Answer
See the detailed answer.
Work Step by Step
a) Since the collision is a perfectly elastic collision, the momentum and the energy are conserved.
$$p_{ix}=p_{fx}$$
$$m_1v_{1x,i}+m_2v_{2x,i}=m_1v_{1x,f}+m_2v_{2x,f}$$
We can assume that the first ball of 100-g is moving to the right so that the second ball of 200-g is moving to the left.
Thus, $v_{1x,i}=4$ m/s, and $v_{2x,i}=-3$ m/s.
Plugging the known;
$$(0.1\times 4)+(0.2\times -3)=0.1v_{1x,f}+0.2v_{2x,f}$$
Thus,
$$0.1v_{1x,f}=-0.2-0.2v_{2x,f}=-0.2(1+v_{2x,f})$$
$$ v_{1x,f}= -2(1+v_{2x,f})\tag 1$$
$$E_i=E_f$$
$$K_{i1}+K_{i2}=K_{f1}+K_{f2}$$
$$ \color{red}{\bf\not} \frac{1}{2}m_1v_{1x,i}^2+ \color{red}{\bf\not} \frac{1}{2}m_2v_{2x,i}^2= \color{red}{\bf\not} \frac{1}{2}m_1v_{1x,f}^2+ \color{red}{\bf\not} \frac{1}{2}m_2v_{2x,f}^2$$
$$ m_1v_{1x,i}^2+ m_2v_{2x,i}^2= m_1v_{1x,f}^2+ m_2v_{2x,f}^2$$
Plugging the known;
$$ (0.1\times 4^2) + (0.2\times [-3]^2)= 0.1v_{1x,f}^2+ 0.2v_{2x,f}^2$$
$$ 3.4= 0.1v_{1x,f}^2+ 0.2v_{2x,f}^2$$
Plugging from (1);
$$ 3.4= 0.1[-2(1+v_{2x,f})]^2+ 0.2v_{2x,f}^2$$
$$ 3.4= 0.1[4(1+2v_{2x,f}+v_{2x,f}^2)]+ 0.2v_{2x,f}^2$$
$$ 3.4= 0.4(1+2v_{2x,f}+v_{2x,f}^2) + 0.2v_{2x,f}^2$$
$$ 3.4= 0.4 +0.8v_{2x,f}+0.4v_{2x,f}^2 + 0.2v_{2x,f}^2$$
Thus,
$$0.6v_{2x,f}^2+0.8v_{2x,f}-3=0$$
And hence, the final velocity of the second ball is either $$v_{2x,f}={\bf 1.667}\;{\rm m/s\; \;\; or} \;\;\;v_{2x,f}=\bf -3\;\rm m/s$$
It is impossible for the second ball to complete in the same direction at the same speed after the collision, so the right answer is
$$v_{2x,f}=\color{red}{\bf 1.667}\;\;{\rm m/s}$$
And hence,
Plugging into (1);
$$ v_{1x,f}= -2(1+1.667) $$
$$v_{1x,f}=\color{red}{\bf -5.33}\;\;{\rm m/s}$$
---
b) When the collision is perfectly inelastic,
$$p_{ix}=p_{fx}$$
$$m_1v_{1x,i}+m_2v_{2x,i}=(m_1 +m_2)v_{xf}$$
So,
$$v_{xf}=\dfrac{m_1v_{1x,i}+m_2v_{2x,i}}{m_1 +m_2}=\dfrac{(0.1\times 4)+(0.2\times -3)}{0.1+0.2}$$
$$v_{xf}=\color{red}{\bf -0.667}\;\;{\rm m/s}$$
The negative sign means the two balls will move to the left.
