Answer
$e=0.0113$
Work Step by Step
We can determine the required restitution as follows:
We apply the conservation of linear momentum in the $x$-direction
$m_Av_{A_1}+m_Bv_{B_1}=m_Av_{A_{x_2}}+m_Bv_{B_{x_2}}$
We plug in the known values to obtain:
$0.5(-6)+(0.5)(4)(\frac{3}{5})=0.5v_{A_{x_2}}-0.5v_{B_{x_2}}$
This simplifies to:
$v_{Ax}-v_{Bx}=-3.6~~~$[eq(1)]
According to the conservation of linear momentum in the $y$-direction
$\frac{4}{5}mvB_1=mvB_{y_2}$
$\implies v_{By}=(\frac{4}{5})(4)=3.2m/s\uparrow$
and $vB_x=3.2tan30=1.848m/s\leftarrow$
We plug in this value in eq(1) to obtain:
$v_{Ax}=1.752m/s\leftarrow$
Now $e=\frac{v_{Ax_2}-v_{Bx_2}}{v_{B1}-v_{A1}}$
We plug in the known values to obtain:
$e=\frac{-1.752-(-1.848)}{4(\frac{3}{5})-(-6)}$
This simplifies to:
$e=0.0113$