Answer
1) When the large-mass object moves initially, the speed of the system is $18.2m/s$
2) When the small-mass object moves initially, the speed of the system is $6.8m/s$
Work Step by Step
1) The large-mass object moves initially
The small-mass object, whose mass $m=3kg$ and velocity $v=0$, was hit by the large-mass object, whose mass is $M=8kg$ and has initial velocity $V=25m/s$.
After that, the system has mass $M+m=11kg$ and travels at velocity $V_f$, which needs to be found.
The whole process only travels in one direction, so we do not need to assume the signs for the vectors.
We assume there are no external forces, so the total linear momentum is conserved. Therefore, $$M\vec{V}+m\vec{v}=(M+m)\vec{V}_f$$ $$V_f=\frac{MV+mv}{M+m}=18.2m/s$$
2) The small-mass object moves initially
The small-mass object, whose mass is $m=3kg$ and has initial velocity $v=25m/s$, collides with the large-mass object, whose mass is $M=8kg$ and has initial velocity $V=0m/s$. After that, the system has mass $M+m=11kg$ and travels at velocity $V_f$.
Similarly, we assume the conservation of total linear momentum and have $$V_f=\frac{MV+mv}{M+m}=6.8m/s$$
