Answer
Temperature differences determine the direction of heat flow, but it is thermal energy that does the flowing, i.e., heat.
Furthermore, if one of the objects is larger, or has a large specific heat capacity, it might only change in temperature by 1 degree while the other object changes by, say, 10 degrees, so it would be incorrect to talk about the flow of temperature.
