what does old and young refer to..

This is in the 2nd to last line of the Part II. By "old and young" the old man means that everyone on the ship resented him for shooting the Albatross, since it had caused punishments, including the lack of water. So they all turn on him and make him their scapegoat, and make him wear the Albatross. The point is that all were united against him - old and young alike.