Why must Mary Jane leave?

Huck makes Mary Jane leave the house and stay with a friend across the river. Before she leaves, he writes down where the money is located so she will be able to find it later on. Huck is afraid that if Mary Jane stays at the house, her face will give away Huck's indiscretion.