Did Rainsford outsmart the General or what it Zaroff’s over confidence that caused him to be defeated?

Likely, it is a little of both. Zaroff's belief that Rainsford had died going over the cliff was premature, and for an experienced hunter, it was an uncharacteristic assumption. Rainsford, on the other hand, successfully outsmarted Zaroff and made him believe he was dead.