I've flown the Inspire several times and have calibrated the compass at each location.  I got lazy and didn't calibrate after moving 8 miles from the previous location, both locations were an open field.  I checked the GPS signal and the locations seemed accurate.  When I took off, I hovered at 6 feet and it began to drift to the right slowly.  I decided to bring it down but there was enough pitch that it tipped over while landing and snapped 3 rotors.  I got lucky.
Is it obvious that the lack of calibration caused the drift?  Could it be the GPS was not solid and that maybe I didn't wait long enought for it to lock on?

Thanks in advance.

Did you trie the calibration of the RC?

I would try an Advanced IMU calibration on a nice undisterbed level surface. Check the sensor readings before and after cal. Compare them to the 'Norm'

Are you 100% sure you didn't lose GPS during that time?