I was wondering if someone knew any rules for flying (at a safe distance) at a balloon festival, technically it's no an airport (US)

I would treat it like any other aircraft...balloons have the right-of-way over drones.  Check and see if there is any temporary no fly zone for the event.  I would also be concerned with electronic interference as all balloons will carry communication device(s)...which could lead to a fly away...which means you should have pretty good liability insurance.