Since we now can turn off Vision Positioning system, does it make sense to do so when we fly especially over water?

Or is it a good idea when we fly at low altitude over water?

I turned mine off to see the difference, and I did not notice much when flying high, but when the P4 was hovering low, it started to move up, and then back down on it's own.  I did not see this behavior before.

Yes, exactly as I would have expected. Up high the ultrasonics are not effective, down low they are the most reliable methods of measuring altitude. So if you switch them off, the aircraft will not hold altitude at low levels as precisely. Mind you, it won't hold altitude up high as well either, but you won't see the changes as easily.