do you have iOSD? It's better that you could link it to your aircraft through the CAN-bus and get data of your next flight. Data are helpful to the analysis of the problem.
I'm assuming your are running a vision or vision+ so no need to worry about iOSD .
Alt reading comes from a small barometric pressure sensor that is inside the Naza-M V2 flight controller which is installed in the Phantom and variations of 10 feet up and down are not unheard of. I think Alt reading is reset every time you switch your phantom on so you should get near zero on power-up, it may drift a bit thereafter since temperature has an influence on the sensor. You may have a sensor defect if you get a lot of drift in the reading.
I just did 3 lights this morning. The first -546 feet at take off and drone would not hover would keep sinking about a foot a second.
Power cycled and tried again.
56' reading at ground and hovered much better. climbed to what I estimated to be 100' and hovered. drone slowly came down
as reading came closer actual ground 0 but still read 16' just off the ground. Then the reading started to go back up as the actual elevation didn't change.
3 rd flight. Power cycled and tried agian.
-556' reading at ground. hovered ok and after a short flight and back reading was -547 at ground 0
then I would suggest you to return the phantom back to your local dealer for arrangement of repair service, if you have bought it from a local dealer; or contact the nearest DJI support center in your region. For the contact info of DJI support center, pls refer to: http://www.dji.com/support.
I have the 2 v+ ver 3.0 and I live on a hill. It does not adjust to the height difference. If I go to 50 ft. and fly over the slope the altitude doesn't change. I'm now flying with a 50 ft. altitude reading going about 20 ft. over trees that are over a 100 ft. tall. Is this normal. Everything is OK as far as the phantom compass and I usually have 10 to 12 satellites.