Accuracy and Precision of P4P RTK
123
26728 100 2019-4-26
Uploading and Loding Picture ...(0/1)
o(^-^)o
patiam
Second Officer
Flight distance : 1093865 ft
  • >>>
United States
Offline

HTA_RP-1 Posted at 7-17 07:58
That is great!  let me try some of my data sets using the steps you have already provided before you go deeper into developing tutorials or workflow.  Will I need to tweak the FL in mm increments as you did in order to arrive at the acceptable compromise or could I just use what you found to be the best FL setting. In other words.... is every camera on each P4RTK going to have a different FL sweet spot when capturing data at the same flt altitude?   or do you expect all acft flown at the same altitude with the same FL setting will produce similar accuracies?

From what I have read and been told by other using this approach, the optimal FL will vary between aircraft, and may even drift for a given camera/aircraft due to conditions or camera age, but should be consistent across all operational altitudes (hopefully). In other words, every camrera/aircraft is potentially different.

You can start with my value if you like, but I suggest starting with the default FL and changing in 0.01mm increments above and then below the default FL to make sure you find the "sweet spot". I think its fairly safe to say that if you see the Z RMSE getting worse (higher) for two or three successive iterations, you can stop going in that direction. Then once you find the optimal FL for one flight, you can at least start there for the others.

Good luck and let us know how it goes!
2019-7-17
Use props
patiam
Second Officer
Flight distance : 1093865 ft
  • >>>
United States
Offline

patiam Posted at 7-17 08:34
From what I have read and been told by other using this approach, the optimal FL will vary between aircraft, and may even drift for a given camera/aircraft due to conditions or camera age, but should be consistent across all operational altitudes (hopefully). In other words, every camrera/aircraft is potentially different.

You can start with my value if you like, but I suggest starting with the default FL and changing in 0.01mm increments above and then below the default FL to make sure you find the "sweet spot". I think its fairly safe to say that if you see the Z RMSE getting worse (higher) for two or three successive iterations, you can stop going in that direction. Then once you find the optimal FL for one flight, you can at least start there for the others.

Here's the latest from Pix4D tech support on the matter
(TL;DR- they suggest doing camera calibration 1st and then FL tweaking only if needed):

****start of Pix4D response*********
Hi patiam,

There are some users that contacted us because of the vertical offset however, we could not identify that this is a global issue and affects all the users processing the DJI Phantom 4 RTK dataset.

The issue could be related to the specific units and therefore, in case the offset is detected, the calibration and testing need to be performed separately for the specific camera.

I discussed the issue with the manager of the team in charge of calibration procedures and camera database and he mentioned that it is advised to perform a camera calibration if the vertical offset is detected.

More specifically:
- Use checkpoints to evaluate the accuracy of the reconstruction and to detect a possible offset.
- Perform the camera calibration using the How to calibrate a Perspective Lens Camera article.
- Use the All Prior option in order to make sure the parameters do not change during processing.
- (Optional) ) If the vertical offset is still present and consistent between the datasets, then we recommend manually changing the focal length by small increments and estimate which value gives the best results. Once the optimal focal length is determined is should be saved and used together with the All-Prior processing option for future projects.

Did you have the change to perform the flight on different heights? Did the manually determined focal length worked well with other flights?

I look forward to your response.
Regards,

​Blaž Vidmar | Technical Support Engineer
****end of Pix4D response*********

The statement "If the vertical offset is still present and consistent between the datasets..." is a bit worrisome & somewhat confusing. The problem with this system is NOT a consistent vertical offset observed across all datasets. It is a Z error that is positively correlated with altitude but extremely variable. Flights at higher altitudes exhibit greater error, but multiple flights at the same altitude do not necessarily exhibit the same error.

Anyway, just wanted to update the thread with my most current comms from Pix4D. Not sure I'll bother with camera calibration if my adjusted FL works consitstently (haven't had a chance to do any 100 m & 120 m AGL flights yet).


2019-7-25
Use props
elaliberte
lvl.1
Flight distance : 33 ft
Offline

Thanks @patiam for this very useful post. I was getting the same problem and by tweaking the focal length in 0.01 mm I was able to get rid of the vertical offset and got the RMS vertical accuracy down from 20 to 2 cm. The optimal focal length was 8.53 mm, compared to the default 8.58 mm.
2019-9-4
Use props
patiam
Second Officer
Flight distance : 1093865 ft
  • >>>
Offline

Glad this thread is providing useful information and solutions. I've been away from the P4RTK for a while now dealing with getting various other DJI & non-DJI aircraft dialed in. But I'll report back when I have more info or data.
Thanks for reporting your success with FL tweaking! (And that's a pretty significant change wrt the default FL!)
2019-9-5
Use props
Skyveyor112
lvl.4
United States
Offline

To all those that contributed to this thread, thank you!.  I appreciate you sharing your learning with me and the group. With your hard work, advice and suggestions, I got phenomenal results. For the first time, I see the potential for GCP free flying.

I calibrated the Phantom for 100m.  Upon completion, the RMS was 0.09ft or 2.74cm on 4 GCPs.

We then flew the same area using terrain awareness with an offset of 100m. Due to variations in the terrain and the granularity of the terrain model, the average AGL altitude was 96m. Not bad.

For the Initial Processing step in Pix4d we selected Use Geometrically Verified Matching, Accurate Geolocation and Orientation and All Prior. I know Pix4d recommends Standard matching, but we've seen better results with Accurate Geolocation and Orientation selected.  After importing the GCPs (for more accuracy we use height above GRS_1980 instead of one of the provided EGMs).  The computed GCP point was almost exactly in the center of the actual GCP with an RMS of 0.099ft. The largest vertical error was .124 ft, 3.78cm. Outstanding. Even with GCPs, I often don't see an RMS that low.

I am thrilled with these results, but more testing is required.  The plan is to lay down more control, especially in areas with large elevations changes from the home point elevation. Not sure when that will happen but I will update you when it's done.

Thanks again for testing, analyzing and sharing.

Dave

2019-11-9
Use props
djiuser_amHicEaayH43
New

Pakistan
Offline

I think this might help
2019-11-9
Use props
mbuonanno
lvl.4

Italy
Offline

Usefull references

Hello to everybody,

I hope I can help you by suggesting these two interesting papers:

Taddia Y., Stecchi F., Pellegrinelli A. (2019) USING DJI PHANTOM 4 RTK DRONE FOR TOPOGRAPHIC MAPPING OF COASTAL AREAS.
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII-2/W13, 625-630.
https://doi.org/10.5194/isprs-archives-XLII-2-W13-625-2019
https://www.int-arch-photogramm- ... LII-2-W13/625/2019/

Peppa, M. V., Hall, J., Goodyear, J., and Mills, J. P. (2019) PHOTOGRAMMETRIC ASSESSMENT AND COMPARISON OF DJI PHANTOM 4 PRO AND PHANTOM 4 RTK SMALL UNMANNED AIRCRAFT SYSTEMS. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII-2/W13, 503-509.
https://doi.org/10.5194/isprs-archives-XLII-2-W13-503-2019
https://www.int-arch-photogramm- ... LII-2-W13/503/2019/

Maurizio

2019-11-27
Use props
CHASCOADMIN
lvl.4
United States
Offline

Awesome thread all! Just a couple of observations that I don't think DJI would want to advertise, but are relative to some of the issues I see people are having. First one being inconsistent vertical offsets. The is a known issue with the use of any GNSS system that does not use localization and relies on a solid RTK fix and an uncontrolled set of satellite constellations. While you are probably ok on a high percentage of the time with the results you will get one-time or infrequent flights it may not be so if you are tied to set control under a localization and are attempting to do many repeated flights over the lifespan of a project. This is not the hardware's fault, but a natural condition of single-point occupation without localization and varying satellite conditions without the ability to control them. This is also why many who expect 2-3cm accuracy all the time every time use PPK, which the P4RTK is capable of, but not part of its native design or software workflow. This is something that needs to change in the next iteration. We flew multiple tests with another operator who had a P4RTK and decided not to purchase at this time because it had no advantages, but actual disadvantages in comparison to our aftermarket PPK configurations.

The second issue is the PPK process as I mentioned. While a year down the road a few companies have figured it out it is not streamlined for a user to do themselves without the need for expensive 3rd party solutions or a slew of other software. The way the data is formatted is not to an ISO standard and is understandable as many companies choose to be sandboxed, but many pains that people are experiencing would be alleviated by a better workflow.

All that being said you guys are getting great results and keep up the good work! It would be a great thing to have this cross-posted on several forums that some of us are trying to help others with. I will do my part even though we personally do not own one.
2019-11-30
Use props
patiam
Second Officer
Flight distance : 1093865 ft
  • >>>
United States
Offline

Thanks for the kind words @CHASCOADMIN, they mean a lot coming from a guy with your chops.

Thanks also for your insights on the advantages of PPK and the need for more standardized formats and workflows. In my previous hydrographic surveying life, we used PPK almost exclusively to great effect.

Feel free to cross-post away!
2019-12-7
Use props
djiuser_UqJEbkxHbEQu
lvl.1

France
Offline

Hello everyone,
I was wondering whereas I didn't find any clear anwser (might be so obvious for everyone but me?!?) about the following:
what will be the best practice regarding photo parameter using P4 RTK in order to achieve photogrammetry?
i.e. using dewarp ON within GS Pro or DJI Pilot or using dewarp OFF ?
In my opinion and regarding Agisoft (that i use) recommandation the photo should be as "natural" as possible ("Metashape operates with the original images. So do not crop or geometrically transform, i.e. resize or rotate, the images").
Does anyone conduct photogrammetry process comparaison over 2 sets of data with those 2 parameters?

However if the answer is to use dewarp as OFF:
Should I assume that built in configuration of the lens found within XMP metadata are just craps regarding to the following post ?:
https://forum.dji.com/forum.php?mod=redirect&goto=findpost&ptid=187711&pid=1854372  and
https://forum.dji.com/forum.php?mod=redirect&goto=findpost&ptid=187711&pid=1856781

Or anyone as other results ?

2019-12-18
Use props
djiuser_UqJEbkxHbEQu
lvl.1

France
Offline

Hello again,
in your experience when using P4 RTK in...RTK mode (i.e. no PPK solution) with a ntrip network:
- does the P4 takes correctly the antennae height (i.e. offsets between ARP and known reference point) provided within RTCM ?
- does it also (much harder to confirm) take correctly the antenna calibration (i.e. offsets between phase center and ARP) of the RTK base?  
I am not so sure about this last one as only the IGS name of the antenna is provided through the RTCM meaning that in a way or another the P4 RTK is able to download and update official antenna calibration files.
Thanks for your answers
2019-12-18
Use props
mbuonanno
lvl.4

Italy
Offline

Hello,


please could somebody using Agisoft Metashape share the default calibration parameters for P4RTK cameras (i.e. DJI FC6310R), plus sensor size (in mm and pixel) and pixel size (in µm) ?
In Pix4Dmapper even the pixel size in micrometers is different from that declared from DJI (2.34527 or 2.39981µm vs. 2.41µm).

Thanks,
Maurizio



2019-12-18
Use props
CHASCOADMIN
lvl.4
United States
Offline

How are you mappers dealing with localization and scale factor? We did a demo of a P4 rtk and I also have several contacts that use them, but I have not seen the ability to do this.
2019-12-18
Use props
djiuser_UqJEbkxHbEQu
lvl.1

France
Offline

djiuser_UqJEbkxHbEQu Posted at 12-18 03:50
Hello again,
in your experience when using P4 RTK in...RTK mode (i.e. no PPK solution) with a ntrip network:
- does the P4 takes correctly the antennae height (i.e. offsets between ARP and known reference point) provided within RTCM ?

I did run some test yesterday:
- set the P4 RTK on a static surface
- 1st observation :initiate RTK base with Antenna Height (AH) =0 and correct antenna calibration file
- 2nd observation :initiate RTK base with Antenna Height (AH) =10 and correct antenna calibration file (dz=15cm)
- 3rd observation :initiate RTK base with Antenna Height (AH) =0 and Null antenna calibration file incorrect  antenna calibration file (dz=0cm)

reading of Ellipsoid Height (he) was done but meaning ~10 to 15 consecutive observations

1= he=75.645
2= he=85.643
3=he=64.646

In summary the HA is well accepted by the DJI P4 RTK but not the antenna calibration file. Which mean that a constant shift should be done on the Z axe  on the final model.
it could be interesting to compare these results by same operations with the DJI base.
2019-12-19
Use props
mbuonanno
lvl.4

Italy
Offline

djiuser_UqJEbkxHbEQu Posted at 12-19 03:10
I did run some test yesterday:
- set the P4 RTK on a static surface
- 1st observation :initiate RTK base with Antenna Height (AH) =0 and correct antenna calibration file

Hi djiuser_UqJEbkxHbEQu,

what do you mean with "antenna calibration file" ?  Where do you input such file?

Maurizio
2019-12-19
Use props
djiuser_UqJEbkxHbEQu
lvl.1

France
Offline

mbuonanno Posted at 12-19 05:59
Hi djiuser_UqJEbkxHbEQu,

what do you mean with "antenna calibration file" ?  Where do you input such file?

Roughly how it work (or supposed to )
The antenna calibration file is declared within the GNSS base station and transmit over RTCM (NTRIP). It only declares the antenna IGS name of the antenna example "LEIAX1203+GNSS"
The rover (P4 RTK in our case) should be able to decode the name and to get the dx dy dz parameters from a database . dx dy dz are the offsets between L1, L2 center phase (the point where the observation are received ) and the ARP (the physical reference point of the antenna).
The electromagnetic physique makes that the L1 L2 center phase in some case could be above the physical radome of the antenna and have more than 15cm of height shift (dz) to the ARP (But more often around 3 to 5cm)

I don’t know how the P4 RTK deals with de DPGS 2 station from DJI. But for (almost) sure when it come to third party base via RTCM NTRIP it DOES NOT take the antenna calibration files for  its RTK phase solution.

I won't go any further but if you are dealing with PPK you should input the antenna calibration file as well.  you have two kind of calibration files relative and absolute on IGS website. nowadays all serious PPK software should deal with the absolute calibration antenna file as it is more accurate.

The consequence for us is that might shift an entire survey from some cm. It might also explain some of the constant errors between Ntrip survey made by patiam (thanks for all your test) and the GPCs.
2019-12-20
Use props
mbuonanno
lvl.4

Italy
Offline

Hi,

two days ago I received from the DJI Support engineers some info about the Phantom 4 RTK camera. I know that for most of you these are not news but at least they can be considered interesting because "official".

Q: what is the precise size in micrometers of the sensor pixel? (Agisoft Metashape suggests 2.41µm, whereas Pix4Dmapper 2.34527µm)
A: The Pixel Size is 2.4μm*2.4μm

Q: 8.8mm is "default" value NOT the true "calibrated focal lenght". Is it correct ?
A: Yes

Q: Can I calculate the TRUE CALIBRATED focal length as ((fx+fy)/2)*pixel_size ?
A: Yes, the actual focal length of the camera can be calculated based on f=(fx+fy)/2*2.4*0.001


Consequently as far as I have understud:

- the XMP tag drone-dji:CalibratedFocalLength="3666.666504" contained into each JPG image of any P4RTK camera is NOT referred to the true Calibrated Focal Length but to the "default" one (3666.666504pixels * 2.4µm/pixel = 8799µm or ~8,8mm). The same value is reported into the EXIF metatadata.
I think we can expect the same with the folowing XMP tags:
drone-dji:CalibratedOpticalCenterX="2736.000000"
drone-dji:CalibratedOpticalCenterY="1824.000000"

- the XMP tag "drone-dji: DewarpData" is the only one containing truely CALIBRATED values for each single camera. Even if they can change due to temperature or mecchanical stresses and should be regularly checked, we could use the calibrated focal length as a starting point.
The "drone-dji: DewarpData" tag contains the following fields:
calibrate_date;
fx, fy, the focal length in pixels;
cx, cy, the optical center expressed in pixel coordinate (origin point as the center of the imager);
k1, k2, k3, the radial distortion parameters;
p1, p2 - the tangential distortion parameters.

Example:
"drone-dji: DewarpData="2019-05-30;3650.780000000000,3644.020000000000,-12.330000000000,25.230000000000,-0.266697000000,0.116250000000,0.000782370000,-0.000359319000,-0.040497400000"

I wish you a good 2020 !
Maurizio
2019-12-28
Use props
fans31104d30
lvl.2
Flight distance : 1014872 ft
Portugal
Offline

mbuonanno Posted at 2019-12-28 05:56
Hi,

two days ago I received from the DJI Support engineers some info about the Phantom 4 RTK camera. I know that for most of you these are not news but at least they can be considered interesting because "official".

After using both Pix4D and Agisoft PhotoScan (now Metashape) since the most earlier versions and considering the results expressed in this forum, I still privilege the use of A. Metashape for getting the biggest accuracy possible in my works.
Best Regards

2020-4-5
Use props
patiam
Second Officer
Flight distance : 1093865 ft
  • >>>
United States
Offline

This is an old thread, but I want to update it with some recent results obtained with a brand-new P4 RTK @ the same test site in the OP:


Methods:
  • RTK corrections via NTRIP from a CORS GPS Basestation
  • Flight altitude: 40m, 80m, and 120m AGL
  • GS RTK "Altitude Optimization" option ON (this adds some oblique image capture automatically)
  • Processing: Pix4D Desktop & Cloud
  • GCPs: none
  • Checkpoints: 4 established benchmarks, marked in 8 images per checkpoint

Results:

X & Y RMSE 1-2 cm at all altitudes
Z RMSE 3-7 cm increasing w/ altitude
X, Y, & Z RMSE all generally within 2xGSD

NTRIP:
RMSE (m)
RMSE/GSD
Altitude (m)GSD (m)
X
Y
Z
X
Y
Z
400.0120.0180.0140.0301.4771.1582.471
800.0250.0180.0170.0420.7040.6961.667
1200.0370.0170.0130.0720.4510.3631.947


Similar-ish results using the D-RTK 2 placed @ the 1st benchmark:
D-RTK 2:
RMSE (m)
RMSE/GSD
Altitude (m)GSD (m)
X
Y
Z
X
Y
Z
400.0130.0130.0530.0191.0114.0911.442
800.0250.0130.0550.0500.5222.1861.983
1200.0370.0220.0520.0550.6011.4061.495



The relatively worse Y accuracy is probably due to error in positioning the D-RTK 2 on benchmark #1 as its base location.


This is in contrast to the results reported in the OP, where we found that Z RMSE got much worse (like 40 cm) at altitudes > 40m unless we did extensive iterative Focal Length calibration to create a custom camera model for Pix4D. Not sure if the oblique images are the fix here, or if DJI improved the optics or something else, but it's good to know we can expect Z accuracy that diminishes with altitude in a reasonable way. My gut is the obliques are the key, but I'm not interested enough to try re-processing without them to find out. For me, knowing this system is working when used in this way is enough.

Also, thanks to all that contributed useful information to this discussion over the last few years. There's a lot of good stuff here!
2021-5-27
Use props
EmanueleT
lvl.1

Ghana
Offline

patiam Posted at 5-27 10:10
This is an old thread, but I want to update it with some recent results obtained with a brand-new P4 RTK @ the same test site in the OP:

Hello Patiam,
this results is great!
We just started our works with P4RTK and we works normally in remote bush location without mobile access so we need to use D-RTK2.

Your results at 120 m is great (I plan 100m AGL normally).
We works in Ghana and the information about local system, geoid and so on is very lack. We have a network of about 300 know coordinate BM that is in WGS84 UTM30 system.

My idea is to convert metric XY in DD and use Z as is (not convert change to ellipsoidal H, that could be very challanging for us, but of course adding pole H).
Of course we plan to measure 4 Check Point by base-rover Leica GNSS.

Do you think that with this approch we can reach your results? Maybe we need a good camera calibration...my idea is to do that with 15 GCP in a test site performing self calibration.

Thanks for your time...
2021-7-22
Use props
patiam
Second Officer
Flight distance : 1093865 ft
  • >>>
United States
Offline

Hi EmanueleT-

Sorry for the late reply, I was helping run a Drone Camp all last week.

Sounds good, should work fine. If the P4 RTK was a true survey grade system you could work in your own UTM Zone 30 XY coordinates, but as you say, you will have to convert to DD (which is straightforward). Working in HAE is not an issue.

I suggest trying to use some of your BM as GCP while retaining some for checkpoints, and seeing how it comes out before worrying about camera calibration- you may not need it.

Have fun, be safe, and let us know how it works out!
2021-8-4
Use props
123
Advanced
You need to log in before you can reply Login | Register now

Credit Rules