This is a new thread to discuss ongoing testing of the performance of the Phantom 4 RTK, specifically the accuracy and precision of mapping products generated by processing imagery data collected with the system.
Previous threads have discussed the setup of the system and best practices for using it with the supplied DJI D-RTK 2 base station and with NTRIP VRS sources of RTK corrections, issues with system functionality, as well as preliminary testing results. Rather than add to one of those discussions, let's start fresh.

We've now flown and processed enough flights to get a feeling for what we can expect from this system. Our approach is as follows:

• We have a ~5ha callibration/test flight area with multiple permanently-marked GCPs that have been precisely surveyed in using RTK static occupation and/or Total Station. There are 5 RTK and 35 Total Station surveyed GCPs. We are very confident in the accuracy of the coordinates of these GCPs as they have been checked and verified repeatedly and used in dozens of calibration flights.
• We have our own CORS station (not part of the official CORS network but continually operating and used by Trimble for their Centerpoint RTX network) that is an NTRIP caster, just 500m from the center of the test area.
• We fly with either the D-RTK 2 base station (located on one of our GCPs) or our NTRIP VRS as the source of corrections. For the former, we enter the lat/lon coordinates of the GCP in decimal degrees and height above ellipsoid (HAE) in meters, with our measured 1.802m antenna height added to the HAE of the GCP.
• We fly multiple, identical consecutive flights using each correction source. Every variable (altitude, speed, overlap, camera settings, etc.) is kept constant between flights.
• We process our data in Pix4D to produce standard mapping products (orthomosaics, DSMs, point clouds, 3d mesh, etc.).
• Our Pix4D processing generally follows a workflow suggested by the developer: WGS 84 Geographic coordinates for input images with HAE set to 0.00m, and NAD83 (2011) UTM Zone 10 for output with NAVD88 vertical by entering the Geoid12B value for the test area (-34.186m) in output coordinates HAE parameter. This is a vetted workflow developed with our previous RTK sUAS, a fixed-wing eBee RTK, for which it has always worked very well.
• We add our GCP positions as checkpoints only; these are not used to correct or georeference the orthomosaic or point cloud, only as validation points against which to compare the results. At least 3 checkpoints are added. Using the Pix4D Ray Cloud Editor we mark the checkpoints in at least 8 images per standard procedures.
• Initial pecision and accuracy estimates are obtained from the Pix4D Quality Report. We conduct further analysis in GIS using the map products and spatial data layers representing our full population of GCPs.
  

Enough of the methods. Next post will be results.

I'll report the results of four recent (2019/04/23) flights made after updating all firmware on the AC and base to ensure everything was current. We had previous difficulties with the D-RTK 2 base station not accepting the GCP coordinates we entered as it claimed (incorrectly) they were > 50m away from its calculated position. It seemed that we had slightly mismatched firmware and updating it fixed that issue, so I'll focus on only the results from post-update flights in order to remove firmware msimatch as a potential source of error.
Two flights were conducted using the D-RTK 2 base station and 2 using our NTRIP VRS. All flights were at 80m AGL (~2.4cm GSD). Data were processed as described in post #1. We even processed one flight using Agisoft to validate the Pix4D processing workflow. The Agisoft results were essentially the same as Pix4D.

Horizontal precision (repeatability) and accuracy for all flights was excellent (a couple cm). The vertical, not so much.

While vertical precision within flights is quite good, between flights it is not, and neither is accuracy. That is, the Z values are inaccurate but internally consistent within a given flight (they're all wrong by similar amounts in the same direction), but not in a consistent way from flight to flight. This means that if one knows the "fudge factor" to apply to a given flight, the data products can be brought into acceptable Z accuracy. Unfortunately, that value is variable and the only way to find that value is to do a bunch of surveying that the P4P RTK was supposed to obviate.

patiam Posted at 4-26 13:35
I'll report the results of four recent (2019/04/23) flights made after updating all firmware on the AC and base to ensure everything was current. We had previous difficulties with the D-RTK 2 base station not accepting the GCP coordinate we entered as it claimed it was > 50m away for its calculated position. It seemed that we had slightly mismatched firmware and updating it fixed that issue, so I'll focus on only the reults from post-update flights to remove firmware msimatch as a potential source of error.
Two flights were conducted using the D-RTK 2 base station and 2 using our NTRIP VRS. All flights were at 80m AGL. data were processed as described in post #1. We even processed one flight using Agisoft to validate the Pix4D processing workflow. The Agisoft results were essentially the same as Pix4D.
Horizontal precision (repeatability) and accuracy for all flights was excellent (a couple cm). The vertical, not so much.

Well done and revealing.  Thanks

@HTA_RP-1-

Thanks, at this point we are resigned to exploring whether we got a "lemon" and the under-performance of our system is due to some deficiency in our particular kit, or if others are having similar difficulties and the product itself is just not what DJI claims. I hope other users will post their experiences here or in their own threads in the P4RTK forum.

@patiam,
Have you made correction for Phantom 4 RTK antenna height error? I made over 10 models already on my site with proper check with Leica RTK GPS and all the time there is an 190mm shift in elevation. DJI said they take it into account but my results tell me something opposite. Also, if you fly in the wind, depending on its strength the antenna height will vary because the drone will change the angle of antenna during flight.
DJI can you comment on this?
Thanks.

@Votan-

Thanks for the suggestion. As you mentioned, the lever arms between the antenna and CMOS sensor on the AC are supposed to be accounted for in the EXIF data, and one can see the instantaneous offsets and angles in the Timestamp.MRK file.

In any event the errors we are seeing are of much greater magnitude and not at all consistent. A systematic and repeatable offset would be much more acceptable than the lack of precision we are experiencing.

patiam Posted at 4-29 08:28
@Votan-

Thanks for the suggestion. As you mentioned, the lever arms between the antenna and CMOS sensor on the AC are supposed to be accounted for in the EXIF data, and one can see the instantaneous offsets and angles in the Timestamp.MRK file.

Thank you for the detailed description patiam. Alarming to say the least. Have you actually identified the GNSS to be the culprit? Is there a way to isolate what exactly is shifting between your acquisitions? Are the vertical coordinates of the images different between missions?

You are reporting check point errors after the photogrammetry, but as you know, there are a lot of steps between image acquisition and point cloud (or DEM) creation. Could it be that something is going wrong during photogrammetry. For example, the P4RTK has a 'distortion correction' option in the planning application. From what I read in the manual, we should not use it. In the image metadata, DJI provides 'Dewarp' data which I'm currently trying to understand. I'm not sure if they correspond to typical lens distortion parameters (focal length, principal point, radial distortion, etc.). In the past, it has not been my practice to pre-calibrate the camera, but maybe it is necessary. Finally, how strong is the geometry of your acquisition? Collecting and processing only nadir images using structure-from-motion algorithms do not always produce the best results. Adding in oblique images is likely to improve results.

Tomorrow I will do some repeat acquisitions to see if I experience the same issues as you.

jgillan Posted at 5-2 15:10
Thank you for the detailed description patiam. Alarming to say the least. Have you actually identified the GNSS to be the culprit? Is there a way to isolate what exactly is shifting between your acquisitions? Are the vertical coordinates of the images different between missions?

You are reporting check point errors after the photogrammetry, but as you know, there are a lot of steps between image acquisition and point cloud (or DEM) creation. Could it be that something is going wrong during photogrammetry. For example, the P4RTK has a 'distortion correction' option in the planning application. From what I read in the manual, we should not use it. In the image metadata, DJI provides 'Dewarp' data which I'm currently trying to understand. I'm not sure if they correspond to typical lens distortion parameters (focal length, principal point, radial distortion, etc.). In the past, it has not been my practice to pre-calibrate the camera, but maybe it is necessary. Finally, how strong is the geometry of your acquisition? Collecting and processing only nadir images using structure-from-motion algorithms do not always produce the best results. Adding in oblique images is likely to improve results.

Patiam and other P4RTK users:

After reading your post about repeatability problems, I did my own exploration. I collected imagery at the same plot twice using the exact same specifications and workflow. The two acquisitions occurred about a week apart. I am using the D-RTK2 mobile station set up over a known benchmark. I am using height above ellipsoid and adding in the antenna height (1.802m). The area of interest is a 1.1 ha rectangle. I flew a grid of oblique images (-60 camera angle) and an S-pattern of nadir images. The difference between the two acquisitions was readily apparent in the 'absolute altitude' metadata in exif of the images. All of the absolute altitude values in the second flight were shifted 2.5 meters lower than the altitude values in the first flight. This of course produced point clouds and DEMs that were 2.5 m vertically shifted. The first flight products were basically spot on, I even had a check point that was within a few cm of true. I essentially thought the equipment was validated until I read your post about repeat problems. The second flight was vertically off by 2.5 meters.

I really want to believe that we are all collectively doing the workflow incorrectly. However, it is more likely the equipment is not consistent for some reason. I will do several more repeatability tests to try to understand what is occurring.

jgillan Posted at 5-4 18:15
Patiam and other P4RTK users:

After reading your post about repeatability problems, I did my own exploration. I collected imagery at the same plot twice using the exact same specifications and workflow. The two acquisitions occurred about a week apart. I am using the D-RTK2 mobile station set up over a known benchmark. I am using height above ellipsoid and adding in the antenna height (1.802m). The area of interest is a 1.1 ha rectangle. I flew a grid of oblique images (-60 camera angle) and an S-pattern of nadir images. The difference between the two acquisitions was readily apparent in the 'absolute altitude' metadata in exif of the images. All of the absolute altitude values in the second flight were shifted 2.5 meters lower than the altitude values in the first flight. This of course produced point clouds and DEMs that were 2.5 m vertically shifted. The first flight products were basically spot on, I even had a check point that was within a few cm of true. I essentially thought the equipment was validated until I read your post about repeat problems. The second flight was vertically off by 2.5 meters.

@jgillan-

THANK YOU! This sort of accuracy and precision testing by multiple users is exactly what we need to establish not only the performance of the system but also how robust and valid our workflows are.


Personally I was happy and encouraged when an early test flight checked out well against my known high confidence benchmarks. It was only after several subsequent flights on the same site failed to deliver the same results that I really began to question the P4PRTK. The workflow hs been vetted over years of RTK flights with other platforms. Perhaps the workflow needs to be tweaked for the P4R or Pix4D is not handling the data correctly but the lack of repeatability suggests the problem is in the acquisition, not the processing.


Again thank you for taking the time to test and for adding to the discussion. The more we share experiences and notes the better our chances to arrive at a system that works for us, or determine that it just isn't going to.


We continue to test. I'll update when I have new info.
For others that would like to contribute - please fly some consecutive flights over the same terrain - preferably where you have some good vertical control (existsing points, surfaces, or other data in which you have confidence), but even if you don't have that, repeat flights over the same ground with as many variables kept constant as possible should return the same results. If they don't, something is not working right. Try it out, and report your results! And thank you!

patiam Posted at 5-4 20:46
@jgillan-

THANK YOU! This sort of accuracy and precision testing by multiple users is exactly what we need to establish not only the performance of the system but also how robust and valid our workflows are.

After some more testing today, I figured out why my 2nd acquisition was 2.5 m lower than 1st acquisition. It had to do with setting (actually not setting) the coordinates of my D-RTK2 mobile base station. When I entered the coordinates and tapped 'settings', it said it was successful. In reality, it did not adopt the coordinates I entered. Instead, it used the GNSS coordinates of the base station which will change slightly every time you start it up. After I figured out this was the culprit, I starting doing different scenarios to see which coordinates it would accept and which ones it would not. I discovered that if I did two consecutive missions with the base station parked at the same location (all equipment powered down between missions), the second mission would not accept the entered coordinates. If I picked up and moved the base station some distance away, it would accept the manually entered coordinates.

I think the strange behavior can be attributed to this passage from the mobile station user guide page 9: "If the D-RTK 2 mobile station is restarted after input coordinates are successfully set, these coordinates will be used only if the difference between the actual coordinates and the set coordinates is less than 5 m. Otherwise, the actual positioning coordinates will be used." Perhaps there is a bug in the code that is preventing me from using manually entered coordinates on two consecutive missions with the base station in the same place.

I also just realized there is a firmware upgrade I need to do. Perhaps that will fix the problem.

More tests ahead.

@jgillan- Good catch! The GS RTK UI is terribly flaky and it is looking more and more like the failsafe/error trapping DJI wrote into the base station coordinate settings entry causes more problems than it solves.

I've recently revisited the only flight results that we've been happy with to see if any patterns emerge- both were on a single day, one NTRIP and one D-RTK 2, and both were at 40m AGL (as opposed to all our other test flights at 80m). Z accuracy was in the 4-5cm range with 3 checkpoints. I'll post some results in a bit after I look at all 35 total station checkpoints as well.

I can't fly again until Monday, but will be testing 40m vs 80m AGL again and also paying very close attention to the UI for any gotchas like the one you've reported.

So I mentioned in a previous thread that we have had at least one flight with very good accuracy and precision. Turns out we've had two, both on the same day and both at 40m rather than 80m AGL, the altitude where the bulk of our testing has been conducted.

Other than altitude, same methdology as all other flights in all respects, one NTRIP and one using the D-RTK 2. I've reanalyzed the results to incorporate not only the 3-4 BMs we use for preliminary checkpoints, but also 34 total station checkpoints. The BM checkpoint analysis was done in both Pix4D and the Z accuracy validated in GIS using the Pix4D DSM, while the TS Z analysis was done only in GIS (I'm not up for marking that many images in Pix4D).

For both flights we managed 2-3cm RMSE. If we could achieve this consistently at any altitude up to 120m AGL, we'd be very happy.

I have an update as well. After figuring out my problem was getting the app to accept the manual coordinates I input for the D-RTK2 mobile station, I have been getting good results. I did two identical acquisitions (as mentioned in previous post) and set out to test repeatability. I did not use typical check points that have independently surveyed x,y, z coordinates. Instead I made DEMs and orthomosaics (using Metashape) for both acquisitions and assessed how identical they were. For x and y repeatability, I found many small features in my orthomosaics (tufts of grass or rocks) and measured how they shifted from one orthomosaic to the next. I found on average the shift to be about 3-4 cm across the 2.2 ha study area. To assess repeatability of z, I created and subtracted one DEM from the other to get a vertical change (did this in ArcMap). These DEMs had 5 cm cell sizes. Looking across dozens of bare-ground areas of my study area, I found most vertical difference to be less than 1 cm. The largest differences were 1-2 cm.

jgillan Posted at 5-7 13:29
I have an update as well. After figuring out my problem was getting the app to accept the manual coordinates I input for the D-RTK2 mobile station, I have been getting good results. I did two identical acquisitions (as mentioned in previous post) and set out to test repeatability. I did not use typical check points that have independently surveyed x,y, z coordinates. Instead I made DEMs and orthomosaics (using Metashape) for both acquisitions and assessed how identical they were. For x and y repeatability, I found many small features in my orthomosaics (tufts of grass or rocks) and measured how they shifted from one orthomosaic to the next. I found on average the shift to be about 3-4 cm across the 2.2 ha study area. To assess repeatability of z, I created and subtracted one DEM from the other to get a vertical change (did this in ArcMap). These DEMs had 5 cm cell sizes. Looking across dozens of bare-ground areas of my study area, I found most vertical difference to be less than 1 cm. The largest differences were 1-2 cm.

@jgillan-

Thanks, sounds good. That method works for accessing between-flight repeatability. What was your flight altitude please?

I flew 38 m AGL which yields gsd of 1.03 cm. This the lowest I could fly and still move at 3 m/s. If you fly at 37 m, the bird only goes 2 m/s which is excruciating to watch.

Thanks, good to know you have good results < 40m AGL as well.

Hello and thanks so much for sharing your information.  Us newbies (I'm 2 months into the aerial world) really appreciate it and hope to return the favour some day!!!!  Just a quick question, can you suggest a method that I can use to determine the height above ellipsoid?  I note you convert the images to your local datum and then shift to your geoid12B by inputting -34.186.  How do you arrive at this figure?  I have a GPS Topcon base station and rover and am learning to use it to log raw data and post process using KLAU software.  Can I measure this easily with my gear do you think or is there somewhere I can go to online?  Appreciate all your commentary!!!  Mark.

Mark Steggs Posted at 5-11 05:46
Hello and thanks so much for sharing your information.  Us newbies (I'm 2 months into the aerial world) really appreciate it and hope to return the favour some day!!!!  Just a quick question, can you suggest a method that I can use to determine the height above ellipsoid?  I note you convert the images to your local datum and then shift to your geoid12B by inputting -34.186.  How do you arrive at this figure?  I have a GPS Topcon base station and rover and am learning to use it to log raw data and post process using KLAU software.  Can I measure this easily with my gear do you think or is there somewhere I can go to online?  Appreciate all your commentary!!!  Mark.

You can find your Geoid12B height either way (using your gear or with an online service). The easiest is just to go to the NGS website and input your Lat/Lon (You don't need super accurate coordinates, the Geoid doesn't vary much on a scale of 10's of meters - you can test this yourself by inputing coordinates near each other).
This is from another thread:

...The difference between Ellipsoid & NAVD88 Orthometric Height defined by GEOID12B (or any other Geoid Model) varies over space. This is because the geoid surface is irregular, unlike the reference ellipsoid.  So you need to find the value that is appropriate for your site (and if your site is quite large, and you want the highest accuracy possible, you should not apply a single correction but one that varies across the area mapped - that can be accomplished a number of ways).
You can get the value for your base station location from a number of sources, and perhaps the National Geodetic Survey (NGS- part of  NOAA, they run OPUS as well) is the best one-stop shop:

• If you're using a known published benchmark the Ellipsoid and Orthometric Height should be in the published station info (as in HTA_RP-1's example). If it's a CORS station, NGS will have it (as well as data from the station if needed for PPK processing, etc.)
• The easiest way to get the value for an known but unpublished location is also at NGS, where they have an interactive toolinto which you can enter Lat/Lon for a single site or a file of multiple sites, which will return the GEOID12B height. While the GEOID12B height varies over space as noted above, it does so on a scale that even the error associated with a decent autonomous GPS Lat/Lon position (3-5m) will still give you an accurate value in most locations. They also have software tools that run in Windows or Unix that you can download and run locally (with downloaded data for your area)
• If you need the value for an unknown location, and you are going to use your RTK GPS to site it in as a new benchmark, you can record data  for at least 15min but not more than 24hrs in a RINEX file, and upload it to OPUS (Online Positioning User Service) for processing. You have to wait until the next day as most CORS stations roll over their files every 24hours, and OPUS needs their data to process your file. You'll need to enter the antenna model and height there when you upload the file. OPUS will either rapid-static or static process the data using a network of nearby CORS stations and send you a report like the one I pasted in my post (I left off the section lising all the stations used and their info). The report will have your previously unknown benchmark coords in earth-centric (XYZ) coordinates for both the NAD83 and IGS08 datums, as well as Lat/Lon in DMS and the Ellipsoid height for both those datums, as well as the NAVD88 Ortho height for NAD 83.
  

Lots of info on how OPUS works and how to interpret the report is here. Your blue square indicates the Ellipsoid and Ortho Heights in m. The red square shows the error of those values (standard deviation for rapid-static, peak-to-peak for static); this is the case for the Lat/Lon's listed above and all XYZ & LAT/Lon positions in the report. Next to every position is it's associated error and the units of the error (yes, the EL HGT error is just 9mm, and while the ORTHO HGT is a bit worse at 24mm, due to the uncertainty in the GEOID12B model, but still... That's pretty dead nutz!).

• If you need to apply the GEOID12B model over a larger area you can download it for different areas and in various formats from the main GEOID12b page, CONUS is here. The model can be applied in some processing software (not yet in Pix4D unfortunately), or you can leave your Pix4D DSM/DEM/XYZ outputs in HAE and apply the GEOID12B model in GIS or elsewhere. Note that the resolution of the downloaded file is 1 arc-minute (about 31 m or 101 ft at the equator), so you won't see any variarion of the model at scales smaller than this. If you need finer resolution then you need to use the interactive tool on the website or the downloadable one to generate heights at dicrete locations, which you can then apply
  

Lots more info on GEOID12B here

Also see this from later in that thread:

Perhaps I should just drop this here...

Have fun and fly safe!

OK so here's some more test results from today.
TL;DR: Regardless of correction source flights @ 40m AGL had acceptable Z RMSE (3-4cm), while 80m AGL flights did not (40-60cm). XY was always great, as usual.

We did 4 flights in rapid succession: 2 using the D-RTK 2 and 2 using our NTRIP VRS. For each correction source we flew at 40m and 80m AGL. Nadir imagery only, no crosslines. The order flown was D-RTK 2 80m AGL, 40m AGL, NTRIP 40m AGL, 80m AGL. The first 3 flights were done on a single battery, and the aircraft was not powered down until the battery swap prior to the last flight. 80m AGL yields 61 images with a a GSD of about 2.4 cm, while 40m yields 141 images @ 1.2 cm. The NTRIP flights have 1 more checkpoint as we moved the D-RTK 2 off of BM1 after the 2nd flight.

This trend was something I suspected after looking at all our previous flights and finding that only the 40m flights had good results. But now I can conclusively say that there is a correlation between altitude and Z accuracy for our P4PRTK, such that it becomes unacceptable at altitudes we need to fly. I don't know if the relationship is linear, but we may try some 60m & 100m AGL flights to see.

Anybody else experiencing this?

I've had this same issue. I think you're barking up the wrong tree as far as geoid vs ellipsoid.

Try taking oblique photos of the area you are surveying and include these in the processing.

This is the subject where I was dealing with the same issue:

https://www.agisoft.com/forum/index.php?topic=10068.0

I believe it is an issue with how the focal length is reported in the metadata, but I'm really not certain.

As far as I can tell, the GPS positions are not the issue.

Agree with Dragline on this one.   We used the Loki PPK system bolt on prior to the RTK being released.  And they harped on about generating a camera calibration for each drone in order to get the accuracy to within survey tolerance without the need for GCPs (Still need QA).

This can be done in two ways (in Agisoft, not sure about Pix4d):

- Using a job that has a whole lot of GCPs, full coverage, and exporting the camera calibration once all GCPs have been applied.  This camera calibration is then applied to the raw images prior to processing.
- Using a calibration mat (we had one issued as part of the loki rollout)  and take 3 rounds of 70 images at different angles to generate a calibration file.

The second option is likely the more accurate, but also a pain in the ass.  We are running some data using the first option over the next few days, but I am confident it will improve the results considerably.

Fingers crossed, and if any one else is already using camera calibration files (XMLs) let me know if it is improving results.

Cheers,
Tim.

Also, if it was a geoid-ellipsoid separation issue, it should be pretty consistent.  However, we are seeing  variances of 200mm+ on our flights in geographically consistent locations.

Will have some results in 24hr for this one though.

For reference, this is some feedback from Alexis at Agisoft:

Hello rmowbs,

If the pre-calibrated data is used that is not fixed, such un-fixed parameters will be only used as Initial step for the autocalibration procedure.

If you fix the parameters before starting the alignment, then they wouldn't be adjusted during the alignment/optimization procedure.

I'm a bit skeptical regarding built-in parameters stored in the meta-information, and assume that autocalibration should be more effective. Or even performing regular "in-field calibration" over the area with GCPs and estimating the IO parameters based on both cameras and markers coordinates. Then re-using this information in the actual flights.
But such pre-calibration should be performed rather regularly. For example, once a month or even more frequent, depending on the number of flights, as the lens parameters could change due to mechanical reasons (during take-off/landing) or even temperature factor.

Dragline49 Posted at 5-13 16:11
I've had this same issue. I think you're barking up the wrong tree as far as geoid vs ellipsoid.

Try taking oblique photos of the area you are surveying and include these in the processing.

I'm not hanging this issue on the conversion to orthometric height at all.  The results are the same whether one keeps everything in HAE or converts to NAVD88 (within a few mm, the noise in GEOID12B).

I'll try adding obliques as you described, and check out the calibration discussion offered. The area in question is quite flat, (parking lot flat) not one that would intuitively benefit from obliques. But we can try...
And the fact is that even with nadir only, results are good @ 40m AGL. And simply flying twice as high results in an order of magnitude worse outcome. There's something wrong when one gets results like that, and these results are in stark contrast to DJI"s published accuracy specs. I know to take those with a giant grain of salt and always do, but this appears to be well beyond that.


Hopefully I have enough data already in the can to generate a camera calibration file if that's the solution. We weren't supposed to need that... I'll have to see about how to do it in Pix4D. We have Agisoft as well but Pix is our goto processing platform.
I'll be overjoyed if there is a processing fix for this, via a change in workflow or Pix4D/Agisoft catching up to something DJI has implemented without sharing the details. Such a fix would be appreciated, if late in coming (but we're used to that. This isn't our first rodeo).


ETA- Now that i've read the Agisoft forum thread (Thanks @Dragline!) it sounds like obliques really may be key to improving results. I know others (@jgillan) have mentioned this previously and I sort of shrugged it off as not being needed, but I'm certainly willing to give it a shot.

Thanks for this.  I received our P4P RTK On site two days ago and did some flights yesterday.  We were using eBee RTK for over a year here and had great results (except the rough landings).  Processing these photos the same way as the Phantom 4 Pro, except I did not set Ground Control Points because I wanted to see the accuracy of the RTK system.  
I had terrible results over 2 flights.  The First flight was No good.  I processed with Agisoft and left the XY/Z Accuracy at the standard 10m.  This gave me 1.777m Error.  I also noticed that the error was greater the further away from the base it was.  I flew again and made sure to check the D-RTK2 Offset because I thought that was the 1.777m error.  It turned out that wasnt the problem.  regardless my second flight, in agisoft I changed the accuracy of XY/Z to .01m, reprocessed and got an error of 8mm which was great.  I finished the Dense Cloud and put that into Civil 3D to check based on random points on the surface that we picked up.  The results I found were on average 13cm Higher than real world, and also the entire surface seemed to be shifted counter clockwise 1m.

I Flew a third time and am just finishing up the processing.  Will post results, so far not impressed with this.  The fact that even the D-RTK2 doesnt have a published offset to phase center is troubling.  It seems DJI didnt bother to consult actual surveyors to design this.  SO far this P$P RTK is no better than the Phantom 4 Pro.

All flights were done at 65m AGL, Converted WGS84 to NAD83 UTM Zone 8N

patiam Posted at 5-13 14:10
OK so here's some more test results from today.
TL;DR: Regardless of correction source flights @ 40m AGL had acceptable Z RMSE (3-4cm), while 80m AGL flights did not (40-60cm). XY was always great, as usual.

You're flying much higher than we are. To keep the GSD at 2cm we're flying at 70m max and many of our sites are near airports so we can't even always go that high. At a local site where construction hasn't been started I've flown it six times over a three month period mixing using Ntrip and the base. There is no appreciable difference that I can measure with a RTK rover. Checking the control I'm typically under .04 feet in deviations which is well below the RTK accuracy specs.

The RTK drone flight topos are at worst .06 feet deviation from each other and most match within a couple  of hundredths. That's plenty good for earthwork and in fact better than I believe I could do with an RTK rover and a conventional ground topo because seeing the breaks on the ground has its own imprecision.

RichardUnderhil Posted at 5-14 09:39
Thanks for this.  I received our P4P RTK On site two days ago and did some flights yesterday.  We were using eBee RTK for over a year here and had great results (except the rough landings).  Processing these photos the same way as the Phantom 4 Pro, except I did not set Ground Control Points because I wanted to see the accuracy of the RTK system.  
I had terrible results over 2 flights.  The First flight was No good.  I processed with Agisoft and left the XY/Z Accuracy at the standard 10m.  This gave me 1.777m Error.  I also noticed that the error was greater the further away from the base it was.  I flew again and made sure to check the D-RTK2 Offset because I thought that was the 1.777m error.  It turned out that wasnt the problem.  regardless my second flight, in agisoft I changed the accuracy of XY/Z to .01m, reprocessed and got an error of 8mm which was great.  I finished the Dense Cloud and put that into Civil 3D to check based on random points on the surface that we picked up.  The results I found were on average 13cm Higher than real world, and also the entire surface seemed to be shifted counter clockwise 1m.

Don't leave the accuracy at 10M. That allows the software to move the position to fit and not only does it take longer, I believe it introduces inaccuracy. The Pix4D custom settings that recognize the accuracy of the capture give good results and every situation where I've seen software treat RTK captures like autonomous captures has had much poorer results.

MTCWBY Posted at 5-15 09:20
You're flying much higher than we are. To keep the GSD at 2cm we're flying at 70m max and many of our sites are near airports so we can't even always go that high. At a local site where construction hasn't been started I've flown it six times over a three month period mixing using Ntrip and the base. There is no appreciable difference that I can measure with a RTK rover. Checking the control I'm typically under .04 feet in deviations which is well below the RTK accuracy specs.

The RTK drone flight topos are at worst .06 feet deviation from each other and most match within a couple  of hundredths. That's plenty good for earthwork and in fact better than I believe I could do with an RTK rover and a conventional ground topo because seeing the breaks on the ground has its own imprecision.

Well we'd like to be able to achieve the stated accuracy throughout the range of altitudes we are legally allowed to fly (up to 120m AGL unless otherwise limited by airspace type), which is what DJI claimed the system could do.
The published specs say :"Mapping accuracy meets the requirements of the ASPRS Accuracy Standards for Digital Orthophotos Class Ⅲ
** The actual accuracy depends on surrounding lighting and patterns, aircraft altitude, mapping software used, and other factors when shooting."

That's interesting because the current version of the  ASPRS Accuracy Standards for Digital Geospatial Data published in 2014 deprecated the Class I/II/III system used in the previous 1990 edition of the standards. And of course there's the **disclaimer, which I guess DJI is counting on covering their butts when they can't meet the stated standard. It's worth noting that the both the outdated 1990 and current 2014 Accuracy Standards for Digital Orthophotos apply to horizontal (XY) accuracy ONLY. There is a different standard for Vertical Accuracy of Digtal Elevation Data for each version, which DJI doesn't mention at all in their specs.

The old 1990 standard and the current 2014 one are pasted below. Class III from 1990 says XY RMSE should be within 3 pixels, which we've been able to achieve with our P4PRTK at both 40m and 80m AGL. If one were to use 1990 Class III for Z as well (again, DJI does not address vertical accuracy in their specs), RMSE in non-vegetated areas should be 5cm or less. We've achieved this at 40m but not 80m AGL. And while DJI don't claim a vertical accuracy standard in their specs, the P4PRTK is being marketed as a mapping tool, and the implication is that it will provide survey-quality results.

The altitude-dependent magnitude of the error has me convinced this is a boresight/calibration issue rather than a GPS position one. The RTK part of the system works as well as can be expected. But the camera orientation and/or sensor calibration are not tight enough, which leads to increasing Z error with increasing distance (altitude). Perhaps adding oblique imagery will help mitigate this, perhaps some additional calibration can be done, and perhaps the processing software still needs tweaking. Hopefully we'll find a way to tighten up the Z accuracy to an acceptable level.

MTCWBY Posted at 5-15 09:23
Don't leave the accuracy at 10M. That allows the software to move the position to fit and not only does it take longer, I believe it introduces inaccuracy. The Pix4D custom settings that recognize the accuracy of the capture give good results and every situation where I've seen software treat RTK captures like autonomous captures has had much poorer results.

Definitely don't leave the accuracy at 10m. Luckily Agisoft will let you load the image accuracy directly from the XMP metadata.Tools>Preferences>Advanced>Load Camera Location accuracy from XMP Metadata.

This is also nice as it allows you to sort images by accuracy and discard any float solutions.

Have some results in regards to improving the accuracy of the initial RTK results.

Thinking that the camera calibration that was generated by Agisoft, or supplied as part of the XMP data was the cause of this error, I ran a few tests.

Firstly, you need to run a flight with each RTK unit over a controlled range (small area controlled with between 5-10 GCPs).

Apply your GCPs as you would have previously done with the Phantom 4 and optimize the camera locations (Selecting the Optimize Cameras or Wand button).

Once it has completed the optimisation using the GCPs, and you are happy with the total error of the GCPs (Should be sub 50mm), save the job as "DATE-DRONE NAME-CALIBRATION" and export the camera calibration as an XML:

Tools > Camera Calibration > Save

Then you will just need to load this camera calibration file prior to processing P4RTK jobs.  

This is done by Tools > Camera Calibration > Load:



Be sure to fix the parameters you want fixed (The above shows the suggested fixed parameters).

Process your job as normal from there.

Results:

Our results from applying the camera calibration have been great.

We ran multiple jobs before and after applying the camera calibration.  

At 180m depth, our GCP check points were 1.3m out in Z value without camera calibration.

This reduced to sub 100mm with the calibration applied.

At 80m, we are seeing sub 25mm without any GCPs being used.

I think this is the fix for the Z value error, and is fairly easy to manage.

I would suggest running the unit over the range when the results start to deteriorate.  And having a separate camera calibration based on temperature ranges.

But looks very promising.  For those using Pix4d, I am sure there is a similar process.

Good Luck

Processed two more jobs -  Sub 30mm without any GCPs applied.  Boo-yah.

**ALWAYS HAVE QA CHECKS TO VALIDATE YOUR CLAIMED ACCURACY IF YOU ARE SELLING A PRODUCT**

YOU DA MAN, DJI4SURVEY!
This is great news, thank you for sharing this. Now to see how to do this in Pix4D!!!

https://support.pix4d.com/hc/en- ... pective-Lens-Camera

Looks to be a very similar process.

Yup. Do you recommend obliques be included in the calibration dataset and ongoing surveys, or does the camera calibration obviate the need for those?

@patiam et al.,

Here is some of the best research I have seen on the topic of oblique imagery and camera calibration for SfM photogrammetry from drones.

https://www.sciencedirect.com/sc ... i/S0169555X16311291

https://onlinelibrary.wiley.com/doi/full/10.1002/esp.3609

The articles are probably behind a paywall if you are not at a university. PM me and I will send out PDFs to anyone interested.  

jgillan Posted at 5-16 12:59
@patiam et al.,

Here is some of the best research I have seen on the topic of oblique imagery and camera calibration for SfM photogrammetry from drones.

thanks @jgillan-

I'm @ a uni so was able to access them. Not sure why I haven't gone back to the lit on this rather than poking around here. We've always had such good results in the past I've never had to worry much about some of these considerations.

So thanks for the references!

DJI4Survey Posted at 5-15 16:45
Have some results in regards to improving the accuracy of the initial RTK results.

Thinking that the camera calibration that was generated by Agisoft, or supplied as part of the XMP data was the cause of this error, I ran a few tests.

Hi DJI4Survey,

I will definitely need to calibrate the camera on my drone as I keep getting an unwanted vertical shift in every survey I have carried out..I have been flying at about 90m from ground level.

Would you tell me how big was the area you used for your calibration, what height above ground level you were flying and how many photos you included?

Thank you, much appreciate!