Hey Lorenzo,

We flew a fairly small area.  Approx.  5 ha.  I would suggest flying at the height that you most commonly fly missions, however, we used 80m for the calibration flight.

Use at least 10 GCPs and make sure that 2 are in the middle of the capture area, with the rest around the perimeter.

That should do the trick,  let me know if your results improve.  We have calibrated 3 of our RTK fleet so far, and the improvement in vertical accuracy has been achieved on each unit.

Cheers,

DJI4Survey Posted at 5-27 15:49
Hey Lorenzo,

We flew a fairly small area.  Approx.  5 ha.  I would suggest flying at the height that you most commonly fly missions, however, we used 80m for the calibration flight.

Thanks for replying with all that precious info,

Anyone else who used GCP's and then calibrated the camera within Pix4d? I can't get my head around the process in Pix4d.

Cheers guys

lorenzo.giusto Posted at 5-28 02:20
Thanks for replying with all that precious info,

Anyone else who used GCP's and then calibrated the camera within Pix4d? I can't get my head around the process in Pix4d.

I'll be trying it this week probably.

patiam Posted at 5-28 06:51
I'll be trying it this week probably.

Great, don't know when I'll have time to go through a calibration flight yet, but yesterday I received a reply from PIX4D support in relation to this matter.

They say I should fly the area at different heights, and I'm thinking to use a subdivision as survey area to tie as tight as I can to vertical control.

I also got told to avoid areas with either vegetation and sand.

Good luck

fans18d74cdd Posted at 5-30 13:37
Great, don't know when I'll have time to go through a calibration flight yet, but yesterday I received a reply from PIX4D support in relation to this matter.

They say I should fly the area at different heights, and I'm thinking to use a subdivision as survey area to tie as tight as I can to vertical control.

We'll you're doing better than I am with Pix4D support, b/c they haven't responded to my posts on their forum. Thanks for sharing the info, let's continue!

I'm very interested in the Pix4D calibration process also.  I added the BAAM Tech PPK system to my P4P and I have the same issue with vertical error with zero GCPs.

fans18d74cdd Posted at 5-30 13:37
Great, don't know when I'll have time to go through a calibration flight yet, but yesterday I received a reply from PIX4D support in relation to this matter.

They say I should fly the area at different heights, and I'm thinking to use a subdivision as survey area to tie as tight as I can to vertical control.

@lorenzo.giusto/fans18d74cdd:

Do you have an active support thread on Pix4D's forum, or have you been communicating directly with someone in Pix4D support?

patiam Posted at 5-31 08:07
@lorenzo.giusto/fans18d74cdd:

Do you have an active support thread on Pix4D's forum, or have you been communicating directly with someone in Pix4D support?

I contacted directly PIX4D through my boss's account.

We are thinking of using a subdivision for the calibration.

The drone came with a calibration certificate of the dealer which states the calibration was conducted in a lab as supposed to an in-situ one.

I'm having your same issues in the field currently.
What I mean by saying that is that if I fly at 60m AGL my survey will be +- 40/60mm.

But as I fly higher the accuracy decreases by a factor that I can't really quantify. To give you an example, my first flights were done at 85m AGL and difference in elevation to GCPs were in the order of 200mm

fans18d74cdd Posted at 6-1 05:51
I contacted directly PIX4D through my boss's account.

We are thinking of using a subdivision for the calibration.

Thanks, I'll do the same as we have a paid account. I was trying to engage their support guy on the forums but all I'm getting is crickets there.

Let us know how your calibration flights & processing work out. I'll probably just use our multiple flights at our test site although it is not optimal according to the Pix4D guidelines (too flat) but I have tons of GCPs there and flights in the can @ various AGL so it's worth a shot. I'll post results here.

I plan on flying a calibration site tomorrow for Pix4D.  I'll post my results.

andrewm Posted at 6-3 12:50
I plan on flying a calibration site tomorrow for Pix4D.  I'll post my results.

That's great! Looking forward to knowing your results

Well that was a waste of time.  I followed all of the instructions from Pix4D and generated a new camera model.

• Youth baseball park selected for test area.  It has various buildings and structures with rich texture.
• Using Drone Deploy the calibration mission was flown at 200 ft agl, 75/75 overlap.  After the planer images were acquired, a series of oblique photos were taken around the perimeter of the area, facing the center.  A total of 119 vertical and 77 oblique photos were collected.
• 10 GCPs were placed around the area and surveyed using a total station
• I have the BAAM Tech PPK system installed on my Phantom 4 Pro, so the PPK data was processed and a geotag file was created for import to P4D
• P4D project was created and processed and a new camera model was created, as per the calibration instructions.
• I flew a second flight (without the obliques), with flight lines perpendicular to the first flight - 117 vertical photos were collected, 200 ft AGL, 75/75 overlap
• For the second flight, I processed Step 1 twice.  Once with the default P4P camera model and once with the camera model I created.  There was virtually no difference between the two results.  RMS Error (ft) of the 10 GCPs used as checkpoints: 0.130 (x), 0.182 (y), 3.69 (z)
  

Perhaps I did something wrong, but the new camera model process in P4D certainly didn't help my results.  I still need at least 1 GCP for decent results.

Thanks for reporting back. That was a lot of work, I'm sorry to hear. it didn't improve your results.

I believe the camera calibration process is being proposed here as a potential solution for altitude-dependent Z error in the P4P RTK specifically, not necessarily for aftermarket RTK/PPK solutions installed on the P4P.
Not saying it wasn't worth a try, but as there are many potential sources of error in these systems I think your PPK solution adds an additional wrinkle that most of this discussion assumes is not in play.

I've held off doing the calibration processing as I have a Pix4D support tech looking at my datasets and have been told camera calibration may or may not be the fix. I'll report what I find either way.

Hey Guys,

I have just calibrated another RTK this morning using the method above in Agisoft.  Prior to camera calibration, Z value error around 400mm at 100m.  After calibration, Z value error sub 50mm over two test flights.  



I am pretty confident that this is the solution for the Z value error, have calibrated 5 of our 10 RTK units so far and it has fixed the issue on all of them.

Pix4d is a more user friendly photogrammetry package, but doesn't allow as much customisation in my experience and is not as nimble to change as Agisoft.  Hopefully they come up with a solution soon.

If anyone running Agisoft can confirm my results, I am sure that would be helpful to the others in this forum.

Cheers

I have an update.  I took a step back and looked into all the settings and variables that can be changed in Pix4D that could affect accuracy then systematically changed each one until I started seeing improvements:

• Calibration Method - by default P4D uses Standard, which is suitable for nadar and oblique imagery. Alternative is optimized for aerial nadar images with accuracte geolocation, low texture content, and relatively flat terrain.  I switched to Alternative, since this more accurately describes PPK / RTK mapping projects.  I saw a slight improvement.
• Camera Optimization (Internal Parameters) - The default setting is All, which reoptimizes all internal camera parameters.  So if you have created a custom camera calibration, P4D only uses that for a starting point.  I changed this setting to None, so the initial camera calibration is held during processing.  This allowed me to manually tweak the focal length, which has a significant impact on vertical accuracy.
• Horizontal and vertical accuracy of geocoded images - By default, it was recommended that both horizontal and vertical accuracy be set to 0.131 feet for my PPK system.  Changing those parameters also has a significant impact on accuracy.
• Camera model - Since the focal length and radial distortion parameters R1 and R2 have the largest impact on accuracy for perspective lens cameras, I used the default P4D camera model for my Phantom 4 Pro and manually adusted the focal length.  I didn't adjust R1 or R2 yet.
  

Here are my results so far.  Accuracies are based on 10 GCPs collected using a total station and marked as check points on all available images.  No "calibration target" or other GCP was used.  For the PPK data, fixed solutions were obtained for all images.

Default Settings

• P4D 3D Maps Defaults
• Geocoded horizontal image accuracy:  0.131 feet
• Geocoded vertical image accuracy: 0.131 feet
  

Results (RMS errors in feet) x, y, z:  0.119709, 0.188054, 3.884623


Modified Settings

• Calibration method set to Alternative
• Camera Optimization (Internal) set to None
• Camera Model Focal Length changed from 8.60 to 8.58
• Geocoded horizontal image accuracy:  0.480 feet
• Geocoded vertical image accuracy: 0.111 feet
  
  Results (RMS errors in feet) x, y, z:  0.150272, 0.155754, 0.138275
  

I need to do more testing with new data sets to see if these settings hold true, but it's certainly a great start.

I re-processed a previous project I flew two months ago:

Default Settings:
0.246582        0.129825        2.383071

Modified Settings:
0.087530        0.120151        0.205327

So far I'm very pleased with these results.

I have some RTK photogrammetry accuracy results to report using Agisoft Metashape.
For this test, I used the DRTK base-station parked over a surveyed benchmark. I flew the drone at 38 m AGL and covered a 1 ha rectangle. For my application I am interested in 3D point cloud modeling of shrubs and trees, so I programmed a 3D mission in GS Pro RTK. The mission had an oblique camera angle of -60 degrees. I also flew an S-pattern with nadir images for orthomosaics. I used all the images from both missions (520) in the photogrammetry project. I did not perform any precalibration of the sensor. Instead I did self-calibration which turned out very good (probably because of the convergent oblique imagery). To assess accuracy, I surveyed 12 independent check points. The RMSE errors were as follows:
longitude: 2.2 cm
latitude: 2.4 cm
elevation: 3.4 cm

jgillan Posted at 6-9 18:23
I have some RTK photogrammetry accuracy results to report using Agisoft Metashape.
For this test, I used the DRTK base-station parked over a surveyed benchmark. I flew the drone at 38 m AGL and covered a 1 ha rectangle. For my application I am interested in 3D point cloud modeling of shrubs and trees, so I programmed a 3D mission in GS Pro RTK. The mission had an oblique camera angle of -60 degrees. I also flew an S-pattern with nadir images for orthomosaics. I used all the images from both missions (520) in the photogrammetry project. I did not perform any precalibration of the sensor. Instead I did self-calibration which turned out very good (probably because of the convergent oblique imagery). To assess accuracy, I surveyed 12 independent check points. The RMSE errors were as follows:
longitude: 2.2 cm

Thanks @jgillan. From my experience, under 40m AGL the results are great (like yours), with or without obliques. It's when you fly higher that the error creeps up...

The more information the better! Everyone share your results!

patiam Posted at 6-9 20:22
Thanks @jgillan. From my experience, under 40m AGL the results are great (like yours), with our without obliques. It's when you fly higher that the error creeps up...

The more information the better! Everyone share your results!

Hi Patiam, in the end I gave up with my own calibration as I could not get any solid results within Pix4d.

But yesterday I had a face to face meeting with the photogrammetry expert at the dealer shop and talked about the problems related to increased flight height = increased vertical misclosure.

After confirming that my workflow was correct, the only thing that he could think of was the Focal length tweaking.
He also told me that he would fly at 100m AGL in order to achieve 2 to 3 GSD vertical accuracy if calibration of camera was spot on.
This time he will calibrate it in the field as supposed to the original "Lab calibration".

I'll update you as soon as I receive the new calibration parameters

Thanks for sharing. I'm standing by waiting for a Pix4D support guy to look at the 4 flights worth of data I shared on Pix4D cloud. He also said they were not convinced that camera calibration was the answer.

I'd bequite  happy w/ 2-3xGSD Z accuracy!

OK so based on @andrewm's post about tweaking settings, and on this seemingly "oh yeah you might want to try this" statement in the FAQ on the Pix4D forum thread for processing P4P RTK data:

What could cause a big offset when comparing the results with checkpoints?

• The horizontal offset could be caused by the horizontal grid corrections and transformations.
  In some cases, when flying fast, close to the ground or in high winds, enabling the linear shutter optimization could reduce the offset.
• The vertical offset could be present due to wrongly selected vertical coordinates system or custom geoid models.
  In case the optimized camera parameters are far from the initial values, e.g. focal length, we recommend using the All Prior processing option.
  

(Highlight mine)
And the fact that the author Blaž also mentioned "All Prior" in my correspondence with him, I decided to reprocess the 4 flights from 20190513 with only that parameter changed. The setting is in Processing Options>Initial Processing>Advanced>Calibration>Camera Optimization>Internal Parameters Optimization (whew!).

Lo and behold, my 80m AGL flight Z accuracy improved (went from 0.434m and 0.621m to 0.165m and 0.111m, for D-RTK 2 and NTRIP, respectively). However, 40m Z accuracy suffered, although not to the same degree as the improvement @ 80m (0.039m and 0.043m to 0.062m and 0.053m, for D-RTK 2 and NTRIP, respectively).

The accuracy at 80m AGL is still nowhere close to 3xGSD (that would be 0.072m, so it's closer to 6-7XGSD) but it is an improvement. The accuracy @ 40m AGL using the "All" setting rather than "All Prior" camera optimization setting is at least close to 3XGSD (0.037m).

What does it all mean? I dunno. Except that better results are possible, and hopefully we'll end up with some processing guidelines that produce more consistent results across the range of altitudes we need to fly.

patiam Posted at 6-11 12:59
OK so based on @andrewm's post about tweaking settings, and on this seemingly "oh yeah you might want to try this" statement in the FAQ on the Pix4D forum thread for processing P4P RTK data:
What could cause a big offset when comparing the results with checkpoints?

The horizontal offset could be caused by the horizontal grid corrections and transformations.

I also tried the All Prior option.  I got better results with that option set to None and by manually adjusting the focal length 0.01mm at a time.

But I only tested a single altitude of 200ft.  I have no idea what my results are at other altitudes.

andrewm Posted at 6-11 13:54
I also tried the All Prior option.  I got better results with that option set to None and by manually adjusting the focal length 0.01mm at a time.

That's a lot of work! You mentioned you systematically tweaked all the available settings but I didn't realize you really tried every permutation! Even for just one flight that's a lot!

SOLUTION:  Get Agisoft.

Jks.  Can you enter calibration settings manually?  Not just the focal length?

patiam Posted at 6-11 14:07
That's a lot of work! You mentioned you systematically tweaked all the available settings but I didn't realize you really tried every permutation! Even for just one flight that's a lot!

My test flight only had about 100 images, so re-running Step 1 only took a few minutes each time.  I started with horizontal and vertical accuracies of the geotagged images, then moved on to the focal length adjustments.  I spent about a day on it.  If I were you, I'd start with my settings and see what you get, then tweak from there.

DJI4Survey Posted at 6-11 14:33
SOLUTION:  Get Agisoft.

Jks.  Can you enter calibration settings manually?  Not just the focal length?

Yes, you can adjust all camera calibration settings in Pix4D.  But I suspect vertical accuracy is highly sensitive to focal length.  So I kept the default calibration and only tweaked the focal length.  When I get more time, I'll also tweak the radial distortion parameters and see what happens.

andrewm Posted at 6-12 05:16
Yes, you can adjust all camera calibration settings in Pix4D.  But I suspect vertical accuracy is highly sensitive to focal length.  So I kept the default calibration and only tweaked the focal length.  When I get more time, I'll also tweak the radial distortion parameters and see what happens.

The Pix4D default camera model for the P4P RTK (FC6310R_8.8_5472x3648) already has a Focal Length of 8.5797, so I tried your optimization settings (Alternative & None) on my 80m D-RTK 2 80m AGL flight, and left the Focal Length as-is.
Unfortunately my results were essentially identical to default settings (Z RMSE = 0.434m). I suppose next I can try changing the FL incrementally as you did, either using default optimization settings or AllPrior...

patiam Posted at 6-12 09:50
The Pix4D default camera model for the P4P RTK (FC6310R_8.8_5472x3648) already has a Focal Length of 8.5797, so I tried your optimization settings (Alternative & None) on my 80m D-RTK 2 80m AGL flight, and left the Focal Length as-is.
Unfortunately my results were essentially identical to default settings (Z RMSE = 0.434m). I suppose next I can try changing the FL incrementally as you did, either using default optimization settings or AllPrior...

What do you have the horizontal and vertical accuracies for the geotagged images set at?

andrewm Posted at 6-13 06:00
What do you have the horizontal and vertical accuracies for the geotagged images set at?

I've been keeping defaults, XY = 0.010m, and Z = 0.020m. These values are reasonable given the quality of the RTK positions. Not sure why one would worsen them, but I do realize you found better results by doing so.
I am thinking that we may be dealing with different sources of error to at least some extent, but I'm willing to try anything that makes sense.
It was suggested to me by Pix4D that I try the Linear Rolling Shutter model rather than Global (even though the latter should be appropriate). Doing so only made things worse (Z RMSE = 0.793m rather than 0.434m using Global Shutter).

By relaxing  the geotagged accuracy it allows more flexibility in the model.  It definitely helped me to keep the vertical tightly constrained, while relaxing the horizontal.

Makes sense, maybe I'll try it. Still pushing Pix4D to come up with a solution.

I'm still doing some testing, but here you can see my results so far.  Errors shown are in feet:

patiam Posted at 6-14 08:21
Makes sense, maybe I'll try it. Still pushing Pix4D to come up with a solution.

Hey Patiam, I got my camera back from the survey equipment shop and recalibrated.
Results are now incredibly spot on mate.

All they did at the shop was to try different focal lengths until residuals were good.
In the specific, all they did was applying increments of one 0.01mm to the focal length.
Hope this helps, even though it sounds like a lot of dicking around.
Geotagging accuracy I've got it setup at 20mm Hz and 20mm Vertical.

Good luck

fans18d74cdd Posted at 6-26 00:43
Hey Patiam, I got my camera back from the survey equipment shop and recalibrated.
Results are now incredibly spot on mate.

@fans18d74cdd-

Thanks, that's great info. Now we have two confirmed use cases where FL tweaking seems to solve the problem. Guess it's time for me to try it as well!

If I understand correctly you've created a custom camera in Pix4D w/ the new adjusted FL and are using that for all flights now, rather than the default. But no changes were made to the aircraft or actual camera itself. Correct?

Thanks again for sharing your results!

patiam Posted at 6-26 07:52
@fans18d74cdd-

Thanks, that's great info. Now we have two confirmed use cases where FL tweaking seems to solve the problem. Guess it's time for me to try it as well!

that is correct

Finally had a chance to reprocess my last batch of 40 & 80m AGL D-RTK 2 and NTRIP flights while tweaking the FL of the camera. Got great results to report so far.
Using the All-Prior optimization I was able to find an adjusted FL that greatly improves the Z RMSE at both 80m and 40m AGL. I used the 4 flights from 20190513 (2 ea. @ 80m & 40m AGL, 1 w/ D-RTK 2 and 1 NTRIP).

As noted previously, just switching to "All-Prior" with the default FL (8.57971 mm) decreased Z RMSE significantly for the 80m flights, from 43cm and 62cm to 16cm and 11cm for D-RTK 2 nd NTRIP, respectively, but not to an acceptable level. Doing so also worsened Z RMSE for the 40m AGL flights by a cm or 2.

But by tweaking the FL in 0.01mm increments, I was able to find FLs that bring Z RMSE down to 2-3cm for all flights! Interestingly, the optimal FL differs by 0.01mm for D-RTK 2 (8.55971mm) and NTRIP (8.56971mm), but splitting the difference (8.56471mm) yields 2-4cm RMSE across all flights, which is an acceptable compromise. X&Y RMSE are unaffected by changing FL as expected.

I left the image geolocation accuracies at the default XY 0.011m abd Z 0.024m. The only camera parameter altered was FL.

Next steps:
1) Apply the new FL to some other existing data
2) Try flying at 100m & 120m AGL to see if results are good throughout our range of operating altitudes.


If using the new FL works as well for other datasets as it does for these 4 flights, we'll be quite pleased.

patiam Posted at 7-15 09:47
Finally had a chance to reprocess my last batch of 40 & 80m AGL D-RTK 2 and NTRIP flights while tweaking the FL of the camera. Got great results to report so far.
Using the All-Prior optimization I was able to find an adjusted FL that greatly improves the Z RMSE at both 80m and 40m AGL. I used the 4 flights from 20190513 (2 ea. @ 80m & 40m AGL, 1 w/ D-RTK 2 and 1 NTRIP).

That is great news!  Looking forward to seeing your 120 m results as that is what we fly for our tree canopy elevation analyses.    Confirm there was no need to fly the double grid or combine nadir and obliques imagery to get you results.  Did you calibrate the camera pretty much as here ?How to calibrate a Perspective Lens Camera In the pix4D knowledge base?

No double grid or obliques. Didn't do the perspective lens calibration. All I did was change Internal Camera Optimization parameter to "All-Prior" and then process each of 4 datasets (single grid, no obliques) with the FL changed in 0.01mm increments. The first one I bracketed the default FL several 100ths mm both + & -  to reveal the trend, and subsequent flights I focused on the side that had the best results. After the first, it was really all about confirming that the FL worked consistently.
Really easy to do, just took an hour or two. The most onerous part was file management and renaming QRs and such to keep them from being overwritten.
Basically:

• Save xxxx project as xxxx_tweakFL.
• Ensure that "All-Prior" internal camera optimization parameter is chosen
• Edit camera FL by 0.01mm and save to db with _tweakFL appended to name (Choose "No" when asked if this edited camera should become default for all images from the original P4RTK camera).
• Reoptimize project
• Create QR and examine RMSE
• Rename QR so it does not get overwritten on next iteration
• Repeat
  

I can write up a more detailed How-To if it would be helpful.

patiam Posted at 7-15 20:14
No double grid or obliques. Didn't do the perspective lens calibration. All I did was change Internal Camera Optimization parameter to "All-Prior" and then process each of 4 datasets (single grid, no obliques) with the FL changed in 0.01mm increments. The first one I bracketed the default FL several 100ths mm both + & -  to reveal the trend, and subsequent flights I focused on the side that had the best results. After the first, it was really all about confirming that the FL worked consistently.
Really easy to do, just took an hour or two. The most onerous part was file management and renaming QRs and such to keep them from being overwritten.
Basically:

That is great!  let me try some of my data sets using the steps you have already provided before you go deeper into developing tutorials or workflow.  Will I need to tweak the FL in mm increments as you did in order to arrive at the acceptable compromise or could I just use what you found to be the best FL setting. In other words.... is every camera on each P4RTK going to have a different FL sweet spot when capturing data at the same flt altitude?   or do you expect all acft flown at the same altitude with the same FL setting will produce similar accuracies?