QBK
 lvl.2
United States
Offline
|
There are several recent threads on the forum asking "Are the images from my P3 supposed to look like this?", e.g.:
Focus quality variation across image view,
Something wrong with my camera or is this the quality?,
and New Firmware - Blurry Photos and Video.
(Update: see also Out of Focus Camera... Please help!)
(Update #2: this post in the first thread linked above demonstrates an impressive (though tricky) technical fix to a similar problem).
So, even though we're all obsessed with geofencing now... my house isn't in a no-fly zone. And I've been getting poor images. So I did some tests to try and figure this out. I hope this is useful to some of y'all, but really what I'm looking for is feedback.
Backstory: I got my P3P in June, flew it around in summer weather in New Mexico and Alaska, and got very good images. Then, in October, my Lightbridge died. I finally RMAed it in December (had a great experience -- thanks, DJI!) and got it back right after Christmas. DJI replaced the whole camera. I updated the firmware, reduced gains to 80 across the board to get rid of vibration, and started flying (in Massachusetts, now -- so it's cold) and shooting. And I was really disappointed with the image quality that I saw. So I started Googling for data, found the threads above, and eventually started doing my own tests.
What I saw in the field: First, a word on methodology. All the images in this post are unprocessed frame captures from 4K video. I'm perfectly aware that this does not maximize image quality! My goal here was to compare the P3P camera against itself at different times. Furthermore, I have a lot more video footage to work with (especially from the old camera). Using unprocessed 4K frame grabs throughout ensures a consistent standard. Finally, this is a more forgiving test than using DNG -- any lens defect that shows up in these tests would be even worse in DNG. Oh, and yes, I did take the plastic cover off the lens (and inspected it minutely for haze, defects, etc).
Most of the images are ~2MB, and I can only attach <1MB files to a post. Downsampling them would completely miss the point of this post! So I'm posting Dropbox links instead.
With that said, here are two shots from my original camera in warm weather (55 - 90 Fahrenheit). Note that I never used it in colder weather.
Original Camera 1
Original Camera 2
Here are two shots from my new camera in temperatures around freezing (30-40 Fahrenheit). I haven't had an opportunity to use it in warm weather.
New Camera (cool temps) 1
New Camera (cool temps) 2
And here are two shots from my new camera in sub-freezing temperatures (15-20 Fahrenheit).
New Camera (sub-freezing) 1
New Camera (sub-freezing) 2
I'd really appreciate comments on your perceptions of the sharpness and detail in these images. (And note that I'm not concerned with composition, color balance, etc!) I'm looking at these at 100% (1:1) magnification, and I'm not comparing them to DSLR images -- I'm interested in the before-and-after comparison.
Here's my take:
1. The old camera is excellent for a small sensor. In particular, the lens is more or less flawless. I see pixel level detail across the frame. Everything is sharp down to the single pixel level. There's no visible CA (chromatic aberration) in these or any other shots. There are strong JPEG artifacts, and it's a small sensor, but I have no complaints here.
2. The new camera is consistently worse except in the lower right corner. The center and left portions of the image range from "slightly blurry and low-contrast" (above freezing) to "dreadful" in the sub-freezing picture of the house and trees. In other images, I see quite a lot of CA. (Which is not a huge concern in itself, but is frequently a symptom of other lens problems... particularly given that it was absent in my original camera).
3. Temperature seems to make a huge difference. The images taken above freezing are okay, and if I'd never seen what my original camera could do, I'd be content with them. But the ones below freezing are seriously blurred. I estimate that their linear resolution is at least 2x worse, meaning these images would have to be downsized from 8MP to 2MP (1080p) to look as sharp as those taken with my original camera.
Resolution testing: To try and get some objective data, I did some tests. First, to see whether vibration was the issue, I shot a bunch of brick walls both (1) while flying and (2) while on the ground. I also varied shutter speed. I found no significant difference, but stationary shots seemed at least as good as flying shots, so all remaining shots were taken with the aircraft resting on a stool.
Then, at about 45 degrees, I set up in front of a brick wall and took a bunch of shots of a couple of resolution charts. I moved them around the frame, captured frames, and then merged the frames to make the following two JPEGs. One shows several shots of a large resolution chart held at 13' distance. The second shows several shots of a small resolution chart held at 13' and 25' distances.
Large resolution chart (merged)
Small resolution chart (merged)
In case you're wondering about my methodology, I was pretty careful about splicing the chart crops into the right location, all editing was done at 100% JPEG quality, and I checked the final merged images to confirm that they look identical (at 3:1) to the originals.
I'll skip over most of the nerdy details of the calculations that I did, and go to the punchline. While the left side is weak, that's mostly because there's a lot of CA (several pixels of red/green banding in high-contrast transitions). The absolute resolution is still darn good. I'm getting about 1.3 to 1.5 pixels/line. In other words, I can resolve 10 alternating black/white lines in a region that's just 13-15 pixels high. This is about as good as you can get (given aliasing, etc).
CA isn't usually a big problem, but when it's this bad it causes a dramatic reduction in micro-contrast. In other words, you can still resolve fine black/white lines, but you see them as dark grey and light grey. This manifests as "blurriness" on real objects (trees, grass, rocks). You need very sharp high-contrast edges to even identify that it's being caused by CA, and you need multiple length scales to really diagnose it. CA this bad can be a symptom of something misaligned in the lens, and I suspect that's the situation here.
Okay, one more test. Above, I said resolution was "1.3 to 1.5 pixels/line". Actually, it's 1.3 in the early photos, and 1.5 in the later ones. That made me wonder if temperature was playing a role (even at 45 F) -- was the camera getting worse over time?
Temperature Test (70F down to 22F): So I made the most boring 4K video ever. I put my Phantom on a stool outside, facing a resolution chart at 13' away, and took a 21-minute video. It was 22F outside, and I started the video inside at 70F before taking it outside, with everything already set up. Afterward, I took 43 frame grabs at 30-second intervals (starting about 8 seconds after the P3P was exposed to cold air), cropped them to the chart, and flipped through them.
Unfortunately, I can't figure out how to make an MP4 out of them without spending a lot of time on it, so I'm just posting four sample times: 0:00, 5:30, 11:30, and 21:00.
(These photos are attached to the post).
Resolution clearly decreases over time. I can't prove that this is due to temperature, but it's a pretty strong implication. The lighting does get a bit flatter over time, but this is all at ISO 100, and I don't think the flat lighting should affect resolution this way.
I want to point out, in particular, how the resolution dies. In the first frame, if you look at the top set of 17 lines, you'll see distinct lines on the right which vanish abruptly at the point marked "6". This is due to sensor resolution. There just aren't enough pixels to resolve more lines, and so we see an abrupt transition from lines to grey mush. Now, look at the last frame. The entire upper set of lines is mush, but we can look at the lower set of 9 lines (at the right edge, their spacing is identical to that of the upper set). The transition from "lines" to "mush" is much, much more gradual. This is due to lens (focus/centering/whatever) failure. So, whereas before the lens outresolved the sensor, now it's not.
Conclusions:
1. My new camera's lens (+alignment) is not as good as my old one. This is causing variation over the frame, CA, and loss of resolution (at least at cold temps). It's visible in 4K frame grabs as well as DNGs (not shown).
2. Given the rash of very similar complaints and issues, I tend to suspect there's a batch of cameras with this feature.
3. Cold temperatures make it much, much worse. At 22F, I'm seeing not just CA, but actual defocusing/decentering (I can't tell the difference since the lens is fixed-focus).
4. Letting the aircraft cool down (at least for up to 21 minutes) doesn't help -- it just equilibrates at "bad".
5. Vibration has nothing to do with this.
Questions:
1. Especially for the DJI folks: is this a legitimate RMA issue, and will it get fixed if I do send it in? Or is this "within spec" (meaning that I just got fantastically lucky with my first camera), and the best I can expect is a replacement camera that's just as bad?
2. Anybody else feel like doing similar tests?
3. Do y'all agree with my analysis of the images given here?
|
-
Resolution chart at 0:00
-
Resolution chart at 5:30
-
Resolution chart at 11:30
-
Resolution chart at 21:00
|