raven4 Posted at 2017-3-9 04:21
What i said was that yes, there is the Dlog option for RAW. But, that's not right. How can that be? RAW isn't video, it's digital data straight from the sensor. Dlog is video that's been extracted from the RAW data and had a log gamma applied to the extraction. The Dlog option is supposed to be applied to a debayered image stream, not to RAW. And, no,  haven't had a chance to check it out, yet. It's been extremely windy here for the last week.

so, let me get this straight... DJI feels they need to lock the ISO on DLOG because its better for us.. and also that you can shoot RAW using DLOG? That makes no sense. Im curious what the decision was for this.

DJI, please remove the ISO lock so I can install the new firmware.

DJI-CAO Posted at 2017-3-8 16:54
Please read my post.

Log gamma affects the exposure. So a total 2.3ev is used to protect the highlight. That means RAW is changed when shooting dlog.

Dear DJI-CAO, thank you for your answers. I have to disagree with your statement and some previous ones. Log gamma does NOT affect exposure - it just affects how certain exposure is recorded, it sets color space for recording side, to intentionally compress stops of shadows more than stops in highlights. That is not difference in exposure, but difference in recording space.

Log color space is designed to protect highlights but it does not protect shadows, in order to squeeze more stops in recording, shadows are more compressed in Log to make space for more stops in highlights, in such way that 18% gray in Log does not fall at same IRE as in  Rec709. So,  managing professional Log workflow incorporates knowing that 18% gray shot in Rec709 should not have same IRE as 18% gray in Log, By design. 18% gray in Log will get to Rec709 level after transforming log-to-Rec709 by proper LUT... It goes the same  for Arri C-Log, Canon CLog, Sony S-Log... Panasonic V-Log L, which was similar to D-Log, has 18% gray at 42IRE, 90% will be at 61 IRE, while super whites are at 90IRE. So, if one wants to overexposed he will do it by himself, not to be forced by fixed ISO, and Pany can record VLog even at ISO800 and ISO1600 and so on... Same goes for Sony pro cameras as well, but it starts from ISO1600 or ISO3200 depending on camera...

So, my question is still:
- why ISO500, if best dynamic range of sensor is at ISO100 (explained by DJI graphs)
- why ISO is fixed? underexposed image is big no-no in pro Log workflow.
- what are IRE values for D-Log at 0%, 18% and 90% reflectance? Have they changed in latest firmware?

Gybo102 Posted at 2017-3-8 19:50
In raw there is no option for DLOG or anything... its.. RAW.
So, I just want to verify... In RAW are we also stuck at iso 500? Since there is no color profile to select, I assume not, but I would like DJI to confirm this.

If DJI did not invent "new wheel" - RAW does not have ISO baked in. Data are recorded as they are read from sensor, at sensors baseline ISO, and just  metadata of intended ISO is recorded in order for  display software to apply adequate gain. So, RAW file data should  not be  changed in any way ny manipulating ISO during recording or during displaying.

DJI-CAO Posted at 2017-3-3 22:28
To explain the native ISO of DLOG, let's start with exposure.

1. Exposure

Hello DJI-CAO, few remarks more:
Regarding point 3: Log is not "digital gain". Even more - Log introduces less gain then regular Rec709.
Regarding point 4: Every pro. Log curve positions 18% below its position in Rec709 which is 50IRE. Arri does it at 39, Canon at 33, Sony at 32 IRE  (in order to make room for highlights). Therefore once more Log does not introduce gain, but it introduces luma compression which is opposite. I really do not understand how you count for positive exposure shift in middle gray... Can anyone explain?
Please look at very informative and correct explanation of Log at:
https://www.thebroadcastbridge.c ... eper-into-log-gamma
Look carefully at figure 9. which is explanatory by itself.

DrMrdalj Posted at 2017-3-9 12:55
Hello DJI-CAO, few remarks more:
Regarding point 3: Log is not "digital gain". Even more - Log introduces less gain then regular Rec709.
Regarding point 4: Every pro. Log curve positions 18% below its position in Rec709 which is 50IRE. Arri does it at 39, Canon at 33, Sony at 32 IRE  (in order to make room for highlights). Therefore once more Log does not introduce gain, but it introduces luma compression which is opposite. I really do not understand how you count for positive exposure shift in middle gray... Can anyone explain?

Well.

ISO500 means applying sensor_gain=1.0 (best dynamic range) and log gamma, the av and tv can be matched to standard metering formula sv=av+tv-bv,

so the native iso500 is a calculated ISO making your device align with other cameras. Please remember ISO100 = 1.0 sensor gain is not correct.

Again, gamma affects exposure. Log gamma will increase the sensitivity, so the native ISO increases, I.e., normal gamma iso100 and log gamma iso500, although sensor gain is kept 1.0x, the 18% gray is comparable.

In the engineering point of view, any amplification in the digital domain is called digital gain. So gamma is a kind of digital gain which is dependent to intensity, not a global gain (ISP gain).

Fixing ISO=500 means we recommend 1.0x sensor_gain currently. Log gamma applies a huge digital gain in the dark side. It might make horizontal fix pattern noise (HFPN) noticeable if the sensor gain is too large. How many sensor gains we can enable depends on the sensor characteristics. We are still test ISO1000 and ISO2000, which correspond to sensor gain 2.0 and 4.0. So it might be included in the next FW.

DJI-CAO Posted at 2017-3-9 18:16
Well.

ISO500 means applying sensor_gain=1.0 (best dynamic range) and log gamma, the av and tv can be matched to standard metering formula sv=av+tv-bv,

DJI-CAO, thank for clarification.
I still find confusing having same gain (1.0x) called ISO 100 when Rec709 curve is applied, and calling same gain (1.0x) called ISO500 when Log curve is applied - more so as LOG curve has LESS OVERAL GAIN then Rec709. You can ch computer with engineers to whiteness that 18% gray falls at lower IRE in Log then in Rec709 transformation.
Why there are such misalignment with regular pro marking and workflow?

DJI-CAO Posted at 2017-3-9 18:16
Well.

ISO500 means applying sensor_gain=1.0 (best dynamic range) and log gamma, the av and tv can be matched to standard metering formula sv=av+tv-bv,

DJI-CAO, thanks for the clarification.
Nevertheless, I still find confusing that same gain (1.0x) is called ISO100 when Rec709 curve is applied, while same gain (1.0x) is called ISO 500 when Log curve si applied - more so as Log curve has LESS OVERAL GAIN then Rec709. You can check out with any pro colorist or engineer to whiteness that 18% has LOWER IRE value (output value)  in Log then in Rec709.

So, I still find confusing your statement that gamma affects exposure - exposure multiplied by gamma  affects output values (such as IRE). Exposure measures "volume of light" while output is "Exposure x Gamma curve" ...

DrMrdalj Posted at 2017-3-10 03:23
DJI-CAO, thank for clarification.
I still find confusing having same gain (1.0x) called ISO 100 when Rec709 curve is applied, and calling same gain (1.0x) called ISO500 when Log curve is applied - more so as LOG curve has LESS OVERAL GAIN then Rec709. You can ch computer with engineers to whiteness that 18% gray falls at lower IRE in Log then in Rec709 transformation.
Why there are such misalignment with regular pro marking and workflow?

Let me tell you more details.

Native ISOs (Rec709 gamma):

Panasonic GH4: 200
Red Dragon: 250
Nikon D600/610: 100
Leica M: 150
SONY RX100: 125

So some cameras say ISO100 is extension because their 1.0x sensor gain is larger than 100. To make system easy to understand, they implement the extended ISO by gamma.

DrMrdalj Posted at 2017-3-10 03:33
DJI-CAO, thanks for the clarification.
Nevertheless, I still find confusing that same gain (1.0x) is called ISO100 when Rec709 curve is applied, while same gain (1.0x) is called ISO 500 when Log curve si applied - more so as Log curve has LESS OVERAL GAIN then Rec709. You can check out with any pro colorist or engineer to whiteness that 18% has LOWER IRE value (output value)  in Log then in Rec709.

Artists and engineers might have different understandings of exposure. However, they are just two profiles of the same thing.

Your measurement is in the linear space. My measurement is in the gamma space. Actually mine is more close to the film era, that is:

linear sensor + gamma = film

Then you understand my calculation of native ISO.

DJI-CAO Posted at 2017-3-10 03:53
Artists and engineers might have different understandings of exposure. However, they are just two profiles of the same thing.

Your measurement is in the linear space. My measurement is in the gamma space. Actually mine is more close to the film era, that is:

It happens that I am educated as engineer but work as artist with strong technical background, so I think I can understand both sides. I understand principle of extending ISO by analog and digital gain curve -that happens in camera processing pipeline prior to output gamma. Therefore I find Rc709 as well as Log curves to be just output transformations of same preprocessed signal (exposure x analog gain x digital gain)...  Naturally, an engineer can sum all pre and post curves in one transformation, but that does not change the result, and does not change principle of "exposure x gamma = output"

Therefore for same poressing gain (call it ISO) and with same illumination (call it aperture x shutter speed) we should get same exposure, while resulting Log recording will have lower output then Rec709 due to different curve applied over exposure. So, if I record Rec709 on SD card in I2, while in the same time I record Log ProRes on CineSSD, there should be significant difference between these two recordings as Log curve has different nonlinear gain then Rec709.

What happens if I shoot Rec709 and Log simultaneously at ISO500? If I understand well, you implement same analog (1.0x) and same digital gain over both streams, which finally  differ just in Rec709 and Log gammas. What happens if I record Rec709 at ISO400 - is there still analog gain 1.0x and digital gain of 2 stops, or analog gain is engaged?  I would not like to expose any engineering secrets, but just would like to stress that Log gamma has less overal gain then Rec709 (in both linear and logarithmic scale) for any given illumination or exposure - and therefore I have no clue why same gain is called ISO100 when you record just Rec709 but it is called ISO500 when you record Rec709+Log simultaneously.

What did I get wrong?

DJI-CAO Posted at 2017-3-10 03:53
Artists and engineers might have different understandings of exposure. However, they are just two profiles of the same thing.

Your measurement is in the linear space. My measurement is in the gamma space. Actually mine is more close to the film era, that is:

I know that this will seem oversimplified, but here  are few basic principles

(light) x aperture x shutter speed = sensor illumination

sensor analog gain x digital gamma gain = overall gain (ISO)

sensor illumination x overall gain = exposure

exposure x output gamma = output signal

Output signal is strongly dependent of output color standard, while exposure is not dependent of color standard.

DrMrdalj Posted at 2017-3-10 05:09
It happens that I am educated as engineer but work as artist with strong technical background, so I think I can understand both sides. I understand principle of extending ISO by analog and digital gain curve -that happens in camera processing pipeline prior to output gamma. Therefore I find Rc709 as well as Log curves to be just output transformations of same preprocessed signal (exposure x analog gain x digital gain)...  Naturally, an engineer can sum all pre and post curves in one transformation, but that does not change the result, and does not change principle of "exposure x gamma = output"

Therefore for same poressing gain (call it ISO) and with same illumination (call it aperture x shutter speed) we should get same exposure, while resulting Log recording will have lower output then Rec709 due to different curve applied over exposure. So, if I record Rec709 on SD card in I2, while in the same time I record Log ProRes on CineSSD, there should be significant difference between these two recordings as Log curve has different nonlinear gain then Rec709.

Almost correct.

The architecture of I2 is quite different from cinema camera. I'm sorry there are some technical secrets cannot be exposed.

When you shoot Arri RAW, you don't need to care about LOG or Rec709. They will tell you the workflow can re-select the gamma. Actually the raw footage is determined after exposure. So the raw can't be changed any more.

1. the exposure is based on the LOG gamma;
2. the intermediate color space is based on the Arri wide Gamut;
3. the final output is selectable, a. LOG, b. LOG->REC709

So the principle is still under-exposure to preserve highlights. The Rec709 is quite different from consumer cameras like SONY alpha 7. It's more like a HDR video.

I2 need to cover different customers and applications, i.e., RAW/ProRes and H265/264 for cinema, documentary, TV show, aerial hobby videos. I2 also need to cover both requirements of photography and cinematography. So the Rec709 of I2 is not from the LOG->REC709. It is a real Rec709 equivalent to the cameras like Panasonic GH4, SONY alpha 7, ... The DLOG is a real LOG gamma similar to V-LOG, C-LOG etc... This is why the exposure of DLOG and Rec709 are different. To compensate the difference in APEX system, the native ISO of DLOG = 2.3ev + ISO100 of Rec709 = ISO500

IOW DJI-CAO, cDNG recorded footage really isn't RAW. It's DJI's debayered version of RAW, recorded in cDNG, 16 bit. Still better than 14(?)-bit compressed ProRes, but, still (losslessly) compressed. Point is, a gamma curve has been applied to cDNG, i.e. Dlog.
As to your point that DJI's REC 709 is like HDR, that sounds like hype, to me. REC709, by definition has a dynamic range of 7 stops. So, if your sensor is seeing something like 12.5 stops, you still have to fit that into a 7 stop REC709 representation. So, your cDNG stores the debayered RAW data as something that's not REC709, but some HDR variant like Arri DI. In fact, using a ARRI IDT in Resolve, and a REC709 ODT, I can get a pretty decent  image out of ACES CCt.

DJI-CAO Posted at 2017-3-10 05:45
Almost correct.

The architecture of I2 is quite different from cinema camera. I'm sorry there are some technical secrets cannot be exposed.

I understand that APEX system is reason for adding 2.3ev  gain, but I have no clear expectation of resulted  output. Presume that we record Log ProRes and Rec709 H264 on I2 X5S - would you be so kind to share D-Log and DJI Rec709 expected theoretical dynamic range and  output values (IRE) for 0%, 18%, 90% and max IRE.

DJI-CAO Posted at 2017-3-10 05:45
Almost correct.

The architecture of I2 is quite different from cinema camera. I'm sorry there are some technical secrets cannot be exposed.

I understand that APEX system is reason for adding 2.3ev  gain, but I have no clear expectation of resulted  output. Presume that we record Log ProRes and Rec709 H264 on I2 X5S - would you be so kind to share D-Log and DJI Rec709 expected theoretical dynamic range and  output values (IRE) for 0%, 18%, 90% and max IRE.

DrMrdalj....If you google "LUTCALC", you'll find a free software utility that calculates gamma curves and LUT corrections for various color spaces and gammas. The author has reverse engineered the DJI Dlog gamma curve. According to his calculations (not saying they're right) 8 bit IRE values for Dlog are 0=0, 18=50, 90=86.

DJI-CAO Posted at 2017-3-10 05:45
Almost correct.

The architecture of I2 is quite different from cinema camera. I'm sorry there are some technical secrets cannot be exposed.

I think lost in this discussion is the quality of the new D-Log itself. I think a lot of consternation regarding locking the ISO at 500, is based on experience with the old D-Log which was, especially in H264, a steaming, artifact filled, pile of something that this forum software will not allow me to say. It's hard to imagine that log pushed to 500iso being anything better, and so people have reacted negatively.
I've run some tests on the new D-Log and posted them over at RCGroups (there are 4 posts over several pages). The good news is that the new, Locked D-Log500 is a substantial improvement over the old log, especially in H264 (usable? yes!) with more standardized clip points (~95ire* instead of the previous ~62ire), generally better noise characteristics, more linear color, and a more "log-like" look. Its much easier to grade (actually quite simple to do without a LUT --in the same way the original Canon Log was easy to grade this way). If you're interested, head on over to the link below and take a look.

*That said, there is an error in the encoding H264 version that DJI should address, regarding the clipping point (red channel stops at 95 ire as it should, but blue and green max out at 100ire. In a funny way it works almost like an overexposure warning, but it really shouldn't function this way. This is visible in Go4 and the NLE, so it doesn't appear to be a metadata error. It's also different from how the ProRes file deals with the clipping point (there all three channels peak at ~95ire as they should).

Finally, CAO, you've already said as much, DJI Locking the ISO at 500 is in a way, your recommendation that it is the best tradeoff between noise and highlight protection. You have mentioned several times the future option extending this ISO by 2x or 4x. The reality is that most cinematographers I know don't push log exposure this way, because it exposes the noise in an already gained shadow. It's imperative that the photographer be able to go the other way  .5x or .25x  --Iso 250 or 125. There are many situations where the extra highlight detail is NOT needed, and so choosing a lower noise option is preferred. Based on my experience with D-log so far, it has plenty of headroom. There's no reason why I wouldn't expose this at a lower value to get better noise characteristics if I could.

A few final requests. DJI has a way of working on improvements that no-one ever asked for. Mostly that's appreciated, as they are still improvements. However, relative to the camera there are some things that just don't compute and should be fixed Listed below:

1) new Go4 app only allows color looks like d-cinelike to be applied to H264, and only when the SSD is disengaged. It would be nice to have these returned to the ProRes side of the equation. Right now the only options are D-Log (nice!) and Standard (ugh). Overall, the organization of the photo settings took a  big dive on this last update.

2) Lack of Lens Corrections on ProRes - apparently you're not going to fix this, but its a serious error in my opinion. This is a classic, tons of people requesting it, ( one or two bad*sses preferring to do it in post, and walk to school in the snow), and instead what we get is the ability to turn the corrections off in H.264. You have to love the irony. Anyone who prefers the distorted version of the stock DJI lens probably should own a mavic and a skateboard.

3) Incorrect lens correction for the 25mm Oly Lens in RAW results in largely uncorrectable, comical chromatic aberrations.

4) Green channel is consistently "low" at all kelvin settings. Neutral tones are almost always slightly red/pink. What color grey card do you guys use to calibrate these things? :-)

Overall, I'm really quite happy with DLog500. Hopefully others will be too and be a little happier waiting for the "unlocking" in the future.

The Quirky, Fantastic new D-Log500

Barry Goyette Posted at 2017-3-10 07:26
I think lost in this discussion is the quality of the new D-Log itself. I think a lot of consternation regarding locking the ISO at 500, is based on experience with the old D-Log which was, especially in H264, a steaming, artifact filled, pile of something that this forum software will not allow me to say. It's hard to imagine that log pushed to 500iso being anything better, and so people have reacted negatively.
I've run some tests on the new D-Log and posted them over at RCGroups (there are 4 posts over several pages). The good news is that the new, Locked D-Log500 is a substantial improvement over the old log, especially in H264 (usable? yes!) with more standardized clip points (~95ire* instead of the previous ~62ire), generally better noise characteristics, more linear color, and a more "log-like" look. Its much easier to grade (actually quite simple to do without a LUT --in the same way the original Canon Log was easy to grade this way). If you're interested, head on over to the link below and take a look.

Thanks for the example and comment,
If I understand your point - you say that D-Log is closer to Canon WideDR gamma then to pro-grade log curves (Cineon, C-Log2, S-Log2, V-Log)... All pro log curves strongly protect highlights but compress shadows, so cinematographers intentionally  overexpose (expose-to-the-right) - if I understand your point you find D-Log to act more as WideDR so you expose for center?

Regarding photographers- they do not need Log curve at all, as they can shoot pure DNG RAW, that is what they usually do. But for cinematographers, I still can not understand how and why there are so many basic issues for camera which costs almost 4000 EUR (with licenses). I think of changing to M600 just for problems with X5 camera  line

Barry Goyette Posted at 2017-3-10 07:26
I think lost in this discussion is the quality of the new D-Log itself. I think a lot of consternation regarding locking the ISO at 500, is based on experience with the old D-Log which was, especially in H264, a steaming, artifact filled, pile of something that this forum software will not allow me to say. It's hard to imagine that log pushed to 500iso being anything better, and so people have reacted negatively.
I've run some tests on the new D-Log and posted them over at RCGroups (there are 4 posts over several pages). The good news is that the new, Locked D-Log500 is a substantial improvement over the old log, especially in H264 (usable? yes!) with more standardized clip points (~95ire* instead of the previous ~62ire), generally better noise characteristics, more linear color, and a more "log-like" look. Its much easier to grade (actually quite simple to do without a LUT --in the same way the original Canon Log was easy to grade this way). If you're interested, head on over to the link below and take a look.

1) cinelike of raw/prores will be included in the future FW.

2) we have difficulties to correct the warping because M43 lenses are comapct and mostly with large distortion. but we still keep on trying.

3) in the next upgrade

4) I read your posts in RCG. Making a LOG for 8bit 420 H264 isn't a happy work. Regarding ProRes, the color tunning is needed after adding contrasts. Simply mapping to Arri and Canon  doesn't work although the color space is close to Arri wide gamut. I prefer CinemaDNG because it is easy to map the footage to any of my favourite cameras. You can try Log-C.

DrMrdalj Posted at 2017-3-10 08:18
Thanks for the example and comment,
If I understand your point - you say that D-Log is closer to Canon WideDR gamma then to pro-grade log curves (Cineon, C-Log2, S-Log2, V-Log)... All pro log curves strongly protect highlights but compress shadows, so cinematographers intentionally  overexpose (expose-to-the-right) - if I understand your point you find D-Log to act more as WideDR so you expose for center?

A couple of things. No I wouldn't compare D-log500 to WideDR gamma, which is a linear gamma. D-log appears to be a log gamma, similar to CLog or CLog3 , but "probably" is not a cineon style log like Arri's Log-C or Canons CLog2. I say probably because DJI hasn't published any data regarding it's encoding and I haven't done the type of test that would determine the "shape" of the log (cineon style logs are typified by a long flat mid section that essentially allows you to adjust exposure within a certain range in post with little penalty in the highlights or shadows). Pseudo logs like Clog, and maybe, D-log have raised shadows (that are compressed somewhat compared to a cineon type gamma, but less than a linear gamma) and a steeper, often less-straight middle portion of the curve. These types of gammas are easier to grade and view, and generally have less noise issues in the shadows.

I've heard a number of things said on this forum that are simply not true, one is that in a linear gamma there is more information in the shadows than in the highlights. In fact the opposite is true, and is one of the reasons why Log Gammas are so valuable. I know it looks like shadows are compressed when you look at a log gamma curve, but the reality is that compared to a linear gamma, the discrimination of shadow values is greatly expanded...In reality, the entire range is being redistributed with (relatively) equal amounts of coding values for each unit of input, instead of linear gamma's extreme favoring of data in the highlights (one of the reasons ETTR became so popular).

When you look at a log gamma curve, it's easy to reference what a linear gamma curve looks like and say that the shadows are compressed, because the midpoint has been lowered. A neutral linear gamma curve is a straight line, which belies the fact that there are many, many less encoding values below 50% than above.  Log Gammas are different. Each stop in a log gamma has almost the same number of encoding values. So when sony moves the midpoint lower in a log curve What's going on is not compression,  but rather a translation of those tones to allow for a greater highlight range.  For instance, you've sited that middle grey is often placed below 40% in log gammas. The reason for this is so that reflected white, the brightest non source, non-specular tone that can be recorded, which typically falls at 91ire in rec709. can be placed at ~60ire. to leave more room for expanded highlight range. As we increase DR and expand that  highlight range, All the tones move down the scale, but in a relatively equal fashion. In a way they all compress, (relative to a camera with lower DR, anyway) This is why, as we increase DR, recording at a higher bit level becomes important, because it allows for a greater number of encoding values for each stop of range.

So for the record, Shadow tones are not compressed in LOG. They are, compared to linear gammas, expanded. Looking at a photo (not the curve) shot in log and comparing it to one shot in Linear, this should be obvious. When you apply a LUT to a log gamma image...what happens?...it compresses both the highlight and shadow tones to look more like a linear gamma. This would be hard to do if they had already been compressed compared to a linear gamma.

Now...Cinematographer's exposing LOG to the right -- There isn't technically much value to ETTR in log. For instance when you shoot RAW stills, your camera and ACR show you 7-8 stops of range relatively centered on the linear RAW range it's recorded. Because it's linear, the information below that 7-8stops has exponentially less data points than the information above that 7-8 stops displayed on your screen. So photographers choosing to ETTR are moving the exposure more towards the area where there are many more data points (encoding values or gradations).

In Log it's different. Each stop contains relatively the same number of encoding values, and so the same number of gradations within each stop. The thing is, as we get to the lower end of the log scale, because the shadows have generally been expanded, we start seeing all the crap in the signal...the noise that was recorded because light IS linear, not log..meaning even though the LOG stop has the same number of encoding values--gradations--the light itself and the sensors ability to record it fall off dramatically. So when shooting Log, generally as long as you are in the middle portion of the curve, there is no difference whether you expose a little to the right or a little to the left...each tone has the same amount of info to work with, so you just adjust in post as you desire. However, you want to steer clear of  both endpoints --- clipping in the highlights and the deepest noise in the shadows, because in log...these are true endpoints, unlike how ACR and your camera show you RAW.

Take a shot like the one I posted on RC Groups yesterday. It has important deep shadows and sunlit portions and sunlit clouds,  yet I still had probably 2-3 stops of adjustment where I could get a good exposure. In this situation, I could choose to simply Expose to the right and probably eliminate or minimize most noise in the shadows. However, If I had a client looking over my shoulder, I'd probably have to spend a few minutes explaining why the image looked overexposed...that I'd fix in in post. In that case, or for someone  like me who really wants to WYSIWYG it...(perhaps a misnomer in LOG), it's important to be able to adjust ISO rather than simply increasing EV at a higher ISO. Thats why we need the option of choosing a lower ISO for D-Log.

DJI-CAO Posted at 2017-3-10 08:49
1) cinelike of raw/prores will be included in the future FW.

2) we have difficulties to correct the warping because M43 lenses are comapct and mostly with large distortion. but we still keep on trying.

Thanks for your answers, CAO. Two things. I think DJI has done an admirable job with the H264 version of D-log500. Other than that highlight problem (Canon had trouble with that last stop too!), it's a very nice implementation. I appreciate all the "unhappy" work!.

Second. Your answer to 2) leaves me confused. DJI seems to be applying lens corrections to the H264/h265 footage just fine, and the metadata attached to the RAW files works generally fine except for the 25mm. Why is ProRes the black sheep of the family? Why is it harder to correct than the others? I'd always assumed it was a processing hardware issue, but your answer indicates otherwise...can you explain?....

(and for the record...said many times here, but not to you by me -- ProRes is a production codec. It's main purpose is ease of use and high quality. Some folks seem to think it is akin to RAW, and so it shouldn't have the corrections but this is an error in thinking --ProRes has ISO,  white balance, gamma and a number of other parameters "baked in". Lens corrections are an "automatic" feature of M4/3 even in RAW--you can't even turn them off in ACR, and there is no current method to correct them accurately in post. I really hope that DJI can find a way to add them..soon. The only saving grace is that D-Log500 in H264 is quite nice...and for the wider(more distorted) stuff, will have to do until we get a better solution for ProRes.

Barry Goyette Posted at 2017-3-10 10:08
A couple of things. No I wouldn't compare D-log500 to WideDR gamma, which is a linear gamma. D-log appears to be a log gamma, similar to CLog or CLog3 , but "probably" is not a cineon style log like Arri's Log-C or Canons CLog2. I say probably because DJI hasn't published any data regarding it's encoding and I haven't done the type of test that would determine the "shape" of the log (cineon style logs are typified by a long flat mid section that essentially allows you to adjust exposure within a certain range in post with little penalty in the highlights or shadows). Pseudo logs like Clog, and maybe, D-log have raised shadows (that are compressed somewhat compared to a cineon type gamma, but less than a linear gamma) and a steeper, often less-straight middle portion of the curve. These types of gammas are easier to grade and view, and generally have less noise issues in the shadows.

I've heard a number of things said on this forum that are simply not true, one is that in a linear gamma there is more information in the shadows than in the highlights. In fact the opposite is true, and is one of the reasons why Log Gammas are so valuable. I know it looks like shadows are compressed when you look at a log gamma curve, but the reality is that compared to a linear gamma, the discrimination of shadow values is greatly expanded...In reality, the entire range is being redistributed with (relatively) equal amounts of coding values for each unit of input, instead of linear gamma's extreme favoring of data in the highlights (one of the reasons ETTR became so popular).

Thanks for your effort to make everything clear.

I do not intend to deviate this discussion in technical direction, as t is mainly devoted to,proper use of new D-Log. Nevertheless, you had many good point in your explanation but I still find some of them f to be misleading, so I will try to point out some of them which I find important. First of all, you can not discuss "luma compression" without accounting for output color space. If output color space is Rec709, as it is on your display, then it is baseline for discussion. Please note that correctly exposed Rec709 will have 3 stops below and 3-4 stops above middle gray, while correctly exposed S-log2 has an exposure range of 6 stops above middle grey and 8 stops under. When you account that in Log  middle gray is positioned lower in encoding values, you will understand wha aret "compressed shadows" - you can not lift shadows without introducing compression artifacts and noise, so people use ETTR to compensate...

So,  Log gamma is not what I would prefer in H264.. For 8 bit H264 some WideDR profile is all what is needed, call it Cinelike, Cinestyle or anything...

On the other hand while using X5S ProRes422HQ in 10bit I would prefer proper documented real Log, with recommended LUTs, and with adjustable gain(ISO) in order to intentionally manipulate DR/SNR tradeoffs when shooting in dusk or dawn...

I really can not understand what would Log or any other gamma would bring to do with RAW CinemaDNG recording, as gammas are intended for liner recording media. For me RAW has to have color gamut defined and everything afterwards is up to the software LUTs.

For client over the shoulder and WYSIWYG there are tons of LUT monitors which can display even graded look - that is not what is expected from Log gamma.

This thread seems to be wandering all over the place.
In regards to the OP's original question, some DIT's have expressed the opinion that, when dealing with log gammas, a different gamma curve has to be defined for each ISO setting. It may well be that there are hardware reasons for DJI to pick a single ISO, or,at least a limited number of them.

raven4 Posted at 2017-3-10 12:45
This thread seems to be wandering all over the place.
In regards to the OP's original question, some DIT's have expressed the opinion that, when dealing with log gammas, a different gamma curve has to be defined for each ISO setting. It may well be that there are hardware reasons for DJI to pick a single ISO, or,at least a limited number of them.

I would be glad if there would be limited number of them, instead of just one fixed value.After all this wandering I can say that I learned that at D-Log500 sensor works at gain 1.0x (same as at ISO100) but digital gain is applied to shift exposure curve according to APEX parameters - it still does not have sense to me, but it must be that there are some sense in it...I will do the tests in next two days and measure for myslef how to expose correctly.

Remark: If this new D-Log is completely different curve then previous D-Log (present on X5 and X5R) why is it called the same? Wouldn't it be reasonable to call it D-Log2 or D-Log2017 or something to distinguish it from previous D-Log (V-Log L)?

DrMrdalj Posted at 2017-3-10 12:33
Thanks for your effort to make everything clear.

I do not intend to deviate this discussion in technical direction, as t is mainly devoted to,proper use of new D-Log. Nevertheless, you had many good point in your explanation but I still find some of them f to be misleading, so I will try to point out some of them which I find important. First of all, you can not discuss "luma compression" without accounting for output color space. If output color space is Rec709, as it is on your display, then it is baseline for discussion. Please note that correctly exposed Rec709 will have 3 stops below and 3-4 stops above middle gray, while correctly exposed S-log2 has an exposure range of 6 stops above middle grey and 8 stops under. When you account that in Log  middle gray is positioned lower in encoding values, you will understand wha aret "compressed shadows" - you can not lift shadows without introducing compression artifacts and noise, so people use ETTR to compensate...

--So,  Log gamma is not what I would prefer in H264.. For 8 bit H264 some WideDR profile is all what is needed, call it Cinelike, Cinestyle or anything...

Until yesterday, I would generally agree with you relative to D-Log. I used it once on my P4 and other than a quick test on the I2 a few months back, I've only used d-cinelike when shooting in H264. After my first tests of the new D-Log500 yesterday, I feel comfortable using it as an 8bit log, with all the limitations that implies. Quality 8bit logs aren't unheard of, and certainly can be used when "normal" grading is the goal. (Canon had one of the most popular cinema grade cameras in history with nothing but an 8 bit log gamma.)  In ProRes I think D-Log500 is outstanding, certainly the most pro grade gamma DJI has ever produced (outside of raw).

Barry Goyette Posted at 2017-3-10 10:34
Thanks for your answers, CAO. Two things. I think DJI has done an admirable job with the H264 version of D-log500. Other than that highlight problem (Canon had trouble with that last stop too!), it's a very nice implementation. I appreciate all the "unhappy" work!.

Second. Your answer to 2) leaves me confused. DJI seems to be applying lens corrections to the H264/h265 footage just fine, and the metadata attached to the RAW files works generally fine except for the 25mm. Why is ProRes the black sheep of the family? Why is it harder to correct than the others? I'd always assumed it was a processing hardware issue, but your answer indicates otherwise...can you explain?....

Well, X5S uses a 20.8Mp sensor which outputs 5.2K, 15.4Mp in 16:9 mode. The readout time of 15.4Mp is only 13ms, similar to SONY FS7. It shall improve the rolling shutter effect. However,the DDR throughput is huge, especially in 444 12 bits. So we have to reduce the times of the pixels passing through DDR. H264 doesn't have this issue because the pixel is only in 420 8bit.

M43 system has excellent lenses. However, most of them are designed for photography, not for cinema. The distortion correction of lenses in 4/5K mode may require tens of line-height which forces the pixels go through DDR.

CinemaDNG has a lens profile inside. It can be recognized by Adobe workflow. However, Premiere and Ae both have bugs to support CinemaDNG(!!@@##). We are pushing them to fix it right now. It's strange Davinci Resolve can't recognize the lens profile and we are contacting them about it.

DJI-CAO Posted at 2017-3-10 17:40
Well, X5S uses a 20.8Mp sensor which outputs 5.2K, 15.4Mp in 16:9 mode. The readout time of 15.4Mp is only 13ms, similar to SONY FS7. It shall improve the rolling shutter effect. However,the DDR throughput is huge, especially in 444 12 bits. So we have to reduce the times of the pixels passing through DDR. H264 doesn't have this issue because the pixel is only in 420 8bit.

M43 system has excellent lenses. However, most of them are designed for photography, not for cinema. The distortion correction of lenses in 4/5K mode may require tens of line-height which forces the pixels go through DDR.

Thanks so much for the response and the information. So what you seem to be saying is the H264 is actually corrected late in the processing chain. I would have assumed otherwise, but if the processing is that intensive relative to the hardware, then I guess it would be the only way to go. Let's hope it is possible in the future, perhaps even in the 422hq version if the 4444 version is impossible.

One final thing. (totally off topic, but you mentioned the data throughput of 444 12bit, so I'll ask). Are you actually outputting a 12bit flavor of 4444xq? We did some stress testing here and saw very little difference between the 4444xq and the 422hq (a slight increase in color resolution (the 444), but zero difference in  gradation) leading me to believe that the 4444xq was actually being delivered at 10bit. Care to comment?  :-}. Did I say how nice it is to have someone from the engineering side here on the forum? :-]

DJI-CAO Posted at 2017-3-10 17:40
Well, X5S uses a 20.8Mp sensor which outputs 5.2K, 15.4Mp in 16:9 mode. The readout time of 15.4Mp is only 13ms, similar to SONY FS7. It shall improve the rolling shutter effect. However,the DDR throughput is huge, especially in 444 12 bits. So we have to reduce the times of the pixels passing through DDR. H264 doesn't have this issue because the pixel is only in 420 8bit.

M43 system has excellent lenses. However, most of them are designed for photography, not for cinema. The distortion correction of lenses in 4/5K mode may require tens of line-height which forces the pixels go through DDR.

DJI-CAO, I would like to thank you personally and officially for you time and effort to share detailed info with us. I find it so rare that there is highly qualified DJI person engaged in the forum community, therefore your presence is much more appreciated.

Regarding ISO500 D-Log vs ISO100 Rec709, what was main point of my confusion, after watching some footage it still seems to me that 500 mark is "just naming convention" as these look very close (as explained already in this conversation).

There is naming confusion in picture and color profiles as well. Until this update X3 had its own flavor of D-Log (call it pseudo-log or regular-gamma-log), while X5 line had D-Log which was of Arri style (very similar to Panasonic V-Log L).These two could work in all camera gains (call it ISO) so user could balance DR/noise tradeoffs by himslef.  Now there is third flavor of D-Log (call it "500") which was not clearly explained outside of this conversation, and which happen to strike current prducts which are already in use (I2, P4P). I kindly sugesst that you communicate these change more cleary in general, and to consider that I2 platform is used (as advertised) by real professionals who do not prefer to get involved in such fundamental changes without preparation. and clear understanding..

Regarding other topics raised by Barry lens correction, etc) I second to them fully.
Thank you once more dor you engagement in this conversation.

Barry Goyette Posted at 2017-3-10 19:07
Thanks so much for the response and the information. So what you seem to be saying is the H264 is actually corrected late in the processing chain. I would have assumed otherwise, but if the processing is that intensive relative to the hardware, then I guess it would be the only way to go. Let's hope it is possible in the future, perhaps even in the 422hq version if the 4444 version is impossible.

One final thing. (totally off topic, but you mentioned the data throughput of 444 12bit, so I'll ask). Are you actually outputting a 12bit flavor of 4444xq? We did some stress testing here and saw very little difference between the 4444xq and the 422hq (a slight increase in color resolution (the 444), but zero difference in  gradation) leading me to believe that the 4444xq was actually being delivered at 10bit. Care to comment?  :-}. Did I say how nice it is to have someone from the engineering side here on the forum? :-]

Well, the following pipeline is used in the ProRes of the CineCore 2.0. It ensures the full dynamic range and computational accuracy as well.

Sensor input (10/12/14) -> RAW Processing (16) -> Bayer2RGB(14) -> Color(14) -> Gamma (12) -> RGB2YCbCr (12/10) -> ProRes(12/10)

Actually, the ProRes in CineCore support 12bits encoding.

But we have to cut 2 bits in RGB2YCbCr in case of XQ to save DDR bandwidth. The truncation in the 12 bit gamma space won't reduce the dynamic range. The optimization of the DDR bandwidth is ongoing and we got some solid results.

Personally, two things make the difference between 444 and 422 unnoticable.

1. Bayer sensor. The R/B is naturally halved compared to G. Because Y is mostly determined by G, the resolution of UV is lightly lower than Y and the loss of UV is not noticable.

2. The loss of UV resolution may blur high saturated color boundaries. However, they're relatively rare in the natural world and human vision is not sensitive to chroma.

Please use 444XQ in your high end projects to earn more because they are definitely convincing to the customer.

DJI has stated that the color values are shared between video and photos. Apparently they're also shared between SSD and SD. This appears to be an example of DJI removing a capability. Why not be able to simultaneously record with two different 'Looks' and 'Color' profiles between the two codecs. SSD and SD are already two entirely different profiles. I'm also very confused by the implementation regarding Looks vs. Color. See attached photos.

Photo 1 - shows 'Color' options with SSD turned off
Photo 2 - shows what is now 'Looks' options with SSD turned on.
Photo 3 - shows 'Color' options with SSD turned on. Color is no longer an option as it's defaulted to 'Rec.709'.

When SSD was turned off, 'Color' had an option for D-Log. When SSD is turned on, D-Log is now only an option under 'Looks'. What is going on here? Where is the consistency? Why must these be shared values? I'm so confused by this.

DJI-CAO Posted at 2017-3-3 22:28
To explain the native ISO of DLOG, let's start with exposure.

1. Exposure

DJI-CAO,

Thank you for the explanation.  An issue I'm running into when I have D-Log selected, is that shooting RAW stills lock the ISO to 500.  There shouldn't be any color profile applied to RAW stills and the ISO should be unlocked.

It's such a hassle to switch between shooting D-Log videos and stills because we have to go through the menus to switch to a different color profile.

Let me go one step further to suggest that the camera settings for videos and photos should have separate "sticky" settings.  Because for video, the ideal shutter speed is around 1/50 or 1/60 and for stills we want to shoot faster shutter speeds.

Thanks.

yoengel Posted at 2017-3-25 10:28
DJI-CAO,

Thank you for the explanation.  An issue I'm running into when I have D-Log selected, is that shooting RAW stills lock the ISO to 500.  There shouldn't be any color profile applied to RAW stills and the ISO should be unlocked.

Actually, shooting still in DLOG is not for the photography purpose. It shoots a sample raw for quick grading. Regarding splitting settings of still and video, I agree with you.

raven4 Posted at 2017-3-5 04:02
ISO and ev  are not the same thing. However, ISO and GAIN are the same. The analogy can be made to film. On a film camera, the more sensitive the film emulsion is to light, the higher the rated ISO. On a digital sensor, the higher the bias voltage(GAIN), the more sensitive the sensor is to light. In both cases, the price you pay for boosting the gain or having more light sensitivity, the more grain or noise you'll get. High speed film is also grainy.

Likewise, with film, if you take a film stock(which is a fixed ASA or ISO) and intentionally underexpose it, then in processing the film, you can overdevelop the film to recover the image. With digital, the same thing can be done. You can underexpose the image, then by using a LUT or boosting the shadows/mids/hi's in a program like Davinci Resolve, you can recover the image. The price you pay with film is, yes, there is more grain (noise) than properly exposing the film. Now, with digital, it's a little different.

raven4 said: (side note: I use a ATOMOS FLAME HDR monitor because it doesn't need a LUT to properly display an HDR image; and, a LUT is built into the display)(sorry, could not figure out how to quote parts of a post)

May I ask you, what your settings on the Atomos Flame are? Once you activate AtomHDR you can chose among Camera (Sony, Red, Panasonic, JVC, but not DJI) Gamma and Gamut. Any recommendations?


Best, Andreas

Andi_U Posted at 2017-4-16 02:34
raven4 said: (side note: I use a ATOMOS FLAME HDR monitor because it doesn't need a LUT to properly display an HDR image; and, a LUT is built into the display)(sorry, could not figure out how to quote parts of a post)

May I ask you, what your settings on the Atomos Flame are? Once you activate AtomHDR you can chose among Camera (Sony, Red, Panasonic, JVC, but not DJI) Gamma and Gamut. Any recommendations?

Best setting I've found is for ARRI.

DJI-CAO Posted at 2017-3-5 05:00
Actually the DLOG is improved in the latest FW.

We will soon release a tutorial to help customers correctly handle the DLOG using Davinci Resolve..

Ah ok, so everybody is using Davinci?, make tutorials of every pro edit software!
to much to ask?

I'm hoping DJI will get back {Color mode dropdown menu} in the next version of the app. And unlock ISO please. Otherwise, everything is cool.

DJI-CAO Posted at 2017-3-9 18:16
Well.

ISO500 means applying sensor_gain=1.0 (best dynamic range) and log gamma, the av and tv can be matched to standard metering formula sv=av+tv-bv,

Hello,
thank you so much for all your information. I have a few questions I am hoping you can answer. I am no expert so please bear with me as I ask these questions.  I have been reading your posts on DLOG and was wondering if you could tell me:
- what are the IRE for DLOG at 0%, 18% and 90%
- how many stops over you recommend for exposure to have the least amount of noise.
- what is the noise floor for DLOG in IRE, what IRE level do I want my shadows above for the least amount of noise

Thank you, Chris