If you are here to understand (why) equivalence (is wrong) then read this: https://dtmateojr.wordpress.com/2014/09/28/debunking-equivalence/
This post is practically a continuation of one of my controversial posts on debunking the myth of full frame superiority. In that previous post I discussed why full frame is actually no better than it’s crop sensor counterpart (Nikon D7000 vs D800) in terms of light gathering capability. Now I will try to discuss another aspect of full frame superiority and explain why it leads people to believe that it is superior to smaller sensor cameras when in fact it is not.
A common source of sensor performance data is DXOMark. This is where cameras are ranked in terms of SNR, DR, Colour depth, and other relevant aspects of the equipment. It is important to note that data from this website should be properly interpreted instead of just being swallowed whole. This is what I will try to cover in this post.
One of the most highly debated information from DXOMark is that of low light performance which is measured in terms of Signal to Noise Ratio (SNR). SNR is greatly affected by the light gathering capacity of a camera’s sensor and this is why this is commonly used to compare the low light performance of full frame and crop sensors. This is also the most misinterpreted data by full frame owners. They use this information to justify spending three times as much for practically the same camera. Let’s see why this is wrong…
Consider the following SNR measurements between the Nikon D7000 and D800:
Isn’t it quite clear that the Nikon D800 is superior to the D7000? Did I just make a fool of myself with that “myth debunking” post? Fortunately, I did not 🙂 I’m still right. That graph above is a normalised graph. DXOMark is in the business of ranking cameras and that is why they are forced to normalise their data. Let’s have a look at the non-normalised graph to see the actual SNR measurements:
Didn’t I say I was right? 🙂
The Nikon D7000 and D800 have the same low light performance! That is because they have the same type of sensor. The D800 is basically just the D7000 enlarged to full frame proportions. Simple physics does not lie. A lot of “photographers” have called me a fool for that “myth debunking” post. Well, I’m not in the business of educating those who are very narrow-minded so I will let them keep believing what they believe is true. But some of us know better, right? 🙂
Let’s not stop here. Allow me to explain why the normalised graphs are like that.
Let me tell you right now that DXOMark is unintentionally favouring more megapixels. That’s just the inevitable consequence of normalisation. Unfortunately, those who do not understand normalisation use this flaw to spread nonsense. The normalised graphs are not the real measured SNR values but are computed values based on a 8Mp print size of approximately 8×10. The formula is as follows:
nSNR = SNR + 20 x log10(sqrt(N1/N2))
where nSNR is the normalised SNR, N1 is the original image size and N2 is the chosen print size for normalisation. In the case of the Nikon D800, N1 = 36Mp and for the D7000, N1 = 16Mp. They are both normalised to a print size of N2 = 8Mp. Based on that formula, the D800 has a SNR improvement of 44.93 up from measured SNR of 38.4. The D7000 though only improves a tiny bit to 41 up from 38. As you can see, although both cameras started equally, the normalised values have now favoured the D800.
This increase in SNR is not because the D800 has better light gathering capability. This apparent increase in SNR is due to downsampling. It’s due to the larger image size and not because of better light gathering capability. Unfortunately, this computed SNR is what the full frame fanbois are trying to sell to uninformed crop sensor users. It is the REAL measured SNR that matters and we will learn later on how important this is compared to just more megapickles.
Go back to that normalisation formula and note the term inside the square root (N1 / N2). Note that if N1 is greater than N2 then the log10 becomes a positive number and the whole term adds to the measured SNR. The term drops to zero for N1 = N2 and that’s why when a D800 image is printed at 36Mp, the SNR is the measured SNR. Same goes for the D7000 when printed at 16Mp. That is why when I blogged about noise performance comparisons I kept repeating that images should be printed at their intended sizes. That’s the ONLY fair comparison. Downsampling is cheating. You do not want to buy a 36Mp camera so you could print it at 8×10. That is an absolute WASTE of money.
The idiots will of course justify by saying “well the good thing with having a larger image is that you can downsample and it will outperform a smaller image“. Well not so fast, young grasshopper. That is not true. We know that SENSEL size generally results in better light gathering capacity (Rain Can Teach Us Photography) although this means smaller image size. Let’s consider the D800 vs D4:
So the real SNR shows the D4 (42.3) being superior compared to the D800 (38.4). Again, when normalised to a 8Mp print, the D800 somehow “catches up”:
Unfair isn’t it? Well, only for smaller prints. Using the same formula to compute the SNR in a 16Mp print, the D4 drops to its real measured SNR of 42.3 while D800 SNR drops to 41.92. So now the D800 is inferior to the D4! How about for a 36Mp print? The D4 drops to 38.77 and the D800 drops to its real measured SNR of 38.4. The 16Mp D4 upsized to a whooping 36Mp print BEATS the D800 in its own game!!!
In the comparison above between two full frame cameras we see that even if the total amount of light, which is proportional to the sensor size, does not change, variations in SNR can occur if resampling is added into the equation. Clearly, total light and resampling are unrelated. Just because one sensor has better noise performance at a given print size does not imply that it has better light gathering capacity. If 8Mp was the only print size we could make, one would think that the D800 is every bit as good as the D4. This is clearly not the case at larger print sizes where the D4 outshines the D800. The same argument can be said for comparisons between sensors of different sizes. Sensor performance should not be judged based on arbitrary print sizes. Sensor performance must be taken at the sensor level.
Think about it: every time you print smaller than 36Mp, you are WASTING your D800. Who consistently prints larger than 16Mp or even 12Mp? As you can see, the superior 16Mp sensor makes a lot more sense. The D800 is a waste of space, time, and money.
In essence, a 16Mp sensor, be it full frame or crop can beat the 36Mp D800 if it has high enough SNR. The crop sensor need not match the superior D4 sensor. A 16Mp crop sensor with the same SNR performance as the 7-year old Nikon D700 will beat the D800 at print sizes of 16Mp and higher.
Let’s summarise what we have covered so far:
0. DXOMark data needs to be analysed. Better SNR performance in normalised data does NOT imply better light gathering capacity of full frame sensors but merely a consequence of larger image size in terms of megapixels.
1. DXOMark normalises their data because they are in the business of ranking cameras.
2. Normalisation to a small print size unintentionally favours sensors with more megapixels.
3. More megapixel does not necessarily lead to superior SNR when downsampled.
4. At larger prints (16Mp and higher), the weakness of the 36Mp D800 sensor begins to show.
5. A good quality crop sensor camera with lesser megapixels can beat a full frame camera with insane megapixels.
Do you believe me now?