A Great Resource To Use Before Purchasing A Camera Body Or Lens

A Great Resource To Use Before Purchasing A Camera Body Or Lens

While researching and deciding on what camera or lens to buy next, there can be a lot of banter, back and forth, and noise on opinions on what camera or lens is right for you. It is possible that some websites, influencers, or average Joe’s can hold slanted biases that may play a role in your purchasing decision, and we don’t want that. So what if I told you that there was a more objective resource to help aid your purchasing decisions? Well, I have a site to share with you: DxOMark.

Purchasing a new camera or lens may not only be an exciting process, but also a crucial one. With the selection out there, there are more options than ever before which can be quite overwhelming. I’m going to assume you may also want to make sure that you spend your hard earned dough in the right places to avoid a blunder buy or having to a cut a loss. Luckily there is a source out there that is objective, accurate, and up-to-date that can play a crucial role in purchasing decisions.

DxOMark Testing Lab (Source: dxomark.com)

As I stated earlier in my introduction, DxOMark can be a useful resource that uses more objective data to render results on what lens or camera performs better than the other or among all products. The site uses lab testing and analyzation to give you results on how cameras or lens perform on a wide-array of important features, including: sharpness, noise, vignetting, chromatic abberration, ISO performance, and several more nuggets of information. I am not being coined to say this, DxoMark is a resource use every time when I’m looking to decide on one lens or opposed to the other. 

While it may be a confusing site to visit upon first glance, Tony Northrup put together a great video for everyone that breaks down how to browse through DxOMark, how it works, and what tests are the most useful in making an informed decision on your next lens or body purchase.

​A DxOMark Demonstration

Now that I've gone through what makes DxoMark a great resource and tool. I'll go through a mock example of DxOMark can help you when comparing lenses. For demonstration purposes, lets say I am a portrait photographer using a Nikon D750, I'm looking to add a 50-58mm prime lens to my collection.

I've narrowed my search down to a couple of lenses, the Nikon 58mm f/1.4 ($1,596) & the Sigma Art 50mm f/1.4 ($949). Let's run a few tests to compare key factors to help aid my purchasing decision. 

Nikon Nikkor 58mm f/1.4 vs Sigma Art 50mm f/1.4

Lens' First Impressions

Now before I completely dive in, I would like to also add that DxOMark performs tests for each combination of lenses on every camera body possible. While lens tests may vary from camera sensor to sensor, your results will be catered to what you're looking for. So for this I'll be selecting a Nikon D750 for all tests. After selecting both the camera body and the specific lens, an overall first impression will be displayed with a few scores. Lets compare the overall first impressions:

Nikon 58mm f/1.4

vs. Sigma Art 50mm f/1.4

Right off the bat you can see an overall score shown, in Northrup's video he explains that this piece of information isn't totally useful since it is an overall score that DxO decides opposed to what is most important you. So we need to rather dissect individual tests in order to find out what is most important to use, which is portraits in this case. So lets look into more importantly the 'Lens Metric Scores' shown:

  1. Sharpness - We can all agree sharpness is vital to selecting the right lens. Here we can see right away that the Sigma is better in that department. DxOMark uses a measure of perceptual mega pixels. To put it simply, the Nikon 58mm can only effectively retain 18 megapixels of sharpness for the 24.3 megapixel camera, opposed to the Sigma 58mm that retains most of the sharpness with a score of 24 P-Mpix. The Sigma is a sharp lens coupled with a sharp sensor in this case. Winner: Sigma 50mm
  2. Transmission - T-stops which are very similar to f-stops, are actually the measurement of the amount of light that reaches the sensor. Opposed to f-stops that measure the amount light the reaches the front element. In this category, both lenses perform well with the Nikon 58mm having the slight edge in this category. This especially important for photographers who use a wide-open aperture. Winner: Nikon 58mm
  3. Distortion - This is simply the distortion the lens gives out. This can be fixed in post production so it is a little to no factor. Both give out for very minimal distortion with the Sigma winning with a slight edge. Winner: Sigma 50mm
  4. Vignetting - Just like distortion this also can fixed in post. To explain the data, both come in around -1.2 EV, which means that the image's exposure on the edge will be -1.2 stop(s) below the center of the image. Both lenses check out similarly in this category. Winner: Draw
  5. Chromatic Aberration - Measured in micrometers. The Nikon and Sigma have 2 and 9 µm respectively. Which means that the Sigma has more chromatic aberration than the Nikon. Winner: Nikon 58mm

For overall first impressions, the Sigma has the edge considering that the Sigma beats out the Nikon by 25% in the sharpness category which is significant. In the other data categories they both come out comparable. 

DxO Score Map & Field

Score Map

The score map is a basic representation of sharpness of the lens at certain focal lengths and f-stops. Since these are both prime lenses, we will focus on the sharpness of the perceptual megapixels at certain f-stops. The spectrum is set from red (not sharp) to green (sharpest). As you can see in the chart above, the sharpness quality for the Nikon 58mm is awful at wide-open f-stops (f/1.4-f/2). In this case the Sigma Art 50mm blows the Nikon lens out of the water showing strong sharpness of perceptual megapixels all the way up to f/16. Winner: Sigma 50mm

Score Field

Now an even more interesting depiction on sharpness is represented through the DxOMark field map, which shows you how sharp the image will be at the center and edges of your image. Same as the score map, the spectrum is set from red (not sharp) to green (sharpest). Let's take a look:

Nikon Nikkor 58mm @ f/1.4 vs. Sigma Art 50mm @ f/1.4 (Both prime lenses)

This data above is rendered with both prime lenses shot at f/1.4 (wide-open). It is almost shocking to see how the performance of the cheaper lens blows the $1,600 lens out of the water. The Nikon lens has virtually no sharpness at wide-open apertures, while the Sigma Art performs almost perfectly tack sharp at f/1.4. Let's take a look at both lenses at f/2:

Nikon Nikkor 58mm @ f/2.0 vs. Sigma Art 50mm @ f/2.0 (Both prime lenses)

The Sigma 50mm is perfectly green which represents a perfectly tack sharp image at f/2, while the Nikon 58mm still lags behind substantiality with most of the image being unsharp at f/2. In fact, the Nikon lens doesn't have respectable performance until it reaches f/5.6 shown below:


Nikon Nikkor 58mm @ f/5.6

With that being said, the Sigma 50mm @ f/1.4 still outperforms the sharpness of the Nikon 58mm @ f/5.6 which is very significant. 

Conclusion

Winner: Sigma Art 50mm ($949)

After reviewing a few of the features it is quite clear the Sigma 50mm outshines the Nikon 58mm, especially for portrait photographers who like to shoot wide-open. While both lenses are comparable when it comes to transmission, distortion, vignetting, and chromatic aberration, the Sigma Art 50mm overtakes the Nikon 58mm easily with its sharpness performance. It's also worth mentioning that the Sigma Art is over $600 cheaper. 

This is a perfect example of how researching can save you potentially hundreds of dollars when splitting hairs between bodies and lenses. It is also a great idea to try out the lenses for yourself by renting or borrowing lenses from a friend. If that isn't an option this is a great aid when putting in the time to research when making a decision on a lens. Again, this is simply an aid to use and is just a piece of the puzzle when deciding on a lens or body to purchase, do your due diligence. 

And for the those who may claim that DxO is full of it, I'll side with Northrup on this one and say, "Prove it."

Log in or register to post comments

10 Comments

Dallas Dahms's picture

Northrup and DXO advising me on what works for me? Yeah, right... I'm a photographer, not a sheep.

I used to love trolling through the dxomark lens results. Until I figured out they were useless. They measure all sorts of things, but do not provide the data that matters most to me: performance at different focus differences and apertures. Sure, we get all sorts of information on performance, but they do not share how they arrived at it. Are these at minimum focus distance? Probably. Unless you are shopping for a macro, not really important. How about at 6-10', where I do 95% of my portrait shots? No clue. Or at infinity, where all of my landscape work is done? Unlikely. Those benches are highly expensive. Roger Cicala has one at Lens Rentals, and his data is far more useful.

Moreover, it is highly suspect that they actually test every combo stated. The amount of time taken to test a lens even on Imatest is outrageously long. They would also need to own every single lens on their list. It looks like there is some extrapolation.

Then there is the whole thing about black box methodology. Do you trust results without transparent methodology that is neither repeatable and reproducible? It sure doesn't pass academic standards.

End of the day, DXOmark is just another tool. It isn't the best, but neither is It the worst. Look at all the tests out there, including those done with Imatest software and Lensrentals.com. But whatever you do, do not think dxomark is the definitive review site for lens performance. Sensor performmance is another matter.

ALEXANDER TARDIF's picture

I've also wondered about their ability to test all of the combinations listed for reasons you've mentioned. One other variable that I'm not sure about in terms of accuracy/reliability impact on their published results is the infamous sample variation. It's a real phenomenon that may, at least in theory, have significant impact on the results of a given lens.

That said... while I appreciate having a free tool like DXO, it's quite irrelevant to my purchasing decisions. No matter the review (unless a body/lens has some truly glaring deficiencies documented by DXO or someone else) when it comes to "splitting hairs" there's no substitute to hands-on experience.

Justin Haugen's picture

What's special about that Nikon 58 1.4G is not measured in sharpness.

I say RENT it, and see for yourself. What I may appreciate can differ from others. It's like buying a car based on spec sheets and never take a test drive before the purchase.

DXO's ratings should be taken with a grain of salt, but not totally disregarded. Their ratings do nothing to test the non-quantifiable features of a lens ie: its character, bokeh, handling or 'feel' of the lens etc.. The Nikon 58 F1.4G is a great example of why you should not base your purchase on their "ratings."

The 58 F1.4G was not designed to do well in test scores it was designed similarly to old Zeiss or Leica lenses that give a more organic or aesthetically pleasing look, (which some find debatable.) Nikon specifically said it was designed to create a “three-dimensionally high-fidelity” look. It makes your subject almost "pop" from the image.

"In the case of the Nikon 58mm f/1.4G lens, strong field curvature, Nano + Super Integrated Coating, distortion, moderate vignetting and optimized optical design collectively contribute to rendering of images with perceived depth, beautiful colors and superb bokeh." .... A quote from NASIM MANSUROV in his review of the lens.

You can find his lens review and many other great reviews here: https://photographylife.com/reviews/nikon-58mm-f1-4g

Having used the lens before for stills and film I can say its not for everyone, it's a specialty lens for sure and if you want a clinically sharp and great general-purpose lens definitely buy the Sigma, especially with the price difference.

The Nikon is a very specialized lens targeted at a specific group of photographers and videographers with a similar feel to a classic Zeiss/Leica with modern electronics. It produces video and stills in a way that can't be measured by a DXO test and if you were to only base your purchase/rental on their finite numbers you could be missing out on a lot of great lenses!

The problem with DXO is the representation of their measurments. Five numbers and that's it. The majority of readers will not click the "+Measurmentts"-button which actually gives you the information everyone should first be looking at. Let me give you an example. According to the first page of DXO measurments, the Sony FE 24-70 f4 has a distortion of 1%. That's true for the 35mm sweet spot but still misleading for the lens as a whole:
http://www.photozone.de/sonyalphaff/867-zeiss2470f4oss?start=1
DXO actually tells you that. It's just that no one seems to be bothering. Probably because if you want to do that, you might as well check out more in-depth reviews... Imo the headline should be more like "An Average But Slightly Misleading Resource To Use Before Purchasing A Camera Body Or Lens". Because that's what it really is.

Only DXO knows what they call "Perceptual Mpixels", they do not show the MTF results, they do not write how many lens of a kind they tested.
I often compare to Lens Rentals https://www.lensrentals.com/blog/category/technical-discussion/lensoptics/
who gives the MTF results and variance.

Ty Mattheu's picture

I used to find them far more useful when I shot Canon. Alas, they do not have the technology to test my brand du jour, Fuji. So I DXO no more.....

Well, that was an interesting video. When I was researching my first DSLR to buy, I had a list of features of my two film cameras: full frame, 6 FPS with their respective motor drives. I thought the best I could do was with an APS camera. The affordable match in 2013 was the Canon 5D Mk III. Sure, I would love the 1Dx; but that price was out of my league as a hobbyist.
The bit about the ISO cheating makes me want to do a "shoot out" between film and digital. For that, I would choose Kodak Ektar 100, Adox Silvermax 100 using my two film cameras against the 5D III. But there is the lens variation: Canon FD 28mm 2.8 and FD 50mm 1.8 vs the EF 28mm f1.8 and the EF 50mm f1.4. Apparently there is no EF 28mm 2.8 or EF 50mm 1.8. But lens design has changed significantly from 1980 to 2016.