Recently I got a friendly comment about the images of a camera review I did. It was all about how the results should be presented. Straight out of the camera, or edited? I want to have a closer look at this question and what could be the answer.
I have reviewed a lot of different cameras and lenses. Sometimes even a printer, tripods, flashlights, and filters. Reviewing cameras is the most fun. Shooting with a new camera is great. Diving into the possibilities of different cameras makes me aware of the pros and cons. But most of all, I have learned to look at cameras from an objective point of view.
I like to test a camera in real life situations, just like I would use my own camera. Here I am shooting with the Sony A9 in a mountain stream.
Throughout the years I reviewed cameras made by Canon, Nikon, Sony, Fujifilm, Panasonic, Olympus, Hasselblad, Leica, and GoPro. I have seen how these camera have developed over the years and how they compare. Judging functions, menu structures, button layout, and ergonomics can be considered personal and subjective. But when it comes down to image quality, that’s quite a different story.
A selection of the cameras I have reviewed in the past.
In my reviews I always try to judge a camera from a user perspective. Not only regarding the use of a camera, but also the image results. One thing I try to avoid is laboratory tests and measurements, presented in nice looking graphics. First of all, I don’t have the tools for such measurements, But I also believe these tests aren’t representative of real-world use.
How Should a Review Present the Results
I started thinking about the way images have to be presented, when one reader asked me if the images I showed were straight out of the camera or not. Although I never thought about this, it’s a good question. The images I present in my reviews are shot in raw file format, and processed in Lightroom Classic to my own liking.
Shooting an evening portrait session with the Leica SL (type 901)
I shoot in raw file format. I process my images to my own liking before I present the image. Any new camera should allow me to process my images in this way. For me, it’s the only way to give an honest opinion about the results. Does the result live up to my expectations, or not? After some careful considerations, I think it is the only honest way of looking at the results.
This way will give me an insight into what the camera is capable of, and how far I can push the limits in post. An unedited image doesn’t tell much. For instance, if I exposure for the highlights the shadows can become very dark. If I would present the unedited result, the image doesn’t tell much. Only if I lift the shadows in post, the capabilities of the camera become visible.
I don't believe the straight out of the camera is representative of what the camera is capable of, but the processed image does. The before-after example above is shot with the Sony a1
When Straight Out of the Camera Images Are Needed
There are situations when a straight out of the camera image is needed. When comparing noise levels from high ISO settings, or exposure latitude is tested, the image should not be processed.
Noise levels and exposure latitude are just two examples of when the image should not be altered in any way. After all, that could change the outcome. I still shoot in raw for these tests, but I never change anything in Lightroom Classic.
Fujifilm is well known for its film simulations, just like on the GFX100 I used at Lofoten, Norway. These film simulations can only be reviewed when shooting in jpeg file format.
I find these kinds of tests to be dangerously close to the laboratory tests and measurements I don’t want to do. Nevertheless, I feel the ISO levels and exposure latitude to be essential. If it concerns lenses, vignetting, chromatic aberrations, lens flares, bokeh, and perhaps even field curvature will need unedited images.
Are You a JPEG Photographer?
But wait. The question of that reader made me think a bit more about the use of processed images in a review. After all, there are photographers who shoot jpeg straight out of the camera. Presenting images that are shot in raw file format and processed in Lightroom Classic aren’t very useful for that kind of photographer.
Shooting dogs in action with the Canon EOS 1Dx mark III.
Besides that, some cameras have a nice jpeg engine and even film simulations or picture profiles that produce great results if you’re into that kind of use. To give a good impression of what the camera is capable of, I think it is necessary to present both processed images as well as straight out of the camera jpeg images.
Things a Good Camera Review Should Address
I don’t want to make reviews that dive deep into the technical details, laboratory tests, and measurements. I think a good review should tell how a camera in real-life situations would perform. For that I believe the following things should be addressed in a camera review.
- A shortlist of specs, only the most important or striking ones
- Looks, dimensions, button layout, and ergonomics
- ISO performance and exposure latitude with unedited images
- Autofocus functionality and performance
- Image quality with processed images
- Image quality with straight out of the camera jpeg
- A look at special or unique functions
- A look at film functions and possibilities
- A conclusion with a list of things to like, and thinks to dislike
- A gallery
Although I have added the film functions and possibilities also, I don’t have enough experience with filming. Therefore, my reviews will address these things only briefly. Often it will be a summary of what the camera is capable of by looking at the specs of the manufacturer.
After a night shooting a moon eclipse sequence the Nikon Z7 I tested kept working perfect even under these cold situation. As did my trusty Canon EOS 5D mark IV.
What Do You Think?
I believe these ten points are the most important chapters in a camera review that is made from a user perspective. The review should give a good idea of what a potential buyer could expect.
Perhaps you find there is something lacking in the ten points I mentioned. Maybe you feel there is another important thing that should be addressed in a review or something that can be left out.
Regarding my original question about the presentation of images, I would like to know what you think about processed images in a review or not. Please leave a comment below with your opinion. I’m looking forward to your response.
I think its very helpful when reviews provide before and after edited images, like you did here with the slider to show how the image looked before and after your edit. I think this is the best of both worlds, as you can see examples of images straight out of camera, and after a little (or sometimes a lot) of post processing
Concern about SOOC is overrated............Ansel Adams taught us that.........
Knowing the starting point the sensor will give you IS important though. So is knowing where you can push it to.
That's true but the comment was in reference to the oversimplification of the term straight out of the camera. There's no doubt that today's software has made us lazy. It is better to understand and use the camera to achieve an image rather than trying to repair the mistakes with voodoo magic.
Ansel was selling photographs, not the equipment.
Sorry, Brian, respectfully, that doesn't make any sense but maybe I wasn't either.........“You don't take a photograph, you make it.”
― Ansel Adams
He did his magic in the darkroom.
When I was buying my PhaseOne XF they were very plain about the RAW photos coming out of the camera often were not pretty and in post C1 made them brilliant. Brilliant?... Maybe or maybe not. But the point does remain the same, in that the RAW photos are often composed knowing the data is in the file and you will be able to get the result you want, or hope you will, later. Terry's remark is on point too, in that taking the best photo you can will always give the best chance to get something really great in the end.
I agree with Matt Edwards
The before/after visual is helpful and provides a good reference of what can be recovered, as well as hint towards white balance tendencies and color casts.
I would like for camera reviews to detail how the meter (evaluative/matrix) performs when not influenced manually.
Were camera/lens/film ever reviewed based on a contact sheet, using a loupe?
The reviews that mostly concern themselves with IQ shadow boost, shooting JPEGs, and PP looks are not what I'm interested in!
Tell me what exactly you find comforting and or challenging while shooting with the camera and or specific lens.
You're photographing in changing light either using MF or AF and using auto ISO can you nail the shot? what part of the process is slowing you down or preventing you from image capture?
Does the EVF black out to long? Are the dials, menus or buttons not accessible or intuitive. When the meter is pushed to the right and you pull up the images in RawDigger did the camera show you the exposure to the wall correctly.? or did you have more or less room. when you read the manufactures claims ie battery life (and not sit there and fire off 440 frames or turn on the camera and video a blank wall for 2:13 but really using it chipping and with in the case of video an external monitor were the manufacturers claims met or did it fall short.
Can you explain what it was like to hold the camera for 8 hours a day for three days making all kinds of images how the camera helped or hindered your ability to grab images. What lenses were spot on and which ones were a challenge.
Most importantly where is or what is the best ISO setting to use that allows for the cleanest files either at ISO, PUSHED or PULLED. Can you shoot the camera in the pouring rain and for how long before it dies, in the hot sun or extreme cold if you have the chance if not admit in the review you couldn't test it.
After that then take some files and push, pull stomp on them!
Regards
Well...the title of this article answers your question. "Should images FROM camera reviews...." You have to have a baseline to start with. That baseline is the out of camera images BEFORE any editing. Editing can be done in an infinite number of ways. How the image looks coming out of the camera is a one-time image. No manipulation, no editing, no nothing. Just straight out, first shot image. You can determine the image quality versus other camera without any outside bias (editing).
Jpeg images are RAW images processed according ro the algorithms of the camera's manufacterer. Either the photographer is processing them in post or the manufacterer is processing them in the camera. There are no unprocessed jpegs. This SOOC issue is for people who don't understand that RAW images need to be processed, one way or the other. Somehow the manufacterer is not cheating, but the photographer is?
Thank you sir for further clarifying.
I think it is best to set at JPEG, then post the image along with other basic settings common to most cameras.
I think both is ideal, but the editet files usually are preferred. In a review im mostly interested in the „maximum achievable potential“, not how the files come out of the camera.
Im mostly a movie shooter, so for movies its obvious that for instance showing the Log-image isnt much help. In movies there is the option of showing rec709 …for images there doesnt seem to be such a standard, so i prefer the final results.
Out of the cam, no editing. I want to see if the camera got the exposure, I want to see the skin tones, the white balance, the low light capability.
What I do already now is that since 2012 sensors are top and in post so many things can be done that such pictures have no meaning. I can pimp a Nikon D3200 (DX) image that it looks "better" than the out of cam D800E (FX) image.
I agree 100%
Software is too good now to trust any image as being legit.
I personally prefer to see edited images as I want to the maximum capability of the gear and since I always edit, it's the norm for me. I also think the type of review matters too.if it's a more technical review then seeing both edited and unedited images is better, but if more real world performance based then edited only is preferable.
Yep! End result. I'll make an educated guess and say that the majority here shoot in RAW and process their images for best results. That's what I want to see in a review; what will the shot look like at the end of the process.
The problem with this is "best results" as it pertains to editing is almost entirely subjective. How you edit an image, versus how I edit an image is likely to have completely different results.
The image straight from camera is objective. 3 photographers standing in the same field at the same spot, framing the same sunset with the same gear and settings, should produce the same image out of the camera. The final edited result may be entirely different between the 3.
I rather see both pre-edit and post-edit in a review, it gives you all of the information and the shortcomings of the camera are not hidden behind post processing.
I can't argue your point...completely valid. Like others have mentioned, I like to see what can be done with the RAW image, how far can it go before it breaks down.
Truth be told, I'm pretty much set on what I would buy if my long lost uncle left me a couple of tube socks full of 100 dollar bills. :-) But, being a gizmo junkie, I like to see what other brand's end results are.
I suppose one could make a case for demonstrating 'clean' edits, rather than artistic edits. i.e. set the white balance to neutral and black/white/mid-tone points correctly. Leave out masking, colour grading, local noise reduction and sharpening etc.
As a review, one could possibly just hit auto in LR which will take care of much of this automatically.
TBH, this is one of the areas that DP Review really shined. Their lab test were completely objective and compared jpeg to raw in different lighting.
If you need to edit your pictures before showing them in a review, you need to include the pos-prosed picture as well.
This is becoming even more necessary with all the AI software they bring out by the minute.
I understand that you would like to show the best work you can do, but if we look at your picture of the field with yellow flowers, I can't help thinking "nice sky replacing" done there.
On the other hand, if you had only shown the unedited picture, I would have thought you was an amature.
Showing both pictures, remove both those negative views from the equation, but also give me, the viewer a much better understanding of, what that camera is capable off.
Best to edit the images, as everyone will edit their images, and pushing an image helps to determine the latitude of the raw image. For example, there have been cameras where pushing a shadow 2 to 3 stops and you end up tons of banding, while other cameras can have shadows pushed 5 stops with decent results and no banding.
Beyond that, the biggest issue in reviews are sites that refuse to post full res images. I remember when the Nikon D800 came out and when looking up reviews and some of the sites would do a review, talk about detail but scale the images down to under 2 megapixels.
It is just idiotic, they could literally use a $300 DSLR and claim it is a D800 and no one would know if they are scaling images down so much. Aside from that another important thing that should be included, are sample raw files. Those are the most helpful parts of a review, as you can actually put the raw file through your work flow and see how well it holds up. As well as experience the flexibility of the files.
Personally I'd expect a complete camera review to both show JPEG quality and editing potential of RAWs. I don't want to shoot RAW and edit every photo I take.
yes only Post unediting RAW files it will be great! :P you cant post Jpegs from the camera Because then are edited.. :P
Straight out of the camera is the only way the reader can get a true idea of what the camera or lens will do. Software has become so good that it is possible to take a horrible picture and adjust colors, exposure, eliminate vignetting, remove sun spots, remove dirt spots, eliminate out of focus spots, even add subjects and add clouds, etc.
When I see a review with all doctored images, I wonder what they are hiding. I don't trust reviews that don't include out of camera images.
I want to thank everyone for responding. It is great to read everyones opinion about this.
Images for camera reviews are shot by photographers and tell you more about them than the camera. People who think they can discern important information about the camera from looking at such images are fooling themselves.
Regarding noise, i believe things are changing with AI based denoising. This is no longer linear and it might be that 1 stop (2x iso) of input image not necessarily translates to 1 stop of noise in output. This makes future evaluation trickier. Of course higher iso images have lower dynamic range, but this is a different issue.
Reading comments above saying that software is so good that you cant distinguish cameras, means that there is no value reviewing equipment wo software since final quality is the only thing which really matters.
I have been frustrated for so long by SOOC reviews. I don’t care what they look like SOOC! Only what is possible matters
Usually both are needed, since there is no standard for post processing. The top tier of reviews, is providing users sample raw files, as that allows them to simply use the raw file in their workflow to see how well it holds up. It also allows for pixel peeping and testing the latitude of the files.
Another reason to avoid showing only what is possible, is that there is no limit to how much editing someone can do. Someone could even go as far as taking an image where the image is noisy, and missed focus, but run it through an AI model to recreate the image, or they could manually fix the issues in the image combining multiple methods. In either case, the representation would not be representative
The middle tier of reviews are straight out of camera, as well as an edited image (both full resolution).
Bottom tier is when you see tiny, ~1 megapixel sample images, since all you really have to go on is the text or the review as as the tiny images make it effectively impossible to tell much about a camera from the images, and in that case, they might as well use stock photos or post photos from other cameras approximating what they would photograph using the camera, as both will be equally helpful to readers.