Why Isn't JPEG Dead and Is HEIF the Solution?

Why Isn't JPEG Dead and Is HEIF the Solution?

It feels like a bad dream, that just got worse. Just when you thought the nightmares were over, you awake in a cold sweat plagued by the same recurring terrors. Yes, the JPEG really is that bad so why do we keep clinging to it? Isn't it time we killed it off and moved on to something better?

The need for JPEG format was a very real one — transfer images across extremely low bandwidth networks for display. Low bandwidth applied to both the storage medium, as well as the transfer channel. That could be a hard disk drive or floppy disk, over dial-up networks, or perhaps the more favored sneaker-net! Up until that point, images had been stored as raw bitmaps (BMP) which contained each individual RGB or grayscale pixel values. This was vastly inefficient for data storage and transfer but was the cornerstone for simple algorithms. Methods to compress images were rapidly developed, such as Run Length Encoding (RLE), which identified contiguous pixels that were the same value and just stored this information. Even within this context, there was a competing trade-off between resolution, file size, and loading speed. I distinctly remember waiting tens of seconds for individual grayscale photos to load on my PC running the MS-DOS operating system. Of course, these compression systems didn't work particularly well for continuous-tone images, such as photos. An alternative was needed that worked with photos over low bandwidths. Enter the JPEG.

JPEG is an acronym for the Joint Photographic Experts Group, which was formed as a sub-group of the ISO/IEC Joint Technical Committee 1, Subcommittee 29, Working Group 1. For simplicity, it was known as ISO/IEC JTC 1/SC 29/WG 1! Given the importance of the problem the two main standards organizations (ISO and IEC) came together and released the JPEG format for digital still images in 1992. Importantly, it was royalty free.

Philosophically, the JPEG committee didn't mind if they butchered the original image, as long as it looked good. That was the key to unlocking the easy storage and transfer of digital images. More specifically, it had to look appealing to the human eye. Regardless of the input data, the RGB layers (or the single grayscale) are reduced to 8-bit data before smoothing to the point that it looks the same — or almost the same — as the original image. The primary setting is "quality" which aggressively increases the amount of compression applied at the expense of efficacy. However, the benefit is smaller files, with compression ratios of 10:1 common, along with faster write speeds.

The algorithm underpinning the file format uses a discrete cosine transformation (DCT) over 8x8 windows of pixels (or kernels). The DCT is "fitted" to the underlying image data — the accuracy of the fit determines the degree of compression, with a closer fit requiring more storage. Whilst effective, the use of a DCT has led to some significant problems in the quality of the final images. Firstly, it is fairly common to see "blocky" artifacts, particularly in heavily compressed images, which result from the underlying 8x8 kernels. Secondly, the DCT kernels are also the reason for posterization which results in pixillated transitions across smooth tonal changes (as the image below shows). Thirdly, every time you re-save a JPEG the DCT is reapplied which causes progressive degradation. One of the most common causes of this is image rotation followed by further edited. As rotation simply changes the pixel identified as the origin of the image, there is no need for re-saving and so this was the reason for the introduction of lossless rotation.

The Future?

The JPEG in its original incarnation was a 1980s solution to a 1970s problem. It met the need head-on and was a perfect fit for the subsequent explosion in digital cameras. However, it rapidly pushed up against the inherent problems noted above and so an alternative was required. The JPEG2000 standard entered service in 2000 and used the discrete wavelet transformation (DWT) to achieve much greater flexibility, but only with mild increases in compression. Ultimately it was too complex for its time, was not backward compatible, and had significantly greater processing requirements. It's a salutary lesson on how standards can shape technological development — the JPEG was never envisaged to be the de facto image format for 28 years and counting, however, that is how it has ended up.

We desperately need a new format for one reason: JPEG is appallingly bad for any kind of long term archival.

If you want a butchered image with no latitude whatsoever to be the lasting record for future generations then go ahead and continue using JPEG, however it really has no place in the photographic arsenal. Of the three primary benefits, arguably none of them exist any longer. Backward compatibility has been important for wide-ranging support in capture and display systems, however, there are good alternatives that need wide adoption. The royalty free status is important which is where we need standards organizations to again step in. The file size is far less important than it ever was; ~24 megapixel sensors strike the sweet spot for resolution and you can happily shoot all day on current memory cards. Higher resolution sensors — such as found in the Fuji GFX100 — still cause a data headache but you find these in the hands of those familiar with the data processing issues. There is perhaps some rationale in JPEG use for photographers that require fast shooting speeds, however with Canon's 1DX Mk III able to shoot at 20 fps for up to 1,000 raw files, this difference is rapidly diminishing as the technology trickles down.

Camera manufacturers are partly culpable for the situation we now find ourselves in and certainly have a part to play. It's notable that even with Adobe's push to popularize the DNF raw format, most still stick to their proprietary types although at least these are widely supported. However, JPEG remains a stalwart when there should be a far wider take-up of other formats. Part of the culprit here — besides a lack of industry focus — has been the failure of manufacturers to continue updating firmware. For example, my aged Synology DS112 NAS runs the same version of the firmware that sits on the latest Diskstation 220j: why can't camera manufacturers be as proactive? The longevity of the JPEG is such a critical issue that manufacturers should feel an implicit requirement to update older cameras.

Of course, there is an heir apparently waiting in the wings in the form of the HEIF format which was developed by the Moving Picture Experts Group (MPEG) and is also royalty free, sports better compression, and has a greater bit depth, all of which are important for mobile shooting. It's the reason Apple incorporated it into iOS in 2017 and, because of this, HEIF is now widely implemented across other operating systems and applications. Canon is the first camera manufacturer to incorporate support (into the 1DX Mk III), however, it is disappointing that other manufacturers haven't yet followed suit.

That said, it remains to be seen if HEIF is the format that stills photographers have been waiting for or if it is being exploited to the best of its abilities. Apple and Canon both appear to be focusing on file size, whilst maintaining the same or better quality. Given the use cases involved, this is in order to minimize upload times. However, what we need is far wider adoption for straight-out-of-camera (SOOC) shooters that implement lower levels of compression. The other format on the horizon, and approaching ratification, is JPEG XL which should provide this functionality, better balancing compression, quality, and encoding speed. What this highlights is how poor camera manufacturers are in pushing their firmware platforms forward: there needs to be a sea-change in how they achieve this and the current paradigm is not working for photographers. Perhaps the Zeiss ZX1 is the way forward that will allow a plug and play architecture for the deployment of a range of new functionalities. Whatever the future holds, current manufacturers are letting their users down.

Apple has never been shy to adopt new standards in the pursuit of performance gains and we desperately need camera manufacturers to likewise follow suit. I for one don't want to see an option for JPEG shooting in any future cameras.

Lead image courtesy Michael Gäbler vis Wikipedia.

Log in or register to post comments

23 Comments

No Information's picture

I appreciate the history lesson and perspective. One comment I can make on this is related to a recent project of mine. I'm not a professional, and have recently come to the conclusion that storing all of my 100,000 raw files from the past 5 years of shooting isn't exactly worth it for me. I rarely go back and edit, and most of the time I'm going back just for the memory, not the quality of the image.
I started to look up how to convert all these raw images into HEIF files for longer term storage and after a few weeks of trying, came up against a big issue: raw to heif conversions currently doesn't preserve metadata. (This is the point where I hope people comment on a solution that I missed!)
My tried and true XnConvert does offer a pretty great raw to heif conversion, but no matter how I tinkered with it, I couldn't get metadata to fully transfer, and that's a big appeal to me since I want to know what camera/lens I used for each photo.
I tried every image converter program I could find that supported heif, including installing ImageMagick and attempting to get LibHeif to work, but alas, my command line skills are pretty weak. I think the conclusion I came to after reading up on all the issues is that the licensing issues around heif are still a bit murky and that's preventing more streamlined tools like we have with jpeg. Unsatisfying answer, not sure if I completely believe it, but after weeks of testing, it was as good as I got.
Fortunately, converting from raw to a high quality jpeg with good chroma subsampling using XnConvert works perfectly, so it seems that when my storage solutions hit a wall, that'll be the route I'll go forward with.

David Cannon's picture

Great article. Thank you for it! I’d love to see a Part II that gives some image samples of JPEGs and HEIFs created from the same RAW files and shows quality and size differences.

Spy Black's picture

JPEG isn't going anywhere. Everyone knows it, everyone uses it, and the vast majority of people are happy with it. Web production will continue with it, phone pics will continue with it, etc. There's just so much momentum in JPEG that I simply don't see it stopping. For the most part in day-to-day use it's more than adequate, practical, and readily available. That's gonna be a tough format to kick.

Rhonald Rose's picture

JPG is not going anywhere...

Mike Dixon's picture

Some of us who remember PAYING for a license to use GIF's in our software might disagree with that.

Timothy Turner's picture

If you get it right in camera jpeg is all you need.

Ed Sanford's picture

The problem with that is jpeg is developing your image (color, sharpening, white balance etc). So-called getting it right in the camera is more about composition. JPEG throws away photons of light that “it feels” are unneeded. That’s why serious photographers shoot raw because after they get it right, they want to complete the image based on their visualization.

Andrew Broekhuijsen's picture

Found the Ken Rockwell alt.

Dan Marchant's picture

It will be a long time before JPEG succumbs because there is simply no need to replace it.
1. It was never created as an archival format, nor is it (or it's replacement) needed for that purpose. You should always keep the highest quality for archive (RAW).

2. You create a JPEG to "transfer images across extremely low bandwidth networks for display". It is a disposable file that can be deleted after it's sent. JPEG meets the need, so JPEG Plus formats are a solution looking for a problem.

3. We don't need higher res/better detail Internet images because the majority of these images are displayed on mobile phones, tablets and laptops which won't noticeably benefit from improved image quality.

4. For those people we need to send higher quality images to we can send full size, low compression JPEGs or TIFFs.

the dikiz's picture

HEIF won't kill JPEG,
Just like OGG failed to kill MP3.

No lossy format will ever kill JPEG.

Robert Edwardes's picture

JEPG has a huge amount of support outside the Photography community. This show up in programing and high level languages like JavaScript and Python where you can get a package from other person for working with, manipulating, and processing for ML/AI stuff. Tech can take forever to change SQL is still being used widely despite being from the 80's

Jan Holler's picture

A great article, thank you. In German there is a saying: "Das Bessere ist des Guten Feind", meaning "the better is the enemy of the good". Sadly this is not always true in the IT-world. JPEG is (still) better for the rest of the world simply because it is supported on basically every modern device with a (colour) screen. The solution to this would be to have the choice between JPEG and HEIF or simply being able to store both at the same time (which you cannot in the newest Canons).
The article does not talk about the container format's options. There is a big advantage with HEIF: In a HEIF container you can store videos and animations besides image data while in a JPEG container only image data is. And maybe the article should have mentioned TIFF: A lossless 24bit format widely used by graphical designers and print shops. A problem later occurred with patent issues which then later led to the exclusion of an important compression format: Lempel-Ziv-Welch (lzw). This was also true for another important container: GIF which can store animations but being inferior as it only got 256bit for the colour lookup table (and therefor images should be rasterized). So, yes, any new image(/video) file format should be realized without patents. HEIF is not as I understand. I think that is also preventing other manufacturers to take the risk and implement it.

Paul Scharff's picture

I don't have problems with JPEGs. Their compression algorithms work fine for their intended purposes, and I'm finding over the last year or so ACR pretty amazing at lifting shadows from them (less so at recovering highlights).

I get there's no comparison to RAW files, but I'm not sure JPEGs are a big problem in search for a solution.

Uri Rubinstein's picture

I have repeatedly experimented with RAW & JPG files out of my Fuji-X cameras, processing with ACR, & find no advantage whatsoever to shooting RAW. That compared to when I was a Nikon photographer & shot only RAW.

Timothy Turner's picture

If you don't like jpeg that's fine but don't take it away from the rest of us. I don't like red so I think the color red should be eliminated from society. That's how rediculous this argument is getting.

Uri Rubinstein's picture

Totally agree. Keep your hands off my camera, thank you.

Jon Sneyers's picture

For a visual comparison between JPEG, JPEG XL, and HEIF/HEIC (or BPG, which is basically the same thing, it's a single frame h265), here's a good comparison tool:
https://eclipseo.github.io/image-comparison-web/

I recommend comparing Large or maybe Medium to the original image.

In my (not so neutral) opinion, for high fidelity compression JPEG XL is best, while for low fidelity, BPG and AVIF (aom, rav1e, svt) are better at hiding artifacts (but they do blur details a lot).

Keith Meinhold's picture

Try sending a client a HEIF in an email. The average person does not understand or care about these formats - they only know what they can see or open. As a graphic designer I ask clients for their logo in EPS to TIFF format, I feel almost lucky if I get a a JPG instead of a GIF. I rarely receive an EPS or TIFF because the sender can not view these formats with typical office software, For example; most email software cant view these formats) but a JPG or GIF they can. Until that changes, we are stuck with these that actually work for the lowest common denominator.

Greg Edwards's picture

Agreed. I’m a graphic designer too, in a print company. Clients only tend to supply files in formats they understand and can view, so it’s very rare they’ll supply eps logos or tiff images. (If I had a penny for every time I’ve been given photos in word documents...)

My main concern is resolution. I can fix most artefacts on logos, but not resolution. So jpeg normally works just fine. If not, 9 times out of 10 find a pdf document on their website and extract a vector logo out of it.

FWIW, many years ago printers swore by cmyk tiffs for raster images which took up a lot of space and postscript for everything else.. But the world has changed nowadays. A lot of projects are now designed for print and screed so we stick to rgb Jpegs for most non-vector images and let the software convert to the intended colour space when required. And even with so much more server space, it seems to run out quicker than ever, so Jpeg along with pdfs keeps projects nice and compact.

Keith Meinhold's picture

Seems we have the same challenges and take the same approach. "If I had a penny for every time I’ve been given photos in word documents..." totally!

Michael Dougherty's picture

This article is very timely. Effective immediately, Adobe is no longer supporting Photoshop CC or RAW on Windows 7 computers. Unfortunately, I only have Windows 7 computers and now I won't be able to open my latest Nikon RAW files for the Z7II. Thankfully, I shoot RAW and fine JPEG simultaneously so I'm not in good shape. I will upgrade to WIN 10 but I also have $1,000 of dollars of other programs to deal with and there will be lots of compatibility problems.
Thank you Adobe! Go JPEG.

Spy Black's picture

Adobe is not the only solution in the world you know.

Andrew Broekhuijsen's picture

To me it makes sense to replace JPG with HEIC in places where the person creating the content shouldn't be expected to know anything about archiving (i.e. your average smartphone snapshot taker). John Smith with his iPhone just wants to take pictures and not worry about how they'll be accessed or what level of quality they'll have in 60 years. Shoot HEIC and hope the format catches on. Cool. That's essentially what Apple did with their default HEIC on iPhone photos, right? I specifically didn't want to shoot HEICs and remember needing to go in and change my settings to shoot JPGs with my iPhone instead at some point.

For photographers who are primarily shooting RAW, I'm aware of the arguments that proprietary RAW formats will eventually be unusbale, and I've considered switching to DNG for long-term storage. But I only use JPG for final presentation. If it's done in Lightroom and I'm ready to share on Facebook, my website, Flickr, Instagram, or give copies to clients, I use 1st generation JPGs. In 99% of cases, I can't tell the difference between a TIFF and a 1st gen JPG that doesn't have overly aggressive compression applied (77% quality seems to be a good rule of thumb cutoff, but filesizes are usually fine even at 100% quality).

All my portfolio pieces are backed up in 3 places as uncompressed, full-res TIFFs, and I still have the RAW files as well.

I'm aware of what makes HEIC better on paper, but HEIC fills no needs in my current workflow and storage scheme. The effort it would take to leverage the format (finding software or plugins to do the conversions, adding an extra step since LR won't directly export HEICs, dealing with headaches when customers try to open them and can't, etc.) is high. The benefit is virtually nothing.