How to select the perfect image format

Allow me, for a moment, to spend some time appreciating the venerable, amazing, JPEG. The twenty-five-year-old bag-of-algorithms that is ISO 10918 aka the JPEG standard has stood the test of time.

 

So, how does it work? And what is it good for?

 

JPEG compression carves an image up into 8×8-pixel-blocks, and then does something a little bit crazy to them, with a whole lot of math. It converts each block’s pixels—lists of R, G, and B values, mapped to sequential points in space—into a list of coefficients, used in equations which describe the block in terms of waves of energy. In technical terms, the JPEG compression algorithm translates an image’s information from the spatial domain to the frequency domain.

 

Practically, this means that JPEG is very good at compressing continuous-tone images: images with a tremendous number of colors, and smooth transitions between them. In other words, JPEG assumes that your image is going to look more or less like a photograph.

Conversely, the JPEG format is terrible at compressing images with crisp edges or high-energy, noisy textures—it’ll put rings around sharp edges and blur out fine detail.

In the early days of the web, if an image wasn’t a JPEG, it was a GIF.

 

The GIF format uses the LZW compression algorithm, which is far simpler than JPEG’s mathemagic. Essentially, the LZW algorithm scans over your image data and generates very short codes for the parts of it that repeat; LZW shortens repetition. Which means that the GIF format is good at compressing images that have large chunks of identical or repetitive data. Images that only have a few colors, in broad swaths, with sharp transitions between them can be stored efficiently and losslessly as GIFs.

 

And even though it’s a simple format, GIF sports a pair of fancy features: transparency and animation.

 

But… GIF is terrible at compressing things that have even a moderate number of colors; heck, the format has a baked-in, hard, 256-color limit. Converting an image with more than that number of colors into a GIF results in lossy posterization, which looks awful.

In short, GIF and JPEG have opposite and complementary strengths. They made a killer team in the web’s early days.

 

But unresolved patent questions surrounding the LZW algorithm inspired a few very smart individuals to take a second crack at designing a lossless image format for the web.

PNG excels with the same sorts of images that GIF does, and brings a few added benefits:

 

• No 256 color limit
• Alpha channel transparency (so a pixel can be partially-transparent, and not simply all-transparent or fully-opaque)
• In all but a few edge cases, superior compression

 

This image from Wikipedia shows off PNG’s ability to compress a full-color image with semi-transparent pixels pretty dang well.

How does PNG beat GIF, when it comes to compression? By adding some layers to its compression stack…

First, the PNG algorithm tries to reduce the amount of data it will need to store by using the pixels that it already knows about to predict the ones that it doesn’t. The format features five different prediction strategies, but basically, the PNG assumes that pixels next to each other will be similar. If this assumption proves true, PNG saves data by only storing the difference between its prediction and the actual value; small numbers take up less space than large ones.

Second, PNG cuts out repetition by allowing the image to refer to previous, identical pixel sequences (instead of storing the same data twice) using an algorithm called LZ77. If you squint, PNG’s LZ77 and GIF’s LZW are achieving the same ends—cutting out repetition—but get there via means which are distinct enough to evade patent lawyers. Everybody wins!

And then, finally, having done all of that, PNG uses a process called “Huffman coding” to boil the remaining values down even further by generating the smallest-possible codes for the most common values (incidentally, the JPEG format uses Huffman coding as a last-step as well).

Combining all three of these (lossless) techniques provides huge benefits over GIF’s single strategy. And sophisticated tools can provide even greater compression ratios by altering the original image data lossily before it’s run through this gauntlet, in order to make it more compression-friendly.

Stepping back, all that you need to know is this: PNG will perform worse than JPEG when it comes to photographs, and better than GIF almost always. So, use it for images with crisp edges and broad swaths of solid color or precisely repeated patterns.

The next generation

As of this writing, those three formats—JPEG, GIF, and PNG—are the only image formats with universal support. Which is to say, they’re the only formats that developers can actually use. But new, cutting edge formats are already here—and they’re spectacular.

WEBP

WebP is an offshoot of Google’s WebM video format; the core of it’s compression strategy is predictive, which is to say, it takes the simple predictive strategy used by the PNG format to the next level. WebP uses one of up to sixteen different prediction strategies for every (variable-sized) block in the image, and can optionally either losslessly or lossily compress the residual difference between the predicted and actual values. The format’s relative complexity provides it with a lot of flexibility; it’s good for a wide variety of imagery (both graphic, if you choose its lossless settings, and photographic, if you go lossy), with (generally) better compression than either PNG or JPEG.

But, it’s Google’s format and it is only supported in Chrome at the moment.

JPEG-XR

Microsoft’s next-gen-format-of-choice, JPEG-XR layers a bunch of new techniques on top of the basic mechanics of JPEG compression, enabling:

• Lossless compression
• More efficient lossy compression
• Alpha-channel semi-transparency

Like WebP, JPEG-XR is a lot more complex, performant, and less-well-supported than its predecessors. Right now, the format is only supported in Internet Explorer and Edge.

<source type="image/webp" srcset="sunset.wepb" />

<source type="image/vnd.ms-photo" srcset="sunset.jxr" />

<img src="sunset.jpg" alt="A colorful sunset" />

Fortunately, there’s another way forward. Front-end engineers can shift this complexity to the back-end, employing smart servers that can send different users different resources from the same URL.

 

Using a service like Cloudinary, developers can deploy dynamic, adaptively-compressed images by adding a few simple characters to their URLs. Stick f_auto onto a Cloudinary URL, and you get adaptability without adding any complexity. The picture markup above boils back down to: