Which offers the best lossy compression performance: JPEG or WebP? Common knowledge suggests WebP - it is a more recent, more sophisticated format - but how well has this been tested objectively?

This study will measure the performance of both JPEG and WebP by plotting compression ratio against quality, as measured using the dsimm utility, at quality settings from lowest to highest. Both JPEG and WebP will be used in the most efficient manner practical in real-world situations: jpegs created by imagemagick with optimised huffman tables, and cwebp on -m 6.

Photographs and illustrations of various styles are tested, but no vector art. All input files are in a format that has never been lossy-compressed prior to testing. Unfortunately I cannot share the test files used - though all are published openly on the internet, I did not request distribution rights and they come from a variety of sources. Mostly DeviantArt. The photographs came from my own family photo album.

All of the illustration tests lead to the same conclusions. Firstly, that WebP outperforms JPEG at all ratios: There is not a single counterexample. At any given file size, for any of the test images, WebP will always represent a higher-quality image by the DSSIM metric. Secondly, that JPEG's highest quality settings are able to encode higher quality than WebP's highest quality settings, but at a much larger file size. For illustrations, WebP is a clearly superior format. Sometimes it wins by a wide margin, sometimes it barely outperforms JPEG - but it always outperforms.

While WebP was clearly superior on the illustrations, the difference is less apparent for photographs. WebP clearly has superiority at the very highest levels of compression, but the difference soon narrows as quality is increased - by WebP quality >95, JPEG is even able to out-perform it on quality at equal file size. It can again be seen that JPEG's highest quality settings can offer the best quality, but only with a correspondingly higher file size, as with the illustration tests.

Photographs

Illustrations

The use of the experimental auto-filter option allows for a very slight improvement at some quality settings on some images, and an equally slight impairment of performance at different quality settings on different images, after testing on three photographs and three illustrations. It appears to have little practical use. It makes no significent difference to file size at any specified quality level, as the filter is only a post-decode operation.

Attempts to evaluate -sharp_yuv and -sns were unsuccessful, as no objective quality metric accurately reflected the subjective appearance. On test illustrations at high quality settings, sharp_yuv and -sns 0 clearly improved the visual quality of the image, but also resulted in little change to ssim, lsim, dssim and psnr - often worsening these metrics. Based on subjective viewing alone, I would suggest that -sharp_yuv should be enabled whenever encoding illustrations or images containing edges of sharp color contrast, and -sns 0 slightly improves performance.