A new open source compression algorithm

Brotli: A new compression algorithm for faster Internet

Brotli: A new compression algorithm for faster Internet
Image by : 

Opensource.com

Brotli is a new open source compression algorithm designed to enable an Internet that's faster for users.

Modern web pages can often be made up of dozens of megabytes of HTML, CSS, and JavaScript, and that's before accounting for images, videos, or other large file content, which all makes for hefty downloads. Such loads are why pages are transferred in compressed formats; they significantly reduce the time required between a website visitor requesting a web page and that page appearing fully loaded on the screen and ready for use.

While the Brotli algorithm was announced by Google in September 2015, only recently have the majority of web browsers adopted it. The HTTP servers Apache and nginx now offer Brotli compression as an option. Besides Google, other commercial vendors (such as Cloudflare and DreamHost) have begun to deploy support for Brotli as well.

Compared to the good ol' gzip file compression and decompression algorithm, Brotli's open source algorithm shrinks data by an additional 20 to 25%. Brotli accomplishes this superior feat by making use of a dictionary that includes the most common words and syllables in English, Spanish, Chinese, Hindi, Russian, and Arabic, as well as common phrases used in machine readable languages (particularly HTML and JavaScript). This gives Brotli a clear advantage in compression and decompression over other algorithms, like gzip, bzip2, LZMA, etc.

To learn more, Cloudflare's engineers offer more details on how Brotli compares to these other algorithms. They provide an insightful exploration with a fascinating viewpoint on the potential impact that Brotli's improved compression may have on real-life applications. Another valuable take on the current state of Brotli (or one as of not too long ago, at least) is by Sam Saffron, co-founder of Discourse.org. He provides another set of benchmarking measures that demonstrate Brotli's size and speed savings, as well as a supportive opinion on why Brotli is mainly available for HTTPS.

With most browsers having added support for Brotli, here at DreamHost we followed suit and began deploying Brotli on our managed WordPress service, DreamPress, starting in December 2016. Since DreamPress uses nginx for TLS termination, we were able to use the Google-built ngx module in order to successfully build and deploy the Brotli library.

While browser support for Brotli is on a swift incline, there are notable holdouts as of this date. Apple Safari still has no plans for supporting Brotli, and Microsoft has plans only for its Edge browser, leaving legacy Internet Explorer users behind in the dust (that's kicked up by speedier alternatives). Microsoft's IIS server does have a free extension to add Brotli compression on the server side.

To check if a website is offering content compressed with Brotli, first make sure that your browser asks for the content-encoding extension br and the response in the header:

(As a sign-of-our-times side note, the initially proposed extension "bro" was rejected for having too much linguistic baggage attached to how the term is used in American English; in any case, naming it br rather than "bro" saves everyone a byte.)

Testing if a site supports Brotli using the tool provided by KeyCDN.com is arguably easier.

While in reality the advantage in speed that comes from using Brotli may not always be so readily apparent to users, it is for real. At the same time, seeing a new open source compression algorithm being developed at all is a rare thing, and that makes it worth appreciating in and of itself. Since we love open source at DreamHost, and all of the stars happened to align, we went ahead and started offering this option. And it works.

Topics

About the author

Stefano Maffulli smiling at the camera
Stefano Maffulli - Stefano Maffulli is Director of Cloud and Community at DreamHost. Previously, Stefano served as the Community Manager for the OpenStack Foundation. He built his career around Free Software and open source: from pre-sales engineer and product manager at Italian GNU/Linux distribution MadeInLinux to Italian Chancellor of the Free Software Foundation Europe, where he also created the FSFE Fellowship participation program.