Skip to content
Documentation

Building cacheable applications: cache layers and cache busting

Every request to your site passes through several caches before it reaches your application, and each one keeps its own copy with its own lifetime. Most reports of "the cache is broken" are one of those layers still holding a version that nothing told it to release, and very often it is the one layer you cannot purge: the visitor's browser.

Two habits solve nearly all of it. Read the response headers before deciding which layer is at fault, and make sure every asset URL changes whenever the file behind it changes.

The layers in front of your application

From the visitor inwards:

  1. The browser cache, plus a service worker if your front end registers one. You cannot purge this and nothing you deploy reaches it. A visitor keeps a file until its max-age expires or the URL changes.
  2. A CDN, if your environment has one in front of it. This can be a CDN offering from Dropsolid or a third party CDN you point at your site yourself. It has its own cache keys, its own lifetimes and its own purge mechanism, independent of everything below it. Not every environment has one. Ask your Dropsolid contact person if you are unsure.
  3. Varnish on your environment. It follows the cache-control headers your application sends, and caches for 120 seconds when you send none. See Controlling if and how long Varnish caches your pages.
  4. Your application's own caches. For Drupal that means the page cache, dynamic page cache, render cache, CSS and JS aggregation and the Twig cache. Below that sits PHP's opcache.

These layers are independent of each other. Clearing Varnish does nothing to a CDN or to a browser. Purging a CDN does nothing to files visitors already downloaded. That is the whole reason cache busting exists: the only way to reach a browser cache is to hand it a URL it has never seen before.

Read the headers before you change anything

Ask the site what it actually returned:

curl -sSI https://www.example.com/path/to/asset.woff2

The browser devtools Network tab shows the same thing, as long as "Disable cache" is off, otherwise you are hiding the layer you are trying to diagnose.

What to look at, in order:

  • X-Cache tells you what the server side caches did. You may see several values, one per caching proxy the response passed through. On the platform the first value is Varnish, any further values come from the CDN layer. HIT means that cache answered, MISS means the request travelled on to the next layer.
  • Cache-Control on the response tells every layer after it, including the browser, what it is allowed to do: max-age, s-maxage, public or private, no-store. This is the header you control, and the one that determines the caching of the page or file in the first place.
  • Age tells you how long a shared cache has been holding the object already.
  • X-Cache says nothing about the browser. If every value is MISS and a visitor still sees the old version, the copy is in their browser or in a service worker, not on our side.

Two headers that mislead people:

  • Expires is a leftover from HTTP/1.0 and Cache-Control overrides it wherever both are present. Drupal sends a fixed date in the past on every response, Sun, 19 Nov 1978 05:00:00 GMT, so it carries no information about your site at all.
  • Last-Modified and ETag describe the version of the object, not whether a cache is serving it. A fresh Last-Modified on a stale-looking page usually means you are looking at a different response than the visitor is.

For the full reference on the standard headers, use MDN's HTTP caching guide.

The one rule of cache busting

Never change the contents of a file while keeping its URL. Change the URL instead.

Anything you serve with a long max-age, and fonts, CSS, JS and images are usually served with up to a year, is sitting in visitor browsers until the URL changes. A deploy does not reach in and take it back.

There are two ways to change the URL:

  • Fingerprint the filename, for example main.a83f19c.css. This is the more reliable option: it works with every cache, including ones that ignore or strip query strings, and it lets old and new versions coexist during a rollout.
  • Add a version query parameter, for example main.css?v=a83f19c. Easier to retrofit into an existing theme. Be aware that some caches and proxies treat query strings differently or drop them, so verify before you rely on it for a high traffic asset.

Either way, the version value must come from the build, ideally a hash of the file contents. A value that a developer has to remember to bump by hand will be forgotten. A value generated per request, such as a timestamp, disables caching completely and costs you the performance you were paying for.

Where it goes wrong per asset type

  • Icon fonts and webfonts. The @font-face rule with its version parameter usually lives in a handwritten CSS or SCSS partial. Regenerating the font to add a glyph changes the font file but not that line, so the build passes, the deploy is clean, every server cache reports a miss, and every returning visitor still sees the old glyph set. Generate that version value from the font file itself, or bump it in the same commit that regenerates the font. This is the most common cause of a bug that only a hard refresh fixes, and it is not something you can ask an end user to do.
  • Aggregated CSS and JS. Drupal appends its own cache-busting value to aggregated files, tied to a counter that changes when you rebuild caches. That covers the aggregate, not the files referenced from inside it. Fonts, sprites and background images loaded by a url() in your CSS each need their own busting.
  • Images, sprites and favicons. Same rule. Overwriting hero.jpg in place is the classic mistake. Write a new filename.
  • Generated derivatives and user uploads. If you regenerate a derivative over its existing path, add a version parameter or write it to a new path.
  • Service workers. A worker can keep serving its own precached copy long after every cache on our side is clean. Version the precache manifest and let the new worker take over active clients, otherwise you have built a cache nobody can purge.
  • The HTML itself. Fingerprinting only helps if visitors get HTML that points at the new URLs. Keep documents short lived, or invalidate them by cache tag when content changes, and let the fingerprinted assets carry the long lifetimes. See Invalidating Varnish automatically with the Dropsolid Purge module.
  • Pages that must never be shared. Personalized pages, anything behind a login, checkout and form flows: set Cache-Control on those responses deliberately rather than relying on a default. A page you assume is uncacheable will be cached for 120 seconds if your application says nothing.

When something still looks wrong

Work from the symptom:

  • Old asset, and every cache in the chain reports a miss. The copy is in the browser or a service worker. Confirm in a private window. If the private window is correct, the fix is a URL change in your build, not a purge on our side.
  • Old page, and a shared cache reports a hit. Something upstream is still holding it. Clear Varnish, purge the CDN if you have one, or wait out the remaining Age. Then fix the Cache-Control header so it does not come back.
  • Correct for you, wrong for some visitors. The response is probably varying on something: a cookie, a Vary header, or a personalized block that got cached into a shared page. See Controlling cacheability of requests with cookies.
  • Wrong only on your own machine. Check your hosts file and your local development caches before anything else.
← Previous Invalidating Varnish automatically with the Dropsolid Purge module Next → Overview