Fastly VCL for Image Format Negotiation

Fastly is one of the few CDNs that hands you the cache as a programmable object: instead of ticking boxes in a dashboard, you write Varnish Configuration Language (VCL) that runs on every edge request. That power is exactly what format negotiation needs — the decision of whether a given client gets AVIF, WebP, or JPEG is a per-request branch, and the correctness of your cache depends on getting the cache key right by hand. This guide is part of CDN & Edge Media Delivery and walks the full VCL request lifecycle for serving format-negotiated images: how to read and normalize the Accept header in vcl_recv, how to fold a compact variant token into the hash in vcl_hash, how to drive Fastly Image Optimizer with format=auto, and how to shield your origin and purge variants with surrogate keys.

The core problem is deceptively simple to state and easy to get wrong: a single URL such as /img/hero.jpg must return different bytes to different browsers, and Fastly must cache each of those byte streams separately without ever serving the wrong one. Get the hash wrong in one direction and you fragment the cache into thousands of near-identical objects; get it wrong in the other direction and you poison the cache, handing an AVIF payload to a Safari 15 client that requested WebP.

The Fastly request lifecycle

Fastly’s VCL exposes a fixed set of subroutines, each fired at a defined point in the request’s journey through the edge. For format negotiation, four of them matter. Understanding where your code runs is the whole game — normalizing Accept in the wrong subroutine either misses the cache lookup or runs after the object is already selected.

  • vcl_recv runs the instant the request arrives, before the cache is consulted. This is where you sanitize the request: normalize the noisy Accept header into a compact token, strip query strings you do not want in the key, and decide which backend or Image Optimizer path applies. Whatever request state exists when vcl_recv returns lookup is what the cache sees.
  • vcl_hash computes the cache key. By default Fastly hashes req.url and req.http.host. Anything else you want to distinguish variants by — your normalized format token — must be explicitly added here with hash_data(). This is the single most important subroutine for negotiation correctness.
  • vcl_fetch runs after the origin (or Image Optimizer) responds but before the object is stored. Here you set the object’s TTL, decide whether it is cacheable, and can rewrite Vary so it reflects your normalized token rather than the raw Accept string.
  • vcl_deliver runs just before the response is sent to the client. This is where you attach debugging headers, surface the served format, and clean up internal headers you do not want to leak.
Fastly VCL request flow with Accept-normalization branch A flow diagram: a client request enters vcl_recv, where the Accept header is inspected and normalized to a compact token X-Fmt of avif, webp, or jpeg. That token feeds vcl_hash, which combines it with the URL and host into the cache key. On a hit the object is delivered; on a miss the request goes to the shield POP and Image Optimizer, then vcl_fetch sets TTL and Surrogate-Key, and vcl_deliver returns the response with debugging headers. Client request Accept: image/avif,… vcl_recv inspect req.http.Accept avif? → X-Fmt: avif webp? → X-Fmt: webp else → X-Fmt: jpeg token vcl_hash hash_data(req.url) hash_data(host) hash_data(X-Fmt) cache hit? hit miss Shield POP + Image Optimizer ?format=auto reads X-Fmt / Accept origin fetched once, shared across POPs vcl_fetch set beresp.ttl Surrogate-Key = asset id vcl_deliver attach Fastly-IO-Info / X-Cache / X-Served-Fmt, strip internal headers, send to client cached object

Why raw Accept cannot be the cache key

The instinct is to reach for Vary: Accept — the same header covered in the Cache-Control headers guide — and let the cache split variants automatically. On Fastly this technically works, but it is a trap. Chrome sends an Accept header for images that looks like image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8, and that string changes across Chrome versions, across Firefox, across Safari, and across every crawler and preview bot on the internet. If your cache varies on the full string, each distinct Accept value becomes its own cache object. You end up with dozens of stored copies of the same AVIF, most of them cold, all of them competing for cache space and all of them representing an origin miss the first time they are requested.

The fix is normalization: collapse the entire universe of Accept values into a small, finite set of tokens — avif, webp, or jpeg — and vary the cache on that. Three tokens means at most three cache objects per asset, each one hot. The negotiation stays correct because the token still captures the only distinction that changes the bytes, while the fragmentation disappears because everything else in the header is discarded before it reaches the hash.

Configuration and parameter reference

The table below is the working vocabulary for the VCL and Image Optimizer settings this guide uses. Every one of them appears in a snippet further down.

Parameter / object Where it lives Meaning
req.http.Accept vcl_recv Raw client Accept header. Read-only signal; never hash it directly.
req.http.X-Fmt vcl_recv → vcl_hash Custom header holding the normalized token (avif/webp/jpeg). You create it.
hash_data() vcl_hash Appends a value to the cache key. Call once per component (url, host, token).
bereq.http.Accept shield / origin request The Accept forwarded to Image Optimizer; drives format=auto.
format=auto Image Optimizer query param Lets IO pick the best format the client supports based on Accept.
auto=webp,avif Image Optimizer query param Explicitly enables AVIF+WebP auto-negotiation output.
beresp.ttl vcl_fetch Edge cache lifetime for the fetched object.
Surrogate-Key origin response header Space-delimited tags for grouped purging (e.g. asset id).
Surrogate-Control origin response header Edge-only cache directives, stripped before reaching the browser.
Fastly-Debug: 1 request header Makes Fastly emit diagnostic response headers (state, POP, hash inputs).
Fastly-IO-Info response header Image Optimizer diagnostics: source/output dimensions, format, quality.

Step-by-step: building the negotiation VCL

Step 1 — Normalize Accept into a token in vcl_recv

The first job is to reduce the incoming Accept header to a single token. Order matters: test for the highest-value format first, because a Chrome client advertises both image/avif and image/webp and you want it to land on avif.

sub vcl_recv {
  #FASTLY recv

  # Only run negotiation for image routes; leave everything else untouched.
  if (req.url.path ~ "^/img/") {
    # Order is significant: AVIF is the best format, so test it FIRST.
    # A Chrome request advertises BOTH image/avif and image/webp — matching
    # webp first would down-tier every AVIF-capable client to WebP.
    if (req.http.Accept ~ "image/avif") {
      set req.http.X-Fmt = "avif";
    } elsif (req.http.Accept ~ "image/webp") {
      set req.http.X-Fmt = "webp";
    } else {
      # Everything else — including a missing Accept header (crawlers,
      # curl without -H) — gets the universally decodable JPEG baseline.
      set req.http.X-Fmt = "jpeg";
    }
  }

  return(lookup);
}

Warning: Do not skip the explicit else. A request with no Accept header at all (many bots, and curl without -H) would otherwise leave X-Fmt unset, and an unset header hashes differently from "jpeg". That silently creates a fourth cache bucket for “no format” traffic. Always terminate the branch with a concrete default.

Step 2 — Fold the token into the cache key in vcl_hash

Normalizing the header does nothing on its own — Fastly’s default hash ignores custom headers entirely. You must add the token to the key by hand. The #FASTLY hash macro emits the default req.url + req.http.host hashing; you append your token after it.

sub vcl_hash {
  # The default macro hashes req.url and the host. Keep it.
  #FASTLY hash

  # Append the normalized format token so avif/webp/jpeg variants of the
  # SAME url are stored as three DISTINCT cache objects. Without this line,
  # the first format cached for a url is served to every client regardless
  # of what they can decode — classic format cache poisoning.
  hash_data(req.http.X-Fmt);

  return(hash);
}

Tradeoff: every value you add to hash_data() multiplies the number of cache objects per URL. Three format tokens is the sweet spot. If you were tempted to also hash on DPR, viewport, or a resize width, do the multiplication first: three formats × five widths × two DPRs is thirty objects per image, and most of them will be cold. Keep the hash surface as small as the actual byte-level distinctions require.

Step 3 — Drive Image Optimizer with format=auto

Rather than pre-generating three files per image at build time, you can let Fastly Image Optimizer transcode on the fly and cache the result. Point your image URLs at an origin holding a single high-quality master (JPEG or PNG), enable Image Optimizer on the service, and append format=auto. IO then reads the forwarded Accept header and emits the best format the client supports.

sub vcl_recv {
  #FASTLY recv

  if (req.url.path ~ "^/img/") {
    # ... X-Fmt normalization from Step 1 ...

    # Ask Image Optimizer to auto-select the output format.
    # auto=webp,avif enables BOTH next-gen outputs; IO downgrades to JPEG
    # for clients whose Accept advertises neither.
    set req.url = querystring.set(req.url, "format", "auto");
    set req.url = querystring.set(req.url, "auto", "webp,avif");

    # Optional: cap output quality to hold file sizes predictable.
    # quality is on a 1–100 scale here (unlike avifenc's inverted quantizer).
    set req.url = querystring.set(req.url, "quality", "80");
  }

  return(lookup);
}

Because Image Optimizer keys its own transform cache on the resolved output format, and your vcl_hash keys the edge cache on X-Fmt, the two layers agree: an AVIF request produces an AVIF transform stored under the avif bucket, and it is never handed to a WebP-only client. IO honours the same format negotiation contract described in the MIME type configuration guide — it sets Content-Type: image/avif on the output so the browser decodes it correctly.

Step 4 — Shield the origin

Without shielding, every Fastly POP that misses fetches independently from your origin. For an on-the-fly transcoding setup that is expensive: the same master image gets re-fetched and re-encoded at dozens of edge locations. Designating one POP as a shield collapses those misses into a single origin fetch that all other POPs read through.

sub vcl_recv {
  #FASTLY recv

  # Route origin-bound misses through the London shield POP.
  # Edge POPs fetch from the shield; only the shield fetches from origin,
  # so the master image is pulled — and IO-encoded — at most once per variant.
  if (!req.http.Fastly-FF) {
    set req.backend = origin_shield_london;
  }

  return(lookup);
}

Tradeoff: shielding adds one internal hop for requests that miss at both the edge and the shield, costing a few milliseconds of latency on cold objects. In exchange it can cut origin traffic and Image Optimizer encode load by an order of magnitude on a global audience. For media where objects are long-lived and hit rates are high, the cold-path latency is amortized to near zero.

Step 5 — Set TTL and surrogate keys in vcl_fetch

The origin should tag each response with a Surrogate-Key so you can purge a single logical asset — across all three format variants — with one API call. Surrogate keys are space-delimited and applied at the object level, independent of the format token in the hash.

sub vcl_fetch {
  #FASTLY fetch

  if (req.url.path ~ "^/img/") {
    # Long edge TTL; content-hashed URLs make this safe.
    set beresp.ttl = 31536000s;   # 1 year

    # Surrogate-Control is honoured by Fastly and then stripped, so the
    # browser sees only Cache-Control. This lets edge and browser TTLs differ.
    set beresp.http.Surrogate-Control = "max-age=31536000";
    set beresp.http.Cache-Control = "public, max-age=86400";

    # If origin did not tag the object, derive a surrogate key from the path.
    # One purge of this key invalidates avif + webp + jpeg together because
    # all three share the same logical asset id, not the format token.
    if (!beresp.http.Surrogate-Key) {
      set beresp.http.Surrogate-Key = regsub(req.url.path, "^/img/(.+)$", "asset-\1");
    }
  }

  return(deliver);
}

Step 6 — Surface diagnostics in vcl_deliver

Finally, expose what happened so you can debug from the client side. Echo the served format and the cache state; strip internal headers you do not want to leak.

sub vcl_deliver {
  #FASTLY deliver

  # Echo the normalized format decision back for curl-based verification.
  set resp.http.X-Served-Fmt = req.http.X-Fmt;

  # fastly_info.state exposes HIT/MISS/PASS; surface it as a simple X-Cache.
  if (fastly_info.state ~ "HIT") {
    set resp.http.X-Cache = "HIT";
  } else {
    set resp.http.X-Cache = "MISS";
  }

  # Never leak the internal token-selection header to the public.
  unset resp.http.X-Fmt;

  return(deliver);
}

Purging format-negotiated variants

Surrogate keys turn a cache-wide invalidation problem into a targeted one. When you re-encode or replace an asset, you do not want to flush the entire cache, and you cannot purge by URL alone because a single URL maps to three cached objects (one per format token). Purging by the shared surrogate key invalidates all three at once:

# Purge every format variant of one asset by its surrogate key.
# This clears the avif, webp, AND jpeg objects that share asset-hero-hero.jpg,
# because the key is on the logical asset, not the format token.
curl -X POST \
  -H "Fastly-Key: $FASTLY_API_TOKEN" \
  "https://api.fastly.com/service/$SERVICE_ID/purge/asset-hero.jpg"

# Soft-purge instead (mark stale, serve stale-while-revalidate) by adding:
#   -H "Fastly-Soft-Purge: 1"
# This avoids a thundering-herd of origin misses right after the purge.

Tradeoff: a hard purge evicts objects immediately, so the next request for each variant is an origin miss and an Image Optimizer re-encode. On a high-traffic asset that can briefly spike origin load. A soft purge marks the objects stale and lets Fastly serve the stale copy while it revalidates in the background — smoother, at the cost of serving the old bytes for a few seconds. This mirrors the stale-while-revalidate behaviour discussed in the Cache-Control headers guide.

Tradeoffs and failure modes

Failure mode Cause Fix
Cache fragments into hundreds of objects Hashing raw req.http.Accept instead of a normalized token Normalize to X-Fmt in vcl_recv; hash_data(req.http.X-Fmt) only
AVIF served to a WebP-only Safari 15 client vcl_hash never appends the format token; first-cached format wins Add hash_data(req.http.X-Fmt) in vcl_hash
Chrome down-tiered to WebP despite AVIF support webp tested before avif in the normalization branch Test image/avif first — Chrome advertises both
Fourth “unset” cache bucket appears Missing else leaves X-Fmt unset for no-Accept requests Terminate the branch with set req.http.X-Fmt = "jpeg";
Origin CPU spikes under global traffic No shield; every POP re-fetches and re-encodes the master Designate a shield POP; gate on !req.http.Fastly-FF
Variant explosion from resize + format Hashing width × DPR × format multiplies objects Constrain resize widths to a fixed allow-list; keep the format axis to 3 tokens
Purge misses some variants Purging by URL, which maps to 3 objects Tag with a shared Surrogate-Key; purge the key
Browser revalidates every load Long edge TTL leaked to the client via Cache-Control Split edge vs browser TTL using Surrogate-Control

Debugging Fastly negotiation

Confirm the normalized decision and cache state

Request the same URL with three different Accept headers and confirm the served Content-Type, the normalized format, and the cache state all agree:

# AVIF-capable client (Chrome-like). Expect content-type: image/avif.
curl -sI -H 'Accept: image/avif,image/webp,*/*;q=0.8' \
  https://cdn.example.com/img/hero.jpg \
  | grep -iE 'content-type|x-served-fmt|x-cache'

# WebP-only client (Safari 14/15-like). Expect content-type: image/webp.
curl -sI -H 'Accept: image/webp,*/*;q=0.8' \
  https://cdn.example.com/img/hero.jpg \
  | grep -iE 'content-type|x-served-fmt|x-cache'

# Legacy client, no next-gen support. Expect content-type: image/jpeg.
curl -sI -H 'Accept: */*' \
  https://cdn.example.com/img/hero.jpg \
  | grep -iE 'content-type|x-served-fmt|x-cache'

Expected output for the first request:

content-type: image/avif
x-served-fmt: avif
x-cache: HIT

If x-served-fmt: avif but content-type: image/webp, your normalization and your Image Optimizer output disagree — usually because format=auto is reading a different Accept than the one you normalized (check that shielding is not stripping it).

Enable Fastly-Debug to inspect hashing and POP routing

Sending Fastly-Debug: 1 makes Fastly emit diagnostic headers, including which POP served the request and the internal cache state — invaluable for confirming that the shield is in the path:

# Fastly-Debug surfaces X-Served-By (POP chain), X-Cache-Hits, and the
# object's Vary/state. A two-POP X-Served-By chain confirms shielding is active.
curl -sI -H 'Fastly-Debug: 1' \
  -H 'Accept: image/avif,image/webp,*/*;q=0.8' \
  https://cdn.example.com/img/hero.jpg \
  | grep -iE 'x-served-by|x-cache|x-cache-hits|vary'

Read Fastly-IO-Info to verify the transform

When Image Optimizer is active, the Fastly-IO-Info response header reports what IO actually did — source dimensions, output dimensions, output format, and quality. It is the ground truth for whether format=auto selected AVIF:

# Fastly-IO-Info example value:
#   ifsz=482301 idim=2000x1333 ifmt=jpeg ofsz=61204 odim=2000x1333 ofmt=avif
# ofmt=avif confirms IO transcoded to AVIF for this Accept header.
curl -sI -H 'Accept: image/avif,image/webp,*/*;q=0.8' \
  https://cdn.example.com/img/hero.jpg \
  | grep -i 'fastly-io-info'

If ofmt reports jpeg for an AVIF-capable Accept, Image Optimizer is not seeing the header it needs — verify auto=webp,avif is present and that no upstream subroutine unset bereq.http.Accept before the shield forwarded it.

Where to go next

The single most consequential — and most error-prone — piece of this pipeline is the normalization step itself. Getting the branch order, the hash_data() call, and the origin Vary exactly right is what separates a clean three-object cache from a poisoned or fragmented one. That exact procedure, with a fully annotated vcl_recv block and curl-based verification for each tier, is covered in Fastly VCL: normalize the Accept header for AVIF.