Core Media Fundamentals & Next-Gen Formats
Every millisecond of Largest Contentful Paint delay and every kilobyte of unnecessary payload translates directly into lost conversions, higher infrastructure bills, and lower Core Web Vitals scores. Getting media delivery right means making precise decisions at every layer: which codec to encode with, which quantizer range to target, how HTTP headers instruct CDN caches to store format-negotiated variants, and how build pipelines automate the transcoding work at scale. This section covers the foundational theory and production patterns across all of those layers.
What this section covers
The topics below form the systematic architecture for next-gen media delivery. Each area is explored in depth across its own dedicated pages.
AVIF vs WebP Compression Benchmarks — SSIM/VMAF scoring methodology, file-size comparisons at matched perceptual quality, and the conditions under which AVIF’s superior compression justifies its slower encode time versus WebP’s broader decode support.
MIME Type Configuration for Modern Media Servers — Correct Content-Type registration for image/avif, image/webp, video/webm, and video/mp4; codecs=av01 on Nginx, Apache, and Caddy. Misconfigured MIME types cause browsers to silently discard <source> elements.
Cache-Control Headers for Image and Video Assets — max-age, immutable, stale-while-revalidate, and the Vary: Accept directive that prevents CDNs from serving a cached WebP to an AVIF-capable client.
Understanding Video Codecs: VP9 vs H.265 vs AV1 — Codec lineage, hardware decoding matrices, licensing considerations, and encode-time vs compression tradeoffs for streaming and background video use cases.
Beneath those four topics sit the narrow, operational questions that actually break deployments. On the image side: the AVIF fallback chain for Safari 14, the audience math behind choosing WebP over JPEG, and the archive-versus-delivery split covered in the AVIF vs JPEG XL decision guide. On the server side: registering the AVIF MIME type on Apache and Caddy and debugging incorrect Content-Type headers on WebM video. On the caching side: choosing a max-age for CDN media assets and stale-while-revalidate for media. On the video side: AV1 vs VP9 encode time on AWS Lambda and hardware AV1 decode support by device class.
Read in order, those pages answer one question per layer: what do I encode, how does the server describe it, how does the cache key it, and how does the client select it. Get any one of the four wrong and the other three stop mattering — a perfectly encoded AVIF served as application/octet-stream is a broken image, and a correctly typed AVIF served without Vary: Accept becomes a broken image for somebody else’s browser an hour later.
Pipeline overview
The diagram below shows how format selection, HTTP negotiation, CDN caching, and client-side loading interact end-to-end. Each stage is a decision point where a wrong setting propagates downstream errors.
Core theory: codecs, compression, and HTTP semantics
Image codec fundamentals
AVIF — derived from the AV1 video codec — encodes still images using the same intra-frame prediction, transform, and entropy coding tools that make AV1 competitive with HEVC for video. The key encoding parameters are --min and --max (quantizer range, where 0 is lossless and 63 is maximum loss — inverted from most quality sliders), --speed (0–10, lower is slower but smaller), and --depth (8 or 10 bits per channel). At matched perceptual quality as measured by SSIM or VMAF, AVIF typically produces files 30–50% smaller than JPEG and 15–25% smaller than WebP.
WebP uses the VP8 intra-frame codec for lossy compression and a purpose-built lossless mode for graphics with sharp edges and flat regions. Its quality parameter (-q 0–100) runs in the conventional direction — higher equals better quality. WebP enjoys wider decode support than AVIF (every browser since Chrome 23/Firefox 65/Safari 14), making it the safer universal fallback before AVIF.
Chroma subsampling governs how colour information is sampled relative to luminance. 4:2:0 halves colour resolution in both dimensions, reducing file size by roughly 50% for chroma data with minimal visible impact on photographic content. 4:4:4 retains full colour fidelity, which matters for text overlaid on images, UI screenshots, and product imagery where colour accuracy is brand-critical. Specifying the wrong subsampling mode for UI graphics in avifenc produces visible chroma bleeding on high-contrast edges.
Inside an AVIF file: container structure
AVIF is not a bespoke file format. It is a HEIF/MIAF profile built on the ISO Base Media File Format — the same box structure as an MP4 — with AV1 bitstreams stored as image items rather than as a timed video track. Knowing the box layout explains most of the confusing failures engineers hit in production.
A minimal still AVIF contains four top-level boxes. ftyp declares the major brand (avif for a still image, avis for an image sequence) plus compatible brands such as mif1 and miaf; a decoder that does not recognise the major brand rejects the file before touching a pixel. meta holds the item metadata: hdlr marks the handler as pict, pitm names the primary item, iloc records the byte offset and length of every item inside mdat, iinf lists item types, and iprp/ipco stores item properties. mdat holds the actual coded bytes. iref links auxiliary items to the primary item.
Inside ipco the four properties that matter are av1C (the AV1 codec configuration record — profile, level, chroma subsampling flags, and monochrome flag), ispe (the image spatial extent, i.e. width and height), pixi (bits per channel), and colr (colour information, discussed below). Alpha is not a fourth channel in the primary bitstream: it is a separate monochrome AV1 item, tagged with an auxC property whose URN is urn:mpeg:mpegB:cicp:systems:auxiliary:alpha, and bound to the primary item through an auxl entry in iref. That design is why some older decoders render an AVIF’s colour correctly but drop its transparency entirely — they parse the primary item and ignore the auxiliary one.
Two more structural features have direct delivery consequences. Tiling splits the image into independently decodable columns and rows so that multi-threaded decoders can work in parallel; avifenc --tilerowslog2 / --tilecolslog2 control it, and enabling tiling on small images costs bytes (each tile restarts entropy coding) while buying nothing, because a 400 px thumbnail decodes in a single frame time anyway. Film grain synthesis stores a grain parameter set instead of the grain itself, letting the encoder throw away expensive high-frequency noise and have the decoder re-synthesise it; it can cut 20–30% off a grainy film scan, but the grain is regenerated at decode time, so pixel-comparison tests against the source will fail by design.
Colour signalling: CICP, nclx, and ICC
The single most common “my AVIF looks wrong” bug is not a compression artefact — it is colour signalling. AVIF carries colour information in the colr box in one of two flavours: nclx (three small integers from ITU-T H.273’s Code Point Identification tables, plus a full-range flag) or prof (an embedded ICC profile). The nclx triple is compact and unambiguous, and it is what avifenc writes by default.
The three code points are colour primaries (1 = BT.709/sRGB gamut, 9 = BT.2020, 12 = DCI-P3), transfer characteristics (13 = sRGB, 16 = PQ/SMPTE 2084 for HDR10, 18 = HLG), and matrix coefficients (1 = BT.709, 0 = identity/GBR, which is what 4:4:4 lossless RGB uses). The fourth value, the full-range flag, decides whether luma spans 0–255 or the studio range 16–235.
Get the full-range flag wrong and the image is not subtly off — it is visibly flat: a full-range source tagged as limited range makes the decoder stretch 16–235 out to 0–255, crushing blacks and clipping highlights. Get transfer characteristics wrong on an HDR asset and a PQ image decoded as sRGB looks grey and desaturated. When both an ICC profile and nclx are present, browsers currently prefer the ICC profile, which means a stale ICC profile inherited from the source JPEG will silently override the correct nclx triple you just set.
Warning: Sharp strips the input ICC profile unless you pass withMetadata({ icc: … }), and avifenc copies it unless you pass --ignore-icc. Pick one path deliberately. For sRGB web delivery, the smallest and most predictable result is: strip ICC, write nclx as 1/13/1 with full range, and let the browser assume sRGB.
Video codec fundamentals
For background video, short clips, and streaming, codec selection involves three axes: compression efficiency, hardware decode availability, and licensing cost. H.264 (AVC) decodes in hardware on every device made in the last decade, carries no runtime royalty for free distribution, and is the universal baseline. VP9 — developed by Google as a royalty-free H.265 alternative — achieves roughly 30–50% better compression than H.264 at matched quality. AV1 extends that improvement by another 20–30% over VP9 but requires more complex decode logic; hardware AV1 decode is now standard on Apple Silicon, recent Intel Arc, AMD RDNA3, and Qualcomm Snapdragon 8 Gen series, but older mobile chips fall back to costly software decode. H.265 (HEVC) matches or exceeds AV1 compression on some encoders but carries mandatory patent royalties. Full codec analysis and hardware matrix: Understanding Video Codecs: VP9 vs H.265 vs AV1.
HTTP negotiation semantics
The Accept request header signals which image formats the browser supports. A Chrome request includes image/avif,image/webp,*/*;q=0.8. Servers and CDNs that serve format-negotiated content from a single URL must include Vary: Accept in the response so intermediate caches store separate variants per Accept value. Without it, a CDN that cached a WebP response will serve the WebP to the next AVIF-capable client, negating the format negotiation entirely.
The header is a weighted preference list, not a capability bitmask. Chrome 120 sends image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8 for <img> subresources; Firefox sends a different list, and both vary by request destination — a navigation request, a CSS background-image fetch, and an <img> fetch can carry three different Accept strings from the same browser on the same page. Negotiation logic must therefore substring-match on the format token (image/avif), never string-compare the whole header, and must treat image/* and */* as “no format preference expressed”, falling back to JPEG rather than guessing.
Content-Type must match the actual bytes. Nginx does not auto-detect AVIF or WebM without explicit MIME type registration. A missing or wrong Content-Type causes Firefox and Safari to reject the resource silently, with no error in the console — only a missing image or blank video. Correct registration is covered in depth at MIME type configuration for modern media servers.
The sequence below traces two clients through the same negotiated URL and shows exactly where the cache key is formed.
Two operational consequences follow from that key. First, the edge must actually receive the Accept header: CloudFront strips unlisted request headers before they reach the cache key, so Accept has to be added to a Cache Policy explicitly. Second, the raw header is far too high-cardinality to key on directly — Chrome, Firefox, Safari, and every embedded webview send subtly different strings, and each unique string becomes a separate object at the edge. The fix is normalisation: collapse Accept to a two- or three-value token (avif, webp, none) in a VCL snippet, a Cloudflare Worker, or a CloudFront Function, and key the cache on that token instead.
Client Hints are the modern complement to Accept. An origin that responds with Accept-CH: Sec-CH-DPR, Sec-CH-Width, Sec-CH-Viewport-Width opts the browser into sending device pixel ratio and layout width on subsequent requests, which lets the origin pick a resolution as well as a format. Two caveats apply: hints are only sent after the opt-in response, so the first navigation never has them (add Critical-CH to force a retry when the decision is load-bearing), and every hint you key on multiplies your variant count exactly as Accept does. Save-Data: on is the cheapest hint to honour — one boolean, and a clear user signal that justifies dropping a quality tier.
Reference data: compression and decode performance
The table below reflects typical results from encoding a 2 MP photographic JPEG at visual equivalence (SSIM ≈ 0.95). Decode times are measured on a 2021 MacBook Pro (M1) running Chrome 120; mobile figures (Pixel 7) are higher due to software-path AV1 decode on older SoCs.
| Format | Typical file size | vs JPEG baseline | SSIM at size | Encode time (Sharp / libvips) | Hardware decode | Safari 14 | Safari 16 | Chrome 85+ | Firefox 93+ | Edge 18+ |
|---|---|---|---|---|---|---|---|---|---|---|
| PNG (24-bit) | 1.42 MB | +407% | 1.00 (lossless) | ~40 ms | Universal | Yes | Yes | Yes | Yes | Yes |
| JPEG | 280 KB | — | 0.95 | ~80 ms | Universal | Yes | Yes | Yes | Yes | Yes |
| WebP (q=80) | 195 KB | −30% | 0.95 | ~110 ms | Partial (Chrome/Edge) | Yes | Yes | Yes | Yes | Yes |
| AVIF (q=80 equiv) | 155 KB | −45% | 0.95 | ~900 ms | Apple Silicon, Chrome 121+ | No | Yes | Yes | Yes | Yes |
| AVIF 4:4:4 (q=80 equiv) | 188 KB | −33% | 0.96 | ~1150 ms | Apple Silicon, Chrome 121+ | No | Yes | Yes | Yes | Yes |
| AVIF (q=60, thumbnail tier) | 96 KB | −66% | 0.91 | ~640 ms | Apple Silicon, Chrome 121+ | No | Yes | Yes | Yes | Yes |
| JPEG XL | 145 KB | −48% | 0.95 | ~1200 ms | None currently | No | No | Experimental | No | No |
Note on AVIF encode time: the ~900 ms figure uses avifenc --speed 6. Dropping to --speed 4 or lower increases encode time 3–5× for marginal quality gains. In CI/CD pipelines, set --speed 6 or --speed 8 for trunk builds and reserve slower presets for pre-release or CDN on-the-fly encoding.
Encoder effort ladder
Encoder “speed” or “effort” presets are the highest-leverage dial in the whole pipeline, because they trade CI minutes against bytes on a strongly non-linear curve. The figures below are for the same 2 MP source through avifenc, single-threaded, normalised so that --speed 6 equals 100% size and 100% time.
avifenc --speed |
Relative size | Relative encode time | Practical use |
|---|---|---|---|
| 0 | 95% | ~1400% | Never in CI; reference-quality one-offs only |
| 2 | 96% | ~620% | Marketing hero images encoded once and cached forever |
| 4 | 98% | ~310% | Release builds where image count is small |
| 6 | 100% | 100% | Default for trunk CI — the knee of the curve |
| 8 | 104% | ~45% | Pull-request previews, developer machines |
| 10 | 113% | ~18% | On-the-fly edge encoding with a warm cache behind it |
Tradeoff: going from --speed 6 to --speed 0 costs 14× the CPU to save 5% of the bytes. On a 4,000-image catalogue that is the difference between an eight-minute and a two-hour build for roughly 30 KB saved per image. Spend the CPU only where the image is served millions of times — and never on user-uploaded content encoded on a request path.
Video format reference
| Codec | Container | Bitrate vs H.264 | HW decode (2024+) | Royalty-free | Safari 14 | Chrome 85+ | Firefox 93+ | Edge 18+ |
|---|---|---|---|---|---|---|---|---|
| H.264 (AVC) | MP4 | baseline | Universal | No (MPEG-LA) | Yes | Yes | Yes | Yes |
| VP9 | WebM | −30–50% | Partial (no Apple) | Yes | No | Yes | Yes | Yes |
| H.265 (HEVC) | MP4 | −40–50% | Wide (incl. Apple) | No (HEVC advance) | Yes | No | No | Partial |
| AV1 | MP4/WebM | −50–60% | Apple Si, Intel 12+, Snapdragon 8 Gen 2+ | Yes | Yes (Safari 17+) | Yes (Chrome 90+) | Yes | Yes |
Canonical delivery pattern
The following snippet represents the production-standard format fallback chain for a Largest Contentful Paint image. Every attribute is annotated.
<picture>
<!--
AVIF source: highest compression, ~45% smaller than JPEG at matched quality.
Served to Chrome 85+, Firefox 93+, Safari 16+.
avifenc must register image/avif in Nginx — see MIME type guide.
-->
<source
srcset="hero-480w.avif 480w, hero-800w.avif 800w, hero-1200w.avif 1200w"
sizes="(max-width: 600px) 100vw, (max-width: 1200px) 50vw, 800px"
type="image/avif"
>
<!--
WebP fallback: ~30% smaller than JPEG; supported in Safari 14+, all Chromium.
Required for Safari 14/15 which lack AVIF support.
-->
<source
srcset="hero-480w.webp 480w, hero-800w.webp 800w, hero-1200w.webp 1200w"
sizes="(max-width: 600px) 100vw, (max-width: 1200px) 50vw, 800px"
type="image/webp"
>
<img
src="hero-800w.jpg"
srcset="hero-480w.jpg 480w, hero-800w.jpg 800w, hero-1200w.jpg 1200w"
sizes="(max-width: 600px) 100vw, (max-width: 1200px) 50vw, 800px"
alt="Landscape hero image showing mountain ridge at sunrise"
width="800"
height="500"
<!--
fetchpriority=high: instructs the preload scanner to fetch this resource
before lower-priority resources. Reserve for the primary LCP element only —
applying to 2+ images triggers bandwidth contention with CSS/fonts.
-->
fetchpriority="high"
<!--
loading=eager is the default; stated explicitly to distinguish from
below-fold images that use loading=lazy. Do NOT set loading=lazy on LCP.
-->
loading="eager"
<!--
decoding=async offloads image decode to a non-blocking thread.
Has no effect on LCP timing but prevents jank during scroll.
-->
decoding="async"
>
</picture>
Tradeoff: width and height attributes prevent Cumulative Layout Shift (CLS) by reserving space before the image loads. Without them, browsers reflow the page when the image dimensions are known, causing CLS scores to spike. The values must match the intrinsic size of the src fallback image.
Pipeline integration
Image transcoding at build time
Sharp (Node.js, backed by libvips) is the standard tool for build-time image conversion. It operates on streams and processes multiple output formats in a single decode pass, making it significantly faster than invoking avifenc and cwebp separately for every source image.
// scripts/optimize-images.mjs
import sharp from 'sharp';
import { glob } from 'glob';
import { basename, dirname } from 'path';
const sources = await glob('src/media/**/*.{jpg,png}');
for (const src of sources) {
const dir = dirname(src).replace('src/', 'dist/');
const name = basename(src, /\.(jpg|png)$/.exec(src)[0]);
const pipe = sharp(src);
await pipe
.clone()
// quality=80 maps to AVIF quantizer ~28 internally.
// effort=4 balances encode speed (0=fast, 9=slowest).
// chromaSubsampling='4:2:0' is correct for photography;
// use '4:4:4' for text-heavy UI screenshots.
.avif({ quality: 80, effort: 4, chromaSubsampling: '4:2:0' })
.toFile(`${dir}/${name}.avif`);
await pipe
.clone()
// quality=82 for WebP is perceptually equivalent to AVIF q=80.
.webp({ quality: 82, smartSubsample: true })
.toFile(`${dir}/${name}.webp`);
await pipe
.clone()
// JPEG fallback: mozjpeg encoder reduces file size ~15% vs libjpeg.
.jpeg({ quality: 85, mozjpeg: true })
.toFile(`${dir}/${name}.jpg`);
}
Video transcoding pipeline
For short video clips (background loops, product demos), a two-pass FFmpeg pipeline producing AV1 (MP4) and VP9 (WebM) covers the full browser matrix:
#!/usr/bin/env bash
# transcode.sh — two-format video pipeline for web delivery
INPUT="$1"
STEM="${INPUT%.*}"
# AV1 via SVT-AV1 encoder (libsvtav1). crf=30 targets ~60% smaller than H.264.
# preset=6 balances speed vs compression (0=slowest, 13=fastest).
# -movflags +faststart places the MP4 moov atom at the front for progressive play.
ffmpeg -i "$INPUT" \
-c:v libsvtav1 -crf 30 -preset 6 \
-c:a libopus -b:a 128k \
-movflags +faststart \
"${STEM}-av1.mp4"
# VP9 via libvpx-vp9. -b:v 0 enables constant quality mode (required for -crf).
# crf=33 for VP9 is roughly perceptually equivalent to AV1 crf=30.
# -deadline good / -cpu-used 4 balances quality vs CPU cost in CI.
ffmpeg -i "$INPUT" \
-c:v libvpx-vp9 -b:v 0 -crf 33 \
-deadline good -cpu-used 4 \
-c:a libopus -b:a 128k \
"${STEM}-vp9.webm"
Tradeoff: SVT-AV1 at preset=6 is approximately 3× slower than libvpx-vp9 for equivalent quality. For CI pipelines processing many videos, consider parallelising across CPU cores or offloading to a cloud transcoding API (AWS MediaConvert, Cloudflare Stream) for trunk builds, reserving local FFmpeg for development previews.
Nginx server configuration
Cache-Control headers and MIME type configuration work together in the server block. The critical pairing is Vary: Accept alongside Cache-Control: immutable:
# nginx.conf — next-gen media delivery block
# Ensure mime.types includes: image/avif avif; image/webp webp; video/webm webm;
# (avif and webm are NOT in Nginx's default mime.types before 1.21.x)
location ~* \.(avif|webp|jpg|jpeg|png|gif|mp4|webm)$ {
# immutable: tells browser not to revalidate during max-age window.
# Requires content-hashed URLs (e.g., hero.abc123.avif) so stale assets
# are never served after a deploy.
add_header Cache-Control "public, max-age=31536000, immutable";
# Vary: Accept is mandatory when serving format-negotiated assets from a
# single URL pattern. Without it, a CDN that cached a WebP response will
# incorrectly serve WebP to an AVIF-capable browser (cache poisoning).
add_header Vary "Accept";
gzip off; # AVIF/WebP/MP4/WebM are already compressed; gzip adds CPU with no benefit.
expires 1y; # HTTP/1.0 compatibility alias for max-age=31536000.
try_files $uri =404;
}
Concurrency, memory, and build determinism
Sharp’s throughput is bounded by libvips’ thread pool, not by Node’s event loop. By default libvips sizes its pool to the machine’s core count, and running Promise.all() over 500 images then hands every one of those images to that same pool simultaneously — memory climbs to hundreds of megabytes and the encoder threads thrash. Two settings keep a build predictable:
import sharp from 'sharp';
// concurrency(1) gives each image ONE libvips worker, so outer parallelism
// (a task queue of N workers) becomes the only knob. Without this, N outer
// workers × M libvips threads oversubscribes the CPU and inflates wall time.
sharp.concurrency(1);
// cache(false) disables libvips' operation/file cache. Keep it ON for a dev
// watch loop (repeated re-encodes of the same source), OFF in CI where every
// input is touched once and the cache is pure memory overhead.
sharp.cache(false);
Determinism matters as much as speed. Encoder output is a function of the encoder version, not just its flags: libaom 3.6 and libaom 3.8 produce byte-different AVIFs from identical input and identical parameters. If your deploy pipeline derives asset URLs from a hash of the encoded bytes, an encoder bump silently invalidates every image URL and forces a full CDN refill. Hash the source bytes plus a short encoder-version tag instead (hero.<srchash>.<enc>.avif); you keep immutability, and an encoder upgrade becomes a deliberate, reviewable change rather than a surprise 40 GB of origin egress.
Cost model: what a format migration is actually worth
Format decisions are easier to defend with arithmetic than with adjectives. Take a catalogue site serving 12 million image requests per month at an 88% edge hit ratio, currently on JPEG at the 280 KB average from the reference table.
Egress. 12,000,000 × 280 KB ≈ 3.20 TB/month leaving the edge. Moving to AVIF at 155 KB gives 12,000,000 × 155 KB ≈ 1.77 TB — a saving of 1.43 TB. At a blended CDN rate of $0.06/GB that is roughly $86/month; at AWS CloudFront’s North American on-demand $0.085/GB it is about $122/month. Origin egress falls too, but only on the 12% miss path: 0.17 TB saved there.
Encode. 4,000 catalogue images re-encoded on every deploy at --speed 6 and ~900 ms each is 60 CPU-minutes per full build. Spread over 8 parallel CI workers that is under eight minutes of wall time, and with content-addressed caching only changed images re-encode at all — in steady state a deploy touches tens of images, not thousands.
Storage. Three formats × five widths = 15 derivatives per source. At an average 60 KB across the ladder, 4,000 sources produce ~3.6 GB of derivatives. At $0.023/GB-month that is under $0.09/month; object count (60,000 objects) matters more than bytes for request-priced storage.
Latency. The user-visible number is the one that justifies the work: on a 4G connection modelled at 1.6 Mbps effective throughput, 125 KB less payload is ~625 ms less transfer time for the LCP image. That is a Core Web Vitals threshold’s worth of improvement from a build-time change alone, which is why format work generally outranks JavaScript work on media-heavy pages.
Tradeoff: the same arithmetic argues against AVIF for low-traffic assets. An image served 400 times a month saves 50 MB of egress — a fraction of a cent — while consuming a second of CPU every build. Apply the aggressive tier to the top decile of assets by request volume and let the rarely-requested remainder sit on WebP.
Tradeoffs & failure modes
| Failure mode | Cause | Fix |
|---|---|---|
| CDN serves WebP to AVIF-capable browser | Missing Vary: Accept header |
Add Vary: Accept to every location block serving format-negotiated assets |
| AVIF silently rejected by browser | Missing image/avif MIME type in Nginx |
Register image/avif avif; in mime.types or types {} block |
LCP degrades after adding fetchpriority=high to hero |
Multiple elements have fetchpriority=high, starving CSS/font fetches |
Reserve fetchpriority=high for exactly one element per page — the primary LCP candidate |
| CLS spikes on image load | Missing width/height attributes on <img> |
Always declare intrinsic dimensions; use CSS aspect-ratio as belt-and-suspenders |
| CI build time doubles after switching to AVIF | Default avifenc uses --speed 4 or slower |
Set --speed 6 in CI; use --speed 8 for developer previews |
| WebM video blank in Safari 14/15 | Safari did not support VP9/WebM until Safari 16 | Provide <source type="video/mp4"> H.264 fallback; AV1/MP4 works in Safari 17+ |
avifenc --min/--max produce unexpectedly large files |
Quantizer scale is inverted: --min 0 --max 63 means lossless→worst, opposite of a quality slider |
For q≈80 quality, use --min 20 --max 40; the lower the numbers, the better the quality |
Vary: Accept breaks CloudFront caching |
CloudFront by default does not forward or key on Accept |
Configure a Cache Policy that includes Accept in the cache key |
| AVIF renders but transparency is lost | Decoder read the primary item and ignored the auxiliary alpha item linked via iref/auxl |
Verify the alpha item exists (avifdec --info); for critical transparency keep a WebP source, which stores alpha in-band |
| Colours look flat and washed out after encoding | Full-range source written with the limited-range flag, so the decoder re-expands 16–235 to 0–255 | Set the range explicitly in the encoder and confirm the nclx triple with avifdec --info |
A stale ICC profile overrides correct nclx values |
Source ICC copied into the AVIF; browsers prefer prof over nclx |
Pass --ignore-icc for sRGB delivery, or re-embed a correct profile deliberately |
| Every deploy refills the entire CDN cache | Asset hash computed over encoded bytes, so an encoder version bump changes every URL | Hash the source bytes plus an encoder-version tag |
Edge hit ratio collapses after enabling Vary: Accept |
Raw Accept strings are high-cardinality, so each browser build gets its own cache entry |
Normalise Accept to an avif/webp/none token at the edge and key on that |
| Thumbnails are larger in AVIF than expected | Tiling enabled on small images; each tile restarts entropy coding | Disable tiling below roughly 800 px on the long edge |
| Pixel-diff regression tests fail on unchanged images | Film grain synthesis regenerates noise at decode time | Compare with a perceptual metric (SSIM/Butteraugli) rather than exact pixel equality, or disable grain synthesis for test fixtures |
Browser & CDN compatibility matrix
Image format support
| Feature | Safari 14 | Safari 16 | Chrome 85+ | Firefox 93+ | Edge 18+ |
|---|---|---|---|---|---|
| WebP (lossy) | Yes | Yes | Yes | Yes | Yes |
| WebP (lossless) | Yes | Yes | Yes | Yes | Yes |
| AVIF (8-bit) | No | Yes (16.0+) | Yes | Yes | Yes |
| AVIF (10-bit HDR) | No | Yes (16.4+) | Yes | Yes (partial) | Yes |
| JPEG XL | No | No | Experimental | No | No |
<picture> element |
Yes | Yes | Yes | Yes | Yes |
srcset + sizes |
Yes | Yes | Yes | Yes | Yes |
fetchpriority attribute |
Yes (15.4+) | Yes | Yes (102+) | Yes (132+) | Yes (102+) |
Video codec support
| Codec | Safari 14 | Safari 16 | Chrome 85+ | Firefox 93+ | Edge 18+ |
|---|---|---|---|---|---|
| H.264 / MP4 | Yes | Yes | Yes | Yes | Yes |
| VP9 / WebM | No | No | Yes | Yes | Yes |
| H.265 / MP4 | Yes (HW) | Yes (HW) | No | No | Partial |
| AV1 / MP4 | No | No | Yes | Yes | Yes |
| AV1 / MP4 (Safari) | — | — | — | — | Yes (17+) |
| WebM container | No | No | Yes | Yes | Yes |
CDN format negotiation support
| Feature | Cloudflare | Fastly | AWS CloudFront | Nginx (self-hosted) |
|---|---|---|---|---|
Vary: Accept respected in cache key |
Yes (automatic) | Yes (with Vary enabled) |
Requires Cache Policy config | Yes (default) |
| On-the-fly AVIF conversion | Yes (Image Resizing) | No (requires custom VCL) | No (use Lambda@Edge) | No (use libvips/Sharp) |
| Automatic WebP conversion | Yes (Polish) | No | No | No |
Cache-Control: immutable honoured |
Yes | Yes | Yes | Yes |
Edge Accept header forwarding |
Yes | Requires bereq.http.Accept |
Requires Cache Policy | N/A |
Performance measurement and debugging
Track LCP and decode performance with PerformanceObserver. Separating network transfer time from decode time reveals whether poor LCP is a bandwidth problem (large file, slow CDN) or a decode problem (software fallback, oversized image relative to display size):
// Monitor image resource timing to separate transfer vs decode cost.
// PerformanceObserver fires after the resource completes loading.
const observer = new PerformanceObserver((list) => {
for (const entry of list.getEntries()) {
if (entry.initiatorType !== 'img') continue;
// responseEnd - responseStart = transfer time (network).
// duration - (responseEnd - startTime) ≈ decode + compositing time.
const transferMs = (entry.responseEnd - entry.responseStart).toFixed(1);
const totalMs = entry.duration.toFixed(1);
const decodeMs = (entry.duration - (entry.responseEnd - entry.startTime)).toFixed(1);
console.table({
url: entry.name,
totalMs,
transferMs,
decodeMs,
});
}
});
observer.observe({ type: 'resource', buffered: true });
Tradeoff: High decodeMs values (>50 ms on desktop) indicate the image is being software-decoded — either because hardware AVIF decode is unavailable (older Android) or the image is being decoded at a size larger than its display dimensions. Confirm decode path in Chrome by checking chrome://media-internals or enabling Image Decode in the Performance panel.
For server-side validation, use curl -sI to confirm headers without downloading the body:
# Confirm Content-Type and Vary headers for a negotiated image URL.
# -H 'Accept: image/avif,image/webp,*/*' simulates a Chrome request.
curl -sI -H 'Accept: image/avif,image/webp,*/*' \
https://example.com/images/hero.jpg \
| grep -iE 'content-type|vary|cache-control|x-cache'
Expected output:
content-type: image/avif
vary: Accept
cache-control: public, max-age=31536000, immutable
x-cache: HIT
If content-type: image/jpeg appears despite the Accept header, either the server is not performing format negotiation or the CDN returned a cached non-negotiated response.
Accessibility and progressive enhancement
Progressive enhancement requires that every format tier degrades gracefully: AVIF → WebP → JPEG for images; AV1/MP4 → VP9/WebM → H.264/MP4 for video. Accessibility overlays apply at every tier:
- Every informative
<img>requires a descriptivealtattribute. Decorative images (background textures, purely visual dividers) usealt=""and optionallyaria-hidden="true"to remove them from the accessibility tree. - Animated media (GIF replacements, background loops) must respect
@media (prefers-reduced-motion: reduce). Either pause the animation or replace it with a static poster image. - Video content that conveys information must include synchronized captions (WebVTT
<track kind="captions">) and audio descriptions for visual-only content. - SVG diagrams embedded inline require
<title>,<desc>,role="img", andaria-labelon the root element. Interactive SVGs (zoom, toggle) require keyboard event handlers.
There is a performance dimension to progressive enhancement too. Because <picture> selection happens in the preload scanner before layout, the browser cannot know the final rendered size unless sizes is present — so an art-directed <picture> with no sizes will frequently download a candidate one step too large. Similarly, <source media="…"> art direction and srcset resolution switching solve different problems and should not be mixed to express the same rule: use media when the crop changes, srcset/sizes when only the resolution changes.
Frequently asked questions
Does AVIF replace WebP, or do I still need both?
Both. Safari 14 and 15 decode WebP but not AVIF, and that cohort is small but non-zero on any consumer site. The cost of keeping the WebP tier is a build step and some object storage; the cost of dropping it is that a slice of iOS traffic falls all the way through to JPEG, losing 30% of the saving you were trying to capture. Drop WebP only once your own analytics show that pre-Safari-16 traffic is negligible.
Why does my AVIF look washed out compared to the JPEG source?
Almost always colour signalling rather than compression. Check the full-range flag first — a full-range source written as limited range crushes blacks and clips highlights across the whole image, which is easy to mistake for over-aggressive quantisation. Then check the transfer characteristics; a PQ or HLG asset decoded as sRGB looks grey and desaturated. avifdec --info prints the nclx triple and the range flag.
Is Vary: Accept enough to make CDN negotiation safe?
It is necessary, not sufficient. It stops cross-format poisoning, but it also makes every distinct Accept string its own cache object, and browsers in the wild emit hundreds of variations. Normalise the header to a small token set at the edge before the cache key is computed, and confirm the result by checking your hit ratio before and after — a drop of more than a couple of points means normalisation is not actually applied.
What encoder speed should CI use for AVIF?
--speed 6 for trunk builds, --speed 8 for pull-request previews. The effort ladder above shows why: below speed 4 the encoder spends 3–14× longer for a size gain in the low single digits. Reserve speed 0–2 for a handful of permanently cached hero images.
Should I gzip or Brotli media files?
No. AVIF, WebP, MP4, and WebM are already entropy-coded; recompressing them costs CPU at origin and edge, adds latency, and typically moves the transfer size by well under 1%. Turn compression off for those content types explicitly — many default server configs compress image/* blindly.
How do I tell whether AVIF is decoding in hardware?
Record a Performance trace in Chrome and look at the Decode Image tasks under rasterisation, or open chrome://media-internals. As a rule of thumb, a 2 MP AVIF decoding in software costs 20–40 ms on a mid-range mobile SoC versus under 10 ms on a hardware path — enough to show up as an INP regression on a page with a dozen images.
Do I need a different strategy for user-uploaded images? Yes. The encode-cost model above assumes build-time transcoding of a known catalogue. For uploads, encode a fast tier synchronously (--speed 8 or WebP) so the asset is immediately usable, then re-encode to the aggressive tier asynchronously and swap the derivative in behind a content-addressed URL. Never run a slow AVIF encode inside a request handler.
Related
- AVIF vs WebP Compression Benchmarks — SSIM/VMAF scoring and file-size data
- MIME Type Configuration for Modern Media Servers — AVIF, WebP, WebM on Nginx and Apache
- Cache-Control Headers for Image and Video Assets — max-age, immutable, Vary
- Understanding Video Codecs: VP9 vs H.265 vs AV1 — hardware decode matrix and licensing
- Lazy Loading, Preloading & Fetch Priorities — IntersectionObserver and fetchpriority patterns
- Responsive Image & Video Delivery — srcset, sizes, and art direction with the picture element