Combining Art Direction with AVIF Fallbacks

Art direction varies the image by viewport; format fallback varies it by browser support. A <picture> element that does both has to express a two-dimensional decision as a one-dimensional ordered list, and the flattening is where things break: put the axes in the wrong nesting order and a Chrome user on a phone downloads the desktop crop, or a Safari 14 user at 1440 px gets nothing at all until the terminal <img> rescues them with a 94 KB JPEG of the wrong composition. This page covers the ordering law that makes the flattening correct, the completeness invariant your build has to enforce, how to preload one cell of the matrix without fetching two, and when to collapse the format axis entirely. It assumes the crop-generation and CLS groundwork from art direction with the HTML picture element is already done.

Prerequisite checklist

Step 1 — Nest the axes: media outer, type inner

The selection algorithm evaluates each <source> in document order and takes the first one where the media condition matches and the type is a format the engine can decode. Both tests must pass; failing either sends the browser to the next sibling. That single rule fully determines the correct nesting.

Group by crop on the outside, by format on the inside. Every media band becomes a contiguous run of sources that share one media value and differ only in type, ordered best-compression-first. The browser walks into the first band whose media matches, then walks along that band until it finds a format it supports. Because a band is contiguous, a format miss inside it lands on the next format for the same crop — which is exactly what you want.

Invert the nesting — group all AVIF sources together, then all WebP — and a format miss sends the browser out of its crop band entirely, into the next format group where the media conditions start over from the widest breakpoint again. It sometimes still lands somewhere sane, which is what makes the bug survive review.

Document Order of a Three-Crop, Two-Format Matrix A grid with three rows for the media bands at 1024 pixels and up, 640 to 1023, and no media condition, and two columns for AVIF and WebP. Each of the six cells is numbered with its position in document order: one and two on the first row, three and four on the second, five and six on the third. A terminal image element spans the width beneath. One cell is highlighted as the one a Safari 14 browser at 1440 pixels selects. type="image/avif" type="image/webp" media="(min-width: 1024px)" 1600 × 900 landscape 1 hero-lg.avif · 68 KB 2 hero-lg.webp · 102 KB media="(min-width: 640px)" 1024 × 576 landscape 3 hero-md.avif · 44 KB 4 hero-md.webp · 68 KB no media attribute 480 × 640 portrait 5 hero-sm.avif · 32 KB 6 hero-sm.webp · 51 KB terminal <img src="hero-sm.jpg"> · 94 KB reached only without <source> support Highlighted: cell 2, the source a Safari 14 client at 1440 px selects — right crop, second-best format. Numbers are document order. Reading left-to-right then down is exactly the order the engine evaluates.

The invariant that falls out of the grid is format completeness per band: every row must have an entry for every format tier your audience includes. A row with an AVIF cell and no WebP cell is a hole — a Safari 14 client whose viewport lands in that band skips the AVIF, finds no sibling with a supported type, and continues into the next band, silently receiving a crop meant for a different viewport. It will not error, it will not warn, and the image will look almost right, which is why this is worth asserting in CI rather than eyeballing.

The complete solution

<!--
  Three crops × two formats, plus a universal JPEG fallback.
  Contiguous runs per media band; best compression first inside each run.
-->
<picture>

  <!-- ─── Band 1: viewport ≥ 1024px, 1600×900 landscape master crop ─── -->
  <source media="(min-width: 1024px)" type="image/avif"
          srcset="/img/hero-lg.avif" width="1600" height="900">
  <!--
    type="image/avif"
      The engine tests this against its decoder registry BEFORE fetching.
      An unsupported type costs nothing: no request is made, no error is
      logged, the engine moves to the next sibling in the same band.
    width/height on the <source>
      Per-candidate intrinsic size. Modern engines reserve the box from the
      SELECTED source's pair, so these must differ per band or the crop swap
      produces a shift even though every file loaded fine.
  -->
  <source media="(min-width: 1024px)" type="image/webp"
          srcset="/img/hero-lg.webp" width="1600" height="900">

  <!-- ─── Band 2: 640–1023px, 1024×576 landscape ─── -->
  <!--
    No max-width is needed: band 1 already claimed everything at 1024px and
    above, so this band's effective range is bounded from above by ordering.
  -->
  <source media="(min-width: 640px)" type="image/avif"
          srcset="/img/hero-md.avif" width="1024" height="576">
  <source media="(min-width: 640px)" type="image/webp"
          srcset="/img/hero-md.webp" width="1024" height="576">

  <!-- ─── Band 3: everything narrower, 480×640 portrait ─── -->
  <!--
    Omitting media entirely makes these two an unconditional match, which
    guarantees the ladder is total: no viewport can reach the <img> unless
    the engine ignores <source> altogether.
  -->
  <source type="image/avif"
          srcset="/img/hero-sm.avif" width="480" height="640">
  <source type="image/webp"
          srcset="/img/hero-sm.webp" width="480" height="640">

  <!--
    Terminal <img> — the element that paints, whichever source won.
      src           only ever requested by engines with no <picture> support
      alt           applies to the selected source, not to src specifically
      fetchpriority high on the single LCP candidate only; see the note below
  -->
  <img src="/img/hero-sm.jpg"
       alt="Harbour cranes silhouetted against a low winter sun"
       width="480" height="640"
       loading="eager" decoding="async" fetchpriority="high">
</picture>

If this hero is preloaded, the preload hint has to reproduce both axes or it will warm the wrong cell:

<!--
  A preload that names only the URL fetches that file unconditionally, so a
  phone would download the desktop AVIF in addition to the crop it actually
  renders. Replicating media AND type makes the preload select the same cell
  the <picture> will select.
    imagesrcset  the same value as the target <source>'s srcset
    media        the same condition; the preload is skipped when it fails
    type         skipped when the engine cannot decode the format, so a
                 Safari 14 client never preloads the AVIF at all
  One preload per band-and-format cell you want warmed. Two cells can be
  declared safely because their media conditions are mutually exclusive.
-->
<link rel="preload" as="image" fetchpriority="high"
      imagesrcset="/img/hero-lg.avif" media="(min-width: 1024px)" type="image/avif">
<link rel="preload" as="image" fetchpriority="high"
      imagesrcset="/img/hero-lg.webp" media="(min-width: 1024px)" type="image/webp">

Warning: never declare a preload for a cell and also leave fetchpriority="high" on the <img> for the same asset — that is two scheduler entries for one byte range, and the LCP image stalls behind its own preload. The mechanics are in preload vs prefetch for video and image assets and the priority side is covered in using fetchpriority to optimise critical media.

What the inverted nesting actually does

Grouping by format first looks tidier in a template — one loop over formats, one over breakpoints — and it produces a ladder whose first entries have no media attribute if you write the unconditional small crop into the AVIF group. The result is a source that matches every viewport on every AVIF-capable engine:

Correct Nesting Versus Format-First Nesting Two vertical ladders of six sources. On the left, sources are grouped by media band with formats inside each band, and a Chrome client at 390 pixels wide walks past four rows and stops on the small AVIF crop. On the right, sources are grouped by format with the unconditional small AVIF first, and the same client stops on row one, receiving the correct file only by accident; a second annotation shows the same ladder written with the large crop first, where the client stops immediately on the desktop AVIF. Media outer, type inner Type outer, media inner 1 · ≥1024 · avif · lg media fails 2 · ≥1024 · webp · lg media fails 3 · ≥640 · avif · md media fails 4 · ≥640 · webp · md media fails 5 · any · avif · sm selected · 32 KB 6 · any · webp · sm not reached 1 · ≥1024 · avif · lg media fails 2 · ≥640 · avif · md media fails 3 · any · avif · sm selected · 32 KB 4 · ≥1024 · webp · lg 5 · ≥640 · webp · md 6 · any · webp · sm Client under test: Chrome 120, viewport 390 × 852. Both ladders happen to yield hero-sm.avif. The right-hand ladder is still wrong: a Safari 14 client at 390 px fails rows 1–3 on type, fails rows 4 and 5 on media, and only lands on row 6 — one format miss carried it across all three crop bands. Move the unconditional AVIF to row 1 of that ladder and every AVIF-capable client, at every width, takes the small portrait crop — a silent, total loss of art direction with no console warning anywhere.

The failure is order-of-magnitude, not cosmetic: a desktop client rendering a 480 × 640 portrait crop into a 1600 px-wide hero slot upscales by 3.3× and the result is visibly soft, while the LCP element is now a 32 KB file that reports as “optimised” in every byte-weight audit you own. The only signal is img.currentSrc, which is why the verification below reads it rather than trusting the markup.

Collapsing the format axis

Six sources per image is fine for a handful of heroes and untenable for a catalogue. The format axis is the one you can delete, because format — unlike crop — can be negotiated on the wire: one URL, Vary: Accept, and the edge returns AVIF or WebP per request. The crop axis can never collapse this way, since the server has no idea how wide the viewport is.

Build Artifacts by Catalogue Size and Strategy A line chart with catalogue size from zero to five thousand source images on the horizontal axis and build artifacts produced on the vertical axis up to thirty five thousand. Three straight lines: the explicit three-crop two-format matrix produces seven files per image, an AVIF plus JPEG scheme produces four, and an edge transform scheme keeps one master per image. 0 10k 20k 30k 0 1,000 3,000 5,000 source images in the catalogue build artifacts produced 3 crops × 2 formats + JPEG — 7 files each 3 crops, AVIF + one JPEG — 4 files each edge transform — 1 master each

At 5,000 images the explicit matrix is 35,000 artifacts; on a Sharp pipeline averaging 380 ms per AVIF encode at effort: 6, the AVIF column alone is roughly 95 minutes of single-threaded CPU per full rebuild. Three ways out, in increasing order of commitment:

  1. Drop the WebP column. Keep AVIF sources and let the terminal <img> JPEG catch AVIF-blind engines. This trades ~35 % more bytes for the shrinking Safari 14/15 slice against a 43 % smaller build. Check your own analytics before assuming that slice is negligible; it is still meaningful on long-lived enterprise device fleets.
  2. Negotiate format at the edge, keep crops explicit. Three sources, each a transform URL with format=auto, no type attribute at all. The crop stays in the markup where it belongs and the format moves into Vary: Accept. Parameter semantics are in configuring Cloudflare Image Resizing URL parameters; the Fastly equivalent is normalising the Accept header in VCL, and the CloudFront cache-key side is the Vary: Accept cache policy.
  3. Hybrid. Explicit build-time matrix for the handful of LCP heroes, edge transforms for the rarely-requested tail. This is what most mature pipelines converge on, and it is worth the extra template branch.

Warning: options 2 and 3 require deleting the type attribute from the edge-negotiated sources. Leaving type="image/webp" on a URL that will return AVIF bytes puts two negotiation systems in disagreement; the engine believes it selected WebP, the edge returns AVIF, and while browsers sniff the actual bytes and usually render fine, any intermediary cache keyed on the declared type will serve the wrong thing.

Verification steps

1. Assert format completeness in the build

The completeness invariant is mechanical, so check it mechanically rather than in review:

// build/assert-matrix.js — fail the build if any media band is missing a format.
// Run against the manifest your Sharp step writes, before templates render.
const manifest = require('../dist/img/manifest.json');
const BANDS   = ['lg', 'md', 'sm'];            // one per media condition
const FORMATS = ['avif', 'webp'];              // every <source> format tier

const holes = [];
for (const image of Object.keys(manifest)) {
  for (const band of BANDS) {
    for (const fmt of FORMATS) {
      const key = `${image}-${band}.${fmt}`;
      // A missing entry means a browser in that band with no support for the
      // OTHER format falls through into the next band and gets the wrong crop.
      if (!manifest[image]?.[key]) holes.push(key);
    }
  }
  // The terminal <img> fallback is a separate requirement: exactly one JPEG.
  if (!manifest[image]?.[`${image}-sm.jpg`]) holes.push(`${image}-sm.jpg`);
}

if (holes.length) {
  console.error(`Incomplete art-direction matrix — ${holes.length} missing:`);
  holes.slice(0, 20).forEach(h => console.error('  ', h));
  process.exit(1);
}
console.log('Matrix complete for', Object.keys(manifest).length, 'images.');

2. Read currentSrc across engine and viewport

One assertion per matrix cell you care about, run in both a Chromium and a WebKit context:

// playwright.spec.js — one case per (engine, viewport) pair.
// npx playwright test --project=chromium --project=webkit
const { test, expect, devices } = require('@playwright/test');

const EXPECT = {
  // engine        viewport width  expected file
  chromium: [[1440, 'hero-lg.avif'], [800, 'hero-md.avif'], [390, 'hero-sm.avif']],
  webkit:   [[1440, 'hero-lg.webp'], [800, 'hero-md.webp'], [390, 'hero-sm.webp']],
};

test.describe('picture matrix', () => {
  for (const [engine, cases] of Object.entries(EXPECT)) {
    for (const [width, file] of cases) {
      test(`${engine} @ ${width}px → ${file}`, async ({ page, browserName }) => {
        test.skip(browserName !== engine);
        await page.setViewportSize({ width, height: 900 });
        await page.goto('http://localhost:8080/hero-demo/');
        // currentSrc is the resolved winner; src and srcset still hold every
        // candidate, so asserting on those proves nothing about selection.
        const src = await page.locator('picture img')
                              .evaluate(el => el.currentSrc);
        expect(src).toContain(file);
      });
    }
  }
});

The WebKit row is the one that matters. Playwright’s bundled WebKit supports AVIF, so it will not reproduce a Safari 14 client on its own — pin an older WebKit build or point the same expectations at a real device lane to exercise the fallback column honestly.

3. Confirm each cell is actually served as declared

# Every AVIF URL must come back as image/avif. A misconfigured origin that
# returns application/octet-stream still renders in Chrome (it sniffs), but
# breaks any cache or proxy keyed on Content-Type.
for f in hero-lg hero-md hero-sm; do
  printf '%-10s ' "$f"
  curl -sI "https://example.com/img/$f.avif" | awk -F': ' 'tolower($1)=="content-type"{print $2}'
done
# Expected: three lines of image/avif

# For edge-negotiated URLs (option 2), confirm the response varies correctly.
# Send an Accept header without image/avif and expect WebP or JPEG back.
curl -sI "https://example.com/img/hero-lg?w=1600" \
  -H 'Accept: image/webp,image/png,image/*;q=0.8,*/*;q=0.5' \
  | grep -iE 'content-type|vary'
# Expected: content-type: image/webp  AND  vary: Accept
# A missing Vary line means the first format cached is served to everyone.

Common mistakes

1. An AVIF source with no WebP sibling in the same band

<!-- WRONG: Safari 14 at 800px skips the AVIF, finds no WebP for this band,
     and falls through to the small crop meant for phones. -->
<source media="(min-width: 640px)" type="image/avif" srcset="/img/hero-md.avif">
<source type="image/webp" srcset="/img/hero-sm.webp">

<!-- CORRECT: every band carries every format tier. -->
<source media="(min-width: 640px)" type="image/avif" srcset="/img/hero-md.avif">
<source media="(min-width: 640px)" type="image/webp" srcset="/img/hero-md.webp">
<source type="image/webp" srcset="/img/hero-sm.webp">

2. Putting the unconditional format source first

An entry with a type but no media matches at every viewport width. Placed above the media-bearing sources it wins universally on any engine that supports its format, and art direction is silently gone. Order is always widest band first, unconditional band last.

3. Preloading the URL instead of the cell

<!-- WRONG: fetched on every client at every width, in addition to whatever
     the <picture> actually selects. Phones pay for a 68 KB desktop AVIF. -->
<link rel="preload" as="image" href="/img/hero-lg.avif">

<!-- CORRECT: the preload is skipped unless this exact cell would win. -->
<link rel="preload" as="image" imagesrcset="/img/hero-lg.avif"
      media="(min-width: 1024px)" type="image/avif">

4. Reusing one width/height pair across bands with different aspect ratios

The reserved box comes from the selected candidate. If band 3 is a 480 × 640 portrait crop but every <source> repeats width="1600" height="900", the browser reserves a 16:9 box and then paints a 3:4 image into it — a guaranteed shift on load and a distorted composition after. Keep the pair on each source truthful to its file, and mirror the same ratios in CSS as described in the parent guide.

5. Encoding all three crops at one AVIF quality

A tighter crop enlarges every artifact along with the subject, so the small portrait derivative needs a higher quality setting than the large landscape one to look equivalent — typically quality: 78 against quality: 75 in Sharp, which costs about 4 KB and removes visible blocking in flat sky regions. The comparative data behind those numbers is in AVIF vs WebP compression benchmarks, and the framework-side wiring for per-variant quality is shown in the Astro Picture component for AVIF and WebP and Vite imagetools responsive srcset generation.