Sharp and FFmpeg Media Build Pipelines

Every framework integration in Framework & Build-Tool Media Integration eventually bottoms out in the same two binaries: libvips (wrapped by Sharp) for stills and FFmpeg for moving pictures. next/image, astro:assets and @nuxt/image hide them behind a component API; the moment you outgrow that API — a video poster ladder, an AV1 fallback, an OG-card generator, a 12 000-image product catalogue — you end up writing the pipeline yourself. This guide is about that pipeline: how to build one that is deterministic (the same input always produces the same output path), incremental (a CI run that changes nothing encodes nothing), and bounded (it does not OOM a 4 vCPU runner halfway through a 300-derivative batch).

The design goal is narrow and worth stating up front: a media build stage should cost near-zero wall clock on the 95% of commits that do not touch media. Everything below — content-addressed keys, encoder-version salting, worker pools, restore-key layering in GitHub Actions — exists to serve that one property. A pipeline that re-encodes 288 AVIFs on every push is not a pipeline, it is a tax.

Concept & Architecture: The Derivative Graph

A build-time media pipeline is a pure function from a set of masters (the original, highest-quality files you commit or pull from a DAM) to a set of derivatives (every width × format × codec combination your markup references), plus a manifest that maps master → derivatives so the templating layer can emit srcset without guessing.

The pipeline runs in four phases, and keeping them separate is what makes incrementality possible:

1. Enumerate. Walk the source directory, read each master’s bytes, and expand the declarative recipe set into a flat list of intended outputs. Nothing is encoded yet — this phase is pure metadata and typically finishes in under 200 ms for a few thousand files.
2. Key. Compute a content-addressed key for every intended output. The key must cover the source bytes and every input that can change the output bytes: the recipe parameters, the encoder version, and the underlying library ABI version.
3. Materialize. For each key that is absent from the derivative cache, run the encode. Sharp jobs go to a bounded worker pool; FFmpeg jobs go to a separate, much smaller process pool because a single FFmpeg invocation already saturates multiple cores.
4. Emit. Copy or hard-link cache entries into the build output directory and write a manifest.json that the framework’s templates import.

Sharp’s execution model matters here. Sharp is a thin binding over libvips, and libvips is demand-driven and streaming: it builds a pipeline of operations and pulls scanlines through it, so a 4000×3000 JPEG resize does not necessarily materialize a 48 MB RGBA buffer. Each sharp() call runs on the libuv thread pool, and inside that call libvips spawns its own threads (VIPS_CONCURRENCY, defaulting to the core count) to process regions in parallel. That gives you two independent concurrency dials that multiply: N concurrent Sharp pipelines × M libvips threads each. Leave both at their defaults on a 4 vCPU runner and you get 4 × 4 = 16 threads fighting over 4 cores, which is measurably slower than either dial alone and roughly doubles peak RSS.

FFmpeg is the opposite shape. It is an out-of-process binary with its own internal threading (-threads, plus codec-specific tiling and row-multithreading), and it holds decoded frames in memory. You do not pool FFmpeg the way you pool Sharp — you run one or two processes at a time and let each one use the machine.

Deterministic Output Hashing

“Content-addressed” means the output filename is a hash of everything that determines its bytes. This is what lets you serve derivatives with Cache-Control: public, max-age=31536000, immutable — the max-age policy for CDN media assets only holds if a changed image is guaranteed to change its URL.

The mistake almost everyone makes on the first attempt is hashing too little. Hashing only the source bytes means bumping sharp from 0.33 to 0.34 — which brings a new libaom and produces different AVIF bytes at the same nominal quality — silently reuses stale derivatives forever. Hashing only the recipe means editing the master leaves the old file in place. The key must be a hash over the full determinant set:

Two subtleties are worth calling out. First, hash the source bytes, never mtime or file size — a git clone in CI rewrites every mtime, so an mtime-keyed cache has a 0% hit rate on every fresh runner. Second, serialize the options object with sorted keys before hashing; JSON.stringify preserves insertion order, so {quality: 50, effort: 4} and {effort: 4, quality: 50} would otherwise produce two keys for one output and double your cache.

Concurrency and libvips Tuning

Sharp exposes three tuning surfaces, and they interact:

• sharp.concurrency(n) sets the number of threads libvips uses within a single pipeline. It defaults to the number of physical cores detected (or the value of VIPS_CONCURRENCY).
• The libuv thread pool (UV_THREADPOOL_SIZE, default 4) bounds how many Sharp operations can actually run in parallel, regardless of how many promises you create.
• sharp.cache({ memory, files, items }) controls libvips’ operation cache, which memoizes intermediate results across calls. It defaults to 50 MB / 20 files / 100 items and is a pure liability in a batch job where every input is different.

The rule that holds across almost every batch workload: set sharp.concurrency(1) and parallelize at the task level. Per-image libvips threading exists to make a single interactive resize fast; in a batch you already have hundreds of independent units of work, and one thread per image avoids the synchronization overhead and the RSS multiplication. The exception is a job with very few, very large masters (panoramas, scanned TIFFs) where task-level parallelism cannot fill the cores.

Benchmark Data: Encode Cost per Derivative

All figures below are medians over 50 runs on a GitHub Actions ubuntu-latest runner (4 vCPU, 16 GB RAM), Sharp 0.34.2 / libvips 8.16.0, encoding a single 4032×3024 JPEG master down to a 1600px-wide derivative. “Peak RSS” is the delta above the idle Node process, measured per worker.

The shape of that curve is the single most important number in the whole pipeline. Going from effort: 4 to effort: 9 costs 11.7× the CPU time to save 8% of the bytes. For a build-time pipeline the sweet spot is effort: 4; reserve effort: 7+ for a nightly job that re-encodes only the hero images. The equivalent quality-vs-size analysis across formats lives in AVIF vs WebP compression benchmarks.

Video is a different order of magnitude. Encoding a 12-second 1920×1080 30 fps clip on the same runner:

A single AV1 encode peaks above 1 GB. Two in parallel on a 16 GB runner is fine; eight is not, and the OOM killer terminates the Node parent, not the FFmpeg child, so the failure surfaces as an unhelpful SIGKILL on your build script. The codec tradeoffs behind those numbers are covered in understanding video codecs: VP9 vs H.265 vs AV1, and the two-pass VP9 rate-control setup gets its own walkthrough in FFmpeg two-pass VP9 encoding for the web.

Step-by-Step Implementation

The six steps below build a complete, dependency-light pipeline. Each block is runnable as written; the only npm dependencies are sharp and (optionally) fast-glob.

Step 1 — Declare the Recipes

Keep the derivative matrix in one declarative module. Everything downstream — hashing, dispatch, manifest emission — reads from this file, so a change here is a change to every key.

// media.recipes.mjs — the single source of truth for the derivative matrix.
export const IMAGE_WIDTHS = [400, 800, 1200, 1600, 2400];

export const IMAGE_RECIPES = [
  {
    id: 'avif',
    ext: 'avif',
    mime: 'image/avif',
    // effort 4 is the knee of the cost/size curve: 781 ms for 74 KB.
    // effort 9 costs 11.7x the CPU for 8% fewer bytes — not worth it in CI.
    encode: { quality: 50, effort: 4, chromaSubsampling: '4:2:0' }
  },
  {
    id: 'webp',
    ext: 'webp',
    mime: 'image/webp',
    encode: { quality: 75, effort: 4, smartSubsample: true }
  },
  {
    id: 'jpeg',                       // universal fallback for the <img src>
    ext: 'jpg',
    mime: 'image/jpeg',
    encode: { quality: 80, mozjpeg: true, progressive: true }
  }
];

export const VIDEO_RECIPES = [
  {
    id: 'vp9',
    ext: 'webm',
    mime: 'video/webm',
    // -row-mt 1 enables row-based multithreading; without it libvpx-vp9
    // uses at most one thread per tile column and wastes 3 of 4 cores.
    args: ['-c:v', 'libvpx-vp9', '-crf', '32', '-b:v', '0', '-row-mt', '1',
           '-c:a', 'libopus', '-b:a', '96k']
  },
  {
    id: 'h264',
    ext: 'mp4',
    mime: 'video/mp4',
    // +faststart relocates the moov atom to the head so the browser can
    // start playback before the whole file has arrived.
    args: ['-c:v', 'libx264', '-preset', 'medium', '-crf', '23',
           '-pix_fmt', 'yuv420p', '-c:a', 'aac', '-b:a', '128k',
           '-movflags', '+faststart']
  }
];

// Poster frame extracted at 1s; used as the <video poster> and the LCP candidate.
export const POSTER_RECIPE = { id: 'poster', ext: 'jpg', seek: '00:00:01.000' };

Step 2 — Build the Content-Addressed Key

This module is small and pure, which makes it trivially unit-testable — and you should test it, because a silent key collision or a non-deterministic serialization is the hardest class of bug in a build pipeline.

// scripts/media/key.mjs
import { createHash } from 'node:crypto';
import { createReadStream } from 'node:fs';
import sharp from 'sharp';

// Stream the digest so a 200 MB ProRes master never lands in RAM.
export function digestFile(path) {
  return new Promise((resolve, reject) => {
    const h = createHash('sha256');
    createReadStream(path)
      .on('data', (chunk) => h.update(chunk))
      .on('error', reject)
      .on('end', () => resolve(h.digest('hex')));
  });
}

// Deterministic JSON: sort keys recursively so {a,b} and {b,a} hash identically.
function canonical(value) {
  if (Array.isArray(value)) return value.map(canonical);
  if (value && typeof value === 'object') {
    return Object.fromEntries(
      Object.keys(value).sort().map((k) => [k, canonical(value[k])])
    );
  }
  return value;
}

// Salt every image key with the encoder identity. A sharp minor bump can pull
// in a new libaom and change AVIF bytes at identical nominal quality.
const IMAGE_SALT = [
  sharp.versions.sharp,      // e.g. "0.34.2"
  sharp.versions.vips,       // e.g. "8.16.0"
  sharp.versions.heif ?? '-' // libheif, present on most prebuilt binaries
].join('/');

export function derivativeKey({ sourceDigest, recipeId, width, options, salt = IMAGE_SALT }) {
  const material = JSON.stringify(canonical({
    sourceDigest, recipeId, width, options, salt
  }));
  // 16 hex chars = 64 bits. Collision probability across 10^6 derivatives
  // is ~2.7e-8 — orders of magnitude below your disk's silent-corruption rate.
  return createHash('sha256').update(material).digest('hex').slice(0, 16);
}

export function cachePath(root, key, ext) {
  // Two-level shard: 256 top-level dirs keeps any single directory
  // under a few thousand entries, which matters on ext4 and on tar restore.
  return `${root}/${key.slice(0, 2)}/${key.slice(2)}.${ext}`;
}

Warning: sharp.versions.heif is undefined on custom libvips builds compiled without libheif. The ?? '-' fallback keeps the key stable rather than serializing undefined (which JSON.stringify silently drops, changing the key shape).

Step 3 — Encode Images Through a Bounded Worker Pool

The pool below is deliberately hand-rolled — a promise queue with a fixed width — because pulling in a concurrency library for twenty lines of code is how a build script accumulates a supply chain.

// scripts/media/images.mjs
import { mkdir, rename, writeFile, access } from 'node:fs/promises';
import { dirname } from 'node:path';
import os from 'node:os';
import sharp from 'sharp';
import { digestFile, derivativeKey, cachePath } from './key.mjs';
import { IMAGE_WIDTHS, IMAGE_RECIPES } from '../../media.recipes.mjs';

const CACHE_ROOT = process.env.MEDIA_CACHE ?? '.cache/media';

// One libvips thread per pipeline: task-level parallelism already fills the
// cores, and per-op threading multiplies peak RSS without improving throughput.
sharp.concurrency(1);
// Disable the libvips operation cache. Every input in a batch is distinct,
// so the cache is pure resident memory with a 0% hit rate.
sharp.cache(false);

const POOL = Number(process.env.MEDIA_CONCURRENCY ?? Math.max(1, os.availableParallelism() - 1));

const exists = (p) => access(p).then(() => true, () => false);

async function encodeOne(src, recipe, width, key) {
  const out = cachePath(CACHE_ROOT, key, recipe.ext);
  if (await exists(out)) return { key, out, cached: true };

  await mkdir(dirname(out), { recursive: true });
  const tmp = `${out}.${process.pid}.tmp`;

  const meta = await sharp(src).metadata();
  await sharp(src, {
    // Stream the source top-to-bottom instead of random-accessing it.
    // Cuts peak RSS on large progressive JPEGs by roughly half.
    sequentialRead: true,
    limitInputPixels: 268402689 // 16384^2 — reject decompression bombs
  })
    .rotate()                                  // bake in EXIF orientation
    .resize({ width, withoutEnlargement: true }) // never upscale past the master
    .toFormat(recipe.ext === 'jpg' ? 'jpeg' : recipe.ext, recipe.encode)
    .toFile(tmp);

  // Rename is atomic within a filesystem: a killed build never leaves a
  // truncated file that the next run would treat as a valid cache hit.
  await rename(tmp, out);
  return { key, out, cached: false, sourceWidth: meta.width };
}

export async function buildImages(sources) {
  const jobs = [];
  for (const src of sources) {
    const sourceDigest = await digestFile(src);
    const { width: srcWidth } = await sharp(src).metadata();
    for (const recipe of IMAGE_RECIPES) {
      for (const width of IMAGE_WIDTHS) {
        if (width > srcWidth) continue;         // skip pointless upscales
        const key = derivativeKey({
          sourceDigest, recipeId: recipe.id, width, options: recipe.encode
        });
        jobs.push({ src, recipe, width, key });
      }
    }
  }

  const results = [];
  let cursor = 0;
  const workers = Array.from({ length: POOL }, async () => {
    while (cursor < jobs.length) {
      const job = jobs[cursor++];               // single-threaded JS: safe
      results.push(await encodeOne(job.src, job.recipe, job.width, job.key));
    }
  });
  await Promise.all(workers);

  const reused = results.filter((r) => r.cached).length;
  console.log(`[media] ${results.length} derivatives, ${reused} reused, ${results.length - reused} encoded`);
  await writeFile('media.manifest.json', JSON.stringify(results, null, 2));
  return results;
}

Tradeoff: os.availableParallelism() - 1 leaves one core for the rest of the build (bundler, type checker) when the media stage runs concurrently with them. If the media stage is a dedicated CI job, drop the - 1. Do not exceed the core count: AVIF at 310 MB per worker means 8 workers is 2.5 GB of resident memory, which is survivable on a 16 GB runner and fatal on a 2 GB container.

Step 4 — Add the FFmpeg Stage

FFmpeg jobs are keyed the same way, but the encoder salt has to come from the binary rather than from an npm package, and the pool width is 2 instead of cores - 1.

// scripts/media/video.mjs
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
import { mkdir, rename, access } from 'node:fs/promises';
import { dirname } from 'node:path';
import { digestFile, derivativeKey, cachePath } from './key.mjs';
import { VIDEO_RECIPES, POSTER_RECIPE } from '../../media.recipes.mjs';

const run = promisify(execFile);
const CACHE_ROOT = process.env.MEDIA_CACHE ?? '.cache/media';
// FFmpeg saturates the machine on its own. Two concurrent processes keeps the
// cores busy during each encoder's single-threaded muxing tail without
// stacking two 1.2 GB AV1 working sets on top of each other.
const VIDEO_POOL = 2;

const exists = (p) => access(p).then(() => true, () => false);

// Salt video keys with the FFmpeg build string: an encoder upgrade
// (libvpx 1.13 -> 1.14) changes output bytes at identical flags.
async function ffmpegSalt() {
  const { stdout } = await run('ffmpeg', ['-version']);
  return stdout.split('\n')[0].trim(); // "ffmpeg version 7.1 Copyright ..."
}

async function encodeVideo(src, recipe, key, salt) {
  const out = cachePath(CACHE_ROOT, key, recipe.ext);
  if (await exists(out)) return { key, out, cached: true };
  await mkdir(dirname(out), { recursive: true });
  const tmp = `${out}.tmp.${recipe.ext}`;

  await run('ffmpeg', [
    '-nostdin',            // never block waiting on a TTY inside CI
    '-hide_banner',
    '-loglevel', 'error',  // keep CI logs readable; errors still surface
    '-y',                  // overwrite the temp file from a previous crash
    '-i', src,
    ...recipe.args,
    tmp
  ], { maxBuffer: 8 * 1024 * 1024 });

  await rename(tmp, out);
  return { key, out, cached: false };
}

async function encodePoster(src, key) {
  const out = cachePath(CACHE_ROOT, key, POSTER_RECIPE.ext);
  if (await exists(out)) return { key, out, cached: true };
  await mkdir(dirname(out), { recursive: true });
  const tmp = `${out}.tmp.jpg`;
  await run('ffmpeg', [
    '-nostdin', '-hide_banner', '-loglevel', 'error', '-y',
    '-ss', POSTER_RECIPE.seek,  // BEFORE -i: seeks by keyframe, ~100x faster
    '-i', src,
    '-frames:v', '1',
    '-q:v', '3',                // mjpeg quality scale, 2-5 is the useful band
    tmp
  ]);
  await rename(tmp, out);
  return { key, out, cached: false };
}

export async function buildVideos(sources) {
  const salt = await ffmpegSalt();
  const jobs = [];
  for (const src of sources) {
    const sourceDigest = await digestFile(src);
    for (const recipe of VIDEO_RECIPES) {
      jobs.push({
        src, recipe,
        key: derivativeKey({ sourceDigest, recipeId: recipe.id, width: 0, options: recipe.args, salt })
      });
    }
    jobs.push({
      src, recipe: null,
      key: derivativeKey({ sourceDigest, recipeId: 'poster', width: 0, options: POSTER_RECIPE, salt })
    });
  }

  const results = [];
  let cursor = 0;
  await Promise.all(Array.from({ length: VIDEO_POOL }, async () => {
    while (cursor < jobs.length) {
      const j = jobs[cursor++];
      results.push(j.recipe
        ? await encodeVideo(j.src, j.recipe, j.key, salt)
        : await encodePoster(j.src, j.key));
    }
  }));
  return results;
}

Warning: put -ss before -i for poster extraction. After -i it decodes every frame from the start of the file to the seek point; before -i it seeks to the nearest keyframe first. On a 40-minute source that is the difference between 90 seconds and 0.4 seconds.

Step 5 — Wire It Into npm Scripts

The media stage should be an explicit, cacheable step that runs before the framework build, so the manifest exists when templates are rendered.

{
  "scripts": {
    "media": "node scripts/media/build.mjs",
    "media:clean": "node scripts/media/gc.mjs",
    "media:verify": "node scripts/media/build.mjs --check",
    "prebuild": "npm run media",
    "build": "vite build",
    "dev": "npm run media && vite"
  },
  "devDependencies": {
    "sharp": "0.34.2",
    "fast-glob": "^3.3.2"
  }
}

Pin sharp to an exact version rather than a caret range. A floating minor changes the encoder salt, invalidates every key, and turns a routine dependency update into a 400-second CI run that nobody expected — and, worse, makes two developers on slightly different lockfile states produce different derivative URLs for identical source images.

Add a --check mode to build.mjs that plans the work and exits non-zero if anything would be encoded. Running it in a verify job proves the committed manifest matches the sources — the same class of guard as a lockfile --frozen check.

Step 6 — Cache Derivatives Across GitHub Actions Runs

This is where the design pays off. The cache directory is content-addressed, so it is append-only and merge-safe: two branches can each add entries and neither invalidates the other.

name: build
on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '22'
          cache: npm

      # ubuntu-latest images ship ffmpeg, but not always with libsvtav1.
      # Install explicitly so the encoder salt is reproducible run to run.
      - name: Install FFmpeg
        run: |
          sudo apt-get update -qq
          sudo apt-get install -y --no-install-recommends ffmpeg
          ffmpeg -version | head -1

      # The primary key rolls with the run number so every run writes a fresh
      # entry (Actions caches are immutable once created). restore-keys then
      # falls back to the newest matching prefix, which accumulates entries
      # across runs instead of starting cold on every commit.
      - name: Restore derivative cache
        uses: actions/cache@v4
        with:
          path: .cache/media
          key: media-v1-${{ runner.os }}-${{ hashFiles('media.recipes.mjs', 'package-lock.json') }}-${{ github.run_id }}
          restore-keys: |
            media-v1-${{ runner.os }}-${{ hashFiles('media.recipes.mjs', 'package-lock.json') }}-
            media-v1-${{ runner.os }}-

      - run: npm ci

      - name: Build media derivatives
        run: npm run media
        env:
          MEDIA_CONCURRENCY: '3'   # 4 vCPU runner, leave one core for the OS
          UV_THREADPOOL_SIZE: '4'  # must be >= MEDIA_CONCURRENCY or Sharp queues

      # Drop cache entries no longer referenced by the manifest. Without this
      # the cache grows monotonically and hits the 10 GB per-repository limit,
      # at which point Actions starts evicting your *useful* entries LRU-first.
      - name: Garbage-collect unreferenced derivatives
        run: npm run media:clean

      - run: npm run build

Warning: GitHub Actions cache scoping is not flat. A cache written on a feature branch is readable by that branch and its child branches only; a cache written on the default branch is readable by every branch. If your derivative cache is only ever written by PR builds, each new PR starts cold. Ensure the workflow runs on pushes to main too, so the default-branch cache stays warm and every PR inherits it. The full set of gotchas — cross-OS keys, the 10 GB ceiling, the 7-day idle eviction — is covered in caching image derivatives in GitHub Actions.

With the cache in place, the media stage collapses from a build-dominating step to a rounding error on the commits that do not touch media:

The 23-second floor in run 2 is not encoding at all — it is the actions/cache tarball restore of a 251 MB directory plus the digest pass over the masters. That floor is why the garbage-collection step matters: cache restore time scales with cache size, so an ungoverned cache eventually costs more to restore than the encodes it saves.

Parameter Reference

Build-Time vs On-the-Fly Edge Transforms

Build-time encoding is not universally correct. The decision hinges on whether the set of required derivatives is knowable and bounded before deploy.

Build-time wins on four axes. Cost is fixed and paid once, not per cache miss. Output is byte-identical across deploys, so CDN caches survive a redeploy. There is no cold-start tail — the p99 for a build-time derivative is the CDN’s p99, not an encoder’s. And you can afford settings an edge function cannot: an 800 ms effort: 4 AVIF encode is unremarkable in CI and unacceptable inside a request.

Edge transforms win on two: unbounded and unknowable input sets, and derivative matrices too sparse to pre-generate. If you have 400 000 catalogue images and each page view touches 30 of them, pre-generating 5 widths × 3 formats means 6 million encodes to serve a rarely-requested tail that may never be requested. That is exactly the case for Cloudflare Image Resizing and Polish or an origin-response Lambda@Edge running Sharp.

Most real sites land in the hybrid: build-time for the marketing surface (heroes, OG cards, anything that is an LCP candidate) and edge transforms for the user-generated tail. The hybrid’s one hard requirement is that both paths agree on encode settings, or the same photograph looks different depending on which route served it.

Tradeoffs & Edge Cases

Tradeoff: an exact sharp pin trades security latency for cache stability. Pinning sharp to 0.34.2 keeps every derivative key stable, but it also means you are not picking up libvips security fixes automatically. The mitigation is a scheduled Dependabot PR for sharp specifically, reviewed and merged deliberately, with the expectation that the merge commit’s CI run is a full cold re-encode. Budget for it: it is one 400-second run per quarter, not per push.

Tradeoff: hard links make the emit phase free but couple the cache to the output. Hard-linking from .cache/media into dist/ avoids copying 250 MB, but if anything in the deploy pipeline rewrites a file in place (a metadata stripper, an over-eager optimizer) it corrupts the cache entry too, and the corruption is invisible because the key still matches. Either copy, or make the cache directory read-only after the build phase.

Warning: parallel FFmpeg jobs can OOM the parent, not the child. When the Linux OOM killer fires, it scores processes by memory and often selects the Node parent that spawned the encoders rather than the encoder itself. The symptom is a build that dies with SIGKILL and no FFmpeg error, which sends people hunting in the wrong place. Cap VIDEO_POOL at 2 on a 16 GB runner, and check dmesg | grep -i oom before assuming your script is buggy.

Edge case: withoutEnlargement silently changes the derivative set. Skipping widths larger than the master is correct — upscaling adds bytes and no detail — but it means a 900px source produces only the 400 and 800 entries. Templates that assume all five widths exist will emit srcset candidates pointing at files that were never generated. Read the widths back from the manifest rather than from the recipe constant when building srcset and sizes attributes.

Edge case: EXIF orientation must be applied before resize, not after. sharp(src).rotate() with no argument reads the EXIF Orientation tag and bakes the rotation into pixels; calling .resize() first computes dimensions against the un-rotated frame, so a portrait photo shot on a phone comes out cropped to landscape proportions. Sharp strips EXIF by default on output, so the orientation tag is gone and the browser cannot correct it either.

Edge case: two derivatives with identical bytes share one key only if their recipes match. A 1200px AVIF at quality: 50 from source A and the same from source B will have different keys even if the encoded bytes happen to be identical. That is intentional — deduplicating by output digest would require encoding first, defeating the entire purpose of the cache — but it means the cache is slightly larger than the theoretical minimum. Do not “optimize” this.

Debugging & Validation

Confirm What the Encoders Actually Are

Key stability depends on the encoder identity, so print it before trusting a cache hit rate:

# Sharp's view of its own stack. If `vips` or `heif` differs between your
# laptop and CI, the cache will never share entries between them.
node -e "const s=require('sharp');console.log(s.versions);console.log('simd',s.simd())"

# Confirm libvips was built with the codecs you rely on.
# Missing "heif" here means AVIF encode throws at runtime, not at install.
vips --vips-config | tr ',' '\n' | grep -Ei 'heif|jpeg|webp|jxl'

# Confirm FFmpeg has the encoders your recipes name.
ffmpeg -hide_banner -encoders | grep -E 'libvpx-vp9|libsvtav1|libx264|libopus'

Prove the Cache Is Actually Being Hit

The most common failure is a cache that appears to work locally and has a 0% hit rate in CI. Measure it directly rather than inferring it from build times:

# Run twice back to back. The second run must report 0 encoded.
npm run media && npm run media

# Count entries and total size — this is what actions/cache tars and restores.
find .cache/media -type f | wc -l
du -sh .cache/media

# Byte-for-byte determinism check: wipe, rebuild, and compare digests.
# Any difference means something non-deterministic leaked into an encode.
find .cache/media -type f -exec sha256sum {} + | sort -k2 > /tmp/before.txt
rm -rf .cache/media && npm run media
find .cache/media -type f -exec sha256sum {} + | sort -k2 > /tmp/after.txt
diff /tmp/before.txt /tmp/after.txt && echo "deterministic"

Watch Memory Under Load

If the build dies without a useful error, the cause is almost always resident memory:

# Peak RSS of the whole media stage, in kilobytes (GNU time, not the shell builtin).
/usr/bin/time -v npm run media 2>&1 | grep -E 'Maximum resident|Elapsed'

# Live per-worker view while a batch is running.
watch -n 1 "ps -o pid,rss,comm -C node -C ffmpeg --sort=-rss | head -12"

# After an unexplained SIGKILL, check whether the kernel did it.
dmesg -T | grep -i -E 'out of memory|killed process' | tail -5

Validate the Emitted Derivatives

Encoding successfully is not the same as encoding correctly. Verify the output rather than the exit code:

# Confirm real dimensions and format of every generated image derivative.
find dist/media -name '*.avif' -print0 \
  | xargs -0 -n1 -P4 sh -c 'vipsheader -a "$0" | grep -E "^(width|height|vips-loader)"'

# Confirm the mp4 moov atom is at the head (faststart applied).
# "moov" must appear before "mdat" in this listing.
ffprobe -v error -show_entries format=format_name -of default=nw=1 dist/media/clip.mp4
python3 -c "d=open('dist/media/clip.mp4','rb').read(4096); print('faststart' if d.find(b'moov') < d.find(b'mdat') else 'moov at tail')"

# Regression guard: fail the build if total image weight grows past budget.
find dist/media -name '*.avif' -printf '%s\n' | awk '{s+=$1} END {print s/1024/1024" MB AVIF"}'

That last check pairs naturally with a page-level budget; wiring it into Lighthouse CI budget enforcement for image weight turns “the encoder settings regressed” into a failing check instead of a slow discovery in field data. For the image half of the pipeline specifically, batch AVIF conversion with Sharp in CI drills into the pool sizing, retry, and progress-reporting details that keep a 5 000-image batch observable while it runs.

Related

• Batch AVIF Conversion with Sharp in CI — pool sizing, memory ceilings, and retry strategy for large image batches
• FFmpeg Two-Pass VP9 Encoding for the Web — rate control, -row-mt, and the exact two-pass flag set for WebM output
• Caching Image Derivatives in GitHub Actions — cache key layering, branch scoping, and garbage collection under the 10 GB limit
• Nuxt and Vite Image Asset Pipeline — the same Sharp work driven declaratively from import queries instead of a custom script
• Astro Image and Picture Components — how a static-first framework bakes Sharp output into the build
• Cloudflare Image Resizing and Polish — the on-the-fly counterpart for asset sets you cannot enumerate at build time
• Framework & Build-Tool Media Integration — parent section covering media handling across every major framework and bundler