Caching Image Derivatives in GitHub Actions

A sharp batch that produces 288 AVIF, WebP and JPEG derivatives costs roughly seven minutes of wall clock on a four-vCPU runner, and almost none of that work changes between commits. GitHub Actions gives you a cache with three properties that make naive reuse fail: keys are immutable once written, entries are scoped to the branch that wrote them, and the repository quota is a hard 10 GB with LRU eviction. This page is the cache-key design, workflow wiring and eviction policy that make an image derivative cache survive all three — it assumes you already have the content-addressed encoder described in Sharp and FFmpeg media build pipelines and are wiring it into CI under Framework & Build Tool Media Integration.

Prerequisite checklist

The key rule: identity in the key, freshness in the filenames

Nearly every broken derivative cache makes the same mistake — hashing the source images into the cache key:

key: media-${{ hashFiles('assets/**/*.{jpg,png,tif}') }}   # WRONG

That key changes the moment anyone touches one photograph, which produces a total miss and re-encodes all 288 derivatives to rebuild an entry that differs from the previous one by three files. The content-addressed directory already tracks freshness at per-derivative granularity: a file named after the digest of its own inputs is either present and correct, or absent. So the cache key does not need to encode what is inside the directory. It needs to encode only which directory contents are legal to reuse at all — that is, the toolchain and recipe identity:

  • runner.os and runner.arch — a libvips build on linux/arm64 produces different AVIF bytes from linux/x64 at identical settings.
  • A hash of media.recipes.mjs and package-lock.json — quality, effort, chroma subsampling and the pinned sharp/libvips versions. These are exactly the knobs whose cost/size curves are mapped in the AVIF vs WebP compression benchmarks, and changing any of them legitimately means every existing derivative is a different image.
  • A manual epoch (media-v3-) you bump by hand when you want to abandon every existing entry.

Everything else — which images exist, which sizes were requested — belongs in the filenames. That is also what makes the emitted URLs safe to serve with a one-year immutable policy, per best practices for setting max-age on CDN media assets: the CI cache and the CDN cache are keyed on the same digest, so a rebuilt derivative can never collide with a stale edge copy.

Because Actions caches are immutable, the primary key must also be unique per run, or the save step is a no-op and the cache never grows past its first write. The idiomatic fix is a run-scoped suffix on the primary key plus a restore-keys prefix ladder that finds the newest compatible entry. The ladder resolves top-down and stops at the first prefix match:

Restore-key resolution ladder for a derivative cache A four-rung ladder. The exact key carries the run id and always misses because it is unique per run. The first restore key matches the newest cache built with the same recipes and lockfile. The second restore key matches any cache for the same operating system and architecture. If nothing matches, the run encodes from cold. How actions/cache resolves a derivative cache lookup Evaluated top to bottom; the first prefix match wins and the rest are skipped. key (exact) media-v3-Linux-X64-a91f3c-18422907631 always misses run id is new every run restore-key 1 media-v3-Linux-X64-a91f3c- newest identical toolchain 288 of 288 derivatives reused restore-key 2 media-v3-Linux-X64- recipes changed old names still valid, partial reuse no match first run on a new epoch or a new runner arch cold encode 412 s on a 4 vCPU runner Never put source-image hashes in any rung — one edited photo would drop the whole ladder to the bottom rung.

Rung three is the one people delete because it looks unsafe. It is not: a recipe change alters the encoder salt, which alters every derivative filename, so restoring an older directory brings back files that the new run will simply ignore and never read. You pay a slightly larger tarball restore in exchange for keeping unrelated derivatives — video posters, untouched sections of the library — warm across a quality tweak. The garbage-collection step below is what stops those ignored files accumulating forever.

The workflow

This uses the restore/save split rather than the combined actions/cache action, because the combined action’s post-job save fires on every run regardless of whether anything changed — which is exactly the behaviour that burns the quota.

name: build
on:
  push:
    branches: [main]        # keeps the default-branch cache warm — see mistake 3
  pull_request:

# Two runs on the same ref would race to reserve the same cache key; the loser
# logs "Unable to reserve cache" and silently discards its work.
concurrency:
  group: build-${{ github.ref }}
  cancel-in-progress: true

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '22'
          cache: npm        # NOTE: shares the same 10 GB repo quota as the media cache

      - name: Install FFmpeg
        run: sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends ffmpeg

      # Identity, not freshness. runner.arch matters because libvips emits
      # different AVIF bytes on arm64; the epoch (v3) is the manual kill switch.
      - name: Compute cache key prefix
        id: k
        run: |
          echo "prefix=media-v3-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('media.recipes.mjs','package-lock.json') }}" >> "$GITHUB_OUTPUT"

      - name: Restore derivative cache
        id: restore
        uses: actions/cache/restore@v4
        with:
          path: .cache/media
          # Unique per run: Actions caches are immutable, so a stable primary
          # key would hit on every run and make the save step a permanent no-op.
          key: ${{ steps.k.outputs.prefix }}-${{ github.run_id }}
          restore-keys: |
            ${{ steps.k.outputs.prefix }}-
            media-v3-${{ runner.os }}-${{ runner.arch }}-
          # Do not repack x64 caches for arm64 consumers; the bytes are not
          # interchangeable and a cross-OS restore would poison the directory.
          enableCrossOsArchive: false

      - run: npm ci

      - name: Build media derivatives
        id: media
        run: npm run media          # must write encoded=<n> / reused=<n> to $GITHUB_OUTPUT
        env:
          MEDIA_CONCURRENCY: '3'    # 4 vCPU runner, one core left for the OS
          UV_THREADPOOL_SIZE: '4'   # must be >= MEDIA_CONCURRENCY or Sharp serialises

      # Delete cache files the current manifest no longer references. Run it
      # BEFORE the save so the saved tarball is the pruned one, not the fat one.
      - name: Garbage-collect unreferenced derivatives
        run: npm run media:clean

      # Only write a new entry when the directory actually changed. On a repo
      # with 40 merges a day this is the difference between 250 MB/run of
      # quota churn and a handful of writes a week.
      - name: Save derivative cache
        if: steps.media.outputs.encoded != '0'
        uses: actions/cache/save@v4
        with:
          path: .cache/media
          key: ${{ steps.k.outputs.prefix }}-${{ github.run_id }}

      - run: npm run build

      - name: Report cache effectiveness
        run: |
          {
            echo "### Media cache"
            echo "- restored from: \`${{ steps.restore.outputs.cache-matched-key || 'cold' }}\`"
            echo "- reused: ${{ steps.media.outputs.reused }}"
            echo "- encoded: ${{ steps.media.outputs.encoded }}"
          } >> "$GITHUB_STEP_SUMMARY"

Tradeoff: gating the save on encoded != '0' means a run that only deletes derivatives never persists the pruned directory, so the tarball shrinks a run later than it could. That is the right side to err on — a stale-but-larger cache costs restore seconds, while unconditional saves cost quota, and quota exhaustion evicts entries you cannot get back without a full cold encode.

Verification

1. Prove reuse on a no-op commit

Push an empty commit and read the job summary. The restore step should log Cache restored from key: media-v3-Linux-X64-… and the media step should report encoded: 0. If it reports a non-zero encode count on a commit that touched no image, your derivative keys are not deterministic — the salt is picking up something that varies between runners (an absolute path, a timestamp, a floating libheif version).

2. Audit the quota with the GitHub CLI

# Total bytes and entry count against the 10 GB repository ceiling.
gh api repos/:owner/:repo/actions/cache/usage \
  --jq '{gb: (.active_caches_size_in_bytes/1073741824*100|round/100), entries: .active_caches_count}'

# Biggest entries first, with the ref that owns each one. Feature-branch
# entries that outlive their PR are the usual cause of quota pressure.
gh cache list --limit 40 --sort size_in_bytes --order desc \
  --json id,key,ref,sizeInBytes,lastAccessedAt

# Delete every entry from a merged branch in one pass.
gh cache list --ref refs/heads/feature/gallery-rework --json id --jq '.[].id' \
  | xargs -r -n1 gh cache delete

3. Time the restore, not just the encode

Cache restore is not free: a 251 MB directory takes roughly 11 s to download and untar on a ubuntu-latest runner, and that cost scales linearly with cache size while the encode saving does not. Compare the two directly:

# In the workflow, wrap the restore and print the delta.
du -sh .cache/media && find .cache/media -type f | wc -l

If restore time approaches the cold encode time for the files you actually use, the cache has stopped paying for itself and the GC step is under-aggressive.

4. Confirm branch inheritance

Open a throwaway PR from a branch created off the current default branch and confirm the restore step matches a key written by a main run. If it comes back cold, your workflow is not running on push to the default branch — see mistake 3.

Common mistakes

1. Hashing source assets into the cache key

Covered above, and worth restating because it is the single most common failure. hashFiles('assets/**') turns a per-file cache into an all-or-nothing one.

# WRONG — one edited photograph invalidates all 288 derivatives.
key: media-${{ hashFiles('assets/**') }}

# CORRECT — key on toolchain identity; freshness lives in the filenames.
key: media-v3-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('media.recipes.mjs','package-lock.json') }}-${{ github.run_id }}

Warning: hashFiles() returns an empty string when the glob matches nothing. A typo in the path silently collapses your key to media-v3-Linux-X64--<run_id>, which still works, still saves, and silently stops tracking recipe changes. Assert the value is non-empty in the key-computation step.

2. Using the combined actions/cache action and saving every run

The combined action registers a post-job save that runs whenever the primary key missed — and with a run-scoped primary key, it always missed. On a busy repository that writes a fresh 250 MB tarball on every merge, which fills the 10 GB ceiling in about 40 runs. The correct alternative is the restore + conditional save split shown above.

The same redundancy appears in matrix builds. A strategy.matrix over three Node versions runs three jobs that each restore, encode, and try to save under an identical key; two of them lose the reservation race and log Unable to reserve cache with key …, another job may be creating this cache. That is harmless but wasteful — three full encodes for one usable entry. Either build media in one dedicated job and hand the directory to the matrix through actions/upload-artifact, or include matrix.node in the key so each variant owns its own entry.

3. Only ever writing the cache from pull request builds

Actions cache scoping follows the branch graph: a cache written on a branch is visible to that branch and to branches created from it, while a cache written on the default branch is visible to every branch in the repository. If your workflow has no push: branches: [main] trigger, the only entries that exist belong to individual feature branches, and each new branch starts from nothing.

Cache scope across branches and forks The default branch cache is readable by every branch. A feature branch cache is readable by that branch and branches created from it, but not by sibling branches. A pull request from a fork can read the base repository cache but its save step is skipped because the token is read-only. default branch: main media-v3-… · 2.1 GB writes the shared entry read feature/gallery own entry · 2.3 GB feature/pricing own entry · 2.1 GB PR from a fork save step skipped feature/gallery/fix inherits parent entry siblings cannot see each other's entries read-only token: restore works, save is a no-op Only the default branch entry is universally readable — if nothing writes it, every branch starts cold.

Fork pull requests are the one case you cannot fully solve: pull_request runs from a fork receive a read-only GITHUB_TOKEN, so actions/cache/save logs a warning and exits successfully without writing anything. Restores still work against the base repository’s default-branch entry, so a warm main cache is what keeps fork contributions from paying the full 412-second encode. Do not reach for pull_request_target to fix this; it runs untrusted code with a writable token.

4. Letting the cache grow without a garbage-collection step

Deleted or reworked source images leave orphaned derivatives behind forever. Because the media cache shares its 10 GB ceiling with setup-node’s npm cache and every other actions/cache consumer in the repository, an ungoverned media directory does not just slow its own restore — it evicts other caches LRU-first, and the first symptom is usually a mysteriously slow npm ci.

Repository cache usage with and without derivative garbage collection Line chart over twenty workflow runs. Without garbage collection, cache usage climbs from 0.9 GB to the 10 GB repository ceiling by run 14, after which least-recently-used eviction begins and starts removing other caches. With garbage collection, usage stays flat at roughly 2.4 GB. Repository cache usage over 20 workflow runs 0 3 6 9 12 GB 10 GB repository ceiling LRU eviction begins run 0 5 10 15 20 no GC, save on every run GC + conditional save Past the ceiling, Actions evicts the least recently used entries — including the npm cache this workflow also depends on.

Your media:clean script should read the build manifest, walk .cache/media, and unlink any file whose key is not referenced. Add a grace window — keep entries touched within the last few runs — so a branch that legitimately references an older derivative set does not have its files deleted by a main build running concurrently.

5. Assuming an entry survives because nothing evicted it

Actions deletes any cache entry not accessed for 7 days, independently of the quota. A repository that goes quiet over a holiday comes back to a cold media cache no matter how much headroom it had. If your encode budget makes that unacceptable, the derivative cache does not belong in actions/cache at all — push it to object storage (S3, R2) keyed by the same content hashes, restore with a aws s3 sync-style step, and keep the Actions cache for node_modules only. The tradeoff is that you now own retention, egress and credentials, but you get an entry lifetime measured in months and cross-repository sharing that Actions cannot offer.