Hotlink Protection with Referer and Token Auth

A Referer allowlist is the cheapest hotlink filter you can deploy and the easiest one to get catastrophically wrong: block on “no referer” and you 403 every browser running Referrer-Policy: no-referrer, every <video> element that omits the header on range requests, and every native app that never sends one. Match with a substring test and https://cdn.evil.net/?ref=example.com sails through. This page shows how to combine referer filtering with the HMAC scheme from Signed URLs and Media Access Control into a single ordered decision at the edge, so the referer check only ever saves work and never becomes the thing that grants or denies access on its own. Everything below assumes the media host is separate from the page host, which is the normal shape in CDN & Edge Media Delivery deployments.

The design rule is one sentence: the referer tier may only produce “allow, skip further checks” or “fall through”; it may never produce “deny” for content that a token could have unlocked. Denial belongs to the token tier alone. Once you accept that constraint the whole thing becomes deterministic and testable.

Prerequisite checklist

The order of evaluation matters more than any individual check. Sec-Fetch-Site is a browser-set forbidden header — page JavaScript cannot forge it, and Chrome 76+, Firefox 90+ and Safari 16.4+ all send it — so when it says same-origin or same-site you can short-circuit with high confidence. Referer is a hint. The token is the proof. Anything that fails the hint but presents a valid proof must still be served.

Ordered hotlink decision ladder at the edge A request for a media file passes four ordered checks. If Sec-Fetch-Site reports same-origin or same-site the request is allowed immediately. If the Referer header is empty the ladder skips the allowlist check and goes straight to token verification. If a Referer is present but its host does not suffix-match the allowlist the request is refused with a hotlink reason. Otherwise the HMAC token over path and expiry must verify, and only then is the object served from a cache key that excludes the signature. GET /media/hero.mp4 1 · Sec-Fetch-Site: same-origin / same-site? browser-set, page JS cannot forge it allow — first party no token required cross-site or header absent 2 · is a Referer header present at all? absent is legitimate, never a denial empty: no-referrer, native app, ranged video → skip to 4 present 3 · referer host suffix-matches allowlist? parsed as a URL, compared on host only 403 · hotlink no origin fetch, no cache write allowlisted 4 · HMAC(path + exp) equals sig? the only authoritative check 403 · token bad, missing or expired serve — cache key excludes sig, exp, kid every allowed path lands on the same object Tier 2 and 3 only ever save work or fall through; tier 4 is the sole authority that can deny a token-bearing request.

What the browser actually sends

Almost every broken hotlink rule fails because its author assumed Referer contains the full embedding URL. It usually does not. Under the modern default policy the header is truncated to the origin, and under several common policies it disappears entirely for cross-origin subresource loads. Write the allowlist against the host, and treat absence as “unknown”, never as “hostile”.

Referrer-Policy to Referer value matrix for cross-origin media Five referrer policies mapped to the Referer header value a cross-origin image or video request carries, what the edge rule can therefore compare, and the correct verdict. no-referrer and same-origin send nothing, so the rule must fall through to token verification. origin and strict-origin-when-cross-origin send the origin only, which is enough for a host match. unsafe-url sends the full URL including the path, which leaks private paths into your logs. Referrer-Policy on the embedding page → what the media host receives Policy on the page Referer on the media request Correct rule verdict no-referrer header absent entirely fall through to token same-origin absent for cross-origin loads fall through to token strict-origin-when-cross-origin (browser default since 2021) https://example.com/ origin only, trailing slash, no path host match works origin https://example.com/ host match works unsafe-url https://example.com/u/9f2/private full path leaks into edge logs match host, discard path Two of the five common policies send nothing at all — a rule that blocks on an empty Referer breaks those users outright. Native apps, server-side fetchers and some ranged video requests also arrive with no Referer.

The complete guard

One Worker, one ordered decision, one cache lookup. Every branch sets an x-guard-reason response header so the 403 taxonomy is queryable in logs from day one — which is what makes a staged rollout possible.

// hotlink-guard.js — Cloudflare Worker in front of a private media origin.
// Deploy with a route on media.example.com/* and a secret binding SIGNING_KEYS.

// Registrable hosts allowed to embed. Compared as an exact host OR a dot-suffix,
// NEVER as a substring: "example.com" must not match "example.com.attacker.io".
const ALLOWED_HOSTS = ['example.com', 'partner.net'];

// Paths that are public on purpose (og:image, favicons, poster frames used in
// social previews). Crawlers send no Referer and cannot hold a token.
const PUBLIC_PREFIXES = ['/social/', '/poster/'];

// Auth parameters. These are stripped before the cache key is computed so that
// every viewer of an asset shares one cached object.
const AUTH_PARAMS = ['exp', 'sig', 'kid'];

const SKEW_SECONDS = 30;          // tolerance in BOTH directions for signer/edge clock drift
const DENY_TTL = 'public, max-age=0, must-revalidate'; // never let a 403 stick in a shared cache

function hostAllowed(refererValue) {
  let host;
  try {
    host = new URL(refererValue).hostname.toLowerCase();  // parse, don't regex — this is the whole fix
  } catch {
    return false;                 // malformed Referer: treat as unknown, fall through to the token
  }
  return ALLOWED_HOSTS.some(a => host === a || host.endsWith('.' + a));
}

function b64urlToBytes(s) {
  const b64 = s.replace(/-/g, '+').replace(/_/g, '/').padEnd(Math.ceil(s.length / 4) * 4, '=');
  return Uint8Array.from(atob(b64), c => c.charCodeAt(0));
}

async function tokenValid(url, env) {
  const exp = Number(url.searchParams.get('exp'));
  const sig = url.searchParams.get('sig');
  const kid = url.searchParams.get('kid') ?? 'k1';   // key id: rotation needs two live secrets
  const secret = JSON.parse(env.SIGNING_KEYS)[kid];  // unknown kid MUST fail closed, never default
  if (!secret || !sig || !Number.isFinite(exp)) return false;
  if (Math.floor(Date.now() / 1000) > exp + SKEW_SECONDS) return false;

  const key = await crypto.subtle.importKey(
    'raw', new TextEncoder().encode(secret),
    { name: 'HMAC', hash: 'SHA-256' }, false, ['verify']);   // 'verify', not 'sign'
  // Canonical string = path + "\n" + exp. The query string is deliberately excluded
  // so a player appending ?_=cachebuster to range requests cannot invalidate the token.
  const msg = new TextEncoder().encode(`${url.pathname}\n${exp}`);
  // subtle.verify compares in constant time; a hand-rolled === leaks byte position.
  return crypto.subtle.verify('HMAC', key, b64urlToBytes(sig), msg);
}

function deny(reason) {
  return new Response('Forbidden', {
    status: 403,
    headers: { 'x-guard-reason': reason, 'cache-control': DENY_TTL },
  });
}

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);
    const site = request.headers.get('sec-fetch-site');   // browser-set; page JS cannot forge it
    const referer = request.headers.get('referer');
    const enforce = env.MODE === 'enforce';               // 'log' during rollout, 'enforce' after

    let reason = null;
    if (PUBLIC_PREFIXES.some(p => url.pathname.startsWith(p))) {
      reason = null;                                       // tier 0: intentionally public
    } else if (site === 'same-origin' || site === 'same-site') {
      reason = null;                                       // tier 1: first-party, short-circuit
    } else if (referer && !hostAllowed(referer)) {
      reason = 'hotlink';                                  // tier 2: known-bad embedder
    } else if (!(await tokenValid(url, env))) {
      reason = 'token';                                    // tier 3: absent referer lands here too
    }

    if (reason) {
      // Log-only mode still records the verdict but serves the bytes, so you can
      // measure the false-positive rate before flipping MODE to 'enforce'.
      console.log(JSON.stringify({ reason, referer, site, path: url.pathname, enforce }));
      if (enforce) return deny(reason);
    }

    // Cache key: auth parameters removed, everything else preserved. Do this AFTER
    // verification and BEFORE the cache lookup, or you cache an unverified response.
    const cacheUrl = new URL(url);
    AUTH_PARAMS.forEach(p => cacheUrl.searchParams.delete(p));
    const cacheKey = new Request(cacheUrl.toString(), { method: 'GET' });

    const cache = caches.default;
    let response = await cache.match(cacheKey);
    if (!response) {
      response = await fetch(cacheUrl.toString(), { cf: { cacheEverything: true, cacheTtl: 2592000 } });
      response = new Response(response.body, response);
      response.headers.set('cache-control', 'public, max-age=2592000, immutable');
      response.headers.delete('vary');   // Vary: Referer here would fragment the cache per embedder
      ctx.waitUntil(cache.put(cacheKey, response.clone()));
    }
    return response;
  },
};

Warning: the PUBLIC_PREFIXES escape hatch is load-bearing. Social crawlers (Slack, Discord, Twitter/X, Facebook) fetch og:image with no Referer and no token; if your open-graph images live under the guarded prefix, every link preview of your site silently breaks and nobody reports it for weeks. Keep preview-sized derivatives on a separate, deliberately public prefix.

Why the host must be parsed, not searched

The single most common bypass is a rule written as http.referer contains "example.com". contains matches the string anywhere in the header value — including inside a query parameter on somebody else’s domain, and inside a hostname that merely ends with your name as a label prefix. Parsing the header as a URL and comparing hostname closes all four variants at once.

Substring match versus parsed hostname match Five Referer values are tested two ways. A substring test for the text example dot com accepts all five, including a partner domain that merely carries the string in a query parameter, an attacker subdomain suffixed onto their own registrable domain, and a lookalike domain. Parsing the header as a URL and comparing the hostname to an exact match or a dot-prefixed suffix accepts only the two genuine ones. Same allowlist entry, two matching strategies Referer header value contains "example.com" parsed hostname https://example.com/gallery/2026 allow ✓ correct allow ✓ correct https://img.example.com/embed allow ✓ correct allow ✓ correct https://example.com.attacker.io/ allow ✗ bypass deny ✓ https://scraper.io/?src=example.com allow ✗ bypass deny ✓ https://notexample.com/steal allow ✗ bypass deny ✓ host === "example.com" || host.endsWith(".example.com") — two comparisons, zero bypasses. Note the dot in the suffix form: endsWith("example.com") alone still accepts notexample.com.

Verification

1. Exercise every branch with curl

Each command asserts one tier of the ladder. Run them against a deployment with MODE=enforce; the x-guard-reason header tells you which tier answered, which is far more useful than a bare 403.

BASE=https://media.example.com/media/hero.mp4
TOKEN="exp=${EXP}&sig=${SIG}&kid=k1"

# a) Allowlisted referer + valid token → 200, and a cache HIT on the second call.
curl -sI -H "Referer: https://example.com/gallery" "$BASE?$TOKEN" \
  | grep -iE 'HTTP/|x-guard-reason|cf-cache-status|age'

# b) Hostile referer → 403 with reason "hotlink", even though the token is valid.
curl -sI -H "Referer: https://scraper.invalid/" "$BASE?$TOKEN" | grep -iE 'HTTP/|x-guard-reason'
# HTTP/2 403 ; x-guard-reason: hotlink

# c) NO referer at all + valid token → must be 200. This is the regression that
#    breaks no-referrer browsers, native apps and social players.
curl -sI "$BASE?$TOKEN" | grep -iE 'HTTP/|x-guard-reason'
# HTTP/2 200

# d) NO referer, no token → 403 with reason "token", not "hotlink".
curl -sI "$BASE" | grep -iE 'HTTP/|x-guard-reason'
# HTTP/2 403 ; x-guard-reason: token

# e) The lookalike-domain bypass. A substring rule returns 200 here; the parsed
#    rule returns 403. This is the single test worth wiring into CI.
curl -sI -H "Referer: https://example.com.attacker.io/" "$BASE?$TOKEN" | grep -i 'x-guard-reason'
# x-guard-reason: hotlink

2. Confirm the Referer the browser really sends

The allowlist is written against reality, not against what you assume the page emits. In Chrome DevTools open Network, click the media request, and read Request Headers → Referer. If the row is missing, your page is on no-referrer or same-origin and tier 2 will never fire — which is fine, provided tier 3 works. Confirm the policy in one line from the console:

// Reports the effective policy and what the media host will therefore receive.
const meta = document.querySelector('meta[name="referrer"]')?.content;
console.log({
  metaPolicy: meta ?? '(none — header or browser default applies)',
  documentReferrer: document.referrer || '(empty)',
  // Chrome exposes the per-request policy on the element itself:
  imgPolicy: document.querySelector('img')?.referrerPolicy || '(inherits document)',
});

3. Roll out in log-only mode and read the taxonomy

Deploy with MODE=log first. Every request is still served, but the verdict is written to the log stream, so you can size the blast radius before enforcing.

# Stream live verdicts and count them by tier over a two-minute window.
npx wrangler tail hotlink-guard --format=json \
  | jq -r 'select(.logs[0]) | (.logs[0].message[0] | fromjson) | .reason' \
  | sort | uniq -c | sort -rn
#  41822 hotlink      <- genuine cross-site embedding, safe to block
#    613 token        <- INVESTIGATE: legitimate clients with no token yet
#      0 (null)       <- allowed

# CloudFront equivalent: query the standard log fields for the same split.
aws logs start-query --log-group-name /aws/cloudfront/media \
  --start-time $(date -d '-1 day' +%s) --end-time $(date +%s) \
  --query-string 'fields @timestamp, cs_referer, sc_status | filter sc_status = 403 | stats count() by cs_referer'

Tradeoff: log-only mode costs you nothing in security (you were not blocking before) and buys you an accurate false-positive count. Skipping it is how teams discover on a Monday morning that their Android app never sent a token.

What the numbers look like after enforcement

On a photo-heavy site with roughly 1.8 M cross-site image requests per day, enabling the guard removed about 86 % of that traffic within one day of flipping to enforce mode. The residual allowed traffic is the two allowlisted partner domains plus token-bearing embeds.

Cross-site media requests across a seven-day hotlink rollout Seven daily stacked columns of cross-site media requests. Days one and two run in log-only mode with roughly 1.9 million requests all allowed. From day three the guard enforces and about 1.5 to 1.6 million requests per day are blocked, leaving roughly 0.23 million allowed requests from allowlisted partners and token-bearing embeds. Cross-site media requests per day (millions), log-only days 1–2 then enforce 2.0 1.5 1.0 0.5 0 MODE=enforce day 1 day 2 day 3 day 4 day 5 day 6 day 7 allowed (partners + token-bearing) blocked (x-guard-reason: hotlink) Blocked requests never reach origin and never write to cache, so origin egress fell proportionally on day 3.

Common mistakes and their corrected form

1. Blocking the empty Referer

# WRONG — rejects no-referrer browsers, native apps and social crawlers.
(http.request.uri.path matches "^/media/") and not (http.referer contains "example.com")

# CORRECT — only reject a Referer that is present AND unrecognised; everything
# else falls through to the token tier, which is the real authority.
(http.request.uri.path matches "^/media/")
  and (http.referer ne "")
  and not (http.request.headers["referer"][0] matches "^https://([a-z0-9-]+\\.)*example\\.com/")

Anchoring the regex at ^https:// and terminating at the first / after the host is what stops the query-parameter bypass in a declarative rule engine that has no URL parser.

2. Treating the referer tier as authorization

Warning: a referer allowlist is bandwidth protection, not access control. curl -H 'Referer: https://example.com/' defeats it in one keystroke, and so does any headless browser. If the content is paid, private, or subject to a takedown obligation, tier 3 must be mandatory for every non-first-party request — do not add an “allowlisted referers skip the token” shortcut, however tempting the latency saving looks.

3. Putting Referer or Sec-Fetch-Site in the cache key

Adding either header to a CloudFront cache policy’s headers list, or emitting Vary: Referer from origin, gives you one cached object per embedding site. On a distribution with 40 partner domains that is a 40× multiplication of your working set and a corresponding collapse in hit ratio — the same failure mode analysed in short-lived signed URLs and CDN cache hit rate and diagnosed with the walkthrough in debugging CloudFront cache misses for images. Read the header, branch on it, then discard it before computing the key.

4. Letting the 403 get cached

// WRONG: the deny response inherits the asset's long TTL and a shared cache can
// pin a 403 in front of a legitimate viewer for a month.
return new Response('Forbidden', { status: 403 });

// CORRECT: deny responses are per-request decisions and must never persist.
return new Response('Forbidden', {
  status: 403,
  headers: { 'cache-control': 'public, max-age=0, must-revalidate', 'x-guard-reason': reason },
});

Cloudflare will not cache a 403 by default, but a cacheEverything rule or an intermediary proxy happily will. Setting the header explicitly costs nothing and removes an entire class of “one user is permanently blocked” tickets. The interaction between deny responses and revalidation semantics is covered in Cache-Control headers for image and video assets.

5. Guarding the open-graph and poster images

Social preview crawlers arrive with no Referer, no cookies, and no ability to acquire a token. Guarding og:image produces broken link previews everywhere your content is shared, and because the crawler caches the failure the damage outlives the fix. Keep preview derivatives — typically a 1200×630 JPEG or WebP — on an unguarded prefix, and accept that those specific files are public. The same applies to <video poster> frames that a third-party player fetches before the token-bearing manifest request.

6. Signing the full query string instead of the path

Media players append their own parameters. hls.js adds cache-busting values to segment requests, some smart-TV players append a session id, and range requests may carry a _ timestamp. If the canonical string covers url.search, every one of those mutations invalidates an otherwise valid token and produces intermittent, device-specific 403s that are miserable to reproduce. Sign pathname plus exp only, exactly as the guard above does, and ignore unknown parameters entirely.