Measuring the LCP Impact of Early Hints

There is no browser API that tells a page it received a 103. The informational response is consumed by the network stack before a document exists, it leaves no entry in PerformanceNavigationTiming, and by the time your JavaScript runs the only trace is an indirect one — a resource that started earlier than it should have been able to. That makes Early Hints uniquely hard to evaluate in the field: a large fraction of your traffic silently never receives the 103 at all, so a naïve before-and-after comparison of aggregate LCP is measuring browser mix and cache warmth as much as it is measuring the hint. This page builds a per-navigation attribution chain for the mechanism described in Early Hints and Speculation Rules for Media, so that the number you report is causal rather than coincidental.

Prerequisites

The last item is worth resolving before anything else, because it decides whether the experiment is possible at all. Early Hints buys back origin think time, so its effect size is bounded above by how much think time there is: a route that returns its first HTML byte in 80 ms cannot yield more than 80 ms of load-delay saving no matter how the hint is configured, and 80 ms at p75 is below what any realistic sample can resolve. Measure think time first, pick the route with the largest window, and run the experiment there rather than on the highest-traffic page.

The signature you are looking for

LCP decomposes into four contiguous, non-overlapping intervals that sum exactly to the metric. Early Hints does not compress LCP evenly across them — it attacks precisely one, and knowing which is what makes the measurement falsifiable.

Sub-part Definition What a working 103 should do
Time to first byte Navigation start to first byte of the HTML Nothing. The 103 is sent before the 200 but TTFB is measured against the final response.
Resource load delay TTFB to the LCP resource’s request start Collapse to zero. The fetch started before the HTML arrived, so the delay is clamped at 0.
Resource load duration Resource request start to response end Shrink slightly, then plateau — the transfer overlaps origin think time instead of competing with the parser.
Element render delay Resource response end to the LCP paint Often grows. The image is ready early, so the bottleneck moves to CSS, fonts or the main thread.

That last row surprises people, and it is the single most useful diagnostic on the page. A configuration that removes 390 ms of load delay but gives 60 ms back as render delay has not failed — it has exposed the next bottleneck. A configuration where render delay grows by more than load delay fell has genuinely regressed, usually because the hinted hero is now competing with a render-blocking stylesheet for the same connection.

LCP Decomposition — Unhinted vs 103 Early Hints Two stacked horizontal bars. The unhinted navigation totals 2180 milliseconds: 620 of time to first byte, 300 of resource load delay, 1100 of resource load duration and 160 of element render delay. The hinted navigation totals 1790 milliseconds: the same 620 of time to first byte, zero resource load delay, 950 of resource load duration and 220 of element render delay. Where the 390 ms actually comes from — median of 9 WebPageTest runs, 4G profile holdback arm 620 300 1100 160 2180 ms load delay collapses to 0 — the signature of a consumed 103 hinted arm 620 950 220 1790 ms TTFB resource load delay resource load duration element render delay TTFB is byte-identical across arms: if it moved, your rollout changed something other than the 103. Render delay grew 160 → 220 ms because the hero now waits on the stylesheet rather than on the network. Net −390 ms. Any report that does not decompose the total cannot tell this apart from a lucky CDN day.

The solution: an arm-stamped, decomposed LCP beacon

The chain has three links: the edge decides and records whether this navigation was hinted, the browser measures the four sub-parts, and the beacon carries both to a sink that can group by arm. Server-Timing is the join — it is the only response-header channel the platform exposes to JavaScript, and PerformanceServerTiming entries hang directly off the navigation entry.

// lcp-attribution.js — one beacon per navigation, arm-stamped and decomposed.
// Load it in <head> with `defer` off so the observer is registered before paint.

// --- 1. Navigation-level context ------------------------------------------
const nav = performance.getEntriesByType('navigation')[0];

// activationStart is non-zero only for a prerendered navigation. Every timing
// before it happened while the document was invisible and must be subtracted,
// otherwise prerendered rows drag the whole distribution left.
const activationStart = nav?.activationStart ?? 0;

// The edge writes: Server-Timing: eh;desc="sent", arm;desc="hinted", nid;desc="<uuid>"
// PerformanceServerTiming is same-origin by default and needs Timing-Allow-Origin
// only when the document itself is cross-origin to the beacon's page context.
const st = Object.fromEntries(
  (nav?.serverTiming ?? []).map(e => [e.name, e.description])
);
const arm  = st.arm ?? 'unknown';   // RANDOMISED at the edge — never inferred in JS
const ehed = st.eh  === 'sent';     // did the edge actually emit a 103 for this request

new PerformanceObserver(list => {
  // The last entry wins: LCP can be superseded until the first user interaction.
  const lcp = list.getEntries().at(-1);
  if (!lcp) return;

  // --- 2. Find the resource timing for the LCP element --------------------
  // lcp.url is empty for text nodes; those navigations carry no image at all
  // and must be excluded from the analysis rather than counted as neutral.
  const res = lcp.url
    ? performance.getEntriesByName(lcp.url).find(e => e.entryType === 'resource')
    : null;

  const ttfb = Math.max(0, (nav?.responseStart ?? 0) - activationStart);

  // requestStart is 0 when Timing-Allow-Origin is missing; startTime is always
  // populated, so fall back to it to avoid a bogus negative load delay.
  const reqStart = res ? Math.max(0, (res.requestStart || res.startTime) - activationStart) : 0;
  const resEnd   = res ? Math.max(0, res.responseEnd - activationStart) : 0;

  // --- 3. The four sub-parts, clamped so they sum to LCP ------------------
  const loadDelay    = res ? Math.max(0, reqStart - ttfb) : 0;
  const loadDuration = res ? Math.max(0, resEnd - Math.max(reqStart, ttfb)) : 0;
  const renderTime   = Math.max(0, lcp.startTime - activationStart);
  const renderDelay  = Math.max(0, renderTime - Math.max(resEnd, ttfb));

  navigator.sendBeacon('/rum/lcp', JSON.stringify({
    nid: st.nid,                    // join key minted by the edge, one per navigation
    arm,                            // 'hinted' | 'holdback' — the experiment assignment
    ehSent: ehed,                   // edge-side truth: was a 103 written to the wire
    // initiatorType 'early-hints' means the BROWSER consumed the hint. Use it as a
    // diagnostic only — grouping the analysis by it conditions on success and
    // biases the result toward fast connections. Group by `arm`.
    initiator: res?.initiatorType ?? 'none',
    protocol: res?.nextHopProtocol ?? '',  // h3 / h2 — 103 is ignored over HTTP/1.0
    lcp: Math.round(renderTime),
    ttfb: Math.round(ttfb),
    loadDelay: Math.round(loadDelay),
    loadDuration: Math.round(loadDuration),
    renderDelay: Math.round(renderDelay),
    element: lcp.element?.tagName ?? '',
    url: lcp.url ?? '',             // confirms the hinted URL is the one that painted
    prerendered: activationStart > 0,
    // transferSize 0 with a non-zero decodedBodySize means it came from cache —
    // cached navigations have nothing for a 103 to accelerate and are excluded.
    fromCache: res ? res.transferSize === 0 && res.decodedBodySize > 0 : false
  }));
}).observe({ type: 'largest-contentful-paint', buffered: true });

Three details in that file are load-bearing and easy to get wrong. buffered: true on the observer is mandatory: the LCP entry is frequently emitted before a deferred script has parsed, and without buffering you lose exactly the fast navigations that make the hinted arm look good. The Math.max(reqStart, ttfb) inside loadDuration is what keeps the four parts summing to LCP when the hero genuinely finished downloading before the HTML arrived — the intervals are defined as contiguous, so the overlap has to be attributed to one part rather than double-counted. And fromCache exists because a repeat view has the hero in the HTTP cache already; there is nothing for a 103 to accelerate, the row contributes a very fast LCP to whichever arm it lands in, and including those rows dilutes the measured effect roughly in proportion to your return-visitor rate. Whether a given navigation lands in that bucket is a function of the cache-control policy on your media assets, so the exclusion rate is worth reporting alongside the result.

The comment on initiator is the most important line in the file. initiatorType === 'early-hints' tells you the hint worked, which means it is downstream of the outcome you are trying to measure: clients on HTTP/1.1 proxies, on browsers without 103 support, or hitting a cold Cloudflare URL that has not harvested Link headers yet all land in the “no early hints” bucket and skew slow for unrelated reasons. Split on the randomised arm and treat the initiator as a compliance rate — “72% of the hinted arm actually consumed the 103” is a legitimate and useful number, but it is a covariate, not a grouping.

Attribution Chain for a Hinted Navigation A four-stage data flow. The edge worker assigns an experiment arm, emits the 103 and stamps Server-Timing. The browser measures the four LCP sub-parts and reads the Server-Timing values. The beacon endpoint receives one row per navigation. The warehouse groups by arm and computes the 75th percentile. Below, the beacon payload fields are listed. Edge worker hash(client) → arm emits 103 for hinted only Browser LCP observer + nav entry reads serverTiming[] Beacon endpoint one row per navigation no pre-aggregation Warehouse p75 by arm, bootstrap CI Server-Timing: eh, arm, nid 4 sub-parts + initiator joined on nid delta ± interval Beacon row — the minimum viable schema arm — randomised at the edge; the only legitimate grouping key ehSent — edge-side truth that a 103 was written to the wire initiator — 'early-hints' when the client consumed it; a compliance covariate ttfb / loadDelay / loadDuration / renderDelay — sum exactly to lcp url + fromCache + prerendered — the three exclusion filters applied before p75

Verification

1. Confirm the hint is on the wire and labelled

# The 103 and the Server-Timing stamp must both be present. --http2 is required:
# informational responses are not surfaced over HTTP/1.1.
curl -sSv --http2 https://example.com/ -o /dev/null 2>&1 \
  | grep -iE '< HTTP/2 (103|200)|< link:|< server-timing:|< timing-allow-origin:'
# Expect, in order:
#   < HTTP/2 103
#   < link: </media/home-hero-1600.avif>; rel=preload; as=image
#   < HTTP/2 200
#   < server-timing: eh;desc="sent", arm;desc="hinted", nid;desc="0f3c…"

# Verify the holdback arm really receives no 103. Force it with the bucketing
# cookie the edge worker reads, and assert the 103 line is absent.
curl -sSv --http2 -b 'eh_arm=holdback' https://example.com/ -o /dev/null 2>&1 \
  | grep -c 'HTTP/2 103'
# Expect: 0

2. Read the signature in DevTools

Reload with the Network panel open and the Initiator column enabled. The hero row should read early-hints; the Timing tab for the HTML document should list the eh, arm and nid entries under Server-Timing. Then paste this into the console to print the decomposition for the current navigation:

// Prints the four sub-parts. On a hinted navigation loadDelay should be 0 and
// the hero's requestStart should be LOWER than the document's responseStart —
// a strictly impossible ordering for any in-document hint.
const nav = performance.getEntriesByType('navigation')[0];
const lcpUrl = performance.getEntriesByType('largest-contentful-paint').at(-1)?.url;
const res = performance.getEntriesByName(lcpUrl).find(e => e.entryType === 'resource');
console.table({
  documentResponseStart: Math.round(nav.responseStart),
  heroRequestStart: Math.round(res.requestStart),
  heroInitiator: res.initiatorType,          // want: 'early-hints'
  startedBeforeHtml: res.requestStart < nav.responseStart   // want: true
});

3. Size the experiment before you run it

LCP distributions are right-skewed, so the 75th percentile — the value Core Web Vitals is defined on — has a much wider sampling interval than a mean would suggest. The table below is bootstrapped from a 40,000-navigation sample with p75 ≈ 2.8 s and an interquartile range of 1.5 s; resample your own distribution rather than borrowing these numbers verbatim.

Minimum detectable p75 shift Navigations per arm Days at 1.4k/day holdback
300 ms 1,800 2
200 ms 4,100 3
100 ms 15,400 11
50 ms 58,000 41

Warning: a 10% holdback on a route that gets 14,000 navigations a day yields 1,400 holdback rows a day, not 14,000. The holdback arm, not the total, sets the schedule — which is why a 50 ms effect is usually not worth trying to prove in the field.

LCP Distribution by Experiment Arm A paired bar histogram over seven LCP buckets. The hinted arm concentrates around 1.5 to 2 seconds while the holdback arm spreads across 2 to 3 seconds. The 75th percentile is 2350 milliseconds for the hinted arm and 2790 milliseconds for the holdback arm. LCP distribution, 19k navigations per arm — the delta lives in the right tail 0 10% 20% 30% <1.0 1.0–1.5 1.5–2.0 2.0–2.5 2.5–3.0 3.0–4.0 4.0+ p75 hinted 2350 ms p75 holdback 2790 ms hinted arm holdback arm Medians differ by 210 ms; p75 differs by 440 ms. Report the percentile.

4. Aggregate at the percentile, with an interval

-- DuckDB over the raw beacon rows. Exclusions first: cached heroes and text LCP
-- elements have nothing for a 103 to accelerate and only add variance.
SELECT arm,
       count(*)                                   AS n,
       quantile_cont(lcp, 0.75)                   AS p75_lcp,
       quantile_cont(load_delay, 0.75)            AS p75_load_delay,
       quantile_cont(ttfb, 0.75)                  AS p75_ttfb,
       avg(initiator = 'early-hints')             AS hint_compliance
FROM beacons
WHERE route = '/'
  AND from_cache = false        -- warm cache: hero already local, hint irrelevant
  AND prerendered = false       -- prerendered rows are a different population
  AND url <> ''                 -- text LCP element, no resource to hint
  AND ts >= now() - INTERVAL 7 DAY
GROUP BY arm;
-- Sanity gate: p75_ttfb must match across arms within ~15 ms. If it does not,
-- the arms differ by something other than the 103 and the result is void.

Common mistakes

1. Comparing before and after a deploy instead of running a concurrent holdback

Turning Early Hints on for everyone and diffing last week against this week folds in weekday mix, campaign traffic, CDN cache warmth after the deploy and any other change that shipped in the same window. Cloudflare’s harvest-and-replay model makes this worse: the first requests to every URL after a purge get no 103, so a deploy-day comparison systematically understates the effect.

Fix: randomise at the edge on a stable client hash, run both arms simultaneously, and keep the holdback for the whole measurement window. Hash something that survives a session — a first-party cookie you set on first visit, not the IP address, which reshuffles on mobile networks and will put the same user in both arms within a single browsing session.

2. Grouping the analysis by initiatorType === 'early-hints'

This is the subtle one. Conditioning on whether the hint was consumed is conditioning on an outcome: the “not consumed” group is disproportionately Safari, Firefox, corporate HTTP/1.1 proxies and cold edge URLs, all of which have different baseline LCP for reasons unrelated to the hint. The measured “improvement” is then mostly a browser-mix artefact and typically overstates the true effect by 2–3×.

Fix: group by the randomised arm. Report avg(initiator = 'early-hints') alongside it as a compliance rate, and interpret the arm-level delta as an intent-to-treat effect — which is the number that matters anyway, since you cannot choose your users’ browsers. If you need the effect on compliers specifically, divide the intent-to-treat delta by the compliance rate: a 250 ms arm-level improvement at 72% compliance implies roughly 350 ms for the clients that actually consumed the hint. That estimate is defensible; splitting the raw data on the initiator is not.

3. Comparing means

LCP is right-skewed and Core Web Vitals is defined at the 75th percentile. A mean is dominated by a tail that Early Hints barely touches, so a change that moves p75 by 440 ms can move the mean by 210 ms and look half as valuable. Worse, a single 40-second outlier moves the mean and never moves p75.

-- WRONG: hides the effect and is unstable under outliers.
SELECT arm, avg(lcp) FROM beacons GROUP BY arm;

-- CORRECT: the percentile CWV is defined on, plus a bootstrap interval so you
-- can say whether the difference is real. 200 resamples is plenty for p75.
SELECT arm, quantile_cont(lcp, 0.75) AS p75, count(*) AS n
FROM beacons GROUP BY arm;

4. Forgetting activationStart and Timing-Allow-Origin

Two silent zeroing bugs. Without the activationStart subtraction, any navigation that arrived via a prerender reports an LCP measured from the hidden render — often near zero — and drags p75 down in whichever arm happens to carry more speculation traffic. Without Timing-Allow-Origin on a cross-origin media host, requestStart and responseEnd are 0, loadDelay clamps to 0 in both arms, and the decomposition reports a perfect result regardless of configuration. The budgeting side of the same speculation interaction is covered in speculation rules prerender and the media prefetch budget.

Fix: assert both in the beacon. If res.requestStart === 0 && res.responseEnd === 0, tag the row timingBlocked: true and exclude it rather than letting it contribute a fake zero.

5. Counting a hint that preloaded the wrong asset as a win

A 103 that preloads /hero-1600.avif while the document’s sizes attribute selects /hero-800.avif fetches both files. Bytes double, LCP may still improve slightly, and no console warning distinguishes this from success. The url field in the beacon exists to catch it: compare the LCP element’s URL against the hinted URL, and alert when the mismatch rate rises. Getting the two to agree is a sizes-arithmetic problem, addressed in how to calculate optimal sizes attribute values.

Tradeoff: you can also detect this in the lab far more cheaply than in the field, at the cost of only covering the viewports you script. Run the check both ways — a lab assertion in CI catches the regression on the day it ships, and the field mismatch rate tells you how many real viewports it affects.

Running the measurement to a schedule

Instrument first and let the beacon bake for two days before assigning a single navigation to the holdback — a decomposition bug discovered mid-experiment invalidates everything collected so far. Then hold 10% back for the number of days your minimum detectable effect demands, analyse at the 75th percentile, and only after that go looking for confirmation in aggregate field data. The 28-day trailing window in the CrUX API will eventually reflect a real improvement, but it has no Early Hints dimension and lags by weeks, so it confirms rather than decides. For a lab counterpart that catches regressions the same day, the WebPageTest filmstrip diff automation for LCP workflow runs the same URL with the hint forced on and off.

Thirty-Day Measurement Schedule A timeline over thirty days. Beacon instrumentation runs days zero to three, a bake and sanity-check period days three to five, a ten percent holdback days five to nineteen, bootstrap analysis days nineteen to twenty-one, and CrUX confirmation from day two through day thirty. Measurement schedule for a 100 ms minimum detectable effect Instrument beacon Bake + sanity Holdback 10% 1.4k holdback rows/day → 19k by day 19 Bootstrap p75 CrUX confirmation 28-day trailing window — confirms, never decides day 0 5 10 15 20 25 30 Do not assign traffic to the holdback until the beacon has produced two clean days — a decomposition bug voids the sample. Keep the holdback running afterwards at 1–2% as a permanent regression tripwire.

Leave a small permanent holdback in place once the experiment concludes. Early Hints is fragile in ways that produce no errors — a purge that clears the harvested Link associations, a proxy upgrade that starts buffering informational responses, a new hero filename that the edge keeps hinting at its old path. A 1% arm that never receives the 103 gives you a continuous baseline to diff against, and turns each of those silent failures into a visible convergence between the two arms rather than a slow, unattributable drift in your dashboards.