Responsive Video Delivery in Next.js and React

Video delivery in component-driven applications demands more discipline than static <video> embeds. As part of the broader Responsive Image & Video Delivery architecture, this guide focuses on the layer between your transcode pipeline and the React component tree: source negotiation across codecs, deferred hydration to protect Largest Contentful Paint, poster pre-loading, and Cache-Control header configuration for edge caching. The output is a <ResponsiveVideo> component that ships zero bytes of player JS until the element enters the viewport, supports the full AV1 → VP9 → H.264 fallback chain, and passes WCAG 2.2 AA audit.

Concept & Architecture

The browser’s built-in <video> codec negotiation works identically to <picture> format negotiation: it reads <source> elements in document order and plays the first src whose type codec string passes HTMLMediaElement.canPlayType(). This means the media pipeline has three distinct responsibilities that must be designed separately.

Stage 1 — Transcode. FFmpeg converts source media into codec/container pairs during your build or CI job. Each output targets a different browser generation.

Stage 2 — Serve. Next.js route handlers and CDN edge rules apply Cache-Control: immutable, enforce Accept-Ranges: bytes (mandatory for mobile seek), and set correct MIME types for WebM and MP4 containers.

Stage 3 — Negotiate. The browser evaluates <source> elements in order. Explicit codecs= strings in the type attribute let the browser decide without a network round-trip.

Hydration control lives across stages 2 and 3: React defers mounting the <video> element until IntersectionObserver signals viewport entry, eliminating eager preload requests for below-the-fold video.

Benchmark Reference

Codec selection has measurable file-size and decode-latency consequences. Use these baselines to justify the transcode overhead in your pipeline.

Tradeoff: AV1 encode time is 5–20x slower than H.264 at equivalent quality. Run AV1 encodes on fast presets (-preset 6 for libsvtav1) in CI and reserve slower presets for offline archival masters.

Step-by-Step Implementation

Step 1 — Transcode Multi-Codec Variants

Run FFmpeg once per source file in your build pipeline. Output three variants: AV1 in mp4 (highest compression, modern browsers), VP9 in webm (broad coverage), and H.264 in mp4 (universal fallback).

# AV1 in MP4 container via libsvtav1
# -crf 30     — constant-rate factor: lower = higher quality (0–63 scale, NOT inverted like avifenc)
# -preset 6   — speed/quality tradeoff; 0=slowest, 12=fastest; 6 is a CI-safe default
# -c:a libopus — Opus audio; required for AV1 containers in Chrome
ffmpeg -i input.mp4 \
  -c:v libsvtav1 -crf 30 -preset 6 \
  -c:a libopus -b:a 96k \
  -movflags +faststart \
  output_av1.mp4

# VP9 in WebM — broader Safari 14+ support than AV1
# -b:v 0      — enable constant-quality mode (required when -crf is set for VP9)
# -row-mt 1   — row-based multi-threading; reduces encode time ~40% on multi-core CI
ffmpeg -i input.mp4 \
  -c:v libvpx-vp9 -crf 30 -b:v 0 -row-mt 1 \
  -c:a libopus -b:a 96k \
  output_vp9.webm

# H.264 in MP4 — IE11+ universal fallback
# -movflags +faststart moves the moov atom to the front for progressive streaming
ffmpeg -i input.mp4 \
  -c:v libx264 -crf 23 -preset fast -profile:v high \
  -c:a aac -b:a 128k \
  -movflags +faststart \
  output_h264.mp4

Step 2 — Configure Next.js Caching Headers

Set Cache-Control: immutable on all video routes. Use content-hashed filenames (e.g. hero-a1b2c3.mp4) so the immutable directive is safe to deploy.

// next.config.js
// All /videos/* paths receive 1-year immutable caching.
// Warning: only use immutable on content-hashed filenames — a filename collision
// will serve stale video from CDN edge until the hash changes.
module.exports = {
  async headers() {
    return [
      {
        source: '/videos/:path*',
        headers: [
          {
            key: 'Cache-Control',
            value: 'public, max-age=31536000, immutable'
            // 31536000 seconds = 1 year; immutable tells CDNs never to revalidate
          },
          {
            key: 'Accept-Ranges',
            value: 'bytes'
            // Required for mobile seek — without this iOS Safari cannot scrub
          }
        ]
      }
    ];
  }
};

Step 3 — Build the <ResponsiveVideo> Component

The component uses IntersectionObserver to defer mounting the <video> element until it is within 200px of the viewport. Before intersection, only the poster <img> is rendered — no video bytes are fetched.

// components/ResponsiveVideo.tsx
'use client'; // Next.js App Router: this component requires browser APIs

import { useEffect, useRef, useState } from 'react';

interface VideoSources {
  av1: string;   // AV1 in mp4 container — primary, highest compression
  vp9?: string;  // VP9 in webm — broad coverage; optional but strongly recommended
  h264: string;  // H.264 in mp4 — universal fallback; always required
}

interface VideoProps extends React.VideoHTMLAttributes<HTMLVideoElement> {
  src: VideoSources;
  poster: string;
  aspectRatio?: string; // e.g. '16/9', '4/3'; defaults to '16/9'
}

export function ResponsiveVideo({
  src,
  poster,
  aspectRatio = '16/9',
  ...props
}: VideoProps) {
  const containerRef = useRef<HTMLDivElement>(null);
  const [active, setActive] = useState(false);

  useEffect(() => {
    if (!containerRef.current) return;
    const io = new IntersectionObserver(
      ([entry]) => {
        if (entry.isIntersecting) {
          setActive(true);
          io.disconnect(); // one-shot — stop observing after first intersection
        }
      },
      { rootMargin: '200px' } // pre-load 200px before entering viewport to avoid flash
    );
    io.observe(containerRef.current);
    return () => io.disconnect();
  }, []);

  return (
    <div
      ref={containerRef}
      style={{
        position: 'relative',
        width: '100%',
        aspectRatio, // reserve layout space before video loads → CLS = 0
        background: '#000',
        overflow: 'hidden'
      }}
    >
      {active ? (
        <video
          {...props}
          preload="metadata"  // fetch only duration/dimensions, not payload
          playsInline          // mandatory for iOS Safari inline playback (no fullscreen forced)
          controls
          poster={poster}
          aria-label={props['aria-label'] ?? 'Video player'}
          style={{ width: '100%', height: '100%', display: 'block' }}
        >
          {/* Explicit codecs= string lets canPlayType() decide without a network probe */}
          <source src={src.av1} type='video/mp4; codecs="av01.0.05M.08"' />
          {src.vp9 && (
            <source src={src.vp9} type='video/webm; codecs="vp9"' />
          )}
          <source src={src.h264} type='video/mp4; codecs="avc1.42E01E"' />
          {/* Static fallback when JS/video is unavailable */}
          <img src={poster} alt={props['aria-label'] ?? 'Video placeholder'} loading="lazy" />
        </video>
      ) : (
        // Poster-only state: renders until IO triggers, zero video bytes fetched
        <img
          src={poster}
          alt=""                // decorative — the video element will replace this
          aria-hidden="true"
          fetchPriority={props.autoPlay ? 'high' : 'auto'}
          style={{ width: '100%', height: '100%', objectFit: 'cover' }}
        />
      )}
    </div>
  );
}

Step 4 — Add Captions and Reduced-Motion Support

Captions are a WCAG 2.2 Level AA requirement (Success Criterion 1.2.2). The prefers-reduced-motion media query disables autoplay for users who have requested reduced animation.

// Usage: wrap with reduced-motion guard at the call site
import { useMediaQuery } from '@/hooks/useMediaQuery'; // project-specific hook

export function HeroVideo() {
  // prefers-reduced-motion: disable autoplay for accessibility
  const reduced = useMediaQuery('(prefers-reduced-motion: reduce)');

  return (
    <ResponsiveVideo
      src={{
        av1: '/videos/hero-a1b2c3_av1.mp4',
        vp9: '/videos/hero-a1b2c3_vp9.webm',
        h264: '/videos/hero-a1b2c3_h264.mp4'
      }}
      poster="/videos/hero-poster.jpg"
      aria-label="Product overview video"
      autoPlay={!reduced}   // honour OS-level animation preference
      muted                 // required for autoPlay in any browser
      loop
    >
      {/* VTT captions — WCAG 2.2 AA SC 1.2.2 */}
      <track
        kind="captions"
        src="/videos/hero-captions-en.vtt"
        srclang="en"
        label="English"
        default
      />
    </ResponsiveVideo>
  );
}

Parameter Reference

Tradeoffs & Edge Cases

Tradeoff: AV1 encode time. libsvtav1 at -preset 6 is 3–5x slower than libx264 at -preset fast. For CI pipelines with many video assets, parallelise FFmpeg jobs and cache outputs by content hash. Reserve AV1 generation for assets over 10 seconds; shorter clips may not recoup the transcode cost.

Warning: Safari 16.4 AV1 software fallback. Safari 16.4 supports AV1 decode but falls back to software decoding on Macs without Apple silicon or Intel Ice Lake+. Software AV1 decode can spike CPU to 100% on a 1080p clip, causing frame drops. Test on real hardware; for Safari-heavy audiences keep VP9 in the source order.

Warning: fetchpriority="high" starvation. Applying fetchpriority="high" to more than one element on a page causes the browser’s preloader to deprioritise CSS and fonts. Limit high-priority poster fetches to the single above-the-fold video element.

Tradeoff: rootMargin size. A 200px root margin pre-fetches video on fast connections before the user sees the element, but on a slow 3G connection it can initiate a large download the user may never watch. Consider reading navigator.connection.saveData and reducing the root margin to 0px when saveData === true.

Warning: SSR hydration mismatch. The active state in <ResponsiveVideo> starts as false on the server and true only after IO triggers on the client. React 18 Suspense boundaries and startTransition do not help here — use a suppressHydrationWarning prop on the container div if the poster/video difference triggers a hydration error in strict mode.

Tradeoff: HEVC/H.265 as a second fallback. H.265 achieves better compression than VP9 on Apple hardware but is absent from Chrome and Firefox. Adding it between VP9 and H.264 in the source list has no effect on Chrome/Firefox (they skip to H.264), while Safari selects it over H.264. Only add H.265 if your analytics show >20% Safari share; the extra transcode/storage cost is rarely justified otherwise.

Debugging & Validation

Confirm codec negotiation via canPlayType() in the browser console before filing a “video won’t play” bug:

// Run in the browser console to check what the current UA actually supports
const v = document.createElement('video');
console.table({
  'AV1 mp4':  v.canPlayType('video/mp4; codecs="av01.0.05M.08"'),
  'VP9 webm': v.canPlayType('video/webm; codecs="vp9"'),
  'H264 mp4': v.canPlayType('video/mp4; codecs="avc1.42E01E"')
  // Returns: "probably", "maybe", or "" (empty = not supported)
});

Inspect response headers to confirm Accept-Ranges and Cache-Control are set correctly:

# -sI: silent mode, headers only; replace with your actual video URL
curl -sI https://example.com/videos/hero_av1.mp4 | grep -E 'Content-Type|Cache-Control|Accept-Ranges'
# Expected:
# Content-Type: video/mp4
# Cache-Control: public, max-age=31536000, immutable
# Accept-Ranges: bytes

Check for eager preload in the Network panel. In Chrome DevTools → Network → filter by Media: if video requests fire on page load for below-the-fold elements, IntersectionObserver is not wiring up correctly — the most common cause is the containerRef being null on mount due to a conditional render.

Lighthouse audit. Run Lighthouse in mobile simulation. Key signals: LCP should be driven by the poster <img> (not the video element itself); aspect-ratio on the container ensures CLS = 0 across all viewports. Any CLS > 0 on a video element usually means the container is missing its aspect-ratio rule.

Validate VTT captions with the W3C validator at https://quuz.org/webvtt/ or via the vtt npm package. Missing or malformed cues cause the captions track to silently fail in Safari — it will show no error in the console.

Related

• Using Next/Image with Custom Loader Configurations — apply the same loader pattern to still images in the same Next.js pipeline
• Implementing Responsive Video with Video.js — add a full-featured player UI while keeping the same codec negotiation strategy
• Art Direction with the HTML Picture Element — translate art-direction breakpoint logic from <picture> to <video> source elements
• Understanding Video Codecs: VP9 vs H.265 vs AV1 — codec fundamentals and encode-time benchmarks behind the fallback chain used here
• Cache-Control Headers for Image and Video Assets — configure edge caching for the immutable video assets served by this pipeline
• Responsive Image & Video Delivery — parent section covering srcset, art direction, and container queries alongside this video guide