A video-on-demand platform is not one upload endpoint followed by one large file download. It is a durable media-processing workflow connected to a latency-sensitive delivery plane. A senior system design answer has to explain integrity, publication, access control, failure recovery, and viewer quality—not just “object storage plus a CDN.”
This guide applies the six-step system design framework to user-uploaded VOD. Live streaming has different latency, ingest, playlist, and recovery constraints and is deliberately out of scope.
Table of Contents
- Clarify Product Semantics
- Estimate Scale
- Design the API and State Model
- Build the High-Level Architecture
- Make Transcoding Durable
- Publish and Deliver Safely
- Design Adaptive Playback
- Handle Failures and Regions
- Measure Viewer Quality
- Common Interview Mistakes
- Quick Reference
- Frequently Asked Questions
Step 1 - Clarify Product Semantics
Confirm the media and product boundary before drawing components.
Core requirements:
- initiate, resume, complete, and cancel a large upload;
- validate and process a source into a device-compatible encoding ladder;
- publish HLS and/or DASH playback metadata without exposing partial output;
- authorize playback and deliver segments globally;
- report processing state and viewer-quality telemetry;
- block or delete content and propagate that decision within a defined objective.
Clarify whether this is public user-generated content, subscription VOD, or an internal media service. That determines moderation, DRM, entitlements, geographic rights, retention, and whether CDN URLs can be public. “DRM details are out of scope” does not make authorization, key separation, and takedown disappear.
Also ask:
- Is a low-quality rendition allowed before the complete ladder is ready?
- Which browsers, televisions, mobile devices, codecs, captions, audio tracks, and accessibility requirements must work?
- Are originals retained forever, for a fixed period, or until the owner deletes them?
- What are the RPO, RTO, processing-time, startup-time, rebuffering, and revocation objectives?
- Which regions may store or process each asset?
Availability is asymmetric. Existing playback usually matters more than new upload or high-quality-rendition completion, but a rights takedown can matter more than serving a cached segment.
Step 2 - Estimate Scale
Keep workload numbers as explicit scenario inputs. Suppose the platform receives one million uploads per day, the average source is 500 MB, the average duration is ten minutes, and viewers watch one billion hours per day at an average delivered bitrate of 3 Mbps.
Source ingest:
1,000,000 × 500 MB = 500 TB/day before replication, incomplete multipart parts, metadata, and backups.
Delivered traffic:
1 billion hours/day × 3 Mbps ÷ 24 hours/day ≈ 125 Tbps average viewer egress. Peak rate, geography, protocol overhead, cache hit ratio, and bitrate mix determine actual capacity. The result motivates distributed edge delivery; it is not a claim that one specific platform has this traffic.
Rendition storage:
Do not multiply source bytes by a fixed number of renditions. For a ten-minute asset whose video and audio ladder totals 15 Mbps:
600 seconds × 15 Mb/s ÷ 8 ≈ 1.125 GB
That is roughly 1.1 PB/day for one million assets before packaging overhead, replicas, thumbnails, captions, and retained old recipe versions. A different duration or ladder changes the result immediately. Lifecycle and deletion policy are first-order cost controls.
Transcoding capacity:
One million ten-minute sources create ten million source-minutes/day. Convert that into GPU/CPU demand using benchmarked real-time factors for each codec, resolution, quality preset, and hardware type. “Six rendition-minutes” are not six CPU-minutes, and sixty chunks do not imply a 60x speedup.
Capacity-plan four separate resources: upload bandwidth, workflow/encoding compute, stored bytes, and delivered bytes. Add headroom for retries, hot releases, cache fill, regional failure, and reprocessing after a recipe or codec change.
Step 3 - Design the API and State Model
The application controls authorization and workflow state; object storage carries the bytes.
POST /api/videos
Idempotency-Key: 9f2a...
{
"fileName": "launch.mp4",
"size": 524288000,
"checksumAlgorithm": "CRC32C"
}
201 Created
{
"videoId": "vid_123",
"uploadSessionId": "upl_456",
"partSize": 16777216,
"expiresAt": "..."
}
POST /api/videos/vid_123/upload-parts/7/url
POST /api/videos/vid_123/complete
{
"uploadSessionId": "upl_456",
"parts": [{ "number": 1, "etag": "..." }],
"size": 524288000,
"checksum": "..."
}
GET /api/videos/vid_123
POST /api/videos/vid_123/playback-session
DELETE /api/videos/vid_123Pre-signed part URLs are short-lived, scoped to a server-selected object key, and constrained by owner, expected size, quota, and upload session. Do not let the client choose an arbitrary bucket key. Treat file extensions and media types as untrusted.
Completing multipart transfer is not the same as accepting content. The service verifies session ownership, consecutive part metadata, total size, and a supported full-object checksum. It then conditionally transitions the asset and writes a processing outbox event. Retries of complete return the same workflow; duplicate or out-of-order object notifications do not create duplicate pipelines. Expired sessions are aborted so incomplete parts stop accruing cost.
A useful state machine is:
INITIATED -> UPLOADING -> UPLOADED -> INSPECTING -> PROCESSING
| |
v v
REJECTED READY | FAILED
READY -> BLOCKED -> DELETEDIf progressive publication is required, add PARTIALLY_READY with a precise minimum-ladder contract. Do not overload READY to mean “some files probably exist.” State transitions use expected versions so stale retries cannot move an asset backward.
Keep different data shapes in suitable stores:
| Data | Authoritative representation |
|---|---|
| Asset ownership, state, policy, active version | Transactional metadata store |
| Upload session and idempotency record | Metadata store with TTL/cleanup |
| Original, staged outputs, published segments | Private object storage |
| Job attempts, leases, dependencies | Durable workflow engine or task store |
| Playback authorization | Entitlement/policy service |
| Viewer telemetry | Bounded event stream and analytics store |
Step 4 - Build the High-Level Architecture
The ingest control plane should never proxy every media byte through ordinary application servers.
flowchart TD
Uploader([Uploader]) -->|initiate / part URLs / complete| Upload[Upload control service]
Uploader -->|multipart bytes| Quarantine[(Private quarantine storage)]
Upload --> Meta[(Metadata + idempotency)]
Upload --> Outbox[(Processing outbox)]
Outbox --> Flow[Durable media workflow]
Flow --> Inspect[Probe, scan, policy checks]
Inspect --> Encode[Chunk and rendition workers]
Encode --> Validate[Validate and package]
Validate --> Stage[(Versioned staging objects)]
Validate --> Publish[Atomic publisher]
Publish --> Meta
Publish --> Origin[(Private published origin)]
Viewer([Viewer]) --> Playback[Playback authorization]
Playback --> Meta
Playback -->|signed URL or cookie| CDN[CDN edge]
CDN --> Shield[Origin shield]
Shield --> Origin
Viewer -.QoE events.-> Events[(Telemetry stream)]Figure 1. Upload bytes go directly to quarantined object storage. A durable workflow validates and writes a complete version before an atomic publisher exposes it; playback authorization then grants bounded access through CDN and origin shield.
Use private storage for both quarantine and published objects. Only the CDN identity can read the origin, preventing users from bypassing edge authorization with a storage URL. Media parsers and codecs process hostile input, so workers run sandboxed with resource, duration, recursion, network, and output-size limits.
Step 5 - Make Transcoding Durable
Transcoding is a dependency graph, not one queue message:
- verify checksum and inspect container, streams, duration, dimensions, frame rate, and audio;
- scan or moderate according to product policy;
- choose a versioned encoding recipe from source properties and the supported device catalogue;
- encode video, audio, captions, thumbnails, and any trick-play tracks;
- validate timestamps, codecs, segment boundaries, media duration, checksums, and ladder completeness;
- package manifests and publish the version.
Do not upscale a low-resolution source simply because the standard ladder contains a 4K rung. Codec choice is a compatibility and cost decision; efficient codecs can require more compute and may not decode on every target device.
flowchart LR
Source[Verified source] --> Probe[Probe + recipe v42]
Probe --> Plan[Aligned GOP and segment plan]
Plan --> V1[Video rendition A chunks]
Plan --> V2[Video rendition B chunks]
Plan --> A1[Audio and captions]
V1 --> Check[Per-task validation]
V2 --> Check
A1 --> Check
Check --> Package[HLS / DASH packaging]
Package --> Whole[Whole-version validation]
Whole --> Commit[Commit active version]Figure 2. A versioned recipe produces aligned representations. Per-task checks are followed by whole-version validation so the publication pointer never exposes an incomplete or incompatible ladder.
Align access points and timestamps across representations so a player can switch without a gap. Segment and GOP length trade compression efficiency and request overhead against seek precision and adaptation speed. Some codecs, rate-control modes, and quality passes need context across chunk boundaries; splitting a one-hour file into sixty pieces therefore does not guarantee sixtyfold acceleration or identical quality.
Each workflow task uses an idempotency key such as (assetVersion, recipeVersion, rendition, chunk). Workers acquire a lease with heartbeats, write to an attempt-specific temporary key, validate the result, and conditionally register one winner. A timed-out worker may still finish, so the lease alone is not fencing. Late attempts must be unable to overwrite a newer result.
Queues normally deliver at least once. Bounded retries handle transient infrastructure failure; deterministic bad media is rejected with a durable reason rather than retried forever. Dead-letter storage aids investigation but does not by itself restore an asset. Track recipe versions and provenance so reprocessing is reproducible and rollback can reactivate a previously validated version.
Step 6 - Publish and Deliver Safely
Write every artifact under a versioned path such as:
/media/{assetId}/versions/{versionId}/hls/master.m3u8
/media/{assetId}/versions/{versionId}/video/1080p/segment-000123.m4sAfter all required objects pass validation, a transaction changes active_version, sets the asset to READY, and emits a publication event. The manifest may already exist in private staging, but no playback session points to it before commit. Garbage collection deletes abandoned attempts and old versions only after a safety window and reference check.
Versioned media segments are excellent long-TTL cache objects, but “immutable bytes” does not mean “no invalidation”:
- manifests can change when a version or ladder is published;
- an entitlement, geographic right, owner deletion, or abuse decision can revoke access;
- signed playback credentials expire;
- a bad encode may require rollback and purge;
- encryption keys and DRM licences follow separate authorization lifecycles.
Use long cache lifetimes for versioned segment URLs and shorter or versioned caching for manifests and policy responses. A takedown first blocks new playback sessions, then publishes a purge/deny version; short credential lifetime bounds residual access. Publicly cacheable unsigned segment URLs cannot be reliably made private by deleting only the database row.
sequenceDiagram
participant W as Media workflow
participant O as Private origin
participant M as Metadata store
participant P as Playback service
participant C as CDN
W->>O: write complete version under new prefix
W->>W: validate ladder and manifests
W->>M: conditional commit activeVersion + READY
M-->>P: version and policy visible
P-->>C: viewer receives bounded signed access
C->>O: fill versioned objects on cache missFigure 3. Publication is a pointer commit after complete validation. The CDN sees only a version that the metadata authority has declared playable.
An origin shield or regional cache collapses misses before private object storage. Protect the origin with concurrency limits, request coalescing, and tested cache-fill capacity. Pre-position only releases whose forecast justifies transfer and cache cost; blanket pre-warming wastes bandwidth and evicts useful content. Multi-CDN can reduce provider concentration risk, but adds routing, observability, token, purge, and consistency complexity.
Cache keys must include every representation-changing dimension but exclude irrelevant signature entropy where the CDN can validate authorization separately. Otherwise each signed URL fragments the cache. Configure allowed methods, range behavior, CORS, content types, and cache-poisoning defenses deliberately.
Step 7 - Design Adaptive Playback
HLS uses a master playlist to describe variant streams and media playlists; DASH uses an MPD with representations and segments. A platform may support one or both according to its device matrix. Common fragmented MP4 media can reduce duplicated packaging when the selected profiles interoperate, but “HLS and DASH” is not automatically one identical output.
sequenceDiagram
participant Player
participant Auth as Playback service
participant Edge as CDN edge
Player->>Auth: request playback session
Auth-->>Player: authorized manifest + bounded credential
Player->>Edge: GET manifest
Edge-->>Player: compatible representations
Player->>Edge: GET conservative startup segment
Edge-->>Player: media bytes
Note over Player: estimate throughput, buffer, decode and viewport
Player->>Edge: GET next aligned segment at selected quality
Edge-->>Player: media bytesFigure 4. The player selects among aligned representations using network and device signals. Authorization may be stateful even though versioned segment-byte delivery is cache-friendly.
The client commonly drives ABR, but the decision is richer than “bandwidth up, resolution up.” It considers recent throughput, buffer occupancy, startup target, viewport, decoder support, battery or data-saver policy, and the cost of quality oscillation. Server policy can restrict the ladder for rights, subscription, experiments, or device safety.
Shorter segments let the player react sooner and can reduce startup granularity, but they increase HTTP requests, playlist size, packaging objects, keyframe frequency, and compression overhead. Longer segments do the opposite. Benchmark startup, seek, rebuffering, quality, CDN behavior, and encoder efficiency rather than declaring a universal segment duration.
Media-byte serving can be stateless and cacheable. The overall playback system often is not: entitlement, concurrent-stream limits, DRM licences, personalized ads, manifests, and rights policy can require state. Keep those control decisions out of the per-segment origin path where possible.
Step 8 - Handle Failures and Regions
Abandoned upload. Expire the session and abort incomplete multipart state after a retention window. The owner can start a new idempotent session without guessing which object is authoritative.
Duplicate completion or event. Conditional state transitions and workflow identity return the existing job. Object notifications may be delayed, duplicated, or reordered; they are triggers, not the sole state authority.
Worker crash or timeout. Retry the specific task. An output becomes usable only after checksum and media validation; attempt fencing prevents a late worker from replacing the winner.
Bad source or one failed rendition. Classify deterministic input failure separately from infrastructure failure. Publish a smaller ladder only if the product's required-minimum policy and whole-version validation permit it; otherwise keep the asset non-playable.
CDN or origin trouble. Healthy edges continue serving fresh cached bytes. Misses use origin shield and bounded concurrency. Decide whether a second CDN, alternate origin, or stale serving is justified by the playback and revocation objectives; “route to the next-nearest edge” is provider behavior, not an application-level guarantee.
Takedown or deletion. Deny new playback, increment policy/version state, revoke or let short-lived credentials expire, purge CDN objects where supported, and asynchronously delete derived data and originals according to retention/legal policy. Record a tombstone long enough to prevent resurrection by delayed workflow events.
The CDN is a global delivery layer, not a complete multi-region architecture. Define a home authority for each asset and fence failover of upload, workflow, publication, and deletion. Processing can run elsewhere only with explicit input/output ownership. Replicate published origins according to RPO, recovery and cache-fill requirements; copying every source and rendition everywhere may violate residency rules or dominate egress cost.
Metadata failover must preserve the active-version pointer, entitlement state, and tombstones. Test regional loss with cache misses, not only hot cached videos. Track replication lag, restore procedures, key availability, purge propagation, and which functions intentionally pause during failover.
Measure Viewer Quality
Server latency alone does not describe playback. Collect privacy-reviewed client telemetry for:
- playback-start success and time to first frame;
- rebuffer count, duration, and ratio;
- delivered bitrate/resolution, quality switches, and oscillation;
- fatal errors by player, device, codec, CDN, ISP, and region;
- seek latency and exit-before-start;
- manifest, licence, and segment request failures.
For the pipeline, measure upload completion, checksum failure, queue age, time-to-ready normalized by source duration, encoder real-time factor, retries, lease expiry, validation failures, per-recipe cost, and rejected inputs. For delivery, distinguish CDN byte hit ratio from request hit ratio, origin-shield load, cache-fill concurrency, egress, purge age, and signed-access failures.
Set SLOs from product tiers and measured device/region distributions. “99% ready within minutes” is meaningless without conditioning on video duration, codec, backlog, and promised tier. Likewise, a global startup percentile can hide a broken television model or region.
Common Interview Mistakes
- Treating multipart completion as trusted content acceptance without checksum, quota, and conditional workflow state.
- Emitting processing work from an unverified or duplicate object event.
- Claiming that sixty chunks make encoding sixty times faster.
- Ignoring aligned timestamps and access points across ABR representations.
- Publishing a manifest before every referenced required object is validated and readable.
- Saying immutable segments remove all invalidation, authorization, takedown, and manifest concerns.
- Exposing a public object-store origin that bypasses CDN access policy.
- Calling rendition-minutes CPU-minutes or treating a fixed rendition count as a storage multiplier.
- Assuming the CDN also solves metadata, workflow, deletion, regional authority, and RPO/RTO.
- Declaring the whole server stateless while omitting entitlements, licences, and concurrent-stream policy.
- Using arbitrary global SLOs without segmenting by duration, device, codec, and region.
Quick Reference
| Decision | Defensible default |
|---|---|
| Upload | Tenant-scoped multipart session, bounded credentials, checksum, cleanup |
| Workflow | Durable DAG, idempotent tasks, leases plus fencing, bounded retries |
| Encoding | Source-aware versioned ladder with aligned timestamps/access points |
| Publication | Versioned staging, whole-version validation, atomic active pointer |
| Blob storage | Private object storage with lifecycle, replication, and deletion policy |
| CDN | Long-lived versioned segments, controlled manifests, origin shield, purge |
| Playback access | Signed URL/cookie or edge token plus private origin and entitlement check |
| ABR | Client selection using throughput, buffer, device, viewport, and policy |
| Multi-region | Asset home authority, fenced failover, explicit residency/RPO/RTO |
| Quality | Client QoE plus pipeline, CDN byte-hit, origin, cost, and revocation metrics |
Frequently Asked Questions
How is video stored and delivered at scale?
Store large source and rendition bytes in object storage, keep searchable workflow and ownership metadata in a database, and deliver published segments through a CDN backed by an origin shield. Size the origin from cache misses and failure scenarios rather than total viewer traffic. Use versioned object paths, lifecycle policies, checksums, replication, and access controls instead of assuming storage is automatically unlimited or public.
What is adaptive bitrate streaming?
Adaptive bitrate streaming publishes aligned representations at different bitrates, resolutions, and codecs plus an HLS or DASH manifest. The player selects compatible segments using throughput, buffer, viewport, decode capability, and user policy. Short segments react faster but increase request and encoding overhead, so segment duration and the encoding ladder must be tested against real devices and networks.
How does video transcoding scale?
Model transcoding as a durable DAG: probe and scan the source, create a versioned recipe, encode aligned chunks and renditions, validate outputs, package manifests, and publish once. Tasks need idempotency keys, leases, bounded retries, deterministic or attempt-scoped outputs, and checksums. Parallel chunks reduce latency, but codec dependencies, quality passes, shared bottlenecks, and the longest task prevent linear speedup.
Why is a CDN essential for a video platform?
A CDN moves cacheable media bytes near viewers, lowers startup and segment latency, and shields the origin from global traffic and hot releases. Versioned segments can use long TTLs, while manifests and authorization policy need shorter caching, version changes, or purge. Private origins, signed URLs or cookies, cache-key discipline, and revocation are still required; immutability does not eliminate access-control invalidation.
Why upload large videos in resumable chunks?
Multipart upload retries only failed parts and can send bytes directly to object storage, avoiding a bandwidth-heavy application proxy. The control service must scope upload credentials to one tenant and object, enforce size and expiry, verify the completed part list and full-object checksum, abort abandoned sessions, and conditionally move the asset state before publishing a processing event.
When is a video available to watch after upload?
Publish only after the required rendition set, timestamps, codecs, segments, and manifests pass validation. Write artifacts under a versioned staging prefix, then atomically commit the playable version and status so a manifest never references missing bytes. A product may expose a validated minimum ladder before higher qualities finish, but that is an explicit progressive-publication policy with versioned manifests.
Sources
- RFC 8216: HTTP Live Streaming - HLS playlists, variant streams, segments, alignment, and switching requirements.
- DASH-IF Interoperability Point Guidelines v5 - DASH media, packaging, MPD, and interoperability profiles.
- Amazon S3 multipart upload overview - multipart lifecycle and integrity metadata.
- Amazon S3 object integrity checks - part and full-object checksum behavior.
- Amazon CloudFront private content - signed URLs/cookies and restricting direct origin access.
Related Articles
- System Design Interview Problems: A Senior's Roadmap - the complete series index.
- System Design Interview Guide: The 6-Step Framework - the method applied here.
- Design a Web Crawler - durable fan-out and work scheduling.
- Design a Distributed Cache - cache policy, hot keys, and failure behavior.
- Design a URL Shortener - versioned caching and revocation trade-offs.
- Design a Payment System - idempotent workflow and reconciliation patterns.
This is Part 10 of a 16-part system design series. Next: Design a Payment System.
