System Design
Design With Sid
system design

How a video is uploaded to Youtube

How a video is uploaded to Youtube
Table Of Content

High-Level System Design

In this post, we'll walk through the key components and workflow involved in building a scalable video upload and processing system for a modern application like Youtube. This blog just gives High Level Design, if you want to dig deep then refer to this detailed guide.

Video Upload and Processing Workflow

1. User Uploads a Video

What Happens?

The user uploads a video using the application interface, such as a website or mobile app.

Services Involved

Load Balancer

  • Balances incoming requests across multiple API servers to prevent overloading a single server.
  • Ensures reliability and fault tolerance.

API Server

  • Handles the user’s request to upload the video.
  • Manages authentication, validation, and routes the upload request to Blob Storage.

2. Original Storage (Blob Storage)

What Happens?

The API server saves the raw uploaded video file to Blob Storage.

What is Blob Storage?

  • A type of object storage designed to handle large amounts of unstructured data, such as videos, images, and backups.
  • Examples: AWS S3, Google Cloud Storage, Azure Blob Storage.
  • Reliable, scalable, and cost-efficient for storing raw videos.

Why Is It Used?

It ensures that the original video file is safely stored and can be retrieved for processing or reprocessing.

3. Transcoding Servers

What Happens?

  • The Transcoding Servers pick up the raw video file from Blob Storage.
  • The video is converted (transcoded) into multiple resolutions or formats (e.g., 360p, 720p, 1080p, etc.).

What is a Transcoding Server?

  • A service or software that converts raw video files into multiple formats suitable for playback on different devices and network conditions.
  • Common tools: FFmpeg, AWS Elastic Transcoder, Google Transcoder API.

Why Is It Used?

To ensure the video is accessible on various devices (mobile, desktop, etc.) and supports different internet speeds.

Output

Multiple versions of the video are generated and sent to Transcoded Storage.

4. Parallel Processing After Transcoding

Once transcoding is complete, two parallel flows occur:

3a. Transcoded Storage

What Happens?

The transcoded video files are stored in Transcoded Storage for distribution.

What is Transcoded Storage?

  • A storage system where the processed video files are kept.
  • Often implemented using blob storage or specialized video storage systems that integrate with CDNs.
  • Examples: AWS S3, Google Cloud Storage, or internal distributed storage systems.

Why Is It Used?

Keeps the processed videos organized and ready for streaming or distribution.

3b. Completion Handler

What Happens?

A signal is sent to the Completion Handler, indicating that transcoding is complete.

What is the Completion Handler?

A service responsible for managing post-transcoding tasks, such as:

  • Updating the Metadata Database.
  • Notifying the user that their video is ready to stream.

5. Metadata Update

What Happens?

The Completion Handler updates the Metadata Database with:

  • Video status: "Ready to View".
  • Additional information: Available resolutions (e.g., 360p, 720p), video length, size, etc.

What is Metadata?

Metadata refers to information about the video, such as:

  • File name, upload time, size, duration, resolutions, and availability.

Cache Involvement

A Metadata Cache stores frequently accessed metadata, reducing load on the database and improving performance.

6. CDN Integration

What Happens?

The processed videos from Transcoded Storage are pushed to a Content Delivery Network (CDN).

What is a CDN?

A CDN is a globally distributed network of servers designed to deliver content (like videos) quickly to users based on their geographic location.

Examples

  • Cloudflare
  • AWS CloudFront
  • Akamai

Why Is It Used?

  • Ensures fast video playback with minimal buffering for users worldwide.

More Optimizations

More Optimizations

Summary

Here’s a high-level overview of the workflow:

  1. User Uploads Video → Sent to Blob Storage.
  2. Transcoding Servers pick up the raw video → Process it into multiple resolutions.
  3. Parallel Processing:
    • Transcoded Storage stores the processed videos.
    • Completion Handler updates metadata and notifies the user.
  4. CDN serves the videos for global accessibility.

By designing a system with these components, we ensure scalability, fault tolerance, and an optimal user experience for video uploads and streaming.

That’s the end of the workflow! Let me know if you’d like to see the source code or explore additional optimizations for this design.