JioHotstar-Scale Live Streaming Architecture: Designing for 3

Handling 3 crore concurrent viewers means that in a single, precise second, 30 million people are watching the same video stream simultaneously.

Accomplishing this scale goes far beyond simply deploying a "fast server." It is a classic distributed systems problem — and in this blog, I will walk you through the thinking process and architectural blueprint that senior infrastructure engineers use to solve it.

Step 1 — The Thinking Process

Before designing any architecture, a seasoned system engineer starts by asking critical questions and running back-of-the-envelope calculations:

Running the Estimations:

code

3 Crore concurrent viewers
= 30,000,000 users
 
Single user bandwidth = ~2 Mbps (720p stream)
Total bandwidth required = 30,000,000 × 2 Mbps
                         = 60,000,000 Mbps
                         = 60 Petabits/sec (peak)
 
Single edge server capacity = ~10 Gbps capacity
Edge servers needed = 60,000,000 Mbps / 10,000 Mbps
                    = 6,000 edge servers minimum

Key Architectural Questions:

Read-heavy or write-heavy? → Pure read workload (video delivery).
Consistency or availability (CAP theorem)? → Availability (a slight delay or lag is acceptable).
Latency requirement? → Under 3-5 seconds of live delay is acceptable.
Failure tolerance? → High availability with a target of 99.99% uptime.

Thinking through these constraints is always the first step of systems architecture.

Step 2 — High-Level Architecture

code

                        ┌─────────────────┐
                        │  Stadium Camera  │
                        └────────┬────────┘
                                 │ Raw Feed (SDI/RTMP)
                        ┌────────▼────────┐
                        │  Media Encoder   │
                        │  (FFmpeg based)  │
                        └────────┬────────┘
                                 │ Multiple Bitrates
                        ┌────────▼────────┐
                        │  Origin Server   │
                        │  (AWS Mumbai)    │
                        └────────┬────────┘
                                 │
              ┌──────────────────┼──────────────────┐
              │                  │                  │
     ┌────────▼───┐     ┌────────▼───┐     ┌────────▼───┐
     │  CDN PoP   │     │  CDN PoP   │     │  CDN PoP   │
     │  Delhi     │     │  Mumbai    │     │  Chennai   │
     └────────┬───┘     └────────┬───┘     └────────┬───┘
              │                  │                  │
           Users              Users              Users

Step 3 — The Video Pipeline

From Stadium to Origin

code

Camera (Raw 1080p 50fps)
    ↓
Hardware Encoder (Haivision / AWS Elemental Live)
    ↓ RTMP / SRT Protocol
Ingest Server (AWS MediaLive)
    ↓
Transcoder — creates multiple renditions:
    ├── 1080p @ 4500 kbps
    ├── 720p  @ 2500 kbps
    ├── 480p  @ 1000 kbps
    ├── 360p  @ 600 kbps
    └── 240p  @ 300 kbps
    ↓
Packager (AWS MediaPackage)
    ↓ HLS + DASH both formats
Origin S3 Bucket / Origin Server

How HLS (HTTP Live Streaming) Actually Works

code

master.m3u8  ← The player requests this index playlist first
│
├── 1080p/index.m3u8
│     ├── seg001.ts  (0-2 sec)
│     ├── seg002.ts  (2-4 sec)
│     └── seg003.ts  (4-6 sec)
│
├── 720p/index.m3u8
│     ├── seg001.ts
│     └── ...
│
└── 360p/index.m3u8
      └── ...

The player dynamically inspects the user's internet speed and automatically selects the appropriate playlist rendition. This mechanism is called Adaptive Bitrate Streaming (ABR).

Step 4 — CDN Layer (The Real Hero)

How CDNs Actually Work

A CDN (Content Delivery Network) caches video segments globally so that incoming client requests do not slam the primary origin server.

code

Without CDN:
30M users → Origin Server (Mumbai)
= Origin server crashes instantly
 
With CDN:
30M users → Nearest CDN Edge Server
Edge Server → Origin (only on cache miss)
= Origin server only handles ~100 unique requests

CDN Points of Presence (PoPs) in India

JioHotstar primarily leverages:

Akamai — 50+ PoPs across India
AWS CloudFront — 10+ India locations
Custom Jio CDN — leveraging telecom backbone routing advantages

code

User in Ahmedabad
    ↓
Nearest PoP: Ahmedabad / Surat edge node
    ↓ (if cache miss)
Mumbai Regional Cache (Origin Shield)
    ↓ (if cache miss)
Origin Server AWS Mumbai

During IPL matches, the CDN cache hit rate is consistently ~99%+ because millions of concurrent users are requesting the exact same video segment keys simultaneously.

🛡️ The Origin Shield Strategy

During a massive live event, if a specific edge node experiences a cache miss for a video block, hundreds of edge servers might simultaneously query the origin server. This is known as the Thundering Herd Problem, which can quickly crash the origin.

To prevent this, an Origin Shield (Regional Caching Layer) is placed between the CDN Edge PoPs and the central S3 Origin. When a cache miss occurs at the edge, the request is routed to the Origin Shield first, which collapses and serializes identical requests, sending a single query to the origin server.

Step 5 — Load Balancer Architecture

code

                    ┌──────────────────┐
                    │   DNS (Route53)   │
                    │  GeoDNS routing   │
                    └────────┬─────────┘
                             │
                    ┌────────▼─────────┐
                    │  Global Load      │
                    │  Balancer (L4)    │
                    └────────┬─────────┘
                             │
           ┌─────────────────┼─────────────────┐
           │                 │                 │
  ┌────────▼───┐    ┌────────▼───┐    ┌────────▼───┐
  │  Regional  │    │  Regional  │    │  Regional  │
  │  LB Mumbai │    │  LB Delhi  │    │  LB Bluru  │
  └────────┬───┘    └────────┬───┘    └────────┬───┘
           │                 │                 │
     ┌─────┴─────┐     ┌─────┴─────┐     ┌─────┴─────┐
     │  App      │     │  App      │     │  App      │
     │  Servers  │     │  Servers  │     │  Servers  │
     └───────────┘     └───────────┘     └───────────┘

GeoDNS resolves the client's location based on their IP address and automatically returns the IP of the nearest data center or CDN edge. This routes traffic efficiently at the DNS level before it even reaches the load balancers.

Step 6 — Auto Scaling Strategy

The Problem

The typical IPL match traffic pattern looks like this:

code

Views
  |
5Cr|              ████
4Cr|           ████████
3Cr|         ████████████
2Cr|      ██████████████████
1Cr|   ████████████████████████
   |___________________________________ Time
   Pre  Toss  Match    Finish  Post

Traffic is highly predictable yet extremely spiky.

The Solution — 3-Tier Scaling

Standard reactive auto-scaling (e.g., scaling when CPU usage > 70%) is far too slow for live sports events. For example, when Dhoni walks out to bat, 5 million users might tune into the stream within 10 seconds. Since standard cloud VM instances (like AWS EC2) take 3 to 5 minutes to spin up and register, relying on reactive scaling will cause immediate system failure!

code

Tier 1 — Scheduled Scaling (Pre-warm / Ladder System)
Capacity is pre-provisioned based on match timelines:
- Pre-warm to 10M capacity 1 hour before the match starts.
- Pre-warm to 20M capacity during the toss.
- Pre-warm to 40M+ capacity for the final overs.
AWS: Scheduled Scaling Policy utilizing pre-baked AMIs and containers.
 
Tier 2 — Reactive Scaling based on Request Rates
Scale out based on "Requests Per Second" rather than CPU or memory metrics.
AWS: Target Tracking Scaling Policy.
 
Tier 3 — Predictive Machine Learning Scaling
Utilizes historical game-day data to forecast traffic spikes and proactively scale compute.

Step 7 — Database Architecture

In a live streaming architecture, the database is rarely touched for the actual video stream delivery. However, it is heavily relied upon for user authentication, subscriptions, session state, and high-concurrency transactional flows (e.g., peak payment checkouts when a wicket falls):

code

┌─────────────────────────────────────────┐
│           Data Layer                     │
├─────────────┬──────────────┬────────────┤
│   Redis      │  Cassandra   │ PostgreSQL │
│  (Cache)     │  (Analytics) │  (Users)   │
├─────────────┼──────────────┼────────────┤
│ - Live view  │ - View count │ - User     │
│   counts     │   per second │   profiles │
│ - Session    │ - Watch time │ - Subscr.  │
│   tokens     │ - Geography  │ - Sharded  │
│ - Rate limit │   data       │   writes   │
└─────────────┴──────────────┴────────────┘

Database Sharding & Active-Active Replication

During critical match moments, user checkouts and subscription validations surge exponentially. To prevent transactional write bottlenecks, the primary relational database (PostgreSQL) is horizontally sharded, combined with multi-region active-active database replication setups to guarantee continuous availability.

Step 8 — Real-Time View Counter

Calculating and displaying a real-time concurrent view count (like 48.6 Crore views) to millions of users is a classic high-scale problem.

Updating an exact counter in real-time is computationally impossible at this scale. Instead, we use an approximation strategy.

code

Approach: Approximate Counting with HyperLogLog
 
Every player → sends heartbeat every 30 sec
                    ↓
             Kafka (message queue)
                    ↓
         Stream Processor (Flink/Spark)
                    ↓
         Aggregates every 10 seconds
                    ↓
              Redis HyperLogLog
                    ↓
         Broadcast to all clients via WebSocket

🧮 The Mathematics of HyperLogLog (HLL)

HyperLogLog is a highly optimized probabilistic algorithm used for estimating the cardinality (unique count) of multi-million element sets.

Memory Optimization: Storing an exact SET of millions of active User IDs would consume gigabytes of expensive RAM. HyperLogLog resolves this by requiring less than 12 KB of memory per key!
How it Works: HLL hashes the incoming stream of User IDs (using MurmurHash3) into binary sequences. It then calculates the mathematical approximation of the unique set by counting the maximum number of leading zeros in the binary hashes. The time complexity is a constant $O(1)$, completely eliminating database locking issues and write contention.
Accuracy: The standard error rate of HLL is approximately $1.04 / \sqrt$ (where $m$ represents the number of registers). At JioHotstar scale, a tiny ~1% discrepancy is statistically negligible and completely acceptable.

Step 9 — Fault Tolerance

What happens when an entire cloud region or primary server experiences a failure?

Multi-Region Active-Active Setup

code

Region: Mumbai (Primary)     Region: Delhi (Secondary)
┌────────────────────┐       ┌────────────────────┐
│  Full stack         │◄─────►│  Full stack         │
│  running           │       │  running           │
└────────────────────┘       └────────────────────┘
         │                            │
         └──────────┬─────────────────┘
                    │
              Both regions actively serve traffic.
              If Mumbai fails, DNS immediately
              routes 100% of the load to Delhi
              via active health checks.
              Uptime is maintained.

Circuit Breaker Pattern

code

// Pseudocode
if (originServer.responseTime > 500ms) {
  serveFromCache(); // Stale content is better than no content
}
 
if (originServer.errorRate > 5%) {
  circuitBreaker.open();
  // Temporarily stop querying the origin, serve cached static assets
  // Retry connection after 30 seconds
}

Step 10 — Security Layer

code

User Request
    ↓
DDoS Protection (AWS Shield / Cloudflare)
    ↓
WAF (Web Application Firewall)
    ↓
Token Validation (JWT / Signed URLs)
    ↓
Rate Limiter (Redis based)
    ↓
Content Served

Signed URLs — Each validated user is provided with a temporary, cryptographically signed URL:

code

https://cdn.hotstar.com/match/ipl2025/final/master.m3u8
  ?token=eyJhbGci...
  &expires=1748700000
  &user_id=dhaval123

Once the temporary security token expires, the edge server immediately terminates the stream, successfully preventing piracy and deep-linking.

Full Architecture Diagram

code

┌─────────────────────────────────────────────────────────────┐
│                    STADIUM                                   │
│  Camera → Encoder → SRT/RTMP → AWS MediaLive               │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                 ORIGIN LAYER (AWS Mumbai)                    │
│  MediaLive → Transcoder → MediaPackage → S3 Origin          │
│  [1080p] [720p] [480p] [360p] [240p]                       │
└──────────────┬──────────────────────────────────────────────┘
               │
┌──────────────▼──────────────────────────────────────────────┐
│                    CDN LAYER                                 │
│  Akamai + CloudFront + Jio CDN                             │
│  50+ PoPs across India                                      │
│  Origin Shield protection                                   │
│  Cache Hit Rate: 99%+                                       │
└──────────┬───────────────────────┬──────────────────────────┘
           │                       │
┌──────────▼──────┐      ┌─────────▼───────┐
│  API Layer       │      │  WebSocket       │
│  (User Auth,     │      │  (Live counts,   │
│   Payments,      │      │   Chat, Score)   │
│   Profiles)      │      │                  │
│  ECS + ALB       │      │  AWS API GW      │
└──────────┬───────┘      └─────────┬────────┘
           │                        │
┌──────────▼────────────────────────▼────────┐
│              DATA LAYER                     │
│  Redis | Cassandra | PostgreSQL | Kafka     │
└─────────────────────────────────────────────┘

Tech Stack Summary

code

Video Pipeline:    AWS MediaLive + MediaPackage + FFmpeg
CDN:               Akamai + AWS CloudFront + Jio CDN
Protocol:          HLS + DASH (Adaptive Bitrate)
Compute:           AWS ECS (containers) + EC2 Auto Scaling
Load Balancer:     AWS ALB + Route53 GeoDNS
Cache:             Redis Cluster (ElastiCache)
Database:          PostgreSQL (RDS) + Cassandra
Message Queue:     Apache Kafka
Stream Processing: Apache Flink
Security:          AWS Shield + WAF + Signed URLs
Monitoring:        Datadog + Grafana + PagerDuty
IaC:               Terraform (I learned it myself! 😄)

Conclusion

Handling 30 to 50 million concurrent viewers is not a problem solved by a single technology, framework, or cloud provider. It is a multi-layered engineering approach built on robust distributed systems principles:

Shift video delivery to the edge using high-capacity Multi-CDNs.
Dynamically adjust quality at the player level using Adaptive Bitrate Streaming (ABR via HLS/DASH).
Seamlessly distribute incoming traffic using L4 Global Load Balancers and GeoDNS routing.
Proactively scale capacity utilizing Scheduled pre-warming (Ladder scaling) and Request-Rate predictive scaling.
Architect for failure with Multi-Region Active-Active failovers and Circuit Breakers.
Protect and secure content using edge-level Web Application Firewalls (WAF) and signed security tokens.

This is the ultimate mindset of a distributed systems engineer — always assume everything will fail, and design every single layer to handle that failure gracefully.

JioHotstar-Scale Live Streaming Architecture: Designing for 3–5 Crore Concurrent Viewers

Step 1 — The Thinking Process

Step 2 — High-Level Architecture

Step 3 — The Video Pipeline

From Stadium to Origin

How HLS (HTTP Live Streaming) Actually Works

Step 4 — CDN Layer (The Real Hero)

How CDNs Actually Work

CDN Points of Presence (PoPs) in India

🛡️ The Origin Shield Strategy

Step 5 — Load Balancer Architecture

Step 6 — Auto Scaling Strategy

The Problem

The Solution — 3-Tier Scaling

Step 7 — Database Architecture

Database Sharding & Active-Active Replication

Step 8 — Real-Time View Counter

🧮 The Mathematics of HyperLogLog (HLL)

Step 9 — Fault Tolerance

Multi-Region Active-Active Setup

Circuit Breaker Pattern

Step 10 — Security Layer

Full Architecture Diagram

Tech Stack Summary

Conclusion