02 Β· Back-of-the-Envelope Estimation¶
TL;DR
- Estimation ka matlab exact answer nahi β matlab hai saaf process dikhana aur sahi order of magnitude pe pahunchna (10Γ ka fark matter karta hai, 20% ka nahi).
- Teen tables zubaani yaad rakho: powers of two (KBβPB), latency numbers (memory β 100 ns, SSD β 100 Β΅s, cross-continent β 150 ms), aur availability nines (99.9% β saal mein 8.8 ghante down).
- Core recipe: DAU Γ actions per user Γ· 86,400 = average QPS, peak ke liye Γ 2β3. Storage = writes/day Γ size Γ retention Γ replicas.
- Bindaas round karo (86,400 β 10β΅), assumptions likh lo, aur har jagah units likho.
- Numbers se decisions lo: "40K QPS reads β cache chahiye", "55 PB β object storage chahiye, DB nahi".
Intuition: estimate karte hi kyun hain?¶
Civil engineer jab pul (bridge) plan karta hai, toh paint ka colour pehle nahi chunta. Pehle poochta hai: kitne trucks, kitne bhaari, kitni baar? Jawab decide karta hai ki footbridge chahiye ya steel ka suspension bridge.
Software mein estimation yahi kaam karta hai. "Twitter design karo" ka matlab 3 servers bhi ho sakta hai aur 3,000 bhi β aur dono bilkul alag designs hain. Paanch minute ka hisaab batata hai ki tum kaunsa bana rahe ho, aur interviewer ko dikhata hai ki tum requirements se design karte ho, buzzwords se nahi.
Table 1 Β· Powers of two¶
Data sizes powers of 2 hote hain, par estimation ke liye inhe powers of 10 maan lo β error 10% se kam hai.
| Power | Exact value | β | Naam | Short |
|---|---|---|---|---|
| 2ΒΉβ° | 1,024 | 1 hazaar | Kilobyte | KB |
| 2Β²β° | 1,048,576 | 10 lakh (1 million) | Megabyte | MB |
| 2Β³β° | 1,073,741,824 | 100 crore (1 billion) | Gigabyte | GB |
| 2β΄β° | β 1.1 Γ 10ΒΉΒ² | 1 trillion | Terabyte | TB |
| 2β΅β° | β 1.1 Γ 10ΒΉβ΅ | 1 quadrillion | Petabyte | PB |
Roz ki cheezon ka size (roughly yaad rakho):
| Cheez | Size |
|---|---|
| 1 ASCII character | 1 byte |
Ek long / Snowflake ID / timestamp |
8 bytes |
| Ek UUID | 16 bytes (string mein 36) |
| Tweet jitna text (280 chars) | ~300 bytes |
| Metadata ki typical DB row | 100 B β 1 KB |
| Compressed photo | 200 KB β 2 MB |
| 1080p video ka 1 minute | ~50β100 MB raw upload, har streamed rendition ~10β20 MB |
Table 2 Β· Latency numbers jo har programmer ko pata hone chahiye¶
Yeh Jeff Dean ki famous list se hain (lagbhag 2010, aaj ke approximations ke saath). Hardware improve hone se exact values badalti rehti hain; ratios matter karte hain.
| Operation | Approx. time | Aise socho⦠|
|---|---|---|
| L1 cache reference | 0.5β1 ns | β |
| Branch mispredict | 3β5 ns | β |
| L2 cache reference | 4β7 ns | β |
| Mutex lock/unlock | ~25 ns | β |
| Main memory reference | ~100 ns | "fast" ka baseline |
| 1 KB compress (Snappy/zstd) | ~2β3 Β΅s | sasta β bhejne se pehle compress karo! |
| 1 Gbps network pe 1 KB bhejna | ~10 Β΅s | β |
| SSD se random read | ~16β100 Β΅s | memory se ~1,000Γ slow |
| Memory se 1 MB sequential read | ~5β10 Β΅s | β |
| Same data center ke andar round trip | ~0.5 ms | β |
| SSD se 1 MB sequential read | ~50 Β΅s β 1 ms | β |
| Disk (HDD) seek | ~2β10 ms | random seeks se bacho! |
| HDD se 1 MB sequential read | ~1β5 ms | β |
| Packet round trip CA β Netherlands β CA | ~150 ms | speed of light hi limit hai |
Units: 1 ns = 10β»βΉ s Β· 1 Β΅s = 1,000 ns Β· 1 ms = 1,000 Β΅s = 10βΆ ns.
Insaani time mein socho β agar ek memory reference 1 second leta: toh SSD read lagbhag 1β2 din, disk seek lagbhag 3 mahine, aur cross-continent round trip lagbhag 5 saal leta. Isliye caches aur data locality itne important hain.
Interview mein yeh takeaways zor se bolo:
- Memory tez, disk slow β hot data RAM mein cache karo.
- Disk seeks se bacho β sequential I/O prefer karo (isi liye LSM-trees aur append-only logs bane β Chapter 6).
- Compression sasta hai β network pe bhejne se pehle compress karo.
- Cross-region calls mehenge hain (~100+ ms) β baatooni calls ek data center ke andar rakho; data users ke paas replicate karo.
Table 3 Β· Availability numbers ("nines")¶
Availability = kitne fraction time system chal raha hai. Service Level Agreement (SLA) woh promise hai jo provider customers se karta hai (AWS, GCP aur Azure zyada tar compute SLAs 99.9% ya upar rakhte hain). Related terms: SLO tumhara internal target hai, SLI actual measured value.
| Availability | Downtime / din | Downtime / hafta | Downtime / mahina | Downtime / saal |
|---|---|---|---|---|
| 99% ("two nines") | 14.4 min | 1.68 h | 7.31 h | 3.65 din |
| 99.9% ("three nines") | 1.44 min | 10.1 min | 43.8 min | 8.77 h |
| 99.99% ("four nines") | 8.64 s | 1.01 min | 4.38 min | 52.6 min |
| 99.999% ("five nines") | 864 ms | 6.05 s | 26.3 s | 5.26 min |
| 99.9999% ("six nines") | 86.4 ms | 605 ms | 2.63 s | 31.6 s |
Availability jodna:
- Series mein (saare parts chahiye): multiply karo. Teen 99.9% parts β 0.999Β³ β 99.7%. Har dependency availability ghatati hai.
- Parallel mein (koi ek kaafi hai):
1 β (1 β a)βΏ. Do 99% replicas β 1 β 0.01Β² = 99.99%. Redundancy availability badhati hai.
Khud try karo:
Estimation toolkit¶
Kaam ke conversions¶
| Fact | Yeh use karo |
|---|---|
| Ek din mein seconds = 86,400 | β 10β΅ (division aasaan) |
| Ek mahine mein seconds | β 2.5 Γ 10βΆ |
| Ek saal mein seconds | β 3 Γ 10β· |
| 10 lakh (1 million) requests/din | β 12 QPS |
| 100 crore (1 billion) requests/din | β 12,000 QPS |
| 1 request/sec lagatar | β 86K/din β 2.6M/mahina |
Core recipes¶
DAU = MAU Γ (roz active hone ka fraction)
Average QPS = DAU Γ (per user per day requests) / 86,400
Peak QPS = Average QPS Γ (2 β¦ 5) β traffic flat nahi hota
Read vs write QPS = read:write ratio se split karo
Storage per day = (writes per day) Γ (bytes per write)
Total storage = storage per day Γ 365 Γ years Γ replication factor
Bandwidth = QPS Γ bytes per request (writes ke liye ingress, reads ke liye egress)
Cache memory = daily read data ka ~20% (80/20 rule: 20% items ko 80% reads milte hain)
Servers = Peak QPS / (ek server kitna QPS sambhale) + headroom
Rule-of-thumb capacities (bahut rough, ek commodity server ke liye):
| Component | Ballpark |
|---|---|
| Stateless web/app server | 1K β 10K simple requests/sec |
| Relational DB (single primary) | ~1K β 10K writes/sec, achhe indexes ke saath reads zyada |
| Redis / Memcached node | ~100K+ ops/sec |
| Kafka broker | ~100K+ messages/sec |
| Commodity server RAM | 64 β 512 GB |
Interactive calculator¶
Koi bhi input badlo, poori chain dobara calculate hoti hai. Defaults neeche Worked Example 1 se match karte hain (100 KB average write hai: 20% posts Γ 500 KB image).
Worked example 1 Β· Ek microblogging service¶
(Numbers practice ke liye banaye hue hain.)
Assumptions β pehle board pe likho:
- 400 M monthly active users, 40% roz use karte hain.
- Har daily user 3 posts publish karta hai aur ~80 posts padhta hai (timeline loads).
- 20% posts mein ~500 KB ki image hai (video abhi ignore).
- Sab kuch 7 saal rakhna hai.
QPS
- DAU = 400 M Γ 40% = 160 M
- Write QPS = 160 M Γ 3 / 86,400 β 480 M / 10β΅ β 4,800 posts/sec; peak (Γ2) β ~10,000
- Read QPS = 160 M Γ 80 / 10β΅ β 128,000 post reads/sec β read-heavy (~27:1) β caching aur precomputed timelines zaroori (Chapter 11).
Storage
- Har post ka text + metadata: id 8 B + author id 8 B + text ~300 B + timestamps/counters ~100 B β ~0.5 KB
- Text/din = 480 M posts Γ 0.5 KB = 240 GB/din β Γ365Γ7 β ~0.6 PB (replication se pehle)
- Media/din = 480 M Γ 20% Γ 500 KB = 48 TB/din β Γ365Γ7 β ~123 PB
Numbers kya bata rahe hain: media text se ~200Γ bhaari hai. Text sharded database mein jaayega; media object storage (S3) + CDN mein. Yeh ek observation poore architecture ko shape karta hai.
Worked example 2 Β· WhatsApp jaisa chat¶
Assumptions: 2 B MAU, 50% daily β 1 B DAU; har user 40 messages/din bhejta hai; average message ~100 bytes; messages server pe 30 din store (deliver/backup hone tak).
- Messages/din = 1 B Γ 40 = 40 B/din
- Average write QPS = 40 B / 10β΅ = 400,000 msgs/sec; peak Γ3 β 1.2 M/sec
- Storage/din = 40 B Γ 100 B = 4 TB/din β 30 din β 120 TB, Γ3 replicas β 360 TB
- Connections: peak pe 30% DAU online β 300 M concurrent WebSocket connections. Ek achhe se tuned server pe ~1 M connections β sirf connections sambhalne ke liye ~300+ chat servers (plus headroom).
Decision: bottleneck storage nahi, concurrent connections aur message fan-out hai β stateful chat servers + writes ke liye optimised key-value store (Chapter 12).
Worked example 3 Β· YouTube jaisa video¶
Assumptions: 2 B MAU, 50% daily β 1 B DAU; har user 5 videos/din dekhta hai; 1% users 1 video/din upload karte hain; average upload 300 MB (raw); hum ~4 renditions store karte hain jo milke raw ka ~1.5Γ hain.
- View QPS = 1 B Γ 5 / 10β΅ = 50,000 views/sec
- Uploads/din = 1 B Γ 1% = 10 M videos/din β ~115 uploads/sec
- Storage/din = 10 M Γ 300 MB Γ 1.5 β 4.5 PB/din (!)
- Egress: agar average view 100 MB stream karta hai β 5 B views Γ 100 MB = 500 PB/din serve β yeh sirf CDN kar sakta hai.
Decision: cost pe CDN egress aur storage haavi hain. Optimisations β sirf popular videos CDN se, better compression, unpopular videos ke kam renditions β first-class design topics ban jaate hain (Chapter 14).
Worked example 4 Β· URL shortener¶
Assumptions: roz 40 M naye short links; har link apni zindagi mein ~50 baar khulta hai (read:write = 50:1); links 15 saal rakhne hain; har record ~250 bytes (long URL + code + owner + timestamps).
- Write QPS = 40 M / 10β΅ β 400/sec (exactly ~460)
- Read QPS = Γ50 β 20,000β23,000/sec; peak Γ2 β ~45K/sec
- 15 saal mein records = 40 M Γ 365 Γ 15 β 219 B records
- Storage = 219 B Γ 250 B β 55 TB (replication se pehle)
- Short-code length: kam se kam 219 B unique codes chahiye. Base-62: 62βΆ β 57 B (kam), 62β· β 3.5 T β β 7 characters.
Decision: chhote records, bahut zyada count, reads β« writes β sharded key-value store plus hot links ke liye aggressive cache. (Poora design Chapter 8 mein.)
Worked example 5 Β· Instagram jaisi photos β cache & servers¶
Assumptions: 500 M DAU; har user 50 photos/din dekhta hai; 0.2 photos/din upload karta hai, 2 MB ki; har photo ka metadata 1 KB; ek server 2,000 QPS handle karta hai.
- Read QPS = 500 M Γ 50 / 10β΅ = 250,000/sec; peak Γ2 = 500K/sec
- Write QPS = 500 M Γ 0.2 / 10β΅ = 1,000 uploads/sec
- Photo storage/din = 100 M Γ 2 MB = 200 TB/din
- Metadata cache: roz 25 B photo views; par unique photos jo dekhi gayi woh kahin kam. Agar hot set pichle 30 din ki photos ka ~20% hai: 100 M Γ 30 Γ 20% Γ 1 KB β 600 GB β 64 GB ke ~10 cache nodes (headroom/replicas ke saath).
- API servers: 500K / 2,000 = peak pe 250 servers, + ~30% headroom β 325.
Tips jo interviewer ko impress karte hain¶
Interview mein estimate kaise karein
- Pehle assumptions bolo, aur confirm karo: "Main 100 M DAU maan raha hoon β theek lag raha hai?"
- Bindaas round karo. 48,213 Γ 6.8 β 50,000 Γ 7 = 350,000. Koi decimal precision expect nahi karta.
- Har unit likho. "5" ka koi matlab nahi; "5 MB/s" ka hai.
- Reads aur writes alag karo β dono alag components ki taraf le jaate hain (cache vs write-optimised storage).
- Numbers ko decisions mein badlo β "toh writes ek DB sambhal lega, par reads ko cache chahiye".
- Sanity-check karo kisi jaani-pehchaani cheez se: "global chat app ke liye 1.2 M msgs/sec plausible hai."
- Zyada time mat lagao. 3β5 minute, phir design pe chalo.
Aksar poochhe jaane waale estimates
QPS aur peak QPS Β· N saal mein storage Β· bandwidth (ingress/egress) Β· cache memory Β· servers / connections ki ginti Β· ID space (kitne bits / characters).
Interviewer follow-ups¶
Peak ke liye 2β5 se multiply kyun? Yeh arbitrary nahi hai?
Traffic daily cycles (shaam ka peak), weekly cycles aur events (cricket final, New Year, sale) follow karta hai. Consumer apps ke liye 2β3Γ peak-to-average common hai; event-driven spikes 10Γ+ ho sakte hain. Point yeh hai ki average nahi, peak ke liye design karo β aur batao multiplier kyun chuna ("user base India-heavy hai, toh shaam ka peak").
Tumhara storage estimate 55 PB hai. Design mein kya badlega?
Koi database itna economically nahi rakhega. Blobs ke liye object storage (S3/GCS/HDFS), database mein sirf metadata (URLs, owners, sizes), purana data cold storage pe (sasta, slow), identical uploads deduplicate karo, aur reads CDN se bhejo.
Kitna cache chahiye, yeh kaise estimate karoge?
80/20 rule: ~20% objects ko ~80% reads milte hain. Cache memory β 20% Γ (roz padhe gaye unique objects) Γ object size. Phir overhead (~1.3β1.5Γ keys aur metadata ke liye) aur replicas jodo. Baad mein real hit-ratio metrics se validate karo.
1 B requests/din β kya yeh bahut hai?
Average ~12K QPS, peak pe shayad 30K β bada hai, par bilkul manageable: kuch dozen stateless servers, ek cache cluster, aur ek sharded/replicated DB. "Per day" ko "per second" mein badalna sabse tez tareeka hai yeh samajhne ka ki number kitna darawna hai.
Practice¶
Practice 1 β Food delivery
India mein ek food-delivery app: 20 M DAU, har user din mein 4 baar app kholta hai aur har baar 10 API calls hoti hain. Orders: 3 M/din. Har order 5 KB likhta hai. 3 saal rakhna hai. Dinner pe peak average ka 4Γ hai. API peak QPS, order write QPS, aur 3Γ replication ke saath order storage nikalo.
Solution. API: 20 M Γ 4 Γ 10 = 800 M/din β 800 M / 10β΅ = 8K QPS avg, Γ4 = 32K peak. Orders: 3 M / 10β΅ β 35/sec avg (~140 peak) β DB ke liye kuch bhi nahi. Storage: 3 M Γ 5 KB = 15 GB/din Γ 365 Γ 3 β 16.4 TB Γ 3 replicas β ~50 TB. Takeaway: API read path (menus, restaurant lists) ko caching chahiye; order writes aasaan hain.
Practice 2 β Snowflake bits
41-bit millisecond timestamp kitne saal ke IDs deta hai?
Solution. 2β΄ΒΉ ms β 2.2 Γ 10ΒΉΒ² ms β 2.2 Γ 10βΉ s β 2.2 Γ 10βΉ / 3.15 Γ 10β· β ~69.7 saal (Chapter 7).
Practice 3 β Bandwidth
Ek video platform 50K concurrent streams 5 Mbps each pe serve karta hai. Egress kitna?
Solution. 50,000 Γ 5 Mbps = 250,000 Mbps = 250 Gbps β 31 GB/s. Ek din mein β 2.7 PB. CDN chahiye.
Practice 4 β Connections
Ek multiplayer game ke 10 M DAU hain; peak pe 10% online, har ek ek WebSocket rakhta hai. Ek gateway server 200K connections rakhta hai. Kitne gateway servers?
Solution. 10 M Γ 10% = 1 M connections / 200K = 5 servers; N+2 redundancy aur headroom ke saath β 7β8.
Revision checklist¶
- KB se PB tak powers-of-two table likh sakta hoon.
- Memory β 100 ns, SSD β 100 Β΅s, HDD seek β 10 ms, cross-continent β 150 ms yaad hai β aur 4 takeaways bhi.
- 99%, 99.9%, 99.99%, 99.999% ka downtime/saal pata hai, aur series vs parallel kaise jodte hain.
- Per-day ko per-second turant badal sakta hoon (Γ· 10β΅) aur pata hai 1 M/din β 12 QPS.
- QPS, peak, storage, bandwidth, cache size aur server count units ke saath estimate kar sakta hoon.
- Har number ko design decision mein badal sakta hoon.