System Design Deep Dives on Atharva Pandey

Lesson 5: Design Twitter/X — Tweet fanout, timeline ranking, trending topics at 500M users

Thu, 14 Nov 2024 00:00:00 +0000

Twitter is the classic system design problem for good reason. It looks like a glorified blog until you start pulling on the threads: how does a tweet from a user with 100 million followers appear in every follower’s timeline within seconds? How do you rank timelines without reading millions of tweets per request? How do you identify trending topics across 500 million users in near real-time? Each of these is a genuinely hard problem, and they interact in non-obvious ways.

Lesson 4: Design WhatsApp — End-to-end encryption, message delivery guarantees, presence

Sat, 14 Sep 2024 00:00:00 +0000

WhatsApp is deceptively simple from a user perspective: you send a message, it arrives. But building a messaging system that handles 100 billion messages per day with end-to-end encryption, reliable delivery semantics, and real-time presence for 2 billion users is a genuinely hard engineering problem. I find this problem particularly instructive because it forces you to confront three things simultaneously: cryptographic key management, message delivery guarantees, and the cost of maintaining online/offline state at massive scale.

Lesson 3: Design Google Docs — Real-time collaboration, OT vs CRDT, conflict resolution

Mon, 15 Jul 2024 00:00:00 +0000

Google Docs is the problem I recommend to every engineer who thinks they understand distributed systems. The surface looks trivial: multiple users editing a document simultaneously. The depth is staggering. When two users type at the same position in a document at the same millisecond, what does each user see? How do you converge on a consistent state without a central lock? How do you preserve the intention behind each edit, not just the characters?

Lesson 2: Design Uber — Real-time matching, geospatial indexing, surge pricing

Sat, 25 May 2024 00:00:00 +0000

Uber is one of the most instructive system design problems because it forces you to think about real-time data pipelines, spatial indexing, and latency-sensitive matching algorithms all at once. I spent a significant amount of time studying this problem specifically because the geospatial angle is something most candidates gloss over. They say “use a database with location queries” and move on. But at Uber’s scale — 5 million trips per day, hundreds of thousands of concurrent drivers and riders — the geospatial indexing strategy is the entire problem.

Lesson 1: Design YouTube — Video upload, transcoding, streaming at scale

Tue, 09 Apr 2024 00:00:00 +0000

YouTube serves over 500 hours of video uploaded every minute and delivers billions of views per day. When I first studied this problem seriously, I made the mistake of treating it as a simple “upload file, store it, serve it” exercise. It isn’t. The interesting engineering is in what happens between the moment a creator hits upload and the moment a viewer’s video starts playing seamlessly on a 3G connection in rural India. That gap is where the system design lives.