Rust Concurrency Masterclass on Atharva Pandey

Lesson 25: Production Concurrency Architecture — Putting it all together

Sun, 22 Dec 2024 10:00:00 +0000

After 24 lessons of building blocks, let’s talk about how they compose in real systems. I’ve shipped concurrent Rust services handling millions of requests per day, and the architecture patterns that survive production are surprisingly consistent. Not because they’re clever — because they’re boring. Boring is good when your pager is involved.

This lesson is the blueprint I wish I’d had when I started building concurrent Rust systems for real.

Lesson 24: Testing Concurrent Code — Loom and beyond

Sat, 21 Dec 2024 09:15:00 +0000

I spent a full week writing tests for a lock-free queue. Ran them a thousand times — all green. Shipped it. Two days later, a production crash. A race condition that occurred roughly once every 50,000 operations under specific timing. My tests never hit it because standard testing can’t explore all possible thread interleavings. That’s when I found Loom.

Testing concurrent code is fundamentally different from testing sequential code. A test that passes doesn’t mean the code is correct — it means the code was correct for that particular thread scheduling. Run the same test with different timing and you might get a different result.

Lesson 23: GPU Computing from Rust — wgpu and compute shaders

Thu, 19 Dec 2024 11:35:00 +0000

The first time I ran a matrix multiplication on a GPU, my jaw dropped. A computation that took 8 seconds on an 8-core CPU finished in 40 milliseconds on a mid-range GPU. That’s a 200x speedup. GPUs have thousands of cores — small, simple cores designed for massively parallel uniform computation.

Rust’s GPU story has gotten remarkably good. wgpu gives you cross-platform GPU compute that works on Vulkan, Metal, DX12, and even in the browser via WebGPU. No CUDA lock-in.

Lesson 22: SIMD — Explicit vectorization

Tue, 17 Dec 2024 08:20:00 +0000

I had an image processing pipeline that took 4.2 seconds per frame on a single core. After rewriting the hot loop with SIMD intrinsics, it dropped to 0.9 seconds. Same core, same algorithm, 4.7x faster. SIMD doesn’t add more cores — it makes each core do more work per clock cycle.

SIMD (Single Instruction, Multiple Data) is the other kind of parallelism. While threads run code on different cores, SIMD runs the same operation on multiple data elements simultaneously within a single core.

Lesson 21: CSP-Style Concurrency — Go channels in Rust

Sun, 15 Dec 2024 10:40:00 +0000

Before I wrote Rust full-time, I spent two years writing Go. The goroutine-plus-channel model gets into your brain. You start thinking about problems as independent processes connected by typed pipes. When I switched to Rust, the first thing I looked for was the equivalent of Go’s select statement. Crossbeam has it — and in some ways it’s even better.

CSP (Communicating Sequential Processes) is the formal model behind Go’s concurrency. The idea: concurrent processes interact only by passing messages through channels. No shared memory. Each process is sequential internally. Concurrency comes from composition.

Lesson 20: The Actor Model with Rust — Message-passing architectures

Fri, 13 Dec 2024 14:20:00 +0000

The first time I saw Erlang’s actor model, I thought it was over-engineered. Every piece of state behind a process, every interaction a message. Then I worked on a system with 40 mutexes, 12 deadlock-prone code paths, and a debugging story that involved printf-ing thread IDs into a file and diffing them. I became an actor model convert that week.

Rust doesn’t have a built-in actor system like Erlang or Akka. But the building blocks — channels, threads, ownership transfer — make building one surprisingly natural.

Lesson 19: Thread-Local Storage — Per-thread state

Wed, 11 Dec 2024 09:55:00 +0000

I was optimizing a JSON serializer that allocated a buffer for every call. Under profiling, those allocations were 30% of the cost. The fix? A thread-local buffer that gets reused across calls on the same thread. No synchronization needed. No contention. Each thread has its own buffer. Throughput doubled.

Thread-local storage is the ultimate escape hatch from synchronization overhead. If each thread has its own copy, there’s nothing to synchronize.

Lesson 18: Debugging Deadlocks and Data Races — Tools and techniques

Mon, 09 Dec 2024 11:30:00 +0000

The worst deadlock I ever encountered wasn’t between two mutexes. It was between a mutex and a channel. Thread A held a lock and tried to send on a full bounded channel. Thread B was the consumer for that channel but was waiting to acquire the same lock before it could receive. Clean deadlock. Took me four hours to find because I was looking at mutex ordering and the real problem was a channel.

Lesson 17: Lock-Free Data Structures in Rust — Beyond mutexes

Sat, 07 Dec 2024 08:50:00 +0000

A few years ago I was profiling a metrics collection service and found that 60% of the CPU time was spent on mutex contention. Thirty-two threads, one mutex protecting a counter map. The actual “work” — incrementing counters — took nanoseconds. The locking overhead was three orders of magnitude more expensive than the operation it protected.

That’s when lock-free data structures become worth the complexity. When the lock is the bottleneck, remove the lock.

Lesson 16: Barriers and Once — Synchronization primitives

Thu, 05 Dec 2024 10:15:00 +0000

I was building a benchmark suite once — eight threads, each measuring throughput of a different operation. The problem was that threads started at different times depending on OS scheduling. Thread 0 might start 50ms before thread 7, skewing the results. I needed all threads to start their measurement at exactly the same time.

That’s what barriers do. Everyone waits at the barrier until the last thread arrives, then they all proceed together.

Lesson 15: Condvar — Waiting for conditions

Tue, 03 Dec 2024 14:40:00 +0000

Early in my career, I wrote a producer-consumer queue using a mutex and a busy-wait loop. The consumer would lock the mutex, check if there’s data, unlock, sleep for 10 milliseconds, and repeat. It worked, but it burned CPU doing nothing and had up to 10ms of latency on every message. My tech lead pointed me to condition variables, and the latency dropped to microseconds while CPU usage went to near zero.

Lesson 14: parking_lot — Faster mutexes

Sun, 01 Dec 2024 09:25:00 +0000

I switched a high-contention service from std::sync::Mutex to parking_lot::Mutex and saw lock acquisition time drop by 30% under load. The API is nearly identical — it was a find-and-replace job. That’s the kind of optimization I like: massive payoff, zero complexity cost.

parking_lot is one of those crates that probably should have been in the standard library. It provides the same synchronization primitives as std, but faster, smaller, and with more features.

Lesson 13: Fan-Out Fan-In in Rust — Parallel pipelines

Fri, 29 Nov 2024 12:50:00 +0000

One of the most satisfying architectures I’ve built was a real-time analytics pipeline. Events came in through a single ingestion point, fanned out to eight processing workers, then fanned back in to a single aggregator that wrote results to the database. Throughput went from 2,000 events/second to 14,000 with zero data loss.

Fan-out/fan-in is one of those patterns that looks simple on a whiteboard but has real subtlety in implementation. Getting the channel lifecycle right, handling errors, managing backpressure — that’s where things get interesting.

Lesson 12: Worker Pool Patterns — Bounded concurrency

Wed, 27 Nov 2024 08:35:00 +0000

I blew up a production database once by spawning unlimited concurrent connections. A batch job that normally processed 100 items suddenly got 50,000. Each item opened a database connection. The connection pool maxed out, then the OS ran out of file descriptors. The database server stopped accepting connections from any service. All because I wrote for item in items { thread::spawn(...) } without thinking about bounds.

Worker pools solve this. Fixed number of threads, bounded queue, backpressure when the system is overloaded. After that incident, I never write unbounded concurrency again.

Lesson 11: Crossbeam — Scoped threads and lock-free structures

Mon, 25 Nov 2024 11:05:00 +0000

Before Rust 1.63 added thread::scope to the standard library, crossbeam was the only ergonomic way to spawn threads that could borrow local data. Even now that std has scoped threads, crossbeam remains essential. Its channels are faster than std::sync::mpsc, it provides lock-free data structures, and its utilities fill gaps the standard library doesn’t cover.

I reach for crossbeam in nearly every concurrent Rust project. Here’s why.

Crossbeam Channels

The biggest win: crossbeam’s channels are multi-producer, multi-consumer (MPMC) and significantly faster than std::sync::mpsc.

Lesson 10: Rayon — Data parallelism made easy

Sat, 23 Nov 2024 15:20:00 +0000

I had a batch processing job — reading 50,000 JSON files, parsing them, running validation, writing results. Single-threaded, it took 12 minutes. I added Rayon, changed .iter() to .par_iter(), and it dropped to 90 seconds. Five characters added to my code. That’s it.

Rayon is probably the most impressive crate in the Rust ecosystem for the effort-to-impact ratio. If you have CPU-bound work that operates on collections, Rayon makes parallelism trivial.

Lesson 9: Send and Sync — The traits behind thread safety

Thu, 21 Nov 2024 09:40:00 +0000

There’s a moment in every Rust developer’s life when they try to send an Rc<RefCell<Vec<String>>> to another thread and get a wall of compiler errors. They Google the error, see something about Send and Sync, patch the type to Arc<Mutex<Vec<String>>>, and move on without understanding why.

I did exactly that for six months. Then I needed to write a custom type that crossed thread boundaries, and I had to actually learn what these traits mean. Turns out they’re beautifully simple once you see the design.

Lesson 8: Memory Ordering — Relaxed, Acquire, Release, SeqCst

Tue, 19 Nov 2024 13:15:00 +0000

Memory ordering is the thing that separates people who use concurrent code from people who write concurrent primitives. I avoided understanding it for years, using SeqCst everywhere like a safety blanket. It worked. But it left performance on the table and, more importantly, left me unable to read half the lock-free code I encountered.

So here’s the actual explanation — no hand-waving.

Why Ordering Matters

Modern CPUs and compilers reorder instructions for performance. Your code says “write A, then write B,” but the CPU might execute B first if there’s no dependency between them. On a single thread, this is invisible — the final result is the same.

Lesson 7: Atomics — Lock-free primitives

Sun, 17 Nov 2024 07:45:00 +0000

I once replaced a Mutex<u64> counter in a hot path with an AtomicU64 and saw throughput jump 40%. Not because mutexes are slow — they’re fast. But for a single integer being incremented by 32 threads, the overhead of acquiring and releasing a lock millions of times per second adds up to real time.

Atomics are the foundation of lock-free programming. They let you do thread-safe operations on primitive values without any lock at all.

Lesson 6: Arc> — The shared mutable state pattern

Fri, 15 Nov 2024 10:30:00 +0000

I remember staring at Arc<Mutex<HashMap<String, Vec<u8>>>> in a codebase and thinking “this is the ugliest type I’ve ever seen.” Three months later, after debugging a similar system in Go that had zero type safety around its concurrent map access, I came crawling back to Rust’s ugly-but-correct approach.

The Arc<Mutex<T>> pattern is everywhere in Rust concurrent code. Understanding why it exists — not just how to type it — is the key.

Lesson 5: Shared State — Mutex, RwLock, and poisoning

Wed, 13 Nov 2024 16:55:00 +0000

The nastiest production bug I ever tracked down involved a Java ConcurrentHashMap that was “thread-safe” in the API sense but not in the logic sense. Two threads would read a key, both see it’s absent, both insert with computed values, and one would silently overwrite the other. The map itself was fine — the access pattern was broken.

Rust’s Mutex won’t save you from logic bugs either. But it will absolutely prevent you from forgetting the lock in the first place.

Lesson 4: Channels — mpsc and beyond

Mon, 11 Nov 2024 09:10:00 +0000

The first concurrent system I built that actually worked well was a log aggregation pipeline. Multiple producers writing log lines, one consumer batching and flushing to disk. No shared state, no locks, no races. Just messages flowing through a pipe.

That experience sold me on message passing. And Rust’s channel implementation makes it surprisingly ergonomic.

The Problem: Shared State Is Hard

You can share state between threads with mutexes. But every shared mutable variable is a coordination point, a potential bottleneck, and a source of bugs. The more threads touching the same data, the harder the code is to reason about.

Lesson 3: move Closures — Sending data to threads

Sat, 09 Nov 2024 14:20:00 +0000

When I first started writing threaded Rust code, I hit the same compiler error about forty times in one afternoon. Something about closures borrowing values that might be dropped. I kept slapping move on closures until things compiled, without really understanding what was happening.

That’s a terrible way to learn. So here’s the actual explanation I wish I’d had.

The Problem: Closures and Thread Lifetimes

When you pass a closure to thread::spawn, the new thread might run for an arbitrary amount of time. It could outlive the function that spawned it. It could outlive the variables it references.

Lesson 2: std::thread — Spawning and joining

Thu, 07 Nov 2024 11:45:00 +0000

A few years back I was reviewing a Go service that spawned goroutines like confetti at a parade — hundreds of them, no tracking, no lifecycle management. When the service shut down, half those goroutines just vanished mid-work. Orphaned database connections everywhere. The Go runtime made it so easy to fire off concurrent work that nobody stopped to think about cleanup.

Rust’s threading model forces you to think about it. Every thread gives you a handle. Every handle demands acknowledgment. You can ignore it — but you have to do so explicitly.

Lesson 1: Why Rust Concurrency Is "Fearless" — The compiler has your back

Tue, 05 Nov 2024 08:30:00 +0000

I once spent three days chasing a race condition in a Java service that only manifested under production load. The bug? Two threads updating a shared HashMap — no synchronization, no errors at compile time, no warnings. Just silent data corruption that showed up as incorrect billing amounts. Three days of my life, gone, because the language didn’t care.

Then I tried to write the same bug in Rust. The compiler said no.