Tokio on

Lesson 20: Production Async Architecture — Connection pools, retries, circuit breakers

Wed, 12 Feb 2025 17:09:33 +0000

This is the lesson that ties everything together. Over the last 19 lessons, we’ve built up from mental models to executors, from channels to cancellation safety. Now it’s time to put it all into a production-grade architecture.

I’ve run async Rust services handling tens of thousands of requests per second. The patterns in this lesson are the ones that survived contact with real traffic, real failures, and 3 AM pages. Nothing theoretical here — just battle-tested code.

Lesson 19: Testing Async Code — Mocking time and I/O

Mon, 10 Feb 2025 11:28:56 +0000

Async code is harder to test than sync code. Not because the logic is more complex, but because time, I/O, and concurrency introduce non-determinism. A test that passes 99 times and fails on the 100th is worse than a test that always fails — at least the always-failing test tells you something.

I’ve developed a set of patterns for testing async Rust that eliminate flakiness. The key insight: mock the things that make tests non-deterministic (time, network, randomness) and let everything else run for real.

Lesson 18: Tracing Async Code — Context propagation across .await

Sat, 08 Feb 2025 15:43:29 +0000

Debugging async code with println! is like debugging a highway pileup by asking each car individually what happened. You get disconnected fragments — “I was going north,” “I hit something,” “I heard a crash” — but no coherent story. With 500 concurrent tasks all writing to stdout, good luck finding which log line belongs to which request.

The tracing crate solves this. It gives you structured, contextual logging that follows your request across tasks, .await points, and even service boundaries. I switched from log to tracing three years ago and haven’t looked back.

Lesson 17: Pin in Async Context — Why futures must be pinned

Thu, 06 Feb 2025 08:27:44 +0000

I’ve been hand-waving around Pin for 16 lessons. Every time we saw Pin<&mut Self> in a poll method, I said “don’t worry about it.” But now it’s time to worry about it, because without Pin, async Rust’s entire safety model falls apart.

The good news: the mental model is simpler than it looks. The bad news: the syntax is still ugly. Let’s deal with both.

The Problem Pin Solves

When the compiler turns your async fn into a state machine, it needs to store local variables across .await points. Sometimes those variables reference each other:

Lesson 16: How Async Executors Work Under the Hood — Demystifying the runtime

Tue, 04 Feb 2025 13:56:18 +0000

There’s a moment in every async Rust developer’s journey where the runtime stops being a black box and starts being a machine you understand. For me, it was when I built a toy executor from scratch. Suddenly, Waker, Context, poll_ready — all of it made sense. Not as abstract concepts, but as mechanical parts.

This lesson won’t make you build a production executor. But it will show you how the pieces fit together, and that understanding will change how you write and debug async code.

Lesson 15: Backpressure — Bounded channels and flow control

Sun, 02 Feb 2025 10:33:45 +0000

The fastest way to kill a production service is to accept work faster than you can process it. I learned this the hard way when a log ingestion pipeline I built consumed 50GB of RAM and crashed because I used unbounded channels everywhere. “It works fine in testing” — yeah, testing with 100 messages, not 10 million.

Backpressure is the mechanism by which a system says “slow down, I’m full.” Without it, fast producers overwhelm slow consumers, and your only flow control is the OOM killer.

Lesson 14: Tower — The service middleware pattern

Fri, 31 Jan 2025 19:18:37 +0000

I avoided Tower for months. The trait bounds looked terrifying, the documentation assumed you already knew what you were doing, and I couldn’t figure out why I’d want it when I could just write functions. Then I needed to add logging, retries, timeouts, and rate limiting to every HTTP handler in a service with 40 endpoints.

Writing those as middleware that composes? That’s Tower’s whole thing. And once it clicks, you’ll never build a service without it.

Lesson 13: Building HTTP Clients with reqwest — Async HTTP done right

Thu, 30 Jan 2025 16:42:09 +0000

Almost every backend service I’ve built makes HTTP calls to something — a third-party API, another microservice, a webhook endpoint. And almost every production HTTP bug I’ve dealt with comes from one of three things: missing timeouts, not reusing connections, or ignoring response bodies.

reqwest is the HTTP client for async Rust. It’s built on hyper and Tokio, handles connection pooling, TLS, cookies, compression, and all the stuff you don’t want to think about. But you still need to use it correctly.

Lesson 12: Async I/O — Files, sockets, DNS

Tue, 28 Jan 2025 08:55:13 +0000

Here’s an uncomfortable truth about async file I/O: on most operating systems, it doesn’t really exist. When you call tokio::fs::read_to_string, Tokio dispatches the operation to a thread pool because Linux’s file I/O isn’t truly asynchronous (yes, there’s io_uring, but Tokio doesn’t use it by default). Network I/O, on the other hand, is genuinely async through epoll/kqueue.

Understanding this distinction matters. It changes how you architect things.

The Async I/O Traits

Tokio defines two core traits that mirror their std counterparts:

Lesson 11: Timeouts, Deadlines, and Graceful Shutdown — Bounded operations

Sun, 26 Jan 2025 12:24:51 +0000

Every production outage I’ve investigated boils down to one of two things: unbounded retries or missing timeouts. A function that “usually takes 50ms” eventually takes 30 seconds because the database is overloaded, and suddenly your entire service is frozen because every thread is waiting on that one slow call.

Timeouts aren’t optional in production code. They’re as important as error handling. And graceful shutdown — cleanly stopping your service when it’s time to deploy — is what separates “my service runs in production” from “my service runs in production well.”

Lesson 10: Cancellation Safety — The silent footgun

Fri, 24 Jan 2025 09:41:22 +0000

This is the lesson I wish someone had shoved in my face before I wrote my first select! loop. I lost three days to a bug where messages were disappearing from a queue. No errors. No panics. Just… gone. Turns out, select! was cancelling a future that had already read the message from the channel but hadn’t finished processing it.

Cancellation safety is the most under-discussed footgun in async Rust. If you use select!, you need to understand this.

Lesson 9: Semaphores and Rate Limiting — Bounded async concurrency

Wed, 22 Jan 2025 15:08:36 +0000

I once crashed a third-party API by spawning 10,000 concurrent requests from an async Rust service. The code was correct — every request completed (eventually). But the API’s rate limiter kicked in after 50 concurrent connections, and we got IP-banned for two hours.

The fix was a single type: tokio::sync::Semaphore. Five lines of code turned “as fast as possible” into “at most N at a time.”

What’s a Semaphore?

A semaphore is a counter with a maximum value. You acquire a permit before doing work, and release it when you’re done. If all permits are taken, acquiring blocks (yields) until one becomes available.

Lesson 8: Async Mutexes — tokio::sync::Mutex vs std

Mon, 20 Jan 2025 07:15:44 +0000

“Should I use tokio::sync::Mutex or std::sync::Mutex in async code?” I’ve seen this question in every Rust Discord server, every forum, every team Slack. And the answer most people give — “always use the async one in async code” — is wrong.

The real answer depends on how long you hold the lock and whether you need to .await while holding it. Get this wrong and you’ll either deadlock your runtime or tank your performance.

Lesson 7: Async Channels — tokio::sync::mpsc

Sat, 18 Jan 2025 13:52:28 +0000

The first real async service I built had a classic architecture: an HTTP handler receives a request, puts work on a queue, a background worker processes it, and the result gets sent back. In Go, this is channels all day. In async Rust, it’s also channels — but you’ve got four different kinds to choose from, and picking the wrong one leads to subtle bugs.

This lesson covers all four of Tokio’s channel types and when to reach for each one.

Lesson 6: Streams — Async iterators

Thu, 16 Jan 2025 10:31:07 +0000

Regular iterators give you one value at a time, synchronously. Futures give you one value, asynchronously. Streams give you multiple values, asynchronously. It’s the obvious combination, and once you start using them, you’ll wonder how you ever processed sequences of async data without them.

I first needed streams when building a log aggregator. I had dozens of log sources, each producing lines at their own pace. I needed to merge them, filter them, and process them in real time. Without streams, that code was a tangled mess of channels and select loops. With streams, it was a pipeline.

Lesson 5: tokio::select! — Racing futures

Tue, 14 Jan 2025 16:19:55 +0000

A couple months ago I was building a WebSocket handler that needed to do three things simultaneously: read from the socket, check a shutdown signal, and send periodic heartbeats. With join!, I’d need all three to complete. But I didn’t want them all to complete — I wanted to react to whichever one happened first.

That’s what select! does. It races multiple futures and gives you the result of the winner. The losers are dropped.

Lesson 4: Spawning Tasks and JoinHandles — Concurrent work units

Sun, 12 Jan 2025 08:44:19 +0000

I remember the exact moment async Rust “clicked” for me. I was building an API aggregator that needed to call five different services. My first version awaited them sequentially — 2 seconds total. Then I spawned them as concurrent tasks — 400ms. Same work, 5x faster, and I didn’t need to think about threads, locks, or shared state.

But spawning tasks isn’t free, and JoinHandle has some sharp edges that nobody warned me about. This lesson covers the patterns you’ll use every day.

Lesson 3: Tokio — The runtime that powers async Rust

Fri, 10 Jan 2025 11:05:33 +0000

Every async Rust tutorial shows you #[tokio::main] in the first example and then moves on like that’s totally self-explanatory. It’s not. That macro hides a lot of important decisions, and if you don’t understand what it’s doing, you’re going to make some costly mistakes.

I once spent an entire day debugging a deadlock that happened because I was using the single-threaded runtime without realizing it. One #[tokio::main(flavor = "current_thread")] vs the default, and my entire application’s behavior changed. That shouldn’t surprise anyone — but it surprised me.

Lesson 2: The Future Trait — What .await actually does

Wed, 08 Jan 2025 14:37:42 +0000

After I understood the mental model from lesson 1, my next question was obvious: what happens mechanically when I write .await? I could accept “it’s a state machine” as a hand-wave, but I wanted to see the gears turning.

Turns out, the answer is beautifully simple once you strip away the syntax sugar. The entire async machinery in Rust boils down to one trait, two types, and a contract.

The Future Trait, For Real This Time

use std::pin::Pin;
use std::task::{Context, Poll};

pub trait Future {
 type Output;
 fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

Three things to note:

Lesson 1: Async Mental Model — Futures, not threads

Mon, 06 Jan 2025 09:22:14 +0000

I spent three weeks writing async Rust code that compiled, ran, and produced correct results — while having absolutely no idea what was actually happening. I was copy-pasting async fn, slapping .await on things, and hoping for the best. Sound familiar?

The problem wasn’t syntax. The problem was that I was thinking about async Rust the same way I thought about threads in Go or Java. And that mental model is wrong for Rust.