Async on

Lesson 8: Runtime Comparison — Tokio vs async-std vs smol vs glommio

Thu, 02 Oct 2025 16:55:31 +0000

Every few months, someone asks me “which async runtime should I use?” and every time my answer is frustrating: “it depends.” But after spending the last seven lessons understanding how runtimes work from the inside, we can finally have a real conversation about what the differences actually are, why they exist, and when each one is the right choice.

I’ve used all four of these runtimes in production. Tokio for most things, smol for a lightweight embedded project, glommio for a storage engine prototype, and async-std briefly before migrating away from it. Let me share what I’ve learned — not from reading documentation, but from debugging real problems at 2 AM.

Lesson 7: Designing a Custom Async Runtime — When Tokio isn't enough

Tue, 30 Sep 2025 11:20:06 +0000

A colleague once asked me, “Why would anyone build a custom async runtime when Tokio exists?” Fair question. Tokio is battle-tested, well-maintained, and fast. For 95% of use cases, it’s the right answer. But I’ve now been in three situations where it wasn’t — and each one taught me something about what a runtime actually does.

The first was a latency-sensitive trading system where work-stealing’s cache invalidation was unacceptable. The second was an embedded system with no allocator. The third was a specialized database engine where we needed precise control over I/O scheduling. Each time, understanding how to build a runtime from the ground up saved the project.

Lesson 6: Work-Stealing Schedulers — Balancing load across cores

Sat, 27 Sep 2025 13:45:50 +0000

I once spent two days debugging a performance issue where our Tokio application was using four cores but only one was doing any work. Three threads were idle, one was pegged at 100%. The problem wasn’t Tokio’s scheduler — it was ours. We’d accidentally created a pattern where all tasks were spawned from and waking on the same thread, and nothing triggered work-stealing because the tasks completed too quickly.

That experience taught me that you can’t treat the scheduler as a black box. If you understand how work-stealing works, you can design your task topology to take advantage of it. If you don’t, you’ll write code that accidentally defeats it.

Lesson 5: The Reactor Pattern — How Tokio actually works

Wed, 24 Sep 2025 09:27:13 +0000

I read the Tokio source code on a Sunday afternoon, expecting to find something inscrutable. Layers of unsafe code, impenetrable abstractions, the kind of thing that makes you question your career choices. What I actually found was a clean, well-documented reactor implementation that I could follow. Not easily — but followably. The architecture is elegant once you see how the pieces connect.

This lesson is about that architecture. Not Tokio’s API — you already know how to use tokio::spawn and TcpStream. This is about what happens underneath when you call those functions. The reactor pattern, the I/O driver, the timer wheel, and how they all feed into the executor.

Lesson 4: epoll/kqueue — Platform event loops

Sun, 21 Sep 2025 17:08:45 +0000

Before I understood event loops, I thought async I/O was some kind of kernel magic. You register interest in a socket, and somehow the OS tells you when data arrives — without blocking a thread. How? I imagined complex kernel subsystems doing heavy lifting behind the scenes.

Turns out, the mechanism is almost embarrassingly simple. The kernel maintains a list of file descriptors you care about. When something happens on one of them, it flips a bit. You ask “what happened?”, it tells you. That’s it. The entire async I/O ecosystem — Tokio, Node.js, nginx, everything — is built on this one primitive.

Lesson 3: io_uring — Zero-copy async I/O on Linux

Fri, 19 Sep 2025 10:33:08 +0000

I ran a benchmark last year that genuinely surprised me. A simple TCP echo server using io_uring was handling 40% more connections per second than the same server on epoll, with measurably lower tail latencies. Not 5% — forty percent. On the same hardware, same kernel version, same application logic. That’s when I stopped treating io_uring as a curiosity and started treating it as the future of Linux I/O.

If you’ve been building async runtimes on epoll (or even kqueue on macOS), io_uring changes the game completely. Let me show you why.

Lesson 2: Building a Minimal Executor — Your own async runtime

Wed, 17 Sep 2025 14:12:37 +0000

The moment I built my first executor from scratch, async Rust stopped being scary. Not because executors are simple — they’re not — but because once you see the machinery, every “mysterious” behavior has an obvious explanation. Futures hanging? The waker isn’t being called. Tasks not making progress? The executor’s run loop has a bug. Performance terrible? You’re probably polling too aggressively or not enough.

So let’s build one. A real, working executor. Not a production-quality one (that’s Tokio’s job), but one that actually runs futures to completion, handles multiple tasks, and demonstrates every concept from the previous lesson.

Lesson 1: Future Internals — Poll, Waker, Context

Mon, 15 Sep 2025 08:45:22 +0000

I thought I understood Rust futures until I tried to implement one without async/await. Not a toy future that immediately returns Ready. A real future — one that yields, gets woken up, and resumes where it left off. That exercise broke every mental model I had and rebuilt it from scratch.

The async/await syntax is one of Rust’s great lies. It looks simple. It feels like you’re writing sequential code. Under the hood, the compiler is generating state machines, threading waker references through call graphs, and constructing self-referential structs that would make most C++ programmers nervous. Let’s rip the lid off.

Lesson 20: Production Async Architecture — Connection pools, retries, circuit breakers

Wed, 12 Feb 2025 17:09:33 +0000

This is the lesson that ties everything together. Over the last 19 lessons, we’ve built up from mental models to executors, from channels to cancellation safety. Now it’s time to put it all into a production-grade architecture.

I’ve run async Rust services handling tens of thousands of requests per second. The patterns in this lesson are the ones that survived contact with real traffic, real failures, and 3 AM pages. Nothing theoretical here — just battle-tested code.

Lesson 19: Testing Async Code — Mocking time and I/O

Mon, 10 Feb 2025 11:28:56 +0000

Async code is harder to test than sync code. Not because the logic is more complex, but because time, I/O, and concurrency introduce non-determinism. A test that passes 99 times and fails on the 100th is worse than a test that always fails — at least the always-failing test tells you something.

I’ve developed a set of patterns for testing async Rust that eliminate flakiness. The key insight: mock the things that make tests non-deterministic (time, network, randomness) and let everything else run for real.

Lesson 18: Tracing Async Code — Context propagation across .await

Sat, 08 Feb 2025 15:43:29 +0000

Debugging async code with println! is like debugging a highway pileup by asking each car individually what happened. You get disconnected fragments — “I was going north,” “I hit something,” “I heard a crash” — but no coherent story. With 500 concurrent tasks all writing to stdout, good luck finding which log line belongs to which request.

The tracing crate solves this. It gives you structured, contextual logging that follows your request across tasks, .await points, and even service boundaries. I switched from log to tracing three years ago and haven’t looked back.

Lesson 17: Pin in Async Context — Why futures must be pinned

Thu, 06 Feb 2025 08:27:44 +0000

I’ve been hand-waving around Pin for 16 lessons. Every time we saw Pin<&mut Self> in a poll method, I said “don’t worry about it.” But now it’s time to worry about it, because without Pin, async Rust’s entire safety model falls apart.

The good news: the mental model is simpler than it looks. The bad news: the syntax is still ugly. Let’s deal with both.

The Problem Pin Solves

When the compiler turns your async fn into a state machine, it needs to store local variables across .await points. Sometimes those variables reference each other:

Lesson 16: How Async Executors Work Under the Hood — Demystifying the runtime

Tue, 04 Feb 2025 13:56:18 +0000

There’s a moment in every async Rust developer’s journey where the runtime stops being a black box and starts being a machine you understand. For me, it was when I built a toy executor from scratch. Suddenly, Waker, Context, poll_ready — all of it made sense. Not as abstract concepts, but as mechanical parts.

This lesson won’t make you build a production executor. But it will show you how the pieces fit together, and that understanding will change how you write and debug async code.

Lesson 15: Backpressure — Bounded channels and flow control

Sun, 02 Feb 2025 10:33:45 +0000

The fastest way to kill a production service is to accept work faster than you can process it. I learned this the hard way when a log ingestion pipeline I built consumed 50GB of RAM and crashed because I used unbounded channels everywhere. “It works fine in testing” — yeah, testing with 100 messages, not 10 million.

Backpressure is the mechanism by which a system says “slow down, I’m full.” Without it, fast producers overwhelm slow consumers, and your only flow control is the OOM killer.

Lesson 14: Tower — The service middleware pattern

Fri, 31 Jan 2025 19:18:37 +0000

I avoided Tower for months. The trait bounds looked terrifying, the documentation assumed you already knew what you were doing, and I couldn’t figure out why I’d want it when I could just write functions. Then I needed to add logging, retries, timeouts, and rate limiting to every HTTP handler in a service with 40 endpoints.

Writing those as middleware that composes? That’s Tower’s whole thing. And once it clicks, you’ll never build a service without it.

Lesson 13: Building HTTP Clients with reqwest — Async HTTP done right

Thu, 30 Jan 2025 16:42:09 +0000

Almost every backend service I’ve built makes HTTP calls to something — a third-party API, another microservice, a webhook endpoint. And almost every production HTTP bug I’ve dealt with comes from one of three things: missing timeouts, not reusing connections, or ignoring response bodies.

reqwest is the HTTP client for async Rust. It’s built on hyper and Tokio, handles connection pooling, TLS, cookies, compression, and all the stuff you don’t want to think about. But you still need to use it correctly.

Lesson 12: Async I/O — Files, sockets, DNS

Tue, 28 Jan 2025 08:55:13 +0000

Here’s an uncomfortable truth about async file I/O: on most operating systems, it doesn’t really exist. When you call tokio::fs::read_to_string, Tokio dispatches the operation to a thread pool because Linux’s file I/O isn’t truly asynchronous (yes, there’s io_uring, but Tokio doesn’t use it by default). Network I/O, on the other hand, is genuinely async through epoll/kqueue.

Understanding this distinction matters. It changes how you architect things.

The Async I/O Traits

Tokio defines two core traits that mirror their std counterparts:

Lesson 11: Timeouts, Deadlines, and Graceful Shutdown — Bounded operations

Sun, 26 Jan 2025 12:24:51 +0000

Every production outage I’ve investigated boils down to one of two things: unbounded retries or missing timeouts. A function that “usually takes 50ms” eventually takes 30 seconds because the database is overloaded, and suddenly your entire service is frozen because every thread is waiting on that one slow call.

Timeouts aren’t optional in production code. They’re as important as error handling. And graceful shutdown — cleanly stopping your service when it’s time to deploy — is what separates “my service runs in production” from “my service runs in production well.”

Lesson 10: Cancellation Safety — The silent footgun

Fri, 24 Jan 2025 09:41:22 +0000

This is the lesson I wish someone had shoved in my face before I wrote my first select! loop. I lost three days to a bug where messages were disappearing from a queue. No errors. No panics. Just… gone. Turns out, select! was cancelling a future that had already read the message from the channel but hadn’t finished processing it.

Cancellation safety is the most under-discussed footgun in async Rust. If you use select!, you need to understand this.

Lesson 9: Semaphores and Rate Limiting — Bounded async concurrency

Wed, 22 Jan 2025 15:08:36 +0000

I once crashed a third-party API by spawning 10,000 concurrent requests from an async Rust service. The code was correct — every request completed (eventually). But the API’s rate limiter kicked in after 50 concurrent connections, and we got IP-banned for two hours.

The fix was a single type: tokio::sync::Semaphore. Five lines of code turned “as fast as possible” into “at most N at a time.”

What’s a Semaphore?

A semaphore is a counter with a maximum value. You acquire a permit before doing work, and release it when you’re done. If all permits are taken, acquiring blocks (yields) until one becomes available.

Lesson 8: Async Mutexes — tokio::sync::Mutex vs std

Mon, 20 Jan 2025 07:15:44 +0000

“Should I use tokio::sync::Mutex or std::sync::Mutex in async code?” I’ve seen this question in every Rust Discord server, every forum, every team Slack. And the answer most people give — “always use the async one in async code” — is wrong.

The real answer depends on how long you hold the lock and whether you need to .await while holding it. Get this wrong and you’ll either deadlock your runtime or tank your performance.

Lesson 7: Async Channels — tokio::sync::mpsc

Sat, 18 Jan 2025 13:52:28 +0000

The first real async service I built had a classic architecture: an HTTP handler receives a request, puts work on a queue, a background worker processes it, and the result gets sent back. In Go, this is channels all day. In async Rust, it’s also channels — but you’ve got four different kinds to choose from, and picking the wrong one leads to subtle bugs.

This lesson covers all four of Tokio’s channel types and when to reach for each one.

Lesson 6: Streams — Async iterators

Thu, 16 Jan 2025 10:31:07 +0000

Regular iterators give you one value at a time, synchronously. Futures give you one value, asynchronously. Streams give you multiple values, asynchronously. It’s the obvious combination, and once you start using them, you’ll wonder how you ever processed sequences of async data without them.

I first needed streams when building a log aggregator. I had dozens of log sources, each producing lines at their own pace. I needed to merge them, filter them, and process them in real time. Without streams, that code was a tangled mess of channels and select loops. With streams, it was a pipeline.

Lesson 5: tokio::select! — Racing futures

Tue, 14 Jan 2025 16:19:55 +0000

A couple months ago I was building a WebSocket handler that needed to do three things simultaneously: read from the socket, check a shutdown signal, and send periodic heartbeats. With join!, I’d need all three to complete. But I didn’t want them all to complete — I wanted to react to whichever one happened first.

That’s what select! does. It races multiple futures and gives you the result of the winner. The losers are dropped.

Lesson 4: Spawning Tasks and JoinHandles — Concurrent work units

Sun, 12 Jan 2025 08:44:19 +0000

I remember the exact moment async Rust “clicked” for me. I was building an API aggregator that needed to call five different services. My first version awaited them sequentially — 2 seconds total. Then I spawned them as concurrent tasks — 400ms. Same work, 5x faster, and I didn’t need to think about threads, locks, or shared state.

But spawning tasks isn’t free, and JoinHandle has some sharp edges that nobody warned me about. This lesson covers the patterns you’ll use every day.

Lesson 3: Tokio — The runtime that powers async Rust

Fri, 10 Jan 2025 11:05:33 +0000

Every async Rust tutorial shows you #[tokio::main] in the first example and then moves on like that’s totally self-explanatory. It’s not. That macro hides a lot of important decisions, and if you don’t understand what it’s doing, you’re going to make some costly mistakes.

I once spent an entire day debugging a deadlock that happened because I was using the single-threaded runtime without realizing it. One #[tokio::main(flavor = "current_thread")] vs the default, and my entire application’s behavior changed. That shouldn’t surprise anyone — but it surprised me.

Lesson 2: The Future Trait — What .await actually does

Wed, 08 Jan 2025 14:37:42 +0000

After I understood the mental model from lesson 1, my next question was obvious: what happens mechanically when I write .await? I could accept “it’s a state machine” as a hand-wave, but I wanted to see the gears turning.

Turns out, the answer is beautifully simple once you strip away the syntax sugar. The entire async machinery in Rust boils down to one trait, two types, and a contract.

The Future Trait, For Real This Time

use std::pin::Pin;
use std::task::{Context, Poll};

pub trait Future {
 type Output;
 fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

Three things to note:

Lesson 1: Async Mental Model — Futures, not threads

Mon, 06 Jan 2025 09:22:14 +0000

I spent three weeks writing async Rust code that compiled, ran, and produced correct results — while having absolutely no idea what was actually happening. I was copy-pasting async fn, slapping .await on things, and hoping for the best. Sound familiar?

The problem wasn’t syntax. The problem was that I was thinking about async Rust the same way I thought about threads in Go or Java. And that mental model is wrong for Rust.