Plugins on

Lesson 2: gRPC-Based Plugin Architecture — How Terraform and Vault do plugins

Thu, 21 Nov 2024 00:00:00 +0000

After I understood how hashicorp/go-plugin worked with net/rpc, the next question was obvious: how does Terraform manage hundreds of community-contributed providers, written by different teams, evolving on different schedules, sometimes in languages other than Go? The answer is gRPC. Terraform’s provider protocol is a protobuf schema, communicated over the same subprocess-plus-RPC architecture from Lesson 1 — but with gRPC instead of net/rpc. That substitution buys you schema evolution, multi-language support, streaming, and a strongly-typed IDL. This lesson shows you how to build a plugin system using that pattern.

Lesson 1: Go Plugins and hashicorp/go-plugin — Extending Go apps without recompiling

Mon, 26 Aug 2024 00:00:00 +0000

The first time I needed a plugin system in Go, I went straight to plugin.Open in the standard library. Ten minutes later I was reading about RTLD flags and shared library loading on Linux, discovering that plugins had to be compiled with the same Go toolchain version as the host, and learning that once a plugin was loaded it could not be unloaded. My enthusiasm dropped sharply. Then I found hashicorp/go-plugin and understood that the Go community had largely agreed: real plugin systems in Go should run plugins as separate processes communicating over RPC, not as shared libraries in the same process.