Pipeline

Let’s be honest: you’ve been tempted. You see a function that takes a string and returns an int, and a little voice in your head whispers, “What if we need to handle other types later? We should make this generic now.” That voice is your inner architect, and while its intentions are noble, it’s often your enemy. In Go, the most powerful design tool is often the concrete type, and the most common design mistake is abandoning it too soon in favor of needless abstraction.

Right, let’s talk about table-driven design. You’ve probably already used this pattern without knowing its fancy name. It’s the moment you realize you’re about to write your fifth if or case statement for what is essentially the same operation, just on different data, and you scream “there has to be a better way!” There is. It’s called table-driven design, and it’s embarrassingly simple. Instead of a sprawling chain of conditional logic, you define your behavior in a data structure—a table—and then you write one, single, elegant loop to process it. The logic is decoupled from the data, which means your code becomes more readable, more testable, and infinitely easier to change and extend. It’s the difference between hand-carving each piece of a model and using a 3D printer. One is artisanally painful; the other is brilliantly efficient.

Right, let’s talk about errgroup. You’ve been there: you have a handful of goroutines doing work, and you need to wait for them all to finish, but if any of them fails, you want to cancel the whole operation immediately. You could roll this yourself with a sync.WaitGroup, some channels, and a context.Context for cancellation, but you’d be writing the same boilerplate for the tenth time this week. Stop that. The fine folks at the Go team felt your pain and gave us golang.org/x/sync/errgroup. It’s essentially a WaitGroup that understands errors and context.

Right, let’s talk about semaphores. You’ve probably heard the term in operating systems or concurrent programming—it’s a fancy word for a counter that controls access to a finite number of resources. In Go, we don’t have a semaphore package in the standard library. Why? Because we can build a perfectly good one, a weighted one at that, in about three lines of code using a buffered channel. It’s one of those elegant “less is more” designs that makes you appreciate Go’s simplicity, even if you occasionally want to throw your keyboard at it.

Right, the Worker Pool pattern. You’ve hit that beautiful moment in your Go journey where you realize that just slapping a go keyword in front of every function call is a fantastic way to trigger a cascading failure, get rate-limited into the next decade, or simply melt your machine’s CPU. Congratulations! Welcome to the big leagues, where we think about bounding our concurrency instead of just unleashing it like a herd of cats.

Right, let’s talk about the pipeline pattern. You’ve probably got some data that needs a series of operations performed on it: fetch it, process it, filter it, transform it, store it. You could write one big, gnarly function that does all of that. But then you’d have a monster that’s impossible to test, reason about, or modify without breaking three other things. We’re better than that. The pipeline pattern is our escape hatch. We break that big process into discrete stages, each a separate goroutine, connected by channels. Data flows in one end, gets worked on, and flows out the other. It’s like an assembly line for your data, and it’s one of the most elegant ways to structure concurrent programs in Go. It makes your code modular, testable, and frankly, a joy to work with.

Right, let’s settle this. You’re about to configure a thing in Go—a server, a client, a database connection, some complex monstrosity you built. You quickly realize your constructor function is getting out of hand. It’s starting to look like NewThing(host string, port int, timeout time.Duration, enableLogging bool, maxRetries int, name string, fluxCapacitorCapacity float64) and it’s an unreadable, unmaintainable mess. You need a pattern. You’ve likely seen two: the Options Struct and Functional Options. Let’s break down why both exist and when to use which.

Let’s be honest: you’ve seen this before. You’re trying to create a Thing, and its constructor is a nightmare. You either have a function with fourteen arguments where the last nine are almost always the same, or you’ve got a dozen different NewThingWithXAndYButNotZ constructors. It’s a mess. It’s un-Go-like. It’s exactly the kind of ceremony we’re trying to avoid. Enter the Functional Options pattern. This is one of those patterns that looks like magic the first time you see it, but once you understand it, you’ll wonder how you ever lived without it. The core idea is simple: we pass a variadic slice of functions to our constructor, and each function operates on the struct we’re building. It’s configuration with a functional flair.