28.7 Connection Pool Tuning: MaxOpenConns, MaxIdleConns, ConnMaxLifetime
Right, let’s talk about connection pools. You’ve probably already used sql.Open() and thought, “Great, I have a database handle.” What you actually have is a handle to a pool of connections, managed by the database/sql package. This is fantastic—it saves you from the monumental pain of managing connections yourself. But like any powerful tool, it has knobs. And if you ignore these knobs, your application will eventually, and spectacularly, fall over in production at 3 AM. I’ve been there. Let’s not do that.
The default settings for this pool are… aggressively mediocre. They’re designed not to break a simple, low-traffic demo app. Your production application is neither simple nor low-traffic. You need to tune these settings based on your actual database’s capacity and your application’s behavior. The three main levers you have are MaxOpenConns, MaxIdleConns, and ConnMaxLifetime.
The Defaults Are Lying to You
Let’s see what we’re working with by default. The sql.DB object hides its stats, but you can actually peek under the hood.
db, _ := sql.Open("postgres", connString)
// Set some sane limits for our example. The defaults are:
// MaxOpenConns: 0 (unlimited!)
// MaxIdleConns: 2 (why?!)
// ConnMaxLifetime: 0 (forever)
// ConnMaxIdleTime: 0 (forever)
db.SetMaxOpenConns(25)
db.SetMaxIdleConns(25)
db.SetConnMaxLifetime(5 * time.Minute)
db.SetConnMaxIdleTime(1 * time.Minute)
// Later, when you're debugging why your app is slow:
stats := db.Stats()
fmt.Printf("Open Connections: %d (InUse: %d, Idle: %d)\n",
stats.OpenConnections, stats.InUse, stats.Idle)
fmt.Printf("WaitCount: %d, WaitDuration: %v\n",
stats.WaitCount, stats.WaitDuration)
The most egregious default is MaxOpenConns: 0, which means “unlimited.” This is a trap. If you don’t set this, your application can—and will—attempt to open more connections than your database can handle. You’ll drown your poor database in connections, leading to errors, lock contention, and a truly miserable time for everyone. The first rule is always, always set MaxOpenConns to a sane value below your database’s max_connections setting.
MaxOpenConns: Your Most Important Setting
This is the maximum number of open connections to the database allowed at one time. Think of it as the concurrency limit for your database operations.
- Set it too low: Your application will spend time waiting for a connection to free up. You’ll see high
WaitDurationin the stats. This adds latency. - Set it too high: You overwhelm the database. Each connection (even an idle one) consumes memory on the database server. Too many connections lead to excessive context switching and can grind the database to a halt.
How to set it? A good starting point is something like (number_of_cpu_cores * 2) + 1, but that’s a vast oversimplification. The real answer is: know your database’s limits. If your PostgreSQL max_connections is 100, set this to 80 or 90, leaving room for other clients (admin tools, cron jobs, etc.). Then, use the stats from db.Stats() to see if you have a high WaitCount. If you do, nudge it up. If your database is struggling, nudge it down.
MaxIdleConns: Don’t Be a Bad Guest
This is the maximum number of connections that can be sitting idle in the pool. An idle connection is one not currently in use, but kept open for immediate reuse.
- Set it too low: You cause churn. If you have 10 requests per second and
MaxIdleConnsis set to 2, you’re constantly closing and re-opening connections. This is expensive. - Set it too high: You’re a bad guest. You’re hogging database resources (memory, mostly) that you aren’t using. If you have 100 idle connections and your app is traffic is spiky, you’re preventing other applications from using those connections.
The best practice is to set MaxIdleConns equal to, or slightly lower than, MaxOpenConns. This ensures your pool can maintain a warm set of connections ready to handle your normal traffic load without causing needless churn.
ConnMaxLifetime: The Preventative Reboot
This is the maximum amount of time a connection can be alive. After this duration, the connection will be closed, even if it’s idle.
Why would you ever do this? Two main reasons:
- Network Infrastructure: Load balancers, proxies, and firewalls sometimes silently drop long-lived TCP connections. This setting ensures your connections get recycled before that happens.
- Database State: If your database server is restarted for maintenance, connections from the old instance become invalid. A finite lifetime helps eventually clear these out.
A value of 30 minutes to an hour is often reasonable. You don’t want it too short, or you’re back to causing connection churn. You can also use ConnMaxIdleTime to be more granular, saying “a connection can be idle for X amount of time before being closed,” which is often a more elegant solution.
The Golden Rule: Contexts Are Your Escape Hatch
No matter how well you tune your pool, there will be traffic spikes, slow queries, or network hiccups. Your code must be resilient. This is where context.Context is non-negotiable.
// This is how you avoid cascading timeouts and connection stalls.
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel() // Always call cancel to free resources!
// The query will now bail after 3 seconds, freeing the connection
// back to the pool for someone else to use.
var result string
err := db.QueryRowContext(ctx, "SELECT some_slow_function()").Scan(&result)
if err != nil {
if errors.Is(err, context.DeadlineExceeded) {
log.Println("Query took too long! Aborted.")
}
// handle error
}
If you don’t use contexts with timeouts, a single slow query or a stuck TCP packet can hold a connection hostage indefinitely, slowly draining your pool until your entire application is just… waiting. It’s a silent, brutal failure mode. Always use Context methods and always set a deadline. Your pool will thank you.