27.3 http.Client: Timeout, CheckRedirect, and CookieJar
Right, let’s talk about the http.Client. This is where we graduate from making simple, one-off requests with http.Get to building a proper, stateful, and frankly, adult HTTP client. The default client is a bit of a naive tourist; it works for a quick trip, but it doesn’t know the local customs, gets lost easily, and has no concept of time. We’re going to fix that.
The magic of http.Client is in its three main levers of power: Timeout, CheckRedirect, and CookieJar. You configure these when you create the client, and then they handle the messy, repetitive work for you automatically. It’s like hiring a very efficient, slightly pedantic intern.
The Timeout That Isn’t Just One Thing
Here’s the first thing the documentation doesn’t scream loudly enough: Client.Timeout is a total request deadline. It covers everything: DNS lookup, TCP connection, TLS handshake, request writing, response reading, and all redirects. It’s the guillotine blade hanging over the entire operation. This is incredibly convenient, but also a bit blunt.
If you need more granular control—say, you want to allow a long time for the connection to establish but then quickly timeout if the response is slow—you have to roll your own using net.Dialer and context.Context. But for 90% of cases, the total timeout is exactly what you want.
package main
import (
"net/http"
"time"
)
func main() {
// This client will give up on the entire operation after 5 seconds.
// This includes spending 4.9 seconds waiting for a TCP connection.
client := &http.Client{
Timeout: 5 * time.Second,
}
resp, err := client.Get("https://httpbin.org/delay/10") // This endpoint waits 10 seconds to respond
if err != nil {
// This will *always* error with "context deadline exceeded (Client.Timeout exceeded...)"
panic(err)
}
defer resp.Body.Close()
// ... handle response ...
}
The pitfall? Not setting one. The default client has no timeout. Your goroutine could hang forever, leaking resources and slowly driving your service to its knees. Always, always set a timeout.
Taming the Redirect Loop
By default, the client will happily follow up to 10 redirects. This is usually sensible, but sometimes you need to intervene. This is what CheckRedirect is for. It’s a policy function you provide that gets called before each redirect.
Why would you care? Maybe you need to log them, or you want to stop after a certain number (the default policy does this for you), or, crucially, you need to handle sensitive headers like Authorization. The HTTP spec says that during a redirect to a different domain, you should strip the Authorization header to avoid leaking credentials to a third party. The Go client does this for you. But what if you’re redirecting within your own domains? The default policy is still to strip it. This is one of those “questionable choices” – it’s secure but can be infuriating.
func main() {
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
// Log the redirect
fmt.Printf("Redirecting to: %s\n", req.URL.String())
// Let's be more specific than the default. Allow 10 redirects total.
if len(via) >= 10 {
return fmt.Errorf("stopped after 10 redirects")
}
// If we're redirecting to our own trusted domain, let's keep the auth header.
if req.URL.Hostname() == "internal-api.example.com" {
// The req.Headers are already set for the upcoming request.
// The default policy has already stripped the Authorization header.
// So we have to re-add it if we have it.
if len(via) > 0 && via[0].Header.Get("Authorization") != "" {
req.Header.Set("Authorization", via[0].Header.Get("Authorization"))
}
}
return nil // Return nil to allow the redirect to proceed.
},
}
// This request will now log and potentially carry auth internally.
resp, err := client.Get("https://external-api.example.com/start")
if err != nil {
panic(err)
}
defer resp.Body.Close()
}
The Cookie Jar: Your Stateful Friend
Without a CookieJar, your client is like a goldfish. It gets a Set-Cookie header from the server, says “neat!”, and immediately forgets it. The next request starts from scratch. For anything that requires login or session state, this is useless.
The CookieJar interface is your solution. It’s a simple store for cookies that automatically appends relevant cookies to outgoing requests and stores new ones from incoming responses. The best part? The net/http/cookiejar package gives you a ready-to-use, in-memory implementation.
package main
import (
"net/http"
"net/http/cookiejar"
)
func main() {
// Create a new cookie jar. This is a pointer, so we need to handle the error.
jar, err := cookiejar.New(nil)
if err != {
panic(err)
}
client := &http.Client{
Jar: jar, // Attach the jar to the client
}
// First request: log in. The server will set a session cookie.
req, _ := http.NewRequest("GET", "https://example.com/login", nil)
req.SetBasicAuth("user", "pass")
resp, err := client.Do(req) // The jar is now active for this client.
if err != nil {
panic(err)
}
resp.Body.Close()
// Second request: the jar AUTOMATICALLY sends the session cookie received from /login.
// You don't have to think about it.
resp2, err := client.Get("https://example.com/profile")
if err != nil {
panic(err)
}
defer resp2.Body.Close()
// Now the client is "logged in" for its entire lifetime.
}
The big pitfall here is that the default cookiejar is in-memory. If you create a new client for every request (a common anti-pattern), you lose all your cookies. The client is the session. You need to create it once and reuse it. For persistence across restarts, you’d need to implement your own Jar that saves to disk or a database, but that’s a story for another day.
So there you have it. A configured client isn’t just about making requests; it’s about defining a resilient, stateful, and policy-driven agent to navigate the chaos of the web on your behalf. Now go build something that doesn’t hang.