44.5 Go 1.23: Iterators and the iter Package
Right, let’s talk about Go 1.23. This is the release where the core team finally decided to give us a proper, language-backed way to handle sequences of data. We’ve been faking it with slices and channels for over a decade, and while that worked, it was a bit like using a screwdriver to hammer a nail—it gets the job done, but you feel a little silly doing it and everyone watching knows there’s a better tool for the job.
The new iter package and the associated language changes are that better tool. At its heart, it introduces two new concepts: iterators and sequences. An iterator is any type that has a Next method. A sequence is a function that returns an iterator. This sounds simple because it is simple, and that’s the beauty of it. The magic happens when you pair this with the new yield keyword and the range loop enhancements.
The Core: iter.Pull and the yield Keyword
The most common way you’ll create a sequence is by using iter.Pull. This function takes another function that contains yield statements and turns it into a sequence. This is where the compiler magic happens.
package main
import (
"fmt"
"iter"
)
func main() {
// Generate a sequence of the first 5 integers
seq := func(yield func(int) bool) {
for i := 0; i < 5; i++ {
// Yield the value to the consumer.
// If the consumer stops iterating, yield returns false and we break.
if !yield(i) {
return
}
}
}
// Now range over the sequence
for v := range seq {
fmt.Println(v) // Prints 0, 1, 2, 3, 4
}
}
Wait, hold on. That seq function looks weird, right? Its signature is func(func(int) bool). This is the standard form for a pull sequence. The inner function you pass to iter.Pull is provided this yield function by the runtime. You call yield to send a value out. The bool it returns is your signal from the consumer: false means “stop sending me things, I’m done,” which is crucial for preventing leaks and wasted computation in more complex generators.
But writing that manual function is a bit verbose. This is where iter.Pull does the heavy lifting for you. The above example is better written as:
func countTo(n int) iter.Seq[int] {
return func(yield func(int) bool) {
for i := 0; i < n; i++ {
if !yield(i) {
return
}
}
}
}
func main() {
for v := range countTo(5) {
fmt.Println(v)
}
}
iter.Pull just gives you a more convenient way to write this, handling the boilerplate. The real power is in yield. It lets you write a generator that lazily produces values. You’re not building a slice in memory; you’re producing values on-demand and pausing the function until the next one is requested. This is a game-changer for large or infinite sequences.
Ranging Over Functions
The most visible change is that you can now range over a function that returns an iter.Seq. This is a brilliant bit of syntactic sugar. The type iter.Seq[int] is literally just a type alias for func(func(int) bool). The range loop knows that if you give it a function of this signature, it should call it, providing the yield function and iterating until yield returns false.
This is why for v := range countTo(5) works. It’s clean, intuitive, and feels like it was always part of the language.
The Other Half: iter.Pull2 and Error Handling
Now, the designers faced a problem. A huge percentage of real-world iterations can fail. Reading from a network socket? Parsing lines in a file? Those operations can error. How do you handle that? You could yield a struct {V T; Err error}, but that’s clunky.
Their solution: iter.Pull2. This gives you a second yield function for an error. The sequence function now has the signature func(yield func(T) bool, yieldErr func(error) bool).
func readLines(filename string) iter.Seq2[string, error] {
return func(yield func(string) bool, yieldErr func(error) bool) {
f, err := os.Open(filename)
if err != nil {
yieldErr(err)
return
}
defer f.Close()
scanner := bufio.NewScanner(f)
for scanner.Scan() {
if !yield(scanner.Text()) {
return
}
}
if err := scanner.Err(); err != nil {
yieldErr(err)
}
}
}
func main() {
for line, err := range readLines("foo.txt") {
if err != nil {
fmt.Printf("Error: %v\n", err)
break // Always break on error!
}
fmt.Println(line)
}
}
This is… a choice. It’s powerful, but now your range loop has two variables. It’s a trade-off. The clear best practice is that the moment you get an error in the consumer, you must break. The sequence function has no other way of knowing something went wrong on your end. Failing to break could lead to you ignoring errors and continuing with garbage data.
The Rough Edges and Best Practices
This is new, so there are quirks. First, you can’t go range over a sequence. The semantics of yielding from one goroutine and consuming in another are complex and not yet defined. If you need that, you’re back to using channels for now.
Second, be hyper-aware of resource management. The defer in our readLines example is critical. The sequence function runs synchronously within the range loop. When the loop breaks (or finishes), the sequence function returns, and your defer runs. This is actually great—it makes resource cleanup much more straightforward than with channels.
Finally, remember that these are pull iterators. They’re lazy. Nothing happens until the consumer starts the range loop. This is generally what you want, but it means any errors during setup (like the os.Open above) can’t be checked until you actually start iterating. There’s no way to validate a sequence before you use it, so design your APIs accordingly.
This isn’t just a new package; it’s a fundamental shift in how we think about traversing data in Go. It’s more efficient, more expressive, and finally gives us an idiomatic way to handle sequences that aren’t just in-memory slices. It’s a long-awaited and very welcome addition.