The yield from expression, introduced in Python 3.3, is a powerful syntactic sugar that significantly simplifies the process of delegating work to a sub-generator. At its core, it handles the tedious boilerplate of iterating over a sub-generator manually, while also correctly propagating values and exceptions, making generator composition both more efficient and more readable.

The Problem yield from Solves

Before yield from, if a generator needed to yield all values from another iterable (like a generator), one would use a for loop:

def old_style_generator():
    for item in range(3):
        yield item
    for item in ['a', 'b', 'c']:
        yield item

# Usage
for value in old_style_generator():
    print(value, end=' ')  # Output: 0 1 2 a b c

While functional, this becomes cumbersome and less performant when dealing with complex generator delegation. More importantly, it fails to properly handle the full generator protocol, which includes sending values into the generator using .send(), throwing exceptions with .throw(), and closing it with .close().

Basic Syntax and Operation

The yield from expression takes a single argument: an iterable object. Its most basic use is to yield every value produced by that iterable.

def new_style_generator():
    yield from range(3)
    yield from ['a', 'b', 'c']

# Usage
for value in new_style_generator():
    print(value, end=' ')  # Output: 0 1 2 a b c

The output is identical, but the code is cleaner. However, the true power of yield from lies beneath the surface, in its handling of the two-way communication between the caller, the delegating generator, and the sub-generator.

The Full Delegation Protocol

yield from establishes a direct channel between the caller and the sub-generator. This means:

  1. Values Sent In: When the caller uses generator.send(value), that value is passed directly to the sub-generator if the delegation is active. If the sub-generator is already exhausted, the value is sent to the delegating generator itself.
  2. Exceptions Thrown In: generator.throw(exc_type) is similarly forwarded to the sub-generator. If the sub-generator handles it and yields more values, the delegation continues. If it propagates a StopIteration or another exception, it is felt by the delegating generator.
  3. Return Values: A critical feature is that yield from expresses a value. When the sub-generator terminates, its return value (carried by the StopIteration exception) becomes the value of the yield from expression. This enables generators to return values, not just yield them.
def subgen():
    received = yield "Subgen ready"
    return f"Subgen received: {received}"

def delegating_gen():
    result = yield from subgen()  # The RETURN value of subgen() is captured here
    yield f"Delegating gen got: {result}"
    return "Delegating gen done"

# Create the generator
gen = delegating_gen()

# Prime the subgen, get its first yield
print(next(gen))  # Output: Subgen ready

# Send a value directly to the subgen
try:
    print(gen.send("Hello!")) # Output: Delegating gen got: Subgen received: Hello!
except StopIteration as e:
    print(e.value)  # Output: Delegating gen done

Why This Matters: Coroutines and AsyncIO

This two-way delegation protocol is the fundamental mechanism that makes native async/await syntax possible in modern Python. An async def function is a coroutine, and the await keyword is conceptually similar to yield from. It suspends the current coroutine and awaits the result of another awaitable object (e.g., another coroutine), effectively creating the same direct channel for execution flow and data passage. Understanding yield from is therefore crucial to understanding the underlying mechanics of asynchronous programming in Python.

Common Pitfalls and Best Practices

  1. Not Handling the Return Value: A common mistake is to use yield from without capturing its value. If the sub-generator’s return value is important, you must assign the result of yield from to a variable.
  2. Exception Handling: Exceptions propagate through the yield from link. If you need to handle exceptions specifically from the sub-generator within the delegating generator, you must wrap the yield from expression in a try...except block.
  3. Priming: Remember that all generators, including the sub-generator, must be “primed” by a next() call to advance them to their first yield statement. yield from handles this priming automatically, which is a key advantage over manual iteration. The initial next() call on the delegating generator automatically primes the sub-generator.
  4. Use with Any Iterable: While most powerful with generators, yield from works with any iterable (e.g., lists, strings). However, sending values or throwing exceptions into a non-generator iterable (which has no send() or throw() methods) will result in an AttributeError.