The for loop is the workhorse of iteration in Python. At its most basic level, it allows you to execute a block of code repeatedly, once for each item in a sequence (like a list, tuple, or string) or, more generally, for each item provided by an iterable. Understanding the for loop is therefore inseparable from understanding the concept of iterables and the iterable protocol that underpins it.

The Iterable Protocol: The Engine Behind the for Loop

Contrary to what it may seem, the for loop does not directly work with sequences by index. Instead, it operates on a more fundamental level through a standardized process called the iterable protocol. This protocol defines how any object in Python can be looped over. The process involves two key concepts: the iterable and the iterator.

An iterable is any object capable of returning its elements one at a time. It does this by implementing the __iter__() method, which must return an iterator object. An iterator is the object that actually performs the iteration. It implements the __next__() method, which returns the next item from the iterable. When there are no more items, __next__() raises the built-in StopIteration exception to signal the loop to terminate.

The for loop automates this protocol entirely. When you write for item in my_iterable:, Python performs the following steps behind the scenes:

  1. It calls iter(my_iterable), which internally calls my_iterable.__iter__(), to obtain an iterator object.
  2. It repeatedly calls next(iterator), which internally calls iterator.__next__(), to get the next value.
  3. It assigns each value returned by next() to the target variable (item).
  4. It catches the StopIteration exception when it is raised and gracefully ends the loop.
# Manual demonstration of the iterable protocol using a list
my_list = ['a', 'b', 'c']

# Step 1: Get an iterator from the iterable (list)
iterator = iter(my_list)

# Step 2: Repeatedly call next() on the iterator
print(next(iterator))  # Output: a
print(next(iterator))  # Output: b
print(next(iterator))  # Output: c
# print(next(iterator))  # This would raise StopIteration

Common Iterable Types

Most container types in Python are iterables. This includes sequences like list, tuple, str, and range, as well as non-sequence collections like dict, set, and file objects.

# Iterating over a string (a sequence of characters)
for char in "Hello":
    print(char)

# Iterating over a dictionary (iterates over keys by default)
my_dict = {'a': 1, 'b': 2}
for key in my_dict:
    print(key, my_dict[key])

# Using .items() to iterate over key-value pairs (also an iterable)
for key, value in my_dict.items():
    print(f"Key: {key}, Value: {value}")

# Iterating over a file object (iterates line by line)
# with open('data.txt') as file:
#     for line in file:
#         print(line.strip())

The range() Function: A Special Iterable

The range() function is a cornerstone of for loops, often used to execute a block of code a specific number of times. It’s crucial to understand that range is not a list; it is a specialized, memory-efficient iterable object that generates numbers on the fly. range(5) doesn’t create a list [0, 1, 2, 3, 4] in memory; it creates a lightweight object that produces those numbers one by one as the loop requests them. This makes it extremely efficient even for very large ranges.

# This loop is efficient even with a very large number
for i in range(1000000):
    # do something with i
    pass

# Contrast with the less efficient, older method (in Python 2, range() did return a list)
# for i in list(range(1000000)): # This would create a huge list first
#     pass

Pitfalls and Best Practices

A common pitfall arises when you try to modify a mutable iterable, like a list, while you are iterating over it. This can lead to unexpected behavior because the iterator may not account for the changes, potentially skipping elements or causing errors.

# Problem: Modifying a list during iteration
numbers = [1, 2, 3, 4]
for num in numbers:
    if num == 2:
        numbers.remove(num) # Modifying the list being iterated over
    print(num)
# Output might be: 1, 2, 4. The value '3' might be skipped.

# Solution 1: Iterate over a copy of the list
for num in numbers[:]:
    if num == 2:
        numbers.remove(num)

# Solution 2: Create a new list (often more readable)
new_numbers = []
for num in numbers:
    if num != 2:
        new_numbers.append(num)

Another best practice is to use enumerate() when you need both the element and its index within the loop. This is more Pythonic and readable than the traditional pattern of using for i in range(len(sequence)).

fruits = ['apple', 'banana', 'mango']

# Less Pythonic
for i in range(len(fruits)):
    print(f"Index {i}: {fruits[i]}")

# Preferred and more Pythonic
for index, fruit in enumerate(fruits):
    print(f"Index {index}: {fruit}")

Finally, it’s important to remember that iterators are exhaustible. Once an iterator has raised StopIteration, it is consumed. Further calls to next() will continue to raise StopIteration. This is why you can only iterate over a zip or map object once. If you need to iterate multiple times, you must either create a new iterator (by calling iter() again on the original iterable) or use a materialized collection like a list.