In Python, every instance and class has a __dict__ attribute, which is a dictionary that stores its writable attributes. This mechanism is the primary way Python implements dynamic attribute storage for objects. Understanding __dict__ is crucial for comprehending how attribute lookup works, how memory is used, and how to perform advanced metaprogramming tasks.

The Instance __dict__

When you create an instance of a class, Python allocates a new dictionary to store instance-specific attributes. This is the __dict__ you access directly from the instance. It is the first place the interpreter looks during attribute lookup on an instance.

class Person:
    species = 'Homo sapiens'  # Class attribute

    def __init__(self, name):
        self.name = name      # Instance attribute

# Create an instance
alice = Person('Alice')

# The instance __dict__ contains instance-specific attributes
print(alice.__dict__)  # Output: {'name': 'Alice'}

# Adding a new instance attribute dynamically
alice.age = 30
print(alice.__dict__)  # Output: {'name': 'Alice', 'age': 30}

The __dict__ is why you can assign new attributes to instances so flexibly. The assignment self.name = name in the __init__ method is fundamentally equivalent to self.__dict__['name'] = name, though the former is the standard and recommended practice.

The Class __dict__

The class itself also has a __dict__ attribute. This dictionary stores the class’s attributes, which include methods, class variables, and other metadata like the __module__ name. It is important to note that the class __dict__ is a mappingproxy object, not a regular dictionary. This is a read-only wrapper introduced in Python 3.3 to prevent accidental direct mutations of the class namespace, encouraging the use of explicit attribute setting (cls.attr = value) or metaclasses for modification.

print(Person.__dict__)
# Output (approximate):
# {
#   '__module__': '__main__',
#   'species': 'Homo sapiens',
#   '__init__': <function Person.__init__ at 0x...>,
#   '__dict__': <attribute '__dict__' of 'Person' objects>,
#   '__weakref__': <attribute '__weakref__' of 'Person' objects>,
#   '__doc__': None
# }

# Trying to modify the class __dict__ directly will fail
try:
    Person.__dict__['new_attr'] = 'value'
except TypeError as e:
    print(f"Error: {e}")  # Output: Error: 'mappingproxy' object does not support item assignment

# The correct way to add a class attribute
Person.new_attr = 'value'
print(Person.new_attr)  # Output: value

How Attribute Lookup Works

The interaction between the instance __dict__ and the class __dict__ is the foundation of Python’s attribute resolution order. When you access instance.attr, the interpreter follows a specific chain:

  1. It first checks the instance’s __dict__ for the attribute name.
  2. If not found, it then checks the __dict__ of the class.
  3. If still not found, it proceeds to check the __dict__ of the parent classes in the Method Resolution Order (MRO). This process explains why instance attributes can override class attributes of the same name.
class MyClass:
    class_attr = "I'm a class attribute"

    def __init__(self, value):
        self.instance_attr = value

obj = MyClass("I'm an instance attribute")

# The instance finds 'instance_attr' in its own __dict__
print(obj.instance_attr)  # Output: I'm an instance attribute

# The instance does not have 'class_attr', so it is found in the class's __dict__
print(obj.class_attr)     # Output: I'm a class attribute

# If we add an instance attribute with the same name, it shadows the class attribute
obj.class_attr = "This shadows the class attribute"
print(obj.class_attr)     # Output: This shadows the class attribute
print(MyClass.class_attr) # Output: I'm a class attribute (unchanged)
print(obj.__dict__)       # Output: {'instance_attr': "...", 'class_attr': 'This shadows...'}

Memory Considerations and __slots__

Because each instance maintains its own dictionary, this provides immense flexibility but can come at a cost of memory consumption. For each instance, a new dictionary must be allocated, which can be significant if you are creating millions of objects.

To mitigate this, Python offers the __slots__ mechanism. By defining __slots__ in a class, you tell the interpreter to pre-allocate a fixed amount of space for the specified instance attributes instead of creating a __dict__ for each instance. This can lead to substantial memory savings.

class PersonWithSlots:
    __slots__ = ['name', 'age']  # Pre-defined instance attributes

    def __init__(self, name, age):
        self.name = name
        self.age = age

alice = PersonWithSlots('Alice', 30)
print(alice.name)  # Output: Alice
print(alice.age)   # Output: 30

# This instance no longer has a __dict__ by default
try:
    print(alice.__dict__)
except AttributeError as e:
    print(f"Error: {e}")  # Output: 'PersonWithSlots' object has no attribute '__dict__'

# Trying to assign an attribute not in __slots__ will fail
try:
    alice.occupation = 'Engineer'
except AttributeError as e:
    print(f"Error: {e}")  # Output: 'PersonWithSlots' object has no attribute 'occupation'

It is a critical best practice to understand that __slots__ is primarily an optimization tool, not a security feature. Its use cases are typically limited to large-scale applications where memory footprint is a proven bottleneck.

Common Pitfalls and Best Practices

A common pitfall arises from misunderstanding the difference between mutable and immutable class attributes. If a class attribute is a mutable object (like a list or dictionary), modifying it through one instance will affect all instances because the attribute is stored in the class’s __dict__, not the instance’s.

class Warehouse:
    inventory = []  # Mutable class attribute

    def __init__(self, name):
        self.name = name

    def add_item(self, item):
        self.inventory.append(item)  # This modifies the class's list

wh1 = Warehouse('North')
wh2 = Warehouse('South')

wh1.add_item('forklift')
print(wh2.inventory)  # Output: ['forklift'] - Surprise!

The best practice to avoid this is to initialize mutable attributes inside the __init__ method, ensuring they are created in the instance’s __dict__.

class CorrectWarehouse:
    def __init__(self, name):
        self.name = name
        self.inventory = []  # Instance-specific mutable attribute

    def add_item(self, item):
        self.inventory.append(item)

wh1 = CorrectWarehouse('North')
wh2 = CorrectWarehouse('South')
wh1.add_item('forklift')
print(wh1.inventory)  # Output: ['forklift']
print(wh2.inventory)  # Output: [] - As expected

Another best practice is to generally avoid directly manipulating __dict__ unless you have a specific, advanced need (e.g., dynamically bulk-assigning attributes). Direct assignment (obj.attr = value) is clearer, more maintainable, and properly triggers descriptors if they are present. Direct access to __dict__ is, however, incredibly useful for introspection, debugging, and metaprogramming, making it an indispensable tool for any advanced Python developer.