62.4 Motor: Async MongoDB Driver
Right, so you’ve decided to use MongoDB. I’m not here to judge your life choices. Maybe you need to store deeply nested, unstructured data that would give a relational database planner a nervous breakdown. Maybe you’re just prototyping and want the flexibility. Whatever the reason, if you’re in Python’s asyncio event loop, you’re not going to use the standard PyMongo driver. It’s synchronous. Blocking. A total party pooper for your beautifully concurrent architecture.
This is where Motor comes in. Think of Motor as PyMongo’s cooler, non-blocking sibling. It’s not a complete rewrite; it’s a thin asynchronous wrapper on top of PyMongo’s core. This is its genius. It leverages all the battle-tested logic of PyMongo but exposes it with async/await syntax, letting your event loop breathe easy while it waits for network calls to the database.
The First Thing You’ll Do (And Probably Screw Up)
Connecting looks almost identical to PyMongo, but with a crucial, awaitable step.
import motor.motor_asyncio
# This part is non-blocking. It just creates the client object.
client = motor.motor_asyncio.AsyncIOMotorClient("mongodb://localhost:27017")
# This part, however, is often necessary and is blocking.
# It forces a connection to check if the server is available.
# Do this in a startup routine, not in the hot path of a request.
try:
await client.server_info()
except Exception as e:
print(f"Uh oh, can't connect to the database: {e}")
The common pitfall? People slap that server_info() call right in the middle of their application startup and then wonder why their whole app hangs on boot if the database is down. You need to handle this gracefully, probably with a retry mechanism. Motor itself is lazy; it won’t actually establish connections until you make your first operation. This is mostly good, but it means your first request might fail if the DB is unreachable, hence the explicit connection check.
How Motor Actually Works (It’s Clever)
Motor doesn’t use a thread pool to fake asynchronicity. That would be cheating. Instead, it plugs into PyMongo’s core and, whenever you issue a command like collection.find_one(), it schedules the network I/O onto a background thread managed by the driver itself (using the greenlet or awaitable extensions in PyMongo). It then returns an asyncio.Future for you to await. This keeps the I/O out of your main thread while still using the efficient, standard MongoDB C driver underneath.
The takeaway: You get true non-blocking I/O without the overhead of a giant thread pool. It’s the best of both worlds.
The Absolute Biggest “Gotcha”: Cursors
This is where everyone gets tripped up. In synchronous code, you do this:
for doc in my_collection.find({"status": "active"}):
print(doc) # This blocks until each doc is fetched
The naive async translation is wrong:
# WRONG! This will not work as you expect.
async for doc in collection.find({"status": "active"}):
print(doc)
Why? Because find() doesn’t return the data; it returns an AsyncIOMotorCursor object. And you can’t directly iterate over it. You have to explicitly ask for results. The correct ways are:
# Method 1: Pre-fetch everything into a list. Good for small results.
docs = await collection.find({"status": "active"}).to_list(length=100)
# Method 2: Iterate properly by calling `next()` on the cursor.
cursor = collection.find({"status": "active"})
while await cursor.fetch_next:
doc = cursor.next_object()
print(doc)
# Method 3: The idiomatic async for loop (which is syntax sugar for Method 2).
async for doc in collection.find({"status": "active"}):
print(doc)
Yes, that last one does work. I told you the first example was wrong, and now I’m showing you it’s right. Welcome to programming. The key is that Motor’s async for is specially implemented to handle the asynchronous iteration of the cursor behind the scenes. The pitfall is assuming all for loops become async for automatically. They don’t. You must use the async for syntax with Motor cursors.
Sessions and Transactions (Where It Gets Real)
Want ACID guarantees in your document store? MongoDB added multi-document transactions, and Motor supports them with async sessions.
async with await client.start_session() as session:
async with session.start_transaction():
await collection.insert_one({"name": "Alice", "balance": 100}, session=session)
await collection.update_one(
{"name": "Bob"},
{"$inc": {"balance": -100}}},
session=session
)
# This will all commit, or none of it will.
The critical best practice here is to pass the session object to every operation within the transaction. Forget it once, and that operation happens outside the transactional context, which is a fantastic way to introduce horrifying data inconsistencies. Motor can’t save you from yourself here.
Best Practices from the Trenches
- Singleton Client: Your
AsyncIOMotorClientinstance is designed to be created once and reused for the entire lifetime of your application. It manages a built-in connection pool. Creating a new client for every operation is a great way to drown your database in connections. - Define Your Indexes Async: You can’t block on app startup. So use Motor to create your indexes asynchronously too:
await my_collection.create_index("email", unique=True). - Watch Your Timeouts: The client has default timeouts, but networks are messy. Always wrap your database calls in
asyncio.wait_for()or similar if you need to enforce a stricter SLA to prevent a slow DB from taking down your entire service. - Aggregation Pipelines: They work exactly like in PyMongo, just with
await. The complexity is in crafting the pipeline, not the Motor code:result = await collection.aggregate([{"$match": {...}}, {"$group": {...}}]).to_list(length=None).
Motor is a brilliantly engineered driver that does one job and does it exceptionally well. It gets out of your way and lets you write clean, asynchronous code while leveraging the full power—and quirks—of MongoDB. Just remember the cursor thing. Seriously, everyone forgets the cursor thing first time.