72.9 faulthandler: Diagnosing Crashes and Segfaults
Right, so you’ve written some Python. It’s beautiful, it’s elegant, and then—without warning—it exits. No traceback, no KeyboardInterrupt, just a sudden, silent return to the comforting glow of your terminal prompt. Or worse, it spits out Segmentation fault (core dumped) and mocks you from the history log.
This, my friend, is where your usual Python tools tap out. print() statements? Useless. The logging module? Never got the message. pdb? Didn’t even get to wake up. Your code has crashed in the C layers, far beneath the comfortable Python runtime where exceptions are raised and caught. This is the realm of dangling pointers, buffer overflows, and corrupted memory. And to diagnose this, you need a different kind of tool. You need faulthandler.
Think of faulthandler as the airbag in your car. You hope you never need it, but when you crash, it’s the only thing that’s going to give you a clue about what happened. It hooks into the low-level signal handlers for crashes (like SIGSEGV for that segfault) and dumps the current Python stack traces of all your threads to standard error right before the process gives up the ghost. It’s your last will and testament, debugging edition.
Enabling faulthandler: The How and When
You’ve got three main ways to deploy this airbag, and the best one depends on how deep into the crashy abyss you are.
Method 1: The Environment Variable (The “I might need this later” option) This is the easiest way to enable it for any script without modifying a single line of code. Just run your script with:
$ PYTHONFAULTHANDLER=1 python my_crashy_script.py
When the segfault hits, you’ll get a beautiful stack trace dumped to stderr. This is perfect for reproducing a bug reported by a user—just tell them to prepend that to their command.
Method 2: The Command Line Option (The “I’m debugging this right now” option)
The -X flag in Python gives you access to a treasure trove of internal options, and faulthandler is one of them.
$ python -X faulthandler my_crashy_script.py
Same result as the environment variable, just a different way to skin the cat.
Method 3: In Your Code (The “I am the architect of my own doom” option)
You can also enable it programmatically. This is useful if you only want it enabled after a certain point, or if you need more control, like writing the trace to a file instead of stderr.
import faulthandler
import sys
# Just enable it, output goes to sys.stderr
faulthandler.enable()
# Or, enable it and write the crash report to a specific file
with open('/path/to/crash_log.txt', 'w') as f:
faulthandler.enable(file=f)
Be warned: if the crash is severe enough, the file I/O might not complete. Writing to stderr is generally more reliable.
A Real, Crashing Example
Let’s make something blow up on purpose. The ctypes module is perfect for this, as it lets us shoot ourselves in the foot with C-level precision.
import ctypes
import faulthandler
# Enable the airbag
faulthandler.enable()
def cause_a_segfault():
"""This function is a crime against memory safety."""
# Get the address of the Python NULL pointer
null_pointer = ctypes.c_void_p()
# Cast it to a function that returns an int and takes no arguments
function_pointer = ctypes.CFUNCTYPE(ctypes.c_int)(null_pointer.value)
# Try to call it. This is, unequivocally, a Bad Idea.
return function_pointer()
print("About to do something very stupid...")
cause_a_segfault()
print("We never get here.")
Run this, and instead of a silent death, you’ll be greeted with something gloriously informative like:
Fatal Python error: Segmentation fault
Current thread 0x00007f0a5c0d7740 (most recent call first):
File "crash.py", line 13 in cause_a_segfault
File "crash.py", line 17 in <module>
Segmentation fault
There’s your crime scene. Line 13 in cause_a_segfault. Now you know exactly where to start your investigation.
Dumping Stack Traces on Demand
Here’s a trick that feels like black magic: you can ask faulthandler to dump the current stack traces of all running threads to stderr at any time, not just during a crash. This is incredibly useful for debugging deadlocks or weird stalls where your application isn’t crashing but it’s definitely not working.
import faulthandler
import time
from threading import Thread
def some_thread_function():
print("Thread started... now I'm just gonna nap.")
time.sleep(3600) # Sleep for an hour
# Start a background thread
t = Thread(target=some_thread_function)
t.start()
print("Main thread running. Press Ctrl+C to dump stacks.")
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
# The magic happens here
print("\n----- Who's doing what? -----")
faulthandler.dump_traceback()
Run this, wait a second, and hit Ctrl+C. You’ll get a dump showing the stack of the main thread (stuck in sleep) and the background thread (also stuck in sleep). It’s like a global pdb where command for your entire application.
The Inevitable Rough Edges
faulthandler is brilliant, but it’s not omnipotent. The most important caveat is that if the crash corrupts the Python runtime’s own internal state, the stack trace it manages to dump might be incomplete or even garbled. It’s doing its best under truly dire circumstances. Also, the output can be quite verbose in a multi-threaded application, but that’s a good thing—you want to see what every single thread was doing at the moment of impact.
It should be enabled by default in development, and frankly, it’s a design oversight that it isn’t. For any non-trivial application that uses C extensions (hello, numpy, pandas, PIL!), faulthandler isn’t an optional tool; it’s essential safety equipment. Enable it early and often. Your future self, staring at a cryptic segfault, will thank you for the breadcrumbs.