72.9 faulthandler: Diagnosing Crashes and Segfaults

Right, so you’ve written some Python. It’s beautiful, it’s elegant, and then—without warning—it exits. No traceback, no KeyboardInterrupt, just a sudden, silent return to the comforting glow of your terminal prompt. Or worse, it spits out Segmentation fault (core dumped) and mocks you from the history log. This, my friend, is where your usual Python tools tap out. print() statements? Useless. The logging module? Never got the message. pdb? Didn’t even get to wake up. Your code has crashed in the C layers, far beneath the comfortable Python runtime where exceptions are raised and caught. This is the realm of dangling pointers, buffer overflows, and corrupted memory. And to diagnose this, you need a different kind of tool. You need faulthandler.

72.8 sys.settrace(): Writing Your Own Debugger or Profiler

Right, so you’ve graduated from print() statements and you’re ready to get serious. You’ve used pdb and maybe even a fancy IDE debugger, and a little voice in your head asked, “How do they do that?” The answer, my friend, is sys.settrace(). It’s the arcane, powerful, and slightly terrifying incantation that allows you to hook into the CPython interpreter’s execution flow. It’s how debuggers, profilers, and coverage tools are born.

72.7 Remote Debugging with debugpy (VS Code, PyCharm)

Right, so your code is misbehaving. But it’s misbehaving on a remote server, in a Docker container, or inside a virtual environment so alien it might as well be on the dark side of the moon. You can’t just slap a print("got here lol") statement in there and run it locally. This is where we graduate from caveman debugging to something with a bit more finesse: remote debugging. We’re going to use debugpy, Microsoft’s brilliantly capable debugger protocol for Python. It’s what lets VS Code’s debugger do its magic, and it plays nicely with PyCharm and other modern IDEs too.

72.6 breakpoint() and PYTHONBREAKPOINT

Right, so you’ve graduated from print("got here") to actual debugging. Congratulations, we’re all very proud. But let’s be honest, fumbling with import pdb; pdb.set_trace() is the digital equivalent of trying to start a fire with two wet sticks. It works, but it’s clumsy, it leaves a mess, and there’s a much better way. Enter breakpoint(). This isn’t just a new function; it’s a cultural shift in Python debugging, and it’s about damn time.

72.5 pdb: Setting Breakpoints and Inspecting State

Right, so print() statements have failed you. They always do. Welcome to the big leagues. When your code is doing something so profoundly idiotic that you can’t even begin to guess why, you need to stop it mid-execution, climb inside its brain, and have a look around. That’s what pdb, the Python debugger, is for. It’s your surgical tool for figuring out what the hell is actually happening, not what you think is happening.

72.4 Structured Logging with structlog

Right, let’s talk about making your logs actually useful. You’ve probably been there: staring at a text file that looks like a frantic, unstructured diary entry written by a machine on three cups of espresso. Timestamp, log level, some vague message… good luck finding the one error in that mess. The default logging module is fine for telling you that something happened, but it’s terrible at telling you the story of why it happened. That’s where structlog comes in. It’s not just a library; it’s a philosophy for turning your logs from a liability into a debuggable, queryable asset.

72.3 Logging to Files, Rotating Handlers, and External Services

Right, so you’ve graduated from print() statements. Good for you. Now let’s talk about doing it properly. Logging to the console is fine for a quick script, but for anything that runs longer than five minutes, you need persistence. You need logs that survive a reboot, that you can grep through at 2 AM when things are on fire, and that don’t fill up your disk and bring the whole operation to a grinding halt. Let’s get into it.

72.2 basicConfig() vs Manual Configuration

Right, let’s settle this. You’ve seen basicConfig() everywhere. It’s the logging equivalent of a friendly “Easy” button. And you’ve probably also seen people creating Logger objects, Handler objects, Formatter objects… and thought, “Why would anyone do that the hard way?” I’m here to tell you that basicConfig() is a fantastic one-night stand, but for a serious, long-term relationship with your application’s logs, you need to do things manually. Let’s break down why.

72.1 The logging Module: Levels, Loggers, Handlers, Formatters

Right, let’s talk about logging. You’ve been using print() statements to debug your code since you wrote your first Hello, World. I get it. It’s immediate, it’s simple, and when your script is three lines long, it’s perfect. But you’re not writing three-line scripts anymore, are you? You’re building applications. And when your application is running on a server at 2 AM and something goes horribly wrong, you’re not going to SSH in to tail -f a bunch of print() output. You need a system. A robust, configurable, and frankly, adult system for understanding what your code is doing when you’re not there to watch it. That system is the logging module.

— joke —

...