Debugging

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. — Brian W. Kernighan

There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors. — Leon Bambrick

Debuggers don't remove bugs. They only show them in slow motion. — Unknown

These quotes were chosen from this collection at softwareengineering.stackexchange.

General tips

Knowing how to debug your programs is crucial and should be ideally taught right from the start instead of a chapter at the end of a beginner's learning resource. Think Python is an awesome example for such a resource material.

Debugging is often a frustrating experience. Taking a break helps. It is common to find or solve issues in your dreams too (I've had my share of these, especially during college and intense work days).

If you are stuck with a problem, reduce the code as much as possible so that you are left with the minimal code necessary to reproduce the issue. Talking about the problem to a friend/colleague/inanimate-objects/etc can help too — famously termed as Rubber duck debugging. I have often found the issue while formulating a question to be asked on forums like stackoverflow/reddit because writing down your problem is another way to bring clarity than just having a vague idea in your mind.

Here are some awesome articles on this challenging topic:

• What does debugging a program look like?
• How to debug small programs
• Debugging guide
• Problem solving skills

Here's a summarized snippet from a collection of interesting bug stories:

A jpeg parser choked whenever the CEO came into the room, because he always had a shirt with a square pattern on it, which triggered some special case of contrast and block boundary algorithms.

See also this curated list of absurd software bug stories.

Common beginner mistakes

The previous chapter already covered examples for syntax errors. This section will discuss some more Python gotchas.

Python allows you to redefine built-in functions, modules, classes, etc (see also stackoverflow: metaprogramming). Unless that's your intention, do not use keywords, built-in functions and modules as your variable name, function name, program filename, etc. Here's an example:

# normal behavior
>>> str(2)
'2'

# unintentional use of 'str' as a variable name
>>> str = input("what is your name? ")
what is your name? learnbyexample
# 'str' is now no longer usable as a built-in function
>>> str(2)
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    str(2)
    ~~~^^^
TypeError: 'str' object is not callable

Here's another example:

>>> len = 5
>>> len('hi')
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    len('hi')
    ~~~^^^^^^
TypeError: 'int' object is not callable

As an exercise, create an empty file named as math.py. In the same directory, create another program file that imports the math module and then uses some feature, print(math.pi) for example. What happens if you execute this program?

See also:

• Think Python: Debugging chapter
• inventwithpython: common runtime errors and common python gotchas
• pythonforbiologists: common beginner errors
• stackoverflow: common pitfalls in Python

pdb

Python comes with a handy built-in library pdb that you can use from the CLI to debug programs. See docs.python: pdb for documentation. Here are some of the frequently used commands (only short form is shown, see documentation for long form and more details).

• l prints code around the current statement the debugger is at, useful to visualize the progress of the debug effort
• ll prints the full code for the current function or frame
• s execute the current line, steps inside function calls
• n execute the current line, treats function call as a single execution step
• c continue execution until the next breakpoint
• p expression print value of an expression in the current context, usually used to see the current value of a variable
• h list of available commands
  • h c help on the c command
• q quit the debugger

Here's an example invocation of the debugger for the num_funcs.py program seen earlier in the Importing your own module section. Only the n command is used below. Lines with the > prefix tells you about the program file being debugged, current line number, function name and return value when applicable. Lines with the -> prefix is the code present at the current line. (Pdb) is the prompt for this interactive session. You can also see the output of print() function for the last n command in the illustration below.

$ python3.13 -m pdb num_funcs.py
> /home/learnbyexample/Python/programs/num_funcs.py(1)<module>()
-> def sqr(n):
(Pdb) n
> /home/learnbyexample/Python/programs/num_funcs.py(4)<module>()
-> def fact(n):
(Pdb) n
> /home/learnbyexample/Python/programs/num_funcs.py(10)<module>()
-> num = 5
(Pdb) n
> /home/learnbyexample/Python/programs/num_funcs.py(11)<module>()
-> print(f'square of {num} is {sqr(num)}')
(Pdb) n
square of 5 is 25
> /home/learnbyexample/Python/programs/num_funcs.py(12)<module>()
-> print(f'factorial of {num} is {fact(num)}')

Continuation of the above debugging session is shown below, this time with the s command to step into the function. Use r while you are still inside the function to skip until the function encounters a return statement. Examples for the p and ll commands are also shown below.

(Pdb) s
--Call--
> /home/learnbyexample/Python/programs/num_funcs.py(4)fact()
-> def fact(n):
(Pdb) ll
  4  -> def fact(n):
  5         total = 1
  6         for i in range(2, n+1):
  7             total *= i
  8         return total
(Pdb) n
> /home/learnbyexample/Python/programs/num_funcs.py(5)fact()
-> total = 1
(Pdb) p n
5
(Pdb) r
--Return--
> /home/learnbyexample/Python/programs/num_funcs.py(8)fact()->120
-> return total
(Pdb) n
factorial of 5 is 120
--Return--
> /home/learnbyexample/Python/programs/num_funcs.py(12)<module>()->None
-> print(f'factorial of {num} is {fact(num)}')

Use q to end the session.

(Pdb) n
--Return--
> <string>(1)<module>()->None
(Pdb) q

You can call breakpoint() or pdb.set_trace() to set breakpoints in the code and use it in combination with the c command.

See also:

• awesome-python: Debugging tools
• pdb tutorial
• docs.python HOWTOs: Basic Logging Tutorial

IDLE debugging

Sites like Pythontutor allow you to visually debug a program — you can execute a program step by step and see the current value of variables. Similar features are typically provided by IDEs as well. Under the hood, these visualizations would likely be using the pdb module discussed in the previous section.

This section will show an example with IDLE. Before you can run the program, first select the Debugger option under the Debug menu. You can also use idle3.13 -d to launch IDLE in debug mode directly. You'll see a new window pop up as shown below:

Then, with debug mode active, load and run a program (for example, num_funcs.py discussed in the previous section). Use the buttons and options to go over the code. Variable values will be automatically available, as shown below.

You can right-click on a line from the text editor to set or clear breakpoints.

See realpython: Debug With IDLE for a more detailed tutorial.