You can find plenty of information about programming languages, concepts and existing code. Less so about personal programming practices. And even less so about the debugging process. Yet, debugging your own and someone else's code is what a lot of programmers do on a daily basis.
Bugs come in a great variety: reliable, scary, expected. There may also arise situations where they can affect one's life. Do not fear! In this battle we have kind friends and strong allies, and we'll meet them here. Let's dive in.
Before we start
When you see an issue in the program's behaviour, check if it is obvious. Is it? Are you sure? It is not obvious? If so, we usually need to take a number of measures. Those measures may and may not include:
Exact reproduction steps. Without them, how can we prepare tests? More interestingly, how can we be sure we fixed it (at least, on your own machine)?
Reproducing the bug. Maybe the pain will go away? It is not likely, but you might get some additional information. Even small bits like "It works on GNU/Linux box, but fails on Windows" help.
Minimizing the input if any. Chop off everything that does not stop the issue from appearing, the smaller the better.
Depending on your work policies, check if the bug has been already reported before, fill in the reports, etc. Do the steps that suit your specific workflow.
Finding sources
Q: When?
A: When you see an error, a thrown exception or some incorrect behaviour. Sometimes you have a stacktrace (trace of all calls from the one which caused the error) and sometimes you don't. To start an actual investigation as to where things go wrong, you need to find the code that's behind the error.
Q: I have a stacktrace! Is this enough?
A: Hopefully. Stacktraces can be rather long, but most often the first 30 or so lines have all the info needed. An exception usually has some message about the error state, which might make things easier.
Q: There is no stacktrace. An error message or some label I have to work with is just emitted somewhere. And this code base has millions of lines of code!
A: Use text search to find where the line is defined - while it might sound obvious, this simple step is often neglected. By using tools like ripgrep or a well-configured grep it takes mere seconds to look up a text pattern, even in a large code base. As a bonus, the error messages are often constructed from parts, so if the search does not return anything at all, you can try smaller chunks and variations of the texts. To narrow down search results for a query consisting of just a couple of common words, try adding quotation marks.
Let's look at a couple of examples using IntelliJ Community code base (with 9+ millions of lines of code it is large and interesting enough, but the tricks shown are not really limited to it).
Case 1: executing rg '"Extract"' gives 2 results, while executing rg Extract gives just 2751.
Case 2: executing rg 'If you already have a 64-bit JDK installed' for the same project returns no result at all, while executing rg 'If you already have a' gives source of the runtime error message: number of bits is baked into the error message and so it cannot be searched directly.
Case 3: say you want to know why you get an error while compiling a particular file using 'Recompile ...' in IDEA's context menu. A rg '"Recompile ' does not yield anything relevant. However, it has a hotkey tied to it: Ctrl+Alt+F9. With a search for
rg '"control shift F9"' we find some keymap files where we see this:
<action id="Compile">
<keyboard-shortcut first-keystroke="control shift F9">
</action>Now that we have the action's ID, we can look it up with rg 'id="Compile"' . This gives us a number of lookups with one being resources/src/idea/JavaActions.xml file containing <action id="Compile" class="com.intellij.compiler.actions.CompileAction"/> . Now we know the exact class and can start from there using a debugger.
So, the search process might need several steps. It might also rely on specific project organization patterns where additional data can help us look up sources even when there are no obvious shortcuts to a specific file among thousands.
Q: Still no luck!
A: It can be something from a library, so just google its text (removing parts looking like identifiers may improve the search results). Another option is that someone really didn't want you to look up this message in the sources.
Exploring sources
Q: When?
A: When you know where the error happens. Maybe because you wrote the code yourself. Or you found it using a stacktrace or had some luck during the search. Now it's time for you to make some sense of it.
Q: I have a debugger, is that enough?
A: Hopefully, alas less often than you want it to be. Even so, having an interactive debugger is a very neat way to explore the particular situation. Learn how to use it with the abundant resources available about stepping, breakpoints and playing with values and stack frames.
Let's look at a couple of examples using IntelliJ Community code base.
Q: The input for the piece of code I debug is big and the error is only triggered after the code in question is executed hundreds of times!
A: Your options are: 1) reduce the input as much as possible (again, preparation measures are helpful); 2) If you know enough about the problem to set up a conditional breakpoint - add one.
Q: The input is enormous even after reducing it. Conditional breakpoints are no good here.
A: Then it's time to resort to one of the most popular and universal techniques - Old Plain Log-driven Debugging. Just add some text output and check it on the next run! See how the execution goes, what data and results are you getting around the wrong piece of code. This step is usually a prerequisite for guessing a correct condition for a breakpoint and if you have one, stop immediately! Proceed with the breakpoint then. Given this technique is so common and easy (thus tempting) to use, it is a good habit to write proper log messages. Take your time and explain the situation, or write something neutral, like numbers. Never write anything which could be offensive to your colleagues in a git log! You will commit and push debug logging leftovers, for sure.
Obligatory edge cases
Q: When?
A: When nothing works out, but certain conditions are met.
Q: The bug does not appear when testing, but appears on the production build.
A: The best bet is a remote debugger for the specific language. As the production and testing environment may differ in subtle ways, connecting via a remote debugger will help with investigating the situation.
Q: I can reproduce the bug, but nobody else from my team can.
A: When it seems like the world itself is against you, it is wise to believe it's not the case. At least this time. The code you write is most likely standing on the shoulders of giants: countless libraries, applications, OS itself, an enormous amount of layers, let alone various specific conditions from your workplace. It is safe to assume that something differs in your environment, and the task is to find out what it is. Time zone? Locale settings? LibreSSL version? Whatever it is, checking everything related might lead to satori. If the error looks unusual enough, try to google it first.
Q: The code creates input for some other program, e.g. a binary for an OS or a text document in specific format for our bank server, which does not accept it.
A: If you have a working sample and a broken one, don't hesitate to ignore the sources for a bit and inspect them properly. Use whatever tools you can find: specific editors, diff checkers, anything will do. If the differences are numerous, try to port them in chunks until the observed behaviour won't change. If you can find out what levers should be pulled, you are usually able to pull them.
Q: The bug is obscure, scary. I can reproduce it, but I can't find the reasons behind it.
A: If it is a regression and you know when things worked, use - git bisect. If you know where the bug might hide, but don't know what change causes what, try to revert related commits. Today "git blame" and "git history" can do wonders! See if you can make the bug disappear in earlier versions, and then you can move on from there. If there are 10000 commits and you don't know which one is wrong,git bisect can still do wonders. Even when the situation seems desperate, with enough perseverance you are able to bisect the code into chewable sizes. It helps you to find the culprit fast, depending on how fast you can check if the bug is present at the current commit.
Q: I change code in various ways again and again, but no changes appear in the application run!
A: As obvious as it sounds, check if the right sources are being edited and executed!
Protecting sources
All of the above might be good and popular when you already see a bug, but sometimes it is better to think of possible issues and help your future self beforehand. This way we can even hunt down invisible bugs.
Consider the following. Say your application queries data from numerous data sources. When executing a query, the data sources are asked to provide a new data set. It is added to a combined data set and everything is returned. You and other people can work over a long time, months and years, editing the code here and there. Eventually, some data sources might become slightly incorrect (for example, due to a not so successful refactoring) and provide data they should not, for example, adding the same data twice. While this does not break the application, it still spends time doing redundant calculations, and worse - it will be completely "invisible" to anyone if the whole process is relatively quick.
The answer is simple: use assertions! Write an assertion which states that nothing in a single dataset can be contributed to the combined data set already. If the code or data sources ever regresses, users will see no impact other than a bit of performance drop due to redundant calculations. But in a test environment a single run with assertions enabled will allow a stacktrace of the issue in no time and re-think the regressed piece.
This approach, when applied in various cases, allows for not just having a more explicit and stricter execution logic for the reader. It also helps with "a bug, but not obviously visible one" kind of issues and makes the code more robust against future changes for years ahead.
Finding new hope
Q: When?
A: When nothing works out and the issue seems really serious.
Q: So there is really no solution then if you repeated "nothing works out" twice, right?
A: Not really. When in serious doubt: just ask someone for help. Someone might know this area of code better. Someone might have more luck today. But be considerate to your colleagues. Ask if the person is struggling with something even more difficult first. If they have time to help you out, remember to thank them and learn from them. You might be able to show someone else next time.
Closing thoughts
Here we took an informal look at a couple of simple advice on how to deal with issues in code. This post is not intended to be a comprehensive guide. It tries to avoid both overused advises like "Try to explain this bug to a plush toy on the desk to understand it better" and language-specific advice. But, if this article helps you deal with bugs even a tiny bit better, it made at least one person's life easier.
By Oleksandr Kyriukhin, part of our team in Prague