Thanks to a tip from “Derek” on a previous blog post about a replay debugger from 1995, I was made aware of the reverse execution ability that was available in the Borland Turbo Debugger version 3.0 from 1992! This is the oldest commercial instance of “reverse” that I have found (so far), and definitely one of the oldest incarnations of the idea overall. Thanks to Google and the Internet, I managed to find a scanned copy of the manual of the product, which provided some additional information. Note that the debugger only does reverse execution, but not reverse debugging since you cannot run in reverse to stop at a breakpoint.
The manual for the debugger (version 5), has a whole section called “Reverse Execution.” It explains the feature like this:
Turbo Debugger’s execution history keeps track of each instruction as it’s executed, provided that you’re tracing into the code. Using the Execution History window, you can examine the instructions you’ve executed and, if you like, return to a point in the program where you think there might be a bug. Turbo Debugger can record about 400 instructions.
Picture of the cover, from Amazon.
This is essentially stepping back along a short history of recorded instructions. The debugger does offer the ability to step into things like INT handlers, so it is kind of “system level”. It seems to assume the program has a single thread of control, and that there is only one hardware processor in the system – in 1992, handling multicore would seem rather unnecessary. There is some mention of Windows calls, so there might be an OS running – but this is still in the days of DOS-based Windows and cooperative multitasking, simplifying the problem quite a bit.
It sounds like you can only step back through code that has been traced or stepped instruction-by-instruction by the user. If you just run forward, reverse is not active. Given the short window of reverse, this is reasonable.
IO cannot be undone, which in the case of an early 1990s PC system basically means the classic 8086 IN and OUT instruction families. There is no mention of PCI in the debugger manual at all.
There are some instructions and debugger commands that cause the recording history to be flushed, such as executing an INT instruction – unless explicitly tracing into it manually. Using the debugger “run” command or stepping over function calls – which makes sense since that is executing without explicit tracing.
I would not characterize the Turbo Debugger as being full reverse debug – there are no backwards breakpoints. All you can do is to move backwards in the execution recording, either one step at a time or moving back to a particular instruction in the history window.
Overall, the feeling of the debugger is very much that of a bare-metal embedded debugger, including commands like resetting a program by basically setting the instruction pointer to the beginning of the program. In this context, single-stepping and capturing all instructions makes a lot of sense, and doing a bit of reverse on that is rather easy (the amount of state that has to be restored is pretty small).
Debugger session record and replay
The Turbo Debugger offers an interesting way to re-activate reverse execution for a program if you accidentally lose the execution history. You can record all keystrokes sent to the debugger itself, as well as to the program being debugged, and then replay them. This is an interesting way to restore both the program state and debugger state, by recording the history of both.
The Turbo Debugger also offered the ability to do remote debug, using a serial cable at 115kbps to connect the debugger with the debuggee. This was done as a memory usage optimization, since the resident debug agent would be a lot smaller than the full debugger. One source on the Internet lists the size as something like 15kB for the debug agent – which is really compact for a 32-bit x86 program.
The feeling you get is that the developers started out with an instruction trace feature, and then realized that it could be generalized to a reverse execution feature. It is rather limited, but for single-stepping debug, it makes perfect sense to be able to back up a few steps. Technologically, this looks like a dead end since it relies on complete control over the machine and implicitly assumes that there is no operating system or other cores getting in the way.