[I previously sent this to the libreswan developers list.] I don't use gdb much. Consequently I'm clumsy using it. Consequently I don't use gdb much. This tool looks like it might make gdb a lot more useful: <https://rr-project.org/> It is a Mozilla research project. But at this point it just works. Have a look at the videos on that page. Idea: (with limitation) you run a program under rr to record that run. When it is captured, you can replay it (forward and back!) as many times as you want, from within GDB. Breakpoints and all work. It supports multithreaded programs, but it only allows one core to be used. (I have no idea whether it supports fork.) For a non-deterministic bug, you keep rerunning the recording step until you observe the problem. You can replay that execution as much as you want without the non-determinism shifting on you. Some kinds of output are lost: in demos of debugging Firefox, Firefox windows were not created during replay. But anything that the program actually examines comes from the recording. The overhead comes in two parts: - run time during recording. That seems fine: less than a factor of two. - RAM requirements. That can get large. It mostly records everything that passes into and out of each system call. If they can debug Firefox with this, I imagine that we could debug even large programs.