When we have to debug applications that use concurrency, perhaps written in Java, all we get from the debugger is a list of threads, perhaps some information about held locks, and the ability to step through each thread separately.

What we usually don’t get is any kind of high-level support for the concurrency abstractions that we used in our applications. We might have used a library for actors, channels from CSP, Java’s fork/join pool, or perhaps even some software transactional memory. This means, in the debugger, we cannot reason anymore about the same concepts we used to built the application. Instead, we can only vaguely recognize them from how the threads interact. We might see that one thread puts an object, which is perhaps a message, into another object, which represents a mailbox. But we don’t get any support to actually follow that message as it is being processed. We also have no way of jumping from the point where we spawn a fork/join task to its activation to see the computation unfold recursively and in parallel. And wouldn’t it be nice if we could simple set a breakpoint on a promise and see where we continue executing when it got resolved?

I am pretty sure these things would make it easier to understand what’s going on in my applications. Unfortunately, debugger support for such high-level concepts is rare or non-existing.

One of the biggest problems is that there are too many different concurrency models, concepts, languages, and libraries. This means designing debugging support for all these variations is a huge problem. And using the same debugger for Akka actors, Erlang, Pony, or your favorite actor library doesn’t seem feasible. Not to mention all those other systems. You want to debug goroutines and their use of channels, Clojure’s core.async, or perhaps Java’s JCSP?

Turns out, building a debugger for all these things isn’t impossible. We can design the debugger in a way that it can remain completely oblivious of the various concurrency concepts. This means, it can be used for a wide range of concurrent systems without having to be adapted specifically for them. To make this work, we propose the Kómpos protocol to communicate the necessary details from a language runtime to the debugger in a concurrency-agnostic way.

For a small demo, see the video below. The abstract for our paper describing the details of the approach, as well as links to it follow afterwards.

For questions or comments, please find me on Twitter @smarr.

Abstract

Today’s complex software systems combine high-level concurrency models. Each model is used to solve a specific set of problems. Unfortunately, debuggers support only the low-level notions of threads and shared memory, forcing developers to reason about these notions instead of the high-level concurrency models they chose.

This paper proposes a concurrency-agnostic debugger protocol that decouples the debugger from the concurrency models employed by the target application. As a result, the underlying language runtime can define custom breakpoints, stepping operations, and execution events for each concurrency model it supports, and a debugger can expose them without having to be specifically adapted.

We evaluated the generality of the protocol by applying it to SOMns, a Newspeak implementation, which supports a diversity of concurrency models including communicating sequential processes, communicating event loops, threads and locks, fork/join parallelism, and software transactional memory. We implemented 21 breakpoints and 20 stepping operations for these concurrency models. For none of these, the debugger needed to be changed. Furthermore, we visualize all concurrent interactions independently of a specific concurrency model. To show that tooling for a specific concurrency model is possible, we visualize actor turns and message sends separately.

  • A Concurrency-Agnostic Protocol for Multi-Paradigm Concurrent Debugging Tools
    S. Marr, C. Torres Lopez, D. Aumayr, E. Gonzalez Boix, H. Mössenböck; In Proceedings of the 13th ACM SIGPLAN International Symposium on Dynamic Languages, DLS'17, p. 3–14, ACM, 2017.
  • Paper: HTML, PDF
  • DOI: 10.1145/3133841.3133842
  • BibTex: bibtex 
    @inproceedings{Marr:2017:CPCD,
  abstract = {Today's complex software systems combine high-level concurrency models. Each model is used to solve a specific set of problems. Unfortunately, debuggers support only the low-level notions of threads and shared memory, forcing developers to reason about these notions instead of the high-level concurrency models they chose.
  
  This paper proposes a concurrency-agnostic debugger protocol that decouples the debugger from the concurrency models employed by the target application. As a result, the underlying language runtime can define custom breakpoints, stepping operations, and execution events for each concurrency model it supports, and a debugger can expose them without having to be specifically adapted.
  
  We evaluated the generality of the protocol by applying it to SOMns, a Newspeak implementation, which supports a diversity of concurrency models including communicating sequential processes, communicating event loops, threads and locks, fork/join parallelism, and software transactional memory. We implemented 21 breakpoints and 20 stepping operations for these concurrency models. For none of these, the debugger needed to be changed. Furthermore, we visualize all concurrent interactions independently of a specific concurrency model. To show that tooling for a specific concurrency model is possible, we visualize actor turns and message sends separately.},
  acceptancerate = {0.64},
  author = {Marr, Stefan and Torres Lopez, Carmen and Aumayr, Dominik and Gonzalez Boix, Elisa and Mössenböck, Hanspeter},
  blog = {https://stefan-marr.de/2017/10/multi-paradigm-concurrent-debugging/},
  booktitle = {Proceedings of the 13th ACM SIGPLAN International Symposium on Dynamic Languages},
  day = {24},
  doi = {10.1145/3133841.3133842},
  html = {https://stefan-marr.de/papers/dls-marr-et-al-concurrency-agnostic-protocol-for-debugging/},
  isbn = {978-1-4503-5526-1/17/10},
  keywords = {Breakpoints Concurrency Debugging MeMyPublication Stepping Tooling Visualization myown},
  location = {Vancouver, Canada},
  month = oct,
  note = {(acceptance rate 64%)},
  numpages = {12},
  pages = {3--14},
  pdf = {https://stefan-marr.de/downloads/dls17-marr-et-al-concurrency-agnostic-protocol-for-debugging.pdf},
  publisher = {ACM},
  series = {DLS'17},
  title = {{A Concurrency-Agnostic Protocol for Multi-Paradigm Concurrent Debugging Tools}},
  year = {2017},
  month_numeric = {10}
}