Over the past years, two major players invested into the future of Python. Microsoft’s Faster CPython team has pushed ahead with impressive performance improvements for the CPython interpreter, which has gotten at least 2x faster since Python 3.9. They also have a baseline JIT compiler for CPython, too. At the same time, Meta is worked hard on making free-threaded Python a reality to bring classic shared-memory multithreading to Python, without being limited by the still standard Global Interpreter Lock, which prevents true parallelism.

Both projects deliver major improvements to Python, and the wider ecosystem. So, it’s all great, or is it?

In my talk talk on this topic at SPLASH, which is now online, I discussed some of the aspects the Python core developers and wider community seem to not regard with the same urgency as I would hope for. Concurrency makes me scared, and I strongly believe the Python ecosystem should be scared, too, or look forward to the 2030s being “Python’s Decade of Concurrency Bugs”.

In the talk, I started out reviewing some of the changes in observable language semantics between Python 3.9 and today and discuss their implications. I previously discussed the changes around the global interpreter lock in my post on the changing “guarantees”. In the talk, I also use the example from a real bug report, to illustrate the semantic changes:

request_id = self._next_id
self._next_id += 1

It looks simple, but reveals quite profound differences between Python versions.

Since I have some old ideas lying around, I also propose a way forward. In practice though, this isn’t a small well-defined engineering or research project. So, I hope I can inspire some of you to follow me down the rabbit hole of Python’s free-threaded future.

Incidentally, the latest release of TruffleRuby now uses many of the techniques that would be useful for Python. Benoit Daloze implemented them during his PhD and we originally published the ideas back in 2018.

Questions, pointers, and suggestions are always welcome, for instance, on Mastodon, BlueSky, or Twitter.