Parallel Dynamic Language VMs

As part of the external pagePyPy project, we are experimenting with ways to develop virtual machines (VMs) for dynamic programming languages that support parallel execution of applications. RPython, the VM construction framework that is part of the PyPy project, currently has a single external pageglobal interpreter lock (GIL) to synchronise multi-threaded execution within the VM. Hence, true parallel execution is poorly supported, but in return VMs get atomic and sequentially consistent execution.

One approach for true parallel execution is to replace GIL-synchronisation with transactional memory (TM). Since TM supports single global lock semantics, a GIL-supported VM can be migrated to a TM-supported VM without changing semantics. This approach puts correctness before performance, as TM adds considerable overhead to the execution. We are actively working in this direction with external pagePyPy-STM.

Another approach is to initially abandon the guarantees enforced by the GIL and instead let VMs handle synchronisation on their own. In contrast to the previous approach, performance is put before correctness. We pursue this approach with PyPy-NoGIL.

These approaches lie on opposite ends of the spectrum of the performance-safety tradeoff. While PyPy-STM offers automatically correct but slow synchronisation, PyPy-NoGIL offers potentially incorrect, manual but fast synchronisation. Both are extremes, which is why we need to explore ways for each approach to move in the direction of the other, possibly finding an interesting middle-ground that offers the best of both. Ideas and theses to work on these issues are very welcome.

Concrete project ideas:

Our prototype of PyPy-STM is a VM for Python. However, the approach should work for other languages too. Hence, we want to explore how well our approach works for other external pagelanguages.

Scope: bachelor/master thesis
Environment: RPython, Python, PyPy
Keywords: programming languages, concurrency, parallel programming, transactional memory
Contacts: Remi Meier, CAB H 83.1, remi.meier at inf.ethz.ch
Supervisors: Prof. Th. Gross, thomas.gross at inf.ethz.ch