Archive | Truffle RSS for this section

A 10 Year Journey, Stop 3: Performance, Performance, and Metaprogramming

The third post of this series is about how I started using Truffle and Graal, pretty much 4 years ago. It might be in parts ranty, but I started using it when it was in a very early stage. So, things are a lot better today.

Concurrency needs Performance, usually

As mentioned in the last post, the result of my PhD was an ownership-based metaobject protocol that is meant to enable VMs to support a wide range of different concurrency models. The major problem with the approach, and also my evaluation, was that I couldn’t show that it is practical. The RoarVM is a simple bytecode interpreter, and the literature on compiling and optimizing metaobject protocols talked only about static systems with restrictions that would make the ownership-based MOP impossible. Worse, MOPs were kind of abandoned by the research community, because performance was an issue. Many researchers moved on to aspect-oriented approaches, at least in part, because aspects are applied more targeted and thus, incur less general overhead than MOPs.

A hard problem, abandoned for 20 years, and nobody really interested in it anymore? Pretty much sounds like it’s a stupid idea, right? It probably was, and perhaps still is. But concurrency researchers typically want to show that their techniques are useful for performance critical applications. And, I wanted to do that, too.

How to get a fast VM?

So, I needed a new platform for my research. One option would have been to take the RoarVM all the way. Build a state-of-the-art JIT compiler for it. Another would be to apply the ideas of the RoarVM to the CogVM and improve its JIT compiler. But building another JIT compiler? That’s a huge undertaking. Would probably take a few person years to get anywhere useful. And, while I am curious about compilers, I am not really seeing myself building more than a baseline JIT compiler.

But what other options do we have? RPython is a pretty interesting project. It promises you a meta-tracing JIT compiler for your simple interpreter. Sounds great. But there’s a catch: RPython doesn’t really have a concurrency story compatible with my goals. There’s a bit of experimenting with STM going on, but no decent shared-memory GC.

And then, there was that Truffle thing, colleagues from Oracle Labs kept talking about. It was just released as open source at that point. Truffle promised that simple AST-based interpreters combined with partial evaluation would run applications as fast as Java on a JVM. Sounds great. And, the JVM got everything I need for my research, too.

TruffleSOM: The first steps with a simple Smalltalk on top of Truffle

Truffle it is, I thought, and started implementing the little Smalltalk I had been toying around with in 2007 on top of it. There was already a Java version, simply called SOM. So, how hard could it be?

Well, turns out, much harder than expected. Me being not really a compiler person, I had no intuition for how partial evaluation would work on my interpreter. And as a testament to how hard it was for me, apparently even to this date, I am third in the ranking of who spammed the graal-dev mailing list most.

I suppose there were three important reasons for it. As I mentioned before, I had no intuition of how the partial evaluation really works, and what kind of optimizations I can expect from the systems. The second reason was that I did not really have access to the expertise. Mailing lists are fine but slow. And people need to be willing to take the time to answer. So, during my endeavor of building a fast Smalltalk, the most helpful conversations were actually in person at conferences with people from the Truffle team, or when I actually got the chance to visit them in Linz. And the third reason, which is fixed by now, was the overall stability of the system. To me it was very surprising that I ran into many bugs in Truffle and the Graal compiler. I could somehow not really believe that the Truffle team didn’t encounter those issues with their JavaScript and Ruby implementations. But as it turns out, each new language implementation does things somewhat different and the languages are just different enough to trigger new edge cases that haven’t been considered before. As far as I know, there is still a little of that happening today with Graal. Every new language leads to one or two bugs being discovered that none of the previous languages hit.

How can this be sooo hard???

All in all, my experience to build a new language with Truffle and Graal was far from pleasant. On the contrary, it was frustrating. I often just didn’t have the knowledge to debug problems myself. And the Truffle team didn’t really have time to teach an outsider all those basics.

So, yeah, I was very close to throwing in the towel. Well, I actually kind of did. At least a little. There was a moment of “fuck this, how can it be so hard? screw Truffle! Let’s look into RPython!”

And PySOM was born. PySOM is a literal port of the SOM bytecode interpreter to Python. SOM is really a simple and small language. If you know what you’re doing, and can type reasonably fast, you can implement it in 3 or 4 days in a new language.

PySOM was the first step. The next step was RPySOM: a port from Python to RPython, which is the Python subset that the RPython toolchain can compile statically into a fast interpreter with a meta-tracing JIT compiler. This experience was sooo much more pleasant. One big reason was that the PyPy+RPython community uses an IRC channel for communication and was super friendly and happy to help with all my problems. Another reason was that I knew Carl Friedrich, one of the PyPy people already for a while, and he guided me through the classic pit falls. And, I suppose, RPython at that point was already much more mature than Truffle used to be. So, fewer crashes, and I think, I didn’t really trigger much bugs in RPython either. And, since it is a trace-based compiler, understanding what the optimizer did was also much easier, because the result mapped much more directly back to the input code of the interpreter than with a fancy graph-based compiler IR. So, yeah, RPySOM was born, and with that additional knowledge, I kind of managed to make TruffleSOM a reality, too.

Some of this story, and what we learned was written down in the Are We There Yet? article.

And Finally, Fast Metaprogramming!

Then it was time to get back to my original problem: how do I get my ownership-based metaobject protocol fast? Well, turns out if you got a fancy JIT compiler, the solution is pretty simple and already existed in other Truffle interpreters: dispatch chains. Dispatch chains are essentially lexical caches for dispatch operations. A generalization of polymorphic inline caches if you will.

Together with Chris Seaton, we published a paper on Zero-Overhead Meta Programming, where we were able to show how all kind of reflective operations can be made fast, and were I was finally able to show that my metaobject protocol can be realized without sacrificing performance.

A bit later, I also wrote up a longer paper comparing meta-tracing and partial evaluation in more detail.

Cutting a long story short: nothing is as easy as it sounds. In total, it took me two years to go from a simple Smalltalk AST interpreter to a system that can take on Java. But, things should be better today. When starting to implement a new language with Truffle, there are now a few tutorials, and other resources, and the platform is much more mature and pleasant to use!

Next week, I might take a break from this series, but there are at least two more posts coming:

  • Concurrency and Tooling, or ‘What is project MetaConc?’
  • and, Growing the SOM Family

SOMns 0.2 Release with CSP, STM, Threads, and Fork/Join

Since SOMns is a pure research project, we aren’t usually doing releases for SOMns yet. However, we added many different concurrency abstractions since December and have plans for bigger changes. So, it seems like a good time to wrap up another step, and get it into a somewhat stable shape.

The result is SOMns v0.2, a release that adds support for communicating sequential processes, shared-memory multithreading, fork/join, and a toy STM. We also improved a variety of things under the hood.

Note, SOMns is still not meant for ‘users’. It is however a stable platform for concurrency research and student projects. If you’re interested to work with it, drop us a line, or check out the getting started guide.

0.2.0 – 2017-03-07 Extended Concurrency Support

Concurrency Support

  • Added basic support for shared-memory multithreading and fork/join
    programming (PR #52)

    • object model uses now a global safepoint to synchronize layout changes
    • array strategies are not safe yet
  • Added Lee and Vacation benchmarks (PR #78)

  • Configuration flag for actor tracing, -atcfg=
    example: -atcfg=mt:mp:pc turns off message timestamps, message parameters and promises

  • Added Validation benchmarks and a new Harness.

  • Added basic Communicating Sequential Processes support.
    See PR #84.

  • Added CSP version of PingPong benchmark.

  • Added simple STM implementation. See s.i.t.Transactions and PR #81 for details.

  • Added breakpoints for channel operations in PR #99.

  • Fixed isolation issue for actors. The test that an actor is only created
    from a value was broken (issue #101, PR #102)

  • Optimize processing of common single messages by avoiding allocation and
    use of object buffer (issue #90)

Interpreter Improvements

  • Turn writes to method arguments into errors. Before it was leading to
    confusing setter sends and ‘message not understood’ errors.

  • Simplified AST inlining and use objects to represent variable info to improve
    details displayed in debugger (PR #80).

  • Make instrumentation more robust by defining number of arguments of an
    operation explicitly.

  • Add parse-time specialization of primitives. This enables very early
    knowledge about the program, which might be unreliable, but should be good
    enough for tooling. (See Issue #75 and PR #88)

  • Added option to show methods after parsing in IGV with
    -im/--igv-parsed-methods (issue #110)

Communicating Sequential Processes for Newspeak/SOMns

One possible way for modeling concurrent systems is Tony Hoare’s classic approach of having isolated processes communicate via channels, which is called Communicating Sequential Processes (CSP). Today, we see the approach used for instance in Go and Clojure.

While Newspeak’s specification and implementation come with support for Actors, I want to experiment also with other abstractions, and CSP happens to be an interesting one, since it models systems with blocking synchronization, also know as channels with rendezvous semantics. I am not saying CSP is better in any specific case than actors. Instead, I want to find out where CSP’s abstractions provide a tangible benefit.

But, the reason for this post is another one. One of my biggest quibbles with most CSP implementations is that they don’t take isolation serious. Usually, they provide merely lightweight concurrency and channels, but they rarely ensure that different processes don’t share any mutable memory. So, the door for low-level race conditions is wide open. The standard argument of language or library implementers is that guaranteeing isolation is not worth the performance overhead that comes with it. For me, concurrency is hard enough, so, I prefer to have the guarantee of proper isolation. Of course, another part of the argument is that you might need shared memory for some problems, but, I think we got a more disciplined approach for those problems, too.

Isolated Processes in Newspeak

Ok, so how can we realize isolated processes in Newspeak? As it turns out, it is pretty simple. Newspeak already got the notion of values. Values are deeply immutable objects. This means values can only contain values themselves, which as a consequence means, if you receive some value from a concurrent entity, you are guaranteed that the state never changes.

In SOMns, you can use the Value mixin to mark a class as having value semantics. This means that none of the fields of the object are allowed to be mutable, and that we need to check that fields are only initialized with values in the object’s constructor. Since Newspeak uses nested classes pretty much everywhere, we also need to check that the outer scope of a value class does not have any mutable state. Once that is verified, an object can be a proper deeply immutable value, and can be shared with out introducing any data races between concurrent entities.

Using this as a foundation, we can require that all classes that represent CSP processes are values. This gives us the guarantee that a process does not have access to any shared mutable state by itself. Note, this is only about the class side. The object side can actually be a normal object an have mutable state, which means, within a process, we can have normal mutable state/objects.

Using the value notion of Newspeak feels like a very natural solution to me. Alternative approaches could use a magic operator that cuts off lexical scope. This is something that I have seen for instance in AmbientTalk with its isolates. While this magic isolate keyword gives some extra flexibility, it is also a new concept. Having to ensure that a process’ class is a value requires that its outer lexical scope is a value, and thus, restricts a bit how we structure our modules, but, it doesn’t require any new concepts. One other drawback is here that it is often not clear that the lexical scope is a value, but I think that’s something where an IDE should help and provide the necessary insights.

In code, this looks then a bit like this:

class ExampleModule = Value ()(
  class DoneProcess new: channelOut = Process (
  | private channelOut = channelOut. |
  )(
    public run = ( channelOut write: #done )
  )
  
  public start = (
    processes spawn: DoneProcess
               with: {Channel new out}
  )
)
So, we got a class DoneProcess, which has a run method that defines what the process does. Our processes module allows us to spawn the process with arguments, which is in this case the output end of a channel.

Channels

The other aspect we need to think about is how can we design channels so that they preserve isolation. As a first step, I’ll only allow to send values on the channel. This ensure isolation and is a simple efficient check whether the provided object is a value.

However, this approach is also very restrictive. Because of the deeply immutable semantics of values, they are quite inflexible in my experience.

When thinking of what it means to be a value, imagine a bunch of random objects: they all can point to values, but values can never point back to any mutable object. That’s a very nice property from the concurrency perspective, but in practice this means that I often feel the need to represent data twice. Once as mutable, for instance for constructing complex data structures, and a second time as values so that I can send data to another process.

A possible solution might be objects with copy-on-transfer semantics, or actual ownership transfer. This could be modeled either with a new type of transfer objects, or a copying channel. Perhaps there are other options out there. But for the moment, I am already happy with seeing that we can have proper CSP semantics by merely checking that a process is constructed from values only and that channels only pass on values.

Since the implementation is mostly a sketch, there are of course more things that need to be done. For instance, it doesn’t yet support any nondeterminism, which requires an alt or select operation on channels.

Cross-Language Compiler Benchmarking: Are We Fast Yet?

Research on programming languages is often more fun when we can use our own languages. However, for research on performance optimizations that can be a trap. In the end, we need to argue that what we did is comparable to state-of-the-art language implementations. Ideally, we are able to show that our own little language is not just a research toy, but that it is, at least performance-wise, competitive with for instance Java or JavaScript VMs.

Over the last couple of years, it was always a challenge for me to argue that SOM or SOMns are competitive. There were those 2-3 paragraphs in every paper that never felt quite as strong as they should be. And the main reason was that we don’t really have good benchmarks to compare across languages.

I hope we finally have reasonable benchmarks for exactly that purpose with our Are We Fast Yet? project. To track performance of benchmarks, we also set up a Codespeed site, which shows the various results. The preprint has already been online for a bit, but next week, we are finally going to present the work at the Dynamic Languages Symposium in Amsterdam.

Please find abstract and details below:

Abstract

Comparing the performance of programming languages is difficult because they differ in many aspects including preferred programming abstractions, available frameworks, and their runtime systems. Nonetheless, the question about relative performance comes up repeatedly in the research community, industry, and wider audience of enthusiasts.

This paper presents 14 benchmarks and a novel methodology to assess the compiler effectiveness across language implementations. Using a set of common language abstractions, the benchmarks are implemented in Java, JavaScript, Ruby, Crystal, Newspeak, and Smalltalk. We show that the benchmarks exhibit a wide range of characteristics using language-agnostic metrics. Using four different languages on top of the same compiler, we show that the benchmarks perform similarly and therefore allow for a comparison of compiler effectiveness across languages. Based on anecdotes, we argue that these benchmarks help language implementers to identify performance bugs and optimization potential by comparing to other language implementations.

  • Cross-Language Compiler Benchmarking: Are We Fast Yet? Stefan Marr, Benoit Daloze, Hanspeter Mössenböck; In Proceedings of the 12th Symposium on Dynamic Languages (DLS ’16), ACM, 2016.
  • Paper: HTML, PDF, DOI
  • BibTex: BibSonomy

Language Research with Truffle at the SPLASH’16 Conference

Next weekend starts one of the major conferences of the programming languages research community. The conference hosts many events including our Meta’16 workshop on Metaprogramming, SPLASH-I with research and industry talks, the Dynamic Languages Symposium, and the OOPSLA research track.

This year, the overall program includes 9 talks on Truffle and Graal-related topics. This includes various topics including optimizing high-level metaprogramming, low-level machine code, benchmarking, parallel programming. I posted a full list including abstracts here: Truffle and Graal Presentations @SPLASH’16. Below is an overview and links to the talks:

Sunday, Oct. 30th

AST Specialisation and Partial Evaluation for Easy High-Performance Metaprogramming (PDF)
Chris Seaton, Oracle Labs
Meta’16 workshop 11:30-12:00

Towards Advanced Debugging Support for Actor Languages: Studying Concurrency Bugs in Actor-based Programs (PDF)
Carmen Torres Lopez, Stefan Marr, Hanspeter Moessenboeck, Elisa Gonzalez Boix
Agere’16 workshop 14:10-14:30

Monday, Oct. 31st

Bringing Low-Level Languages to the JVM: Efficient Execution of LLVM IR on Truffle (PDF)
Manuel Rigger, Matthias Grimmer, Christian Wimmer, Thomas Würthinger, Hanspeter Mössenböck
VMIL’16 workshop 15:40-16:05

Tuesday, Nov. 1st

Building Efficient and Highly Run-time Adaptable Virtual Machines (PDF)
Guido Chari, Diego Garbervetsky, Stefan Marr
DLS 13:55-14:20

Optimizing R Language Execution via Aggressive Speculation
Lukas Stadler, Adam Welc, Christian Humer, Mick Jordan
DLS 14:45-15:10

Cross-Language Compiler Benchmarking—Are We Fast Yet? (PDF)
Stefan Marr, Benoit Daloze, Hanspeter Mössenböck
DLS 16:30-16:55

Thursday, Nov. 3rd

GEMs: Shared-memory Parallel Programming for Node.js (DOI)
Daniele Bonetta, Luca Salucci, Stefan Marr, Walter Binder
OOPSLA conference 11:20-11:45

Efficient and Thread-Safe Objects for Dynamically-Typed Languages (PDF)
Benoit Daloze, Stefan Marr, Daniele Bonetta, Hanspeter Mössenböck
OOPSLA conference 13:30-13:55

Truffle and Graal: Fast Programming Languages With Modest Effort
Chris Seaton, Oracle Labs
SPLASH-I 14:20-15:10