Performance Results
This report was generated on 2023-06-15 14:44:48.
The results are usually normalized either to Java JDK8 C2 or Java JDK8 Int, depending on whether we are interested in best or interpreter performance.
For VMs with just-in-time compilation, we report results based on the measurements starting from iteration 100. While this does not guarantee that we measure “peak performance”, it gives us relatively stable results for most language implementations.
Overview
Just-in-Time Compiling Language Implementations
The following set of language implementations reaches the performance of Java on our set of benchmarks within a factor of 2 to 3 on average. To allow for a more detailed assessment of these implementations, we exclude slower ones from the following plot.
Interpreters
Looking at interpreter performance, the plot looks quite a bit different. Here we use Java JDK8 Int as baseline.
First Iteration
Looking at interpreter performance, the plot looks quite a bit different. Here we use Java JDK8 Int as baseline.
Java Performance
Interpreter
C2
JavaScript Performance
Python Performance
Ruby Performance
SOM Performance
SOM Interpreters Performance
SOM Optimizations Performance
Performance Overview Data
The following table contains the numerical representation of the results depicted above.
| Runtime Factor over Java | |||
|---|---|---|---|
| median_ratio | min_ratio | max_ratio | |
| Java JDK17 C2 | 0.98 | 0.84 | 1.37 |
| Java ?? C2 | 0.99 | 0.85 | 1.37 |
| Java-Temurin-17-jit | 0.99 | 0.85 | 1.35 |
| Java-Temurin-8-jit | 1.00 | 0.96 | 1.02 |
| Java JDK8 C2 | 1.00 | 1.00 | 1.00 |
| Java-Temurin-20-jit | 1.01 | 0.78 | 1.36 |
| GraalVM Native EE 22.3 | 1.36 | 0.53 | 1.90 |
| TruffleSOM AST Hotspot EE base | 1.98 | 0.88 | 3.99 |
| TruffleRuby EE 22.3 Hotspot | 2.13 | 0.91 | 4.23 |
| Node.js 20.3 | 2.17 | 0.91 | 2.82 |
| TruffleSOM BC Hotspot EE base | 2.18 | 0.96 | 4.99 |
| Bun 0.6.9 | 2.18 | 0.82 | 4.43 |
| Graal.js EE 22.3 Hotspot | 2.31 | 0.68 | 5.62 |
| TruffleSOM AST Hotspot CE base | 2.58 | 0.99 | 3.61 |
| TruffleSOM BC Hotspot CE base | 2.91 | 1.11 | 5.58 |
| GraalPython EE 22.3 Hotspot | 2.91 | 0.81 | 5.13 |
| PySOM BC JIT | 3.09 | 1.11 | 8.35 |
| PySOM AST JIT | 4.31 | 0.99 | 19.86 |
| PyPy 3.8 7.3.9 | 4.53 | 1.04 | 12.25 |
| Ruby 3.1.0 yjit | 25.33 | 9.97 | 61.21 |
| Java JDK20 Int | 27.50 | 4.71 | 57.18 |
| Node.js 20.3 jitless | 35.68 | 9.97 | 81.27 |
| Bun 0.6.9 jitless | 37.33 | 5.84 | 48.44 |
| Ruby 3.1.0 | 41.31 | 12.88 | 89.81 |
| Java-Temurin-8-int | 46.05 | 5.13 | 181.77 |
| Java JDK8 Int | 46.08 | 5.18 | 183.73 |
| CPython 3.11 | 48.92 | 10.95 | 79.19 |
| Java-Temurin-17-int | 54.43 | 4.48 | 179.82 |
| TruffleSOM AST Native Int EE uber | 55.00 | 1.19 | 132.49 |
| Java JDK17 Int | 57.13 | 4.60 | 188.28 |
| PySOM AST Int | 102.15 | 18.99 | 138.87 |
| TruffleSOM AST Native Int EE super | 106.77 | 18.39 | 167.66 |
| PySOM BC Int | 109.26 | 20.78 | 170.66 |
| TruffleSOM AST Native Int EE base | 114.81 | 23.20 | 170.03 |
| TruffleSOM BC Native Int EE base | 183.96 | 33.74 | 268.33 |
| PyPy 3.8 7.3.9 Int | 227.02 | 58.02 | 377.57 |
| Graal.js EE 22.3 Native Int | 233.02 | 36.73 | 505.37 |
| TruffleRuby EE 22.3 Native Int | 358.05 | 50.22 | 608.99 |
| ykSOM Int | 600.83 | 110.45 | 802.59 |
| GraalPython EE 22.3 Native Int | 738.71 | 68.74 | 1289.49 |
| SOM-RS BC Int | 1670.33 | 420.26 | 3127.36 |
| SOM++ Int | 3118.74 | 620.67 | 6614.88 |
| SOM-RS AST Int | 3673.31 | 822.01 | 6033.10 |
| CSOM Int | 5430.41 | 1364.01 | 15498.90 |
Details for all Benchmarks
The following plots show results for each of the benchmarks.
With JIT Compiler
Truffle Impl. Only
