Project Renaissance: Harness Emergence

Avoid the Concurrency Trap by Embracing Non-Determinism

A collaboration of: IBM Research, Portland State University, Vrije Universiteit Brussel

Overview

Ly and Sly

Languages for Non-deterministic Programming

  • Ensembles: collections representing a whole, multi-part entities
    e.g. a flock of birds
  • Facilitate map/reduce-like parallelism
  • Message adverbs and gerunds control parallelism and semantics
  • Sly is an extension to Smalltalk-80
  • Ly: JavaScript-like language
    implemented in Smalltalk

RoarVM

The Manycore Squeak VM

  • VM for Squeak and Pharo Smalltalk
  • Derived from original SqueakVM
  • Runs on multicore and manycore: tested up to 59cores
  • Standard Smalltalk but parallel: [42 primeFactors] fork
Showcase
Downloads

Renaissance/Sly Images

One-Click-Image
binary + image
MacOS X
Renaissance image
prepared for Sly3 introduction
Archive
Latest source code snapshots
github (release pending)
Archive Archive
Latest pre-build images
(release pending)
Hyperlink

RoarVM Binaries and Source Code

One-Click-Image
binary + image
MacOS X
RoarVM binary
release build
MacOS X Linux Tilera
RoarVM binary
debug build
MacOS X Linux Tilera
Latest binary snapshots Hyperlink
Latest source code snapshots
github
Archive Archive
Get Started

Brief Introduction to Sly

      "numbers and squaredNumbers are ensembles
       indicate with the % sign.
       Sending a message to them will cause
       all members to evaluate the message,
       ideally in parallel."
      numbers := Sly3Ensemble
          withMembersFrom: {1. 2. 3. 4. 5}.
      squaredNumbers := numbers squared.
         "squaredNumbers = %{1. 4. 9. 16. 25.}"

      "Gerunds modify a function
       to be a reduction operation"
      %{true. true. false. true} andING

      "Need a cartesian product?
       The RoundLy adverb will execute the plus
       by iterating over the operands"
      %{1. 2. 3} plusRoundLY: %{10. 20. 30}

      "For more examples, see Sly3SyntaxTests
       in the Sly3 image."
      Browser newOnClass: Sly3SyntaxTests class  
  

Additional Material

Everything You Know (about Parallel Programming) Is Wrong!: A Wild Screed about the Future
Invited talk at DLS'11, SPLASH'11 by David Ungar
Recording of the talk at CMU Recording of the talk at CMU
Harnessing emergence for manycore programming: early experience integrating ensembles, adverbs, and object-based inheritance
David Ungar and Sam S. Adams, IBM Research
©ACM, 2010. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in proc. of SPLASH'10.
PDF Link
Inconsistency Robustness for Scalability in Interactive Concurrent‑Update In-Memory MOLAP Cubes
David Ungar, Doug Kimelman, and Sam Adams, IBM Research
The work was published in proc. of the Inconsistency Robustness 2011 Symposium.
PDF Link

Building the RoarVM form Source Code

    ## Setting up and running the RoarVM on *nix-like Systems
    cd /tmp         # moving somewhere safe

    #  Get the VM code from github
    git clone --depth 1 git://github.com/smarr/RoarVM.git
    cd RoarVM

    # Read the INSTALL.rst file and make sure all prerequisites are meet
    open INSTALL.rst || less INSTALL.rst

    cd vm/build
    ./configure
    make -j -l 1.5   # those parameters let make use all your cores
    # optionally, install it: sudo make install

    # get the latest image
    cd ../run
    wget http://soft.vub.ac.be/~smarr/roarvm/Sly3.zip
    unzip -j Sly3.zip

    # Start the VM using 2 cores and 512 heap
    ../build/rvm -num_cores 2 -min_heap_MB 512 renaissance.image
    

Additional Material

The Sly3 Programming Language
An introduction and analysis by Pablo Inostroza Valdera
PDF Link
Guide to the Optimizations of the Sly3 Evaluation Mechanism
Project report by Pablo Inostroza Valdera
(release pending)
PDF
Hosting an object heap on manycore hardware: an exploration
David Ungar and Sam S. Adams, IBM Research ©ACM, 2009. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in proc. of DLS'09.
PDF Link
Multiprocessor Smalltalk: a case study of a multiprocessor-based programming environment
Joseph Pallas and David Ungar, Stanford University
Link
Get Involved

What next?

The RoarVM is a research virtual machine, and the main goal is to make it easy to experiment with new VM techniques and languages. Thus, the RoarVM is not optimized for performance.

For the Smalltalk community as a whole, the approach with the most long-term benefit would be to integrate the techniques tested in the RoarVM into the standard SqueakVM and the Cog just-in-time compiler.

To facilitate an integration, the RoarVM uses the same code base:

  • Fully Compatibility: load and save images from the CogVM, StackVM, Standard interpreter + RoarVM
  • Tracking Upstream changes with Git integrate-with-upstream
  • Compatible with standard plugins
  • The RoarVM can just replace interp.c

The iPad version of the RoarVM uses the standard Squeak iOS code, and replaces just the interpreter. Ok, we also added some code to be able to handle input in our images, but it is still Squeak-compatible!

RoarVM on the iPad2 with two cores!

Showing the Sly image on the iPad2 simulator running on two cores.
Project

Retrospect

The Renaissance project started in IBM Research and after involving Andrew P. Black from the Portland State University and Theo D'Hondt from the Vrije Universiteit Brussel it became an open source project.

In November 2010, IBM Research released the source code of the RoarVM and the Sly language implementation officially under the Eclipse Public License to invite feedback and participation in the project.

SPLASH'10 Birds of a Feather

The main question, can we use non-deterministic programming to solve today's concurrency problems, was discuss during a birds of a feather meeting at SPLASH 2010.

Project Infrastructure

Involved People

IBM Research
Renaissance

  • Sam S. Adams
  • David Ungar
  • Doug Kimelman

Portland State University
Andrew Black

  • Andrew P. Black
  • Max OrHai

Vrije Universiteit Brussel
Software Languages Lab

  • Theo D'Hondt
  • Stefan Marr
  • Pablo Inostroza Valdera
  • Mattias De Wael
  • Reinout Stevens
  • Wouter Amerijckx
Goodies

Smalltalk Goodies

SMark

SMark is the attempt to build a common benchmarking framework for Smalltalk. It inspired from the metaphor used for unit testing in SUnit, and thus, a benchmark is implemented by adding #benchMyBenchmark to a subclass of SBenchmarkSuite. The code is originally based on PBenchmark the benchmark framework used for the PinocchioVM and RoarBenchmark a framework used for performance regression testing of the RoarVM.

Transporter

Since the Renaissance project started from a minimal Squeak image without support for Monticello, we needed a structured way to manage our source code.

Transporter is a tool to file out code into standard Squeak file-out format and aid it with a very basic package semantics to enable efficient source code management using Git or other file-base version control systems.

Virtual Machine Goodies

Pauseless Concurrent Garbage Collection

Not yet completely ready, but on a good way (just one bug left!) is a new garbage collector for the RoarVM. It is inspired by Cliff Click's Pauseless GC Algorithm and thus, will provide a scalable way of garbage collection that avoids global synchronization which interrupts all cores. Avoiding global synchronization is key to performance on systems with more than 64 cores. The RoarVM will be the first open-source interpreter VM which comes with such a pauseless GC.

Manycore Smalltalk Scheduler

Another limitation for scalable fine-grained parallelism on the RoarVM is its global scheduler. This project brings a work-stealing-based scheduler to the RoarVM to have better scalability for Sly's fine-grained parallelism, or to facilitate fork/join-like usage of the standard Smalltalk.

ReBench

ReBench is our tool to execute the virtual machine benchmarks. Fully integrated with our Codespeed instance allows ReBench us to continuously track the performance of the RoarVM.

ESUG Award Application

Application for the Innovation Technology Awards

Hereby, we, the whole Renaissance team, would like to apply for the ESUG Innovation Technology Awards 2011.

Name of the Software: Sly and the RoarVM: Exploring the Smalltalk Manycore Future

License information: Sly and the RoarVM are open source under the Eclipse Public License

Team: We are students and full-time researches from IBM Research, Portland State University, and Vrije Universiteit Brussel. Please see the project section for details.

Downloads: please see the downloads section.

A non-deterministic collection sort on the RoarVM

Video by Max OrHai.

Feedback

We are open for your feedback. Please ask us if you have any questions on how to use the RoarVM, or how to use Sly.

The VM-dev mailinglist is one way to reach us. Other options are to follow us on Twitter @roarvm or watch and fork us on GitHub.

Another option is to use Disqus to leave us your feedback below.

Fork us on GitHub!
Sly   RoarVM