FFast: Good Idea, Too Bad About the Implementation

I just read a short paper by Antoine Trouvé and Kazuaki Murakami from the RAPIDO 2010 workshop on “rapid simulation and performance evaluation”. The paper is “FFast: Efficient Application of Compiled Simulation Techniques To A Fast ISS Over a Virtual Machine”. It explores the interesting idea of how an existing virtual machine infrastructure can be used to build a fast instruction-set simulator, and in the extension, a full system simulator.

To me, this idea is worth exploring, since using a mature VM like the .net CLR (used in this paper) or a JVM would offer a shortcut to get high-quality code generation for a JIT compiler. It could also offer other benefits, as these environments support many advanced configuration and management features. I have touched on this topic before, in the posts “Dream ESL Language” (VM as the basis for a simulator) and “The JVM as Universal Parallel Glue” (that a common VM can  offer huge benefits for an ecosystem).

In the paper, the authors show how they have built an ISS for MIPS which runs at 1 MIPS in basic interpretive mode, but at up to 225 MIPS in the most optimized mode. Decent performance on a 2.6 GHz Core 2, but still an order of magnitude compared to the fastest commercial offerings available. However, I think these numbers are not particularly interesting or relevant.

First of all, they only check the performance on basic user-level programs with no I/O, since there is nothing but the CPU present and they thus cannot run an operating system. This makes the numbers essentially “peak” numbers, for small programs, which is not particularly realistic. Second, their implementation does not go straight to bytecodes, but rather to C# code. This is not how a high-performance solution would work, as it is obvious that they struggle to get performance even on these small benchmarks using that approach. Too much effort seems to be spent on gaming the C# runtime system and compiler, in my opinion.

Thus, the paper does not really reveal anything useful in terms of “is building a JIT ISS using a VM a viable idea”? It probably tells us that it is not necessarily a broken idea, but there is a lot of work to bring the solution up to the level of native C-based solutions.

It is clear that the implementation effort in this case is lower, and that the porting cost to new hosts is also very low, compared to the native C-based approaches used in current industrial solutions. C# should also be more productive than using something like C++ for building a software system.

The most interesting aspect of the idea, and one which the authors do not explore at all, is using the power of the .net CLR to build a dynamic full-system simulator. Using the CLR, it should be trivial to build a solution where hardware models can be separately compiled and loaded dynamically at runtime. Using .net “properties”, it might be possible to support user inspection of a running system. Maybe the .net programming tools offer some really interesting possibilities for the debugging of full-system simulators. However, none of this is currently explored, which is a real shame. I guess I could hope that the authors read this short critique  and get some more ideas for future work, as I really think that virtual platforms could be built on top of virtual machines.  That idea is worth exploring in research.

On the nitpicking side, as always when reviewing academic papers in this blog, the authors seem to be unaware of some very relevant previous work. In particular, they should have mentioned Qemu and Simics. They could have used Qemu for MIPS as a point of comparison to compare speeds between their approach and a native C approach. As it is right now, the reference list looks like a fairly random walk around the DAC and DATE communities, but with little insight into the actual virtual platform or full-system simulation tools available today.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.