The paper describes a simulation for a network-on-chip based homogeneous system containing a “ARM-subset” ISS instances with local instruction and data caches, some local RAM, and also some shared RAM. Each core runs its own local software load, there is no SMP operating system. All communication between cores is over shared memory, using explicit operations across the NoC. All cores run a single cycle before they check communications from their neighbors.

This last point is crucial to understanding why this is feasible at all – in general, simulating a general shared-memory multiprocessor machine on a shared-memory multiprocessor falls down on the synchronization overhead. If your simulation semantics dictate that you synchronize every cycle anyway, and you do not try to optimize each core simulator, there is clearly decent room for parallel execution. By including the cache, they increase scalability, since there is more work per target cycle that can be run in isolation.

After reading the article, I am impressed by their work – just getting this to work is pretty good work. But there are quite a few questions which are not really answered in the article and which are crucial to understanding just how well GPGPUs could be used for this kind of ISS work.

• The targeted level of abstraction is a bit confusing. The authors claim it is “instruction accurate and not cycle accurate”, but still simulate caches and cycle-based communications across the NoC. If I read the paper right, communications will take a varying number of cycles depending on the distance for messages to travel. This is more detailed than a typical “instruction accurate” simulator.
• The target system does not run an OS – that might (but I do not know) be an advantage for their approach, since it probably implies less variation in the instruction flow in cores, potentially enhancing the amount of time that all ISSes in a thread group in the GPU can execute the same instruction. This would seem crucial, as if each ISS was running a totally different program, the instruction execution part of the code would be running serialized.
• They should really try to run the same kind of simulation on a high-end x86 CPU like an Intel Sandy Bridge with 8 or more hardware threads. I wonder if their scaling might not work just as well there – and with a much faster serial execution engine. This should give  a much more relevant point of comparison for GPU vs CPU execution of the simulator than…
• the comparison object they use right now, a JIT-accelerated multicore simulation using OVP seems pretty irrelevant since it is not doing the same thing at all. That simulator does not simulate the caches or NoC, just a large number of isolated processors. They also do not run a parallel program on OVP, but rather a large number of single-core fibonacci and dhrystone programs. Thus, the fact that OVP uses a large temporal decoupling time slice does not matter for semantics. It just does not seem like a very relevant comparison point. OVP and their simulator try to solve different problems – fast execution of general code vs. performance profiling of massively parallel machines.
• As I understand it, the given “S-MIPS” numbers in the evaluation tell us the total number of MIPS that we get out across all target cores. That seems to peak around 2000 – which isn’t necessarily that fantastic if we compare to high-performance ISS work in general where a few GIPS is definitely achievable. It is pretty good considering the level of detail here, though, where i would expect a normal ISS + cache simulator to produce at most a few MIPS. Once again, the authors need to be a bit more precise as to what they compare to what.
• Not having an MMU and not implementing any interrupts or exceptions in the target machines avoids a large part of the complexity of a real ISS. That complexity might well be too much for the quite rigid execution environment of a GPGPU.
• They missed that Simics, unique among instruction-accurate mainstream simulators, is parallel since version 4.0.

So, overall, this paper does not really tell us much whether a GPGPU can be used for instruction-set simulation in general. It does tell us that it might be doable, but there are many crucial complications which are not addressed.

First, “Rethink your project planning with a virtual platform“, which talks about how virtual platforms can change your entire project planning. Essentially, by reducing project friction and risks related to hardware availability, software integration, and show-stopper bugs, you can make projects work much better.

Then we have “Transporting bugs with virtual checkpoints“, which is a shorter, popular science, version of the paper I published last year at S4D. This describes how you can use checkpointing in a virtual platform to communicate bugs across time, space, and teams.

It is way too common to be so busy running around being inefficient that there is no time to think about how to become more efficient. Change also requires some discipline to actually keep pushing at habits until they change for the better.

All of this can be illustrated by tying shoes.

The result: not exactly useful. Netscape 4 was bad back then, and it does not work at all with the current style of web coding.

The title of this blog post is based on one fun little tidbit in the article. Steve actually keeps a functioning BBC Micro system around in the shape of an emulator.  But the emulator is not running on his current PC directly, but on top an emulated Acorn Archimedes machine.  A machine from 1981 emulated on top of a machine from 1987 that is in turn emulated on top of a machine from this year.  A real-life example of how nested emulation can help save our digital heritage (an interesting topic I blogged about before).

He also has a physical BBC Micro around, and it is impressive just how high-quality construction could be back then. I still believe the old IBM keyboards are the best ever made, while the cheap laptop I am writing this on probably costs less to manufacture than that keyboard did… obviously at the expense of keyboard feeling.  Grumble grumble kurmudgeon me, right? Steve loves the way the BBC Micro keyboard was designed, and claims that it could hold up to 10 years of abuse by children. In contrast, my old ZX Spectrum had some 4 keyboard membranes worn out in the six years or so I used it. But that is what cheap gives you.

He also touches on the topic of just how inaccessible computing is getting in terms of getting to grips with the machine and the hardware level. Back in the 1980s, you could pretty easily build things and attach them to your home computer.  Slow buses and low-level access and no stupid operating system getting in your way facilitated that.  He proposes people using a PIC development kit, but maybe an Arduino is a better choice.  I like LEGO Mindstorms, but that is indeed a very high-level view of hardware and robotics.  Not at all like the direct programming experience I had with my first ZX Spectrum back in 1984.

It is also funny to read how 1983-era 16-bit processors were not keeping up with memory – too complex instructions made the machine use memory too rarely. How the world had changed!

Recommended reading!

He showed a very interesting discussion from a few years ago on the x86 memory model and the implementation of spinlocks in the Linux kernel. Various experts went back and forth over whether the final MOV that sets a lock variable to 1 needed to be prefixed by LOCK or not. The discussion ended when Linus Torvalds said “I know that it is needed”. Only to see an Intel architect finally intervene and say “you know, really, it isn’t needed”. This was followed by a series of releases of Intel manuals documenting the x86 memory model, with increasing precision in each release. Intel also actually changed the published rules along the road, withdrawing some optimizations as they realized that they would break existing software.

Note that such a description of a memory model must both describe existing hardware, and serve as the guideline for future hardware. Therefore, there are optimizations that are not implemented today but which are possible given the rules. Such optimization opportunities can be removed from the rulebook as long as they have never been part of shipping hardware, so it is not as crazy as it might sound.

Anyway, the point that Francesco made was both to tell an interesting story from history, and making the point that describing and understanding memory models is hard. I certainly agree with that. I recall an ISCA many years ago when some computer architecture professors all agreed that very few people really understand consistency and weak memory models.

To make life easier for programmers, Francesco and Peter Sewell (in Cambridge) has defined their own set of rules for x86 memory consistency. This is not an architecture spec, but a rule set for regular programmers. It is found at http://www.cl.cam.ac.uk/~pes20/weakmemory/. Essentially, the conclusion is that x86 in practice implements the old SPARC TSO memory model.

They have also attempted to formalize the Power Architecture memory model. Both the actual memory model and their model of it can only be described as very complex. The programmer’s model is expressed in terms of store queues, speculative instruction execution, and commits of instructions. Not something you easily keep in your head. It is interesting to note that ARM MPCore essentially copied the Power Architecture.

He showed an interactive simulation of the Power memory model, and the way that you need to think about it in terms of propagating information between threads and committing them. It is possible to propagate values and then another propagation overrides a value before the thread commits… Fun. Or a headache.

The big take-away from the talk for me is that it confirms the observation made may times before that SPARC TSO seems to be the optimal memory model. It is sufficiently understandable that programmers can write correct code without having barriers everywhere. It is sufficiently weak that you can build fast hardware implementation that can scale to big machines.

Maybe TSO does not theoretically scale in the same insane way as Power or Alpha does/did. But the cost of that theoretical scalability is that programmers might have to litter their code with sync operations just to get it to run correctly. With too many sync operations, the code will run very slowly negating any advantage on the hardware level. Note that sync operations can be very expensive. Doug Lea, in the audience, pointed out that a sync can cost up to 300 cycles on a POWER5.

The key idea of the article is to put the debug service in a thread and the debugged SystemC system in another thread, and stop SystemC using a mutex. Yes, you have to do that.

But the really interesting part is how to connect the debugger into the virtual platform, and what that requires from the models and processors and the infrastructure. Unfortunately, the article is pretty silent on that. There is some talk of breakpoint handling required in the ISS, and how to update target memory that mostly corresponds to the debug interface of SystemC TLM-2.0 in scope.

Also, nothing about multicore debug and how to deal with temporal decoupling and debugging, or the need for repeatability across runs. Or breakpoints on things like hardware accesses and internal actions in the simulator.

Too bad.

TI deploys two totally different platforms for hardware and software development, which makes perfect sense.  The goals are so different between a high-speed software development platform and performance-accurate hardware design platform that trying to force them together would likely just create a bad compromise that is bad for everybody.

Additionally, FPGAs are used to create timing-dependent low-level code, where you need both timing accuracy and decent speed.  It is worth noting that the performance model is mostly “dataless” – it models the timing of actions and their dependencies, but not their values and computations.

The different models serve different purposes, require different levels of effort to use, and become available at different times during the project. The SystemC performance model is always available first and is always the simplest to create and use. The virtual platform is the next to become available. It is used for software development and has very little timing accuracy.  TI uses Virtio technology to create this model rather than SystemC.

Given the number of ultimately failed attempts I have seen at making timing and function available in the same model but as orthogonal concerns, this observation in the article is very insightful:

It would appear the choice of two different technologies for the virtual platform and the performance model is inefficient, wasting potential code reuse. However, the two have completely different (almost fully orthogonal) requirements, and at module level almost no code reuse is possible.

Maybe this is an impossible dream in the general case.

One somewhat surprising statement in the article is that there is no real software available to use in the SoC design phase. Often, virtual platforms are sold as being able to use “the real software” when designing hardware. But in the case of TI, the software is mostly written by their customers, with little available for TI to use. Thus, they are forced to design their own test cases to drive the hardware design process.

The requirements on the simulation technology are first and foremost ease in creating test cases and models and credibility of results. The emphasis on test-case creation is a consequence of the complexity of the devices and of the way in which an SoC platform such as OMAP-2 is used: because the whole motivation is to be able to move from marketing requirements to RTL freeze and tape-out in a very short time; and because in many cases large parts of the software will be written by the end customer and not by the SoC provider (Texas Instruments, in this article), the performance-area-power tradeoff of a proposed new SoC must be achieved without the aid of “the software.”

The platform they built is all based on clock-cycle-level interfaces (CC), which is very natural when the primary use case is hardware design.

The primary component optimized in the TI design process is the on-chip interconnect structure, called the “NoC” in the article. Each SoC variant is built from a set of (usually already existing) devices and processor cores. The main work of the integration is designing an appropriate NoC for the SoC. The NoC design is crucial to the actual performance level the final SoC product will have.

By playing with the topology, the level of concurrency, and the level of pipelining in the NOC, it’s possible to create SoCs from the same basic modules with quite different capabilities.

The only real instruction-set simulators used are CC-level models of DSPs, used for software optimization taking but contention into account. No models of the ARM control cores are used. Mostly, processors are represented by stochastic or trace-driven traffic generators that put transactions on buses but do not actually run any real code.

The stochastic processor models are very powerful and provide traffic that is very similar to a real processor.  A very elegant property of such models is that it is very easy to change the parameters of the model to model quite different software/processor scenarios. Compared to writing real test programs for a full ISS, this is much faster and allows for the exploration of more alternatives.

The stochastic models are used along side function-graph breakdowns of software, essentially models that say that an application does A, then B, then C, and that maybe D can happen in parallel. This model of an application is connected to the hardware simulation and can control when things happen and what goes on in parallel. It amounts to a simple model of what an RTOS would do, to some extent.

Configurability is a key theme throughout the OMAP architecture exploration platform. SystemC being what it is, it is limited to configuration at start-up time, but that is perfectly sensible for an architecture exploration use case where you want to setup and platform and test its performance. Dynamic reconfiguration during a run is not that important.  TI has spent a great deal of effort in making the system easy to configure using parameter files.

The article goes into many more fascinating details on the models used.  I can only say one thing: read it, if you have any interest in these kinds of issues.

Good work, James!

This is a small detail, but one where I have always had a feeling that some fundamental assumption was missing in my discussions with various people from the hardware design community. It always seemed that hardware designers assumed a different basic design – and Grant Martin explained it very well just what that was. They only check for interrupts at the beginning of a time slice. Which makes interrupts less precise  versus the code, but also makes the core interpreter fairly simple since all it has to do is to churn through instructions.

There is another solution, which is employed in Simics, where the processor can take an interrupt at any point in a time quantum. To do this, the processor needs to be aware of what is going to happen. The essentials of the solution is to have devices call the processor and tell it that they intend to interrupt it at some point T in time. The processor simulator then makes sure to stop and give the device model a chance to act at that exact point in time. It is obvious that this solution is easily generalized to cover all time callbacks needed to drive device work. A significant part of the responsibility for running the event-driven simulation is moved into the processor core.

Making the event queue visible to the processor also gives the processor a chance to hypersimulate, or skip idle time. Since it knows the next point in time that something will happen (either the end of a time quantum or an event posted by a device), it can very easily, safely, and repeatably jump forward in time without any impact on simulation semantics.

When dealing with multiple processors, this means that each processor will have precise interrupts from the devices that are close to it. Timers and IO interrupts tend to work closely with a certain processor for a prolonged period of time. Interrupts between processors suffer a time-quantum delay sometimes, but that is no worse than the solution of checking all interrupts at time-quantum boundaries.

Qemu uses a solution which is a mix of the two. According to the 2005 Usenix paper, devices do call into the processor to announce an interrupt, but this is handled by “soon” returning to the processor main loop. Processors are not responsible for keeping track of interrupts, making it very imprecise and not very repeatable when interrupts will happen.

Thus, we can see that there are a few different ways to implement interrupts in virtual platforms. Each approach comes from a different tradition and features different trade-offs.

I was a bit surprised by the comment in the Tensilica chapter that only correctly synchronized programs will work on a temporally decoupled simulation. In my experience, temporal decoupling is transparent to software functionality – all software runs. The perceived timing of operations can be different, and some tightly-coupled code might behave in suboptimal ways, but it certainly runs and works. And lets you observe  parallel code errors.

Temporal decoupling is necessary in any fast platform, and its effect on semantics are really minor. With the simple tweak of having a processor know when interrupts might happen, it will also not affect the device-processor interface very much, maintaining very tight synchronization between processors and their controlled hardware.

I got HAVEGE up on a Simics x86 target model with Linux pretty quickly, and ran the two provided tests. Ent, which is a quick entropy test, and nist which supposedly much more thorough.

To my surprise, they both said the randomness we got was totally acceptable. This would seem to invalidate the fundamental assumption of HAVEGE – that it needs to collect randomness from hardware in order to produce good-quality randomness. To try to understand a bit more of what was going on, I took at look at the execution using Simics Analyzer (the dredd.motherboard.processor lines are the processors, and the orange part is the HAVEGE program, yellow is the kernel):

Zooming in a bit:

We can see that the program is regularly interrupted by the OS, which could be  a reason for random timing variations. The instructions run by the OS should vary in count, which would disturb the time stamp counter values read by the HAVEGE program. That could be sufficient to cause random variations, essentially showing that HAVEGE really works well just from OS noise – even in an otherwise idle machine.

However, at this point I started to have my doubts. Something did not feel right.

So I tried to remove all variations from the HAVEGE program. I replaced the “HARDTICKS” macro in HAVEGE with the constant 0 (zero) rather than reading the time stamp counter of the processor. This immediately failed the randomness test.

However, when I used the constant 1 (one) instead, the ent test passed. And even nist almost passed with only a single missed test out of the 426 tests executed.

Thus, the conclusion is that we do not know how well HAVEGE ‘s collection of hardware randomness works, since the evaluation software is too weak. In essence, we do not know if the collection of hardware randomness matters or not, as the proposed measurement hides the randomness behind a pretty good PRNG algorithm.

Ideally, we would need a measurement that would evaluate the predictability of the randomness generated. Or at least one that can correctly estimate the impact of the variation of low-level hardware timing on the quality of the final random numbers. Unfortunately, that is not the case here, throwing the entire idea into doubt.

I also have to apologize for the slow blogging in January of 2011. There was too much going on at work and quite a few days taking care of sick kids. Hopefully, the pace can improve going forward.