What?

Determinism really means this:

• Given a certain initial state
• And a certain sequence of external inputs
• The end result and state of the simulation will always be the same

The key to note is that you need to require both the starting state and the sequence of external inputs to be the same in order to get the same result. If either of these change, you can well get a different result. Implementing a deterministic simulator requires all internal events and activities in the simulator to be performed in the same order and at the same time in each simulation run. It means that the host computer environment state cannot be allowed to affect the simulator execution, and that in turn means that all sorting of internal events have to be done in defined orders in all instances.

I have a story about how hard that can be in practice. I once talked to some compiler developers who had the issue that when recompiling the same program with the same set of compiler options, the results might come out different, even on the same machine. The problem was that each run of the compiler was done in a different overall system state, and this might affect how the OS memory allocation functions allocated items in memory. It turned out that in some cases, the precise value of the pointers to the items in a complex data structure were used by standard libraries to handle iteration over nodes in the data structures. Thus, a different memory allocation pattern gave a different iteration order and a different traversal order of nodes, and in the end an almost arbitrarily different result. The correct solution they had to implement was to use a defined lexical ordering to traverse and iterate, not anything dependent on the state of the host machine. It is nothing different in a simulator: define the order of everything, in order to be deterministic.

Why?

The crucial benefit that determinism brings to a simulation in general and a virtual platform in particular is repeatable debugging. With determinism and an appropriate recording mechanism (and most practically checkpointing) you can rely on being able to repeat a run resulting in a bug any number of times with the precise same sequence of events in the simulation. In particular, the same sequence and timing and timing relative to instructions executed for events visible to and relevant for the software running on the virtual platform. Especially for multicore and parallel computing systems this is incredibly powerful, and something that just cannot be achieved on physical hardware (due to its inherent randomness and chaotic behavior, see my 2006 and 2007 ESC Silicon Valley talks for more on this, at my publications and presentations pages).

If you assume stability of the simulation infrastructure and the simulation platform, determinism also makes debugging the simulation itself easier. Often, a bug in a simulation model is repeatable, and with determinism, it is easy to repeat the same external stimulus sequence to the module and debug it repeatably.

Determinism also makes it easy to detect change in the behavior of a simulation: if the same simulation setup results in a different result or final simulation state, you know something in the setup (model, model parameters, or software) changed. There is no randomness that cause changes without some fundamental parameter being changed. Such boring reliable behavior is generally exactly what you want when testing and debugging large, complex systems.

Obviously, once determinism becomes a requirement, missing determinism in a model is a bug in itself — and finding such bugs can certainly be interesting exercises.

Why Not?

Just like for checkpointing, one reason not do to determinism is that it is hard, as discussed above.

The most common reason that people claim to want to avoid determinism is that they want to explore alternatives within their simulation. Basically, there is a need for variability that would seem to be at odds with determinism. The typical argument is that “if my simulation model contains a non-deterministic choice, I want the simulation to expose that and not just make the same decision every time”. This is where determinism tends to be considered evil. However, this argument is not correct.

If we take the case that at some point P in a simulation run there are two different events E and F that can fire (since they are both posted to the same point in virtual time), a deterministic simulator will always select one and the same. This is necessary to reap the system-level benefits discussed above. However, nothing prevents us from programming a change from this behavior into our system explicitly, introducing controlled and repeatable variation. In such a setup, we will have a random decision being made in each simulation run, but one where the outcome in any particular run can be repeated by setting the same random seed parameter.

This brings the best of both worlds: variation to expose issues where there is potential non-determinism or lack of synchronization in the model, and perfect repeatability of the issues this poses in terms of target software and simulation system behavior. The reason for the simultaneous readiness can be considered to be lacking synchronization in the model, in general, and such a randomizer of behavior will expose that at several different levels. But uncontrolled randomness is not the answer.

Another common misconception is that at a higher level, determinism in a virtual platform means that target software will always run in the same way. That is not true, and misses the importance of state in the deterministic behavior equation. If the initial state when a program starts is different, a different execution will result. If software is run on top of any non-trivial operating system, there is plenty of such variation. In one of our simplest Simics demos, we show this by running an intentionally buggy race-condition-ridden program. Each time it is run, it hits a different number of race conditions. But thanks to determinism (best demoed using reverse execution), we can repeat each run perfectly.

Thus, determinism is not equal to constant behavior or lack of variation.

The reverse argument

Finally, determinism is the simplest way to implement reverse execution: if you have recording, determinism, and checkpointing, you can easily virtually reverse the execution by going back to a checkpoint and replay the execution from that point. If you stop one instruction before the current instruction, you have in essence stepped backwards one step in time. This is how both VMWare and Simics implement reverse execution and debugging. And it could not happen without determinism.

In practice, threading as a paradigm for software models of hardware circuits connected to a programmable processor brings more problems than it provides benefits in terms of “natural” modeling.

As I see it, the main alternative modeling paradigm is to use a classic event-driven system, where all activity is triggered by events and run the associated code to completion. This makes execution occur in a series of simulation steps in various part of the system, rather than as a set of (pseudo) concurrent tasks.

Threaded Problems

The most common complaint with threading is performance. This has become very clear in the case of using SystemC for transaction-level modeling. All advice in how to do good and fast TLM coding tells us to use SC_METHODs, which are essentially callbacks that are not active objects in their own right. Note that SystemC models found in the wild are often built on SC_THREADs despite this advice, as that is the “easiest” way to do things. Some convenience systems part of the OSCI TLM-2.0 library also rely on threads to convert between AT-style asynchronous and LT-style synchronous function calls (which is pretty unavoidable, but not applicable in the realm of high-performance simulation for virtual platforms).

Furthermore, using threading as a paradigm (even cooperative single-active-thread cooperative threads like in SystemC or classic MacOS) bring with it the problems of concurrent programming, in that you suddenly need to care about protecting data structures against conflicting accesses, worry about deadlocks, and similar concurrent programming issues. Without threads, all such issues go away.

Note that using threading as a modeling paradigm with truly concurrent execution of models will make the execution have all the problems of parallel programs, especially non-deterministic execution and hard-to-find bugs. At least a cooperative multitasking system tends to be deterministic in the way it goes wrong.

Threading as a hardware model programming style therefore makes concurrent multithreaded simulation harder rather than easier to achieve. Especially if the semantics of the simulation system specifies an interleaved model of execution as the semantics, which is the case for SystemC. In this cases, there is no way to really make SystemC parallel without adding parallelism as some extra library.

However, one of the biggest practical problems with threading is the problem of inspecting, changing, and checkpointing simulation state. With threads, you end up having state stored in local variables on the stacks in the system, as well as in processor registers, the program counter, and other places that are hard to get to from the outside.  This is not just me saying this, I found this well said in the sampalib white paper :

Using threads means that part of the simulation state is in stacks, which may limit the ability to persist the state of the simulation in checkpoints.

Using wait() implies context switch which are costly in terms of simulation speed, and thus often discouraged in guidelines for modeling SystemC™ models

To furthermore drive this point, all librariesfor general program state serialization that I have seen (for C++ and Java, for example) also rely on explicit state stored in objects, and explicitly do not support the “transient” state held in local variables and the program counter. Essentially, only heap-allocated objects are handled in serialization solutions.

Event-Driven Solutions

An event-driven transaction-level hardware simulation is coded in a different way from a naive threaded implementation (but not that differently from a more sophisticated threaded program).

Each device model has to make its state explicit as a set of variables, and preferably also declare these for access for an external tool using something like GreenSocs GreenControl or Simics Attributes. It also has to expose a set of functions to be called when events happen or other devices in the simulation system send a transaction into the device model.

Additionally, you should encapsulate all state in a model inside the model object and not expose it for direct access from the outside. A pure object-oriented style with accessor functions for everything is required for best modularity.

The advantages of this model are clear:

• Concurrency problems are reduced, since each function call will run to completion before any other object or function is activated. There is no need to worry about shared data variables, as they should not exist.
• Checkpointing and inspection is facilitated, since all state is now explicit and declared.
• Performance is typically increased, since there is no need to do context switches between threads. Locality is also increased by having functions run to completion before returning.
• True concurrency is easier to achieve, since each model can quite easily be considered a local-state, shared-nothing, explicit message-passing component similar to Erlang threads. This makes it possible for the simulation scheduler to run multiple models concurrently on multiple host threads. For more on this topic, see my SiCS Multicore Days 2008 presentation on how Simics was threaded.

The downside is that some people consider the programming more complicated. Which is really a matter of appearance over substance: event-driven programming tends to be more robust and easier to follow in the long run, since threaded programming makes things a bit too implicit.

Here is the basic example of a thread that does some periodic work.

Threaded style:

Thread_for_D():
  loop forever:
    do work...
    wait(some time)

Event-driven style, where we just repost an event each time we are called:

Time_callback():
  do work...
  post event(some time, Time_callback)

Another advantage of event-driven models is that such a paradigm makes it clear that you need to be able to accept any call into the model at any time. This makes for more robust code, since it is quite easy to (intentionally or by mistake) encode an expectation on the sequence of activity in a threaded that might not be what actually happens at run-time. In particular, the state of any protocol being acted on will need to be explicitly rather than implicitly represented.

There is much more to be said on how to code in this style, but there are long papers out there to read on this.

High-Performance Event-Driven Simulation

Note that in high-performance virtual platform-style simulation, processors will usually be a special case in both threaded and event-driven styles. That is since the flow of instructions that they execute constitute very many very small actions that cannot affort a context switch between each. Here, the advantage of the event-driven model is even clearer, given some special-casing of processors. This is another long story that I will not reiterate here, but basically, most events as discussed above will be memory accesses from a processor to read and write device registers, and each such memory access can be handled in a single simulation step. No need to switch context or do anything but handle a simple function call. By not having a wait() call to deal with, this mechanism can be kept simple and cheap — which is essentially using an SC_METHOD in SystemC. But in the complete absence of SC_THREADs and their ilk, many other things can be optimized even better.

The End

What I wanted to provide in this almost-article-length post was an idea for the problems that I see threads cause as a modeling paradigm for hardware models, and the advantages offered by a reactive event-driven style. For some reason, this is misunderstood in the modeling community at large, probably because most operating systems and simulation systems in common use today present various forms of threads as the way to model concurrent behavior. However, threads as a prominent user-level programming model are known to be bad in many ways… and modeling is no exception to this rule.

Note that I realize that threads are needed at some level in order to take advantage of multicore hardware, but I think they are best hidden inside a simpler framework that presents a simpler understandable semantics to the user.

What was quite striking this year was the greater difference of opinion between the speakers. I guess that in 2007, most of the discussion was on the level of “ouch, here comes multicore and what are we going to do about it”. This year, we got a bit deeper and with one more year of experience and massive research work, the collective world of multicore have made some progress and gained insights. And that’s when the differences start to show up; the fact that we have differences of opinion tells us that we are starting to dig into details and turning up different answers due to different viewpoints and user experiences.

So where were the differences this time?

• Heterogeneous vs homogeneous cores (on a single chip). Kunle Olukotun clearly supported the heterogeneous style (which is what you with Sun’s Niagara that he designed the basis for). Erik Hagersten was more interested in the difference between thin and fat cores of the same basic ISA, and Anant Agarwal was strongly in favor of completely homogeneous systems (which is what they build at Tilera). In my biased view, I think the argument for heterogeneous in pure energy efficiency is always going to prevail. See some of my previous blog posts on this topic, for some background:
  • DNS Hardware Acceleration.
  • Interview with Kunle Olukotun at the Register.
  • Homogeneous vs heterogenous.
  • Homogeneous vs heterogeneous, continued.
  • IBM Z6 accelerators.
  • Montalvo and heterogeneous x86.
• Domain-specific vs general-purpose programming languages. The same sides here, with Kunle advocating domain-specific languages, and Anant and David Padua more in the general-purpose camp. I like domain-specific better, it seems to rhyme more with what I see people actually doing today to increase programming productivity overall.
• Memory bottleneck or not? The most interesting discussion came when memory bandwidth and cache sizes were discussed. One quite common school of thought over the past few years teach that caches per core will shrink, and bandwidth to get data into and out of a chip is going to be a severe restriction on what can be done. Not all in the panel agreed with this, there was the idea (mostly from Kunle) that in some way the massive bandwidths and low latencies achievable within a chip (compared to between chip in a classic discrete-processors multiprocessor) could make this less of a problem. Personally, I think this is going to be some kind of problem, but maybe not as much as passing data around faster might reduce the need to store it temporarily. Despite the need for more bandwidth, nobody really agreed with Erik’s thought that maybe it makes sense to build chips that do not max out on the number of cores they contain, but rather try to balance core count with achievable IO bandwidth. That idea has some merit.
• Core counts. Moore’s law tells us there are going to be thousands of cores on a chip fairly soon… but if we do not manage to make good use of them, maybe the growth in core counts will slow soon. Putting four or six or eight cores into a general-purpose system makes sense today, but more than that might turn out to be a waste for the vast majority of users that do not have problems to solve and programs to run that can make of more than that. In the same sense, maybe it is better with slightly fewer more powerful cores than a maximum amount of minimalistic cores, considering the state of software available today. So it sounds like a fairly divergent future here.
• Shared memory or local memories? Most of the seemed to be in the camp proposing that shared memory is too convenient not to have, even when it really is bad for you. Several bad jokes comparing shared memory to alcohol, and the moderator of the panel suggesting that a good way to avoid the hangover of shared memory is to stay drunk… whatever that means in practice.

Somethings were generally agreed upon, though.

• Programming is an issue, shared-memory or local-memory or whatever. the idea for the solution varied, however, as discussed above.
• Cores will still be plentiful and that operating-systems focusing on sharing time on a single very valuable core is an idea of the past. The keyword for the future is spatial sharing and reducing the overhead of management (I have some previous blog posts on this topic, especially on the subject of IMA and real-time control when cores are free).
• Virtualization and isolating partitions of a multicore chip from each are necessary mechanisms. Running multiple different operating systems on a single chip will be quite normal, probably under the control of some global hypervisor.

Any comments on this from my small audience? I think the topics under discussion are quite fascinating and the kind of issues on which the success of major chip design projects will be decided. A good architecture with a good programming model has a great chance of success (as long as it looks like a continuation of something existing ).

Kunle Olukotun‘s presentation on the work of the Stanford Pervasive Parallelism lab included a diagram where they showed a range of domain-specific languages (DSL) being compiled to a universal implementation language. That language is currently Scala, and in the end all applications end up being compiled into Scala byte codes, which are then optimized and dynamically reoptimized and executed on a particular hardware system based on the properties of that system. Fundamentally, the problem of creating and compiling a DSL, and combining program segments written in different DSLs, is solved by interposing a layer of indirection.

But this idea got me thinking about what the best such intermediary might be for large-scale general deployment.

And my conclusion is that the Java Virtual Machine might be the best candidate. Not the JVM as it is today, though. Here is my idea:

• The Sun Java JDK and its optimized HotSpot VM is now open-source, thanks to the OpenJDK. This opens the door to new innovation based on solid technology.
• The HotSpot is a pretty good VM, and therefore other languages are starting to use it as a potential backend. For example, Python can be compiled to the JVM, as can Ada, and I expect many other language environments to follow suit. The reason is that developing and optimizing a VM is hard work, and if there already is a good one in existence, targeting that is easier than doing you own.
• I think that long-term, this might well replace C as the universal language that you target when you do special-purpose code generators from custom languages… which are really DSLs.
• Thanks to this foreseen ubiquity of the OpenJDK JVM as a universal byte-code execution machine, it will provide a single point of leverage across a large range of applications in a multitude of programming languages.
• As demonstrated by the work of the PPL and the approach taken by RapidMind, the idea of using an abstract byte code for software delivery makes very much sense in a heterogeneous and networked environment. It also provides a good infrastructure for analysis and optimization. It simply is very sensible.

However, the JVM as it stands today is not really suitable for this. It will need some extensions, which I am not the man to invent. With an open-source common JVM, such innovation will be easier to do. Thanks to Sun for opening up Java! For example:

• Support for dynamic languages like Ruby and Python: not the same dependence on Java-type static typing and Java types. They work well for Java, but less so for other languages. It would be nice with lists for real as well, and not just as a library container.
• Support for threads. Not OS threads, but the typical very light-weight threads used in environments like Erlang and OZ. Or even lighter, like the serial units of computations in the kernels of RapidMind and CUDA and similar GPGPU efforts.
• Support for SIMD operations, to express data-level parallelism which is often pretty easy to find on a source-code level.
• Support for data blocking, locality, tiling of some kind, to control data locality. Maybe this already exists in X10 (which I heard about at last year’s Multicore Day).
• Support for communication using messages, and I assume that the best model for expressing the threads is through local data, share-nothing, message passing. With a special case for sharing large data blocks.
• Some kind of data sharing mechanism that is more structured and understandable for a runtime system than pure locks & shared data.
• And a system that takes such an advanced byte code and makes it run well on any particular machine, be it a Tilera Tile, an 8-core P4080, a 128000-core BlueGene, a GPGPU, or a plain middle-of-the-road UltraSparc T2.

So there is some work to be done. But I really think this idea has some merit… if only I had research funding and some good students. Or a crazy VC.

Drizzle is really aiming at what they see as the long-term future: 64 cores or more. MySQL now being owned by Sun, it sounds like they caught the Niagara bug… but of course they are also looking at x86 hosts. They are doing this in part by tackling a type of problem that is mostly read access, not intense on write activity. Drizzle also removes some heavy-duty DB features like stored procedures to focus on applications that make plain simple SQL queries. Only loads of them. Examples are typical web properties like Yahoo, Google, and eBay.

What was also interesting was the general attitude to be modern in the code. All old 16-bit (x86 yuck) code is gone, as well as 32-bit support (any serious DB server is 64-bit machine anyway). Also, C99 is used throughout. Quite a refreshing take on something that did start with a decade-old code.

The project is hosted at https://launchpad.net/drizzle, where there is more information, code, etc. I have not really checked that part out.

But if you have time, do listen to the podcast. FLOSS weekly is one of my regular listens.

The fundamentals of the GPU model are the following:

• It presents a fixed pipeline of stages through which data is streamed and transformed into final output.
• Some stages are fixed-function (with some parameters), some are fully programmable.
• The programmable stages are programmed in a local state, simple input-to-output transformation style with no access to global variables or any way to affect other computations. In this respect, the model is similar to DSP programming with its DMA-in, compute, DMA-out style. If a bit more automated.
• There is shared global state — but it is read-only, for parameters (textures, etc.)
• Parallelism is present in two dimensions: each stage operates on lots of data in parallel, and all stages execute concurrently.
• The really tricky transformations of the input data stream that involve dependences between data items are encapsualted inside the fixed-function stages. In essence, this lets a few experts take care of the hard part of programming, and presents a streamlined simple model .
• It is possible to users to destroy performance with badly written programs, but the typical use case and hardware design rests on users doing sensible things within a fairly narrow domain.
• Code is compiled into byte codes, which are then translated and optimized for a particular GPU by the driver in the final PC running the application. This two-stage just-in-time compilation (or dynamic recompilation, or whatever we want to call it) technique is a known good way to combine performance with portability.

This model is an interesting pattern as it has been extensively proven in practice. There are large numbers of programmers doing graphics programs, and they seem to have not too big problems in getting GPUs to run massively parallel computations. If you think about it, that is a pretty major success story! It also validates that the idea of “no shared state” that keeps being brought up does simplify programming. The model above is quite similar to what you have in Erlang/OTP for example.

The lesson that can be drawn from this for other domains is likely that you need to create a framework (both in concept and in implementation) for processing that makes the code users write simple, single-threaded, and straightforward. The framework then automatically runs lots of little sequential snippets in parallel, and takes care of resource scheduling and the data flow.

Obviously, there are domains where this does seems harder to do than in other domains, but I think this is the pattern for the future. Unless most parallel applications are “easy” to create, we will not make much use of parallelism. And I think that this can usually be the case.

In particular can see this kind of framework being quite possible for things like packet processing in various network transforms like firewalls, routing, switching, and virus scanning. A smart hardware manufacturer in the networking market should provide these frameworks, just like the graphics chips providers are today.

Finally, here is a nice-looking link to the article, as generated by ACM’s online magazine publishing system:

Link to article at ACM March/April 2008 issue of Queue

asdf

The most important data in the PhD thesis is really just how much work it is to do an SMP port of an OS kernel. And how hard it is to get performance up to good levels even with several years of work. Really emphasizes the point that hard work and perseverance and just lots of calendar time is what it takes to create a good SMP OS. That’s why Solaris and AIX are still years ahead of Linux in this respect — you just need to hit the snags, fix them, retest, and hit the next snag. It takes time to polish, basically.

So, if you have any interest in multiprocessor operating systems, Simon’s work is well-worth a read. Also check out his blog at http://simonkagstrom.livejournal.com/.  And by the way, he did pass.

…the discussion, and the need to constantly define our terms (and redefine them, and discuss them when people disagree) makes me wish that the world of electronics, system and software design had some agreement on what the right terms are and what they mean…

I think this is a good idea, but we need to keep the core count out of it…

The reason for the confusion of terms and the strong will to create new terms all the time is really that people feel that there is a real difference between a dual-core x86 processor used in a laptop and a highly integrated 100-core-or-more embedded design for traffic processing in a large switch. And for that reason, they want to define a term to define themselves out of the mainstream desktop/server space with a few large cores.

But the number of cores is probably the least useful parameter to use as a differentiator. If 4 cores is multicore and 32 cores manycore today, in a few years time the decrease in feature width will have moved 32 cores into multi and 128 cores into many… etc. So that is really something is bound to change over time.

I think that rather we need to look at other aspects of a chip design, in particular those that are not just straight multiplication of features. Those aspects that really matter to the kinds of programs the chip takes nicely to, and that architects have to think hard about.

Programming models are not the right answer to this. As Grant says, programming models need to be put in a taxonomy of its own:

A kind of taxonomy of multicore related terms, together with a taxonomy of programming models (SMP, AMP, etc.) that everyone could be referred to when these discussions are held and that everyone could begin to build a consensus around would be of great value to all.

If nothing else, we all know that any programming model can be put onto pretty much any piece of silicon, given a sufficiently thick layer of middleware. It might not be the most efficient way to program any particular hardware in terms of hardware resources used, but someone is going to do it anyway.

So what is left in the chip taxonomy?

I think we need to look at things like where memories are located (global, local to each core, shared by a small group), number of levels of memories, whether they are caches or program-controlled. How interrupts and IO are routed is another interesting aspect. Can any core do anything, or do we have master nodes that can do more things? Are all cores equal in terms of performance and computational ability, or do they differ?

As Grant says, a great subject for academia to dig into.

The comments at the end of the post about some secret activities from the Multicore Association by Markus Levy makes me agree with Grant: please get the ideas and drafts out into the open, and make sure to get the widest input possible!

Sun just bought Montalvo whose hardware I blogged about some while ago. And just like the Apple acquisition of PA Semi, the question of “why” appears. Some analysts blame the simple fact that both Montalvo and PA Semi simply needed to be acquired, since their venture capitalists did not want to put in the next 100 million USD needed to go to silicon (Montalvo) or really expand on the opportunity already at hand (PA Semi). Here is my crazy guess.

Look at the following:

• Sun has seen great success with the UltraSparc T line of processors, which are basically “lots of simple cores on a single chip for thread-parallel applications”.
• Sun is investing in Solaris for x86 and has great success with its x86-based servers (based on AMD processors).
• Montalvo is building something quite similar to “lots of simple cores on a single chip” for x86. Which should run Solaris-x86 and most other x86 operating systems.
• Sun has been buying companies and key components for a while now (AMD processors, Fujitsu processors, the company Afara that created the UltraSparc T line).

So my guess is… based purely on technological similarities and no indirect approaches and conspiracy theories. It assumes that Sun does want to make use of Montalvo’s tech as it currently stands:

• Sun buys Montalvo to build x86-based UltraSparc T-style machines for throughput computing. Nice complement to the current high-single-thread-performance AMD-based x86 machines.

Note 1: The indirect approach theory here is that Sun wants to use Montalvo to put cost pressure on AMD, just like there is speculation that Apple is going to use PA Semi to put cost pressure on Intel.

Note 2: This would also put Sun into direct chip competition with Intel Atom-based designs… which might be slightly less clever. Never mind, do it anyway

Note 3: I have no insider on anything on this, this is totally based on speculation from public tech facts.

Note 4: Similar ideas are bandied in a rumor comment at VentureBeat from early April 2008:

But Sun could also choose to avoid a fight with Intel, using the patents to protect itself and to employ the techniques for power savings in its own future SPARC microprocessor offerings. Sun’s most ambitious processors already employ many equal-sized cores on a single chip; the asymmetric architecture of Montalvo’s chips might add interesting capabilities to Sun’s SPARC line-up. In any case, Sun could be picking up the assets at a fire sale price and using them for strategic leverage.

Our talk will be on how approximate you can be in simulating multicore computers, and still get useful results out from the software running on the simulator. It is something that we at Virtutech have spent a lot of time working on, and we want to bring our results to a wider community. Really exciting to present, and it is a pity that I could not be there myself.

Any case, the panel said the following, including feedback from the audience (including the author of this post):

• Gary Smith laid out some data indicating that software is overtaking hardware as the main effort even for the core SoC designers for multicore. Today, some 50% is software, by 2012, 75% of the design cost will be software.
• Rudy lamented the difficulty of doing anything with regular unrestrained C code, and
  described some IMEC research where they restrict down C to make it palatable. Probably related to their work on “2D” VLIW architectures. Their “clean C” can easily be analyzed and create statically scheduled code (my interpretation) that runs well without caches and cache coherency. The audience asked how you could actually write a program without using pointers… good point. I think it can work nicely for data crunching, but fails horribly for control-oriented and dynamic codes.
• Grant made the point that there is no need to panic. Today, we can see people actually building successful multicore systems with today’s tools. It is a bit of muddling through, but it does get through. There is a lot of truth to that, but it misses the issue of converting or containing all legacy code that has a hard time moving from a single to multiple processors.
• Grant also said that the applications he had seen the most success with where those typically called “embarrassingly parallel”. But why should anyone be embarrassed that they have such nice problems? There is nothing to be embarrassed about, rather you should be proud of having such a nice system/algorithm.
• Simon Davidmann echoed my favorite theme that simulation is a key tool to develop software for multicore, as it gives you insight and control.
• When asked about hardware debug support, Simon was downright negative and wanted it all in simulation. Grant said that the proper solution was a mix of hardware debug and software simulators, which I agree with (see http://jakob.engbloms.se/archives/17 for some more thinking on this topic).
• Someone pointed out that hardware debug is sometimes taken out of volume chips and is only used in development versions — apparently, that is common in automotive, as the cost of each shipping chip is of utmost importance. Less practical for larger machines, though, where you cannot easily build a development version of your rack/router/switch/server…
• When asked about how to handle billions of lines of legacy code in a mix of C, C++, Java, and other languages, Rudy sounded downright exasperated. He seemed to be most comfortable with expressing algorithms and mapping them down to hardware using tools, rather than trying to deal with managing a zoo of legacy code… that is what I mean with the data-plane/data-crunching mindset of this panel.

Overall, an interesting panel, but a bit disappointing in the lack of large-scale software thinking. Would have been nice to mix in someone with an HPC, server, or large-scale control-plane embedded-systems background in the panel.

• Several cores share the memory controller(s) — quite likely, since there is not room for more one controller per core in >2 core machines. The limitation is both because memory controllers are large and complex beasts, and because the pins needed for each memory interface makes it hard to have more than a few on a single chip. While the real estate for processors easily lets us put 4, 8, or 16 cores on a chip today.
• Modern DRAM controllers are not a strict fifo queue, but attempt to optimize memory bandwidth by prioritizing accesses that are directed to the currently open rows in the banks in the available DRAMs.
• The scheduling strategy used today (as they claim) can be easily monopolized by a thread with a high rate of memory accesses and good sequential locality. There is no attempt to provide fairness between cores or programs.

A simple stream benchmark doing a sequential read through a large array is a simple example of what they term an MPH — Memory Performance Hog. In experiments on real hardware and in simulation they show how it can kill the performance of simultaneously executing programs with somewhat more random access patterns.

So what to make of this?

First of all, this is a real attack, in the respect that this sort of thing can and do happen on current hardware with current software out in the field. How dangerous it is in practice is hard to tell, but it could be an issue for various cases where users are sharing a computer. A bit like the old “fork bomb” on Unix systems. I remember being thrown out of shared Solaris machines a few times due to these (several times unintended by beginning Unix programmers making honest mistakes).

It is more interesting in the context of embedded systems and integrated modular avionics (IMA). As I stated in a few earlier blog posts(63 and 58) I think that the best way to host multiple different applications on a multicore processor is to partition applications sparially across cores. This should be more efficient, simpler, and safer than sharing all the cores across partitions using time sharing.
However, this attack does reflect critically on that idea: if it is this simple to hog the memory and thus kill performance of other cores and applications, it might not be particularly safe to have each core run an independent set of applications of different criticalities. It does mean that in order to ensure performance isolation between applications, you will need additional hardware support of one form or the other. Could be a better DRAM scheduler (as the paper proposes), or a static allocation of a DRAM controller to each core (which is likely infeasible due to pin constraints), or DRAM controllers that do a slightly inefficient but safe allocation of a portion of their bandwidth to each core.

In the meantime, maybe the ugly temporal sharing of the entire chip is the “best” way ahead, as it at least is proof against this kind of attack based on parallel execution of partitions.

To be critical, trying to compete with Intel might not be the best idea around… but it never hurts to try. Also, the name is not unique, there is already a montalvo.com that is not montalvosystems.com. I think the old name “Memorylogix” was more interesting and less prone to website name collisions (yes, it seems to be the same company that briefly surfaced with some stripped-down x86 processor back in 2002 — I have an MPR article to prove it).

Basically, he echoes my sentiment that multiprocessing systems often offer clear performance and efficiency advantages, especially heterogeneous systems. Programming such systems are a bit more complicated, admittedly, but the resulting system advantages are clear. Quote:

A surprising number of embedded computing systems use multiple processors. There are several good reasons for doing this. Segregating different real-time tasks onto different CPUs makes it easier to determine whether they’ll meet their deadlines. Thanks to the yield characteristics of VLSI, using several smaller CPUs also can be cheaper than using one big CPU.

And later:

Like multiprocessing, heterogeneous multiprocessing has some downsides. If you think it’s hard to debug a program that runs on two identical CPUs, you’ll be totally
mystified by a program that runs on a reduced-instruction-set computing processor and a digital signal processor. Programming heterogeneous systems generally requires using multiple sets of programming tools and being extra careful at the boundary between the two systems.

But heterogeneity builds on the advantages of multiprocessors to provide further benefits. Many design studies have shown that specializing the instruction set of a CPU to the task that it runs saves substantial energy and improves performance (see Chris Rowen’s “Reducing SoC Simulation and Development Time,” Computer, Dec. 2002, pp. 29-34). If you start with a uniform multiprocessor and replace CPUs with new processors that are specialized to tasks that actually run in that part of the system, you can gain performance without substantial changes to your software.

This might be simplifying the problem a bit, but overall I think that he is right in his assessment.

Thanks for an optimistic and realistic piece!

2. A few more ways of multicores will be gobbled up by eyecandy work as Linux, Windows, and OS X keep fighting on what OS looks the best. And this means lots of easily parallelized threads.

3. Long-term, things look bleaker. How do we make use of a 32-way or even 128-way general-purpose machine?

In the embedded systems that I know and love, claim 1 certainly holds up in many cases. Control-plane applications in core network and telecom systems do feature piles of threads today, and can quite easily be scaled out onto a few cores using SMP. This is what ARM has also been advocating is the case for most of the mobile phone workloads that today run on single ARM cores. Using an ARM multicore will work fine up to four cores, since there is ample threads to go around inside a modern phone. All you need is the OS to be SMP capable, and that seems to be finally happening with the last big RTOSes announcing SMP versions this fall.

Note that there is a different way of using initial multicores in the embedded world, by consolidating what used to be several processors onto a single chip. Basically, using a dualcore processor as a natural replacement for two singlecore processors, pretty much running the same workload. This scenario uses two (or more) different operating systems in AMP mode (see http://jakob.engbloms.se/archives/22). In this way, it is quite likely that quite a few systems can take advantage 2, 3, 4, and maybe even 8-way systems without much work.

Claim 2 makes no sense in the embedded field. At least not in the sense that “your platform software provider will add end-user benefits that eat up more CPU and that does not require you to update your own code”. Maybe you could claim this for mobile phones, but mainstream mobile phone OSes like Symbian have not exactly been aggressive on this front. People don’t seem to be looking for eye candy of that kind in phones — currently at least (update: see comment on this, the iPhone could be changing this tenet).

Claim 3 is applicable. At least for control-oriented applications that run on general-purpose shared-memory machines. Unless you count on a continuation of the consolidation trend: imagine a system where you combine more and more boards from a current rack onto a single chip, or add “more boards” by adding in more AMP operating system instances. It makes sense, since in many cases the actual applications feature ample parallelism that today is exploited by using multiple boards or discrete processing units working in close cooperation to handle the volumes of work present.

For media and radio interface applications, you have a real easy time to use “any” amount of parallelism. But that is more similar to the GPUs used in current PCs than the case for the main processor(s) which is being discussed here.

Long-term, PC/desktop/server computing and embedded computing do have some common challenge of using many cores effectively. But the advantage of embedded computing is that most application domains are effectively parallel by nature, and “all” you have to do is find a way to move that parallelism onto a single chip.

His final statement is that:

Our industry press thrives on a good crisis. The switch to multicore processors has presented the brain trust with the opportunity to drum up a convincing one, and they haven’t let us down. Just try to take it with a grain of salt. The crises we’ve had in the past have mostly been resolved with boring, step-wise evolution, and this one will be no different. Maybe 15 or 20 years from now we’ll be writing code in some new transaction based language that spreads a program effortlessly across hundreds of cores. Or, more likely, we’ll still be writing code in C++, Java, and .Net, and we’ll have clever tools that accomplish the same result.

I think he is right about this, and that the end result will be a set of fairly ugly domain-specific frameworks that makes parallel programming reasonably easy. Just like GUI coding frameworks popped up when GUIs were new, relieving you of the tediousness of writing all the plumbing code. But it took a few years to nail down what was to go into a framework and their

The following quotes on programming for the Clearspeed chips describes a solution that I find very attractive: use higher-level programming languages that describe an actual problem or computation to be performed, and then let a compiler/code generator take care of generating a suitable parallel implementation:

For one, it notes that a number of applications such as Matlab and Mathematica can run on the CSX600 chips without any changes to the underlying code thanks to work done by ClearSpeed and the software makers and the presence of friendly ClearSpeed libraries.

Using libraries is one solution suitable for certain classes of problems, and can cover a fair amount of the supercomputing market where the number of kernel algorithms used tend to be fairly limited. I wouldn’t try using these tools for programming a telecom switch, but that is not what the hardware is designed for either. The CSX600 chip contains 96 fully floating-point coprocessors, and there are solutions out there using massive numbers of the chips (144, according to The Register, for a total of 13824 processors). Scalability like that is pretty cool.

So it seems that by targeting a selected set of applications, Clearspeed does manage to produce a decently programmable solution. Also, one has to presume that the value to the end users of the solution is great enough that the time spent programming is worth its while.

Another example of a domain-limited solution with great power is what Acceleware is doing:  take the hardware and the software from Nvidia for using GPUs as accelerators, and code a solution applying it to a certain problem domain. This makes it very easy for the end-users to pick up, since they basically buy a package targeted to their problem, with all the hard parts already taken care of. In the case of Acceleware, the problem is electromagnetic simulations, and the customers are big companies like Nokia and Samsung. These end customers only need a month or so to incorporate the acceleration effect into their custom programs. Everybody wins, and the value of Acceleware is in letting a group of users for GPU acceleration share the cost of creating a platform to work from. Classic play in high-tech.

Finally, The Register looks at a few players working with FPGAs as their acceleration platform. This has theoretical immense performance and performance/power, but also a much steeper learning curve for programmers.  In my favorite field of embedded, I rather see FPGAs and on-chip FPGAs being used to create accelerated peripherals or simple algorithms embedded in hardware rather than as general-purpose math accelerators. The difference might not be that big in theory, but it does impact what the style and target programs for the programming tools are. And there is a big difference between floating-point math and the operations needed to decode video or do parallel table lookups.

On x86, the virtualization uses the virtualization extensions in recent editions of Intel and AMD processors. There is no such support on the C64+ DSP series, so they fall back to paravirtualization, modifying the operating systems to play nice with the virtualization layer. Nothing particularly magical about this.

The interesting part is really that people are considering using Linux which is usually very hard to tune for non-standard hardware platforms to run code on a real high-power DSP. In general, that is considered a bad idea since what you want is a thin layer of software to let you do compute programs that make maximum use of the processor. Doing interrupts and other typical OS work on a DSP is a bad idea since it breaks program flow. Some instructions sequences on a C64-type DSP even require you to turn off interrupts in order to work correctly! So I guess the performance of Linux programs are going to be pretty poor… However, the idea here seems to be to move some control-plane functions onto the DSPs in the system, which is going to be poorly performing code anyway.  I guess it makes sense if you have spare cycles on the DSP to be able to do away with the additional control-plane processor (which is usually a small slow processor anyway on the kind of base station software being targeted here).