Cary’s articles deal mostly with database tuning in the Oracle ecosystem, but most of his observations apply to any kind of programming with a performance requirement. It is worth a read. It was good to see him dissect performance, including obvious – but not really obvious – concepts like the difference in usefulness between average and worst-case response times from a user perspective.  In essence, you need to watch the spread of response times, and try to keep the worst times from getting too bad, rather than just look at an average that might conceal extremes that frustrate users.

Cary also made the comment noted in the title of this post. In his opinion, the performance instrumentation built into Oracle has an overhead of -10% – or even -20% or -30%, since it enables optimizations that would otherwise have been impossible to do. This is something worth noting in general – overhead that looks bad when considered as a local cost might be a net benefit in the grand scale of things, by enabling measurements and insight that let a program run much faster.

The ACM case study on Photoshop can be found online as a resource at the ACM Queue, with what seems to be mostly the same content. It was written by Clem Cole, at Intel, who interviews Russell Williams of the Photoshop team. It is very instructive to see how the Photoshop team has built an application that works well with 2 to 4 and maybe 8 cores, but that really needs to reconsider parts of its architecture to scale beyond 8.

Clem from Intel pushes Russell by bringing up various examples of next-generation architectures, in particular the fact that clusters-on-a-chip and NUMA memories look inevitable. The Photoshop people seem to take a wait-and-see approach to this: they first want to see some architecture have real traction in the market before they commit and rearchitect their software to make use of it.

The problems of debugging parallel software are also brought up. There used to be a simple bug in the asynchronous I/O system in Photoshop that took ten years to uncover!  Essentially, the programmers had not considered atomicity properly in the presence of multiple threads. With that kind of example, it is not surprising that the Photoshop programmers are very careful when planning and performing parallelizations.

The target domain of Photoshop is to some extent naturally parallel, but not as much as I would have thought. Since a user might operate on any part of an image, large or small, and maybe start and then abort an operation, it is not just a matter of splitting a image evenly across threads or cores. There is a significant amount of variation in just how parallel things can be in Photoshop.

Photoshop has had an easy-to-use parallelization system in place since around 1994, which lets programmers write simple serial computational kernels which are automatically applied to parts of an image in parallel. The Photoshop program itself takes care of the synchronization between kernels, and the kernels can be simple and robust and without any parallel code inside. This is a pattern that has been seen before, and which does make a lot of sense – if it can be applied successfully. Apparently, this is not necessarily the easiest thing to scale beyond four cores.

The main performance limitation for Photoshop performance keeps being memory bandwidth, rather than raw compute performance. This also limits the need to aggressively scale to higher levels of parallelism: as long as multiple threads do not give more bandwidth, it has proven hard to use more than two or three threads on any multicore processor as that is sufficient to saturate the memory system. Apparently, this is different on the Nehalem (Core i7/i5/i3) generation of Intel multicore processors, where each core has a dedicated non-stealable slice of the memory bandwidth.

For the near future, it seems that the big step for Photoshop is going the route of using GPUs for acceleration, rather than 10+ core main processors.

S4D Talks, Themes, and Topics

More is available in “S4D part 2“.

Tracing and Instrumentation

The papers presented covered a wide variety of topics from a variety of angles. Still, everybody felt that two topics kept coming back in various forms in a majority of the papers and discussions: tracing and instrumentation.

Code instrumentation is not a dirty word anymore. The traditional judgment that inserting probes into your software is plain bad does not apply anymore, at least not in the minds of the people at S4D. Instrumentation was applied to drivers, OS kernels, and regular user-level software. I think the key insight is that there is clear value in having the developers that write a piece of software also mark points of interest in the code. When analyzing a trace of an execution, that means that the information in the trace becomes meaningful to the software developers, as it is on the right level of abstraction. Instrumentation naturally produces traces, which can be fed out using shared memory, networks, special-purpose hardware, and more.

One of the instrumentation trace solutions presented (the SVEN system from Intel Digital Home presented by Pat Brouillette), actually leaves the instrumentation in place in the shipping customer systems. In this way, you cannot really claim that instrumentation is intrusive – it is just part of the software, always. Customers can even activate the tracing in deployed systems, and ship the traces back to the developers for analysis of bugs found in the field. It is another approach to record and replay that touches on my paper on transporting bugs with checkpoints.

The increased interest in instrumentation probably has something to do with the nature of the systems that are being addressed. For systems using shared memory multicore hardware and general-purpose operating systems, the cost of instrumentation is easier to take than for very small constrained embedded systems. Essentially, as systems get more complex, instrumentation becomes more tractable.

Instrumentation can interact with hardware trace and debug functions is a neat way to build a system which is more powerful than a hardware or software system would be on its own. Especially for software stacks involving hypervisors and multiple complex operating systems, that is likely necessary.

Once we have a trace, just like last year, we need to have tools for analyzing the tons of data you get from tracing a modern system. ST talked about a tracing system that generated 100s of gigabytes of data.

One trace aspect that kept coming up was the need for time stamps on trace data. To reconcile multiple traces and understand how different concurrent units talk to each other, a global time stamping mechanism is crucial. There seems to be work on hardware to support this.

Security, Secrecy, and Debug

I moderated a panel on hardware support for debug, and posed the question on how to balance security and the need to debug. This generated a number of interesting answers from the panel and the audience.

The conflict between debuggability and secrecy is there. Even from the same customer you first get “you have to make the internal state of the controller inaccessible and hidden to avoid customers modifying their engines”… and then when a problem appears in the field, they ask for a way to analyze and trace that very same system. Hard to support both requirements in a reasonable way.

A sophisticated solution to debug security from companies like ARM, Infineon, and ST is debug that can be enabled using key exchange. The chips are built with a “locked door” in place, but the keys to the door are kept well-guarded. In this way the same chip can be used in development and in the field.

To support debug of systems involving secure modes like ARM TrustZone, ARM has defined several levels of access in their CoreSight hardware modules. This makes it possible for a debugger to be restricted to just debugging user-level code, just OS and user-level code, or all of the software stack. To me, this sounds like it could allow mobile phone manufacturers to “securely” let their application developers use hardware-based debug, without compromising operating systems or secure boot modes.

The classic technique of using fuses to turn off functions is also relevant, at least for systems with moderate levels of security. This can certainly be overcome using special tools to peel off the top of chips and reconnect the fuses, but the panel seemed to think that that level of attack was in general not worth protecting against. However, the audience pointed out that  this was actually being done to automotive engine controllers and there are people making a good living from such antics.

ESCUG Meeting

The ESCUG meeting was a mix of fairly slick commercial presentations from OVP/IMperas chief Simon Davidmann and SystemC guru John Aynsley, and research presentations of varying quality.

One thing that struck me was that the academics spent a significant time in all presentations about how their approaches were compatible with the existing SystemC structure, where they host their open-source efforts, etc. I guess that is good in that they show a certain concern for reality – but it is also a bit sad that they did not get time to actually talk that much about the core ideas they were bringing forward. I am personally much more interested in new ideas than infrastructure and project management. It does not bode well for European research if this is what people are forced to produce, in lieu of real innovation.

Thorsten Grötker’s Keynote

On Wednesday morning, Thorsten from Synopsys did a look back over the history of SystemC, free from product pitching. He only mentioned Synopsys in his introduction, where the high-level message was that the embedded software is really the key problem for industry today. I cannot disagree with that.

During the SystemC parts of his talk he did say a few things that I did not quite agree with… in particular that TLM was unknown prior to 1999. It was not called that, but it certainly existed in the field of full-system simulation. The main problem is that Thorsten only sees the EDA history of modeling, not the computer architecture and software-driven work that did simulations as far back as 1950 (the famous Gill paper), and fast simulation since at least 1967.

He also claims that with SystemC you have a single language for both detailed and TLM models. That is true… but you still need multiple models, one at each level of abstraction. So yes, one language, multiple models. However, that gluability really comes with a performance and complexity cost. It makes it too easy to slip into bad modeling even in TLM.

An interesting theme that Thorsten picked up from John’s talk at ESCUG is the use of SystemC to model software and RTOS, using the upcoming process control extensions. If you stretch that into the area of software synthesis, it means that SystemC is going to collide with the field of model-driven software development. Will you use SystemC, coming from the hardware world, or UML/MATLAB/Domain-specific languages coming from the software world?  Thorsten makes the interesting point that in order to integrate with that world, SystemC will require some concepts from that world (like pins and clocks enable interaction with RTL). I am not sure that is true, necessarily, I think you can just as well create point adaptors to the same effect.

Getting to Southampton

The University of Southampton hosted the event, and it took place in the university lecture halls. That means that we got free very fast WiFi (unlike any commercial conference venue I have ever seen). The university campus was full of services (unlike the desolate place that last year’s FDL/S4D choose). Housing in the Glen Eyre residential halls was a bit spartan but functional. Felt like being back in my days as a student living in student housing.

The instructions from the conference about how to get to the conference was a bit confusing and incomplete. In practice, it is very easy to get to Southampton from both Gatwick (direct train) and Heathrow (NationalExpess bus 203). At Heathrow, I had a bit of luck with the bus to Southampton. The instructions from the NationalExpress website had me believe that I had to get from Terminal 5 where we landed to the central bus station and then catch the bus at 15.00. As we landed 40 minutes late (14.40), this looked very hopeless… until I found the NationalExpress counter in the arrivals hall at Terminal 5 and they told me the bus would leave at 15.30. Nice, no stress. The bus to Southampton even had free Wifi on board!

Once in Southampton, you then had to take the bus U1A out to the university campus, and finding a bus stop for that was the most difficult part of the journey, actually. Some of the buses from Heathrow stop at Southampton university.

See also “S4D Part 2” for a few more tidbits from S4D.

This was the first multicore event that I have been to where we did not have a keynote speaker or technical paper from a hardware company. So there was really nothing here directly about how to build multicore chips. Rather, the workshop tended to be about how to program, use, measure performance on, verify software for, and generally work with multicore chips. From the perspective of software people, rather than hardware designers.

Obviously, hardware aspects enter into such talks, but it is the perspective of a user, not a designer. For example, a hardware designer could explain how an atomic compare-and-swap is optimized in a multicore device. But here, we saw measurements on the actual operation latencies observed on real machines using such operations. Quite refreshing, and closer to my personal interests.

The keynote by Andras Vajda of Ericsson was quite interesting. The slides are not online, but the main points that I picked up and that I might not have considered before:

• Software development costs can mean that the cheapest, fastest, most efficient hardware is not necessarily the most economic. Too hard to code for means the software development time and effort removes the advantage. Obvious, but worth reiterating. Software is king.
• The workload on a cellular basestation can sometimes be highly linear and single-threaded. For example, serving a single terminal with a very high bandwidth LTE connection. And suddenly shift to a massively parallel workload as a crowd of a thousand all suddenly appear and start doing data downloads. And then go back to serial again. This means that the age-old argument that signal processing naturally “conveniently concurrent” (and here) is not always true. Nice point!
• Thus, we need adaptable architectures that can trade serial and parallel performance over time, and rebalance quite quickly. In the same chip.
• He is a firm believer that homogeneous systems will win out in the end, I still hold on to a belief in accelerators and offload engines and DSPs. This is partially because of an admitted focus on servers and services processors, and not on the baseband and signalling side. Makes sense.
• Domain-specific languages (DSL) are the future of efficient programming. Agree.

On the topic of DSLs, there was a question about the cost to support them. To me, that is a non-issue. In the organizations that I have worked, it seems that maintaining a useful DSL requires at most one engineer. Developing one, a few good computer scientists for a fairly limited time. In any case, they tend to appear organically when good programmers generalize repeated tasks.

I gave a keynote about how multicore has impacted virtual platforms (in particular, Virtutech Simics) with the following main points:

• Multicore targets increase the performance pressure on a virtual platform, as more processors will have to be simulated.
• Multicore hosts means that sequential performance of the host is going down compared to the aggregate parallel performance demands from the targets.
• To handle large target systems, the virtual platform itself has to run multithreaded on a multicore host. Getting this in place is a major, interesting, and sometimes painful process.
• Once you have a parallel virtual platform, multicore hosts provide a very nice boost in scalability and the manageable system sizes. A single multithreaded virtual platform process is also a bit easier to manage from a user perspective.
• All features in the virtual platform have to be multicore and multimachine-aware… meaning that they often get a bit harder to use initially, as there is no “default processor” you can fall back to for debugging setups etc. Everything has to be explicitly targeted.
• Multicore targets have proven to  be a great sales driver for virtual platforms, as debugging software on a physical multicore, multichip, multiboard system is just too painful.

Overall, this was a fun event, looking forward to next year at Chalmers!

There were several topic areas, such as automotive, consumer, and networking. Networking was mostly focused on the issues of multicore hardware and software.

Of particular interest to me was to see a Freescale QorIQ P4080 8-core networking/control-plane processor live for the first time. This chip was announced in the Summer of 2008, with a full ecosystem of software support thanks to Virtutech Simics. Now, when the silicon is here, software is indeed running on it thanks to the long headstart development got with the virtual platform. Note that several demos at the event used the Simics simulator to show the software support for the P4080, as there was only a single chip to go around.

I would have loved to have a meaningful picture of the first P4080 in Europe, but a chip is not really very photogenic – the P4080 processor was in an open computer case, but covered with a 10 cm-high heat sink which made it fairly hard to actually see. That’s the challenge with infrastructure things: they are not designed to be seen… just to do their job well. If you have a new consumer electronics processor, you can at least drive a screen quickly or something. But watching 28 Gbps of Ethernet traffic is not as easy

Jonas Svennebring of Freescale gave a good talk about how the process of bringup on the P4080 had worked out. It was a total validation of the methodology of using virtual platforms, at different levels of abstraction, and slipping in a bit of hardware emulation as well.

Freescale started software development on the functional fast model, and when clock-cycle-level detailed models of subsystems became available, they started using them as well for performance validation for small pieces of code. Any discrepancies in behavior between the two models was then used to correct the models and documentation. Finally, as the RTL for the silicon began to become available, they used a few emulation setups to run parts of the actual RTL (the emulator could only handle a subset of the entire chip), and validate the performance numbers in the detailed model and the behavior of both models. In the end, when the first silicon became available, Linux was up in a very short time (I cannot give the exact number, but it was a matter of days rather than weeks).

This is the typical iterative process that all chip designers are implementing today: using virtual platforms you can get a head start on development of software, and then as more details become available, you tune models and update both designs, models, and software, iterating towards a hardware/software combination that just works once the silicon realization of the hardware comes around.

So that was all cool.

Jonas also showed a die photo of the QorIQ, and that confirmed by opinion from the SiCS Multicore Day: embedded multicore is not just about processor cores and cache, it is very much about accelerators to help offload repetitive work from the processing cores. More than half the chip was such acceleration logic! To me, this is a clear confirmation that heterogeneity is the future of hardware design, and a useful way to spend hundreds of millions of transistors to boost SoC performance.

The same was true for most other Freescale hardware showcased at the event. For example, there was the MPC5606S dashboard processor, running an LCD display with lots of dynamic graphics with 0.2% CPU load on a 60 MHz e200 Power Architecture processor. All the work was done by its display driver and accelerator. It is hard to argue with that kind of efficiency. That chip did not need a heatsink, either. It was just mounted on the back of an example board with no need for any external logic chips. Apparently, it could also have moved some physical gauges and blinked LEDs, but that demo was considered too distracting for this particular setting.

I also gave a talk at the DWF, about debugging software on multicore using virtual platforms. That was fun, as always. Need to get out more on the road and talk in conferences, I think

I think that for next year, the organizers need to find keynote speakers that are not from the general computing multicore world. The Microsoft talk this year was a step in that direction, as it rather came from multicore programming than multicore hardware. Richard and Anders gave very interesting and good talks, no doubt about it. But it would have been nice with someone from ARM or Freescale or Tensilica or TI or ST or Ericsson or Cisco talking about the kinds of multicore embedded hardware that is being developed and used today. For example, the “next new thing” touted by the keynotes this year was GPGPU. Interesting for HPC and desktops, certainly. But pretty irrelevant for most of the people that I know. GPUs are huge, expensive, and power hungry.

GPGPU was one part of the theme this year. It is definitely catching on as the way to do number crunching in the desktop, server, and HPC world. It is not the universal panacea for any kind of parallelism, however, as Hazim and I noted in the panel discussion that ended the day. There are applications (such as parallel Simics…) that scale well on general-purpose cores, but that will never ever work on GPUs. In general, the class of problems that work on GPUs is pretty limited to massive data-parallel problems like image and video manipulation.

In the eternal homogeneous vs heterogeneous debate (follow the tags in my blog for more posts on this topic), GPGPU was grudingly accepted as a good candidate for something that will not be homogeneized with the main processors. Additionally, Richard Kaufmann gave some hints that Intel or AMD are coming out with new chips with more accelerators on board… I guess it will be security, as is already done by Sun and IBM. When I brought up the topic of more accelerators like pattern matching, compression, and the other things we see in chips from Freescale, Cavium, and others, the response was very “can only be economical for very high volume applications”.

It is striking how the GPGPU idea is bringing the classic telecommunications DSP-data plane/CPU-control plane division into the desktop and server space. Without any recognition being paid or any experience being reused from the 40 years that that has been done in telecoms and consumer electronics… as Jack Ganssle often says, us embedded folks get no respect.

In terms of programming, this year was all about general programming languages. Hazim from Microsoft talked about (and demoed) the quite pervasive addition of parallelism to both native C/C++ and managed .net code in Visual Studio 2010. Microsoft is dead serious about parallel programming, and are bringing out a whole set of different libraries and support structures to allow easier expression of parallel code. In the “LINQ” data query language subset of C#, you could add some easy modifiers to “foreach” statements to make them parallel, for example. Having a language that is your own and which you can extend at will certainly pays off in terms of innovation here. C++ moves far slower than C#, that is becoming clearer and clearer. C# and its cousins in the .net system seem to be sneaking in lots of powerful language design ideas from places like Python, and also results from Microsoft’s powerful group of language researchers.

When I tried to bring up the idea of using domain-specific languages to program parallel applications, Hazim had the wonderful comment that “that might be applicable in certain domains…” — yes, that is the idea. By being narrow in terms of target domains, you gain expressive power and semantic insight that helps move programming from “how” towards “what”. But it sounds like domain-specific is a foul word inside of Microsoft — when the audience asked whether LINQ was not a exactly a domain-specific language for data access, Hazim was a pains to point out that it is Turing-complete and that someone had managed to write a Raytracer using it… interesting. This feels more political than market-based. I guess Micro

Richard Kaufmann had some interesting notes on throughput vs TTC (time-to-completion) jobs in servers. In the “cloud computing” era, throughput is much easier to scale: just add more servers. Classic HPC is more oriented towards TTC, as you do want your results within a reasonable time. Quite often, you can most work into a throughput-oriented style by simply running lots of jobs in parallel rather than pushing through a series of jobs sequentially. Note however that we have the entire field of real-time control, real-time communications, etc., that do not work like this. But that is not the market that HP is building servers for, or that Intel and AMD are servicing.

Outside the keynotes, Per Holmberg of Ericsson gave an interesting presentation on the adoption of multicore in the control plane of the Ericsson CPP platform. The core of his talk was the observation that in these kinds of systems, multicore is not such a big revolution.

They have been distributed since the beginning. Thus, scaling by adding more processors (with local memories) is easy and multicore is only a packaging change from that. Also, most performance-intense operations are already offloaded onto DSP groups, network processors, ASICs, or FPGAs. There is not much parallelism left for the control plane to exploit. Essentially, only functions that unexpectedly become performance bottlenecks due to changes in traffic patterns are likely candidates for parallellization. Interesting point, and might be why the EETimes noted that multicore is slow to catch on in communications (the article is a bit flawed).

Patrik Nyblom from Ericsson held a talk about how the Erlang runtime engine was parallelized. From a practical perspective, the most interesting aspect was that this made applications parallel without changing a single line of code in the applications. Of course, applications had to be threaded to start with, but that is the most natural way in Erlang. He mentioned systems containing up to a quarter of a million threads — hard to do that in anything except Erlang.

He described how they had evolved from a simple implementation that worked well on synthetic benchmarks to a truly industrial-strength implementation. The difference was quite radical, as real codes feature more complex communications patterns, and make heavy use of device drivers and network stacks. This process forced the use of more and finer locks, and rethinking the balance between shared and separate heaps for threads.

They also had the opportunity to test their solution on a Tilera 64-core machines. This mercilessly exposed any scalability limitations in their system, and proved the conventional wisdom that going beyond 10+ cores is quite different from scaling from 1 to 8… The two key lessons they learned was that no shared lock goes unpunished, and data has to be distributed as well as code. Very interesting to hear this story from real software developers solving real problems.

The next multicore event taking place around here is the Second Swedish WOrkshop on Multicore Computing (MCC 2009), in Uppsala, November 26-27.

Update: note that the presentations from the event are available via http://www.multicore.se/.

It is one of the more comprehensive introductions to how to think about and use multicore architectures in the high-end embedded space. It is free to download and print, but if you want a printed copy, such can be ordered at a price of (I am told) 15 USD (did not try it myself).

The PDF is at http://www.freescale.com/files/32bit/doc/ref_manual/EMBMCRM.pdf .

It will also be linked from the “Documentation” section for most Freescale multicore chips’ information pages.

What?

Determinism really means this:

• Given a certain initial state
• And a certain sequence of external inputs
• The end result and state of the simulation will always be the same

The key to note is that you need to require both the starting state and the sequence of external inputs to be the same in order to get the same result. If either of these change, you can well get a different result. Implementing a deterministic simulator requires all internal events and activities in the simulator to be performed in the same order and at the same time in each simulation run. It means that the host computer environment state cannot be allowed to affect the simulator execution, and that in turn means that all sorting of internal events have to be done in defined orders in all instances.

I have a story about how hard that can be in practice. I once talked to some compiler developers who had the issue that when recompiling the same program with the same set of compiler options, the results might come out different, even on the same machine. The problem was that each run of the compiler was done in a different overall system state, and this might affect how the OS memory allocation functions allocated items in memory. It turned out that in some cases, the precise value of the pointers to the items in a complex data structure were used by standard libraries to handle iteration over nodes in the data structures. Thus, a different memory allocation pattern gave a different iteration order and a different traversal order of nodes, and in the end an almost arbitrarily different result. The correct solution they had to implement was to use a defined lexical ordering to traverse and iterate, not anything dependent on the state of the host machine. It is nothing different in a simulator: define the order of everything, in order to be deterministic.

Why?

The crucial benefit that determinism brings to a simulation in general and a virtual platform in particular is repeatable debugging. With determinism and an appropriate recording mechanism (and most practically checkpointing) you can rely on being able to repeat a run resulting in a bug any number of times with the precise same sequence of events in the simulation. In particular, the same sequence and timing and timing relative to instructions executed for events visible to and relevant for the software running on the virtual platform. Especially for multicore and parallel computing systems this is incredibly powerful, and something that just cannot be achieved on physical hardware (due to its inherent randomness and chaotic behavior, see my 2006 and 2007 ESC Silicon Valley talks for more on this, at my publications and presentations pages).

If you assume stability of the simulation infrastructure and the simulation platform, determinism also makes debugging the simulation itself easier. Often, a bug in a simulation model is repeatable, and with determinism, it is easy to repeat the same external stimulus sequence to the module and debug it repeatably.

Determinism also makes it easy to detect change in the behavior of a simulation: if the same simulation setup results in a different result or final simulation state, you know something in the setup (model, model parameters, or software) changed. There is no randomness that cause changes without some fundamental parameter being changed. Such boring reliable behavior is generally exactly what you want when testing and debugging large, complex systems.

Obviously, once determinism becomes a requirement, missing determinism in a model is a bug in itself — and finding such bugs can certainly be interesting exercises.

Why Not?

Just like for checkpointing, one reason not do to determinism is that it is hard, as discussed above.

The most common reason that people claim to want to avoid determinism is that they want to explore alternatives within their simulation. Basically, there is a need for variability that would seem to be at odds with determinism. The typical argument is that “if my simulation model contains a non-deterministic choice, I want the simulation to expose that and not just make the same decision every time”. This is where determinism tends to be considered evil. However, this argument is not correct.

If we take the case that at some point P in a simulation run there are two different events E and F that can fire (since they are both posted to the same point in virtual time), a deterministic simulator will always select one and the same. This is necessary to reap the system-level benefits discussed above. However, nothing prevents us from programming a change from this behavior into our system explicitly, introducing controlled and repeatable variation. In such a setup, we will have a random decision being made in each simulation run, but one where the outcome in any particular run can be repeated by setting the same random seed parameter.

This brings the best of both worlds: variation to expose issues where there is potential non-determinism or lack of synchronization in the model, and perfect repeatability of the issues this poses in terms of target software and simulation system behavior. The reason for the simultaneous readiness can be considered to be lacking synchronization in the model, in general, and such a randomizer of behavior will expose that at several different levels. But uncontrolled randomness is not the answer.

Another common misconception is that at a higher level, determinism in a virtual platform means that target software will always run in the same way. That is not true, and misses the importance of state in the deterministic behavior equation. If the initial state when a program starts is different, a different execution will result. If software is run on top of any non-trivial operating system, there is plenty of such variation. In one of our simplest Simics demos, we show this by running an intentionally buggy race-condition-ridden program. Each time it is run, it hits a different number of race conditions. But thanks to determinism (best demoed using reverse execution), we can repeat each run perfectly.

Thus, determinism is not equal to constant behavior or lack of variation.

The reverse argument

Finally, determinism is the simplest way to implement reverse execution: if you have recording, determinism, and checkpointing, you can easily virtually reverse the execution by going back to a checkpoint and replay the execution from that point. If you stop one instruction before the current instruction, you have in essence stepped backwards one step in time. This is how both VMWare and Simics implement reverse execution and debugging. And it could not happen without determinism.

The z10 is the latest generation of the classic IBM mainframe family that started with S/360 back in the 1960s. The simulation for just running the firmware of these beasts is making most other virtual platforms look positively puny – focusing on single SoCs for consumer or digital devices. It also shows that virtual platforms as a technology can scale all the way from single-core bare-metal simple machines that are useful for developing initial software for simple embedded systems up to servers and racks containing hundreds of processing units and very diverse hardware.

The teminology used is unusual, compared to the EDA/ESL and computer architecture research worlds. But it is good. The key concept is a “VPO”, Virtual Power On. For a computer of this class, doing Power On is a major event, and calling it a “boot” does not really cover its full complexity, involving many different layers of software running on the same and different computers. The VPO was targeted at four months prior to hardware tape-out — and this means that at that point in time the virtual system would be complete and the firmware complete enough to do a power on.

The simulation system used for the z10 mixes IBM’s in-house CECsim with Virtutech Simics. CECsim executes the code for the central zSeries processors, while Simics simulates the FSP-1 “flexible support processor” based on the Power Architecture. In previous generations of simulation, the FSP code had been host-compiled and run on an x86 workstation instead of running the actual Power Architecture binaries. Running the real binaries brought additional verification value to the software, finding 3 times more bugs than in the previous host-based simulation:

Because the Simics environment now enables us to execute all FSP code in simulation, a far greater amount of code is simulated. Correspondingly, the number of defects found in simulation also increased, by more than 33(Table 2).

The article also describes how hardware-accelerated simulation of the actual VHDL of complex new IO chips were used to validate the bits-and-cycles-level interfacing between code and the logic, as well as to validate the logic design itself.

Overall, the article is one of best presentations of comprehensive use of various types of simulation tools and techniques to remove firmware defects as early as possible in the system development project.

For more on the history of this, I refer to a previous blog post here, “The 1970 rules strikes again“, where I described some late 1960′s mainframe simulation technology and its uses. Also, browse the back issues of the IBM JRD archives, there are lots of nuggets to be found there!

The first Swedish Workshop on Multicore Computing (MCC) will take place in Ronneby on November 27 and 28, 2008. The call for papers is now out, and it is open until September 26. If you have something cool to present or publish about multicore computing, and happen to be here in Sweden, please do submit an abstract!

Disclosure: I am in the program committee for this event.

It is a simple idea, using multiple host cores to run a virtual platform, with fairly amazing results. Now, using a single computer we can run fairly incredible simulations that were the realm of pure fantasy just a few years ago. We also got a nice new little box to demonstrate it with, an eight-core Dell with 16 GB of RAM. With 64-bit Linux, this thing makes my Core 2 Duo laptop with 32-bit Vista look like yesteryear’s snail…  And creates that giggling feeling that a really impressive new toy brings up in even the most grown up boys. Booting a 16-machine network of PowerPC boards was so fast it was not demoworthy.  I think we have to up the ante to some 100 target machines to make it interesting, and I have no doubt that a combination of multithreading and idle-loop optimization will make that thing be usefully interactive from the target command lines. There are many other wild things we could try on that demo box, once it gets back from the Power Architecture Conferences tour.

The interesting bit is really the uses that this system is put to:

In promoting the ETSS project a program simulator based on an idea of partial interpretation has been constructed, and its principle and design are described in the paper. This new approach has been introduced to provide the simulator with such features as high speed and high accuracy in simulation and simplification in implementation. The essence of the idea of partial interpretation is using direct execution of instructions by hardware and simulation of them by an interpreter in combination, wherewith the hardware interrupt mechanism intermediates the two phases of the whole simulation. An interruption takes place when executing a “privileged” instruction, which triggers the simulation of the instruction. The other type of instructions are normally rendered to direct execution by hardware. The simulation method for devices operating in parallel is also described with respect to the timing control and scheduling. A program simulator of this type provides a powerful tool for debugging “supervisor ” programs and opens a new approach to system expansion.

Note that last part. This is essentially a virtual machine used for operating-system debug. So far, the earliest mention of this idea that I have found. There are similar ideas in a classic 1972 IBM paper. If anyone has seen anything older, please comment and tell me!

It is also fun reading these old papers… they are usually scanned from a paper copy, and therefore really show how papers looked and felt forty years ago. Long before desktop publishing, or even TeX.

The Rock is a very interesting piece of work. It tries to be both a throughput-oriented design like the Niagara/Ultrasparc T machines, and a single-thread high-performance design. Even though on balance, it is more skewed towards the throughput computing aspect. What is very cool is how they use additional threads to help boost the performance of a main thread using “scout threads” (a concept I saw presented back at ISCA 2004). This makes it possible to use threads to either boost single-thread performance OR do throughput, creating a more flexible design than is usually the case. It is also the first commercial implementation of transactional memory. And 16-way. And due for next year.

So far, Rock seems like a very successful and very visionary project that is trying in yet another way to gain momentum by pure hardware innovation. Just like the UltraSparc T line, Sun is trying to out-invent IBM and Intel/AMD. Who seem to be mostly progressing by just piling on more of the same old features. I really hope this play goes well, if we were down to just IBM/PPC & System Z and Intel-AMD/x86-64 on the server and desktop side, the world would just be too boring.

The Intel and IBM talks on programming were both grounded in the idea that to make people accept a new programming language/API, it has to be an evolution of what the programmers already know. Which pretty much ties us down to C/C++/Java/C# with extensions and modified semantics.

X10 is basically Java with some nicely considered features to support local and global memories and programs that can scale to BlueGene-style massively clustered machines. Tim basically tells everyone to stop inventing new languages and focus on improving existing frameworks like MPI and OpenMP in collaboration with industry. Presented in a very funny style, Tim is a great presenter, and tries hard to get the audience to react. In this crowd, most people agreed. Except the Erlang people, who feel that they do have a better solution to multithreading and multicore than any patched-up language in the C-Java family. I must agree with them, and I do feel that Erlang today is mature enough to serve that purpose.

The panel session at the end was very entertaining, where some people (including myself and Joe Armstrong) tried to ask tough questions to the keynote speakers (and Ulf Wiger of Ericsson). Quite engaging and a rare chance to directly engage with some industry heavyweights who otherwise tend to sit on the other side of the Atlantic.

I think the prize for coolest tech of the day goes to QuviQ, a spin-off from Chalmers doing automated testing tools that really work well for parallel and distributed systems. Their method of minimizing the trace of a failed test case is really interesting, and finds things that no human tester would ever find.

I also presented a talk on “Debugging Multicore Software using Virtual Hardware”, in the breakout sessions. I guess our Tools track was the least visited of the three tracks, but the audience asked some good questions. And there were some good discussions afterwards.

However, to summarize the day, I am a bit disappointed that not more is being done on the hardware side to help people debug their multicore and multiprocessor parallel programs. Transactional memory is all nice and dandy and can help simplify low-level locking primitives for threaded programs. But I would like to see much more in terms of smart tracing, hardware breakpoints and triggers, massive synchronized stops, and similar features. And instructions and features that make parallel expressions simpler. Here, the embedded folks doing things like ARM CoreSight seems to have been much more successful than the server-class designers at Sun, Intel, and IBM. But even ARM do not spend more than 10-15% of the chip area on debug support.

I think it would be interesting to see what would happen if you could spend 25-30% of the chip on some seriously powerful debug features. Full support for remote control of all cores at the same time, lots of bandwidth for debug data and commands, and fat traces of all traffic on and off the chip. Performance and event counters everywhere. That would make the peak performance of chip likely less than a competing chip not spending as much space on debug support — but it would make achieving a high utilization much easier, and that might actually make the debug-intense chip more economical. Would be interesting to try. But I guess nobody would dare to buy such a design.