Go over there to see what he has to say.

The following appears to be to sequence of events:

• Cadence press release, in September, about their “Palladium Incisive Palladium Dynamic Power Analysis and Cadence InCyte Chip Estimator“, quoting Ran:

Cadence Incisive Palladium Dynamic Power Analysis enables SoC designers, architects and validation engineers to quickly estimate the power consumption of their system during the design phase, analyzing the effects of running various real software stacks and other real-world stimuli. The new offerings also include the Cadence InCyte Chip Estimator, which can now provide what-if power analysis through exploration of different low-power techniques. The InCyte Chip Estimator also generates automatically the Si2 Common Power Format (CPF), which helps drive architectural power specification and intent into implementation and verification.

• Frank Schirrmeister blogged “On Chameleons, Low Power and the Marketing Power of Copy Editing“, basically saying that what Cadence was selling was something that was bound to the RTL level and thus arriving with estimates pretty late in the design process, after most important architecture decisions had been made. Instead, he proposed a flow using virtual prototypes that contained a sequence of successively better estimates, from the usual initial spreadsheet to estimates actually derived from RTL later in the process (or for IP blocks that already exist). Synopsys is not alone in this, Neosera and ChipVision are after similar ideas. I think this approach makes excellent sense, following the idea that getting some kind of approximate feedback from a complete system early in the process is better than getting lots of details from a small part of a system late in the process.
• Ran Avinun then blogged a reply to Frank, at “The Power of Cadence System Power Flow vs. Viewing from the Top“. His contention there is that virtual prototypes have their uses, but that real designers will be using hardware accelerators, as that provides the key accuracy needed to do real power work. Also, he sees the creation of a virtual platform as a big problem, and cites a number of cases where running the actual semi-final RTL with power simulation was key to project success. Also, Ran sees the time needed to create a virtual platform as a big obstacle.
• Frank then replied to the reply, at “Hammers, Nails and the Spirits That I Called …“… where he points out that Ran has some misconceptions about virtual platforms, admits that the Cadence flow works well, but that it does miss the point of early power estimation before the design is too frozen to be much changed. There is a pretty but hard-to-read diagram in the post, from a 2005 article he wrote while at ChipVision in Germany, pointing out the need to evaluate designs with actual test data from the real world.

What do I make of all of this?

Ran’s Points

I must admit that I think the Palladium hardware simulation accelerator boxes are very cool pieces of hardware, which at least used to be based on custom logic systems that use several cycles of a fixed sized hardware to simulate multiples of the hardware’s based emulation capacity (so 10M capacity system can use 10 cycles per target cycle to simulate 100M, for example). However, I do agree with Frank that these are dependent on having actual RTL in place to be of much use.

Another issue with hardware emulators is their overall availability: compared to the number of PCs available in an organization, they are going to be very limited. As discussed in many different forums, a key advantage of a pure virtual platform is that it can turn any programmer’s PC into a target system running the real target software. Without having to book time on a limited set of physical target machines, and hardware accelerators are such limited-in-supply hardware machines. So a virtual platform is much more available to people within, and especially outside, a design organization. Also, unless you are happy to release RTL for your design to people outside your organization, hardware acceleration is going to do little to help your end users get the most out of your design, pre-silicon.

My final gripe with hardware emulators is their limited scope. They tend to max out at a the borders of a single chip, or less. A virtual platform, on the other hand, has much more room to scale, to include multiple chips, multiple boards, or even complete racks and networks of networks. You cannot really do that in any hardware simulation, as it involves too many billions of gates running too many billions on instructions. The general rule of simulation still applies with hardware acceleration: you need to increase the level of abstraction to handle larger systems.

As to the problem faced by Ran’s customers, having RTL but no virtual platform: what were they thinking of? Seriously, if you want to do design today a virtual platform should be your starting point, not an afterthought. Time and again, we see examples today where using virtual platforms gets chips to customers ahead of time and provides the ability to test ideas before committing to final RTL. It seems that Ran agrees with this need, but his means are different:

“As was stated above, big reason our customers use RTL emulation platforms is for accuracy, and while virtual platforms can offer certain performance, eventually the need to accuracy becomes critical and can not be overlooked, even for initial performance and power estimation analysis. Frank seems to forget in his statement above that the average bring-up time of new virtual platforms takes 6-12 months while the average bring-up time of many emulated designs takes days.”

The time to create a virtual platform is actually pretty short, if you do it at a sufficently abstract level of detail and don’t worry too much about cycle accuracy. Also, that bringing up of an emulation depends on having a detailed RTL-level description to start with… which is not necessarily the case. I must say that the cited six to 12 months for a VP (for a single SoC as discussed here) sounds reasonable to me — if you are building a cycle-level model that tries to emulate the final timing (which might not be really feasible at at all). If you work at a higher level of abstraction like loosely-timed TLM, that time shrinks by a factor of ten or so. I agree that in the end, accuracy is critical – but before you get there, the approximations used by the VP will have gotten you pretty far in terms of software development and architecture testing.

Ran is also afraid of the lack of accuracy:

Now, even if you build this platform successfully 9-12 months in advance, how do you know that your virtual platform representing your real design? How do you connect it to your verification and implementation environment and realistic power information? Frank seems to overlook these things. Looking at the analogy of the story described at the blog above, using a system-level platform that is not targeting the actual hardware for performance analysis and power trade-offs guarantees that the Chamelon will become a snake and you will get bitten.

As with all simulations, virtual platforms needs to be used with care and understanding. It might also once again be a matter of system scale: for RTL simulation, you are looking inside a single chip, and the detailed design to save power there. With a VP, you might be looking at whether a particular OS kernel does even care to try to turn off unused hardware at all… and that might be just as important in the end as being accurate in how functional units turn on and off inside an accelerator.

In today’s software-driven systems that mostly consist of existing off-the-shelf hardware, not any particular SoC that is being designed right now, the large-scale behavior and smarts of the software in a setting containing lots of chips and functions is far more important than optimizations inside a chip.

Frank’s Points

Since Frank is a virtual platform supporter just like me, I instinctively agree with his points about VPs being pretty fast to develop and available long in advance of actual silicon. I like the way he deals with power in the ARM DevCon presentation cited (do have a look at it), but still there are some lingering doubts and issues…

What I have a hard time understanding is just how detailed the virtual platforms need to be. The use of SystemC TLM-2.0 LT is sensible for speed, but it seems from the DevCon presentation that the main emphasis is on AT-level (and therefore pretty slow) timing-accurate simulations that look at power cycle by cycle in the target. If that is the case, I think we could almost just as well go get ourselves a hardware accelerator, as cycle-level models (even if transaction-driven)

However, Frank also says this which I cannot but agree with: you should not always run around with a hammer and look at everything like a nail — any reasonable chip design process needs both virtual platforms and hardware accelerators, one cannot really replace the other:

When discussing this matter with a friend, he pointed out rightfully so that both Ran’s and my post suffer from “Hammer and Nail-itis”. In fact, he pointed out, the combination of Cadence’s estimators (InCyte), C based synthesis, Palladium, and Synopsys virtual would be pretty powerful! It’s a good thing then that we acquired Synplicity which brought us Synplify high-level synthesis and Confirma FPGA Prototyping to Synopsys, and of course, that we have existing interfaces between our Virtual Platforms and Eve’s solutions.

Conclusion

To me, the lesson from this discussion is clear: A virtual platform should be the starting point of a new design, but once you get down to RTL, hardware acceleration is really pretty useful. You need both, and VP should come first, not second. It is not an either-or issue, rather I expect system and chip designers to use both tools, and the only question is what should come first, which I think is naturally the simulation in the form of a virtual platform. That also allows the chip to be set into a system context, which is otherwise pretty hard before silicon arrives, and something that large system integrators are screaming for.

However, like all host-compiled execution (which I also like to call API-level simulation) it suffered from some problems, and virtual platforms today might offer the speed of host-compiled simulation without all the problems.

The problems are these:

Most companies that are using host-code execution today use “explicit access”.  This means they require all places in the code that access the hardware to call read() and write() functions so every hardware access goes through a common set of functions and then they use #ifdef to change the hardware accesses to call the simulator if they are doing verification with host-code execution. If they are running on the target system, then pointer dereferences are used.

…

This is where implicit access came in. It provided a way to automatically trap pointer dereferences that were reading and writing to hardware locations and convert the load or store instruction into a simulated read or write. For reads it would put the result into the proper host CPU register and the user had no idea that a line of C code would magically turn into a bus transaction on a Verilog BFM

Yes, that is a right pain, and I have seen lots of solutions for it, none of which have the elegant simplicity of a processor simulation. The “implicit access” system is basically trying to trap memory accesses without overtly changing the source code of a program. I guess the best way to do this is binary instrumentation, but it is still very hard to get to work right and robustly. A simulator is simply much simpler in principle here.

Jason continues later on:

Given the hassle of host-code execution I would prefer to cross compile the software and run the target instruction set. Beyond the implicit or explicit access issue, this also eliminates issues with differences in data type sizes, data structure layout, byte order (endianess) and other differences between the host and target processor.

That is absolutely true! Jason does not mention the additional fun of what happens when the target is running an OS that is happily fielding interrupts, scheduling software tasks, etc. Also, that having to maintain a separate build target and maybe code variant is very expensive, process-wise. The expense that a good virtual platform incurs can be paid for pretty quickly once such reduced friction costs are factored in.

So I guess I pretty  much agree with all that Jason is saying, and thanks him for mentioning Simics. Thanks for the insights into what was done in the 1990s, it always interesting to get pointers to old fundamental and interesting work.

About how the virtual platforms actually work inside: it is not that complicated in principle (but pretty hairy to get it quite right and fast in practice). You have to simplify the timing of the target processor, you have to convert from target processor binaries to host binary format using some kind of just-in-time compilation technique (also called dynamic binary translation or code morphing), and you have to provide some kind of direct access to target memory for the target processor simulation (like the DMI feature in SystemC TLM-2.0, but usually the difficult bits are on the CPU side of that, not the memory side).  The most interesting bit is how to build the surroundign system model to not slow the CPU model down, and for this I can recommend a couple of pieces of writing:

• My ESC 2008 general intro to the subject of virtual prototypes (slides, paper)
• Virtutech white paper on system modeling

The most important data in the PhD thesis is really just how much work it is to do an SMP port of an OS kernel. And how hard it is to get performance up to good levels even with several years of work. Really emphasizes the point that hard work and perseverance and just lots of calendar time is what it takes to create a good SMP OS. That’s why Solaris and AIX are still years ahead of Linux in this respect — you just need to hit the snags, fix them, retest, and hit the next snag. It takes time to polish, basically.

So, if you have any interest in multiprocessor operating systems, Simon’s work is well-worth a read. Also check out his blog at http://simonkagstrom.livejournal.com/.  And by the way, he did pass.

…the discussion, and the need to constantly define our terms (and redefine them, and discuss them when people disagree) makes me wish that the world of electronics, system and software design had some agreement on what the right terms are and what they mean…

I think this is a good idea, but we need to keep the core count out of it…

The reason for the confusion of terms and the strong will to create new terms all the time is really that people feel that there is a real difference between a dual-core x86 processor used in a laptop and a highly integrated 100-core-or-more embedded design for traffic processing in a large switch. And for that reason, they want to define a term to define themselves out of the mainstream desktop/server space with a few large cores.

But the number of cores is probably the least useful parameter to use as a differentiator. If 4 cores is multicore and 32 cores manycore today, in a few years time the decrease in feature width will have moved 32 cores into multi and 128 cores into many… etc. So that is really something is bound to change over time.

I think that rather we need to look at other aspects of a chip design, in particular those that are not just straight multiplication of features. Those aspects that really matter to the kinds of programs the chip takes nicely to, and that architects have to think hard about.

Programming models are not the right answer to this. As Grant says, programming models need to be put in a taxonomy of its own:

A kind of taxonomy of multicore related terms, together with a taxonomy of programming models (SMP, AMP, etc.) that everyone could be referred to when these discussions are held and that everyone could begin to build a consensus around would be of great value to all.

If nothing else, we all know that any programming model can be put onto pretty much any piece of silicon, given a sufficiently thick layer of middleware. It might not be the most efficient way to program any particular hardware in terms of hardware resources used, but someone is going to do it anyway.

So what is left in the chip taxonomy?

I think we need to look at things like where memories are located (global, local to each core, shared by a small group), number of levels of memories, whether they are caches or program-controlled. How interrupts and IO are routed is another interesting aspect. Can any core do anything, or do we have master nodes that can do more things? Are all cores equal in terms of performance and computational ability, or do they differ?

As Grant says, a great subject for academia to dig into.

The comments at the end of the post about some secret activities from the Multicore Association by Markus Levy makes me agree with Grant: please get the ideas and drafts out into the open, and make sure to get the widest input possible!

Sun just bought Montalvo whose hardware I blogged about some while ago. And just like the Apple acquisition of PA Semi, the question of “why” appears. Some analysts blame the simple fact that both Montalvo and PA Semi simply needed to be acquired, since their venture capitalists did not want to put in the next 100 million USD needed to go to silicon (Montalvo) or really expand on the opportunity already at hand (PA Semi). Here is my crazy guess.

Look at the following:

• Sun has seen great success with the UltraSparc T line of processors, which are basically “lots of simple cores on a single chip for thread-parallel applications”.
• Sun is investing in Solaris for x86 and has great success with its x86-based servers (based on AMD processors).
• Montalvo is building something quite similar to “lots of simple cores on a single chip” for x86. Which should run Solaris-x86 and most other x86 operating systems.
• Sun has been buying companies and key components for a while now (AMD processors, Fujitsu processors, the company Afara that created the UltraSparc T line).

So my guess is… based purely on technological similarities and no indirect approaches and conspiracy theories. It assumes that Sun does want to make use of Montalvo’s tech as it currently stands:

• Sun buys Montalvo to build x86-based UltraSparc T-style machines for throughput computing. Nice complement to the current high-single-thread-performance AMD-based x86 machines.

Note 1: The indirect approach theory here is that Sun wants to use Montalvo to put cost pressure on AMD, just like there is speculation that Apple is going to use PA Semi to put cost pressure on Intel.

Note 2: This would also put Sun into direct chip competition with Intel Atom-based designs… which might be slightly less clever. Never mind, do it anyway

Note 3: I have no insider on anything on this, this is totally based on speculation from public tech facts.

Note 4: Similar ideas are bandied in a rumor comment at VentureBeat from early April 2008:

But Sun could also choose to avoid a fight with Intel, using the patents to protect itself and to employ the techniques for power savings in its own future SPARC microprocessor offerings. Sun’s most ambitious processors already employ many equal-sized cores on a single chip; the asymmetric architecture of Montalvo’s chips might add interesting capabilities to Sun’s SPARC line-up. In any case, Sun could be picking up the assets at a fire sale price and using them for strategic leverage.

2. A few more ways of multicores will be gobbled up by eyecandy work as Linux, Windows, and OS X keep fighting on what OS looks the best. And this means lots of easily parallelized threads.

3. Long-term, things look bleaker. How do we make use of a 32-way or even 128-way general-purpose machine?

In the embedded systems that I know and love, claim 1 certainly holds up in many cases. Control-plane applications in core network and telecom systems do feature piles of threads today, and can quite easily be scaled out onto a few cores using SMP. This is what ARM has also been advocating is the case for most of the mobile phone workloads that today run on single ARM cores. Using an ARM multicore will work fine up to four cores, since there is ample threads to go around inside a modern phone. All you need is the OS to be SMP capable, and that seems to be finally happening with the last big RTOSes announcing SMP versions this fall.

Note that there is a different way of using initial multicores in the embedded world, by consolidating what used to be several processors onto a single chip. Basically, using a dualcore processor as a natural replacement for two singlecore processors, pretty much running the same workload. This scenario uses two (or more) different operating systems in AMP mode (see http://jakob.engbloms.se/archives/22). In this way, it is quite likely that quite a few systems can take advantage 2, 3, 4, and maybe even 8-way systems without much work.

Claim 2 makes no sense in the embedded field. At least not in the sense that “your platform software provider will add end-user benefits that eat up more CPU and that does not require you to update your own code”. Maybe you could claim this for mobile phones, but mainstream mobile phone OSes like Symbian have not exactly been aggressive on this front. People don’t seem to be looking for eye candy of that kind in phones — currently at least (update: see comment on this, the iPhone could be changing this tenet).

Claim 3 is applicable. At least for control-oriented applications that run on general-purpose shared-memory machines. Unless you count on a continuation of the consolidation trend: imagine a system where you combine more and more boards from a current rack onto a single chip, or add “more boards” by adding in more AMP operating system instances. It makes sense, since in many cases the actual applications feature ample parallelism that today is exploited by using multiple boards or discrete processing units working in close cooperation to handle the volumes of work present.

For media and radio interface applications, you have a real easy time to use “any” amount of parallelism. But that is more similar to the GPUs used in current PCs than the case for the main processor(s) which is being discussed here.

Long-term, PC/desktop/server computing and embedded computing do have some common challenge of using many cores effectively. But the advantage of embedded computing is that most application domains are effectively parallel by nature, and “all” you have to do is find a way to move that parallelism onto a single chip.

His final statement is that:

Our industry press thrives on a good crisis. The switch to multicore processors has presented the brain trust with the opportunity to drum up a convincing one, and they haven’t let us down. Just try to take it with a grain of salt. The crises we’ve had in the past have mostly been resolved with boring, step-wise evolution, and this one will be no different. Maybe 15 or 20 years from now we’ll be writing code in some new transaction based language that spreads a program effortlessly across hundreds of cores. Or, more likely, we’ll still be writing code in C++, Java, and .Net, and we’ll have clever tools that accomplish the same result.

I think he is right about this, and that the end result will be a set of fairly ugly domain-specific frameworks that makes parallel programming reasonably easy. Just like GUI coding frameworks popped up when GUIs were new, relieving you of the tediousness of writing all the plumbing code. But it took a few years to nail down what was to go into a framework and their

Freescale demonstrated a hardware reference platform with a number of security features for future mobile devices, its i.MX31 and i.MX31L multimedia applications processors. Based on the Arm 11 core designed by Arm Holdings, the chips have a run-time integrity checker that verifies the digital signature of code before executing it. This can help stop malware sneaking onto the device — although it could also be used to lock down a mobile device and prevent the installation of third-party applications, much as Apple has attempted to do with its iPhone.

Prototypes are often designed with additional standard circuitry to make it easier to observe their behavior under test. Probes applied to that circuitry, known as a JTAG interface, can even be used to issue debugging instructions to the microprocessor. The connections for the prototype’s JTAG interface often survive — in different positions on the circuit board — right through to final production. Identifying where these points were located on Apple’s iPhone became one of the goals of those trying to unlock the devices as access to it might have allowed them to debug Apple’s security code.

This is the same concern that was expressed in Strombergsons comments to my post on hardware support for parallel programming. Basically, that remnants of debug support for the development phase can be used in deployment to hack into the device.

And that is pretty hard to get around if you assume you want to do debugging on the device. Which I guess is needed for almost all devices, if nothing else in order to analyze performance on actual hardware. Otherwise, I do believe that virtual prototype platforms and simulators like Simics is a key technology to develop safe applications safely — using a virtual system for debug, you have no need for debug backdoors on the final hardware. That backdoor is only there in the virtual hardware, not in the physical manifestation of the hardware. Of course, it is then key that the virtual debugger cannot be used by bad guys to break into the software. I think that can be worked around by making it impossible to get a complete system image off of a target system, which is back to physical security.

I think we need some kind of thinking akin to the key tenet of crypto theory, that information should be safe even if all algorithms and mechanisms involved in encrypting it is known. The secrecy of the key is all that is needed. In the same vein, we need to ensure that a piece of software running on a particular piece of hardware is protected against access and intrusion, if the attacker gets access to the source code of the program and a simulator for the hardware or even debug hardware. There has to be some kind of “key” mechanism that can be used to ensure this.

• C/C++ is going to be the dominant language for embedded for the near future. Nothing really surprising at that.
• C/C++ being dominant means that parallelism in multicore processors, especially shared-memory systems, will be harder to exploit. That is certainly true.
• Tool vendors have no good idea about what to do next.
• You cannot expect to get traction with a new language.

In a sense, blaming the market for not having the good sense to adapt new tools to tackle multicore.

I don’t think things have to be that bleak.

I do believe that there are some hopeful points that indicate that engineers and vendors are beginning to move towards ways to explore multicore:

1. Increased availability of shared-memory APIs like OpenMP and Pthreads is happening, even if that does not really help much considering that shared-memory programming itself is pretty broken.

2. I do not quite agree with the quote by Erik Heilikka that embedded is a trailing-end adopter of new programming languages:

“The inability of C/C++ code to parallelize coupled with its ubiquity throughout the embedded market is a major issue for multi-core going forward,” Heikkila wrote in a follow up email to EE Times. “Any alternative parallel programming languages certainly won’t materialize in the embedded market, but instead will more likely gain momentum in a more mainstream computing market before making its way into embedded applications,” he added.

Au contraire, the embedded market have many times adopted good solutions that are very different from mainstream software practice.

• High-level model-driven tools like Matlab/Simulink and NI LabView are starting to generate parallel code from a higher-level abstraction. This is one very important way forward.
• The use of UML and SDL and similar modeling/programming languages in the telecom sector should open for better code generators straight to parallel code.
• Erlang is a good example of a good language for parallel systems that started in embedded and is now going mainstream.

So thanks to the domain-specific nature of most embedded work, tools can be created that support the creation of parallel implementations without forcing a programmer to care about shared memory and C. And the tools already exist.

I just have to take issue with this.

The assumption behind this is that CPU power and memory sizes just keep increasing as time goes on, and that you can take this for granted. But that is exactly what is not happening right now, if you count single-thread performance. You have to be parallel to get this benefit. For me, this means that I would be a bit more careful, unless I was certain that my system and my programmers were really clear on how to make the application in question scale out well as cores multiply but single-thread performance stays the same.

To be fair, Joel also points out that a big breakthrough for him would be a better way to do Javascript that increases programmer efficiency and avoids cross-platform difficulties (use some other language frontend and generate Javascript optimized for a particular browser platform). That kind of software innovation sounds a whole lot more plausible to me as a bet to just keep programming bloated inefficient Javascript.

But on the idea that faster hardware will let horribly large Javascript programs run better: I have a hard time seeing that, if more compute power is delivered by parallel machines rather than faster single-thread performance. Parallel Javascript seems like a really bad idea. Threaded programs running inside my browser? Written in a horrible language like Javascript? I can just see it crashing and crashing and crashing all the time. No thanks.

Another problem with Joel’s idea is that sometimes hte hardware platform you target exists now and is fixed. If you code for a particular mobile phone that is on the market currently, you cannot expect it to magically improve its speed over time. It is here and you have to live with that. Same thing for certain embedded applications where hardware remains stable for a long time due to certification and safety concerns. Not everyone can enjoy continuous hardware upgrades…

Finally, sandboxing is GOOD. Not bad as Joel says. If there is one thing that it makes sense to “waste” processor cycles and memory on, it is security. Isolating software from other software and keeping code coming off of the Internet tightly locked up in a little sandbox is a good thing. If we did things more like that, we would have much fewer security problems.

I agree with Steve Gibson of grc.com and Security Now — Javascript is a very bad idea from a security perspective. But it does make for nice dancing applications on the web. Too bad we cannot both have our cake and eat it, too.