Adding electronics to systems that used to be mechanical has been the great wave of innovation for a quite a while now. Modern transportation just would not work without all the electronics and computers inside (someone once quipped that a modern fighter is just a plastic airplane full of software), and so much convenience has been provided by automation and smarts driven by electronics. However, this also introduces brand new ways that things can break, and sometimes I wonder if we really are not setting ourselves up for major problems when the electrons stop flowing.

I was confronted with a particularly silly example the other day at the gym. There was a handwritten note taped to the wall next to the tap in the locker room, saying essentially that “This tap does not work since the battery is out. Facilities services have been notified and should fix this shortly.”

What? A water tap that does not work since its battery got depleted? What about attaching it to mains power? Or even just making it mechanical, using some kind of directly acting mechanical actuator to turn the water on and off? Or just having a manual override or emergency actuator? This was not a life-or-death situation, but it did showcase the vulnerability of electrically powered machinery when the power goes out. We have far too many of these unnecessarily electronic system in society today. I think we need to think more about robustness and less about convenience.

A few days later, I was pointed at an example that could actually be life-or-death. The Littmann Electronic stethoscope. As you can see on the right, the sensor part of this device has an on/off switch, a small display, and some buttons to control its many functions. It does add some really useful features to the device. With electronics, you can do noise reduction and bring out the essential sounds from the background. Collected sounds can be transmitted to a computer for recording, storing, and analysis. It is a very good example of how electronics can enhance and in many ways transform a traditional instrument by making it so much smarter. A completely brilliant product development that I could see myself being part of developing. But.

Battery dead, device inoperable.

A classic stethoscope is a nicely mechanical device that a doctor can use anywhere, anytime, with no need for a hospital, electricity, or any other facility. I find that very reassuring (probably having read too much disaster recovery and war history literature). Imagine the situation in the emergency room when a doctor has to stop looking after a patient to chase a new battery for their stethoscope (please hold it right there…). Maybe it should have had a mechanical backup. Or maybe it is simply a matter of thinking about the tools of the trade before going out into the bush where spare batteries will not be found.

Speaking about batteries, the most obvious example of big problems due to an empty battery is the modern smartphone. A friend of mine pointed out that unless you keep your phone charged during the day in the office, it might not have enough battery left on the way home to get you home. Why? Since the way to buy tickets for the commuter train is to use an app on the smartphone. Seems convenient, and is convenient, but the number of ways that this can break is scary when you think about it. Apart from local failure on the phone (in particular, finding a modern phone with a battery that can survive a day of use is hard), you also rely on the phone network and the sales servers all being online at the same time. If any link fails, it will make it a bit harder to get home (or to work, for that matter).

Overall, I guess my feeling is that we need to think a bit more about how to build systems that are robust in the face of power failures and network failures. A bit of mechanical backup does not hurt, and avoiding some features to preserve robustness would seem prudent.

Pretty much everything today is being billed as “cloud” as soon as something is connected to the Internet. That does not seem right to me. We need a stricter definition that actually helps focus the mind and reduce the space to something essential. To me, cloud means: centralization, specialization, and data transfer.

Centralized. There has to be something running in a central location (and a set of data centers like what Facebook is using does count as central location, even if it is actually a whole host of different sites in practice). Cloud-based services is a total reversal of the trend towards pushing computing out into the end points and people’s own machines that was dominant until quite recently. Minicomputers, microprocessors, home computers, PCs, … all of the past trends in hardware have pushed computing out closer to people. Now, with the rise of cloud services, computing is being centralized again.

The most widespread and successful cloud services tend to have custom local clients on connected machines, but this does not mean that they are not centralized. In practice, a web page is not a sufficient interface for most interesting services, especially not on mobile devices.  In a way, cloud services are sufficiently complex that they make sense to embed in custom GUIs, and they benefit greatly from a deep integration with the local file system and data store. A web browser for all its power does not offer the same ability to access to the local machine and integrate into the functions of the OS as a real application.

The centralization aspect means that I would not count peer-to-peer file sharing services as cloud.

Specialized. From an economics perspective, it seems to me that many of the best cloud-based services are examples of classic Smithian specialization by division of labor. Using the Internet, it is possible to aggregate many but minor demands for a particular specialized service into a total demand that makes it possible to create a business focused on doing just one thing and doing it really well. The prior state was many players doing the same thing as part of a broader business, with less skills specialization. The companies providing the services become specialists in a particular function, and can thus do it more efficiently.

This is nothing new in itself, the only difference is the delivery mechanism.  For example, consider the traditional highly skilled consultant that do the same occasional but valuable work for several clients.  Another example is the creation of standard software that is used by many different customers (instead of each customer building their own). In the past, things like specialized operating-system vendors, commercial software vendors (and standard software packages), and open-source software have all been the result of the same kind of aggregation of demand and specialization of provisioning.  Indeed, many of the current business-process-as-a-service or software-as-a-service vendors could have been sellers of well-written standard software a decade ago.  The main difference is that with cloud services, the software runs on the vendor premises instead of on the customer premises.

Data-based. Cloud-based services have to move some kind of data to the servers for processing, and that data processing on the server has to achieve something that just cannot be achieved locally. For mobile applications in particular, this is becoming a very common design pattern (in part to achieve better security). Which really is back to terminals and mainframes, with a fancier terminal and a less imposing mainframe, but still the same idea. Proven in use for a very long time. If there is no data transfer off of the local device, it is not a cloud service.

An interesting modern spin on centralized data processing that is very common in cloud systems is that the services correlate and aggregate the data from many individual users and then feed the result of the aggregation back to the individuals. This creates a network effect where the service provides a collective value to its user base. Social networks definitely belong here, and even more services like Waze. The new thing here is really the individual getting direct personal benefit from the service, while traditional centralized data processing tended to be done for the benefit of the organizing organization. Giving back to the individual users is thus a crucial component to a successful cloud service.

Using the cloud as a mirror and replication service is a common variety of data processing. Cloud storage like Dropbox or the Google and Apple sync between mobile devices and computers are examples of this. Here, a server is used instead of more classic point-to-point peer-to-peer data movement, based on the fact that it is often easier to talk to an Internet server than to the computer next to you (even within a home network, dropbox usually synchs things more transparently than trying to make the devices copy data between them, in my experience).

Price is not an interesting factor to me. Often when people present cloud services, they often describe “low price” or “cheaper” as a key part of their value proposition. To take an utterly unoriginal example, Salesforce.com provides even the smallest firm with access to world-class IT support for their sales force, at a very low price. But that low price really results from the division or labor and specialization. The reason is that the provider here is more efficient than old providers, but not that they are on the Internet per se. The Internet is an important enabler, for sure, but not in itself a cause of low prices. I can certainly see room for cloud services that are very high priced, as long as they provide a corresponding value. Business as usual, from a product management perspective.

So, in summary, there is something really interesting with Cloud, and I want to pursue. But I find it more interesting to see how it can be used in high-value business-to-business transactions, rather than your typical consumer-oriented play. I just like hard problems, I guess.

update, after some  more reading and thinking

Standardization. One aspect that seems very common when people describe cloud plays is the idea that you sell a very standard product. Compared to the amount of customization and variation (in which hosts to run on, for example) seen in standard PC software, a centralized server park does allow a new level of standardization and simplification of the software. A note in the Bessemer Ventures rules for cloud businesses really brings home this point: only build a single variant of your software, as that lets you innovate faster. This makes sense, since the server end of a cloud system is theoretically at least totally under your control and can be made very homogeneous in terms of hardware, operating systems, and middleware. Thus, one possible additional criterion for a cloud play is that it is highly standardized and offering the same thing to many users and not making allowances for variations. This would seem to lead to higher efficiency in development and operations, and is an economical argument for specialization. Why would the user need to care about this, leave it up to the experts!

I am less sure about the Bessemer statement that you never should allow on-premise deployment for customers — there are cases where that makes eminent sense, in particular for very secure systems. Sure, banks have standards for their IT providers and similar, but there are cases where physical control is necessary and sane. Hopefully, you can then make the customer use your favorite software stack to run the on-premise version.

For Simics, the result looks like this:

Note that all services, and all GUI views, are driven across a single connection to the Simics process. We do not need a special debug connection for each target system, and we do not need to separate debug and analysis and general Simics things. They all travel across the same single connection, thanks to the service orientation of TCF.

It is important to note that TCF is a completely open-source project under Eclipse, and it is available for any and all debugger and tools vendors to use.

I read a number of articles and blog posts discussing how to move Angry Birds save games, but nothing that was particular to Bad Piggies. However, the pattern that emerged was pretty clear. The save game state is just a bunch of files, and if you can simply get them from one device to another you should be fine. There were some indications that  maybe with Android 4.0 or later you would need to root your device in order to get at the files, which was a bit worrying.

Turned it out it was really easy, even though I had to take a small detour.

First, I attached the Nexus 7 to my PC. It was no problem to navigate to Internal storage/Android/data/com.rovio.BadPiggiesHD/and just copy the whole files directory to to my PC’s disk. I then attached the Xperia to the same machine, and tries to locate the same directory. It was not there. Instead, I copied the files folder to another location (on an SD card, but I don’t think that matters). On the phone I then used a file manager program to copy the folder from the SD card to the internal memory. On the phone,  I could see the target location and copy the files folder into it (image shrunk to 50% to fit the blog format):

And then it just worked.

Thanks to Rovio for making it that easy. And thanks to Android for being a mobile OS with a file system that is visible. Having used Apple iOS devices for quite a few years, I did not realize just how useful it was to have an exposed file system until I started using Android. Back on the old Symbian phones I used to use, there just was not that much to do with a file system, and they were never mountable as file systems to a PC anyway. Now, with high-resolution screens and the ability to watch movies and read PDFs in a way that makes sense,  having a file system makes so much more sense.

There is probably some way to extract save games from the older Angry Birds games from my iPods to my Android devices, I need to look into that. It should just end up being the same types of files in the end, I assume.

I wonder why Rovio does not offer the service to sync games across devices. The only way I can think of that makes sense is running some kind of server that would sync user’s save games across devices. A hassle to maintain, and users would need to establish accounts just for this purpose. Not sure that I would. Considering the average cost per game sold of only a few dollars, the cost of running servers might well be quite high compared to the average income per user. On the other hand, they might even be able to charge a dollar or so for the service. It could also interact in strange ways with in-game purchases. For games that are guaranteed identical on all devices it is easy to sync progress. But what if I have purchased a set of levels on one device, but not on another, how would you deal with that in a robust way?

Offering a simple way to copy progress to a file would spoil the game a bit (users sharing saved games to avoid the effort to get three stars on all levels and similar thresholds). It is also something you cannot really do on an iOS device in a natural way, as there is no exposed file system.

My starting point is the marketing hype surrounding core counts (something we saw in PCs a few years ago). It sure sounds powerful with an “octacore” device, right? Beats an old dualcore any day, right? To some extent this is marketing hype, but it also does show that there is a need for more performance in phones and tablets. I have also been looking for a new TV lately, and was quite flabbergasted to see several brands touting dual-core processors as a key feature of their offerings. Processor specs for selling a dumb thing like a TV? Things are weird. Sure, Tim Cook of Apple blasted this as being stupid marketing “because they cannot provide a great experience” — but the truth of the matter is that when vendors compete within a single software ecosystem, this is what you compete on. The experience of Android or Windows is mostly the same across devices, so differentiation has to come from the hardware, and core counts are easy to understand. Just like horsepower in cars.

But the key question is: can we really use more cores than two or four in any meaningful way?

Looked at objectively, there are four ways to improve performance of the processor portion of an SoC:

• More cores
• Better microarchitecture
• Higher clock frequency
• Allow the device to go hotter and use more power

In ARM land, there is clearly room for all four, while in Intel PCs, it is hard to do more than minor improvements in anything except core count. So we are left with a more interesting competitive space than most. The better microarchitecture arena is a place where Qualcomm and Apple have shown that you can do better than ARMs standard cores.

Some smart and brave souls at ST Ericsson (one of which I met many years ago and have great respect for) put out an interesting whitepaper on the quad-core mania, making the case that their new very-high-clock frequency dual Cortex-A9 is more useful than a lower-clocked quad-core Cortex-A9. Basically, the software just does not typically make use of multiple cores in a good way, while driving the clock frequency higher will immediately accelerate the software that people really care about. When it comes to reducing the latency of operations, higher clocks tend to be hard to beat. Still, some marketing person at ST Ericsson decided that quad is the word of the day, and dubbed their Novathor L8580 2.5 GHz dual-core a “eQuad”, for something equivalent to a quad. Silly. Had the core been a bit more modern, I think this could have been a real winner. For most real applications, this should be much much faster than the last-generation 1.0 to 1.5 GHz Cortex-A9 dual-cores or quad-cores.

Another interesting new release was the Renesas Mobile MP6530. It is a chip targeting the midrange of smartphones, using an ARM bigLITTLE setup with two Cortex-A15 and two Cortex-A7 cores in a single core complex. What I found interesting here was that for the first time, I saw a bigLITTLE setup described as activating both types of cores at once (look at the diagram on the press release page). Until now, I have only seen designs that switch between only using A7 cores and only using A15s. It was just a matter of time before software matured to this point (the OS scheduler needs to understand that it is dealing with different types of cores, and behave accordingly), but still good to see it happening. I expect the same idea to be used on other mixed-core setups, would be strange if it was Renesas-specific.

This setup makes eminent sense to me. Living with multicore PCs for several years now, it is clear that having more than one core is useful for any multitasking environment. There is enough work simmering around even in a lightly-used system that using two cores clearly reduce latencies and provide a smoother experience with less locking-up of the UI. As long as nothing heavy is running, you can use the two power-efficient A7s and get a nice long battery life. What is interesting is what happens when a heavy task with high demands on latency appears. I think a good example is a web browser doing a rendering pass or running a Javascript application, or a game with heavy AI (which is pretty serial in general it seems). In this case, to get a snappy user experience you can kick in a powerful core at high frequency and get the job done quickly. Perfect, just activate one of the A15 cores and keep the rest of the background noise running on the A7s. If something really heavy comes along, you activate the second A15 core too.

So far, so good. But what about the new octacores sporting four Cortex-A15 along four Cortex-A7? To me, this seems strange. Assuming that we are able to use any combination of cores, this setup would only make sense if four A7s could somehow do a better job than a single A15 while using less power. I heard some numbers indicating that the magic difference is about 3 in performance… so in that case, just what are all these A7s there for?  It would make more sense with a hexacore device – 2 A7s for background noise, and then you turn on 1 to 4 A15 cores to process progressively heavier software tasks. I have no doubt that we can imagine and invent software that can make good use of any number of powerful cores – but where is the case for a large number of weak cores in a client machine (it makes perfect sense on a server)?

Considering the existence of a reference platform from ARM with two A15s and three A7s, you can clearly mix the numbers. No need for the same number of A7s and A15s. Maybe this will change as the use of truly heterogeneous multiprocessing spreads within the ARM ecosystem, while right now the safe bet is a quad + quad setup where you switch from one cluster to another. I have no real data and can only speculate.

What I would like to see done is to run long-term CPU profiling on an all-cores-active-all-the-time bigLITLLE platform and see just how much use you get out of each type of core. Unfortunately, I do not have that right now, and maybe nothing such will be on the market for quite some time.

In conclusion, I think there is some merit to bumping up mobile processors to at least quad, and there is definitely potential in mixing fast and slow cores. Making good use of eight cores seems a bit unlikely though (unless you switch between two separate clusters). One wonders if that silicon area could not have been used for something else instead.

I recently listened to the episode on Bliss, by the Radiolab podcast. As always, Radiolab manages to take a theme and connect all kinds of things to it. In this case, bliss as in happiness turned into Bliss, the man, and his invention of Symbolics. Symbolics was an attempt to create a rational language based on symbols that would not allow the manipulation of human opinion or feeling like regular languages do. It was an attempt to create an antidote to the manipulations of dictators, tricksters, and populists (Bliss himself had been briefly interned in a pre-war German concentration camp, so he definitely knew what words could do). He designed a symbolic writing scheme that was intended to only communicate ideas clearly and unambiguously and with no room for demagogery and oratory. In the end, nobody wanted to use the language for its original purpose.

However… it turned out to be very useful for communicating with children with certain types of handicaps. This use was pioneered by a nurse in Canada, and she reached out to Bliss to thank him for his great invention and the joy it brought to the children. She also found that it was a great way to help the children learn regular English or French. The Blissymbolics were being used to bridge into a regular language, not as a permanent communications method on its won.

At this point, we have a classic “pivot” moment for Blissymbolics. Think of this as a startup. Bliss has discovered that the original idea for what to do with the symbolic language does not seem to work. But there is a new adjacent area where there is real utility and a great chance of success. Any startup theorists knows what to do: Pivot – stay grounded where you are, but change direction.

Charles Bliss did not pivot.

Instead he hated the idea, since it went counter to all that he wanted to do. He did not want people to use regular language, he wanted them to use his superior code. Thus, acrimony, lawsuits, and pain ensued. In the end, the conflict pretty much killed his idea as a practical tool. It is a sad story. We can either view Bliss as a stubborn man in love with his original idea who did not understand the need to change direction to be successful. Or we can view him as an idealist who did not sell out despite the lure of success, where success required violating his original idea. When looked at as a business, he clearly did the wrong thing. When looked at as an expression of art or ideology, it is less clear. Still, I think he should have been happy to help people, even if it was not quite in the way he envisioned.

For a startup, I think the story of Bliss illustrates the need to balance the love of the original idea and application with the need to accept that it might be used in unintended ways. Whether one embraces the pivot will depend on what the situation is. If it is just a different way to earn money and make people happy, do it. If it ends up being something morally challenging, maybe the pivot should not be done. In the most extreme, turning plowshares into swords might earn you money, but is it right?

Sorry for the rambling, but I think it was an interesting story.

The demo is now online at the Wind River Youtube channel, and the setup is explained in a blog post at the Wind River company blog which is worth reading before watching the video.

Schlock Mercenary

From 2008-06-23:

I do simulation tools for a living, but I still agree that there is no replacement for reality tests as well. Doing all those simulation runs probably indicates that nothing is totally broken, but you never know what happens when reality confronts the system with a situation you did not think about including in the simulation. I just love the line “create indistinguishably perfect virtual world”…

And indeed, things do turn out to be a bit different in reality.

From 2008-11-21:

Go and read the entire story – it is fun all along.

LEGOs seem to be a favorite analogy for people bemoaning the state of software development today. “If only it would be as simple as putting Legos together” is a common enough statement, along with various proposals to make software that is Lego-like. Sometime, I wonder if people making these statements have actually tried to build anything non-trivial from Lego recently. Here, I will look a bit closer at the Lego-programming analogy. There is indeed quite a lot to it, but it is not all about child-level simplicity. I think there are some good lessons that can be learnt from analogizing Lego and programming.

The naïve attractive idea of Lego is, I think, is that “any piece can be combined with any other piece”. Technically, this is mostly true in my experience, but with a significant number of exceptions that might surprise someone who does not have kids of Lego age. Even if we ignore the modern Technic beams which are really a separate system with its own logic, there are many pieces in Lego that can only be attached to certain other types of pieces due their geometries (such as this).

Still – the basic unit of interface is almost universal, the Lego stud. There is the first lesson to be drawn from Lego – design for connection from the start. The reason all the pieces fit is that they all use the stud, and the reason that they all use the stud is a conscious explicit design from the very start. It is not something that was retrofitted into an existing set of random blocks – Lego pieces were designed starting with the stud and its geometry, and then designed so that each new piece uses the same-size stud and the same geometrical logic. For software, the lesson is clear: if you want Lego-style assembly, you need to start with very strong standard interfaces and build the software for these interfaces. You cannot really hope to componentized and “Legoize” an existing body of code. The connectivity is more important than functionality.

So, any two Lego pieces are easy to fit together. However, that does not mean that it is easy to build something that works in an interesting way or looks like what you had in mind. Indeed, if you look at the way modern Lego kits are designed, it is amazing how many smart solutions are employed to attach pieces at various angles.

The helicopter shown on the left is a typical modern lego design: plenty of pieces that fit at 90 degrees to the vertical, a far cry from the old-style design shown on the right. The sophistication of the design and the great variety of pieces needed is much greater for the helicopter. I think what mostly people want out of their software is more like the helicopter than the tree.

The basic composition of pieces is easy – but advanced construction is a skilled job. Your average seven-year-old will not be able to design the kits they buy, even though it is easy to follow the instructions and build them. Ease of assembly does not necessarily mean ease of actual design. If you have something particular in mind, phrasing it into Lego bricks can take a long time and a lot of trial and error. That trial and error is certainly usually quick and easy, but it never replaces some planning and forethought so that you do not have to tear down an entire building because you forgot to put some crucial part in the middle of a now-finished wall…

There is the second lesson: even for a software system where we can build like Legos, I would not expect to see regular users create superb software for sophisticated work – that would still be a rare skill. Ease of construction removes some of the accidental difficulties of software construction, but it still leaves the essential problems to be solved.

Another Lego lesson is that to build something that truly works (or looks like something real), you need rich palette of pieces. In theory, you could away with just the simplest old square pieces and basically build a 3D model by using voxels like most of the older models you find in Legoland. But that requires both very large models and very fiddly work, and honestly is not something within the reach of most users. Using tens of thousands of pieces is time-consuming, even when they are Legos.

Thus, modern Lego is awash in special-purpose pieces. To get an idea, look at the list of pieces introduced each year at Brickset. 1000s of new pieces every year. They still follow the Lego system, but it is not just square bricks anymore. Look at the piano on the left: the little microphone and the glass are a special pieces (following the size rules of the Lego minifig hands). To build the rest of it, existing parts are reused, but it is built from 11 different types of bricks, only 5 of which are used more than once (see instructions).

Sometimes when you really need something special, you just have to craft it with no heed given to the idea of composability and flexibility. Like the Lego bicycle seen on the right, which are basically three pieces: frame, and two wheels (front light and helmet are optional). Makes for a perfect model of a bike, but will not help you build anything else.

In summary, the third lesson seems to be that in order to compensate for a rigid way to connect things, it seems you need to introduce an incredible host of special-purpose components to do just that particular thing that cannot be done from heavy standard bricks.

An attractive property of Lego that might or might not be desirable in software is that even in a finished model, you have plenty of places where you can attach new things. Unless you carefully cover all studs with little flat plates, anyone can add things to the model in almost any place. Indeed, extensibility has been a common theme for long-term successful software: by allowing users to extend and build on the basic software, the power and staying power of the software is typically greatly extended. Look at tools like Emacs, gdb, good old Microsoft Word, or Eclipse: by having ways for users (and OEMs and integrators and VARs and consultants) to hook in their own scripts and extensions, you get a software ecosystem that keeps users coming back. Trying to keep users out with a shiny polished surface where things just slide off tend to result in shorter-lived software.

Finally, I have to recount a story from many years ago, when I was studying in Germany. My professor in software engineering made a remark that RAD (rapid application development) tools where like building with Lego Duplo: you quickly get something built, but you do not have much control over how it looks. I agree. You can build big things very quickly with Duplos, but they do require a lot of imagination to interpret the creation. Like the rocket on the right. Took no time at all to build, and it is very large compared to regular Lego (see minifig mechanic to get the scale).

So, to conclude, I think the Lego analogy has much going for it. But it is not “anyone can build anything”. The lessons I think that can be drawn from Lego for software designers:

• Design for modularity and connectivity from the very start
• Force all modules to be connectable, even if that contorts the shape of the module a little (global goals override local optimizations)
• Ease of assembly does not make design and architecture go away
• Sophisticated software design will still be a skill
• Expect to see many special-purpose modules created to solve very particular tasks
• Expect modularity to show through in the end product – it will not look quite the same as something monolithic
• Allow extensions throughout the life of a piece of software – embrace user extensibility

The conventional wisdom (see for example http://archive.arstechnica.com/cpu/4q99/risc-cisc/rvc-1.html) since the PentiumPro has been that for a mainstream processor, the decoder logic is such a small part that it does not really matter. Behind that decoder, both a high-end RISC and a high-end CISC do pretty much the same thing, and the sheer weight of design manpower and manufacturing advantages that Intel has had has ensured that their processors have been the fastest or at least very competitive for the past decade and a half. CISC vs RISC also turned out to be a blurry boundary – some PowerPC processors did the same trick as Intel processors, breaking up their instructions into microoperations before sending them down the pipeline. Indeed, Power Architecture is sometimes not even considered RISC, but rather something of hybrid.

It seems that market share is just as important for processor performance as technical elegance and ingenuity – a bit sad, but pretty true. At least, this holds true in the “old mainstream” desktop and server markets.

Still ISA matters. And even within a family, ISA changes can have a huge impact on performance.

Obviously, we have specialized ISAs like DSPs and network processors and GPUs. In these cases, the ISAs let us express computations in ways that they cannot be on a general-purpose processor, often trading ease of writing general code against performance on a certain class of computation. Efficiency is maybe an order of magnitude better compared to a GPP for these processors, doing the same work. A high-end GPP can often match the absolute performance of a specialized processor, by the brute force of high clock speeds, large caches, and aggressive out-of-order pipelines. But in doing so, it uses 10-100 times more energy.

Inside a mainstream ISA, the trend of the last decade has been the successive addition of small and large sets of specialized instructions for doing various important computations. Floating point units have been essentially replaced by vector processing instruction sets such as MMX, SSE, Altivec, VFP, and Neon. Crypto instructions have entered the mainstream on all major architectures. Power Architecture has included binary-coded-decimal instructions, and IBM mainframes have instructions that do string copies in the L3 cache. Configurable architectures like Tensilica show the value in adding a few well-chosen application-specific instructions. Adding features to existing ISAs has been proven to have very good bang for the buck.

Another, more interesting, aspect is the modernization of old ISAs. The actual cause for this blog post was that the Microprocessor Report estimated that an ARM Cortex-A57 (and A53) core would gain 10% performance when running in ARM AArch64 rather than AArch32 mode. That is pretty significant – the same processor core, the same program, just compiled using a different set of instructions. It is especially interesting for ARM, since the ARM v8 AArch64 64-bit instruction set is pretty much totally different from the old ARM 32-bit instruction set. ARM took the chance to modernize and remove old “good ideas at the time” aspects that are hard to implement efficiently today, such as predication on most instructions, the wonderfully complex shifting operands, and pc-relative constant pools swimming around in the instruction stream (see http://www.realworldtech.com/arm64/  for more details). AArch64 also finally makes ARM an architecture with 32 registers, rather than the old 16, which makes compiler optimizations much easier.

This means that AArch64 improves performance in two dimensions: it allows better more streamlined silicon implementations compared to the old ARM, and it makes the life of a compiler a bit easier with fewer spills and more data kept in registers. I think it is mostly this second effect that is seen for the Cortex-A57 and A53.

I think this is good proof that ISA design does matter. Another similar data point is the effect of going to x86-64 from x86-32. Typically, you can see a five to ten percent improvement in the speed of compute-intense code from the better register allocation allowed by having eight extra registers and the overall somewhat cleaner instruction set. This is a different approach compared to ARM, since x86 maintains the 32-bit instruction set and simply extends it, where ARM basically changed the ISA completely.

Thus, my conclusion is that the design of an ISA can have significant effect on the performance of a processor, even when all other factors are held constant. What this “best” design is seems to change over time – in the 1970s, CISC designs like the 8008 and 8086 were dominant as they let you do more with each slow fetch from memory. In the 1980s, with faster memory, RISC used a bit more program space to create an instruction set that could map directly to a very efficient pipelined processor. In the 1990s, with more complex processors, RISC or CISC turned out to be equivalent. In the 2010s, it is clear that efficiencies can be achieved by designing a clean ISA that is easy to implement in many different ways in out-of-order speculative high-frequency designs. The key today is hardware-independence, where it used to be the close co-design of instructions and hardware.

On the ultra-low-end, I think the classic RISC idea of “simple to implement” instructions still have clear value. Something like the ARM Cortex-M0 can be implemented in 12000 transistors, on the same level as a 1970s 8-bit design – while being fully 32-bit and clocking in at many times the performance of these old machines. It is even half the size of an Intel 8086. It is hard to see how an x86 processor could ever be cut down to this kind of size – unless of course Intel does the same as ARM did, and cut out most of the instruction set into an ultrareduced version. Note that a classic simple RISC like the MIPS R2000 was still about twice the size of a 68000 processor, so it is not necessarily the case that RISC means small. You need to design for smallness too.

Running Simics on itself is pretty cool, and shows the power of the simulator and its applicability even to really complex software.

In particular, Bad Piggies is perfectly determinstic.

The details are all in the Wind River blog post, but I felt I wanted to share some cute screenshots from the game here.

First, we see how levels most usually end, with a broken vehicle and a bouncing pig. The main departure from reality in the physics engine in Bad Piggies is that fact that the pig is made from some indestructible material and can take any amount of pounding without getting more than a few cartoon stars around the head. Perfectly OK, as that makes for a compelling game.

Second, the animations are also just wonderful, the pig (and the queen that sometimes tags along) look genuinely scared as they jump fly and bounce around.

The game is highly recommended, and I have seen people just watching a run laugh out loud as things do not work out too well… it is like a classic Roadrunner cartoon come live, or silent movie physical humor.

(Screenshots taken from the HD version on a Google Nexus 7, and then edited down to size).

However, I do not quite agree with Bill about the absolute uselessness of the intermediate types of models, like SystemC TLM-2.0 AT.  Basically, what is traditionally called “cycle accurate modeling” (while not derived from RTL).

I love the illustration on their blog post:

But that land of dragons sometimes have to be visited, for those daring knights of design that need to peek into the future.

If the goal is to build a model that describes a piece of hardware at cycle accuracy, manually coded AT models are the wrong way to go (see this blog post about the futility of  cycle accurate model building). You will never be quite right, and the resulting model won’t be that much faster than a model derived from RTL.

However, if we look at the case of designing new hardware, the dragon area is the right place to be. Before you have RTL, you cannot be cycle-accurate to the final implementation, since that does not yet exist. Rather, you want a model that lets you try various latencies and pipelines and parallelization strategies and estimate the eventual performance. This has been the standard way to develop new processors since the late 1950s, and still is. James Aldis of TI had a nice description of the process in an article from last year.

The key take-away for me is that anyone who does not either have access to the RTL of a processor or a direct connection to the design team pretty much have to give up on hoping for “cycle accurate” models. In any case, once we start moving away from the processor, out into peripherals and then off-chip, we soon run out of precise models no matter what. A virtual platform can be perfectly useful for most use cases, even when it is only a fast platform with no attempt at cycle accuracy.

Printf debug

For some people, log printouts from a program is dirty and primitive. Very often when discussing debug methodologies, everyone starts with “printf debug” and implicitly considers everything else superior. In many cases this is true. When solving comparatively simple linear problems, proper use of a typical debugger ought to solve a problem in much less time and with much less intrusion into the target system. However, once you get to system composed of many quite independent parts, with unpredictable sequencing of actions and concurrent activity, logging is actually a pretty practical and powerful way to diagnose a system. In such a case, the key is to be systematic about how logging is done and to make sure it is much more than just a sprinkling of print statements in random places in the program. What good logging brings you is the executable embodiment of the knowledge about the most interesting and important events in the flow of a program, put there by the programmer writing the code. It is high-level semantic information, much easier to digest than a line by line trace or stepping of the code of the program.I find it hard to see a better way to put that in than by annotating the source code with log statements – but they had better be statements that are clearly about logging, and that direct the logs to some kind of processing system that allows selection and sorting of the output.

Of course, logging can also be done badly, in which case nothing important is ever logged, or millions of uninteresting things are logged, hiding the few relevant events. I have seen this many times. But when done right, logs help you zero in on a problem at a very high pace, without having to really sit down and inspect the code. Often, good logs let you pinpoint a problem and report it to the programmer without needing more than a trace of the log messages that show that something bad is going on in the code modules executing.

Single sequence

One problem with logging that almost always comes is that you have many different log sources and many different log systems (the ACM Queue article has several examples of this), and thus getting a consistent correctly ordered view of events can be very difficult or impossible.  Obviously, this is something that can be solved for a system over which you have some amount of control. It would seem to me that logging all systems through a single conduit makes the most sense – but also requires some changes to existing mechanism.  In many cases, it would seem possible to just change existing log collectors to push the final results through to some unified infrastructure.

The typical real-world case is shown below, contrasted with an ideal case of a single log conduit that creates a single consistent view of what is going on.

Back at the S4D in 2010, Pat Brouillette presented a paper on Intel’s internal SVEN system that still presents the best logging design I have seen (but I have not seen every system on the planet so there are probably more that are as sophisticated).  What I found particularly attractive and useful in their approach was that they logged in user-level software, drivers, operating systems, and the hardware itself. Thus, you could follow a chain of events up and down through the stack.  As illustrated above, most existing log systems only apply at one of these levels, making it hard to really see what is happening. This system pretty much achieves the unified log illustrated above.

The result of a unified log is that you can follow the execution through the software stack, even across changes from user-level to kernel-level code and the actions of drivers on the hardware. This is incredibly useful for systems where software and hardware need to interact closely. This is something really hard to do using other debug methods, as most debuggers lose the trail when changing execution modes. Even when they can indeed step along, the volume of code that needs to be followed is so big that it is not very likely that debug information and source code is available for all of it.

Hardware support

A key to making the unified log approach work is to have hardware support for logging. This gets around the common complaint about logging that the overhead affects the system and swamps IO. In particular for resource-constrained system, logging can quickly start to impact performance and in particular IO performance (it seems to be less of an issue in the server space). But there are ways around that, if we allow the hardware to help us. Thus, we want to get something like this into our embedded system:

The simplest approach is to have some special memory-mapped locations where log information is dumped, and then passed on via a debug interface to the outside world.  This creates a side channel which avoids mixing in log messages into the useful data streams being sent over serial lines, networks, or other application communications channels. ARM has support for this in their trace facilities for hardware debug today, and I have seen it used to both implement efficient logging as well a general printf (using the output channel instead of serial ports to get data back to the debugger).  We also did something similar with Simics and the LDRA code-coverage tools.

In a panel at the S4D in 2012 I proposed using a special instruction in a processor’s ISA to do logging (of a single integer or a few integer values stored in registers).  I think this could work very well, but it would reuire some kind of discipline to allocate log ID numbers to various parts of the stack. This kind of approach would work best when the goal is to know where you have been executing with a small amount of context data. If there is a need to dump large amounts of data (like typical IT logs), it probably does not work that well. The advantage is that it can be used within any code, as there is no need for a call into a system library (print statements are typically verboten inside interrupt code, for example). Today, we can do this kind of logging inside virtual platforms by using “magic instructions” (no-ops that the simulator recognizes as special and which can be handed off to instrumentation modules or scripts), but it would be a very handy thing to have in hardware as well.

Analyzing the results

Given some kind of log mechanism, the final step is to analyze the logs to figure out what is going on and whether something is going on that should not. This is part that I am currently looking at ways to address – it is much easier getting things out of a system than to really understand them and draw the right conclusions. There is a huge body of research that should apply here, but just not sure about how the pieces should fit together for my target of “general embedded systems”.

First and most importantly, I must revise my previously published history of reverse execution. It turns out I was wrong about Lauterbach. Rather than having something that was record-replay debug as I thought, it turns out that their CTS, Context Tracking System, actually is a working reverse debugger. And it has been that way since 1999, beating Green Hills Time Machine to market by quite a few years. Thus, the award for “first reverse debugger based on hardware trace recording” has to be reassigned to Lauterbach. I was presented with a Trace32 newsletter from 1999 where it was very clear that the CTS allows a user to move backwards in time in the trace, tying the point in time to source code and registers. The CTS can also be given commands to step backwards in time until some condition is fulfilled, which is in essence reverse breakpointing.

I have updated my post from early 2012 to reflect this new understanding. That is the charm of blogging – you are at liberty to go back rewrite what you wrote when new facts appear. Still, I keep the old text around but with strike-through to show what I originally wrote, even if it was indeed wrong. Better to be clear on what has been revised than to silently change things.

Second, as noted before, weak memory models complicate replay for reverse execution. In a typical host-based approach to reconstruction-based reverse debug, the replay of a parallel execution is done in serial (with interleaving at all points that communication was detected). This allows race-condition debug for the case that races are caused by accessing shared memory without proper locking. However, if races arise not from the program design but from the hardware behavior, these will not be reproduced. In a weak memory model like ARM employs, it is theoretically possible for certain memory operations to take noticeable time to propagate from one processor to another. Such conditions will not be replayable when using a single processor to reproduce a parallel scenario. If nothing else will the replay on a single processor ensure perfect memory consistency between threads. There is no way that hardware can allow such scenarios to be replayed without very special support being designed into the hardware.The only method I can imagine that can handle this correctly is to use cycle-accurate simulators that are still deterministic and applicable to reconstruction, running with a perfect model of the memory system – which means it will be painfully slow, but when debugging complex hardware-software memory consistency bugs, that might be the only tool that works.

This year, S4D took place in Wien, colocated with FDL at TU Wien. The main building of the TU is really nice (but we were in the more modern microelectronics building). Here is a night shot of the newly renovated main building:

S4D is a small workshop, and the papers presented only provide part of the picture as to what the hot topics under discussion were. This year, the main themes that I picked up were:

• The bandwidth limitation for hardware-based debug.
• Log-based debugging, automatically looking for errors in logs.

The bandwidth limitation is very important. If you want to use hardware debug circuitry inside an SoC, you also need to be able to talk to it. The complexity of the chip and the capability of on-chip debug grows with Moore’s law – but off-chip bandwidth does not. The number of pins is also growing slowly, if at all, and thus there is strong pressure to use the pins and bandwidth for “real work” and not for debug. The result of this are a few trends in technology for debug:

• Doing more debug processing on the chip, without having to turn around in an off-chip interface box or debugger host (several research papers describes approaches to have checkers and inspection code run on the chip, in a coprocessor or even on one of the regular cores).
• Aggressive compression of data sent off-chip (the ARMv8 debug architecture presented by Michael Williams of ARM only traced mispredicted branches off-chip — expecting the debugger to reconstruct the flow from a minimal amount of information).
• The use of software debug agents and the software interface of on-chip debug hardware becomes more important. In particular for devices such as smartphones, where there is no dedicated hardware debug port and the debug might be done over USB, Bluettoth, or Wifi. Thus, the exposure of hardware breakpoints and similar functions to software becomes more important to let users actually take advantage of the debug power of a modern SoC. Hopefully, all other silicon vendors will follow the lead of ARM and expose really powerful features in the hardware to software agents so we can get away from silly things like rewriting code to plant breakpoints (and allowing full data write and read breakpoints in software agents).
• Simulator-based debug offers a way to get around the issue by having virtually infinite bandwidth (potentially at the cost of slowing down the target, obviously).

Log-based debug is a favorite topic of mine, and it has been on the agenda for S4D since it started (see reports from 2010 and 2009).

• This year, the most interesting idea was a hardware unit (generated into an FPGA) that watched as a target was executing, looking for traces that satisfied properties expressed using past-time Linear Temporal Logic (ptLTL). ptLTL seems quite well-suited for the task of watching traces of events fly by – it allows looking backwards just a bit, which makes it much more powerful than just looking at the current state, but still it can be implemented quite efficiently.
• Users are clearly using logs to diagnose issues in running systems, and a key problem there is finding issues in huge logs. This is nothing new.
• There was a discussion over how to handle explicit log and instrumentation calls in software. Should they remain in the target software as it ships, or be removed? How does that affect certification and validation?
• If we introduce hardware-supported log instructions in the ISA, couldn’t they also be used as a timing-fault-injection mechanism? Basically, with a settable pipeline stall? Such single-cycle overhead instructions should be possible to keep in the shipping software, as they do not lower performance too much. And if single-cycle disturbances kill your real-time system, it is too close to the edge anyway.

One idea that I threw out but that met very little agreement from the S4D and FDL participants was the notion that we should build systems that accept and tolerate and recover from errors, rather than hoping to make them bug-free and with timing under perfect control. I instinctively find the idea a bit repugnant, being schooled in the precise tradition of computer science where we expect programmers to fix bugs, not just work around them. But in practice, I realize that this might be the right thing to do as our systems get so complex we cannot hope to precisely understand them or diagnose issues in the lab. A typical example of this apprach was published in the ACM Queue last year – basically, using a malloc system that minimizes the effects of buffer overruns, double-free, and similar common causes of crashes. A variant of this is actually shipping in Windows 7 already. People building safety-critical systems do not want to have to do this, but at some point we probably need to go statistical rather than showing that our software is correct. There is some interesting work in making software become more continuous than discrete in behavior, paving the way for statistical analysis of errors. But that was not a topic of S4D, at least not this year.

I presented a paper on the history and techniques of reverse debugging, and received some good feedback from the audience. Someone pointed out that with a weak memory model, record-replay on hardware is not guaranteed to reproduce all bugs since concurrency bugs related to the memory model are not inside the controlled area. On x86, where most work has been done, the use of a TSO-like memory model makes this point fairly unimportant. But on ARM and Power architecture, it is indeed relevant. Another member of the audience found it funny that he believed that he had indeed invented record-based debug back in 2001 – but my presentation of history showed that there was ample work before that. Just goes to show how hard it is to know the history of computer science, many ideas are never widely circulated. Finally, there is a lead indicating that there was some kind of reverse-breakpoint trace-based debugger in the market around 1999. I hope to learn more and do a full blog post on this once more data emerges.

Once again a good workshop, and my only wish for next year is that many more people show up so we can get a broader discussion!

Many-Core Missing

Five years have passed since the first SiCS Multicore Day in 2007 (making this the sixth event), and in the introduction by Erik Hagersten he looked back at some of the predictions made back then. One missed prediction stood out clearly. The idea that by now, 128 cores would be mainstream in personal computers. My theory of why this has not happened is simple. GPGPU. GPUs have eaten up the easy parallelism. Instead of using massively multicore regular processors, heavy-duty personal computing has been shifted onto GPUs. With the disappearance of these workloads, there has been little pressure on main processors to become more parallel as there would not be much to gain from that, performance-wise. GPUs have turned out to be perfect for massively dataparallel work in media and other areas (including tasks like cracking password hashes and mining for bitcoins), achieving performance orders of magnitude higher than what could be hoped for with a multicore main processor – while costing less and using comparatively little power.

The prevalence of GPGPU on the desktop is not mirrored in the top supercomputers, however. According to Erik Hagersten, there is no real GPGPU machine in the top-500 supercomputer list at the moment. Maybe 5% of the performance and 3% of the chips are GPUs. I suspect part of this might have to do with the kinds of tasks being done. HPC at the high-end probably requires more flexibility and programmer control than GPUs can offer.

Programmability might be more important in architectural design for HPC, as HPC users tend to be programmers. Most regular computer users, on the other hand, just use software written by someone else. Thus, it is enough that a few people go through the hard work of coding in CUDA or OpenCL or similar toolkits, and the results of their work can be spread across a very large user base. GPGPUs are perfect to provide “performance for the rest of us”, for common tasks coded by a few expert programmers.

Homogeneous, Heterogeneous

The debate over GPGPU is part of a bigger debate about homogeneous vs heterogeneous compute systems (see previous blog posts like this, this, this, and this). The debate is still going on, with the same intensity as it always have. To me, that would seem to indicate that hardware accelerators are here to stay, even if some people do not really like them.

This year, the primary example of the drive to homogeneity was Intel’s recently announced “more than 50 x86 cores on a chip” Knight’s Corner (Xeon Phi). The argument for the chip is very much programmability: “just a large x86 box that runs Linux”. But I guess you do need special compilers or libraries to make use of the big somewhat Cray-like vector unit (512-bit SIMD unit) each core has been equipped with. At least special optimization will be needed to make the best use of the chip, just like you always need to do when performance matters.

The UltraSparc T5 presented by Rich Hetherington from Oracle fell somewhere inbetween. It has 16 identical cores, but can tweak how it uses the SMT threading to make a core run a certain serial task faster than it otherwise would. This is a step towards the kind of heterogeneous performance in a single ISA that ARM is going after with their Cortex-A15/A7 bigLITTLE approach – but without the same span in performance, and also with less impact on the overall flexibility of the chip. The T5 also removed the special crypto accelerator hardware that used to be there, instead adding a few crypto instructions to the ISA.

The reason they moved crypto from an accelerator into the ISA was that it turned out to be costly to use a separate hardware unit for small pieces of data. There is OS overhead in invoking an accelerator, and that requires a decent size buffer of data to work on. With instructions in the ISA, you can work on a single word and still get performance gains. User-level software also have a far easier time accessing it, as the instructions are just part of the regular instruction stream. Interestingly, ARM (as presented by Stephen Hill) had done the exact same thing for crypto, for the same reason. This is an important point for hardware accelerators in general: the driver overhead has to be managed, sometimes by mapping hardware straight to individual programs (I made a simple experiment a few years ago that showed this nicely). On the other hand, everything put into the ISA risks making the entire processor a bit slower and power hungrier, making general ISA extensions something done with great care. Hardware accelerators can be removed from a certain SoC if they turn out not to be needed, not so easy with ISA components.

Stephen Hill from ARM clearly believed in heterogeneity, with four types of processing on a typical chip:

• Big core (ARM Cortex-A15 today)
• Little core (ARM Cortex-A7) – to create the kind of bigLITTLE setup that allows for a bigger span of power-performance settings.
• GPU (from ARM, that means Mali T604 today) – they clearly see that GPGPU is moving into the mobile space very quickly, doing the same kind of work that it has done on the desktop, and with the same effect of reducing the need for general processor cores.
• Special-purpose accelerators – except when merged into the ISA, as noted above.

In researching some of the material from James Larus’ talk, I also came across an interesting talk from Surge 2011 where Artur Bergman from fastly.com tell how they have optimized their content delivery network by only relying on plain processors and not using any network processing offload, router ASICs, etc. Too hard to use, to easy to make errors and have the software crash, and “Xeons are simply faster”. Note that the word “energy” is never mentioned in his talk.

Software Needs to be 100x Better

The software perspective was presented by James Larus from Microsoft research. His talk made many interesting points, but I think the main points were that:

We are not even trying to make efficient systems today, throwing away billions of clock cycles on plain pure overhead. Example: IBM had investigated the conversion of a SOAP (text) date to a Java date object in IBM Trade benchmark.268 function calls and 70 object allocations.  There is great modularity and nothing obviously wrong in the code.  About 20% of memory is used to hold actual data, the rest is hash table, object management overhead, etc.  In general, objects are small and waste is large. Great for programmers, bad for machines. We could and should find ways to do better in programming than this, need to find a way to make performance an abstraction we can work with.

Languages should be as efficient as they can. Today, common runtimes like PHP, Python, and Ruby are very far from optimal. They work “well enough”, but it should be pretty easy to make them 10 to 100 times more efficient with known compiler techniques. This should be done in addition to parallelism and distribution, it is almost criminal to leave that much performance on the table when it is so easy to get. Positive example: the 100x performance improvement for Javascript in recent years shows what can be done once it becomes important enough.

Note that Larus is not advocating going back to assembly language – there is far too much value in programmer productivity – but just that we remove unnecessary waste from our systems while advancing the state of the art in programming languages.

Distributed systems are the new norm. Why don’t we teach it? All programmers should need to understand how build systems from many separate parts. In particular, the impact of IO and network traffic on software performance. Distribution is not free either.

For an example of how bad things can be, he brought up a nice introspective talk from Surge 2011, about the Etsy website:

• The original talk on Youtube
• ArsTechnica coverage

So, that’s my summary of an interesting day.