I read some news (ExtremeTech, Techcrunch) about how “smart” wifi-connected locks sold by Lockstate got bricked by an automatic over-the-network update. This sounds bad – but it is bad for a good reason. I think the company should be lauded for actually having the ability – and laughed out for royally botching it.
Today, when developing embedded control systems, it is standard practice to test control algorithms against some kind of “world model”, “plant model” or “environment simulator”.
Using a simulated control system or a virtual platform running the actual control system code, connected to the world model lets you test the control system in a completely virtual and simulated environment (see for example my Trinity of Simulation blog post from a few years ago). This practice of simulating the environment for a control computer is long-standing in the aerospace field in particular, and I have found that it goes back at least to the Apollo program.
Doing continuous integration and continuous delivery for embedded systems is not necessarily all that easy. You need to get tools in place to support automatic testing, and free yourself from unneeded hardware dependencies. Based on an inspiring talk by Mike Long from Norway, I have a piece on how simulation helps with embedded CI and CD on my Software Evangelist blog on the Intel Developer Zone.
I have a two-part series (one, two) on testing posted on my Software Evangelist blog on the Intel Developer Zone. This is a long piece where I get back to the interesting question of how you test things and the fact that testing is not just the same as development. I call the posts Mindset and Toolset
It is really sad that the European Space Agency (ESA) lost their Schiaparelli lander last year, as we will miss out on a lot of Mars science. From a software engineering and testing perspective, the story of why the landing failed rather instructive, though. It gets down to how software can be written and tested to deal with unexpected inputs in unexpected circumstances. I wrote a piece about this on my blog at the Intel Developer Zone.
The SiCS Multicore Day took place last week, for the tenth year in a row! It is still a very good event to learn about multicore and computer architecture, and meet with a broad selection of industry and academic people interested in multicore in various ways. While multicore is not bright shiny new thing it once was, it is still an exciting area of research – even if much of the innovation is moving away from the traditional field of making a bunch of processor cores work together, towards system-level optimizations. For the past few years, SiCS has had to good taste to publish all the lectures online, so you can go to their Youtube playlist and see all the talks for free, right now!
UndoDB is an old player in the reverse debugging market, and have kept at it for ten years. Last year, they released the Live Recorder record-replay function. Most recently, they have showed an integration between the recorder function and Jenkins, where the idea is that you record failing runs in your CI system and replay them on the developer’s machine. Demo video is found on Youtube, see https://www.youtube.com/watch?v=ap8552P5vss.
This really happened last week, but I was in the US for the DAC then. I did another blog on Intel Software blog, about a white paper that Wind River put out about how they use Simics internally. The white paper is a really good set of examples of how Simics can be used for software development, test, and debug – regardless of how old or new the hardware is. It also touches my favorite topic of IoT simulation and scaling up – Wind River is actually using Simics for 1000+ node tests of IoT software! Read on at https://blogs.intel.com/evangelists/2016/06/06/wind-river-uses-simics-test-massive-iot-networks/
A long time ago, when I was a PhD student at Uppsala University, I supervised a few Master’s students at the company CC-Systems, in some topics related to the simulation of real-time distributed computer systems for the purpose of software testing. One of the students, Magnus Nilsson, worked on a concept called “Time-Accurate Simulation”, where we annotated the source code of a program with the time it would take to execute (roughly) on the its eventual hardware platform. It was a workable idea at the time that we used for the simulation of distributed CAN systems. So, I was surprised and intrigued when I saw the same idea pop up in a paper written last year – only taken to the next level (or two) and used for detailed hardware design!
Continue reading “Time-Accurate Simulation Revisited – 15 years later”
Thanks to the good folks at Vector Software, I was pointed to a conference recording on Youtube, from the Google Test Automation Conference (GTAC) 2015 (Youtube video). The recording covers quite a few talks, but at around 4 hours 38 minutes, Brian Gogan describes the testing used for the Chromecast product. This offers a very cool insight into how networked consumer systems are being tested at Google. Brian labels the Chromecast as an “Internet of Things” device*, and pitches his talk as being about IoT testing. While I might disagree about his definition of IoT, he is definitely right that the techniques presented are applicable to IoT systems, or at least individual devices.
In a blog post at Wind River, I describe how the Wind River Helix Lab Cloud system can be used to communicate hardware design to software developers. The idea is that you upload a virtual platform to the cloud-based system, and then share it to the software developers. In this way, there is no need to install or build a virtual platform locally, and the sender has perfect control over access and updates. It is a realization of the hardware communication principles I presented in an earlier blog post on use cases for Lab Cloud.
But the past part is that the targets I talk about in the blog post and use in the video are available for anyone! Just register on Lab Cloud, and you can try your own threaded software and check how it scales on a simulated 8-core ARM!
I have a long-standing interested in debugging in general and reverse debugging in particular and the related idea of record-replay debug (see a series of blog posts I did a few years ago on the topic: history 1, history 2, history 3, S4D report, updates, Simics reverse execution, and then Lab Cloud record/replay). Recently, I found out that Undo Software, one of the pioneers in the field, had released a product called “Live Recorder“. So I went to check it out by reading their materials and comparing it to what we have seen before.
I just added a new blog post on the Wind River blog, about how you do fault injection with Simics. This blog post covers the new fault injection framework we added in Simics 5, and the interesting things you can do when you add record and replay capabilities to spontaneous interactive work with Simics. There is also a Youtube demo video of the system in action.
I had the great honor to be on a panel discussing IoT Security at the DAC back in June. The panel was part of the Embedded Techcon event that took place essentially as a little embedded corner inside the DAC – it was held in a couple of conference rooms next to the regular DAC sessions, and attendees were also mostly attending the DAC in general. Not a bad idea for meshing embedded and hardware design. The panel was a great one, and David Kleidermacher from Blackberry gave me a great take-away: unless security is allowed to gate releases of products, it is hard to think you take security seriously.
The Security Now Podcast number 497 dealt with the topic of Vehicle Hacking. It was fairly interesting, if a bit too light on the really interesting thing which is what actually went on in the vechicle hack that was apparently demonstrated on US national television at some point earlier this year (I guess this CBS News transcript fits the description). It was still good to hear the guys from the Galois consulting firm (Lee Pike and Pat Hickey) talking about what they did. Sobering to realize just how little even a smart guy like Steve Gibson really knows about embedded systems and the reality of their programming. Embedded software really is pretty invisible in both a good way and a bad way.
I have been thinking about the role and prestige of testing for the past several years. Many things I have read and things companies have done indicate that “testing” is something that is considered a bit passe and old-school. Testers are dead weight that get into the way of releases, and they are unproductive barnacles that slow development down. Testers can all be replaced by automatic testing put in place by brilliant developers. The creative developer types are the guys with the status anyway. I might be exaggerating, but there is an issue here. I think we need to be acknowledge that testers are a critical part of the software quality puzzle, and that testing is not just something developers can do with one hand tied behind their back.
There is a new post at my Wind River blog, about the Trinity of Simulation – the computer, the system, and the world. It discusses how you build a really complete system model using not just a virtual platform like Simics, but you also integrate it with a model of the system the computer sits in, as well as the world around it. Like this:
Read more about it in the blog post, and all the older blog posts it links to!
Last year, I concluded a programming project at work that clearly demonstrated that real programming tasks tend to involve multiple languages. I once made a remark to a journalist that there is a zoo of languages inside all real products, and my little project provided a very clear example of this. The project, as discussed previously, was to build an automated integration between a simple Simics target system and the Simulink processor-in-the-loop code testing system. In the course of this project, I used six or seven languages (depending on how you count), three C compilers, and three tools. Eight different compilers were involved in total.
There is a new post at my Wind River blog, an interview with Andreas Buchwieser from the Wind River office in München. It discusses how Simics can be applied to the field of safety-critical systems, including helping test the software to get it certified. Really interesting, and in particular it is worth noting that qualifying tools in the IEC 61508 and ISO 26262 context is much easier than in DO-178B/C. The industrial family of safety standards have been created to allow for tools to help validate an application without forcing incredibly high demands on the development of those tools.
There is a new post at my Wind River blog, about how Simics is used to simulate large wireless networks for IoT (Internet-of-Things) applications.
It is funny for me to be back at the IoT game. A decade ago (time flies, doesn’t it?), at Virtutech, I and Johan Runeson took part in an EU research project on exactly this topic. Unfortunately, we had to back out of that project due to economic circumstances and failing management commitment, but we still learnt a few things that were relevant now that we are back in the IoT game. In particular, how to simulate wireless networks in a reasonable way in a transaction-level simulator. Thus, payback for the investment took 10 years to arrive, but it did arrive. To me, that underscores the need to be a bit speculative, take some risk, and try to explore the future.
I am going to be speaking at the 2015 Embedded World Conference in Nürnberg, Germany. My talk is about Continuous Integration for embedded systems, and in particular how to enable it using simulation technology such as Simics.
My talk is at 16.00 to 16.30, in session 03/II, Software Quality I – Design & Verification Methods.
There is a new post at my Wind River blog, about how you can use Simics to enable the automatic testing of pretty much any computer system (as long as we can put it inside a simulator). This is a natural follow-up to the earlier post about continuous integration with Simics and Simics-Simulink integrations — automated test runs is a mandatory and necessary part of all modern software development.
The September 2013 issue of the Intel Technology Journal (which actually arrived in December) is all about Simics. Daniel Aarno of Intel and I served as the content architects for the issue, which meant that we managed to contributed articles from various sources, and wrote an introductory article about Simics and its usage in general. It has taken a while to get this journal issue out, and now that it is done it feels just great! I am very happy about the quality of all the ten contributed articles, and reading the final versions of them actually taught me some new things you could do with Simics! I already wrote about the issue in a Wind River blog post, so in this my personal blog I want to be a little bit more, well, personal.
As an old embedded systems and real-time guy, I have always worked with computer systems that are in some way tied to their environment. Simics has often been used to model such computer systems, inside of customer organizations. Which makes it a bit hard to show… however, recently I have cooked up a demo showing Simics simulating a computer system alongside a physical system.
I just put out a post on the Wind River blog, pointing to both a video of my own “water heater” demo and some other Youtube videos showing Simics integrated with simulations of the real world. A screenshot of my setup in action is shown on the side of this post.
I just read the EETimes coverage of the recently concluded court case in the US, where Toyota settled for 3 million USD in damages due to experts finding that the software in a 2005 Camry L4 could indeed cause “unintended acceleration”. In the particular case that was concluded, the accident resulting from the issue caused one driver to be injured and one driver to get killed. This feels like it could be the beginning of something really good, or just as well this could go really wrong.
Debugging – the 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems by David Agans was published in 2002, based on several decades of practical experience in debugging embedded systems. Compared to the other debugging book I read this Summer, Debugging is much more a book for the active professional currently working on embedded products. It is more of a guidebook for the practitioner than a textbook for students that need to learn the basics.
This blog post is a review of the book “If I Only Changed the Software, why is the Phone on Fire“, (see more information on Amazon, for example), by Lisa Simone. The book was released in 2007, on the Elsevier Newnes imprint. It is a book about debugging embedded systems, written in a murder-mystery style with a back story about the dynamics of an embedded development team. It sounds strange, but it works well.
It is quite interesting to see how Qualcomm has emerged as a major player in the “processor market” and is trying to build themselves into a serious consumer brand. I used to think of them as a company doing modems and other chips that made phones talk wirelessly, known to insiders in the business but not anything a user cared about. Today, however, they are working hard on building themselves into a brand to rival Intel and AMD. At the center of this is their own line of ARM-based application processors, the Snapdragon. I can see some thinking quite similar to the old “Intel Inside” classic, and I would not be surprised to see the box or even body of a phone carrying a Snapdragon logo at some point in the future. A part of this branding exercise is the Snapdragon Batteryguru, an application I recently stumbled on in the Google Play store.
Adding electronics to systems that used to be mechanical has been the great wave of innovation for a quite a while now. Modern transportation just would not work without all the electronics and computers inside (someone once quipped that a modern fighter is just a plastic airplane full of software), and so much convenience has been provided by automation and smarts driven by electronics. However, this also introduces brand new ways that things can break, and sometimes I wonder if we really are not setting ourselves up for major problems when the electrons stop flowing.
Logging as as debug method is not new, and I have been writing about it to and from over the past few years myself. At the S4D conference, tracing and logging keeps coming up as a topic (see my reports from 2009, 2010 and 2012 ). I recently found an interesting piece on logging from the IT world in the ACM Queue (“Advances and Challenges in Log Analysis“, Adam Oliner, ACM Queue December 2011). Here, I want to address some of my current thoughts on the topic.