I have a two-part series (one, two) on testing posted on my Software Evangelist blog on the Intel Developer Zone. This is a long piece where I get back to the interesting question of how you test things and the fact that testing is not just the same as development. I call the posts Mindset and Toolset
It is really sad that the European Space Agency (ESA) lost their Schiaparelli lander last year, as we will miss out on a lot of Mars science. From a software engineering and testing perspective, the story of why the landing failed rather instructive, though. It gets down to how software can be written and tested to deal with unexpected inputs in unexpected circumstances. I wrote a piece about this on my blog at the Intel Developer Zone.
This really happened last week, but I was in the US for the DAC then. I did another blog on Intel Software blog, about a white paper that Wind River put out about how they use Simics internally. The white paper is a really good set of examples of how Simics can be used for software development, test, and debug – regardless of how old or new the hardware is. It also touches my favorite topic of IoT simulation and scaling up – Wind River is actually using Simics for 1000+ node tests of IoT software! Read on at https://blogs.intel.com/evangelists/2016/06/06/wind-river-uses-simics-test-massive-iot-networks/
Thanks to the good folks at Vector Software, I was pointed to a conference recording on Youtube, from the Google Test Automation Conference (GTAC) 2015 (Youtube video). The recording covers quite a few talks, but at around 4 hours 38 minutes, Brian Gogan describes the testing used for the Chromecast product. This offers a very cool insight into how networked consumer systems are being tested at Google. Brian labels the Chromecast as an “Internet of Things” device*, and pitches his talk as being about IoT testing. While I might disagree about his definition of IoT, he is definitely right that the techniques presented are applicable to IoT systems, or at least individual devices.
I just added a new blog post on the Wind River blog, about how you do fault injection with Simics. This blog post covers the new fault injection framework we added in Simics 5, and the interesting things you can do when you add record and replay capabilities to spontaneous interactive work with Simics. There is also a Youtube demo video of the system in action.
The Security Now Podcast number 497 dealt with the topic of Vehicle Hacking. It was fairly interesting, if a bit too light on the really interesting thing which is what actually went on in the vechicle hack that was apparently demonstrated on US national television at some point earlier this year (I guess this CBS News transcript fits the description). It was still good to hear the guys from the Galois consulting firm (Lee Pike and Pat Hickey) talking about what they did. Sobering to realize just how little even a smart guy like Steve Gibson really knows about embedded systems and the reality of their programming. Embedded software really is pretty invisible in both a good way and a bad way.
I have been thinking about the role and prestige of testing for the past several years. Many things I have read and things companies have done indicate that “testing” is something that is considered a bit passe and old-school. Testers are dead weight that get into the way of releases, and they are unproductive barnacles that slow development down. Testers can all be replaced by automatic testing put in place by brilliant developers. The creative developer types are the guys with the status anyway. I might be exaggerating, but there is an issue here. I think we need to be acknowledge that testers are a critical part of the software quality puzzle, and that testing is not just something developers can do with one hand tied behind their back.
There is a new post at my Wind River blog, about the Trinity of Simulation – the computer, the system, and the world. It discusses how you build a really complete system model using not just a virtual platform like Simics, but you also integrate it with a model of the system the computer sits in, as well as the world around it. Like this:
Read more about it in the blog post, and all the older blog posts it links to!
There is a new post at my Wind River blog, an interview with Andreas Buchwieser from the Wind River office in München. It discusses how Simics can be applied to the field of safety-critical systems, including helping test the software to get it certified. Really interesting, and in particular it is worth noting that qualifying tools in the IEC 61508 and ISO 26262 context is much easier than in DO-178B/C. The industrial family of safety standards have been created to allow for tools to help validate an application without forcing incredibly high demands on the development of those tools.
There is a new post at my Wind River blog, about how Simics is used to simulate large wireless networks for IoT (Internet-of-Things) applications.
It is funny for me to be back at the IoT game. A decade ago (time flies, doesn’t it?), at Virtutech, I and Johan Runeson took part in an EU research project on exactly this topic. Unfortunately, we had to back out of that project due to economic circumstances and failing management commitment, but we still learnt a few things that were relevant now that we are back in the IoT game. In particular, how to simulate wireless networks in a reasonable way in a transaction-level simulator. Thus, payback for the investment took 10 years to arrive, but it did arrive. To me, that underscores the need to be a bit speculative, take some risk, and try to explore the future.
I am going to be speaking at the 2015 Embedded World Conference in Nürnberg, Germany. My talk is about Continuous Integration for embedded systems, and in particular how to enable it using simulation technology such as Simics.
My talk is at 16.00 to 16.30, in session 03/II, Software Quality I – Design & Verification Methods.
There is a new post at my Wind River blog, about how you can use Simics to enable the automatic testing of pretty much any computer system (as long as we can put it inside a simulator). This is a natural follow-up to the earlier post about continuous integration with Simics and Simics-Simulink integrations — automated test runs is a mandatory and necessary part of all modern software development.
The September 2013 issue of the Intel Technology Journal (which actually arrived in December) is all about Simics. Daniel Aarno of Intel and I served as the content architects for the issue, which meant that we managed to contributed articles from various sources, and wrote an introductory article about Simics and its usage in general. It has taken a while to get this journal issue out, and now that it is done it feels just great! I am very happy about the quality of all the ten contributed articles, and reading the final versions of them actually taught me some new things you could do with Simics! I already wrote about the issue in a Wind River blog post, so in this my personal blog I want to be a little bit more, well, personal.
As an old embedded systems and real-time guy, I have always worked with computer systems that are in some way tied to their environment. Simics has often been used to model such computer systems, inside of customer organizations. Which makes it a bit hard to show… however, recently I have cooked up a demo showing Simics simulating a computer system alongside a physical system.
I just put out a post on the Wind River blog, pointing to both a video of my own “water heater” demo and some other Youtube videos showing Simics integrated with simulations of the real world. A screenshot of my setup in action is shown on the side of this post.
Debugging – the 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems by David Agans was published in 2002, based on several decades of practical experience in debugging embedded systems. Compared to the other debugging book I read this Summer, Debugging is much more a book for the active professional currently working on embedded products. It is more of a guidebook for the practitioner than a textbook for students that need to learn the basics.
It is quite interesting to see how Qualcomm has emerged as a major player in the “processor market” and is trying to build themselves into a serious consumer brand. I used to think of them as a company doing modems and other chips that made phones talk wirelessly, known to insiders in the business but not anything a user cared about. Today, however, they are working hard on building themselves into a brand to rival Intel and AMD. At the center of this is their own line of ARM-based application processors, the Snapdragon. I can see some thinking quite similar to the old “Intel Inside” classic, and I would not be surprised to see the box or even body of a phone carrying a Snapdragon logo at some point in the future. A part of this branding exercise is the Snapdragon Batteryguru, an application I recently stumbled on in the Google Play store.
Adding electronics to systems that used to be mechanical has been the great wave of innovation for a quite a while now. Modern transportation just would not work without all the electronics and computers inside (someone once quipped that a modern fighter is just a plastic airplane full of software), and so much convenience has been provided by automation and smarts driven by electronics. However, this also introduces brand new ways that things can break, and sometimes I wonder if we really are not setting ourselves up for major problems when the electrons stop flowing.
Logging as as debug method is not new, and I have been writing about it to and from over the past few years myself. At the S4D conference, tracing and logging keeps coming up as a topic (see my reports from 2009, 2010 and 2012 ). I recently found an interesting piece on logging from the IT world in the ACM Queue (“Advances and Challenges in Log Analysis“, Adam Oliner, ACM Queue December 2011). Here, I want to address some of my current thoughts on the topic.
After some discussions at the S4D conference last week, I have some additional updates to the history and technologies of reverse execution. I have found one new commercial product at a much earlier point in time, and an interesting note on memory consistency.
Last week, I attended my fourth System, Software, SoC and Silicon Degug conference (S4D) in a row. I think the silicon part is getting less attention these days, most of the papers were on how to debug software. Often with the help of hardware, and with an angle to how software runs in SoCs and systems. I presented a paper reviewing the technology and history of reverse debugging, which went down pretty well.
I am going to the S4D conference for the third year in a row. This year, I have a paper on reverse debugging, reviewing the technology, products, and history of the idea. I will probably write a longer blog post after the conference, interesting things tend to come up.
I am going to be talking about how to transport bugs with virtual platform checkpoints, in the Software Tools track at the Embedded Conference Scandinavia, on October 3, 2012, in Stockholm (Sweden). The ECS is a nice event, and there are several tracks to choose from both on October 2 and October 3. In addition to the tracks, Jan Bosch from Chalmers is going to present a keynote that I am sure will be very entertaining (see my notes from a presentation he did in Göteborg last year).
I am scheduled to talk at the SiCS multicore day 2012 (like I did back in 2009 and 2008). The event takes palce on September 13, at SiCS in Kista. My topic will be on System-Level Debug – how we can make debuggers that work for big systems.
This year, the multicore day is part of a bigger Software Week event, which also covers cloud and internet of things. See you there!
There is a new post at my Wind River blog, about how Simics was used to kick-start the development of the 64-bit version of VxWorks. It is an interesting example of how to use a virtual platform as a model of something much simpler and gentler than actual hardware systems.
There is a new post at my Wind River blog, about the testing on an integrated software stack in simulation. I base the discussion on the very interesting report about the Toyota “unintended acceleration” problems and the deep investigation into the control software of the affected vehicles performed by a NASA team (!). The report covers a lot of different tools, but also notes that about the only thing not done was to integrate the complete software stack in simulation.
There is a new post at my Wind River blog, about how you can use a virtual platform to complete work faster. Not by making the virtual platform execution of target code faster, but by optimizing the way you work and taking advantage of the features of a virtual platform.
There is a new post at my Wind River blog, about iterative hardware-software interface design. It is a discussion with some examples of why hardware designers would do well to use virtual platforms to include software designers in the loop when designing new devices and their programming interfaces.
I have a fairly lengthy new blog post at my Wind River blog. This time, I interview Tennessee Carmel-Veilleux, a Canadian MSc student who have done some very smart things with Simics. His research is in IMA, Integrated Modular Avionics, and how to make that work on multicore.
I just found the blog of an old real-time researcher friend of mine, John Regehr at the University of Utah.
It is at http://blog.regehr.org/ and covers a range of embedded topics relevant to his academic research (which is more embedded that most).