The introduction of non-volatile memory that is accessed and addressed like traditional RAM instead of using a special interface has some rather interesting effects on software. It blurs the traditional line between persistent long-term mass storage and volatile memory. On the surface, it sounds pretty simple: you can keep things living in RAM-like memory across reboots and shutdowns of a system. Suddenly, there is no need to reload things into RAM for execution following a reboot. Every piece of data and code can be kept immediately accessible in the memory that the processor uses. A computer could in principle just get rid of the whole disk/memory split and just get a single huge magic pool of storage that makes life easier. No file system, no complications, easy programmer life. Or is it that simple?
Continue reading “Non-Volatile Memory is Different from Non-Volatile Storage”
A while ago, I visited my Intel colleagues in Costa Rica and ran a workshop for university teachers and researchers, showing how Simics could be used in academia. I worked with a very smart and talented intern, Jose Fernando Molina, and after a rather long process I have published an interview with him on my Intel blog: https://software.intel.com/en-us/blogs/2017/12/05/windriver-simics-to-inspire-teachers-costarica
Continue reading “Intel Blog Post: Using Wind River® Simics® to Inspire Teachers and Researchers in Costa Rica”
Over time, Intel and other processor core designers add more and more instructions to the cores in our machines. A good question is how quickly and easily new instructions added to an Instruction-Set Architecture (ISA) actually gets employed by software to improve performance and add new capabilities. Considering that our operating systems and programs are generally backwards-compatible, and run on all kind of hardware, can they actually take advantage of new instructions?
Continue reading “Intel Blog: Question: Does Software Actually Use New Instruction Sets?”
A blog post from Undo Software informed me that Microsoft has rather quietly released a reverse debugger tool for Windows programs – WinDbg with Time Travel Debug. It is available in the latest preview of WinDbg, as available through the Windows Store, for the most recent Windows 10 versions (Anniversary update or later). According to a CPPcon talk about the tool (Youtube recording of the talk) the technology has a decade-long history internally at Microsoft, but is only now being released to the public after a few years of development. So it is a new old thing 🙂
Continue reading “A new (and old) Reverse Debugger – Microsoft WinDbg”
Intel Software Guard Extensions (SGX) is a pretty cool piece of technology that aims to make it possible for user programs to hide secrets from other user programs and the operating system itself. It establishes enclaves in the system that hides the data being processed and the code processing it from all other software. The original application for SGX was to support client-machine features like DRM, to create a safe space on a client that a server can trust. Recently, the people behind the Signal messaging system have provided a really interesting example of an application that makes use of the of SGX “in reverse”, to make it possible for a client to trust a server.
Continue reading “Signal Secure and Private Contacts Matching – Using SGX”
Integration is hard, that is well-known. For computer chip and system-on-chip design, integration has to be done pre-silicon in order to find integration issues early so that designs can be updated without expensive silicon re-spins. Such integration involves a lot of pieces and many cross-connections, and in order to do integration pre-silicon, we need a virtual platform.
Continue reading “Intel Blog Post: The More the Merrier – Integration and Virtual Platforrms”
IEEE Spectrum ran a short interview with Thomas Knoll, the creator of Photoshop, who made a very interesting point about the move to subscription-based software rather than one-time buys plus upgrades. His point is that if you are building based sold using the “upgrade model”, developers have to create features that cause users to upgrade. In his opinion, that means you have to focus on flashy features that demo well and catch people’s attention – but that likely do not actually help users in the end.
Continue reading “Subscription Software = Better Programmer-User Alignment (?)(!)”
I have just published a piece about the Intel Excite project on my Software Evangelist blog at the Intel Developer Zone. The Excite project is using a combination of of symbolic execution, fuzzing, and concrete testing to find vulnerabilities in UEFI code, in particular in SMM. By combining symbolic and concrete techniques plus fuzzing, Excite achieves better performance and effect than using either technique alone.
Continue reading “Intel Blog: Finding BIOS Vulnerabilities with Symbolic Execution and Virtual Platforms”
Human Resource Machine, from the Tomorrow Corporation, is a puzzle game that basically boils down to assembly-language programming. It has a very charming graphical style and plenty of entertaining sideshows, hiding the rather dry core game in a way that really works! It is really great fun and challenging – even for myself who has a reasonable amount of experience coding at this low level. It feels a bit like coding in the 1980s, except that all instructions take the same amount of time in the game.
Continue reading “Assembly Programming as Game: “Human Resource Machine””
Today, when developing embedded control systems, it is standard practice to test control algorithms against some kind of “world model”, “plant model” or “environment simulator”.
Using a simulated control system or a virtual platform running the actual control system code, connected to the world model lets you test the control system in a completely virtual and simulated environment (see for example my Trinity of Simulation blog post from a few years ago). This practice of simulating the environment for a control computer is long-standing in the aerospace field in particular, and I have found that it goes back at least to the Apollo program.
Continue reading “Simulation and the Apollo Guidance Computer”
In the early 1990s, “PC graphics” was almost an oxymoron. If you wanted to do real graphics, you bought a “real machine”, most likely a Silicon Graphics workstation. At the PC price-point, fast hardware-accelerated 3D graphics wasn’t doable… until it suddenly was, thanks to Moore’s law. 3dfx was the first company to create fast 3D graphics for PC gamers. To get off the ground and get funded, 3dfx had to prove that their ideas were workable – and that proof came in the shape of a simulator. They used the simulator to demo their ideas, try out different design points, develop software pre-silicon, and validate the silicon once it arrived. Read the full story on my Intel blog, “How Simulation Started a Billion-Dollar Company”, found at the Intel Developer Zone blogs.
A new entry just showed up in the world of reverse debugging – Simulics, from German company Simulics. It does seem like the company and the tool are called the same. Simulics is a rather rare breed, the full-system-simulation-based reverse debugger. We have actually only seen a few these in history, with Simics being the primary example. Most reverse debuggers apply to user-level code and use various forms of OS call intercepts to create a reproducible run. Since the Simulics company clearly comes from the deeply embedded systems field, it makes sense to take the full-system approach since that makes it possible to debug code such as interrupt handlers.
I have also updated my history of commercial reverse debuggers to include Simulics.
Continue reading “Simulics – A New Commercial Reverse Debugger”
Intel CoFluent Technology is a simulation and modeling tool that can be used for a wide variety of different systems and different levels of scale – from the micro-architecture of a hardware accelerator, all the way up to clustered networked big data systems. On the Intel Evangelist blog on the Intel Developer Zone, I have a write-up on how CoFluent is being used to do model just that: Big Data systems. I found the topic rather fascinating, how you can actually make good predictions for systems at that scale – without delving into details. At some point, I guess systems become big enough that you can start to make accurate predictions thanks to how things kind of smooth out when they become large enough.
Simics and other simulation solutions are a great way to add more variation to your software testing. I have just documented a nice case of this on my blog at the Intel Developer Zone (IDZ), where the Simics team found a bug in how Xen deals with MPX instructions when using VT-x. Thanks to running on Simics, where scenarios not available in current hardware are easy to set up.
Continue reading “Intel Blog: Why Target Variation Matters (finding a Xen bug)”
UndoDB is an old player in the reverse debugging market, and have kept at it for ten years. Last year, they released the Live Recorder record-replay function. Most recently, they have showed an integration between the recorder function and Jenkins, where the idea is that you record failing runs in your CI system and replay them on the developer’s machine. Demo video is found on Youtube, see https://www.youtube.com/watch?v=ap8552P5vss.
Continue reading “UndoDB Recording during Automatic Testing”
Last year (2015), a paper called “Don’t Panic: Reverse Debugging of Kernel Drivers” was presented at the ESEC/FSE (European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering) conference. The paper was written by Pavel Dovgalyuk, Denis Dmitriev, and Vladimir Makarov from the Russian Academy of Sciences. It describes a rather interesting approach to Linux kernel device driver debug, using a deterministic variant of Qemu along with record/replay of hardware interactions. I think this is the first published instance of using reverse debugging in a simulator together with real hardware.
Continue reading “Reverse Debug with Hardware in the Loop”
My first blog post as a software evangelist at Intel was published last week. In it, I tell the story of how our development teams used Simics to test the software behavior (UEFI, in particular) when a server is configured with several terabytes of RAM. Without having said server in physical form – just as a simulation. And running that simulation on a small host with just 256 GB of RAM. I.e., the host RAM is just a small fraction of the target. That’s the kind of things that you can do with Simics – the framework has a lot of smarts in it.
It was rather interesting to realize that just the OS page tables for this kind of system occupies gigabytes of RAM… but that just underscores just how gigantic six terabytes of memory really is.
How important is the documentation (manual, user guide, instruction booklet) for the actual quality and perceived quality of a product? Does it materially affect the user? I was recently confronted by this question is a very direct way. It turned out that the manual for our new car was not quite what you would expect…
This is just the first page, and as you can see if you know Swedish or German or both, it is a strange interleaving of sentences in the two languages.
Continue reading “Does Documentation Matter for a Product?”
A comment on my old blog post about the history of reverse execution gave me a pointer to a fairly early example of replay debugging. The comment pointed at a 2002 blog post which in turn pointed at a 1999 LWN.net text which almost in passing describes a seemingly working record-replay debugger from 1995. The author was a Michael Elizabeth Chastain, of whom I have not managed to find any later traces.
Continue reading “A Replay Debugger from 1995!”
I love bug and debug stories in general. Bugs are a fun and interesting part of software engineering, programming, and systems development. Stories that involve running Simics on Simics to find bugs are a particular category that is fascinating, as it shows how to apply serious software technology to solve problems related to said serious software technology. On the Intel Software and Services blog, I just posted a story about just that: debugging a Linux kernel bug provoked by Simics, by running Simics on a small network of machines inside of Simics. See https://blogs.intel.com/evangelists/2016/05/30/finding-kernel-1-2-3-bug-running-wind-river-simics-simics/ for the full story.
Continue reading “Intel Blog: Finding a Linux Kernel bug by running Simics on Simics”
A new record, replay, and reverse debugger has appeared, and I just had to take a look at what they do and how they do it. “rr” has been developed by the Firefox developers at Mozilla Corporation, initially for the purpose of debugging Firefox itself. Starting at a debugger from the angle of attacking a particular program does let you get things going quickly, but the resulting tool is clearly generally useful, at least for Linux user-land programs on x86. Since I have tried to keep up with the developments in this field, a write-up seems to be called for.
Continue reading “rr– The Mozilla Reverse Debugger”
A long time ago, when I was a PhD student at Uppsala University, I supervised a few Master’s students at the company CC-Systems, in some topics related to the simulation of real-time distributed computer systems for the purpose of software testing. One of the students, Magnus Nilsson, worked on a concept called “Time-Accurate Simulation”, where we annotated the source code of a program with the time it would take to execute (roughly) on the its eventual hardware platform. It was a workable idea at the time that we used for the simulation of distributed CAN systems. So, I was surprised and intrigued when I saw the same idea pop up in a paper written last year – only taken to the next level (or two) and used for detailed hardware design!
Continue reading “Time-Accurate Simulation Revisited – 15 years later”
Intel is a big Simics user, but most of the time Intel internal use of Simics is kept internal. However, we recently had the chance to interview Karthik Kumar and Thomas Willhalm of Intel about how they used Simics to interact with external companies and improve Intel hardware designs. The interview is found on the Wind River blog network.
It is also my last blog post written at Wind River; since January 18, I am working at Intel. I am working on ways to keep publishing texts about Simics and simulation, but the details are not yet clear.
I just posted a short blog post on the Wind River blog, introducing a video demo of the Web API to Wind River Helix Lab Cloud. In the post and video, I show how the Lab Cloud Web API works. For someone familiar with REST-style APIs, this is probably baby-level, but for me and probably most of our user base, it is something new and a rather interesting style for an API. Thus, doing a video that shows the first few steps of authentication and getting things going seems like a good idea.
I have read some recent IBM articles about the POWER8 processor and its hardware debug and trace facilities. They are very impressive, and quite interesting to compare to what is usually found in the embedded world. Instead of being designed to help with software debug, it seems the hardware mechanisms in the Power8 are rather focused on silicon bringup and performance analysis and verification in IBM’s own labs. As well as supporting virtual machines and JIT-based systems!
Continue reading “Hardware debug and measurement in the IBM POWER8”
I have a long-standing interested in debugging in general and reverse debugging in particular and the related idea of record-replay debug (see a series of blog posts I did a few years ago on the topic: history 1, history 2, history 3, S4D report, updates, Simics reverse execution, and then Lab Cloud record/replay). Recently, I found out that Undo Software, one of the pioneers in the field, had released a product called “Live Recorder“. So I went to check it out by reading their materials and comparing it to what we have seen before.
Continue reading “UndoDB Live Recorder: Record/Replay for Live Systems”
The recent news that a hacked version of Apple Xcode has been used to insert bad code into quite a few programs for Apple iOS was both a bad surprise and an example of something that has been hypothesized for a very long time. For the news, I recommend the coverage on ArsTechnica of the XCodeGhost issue. It is very interesting to see this actually being pulled off for real – I recall seeing this discussed as a scenario back in the 1990s, going back to Ken Thompson’s 1983 ACM Turing Award lecture.
Continue reading “A Compiler that Inserts Bad Code – For Real”
I just added a new blog post on the Wind River blog, about how you do fault injection with Simics. This blog post covers the new fault injection framework we added in Simics 5, and the interesting things you can do when you add record and replay capabilities to spontaneous interactive work with Simics. There is also a Youtube demo video of the system in action.
I have read a few news items and blog posts recently about how various types of software running on top of virtual machines and emulators have managed to either break the emulators or at least detect their presence and self-destruct. This is a fascinating topic, as it touches on the deep principles of computing: just because a piece of software can be Turing-equivalent to a piece of hardware does not mean that software that goes looking for the differences won’t find any or won’t be able to behave differently on a simulator and on the real thing.
Continue reading “Breaking and Detecting Simulators and Emulators”
The Security Now Podcast number 497 dealt with the topic of Vehicle Hacking. It was fairly interesting, if a bit too light on the really interesting thing which is what actually went on in the vechicle hack that was apparently demonstrated on US national television at some point earlier this year (I guess this CBS News transcript fits the description). It was still good to hear the guys from the Galois consulting firm (Lee Pike and Pat Hickey) talking about what they did. Sobering to realize just how little even a smart guy like Steve Gibson really knows about embedded systems and the reality of their programming. Embedded software really is pretty invisible in both a good way and a bad way.
Continue reading “Security Now on Hacking Vehicles – A Bit of Let-Down”