This post is about the use of indirection in a model to increase its flexibility and ease of use, at the cost of a bit more work for the first model to be created.In particular, indirection in the sense of having explicit objects in a simulation to represent things like networks and cables connecting virtual machines.

There is a well-known saying (by David Wheeler) that “any problem in computer science can be solved with another layer of indirection”. Among computer architects, this is often used with addition “…or a cache”. I think this is true, most of the time. The number of times that adding some indirection to an architecture for a program has simplified it — or made it feasible at all — are too many to count. It is at the very core of object-oriented programming, and the number of times you end up passing around function pointers is innumerable.

In the world of virtual platforms, there is one particular area where I see a pretty useful layer of indirection missing. Networks. Many virtual platform solutions offer various ways of connecting a virtual platform to a physical world, for interfaces like USB, Ethernet, or serial. Most virtual platforms achieve this by making the virtual hardware directly connect to the outside world.

Here is an illustration for Ethernet, where I have included a PHY in the picture. Quite often, you don’t even get that, just an Ethernet device that includes its PHY and connects out to a physical network. That’s what Qemu tends to do, for example.

This approach is the simplest when your thinking is that you will model a single device, and simulate one virtual machine at a time, connecting to the physical network to receive stimulus. For USB, this means the useful feature of connecting a camera or USB disk on your PC to the virtual machine. And as a bonus, you can connect multiple machines together using some form of cross-connection on the PC (such as TAP network interface).

However, there is a much better structure that is employed in some simulators. It is based on making each network an explicit object in the simulation, and have all virtual devices talk to the virtual network. Connections to the physical world are then handled by the virtual network, or, even better, by another device attached to the same virtual network.What you also get is the ability to connect multiple virtual devices to each other over the virtual network, and to easily write simulation modules that inspect or do fault-injection on the network traffic.

The picture below illustrates the idea for Ethernet:

The cost of this architecture is that you have to create the virtual network object, and invent the interface between devices and the network. This increases the cost for the first network device you create, and if all  you are tasked with is that single device, I can see why some simulation designers took the direct route. However, if you think about the task of creating tens of devices connecting to the same type of network, the “cost” of creating a virtual network is actually negative. Using an indirect approach like this makes creating each device simpler, and each device immediately gets the benefit of all the services that have been added to the virtual network. As long as a device can connect to the virtual network, it can connect to the physical network without any extra coding or cost.

Encapsulating entire networks with multiple virtual machines within a single simulation session is also very beneficial for control, inspection, and determinism. Relying on a physical connection between virtual machines makes all packets pass the unreliable and random real world on their way between machines, destroying any determinism or control you might have hoped to incur.

In the world of SystemC simulation, an indirect approach like this is also a way to overcome some silly language limitations. Unbelievable as it might sound to the uninitiated, in SystemC you set up a simulation once into a single static setup (in something called the elaboration phase), and then that is what you simulate. There is no option to setup connection between modules or even add new modules to the simulation after the initial setup. Here, you can use a layer of indirection as a work-around. At the  start of simulation, connect all devices that might at some point in time be connected to a particular network to that network. During simulation, configure and reconfigure the network module to only allow traffic from and to certain modules, essentially creating a useful illusion that they are connected and disconnected from the network.

I hope I have convinced you: if you ever build a virtual platform, make sure to make all connections indirect.

The article is highly recommended for its long historical look at network processing and network processing offload. As the balance  between speeds of networks, processors, memory, and interconnects between network cards and the rest of the system has changed over the years, it is an idea that occasionally (four or five times since the 1970s) has made sense. However, in the end, Mike thinks that it usually does not, and for a machine with multiple cores and a modern fast interconnect, it is hard to see how a hardware accelerator can actually help speed things up much when the coordination between the hardware and the software is accounted for. Even if there would appear to be a big bottleneck somewhere today, we can be sure that it wil be removed in the next generation of hardware, rendering the market window for an accelerator quite short.

I read this article as another great motivation for the need to carefully consider the functional design of the hardware-software interface for acceleration devices. For simple data-pumping or media-processing units, this looks easy. For something as complex as TCP/IP processing, it is not. I think the key is that for TCP, we have something that is much more like control-plane processing than data-plane processing, and that is harder to efficiently integrate between hardware and software. Also, there is not really that much work left to offload once data copies have been architected in the right way (and I read Mike’s article to say that we now know how to do this in a sufficently few-copies way that software is close to optimal in architecture).

From a market perspective, it would also indicate that the acceleration circuits that are in common use today are by definition those that make sense. Having hardware-accelerated graphics and video decoders does seem to help build more efficient and attractive computer systems, as do cryptography accelerators. With this view, it will be interesting to see which of all the accelerators found in modern networking SoCs like those from Freescale and Cavium will survive the test of time. I am willing to put a small bet that pattern-matching engines for traffic inspection is one of them. Apart from that, hard to say.

So go read that article before you start designing your next brilliant accelerator for a common expensive operation.

It also reminds me of a whitepaper I wrote early this year on how to evaluate performance of a hardware accelerator in the context of a full system with a full software stack, considering the details of the hardware-software interface.