However, this did get me thinking some about the articulation of problems.

Reflecting over my experience from almost a decade in industry following my decade in the university, one thing that does stand out is a certain shift of focus in how I think about products and solutions. When I was an undergrad and grad student, the natural focus was on the solution. How did it work? Now, I tend to look for the problem. What can it do, why is it relevant?

To go back to the experiences that prompted this post: If someone cannot give me a good description of the problem they are trying to solve with their work, I have a hard time motivating myself to listen.

To be fair, there is usually a real and relevant problem behind most research. It is just that it might be implicit in the mind of the researcher, rather than explicit and part of the design and planning process. I sometimes find myself having to ask a series of fairly tough questions to try to pry out the problem from underneath the layers of assumptions about problem domains and what’s important.This can make people uncomfortable, but I hope that it is all for the best in the end.

Bringing out the problem into the open and making it explicit makes it much easier to discuss about it and consider the direction and potential value of research. Is the problem relevant for one or one million people? Is there some other problem that can be solved with the same methods, with a small shift in focus? Making the problem explicit can be very powerful as a methodology.

It is amazing just how much time I and my colleagues have spent over the past years trying to pin down the problems that we want to attack with the Simics product. When you have a product to sell and you depend on selling it, articulating the problem does become very important. The value of a solution to a customer comes from solving problems, nothing else.

There is one more trick to problem formulation that needs to be considered and that I think can really help anyone formulate their problem statement. When a business grows, you have to add a bigger sales force. To make new sales people productive as soon as possible, the marketing and product people have to be able to articulate just which types of accounts (companies) and which types of projects and people inside the companies to look for.  Equally (or even more) important is making it very easy to understand when to walk away from a discussion, since there is no match between the customer problems and your solution, and therefore no chance of closing a deal. Thinking about a solution or product in this light does make a difference.

The question I think any researcher should ask themselves before presenting their work is: how can I explain it in such a way that someone in the audience can explain to a third person what it is all about? Which problems are being considered – and which problems are not being considered? Can someone in the audience pick up on what I say and carry on to a third party – who might be that perfect contact I need in order to further my career as a researcher?

That is the true definition of knowing your problem, I think.

To paraphrase Bill Clinton, it’s the problem, stupid.

Basically, the article describes the changes in mindset that a development team goes through as they progress from research to enthusiastic early adopters to normal users. The large population of normal users want useful results, not necessarily a deep understanding for how a tool works. In my experience, this leads to “ah, why didn’t we see that” moments as big (and in hindsight, obvious) flaws in the product and its user interface are revealed.

It also leads to “aich, why do we have users” moments when users want something that just is not feasible, sensible, or even sane. There is a nice quote from the article that really made me laugh:

Users really want the same result from run to run. Even if they changed their code base. Even if they upgraded the tool. Their model of error messages? Compiler warnings. Classic determinism states: the same input + same function = same result. What users want: different input (modified code base) + different function (tool version) = same result. As a result, we find upgrades to be a constant headache. Analysis changes can easily cause the set of defects found to shift…

There are so many similar requests made by users of technically advanced tools… and finding ways to get around them is part of the real fun of technical marketing. Reconciling impossible requirements with the technically feasible in some way that meets customer expectations is a very nice challenge to go through.

In this case, the fundamental issue is that the Coverity tool is an approximation, not an exact tool. Small fluctuations in input or the tool itself can change the set of issues flagged in a body of code. If a tool tried to be exact, its runtime complexity would explode. Even going some distance in that direction like the Polyspace Verifier (now part of the Mathworks) drastically increases the runtime. For more on static analysis in practice, I recommend a MSc Thesis from Linköping.

The article is a good read for anyone interested in either static checking as a technology, or in how to commercialize good research.

What Thomas is showing is just how efficiently today’s economy manages to hide complexity from consumers (users). That toaster is just there on the shelf at a very low cost. If you take it apart, you will note that it is made from plastic which has been moulded to shape, and various bits and pieces of steel and copper wires. At that level, you feel that you could almost build it yourself. However, that is just the tip of the iceberg. What the toaster project reveals is the next level of abstraction and information hiding going on: that copper wire contains an enormously complex process in its making. From ore extraction, energy production to fuel the process, copper foundries, factories converting raw copper into wires, and a huge logistical machine to move things around.

In essence, we have a very nice example of information hiding and abstraction. As a user of the toaster, I do not need to understand how it works, and I do definitely not have any idea of the huge chain of suppliers leading up to its presence on my breakfast table.

That’s where are going with computers, but it is going to take time. Today, most users are fairly well shielded from how computers really work. Until they break down, at least. As programmers, we are less lucky. In practice, most good programmers end up understanding at least the basics of assembly language and the memory hierarchy of the machine.

What is hidden today is mostly the innards of the silicon. I have no real idea of how a processor works at the level of transistors and electrons. I don’t have to care about that, while any computer user fifty years ago probably had a decent understanding of the electronics. If nothing else, that was how you investigated hardware faults and actually built computers in a factory. Before integrated circuits, the electronic bits were much more exposed.

I think the current trend towards virtualization in the IT space and virtual platforms in the system design space is showing that the abstraction stack we are using in computing is getting deeper and more opaque. It takes some getting used to, but in the end, we have to realize that most computer programmers will be like the toaster user. All they want is a virtual toaster that toasts virtual bread in a way that lets them do their job.That is: write software that really does not care that much about the particulars of the hardware it is running on.

For the designer of a toaster (or even worse, the manufacturer of copper wire or the oil producer for the raw materials for the plastics), this takes some getting used to. We have to accept that in many cases, a simple abstraction is sufficient to help programmers get moving. There is no need for perfect timing accuracy or all the details of bus transactions. As long as what comes out is sufficiently similar to toast (a virtual toaster spitting out candy would be a bad abstraction), most users are happy.

Brian Bailey touched on this in a blog post following DAC, called “Accuracy does not imply accuracy“. Same idea as the toaster: you have to accept less detail, more abstraction, to get somewhere useful. Not everyone needs to go back to basics… and doing so tends to be counter productive in the end.

It is late now, but I think I will have toast and jam for breakfast tomorrow. Writing this got me hungry.