David Kanter on Benchmarking and Simulation (Tech Report podcast)


The Tech Report podcast did an interview with David Kanter earlier this year. David Kanter is an industry veteran who runs http://www.realworldtech.com/, as well as being a regular contributor to the Microprocessor Report. I have read the MPR since my PhD days, and it is still one of the best places for information on new chips. The subject of the podcast episode was an analysis of the somewhat mysterious Softmachines “VISC” processor architecture. However, it rather turned into a very good discussion on how you do performance predictions, performance projections of competing systems, and the nature of benchmarking and benchmark numbers.

When you listen to the podcast, it breaks down into three parts. One is about how to architect hardware and estimate performance, one is about what Softmachines might actually be doing, and finally a Q&A about all kinds of PC-hardware-related issues. To me, the most interesting part was the first, about how to measure performance. David Kanter explains very well how computer architects work, and the power and limitation of simulation-based numbers and the SPEC benchmarks.

He points out that Softmachines have been doing some fairly fudgy stuff with their performance numbers, like mashing together SpecInt and SpecFP numbers. You would like to see them presented separately. David talks about how Spec has been broken by compilers several times, and the somewhat interesting choice of compilers in the Softmachines numbers.

On the subject on the VISC architecture, David seems to be a bit in doubt as to whether they can really deliver what they promise. He himself worked for a startup with a similar-sounding idea (Strandera) to do speculative threading, and knows how hard it is to get general-purpose software to go along with being threaded.

Just go and listen to the podcast, it is a good 90 minutes of educational material!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.