testing – Observations from Uppsala

Testing Mistral Le Chat (Coding and Understanding Code)

I am coming back to my project of testing AI models, local and in the cloud, on a few coding problems that seem surprisingly difficult. The models I used in my previous posts (analyze code, write code, reason about code) were mostly from the US or China, even though I did try the French Mistral-7B model. In this post, I test the full-power cloud-based Mistral Le Chat, as well as the midsize local Codestral model. Time to try the European AI!

More Ghostwrite Bugginess with RISCVuzz

In my previous blog about the Ghostwrite vulnerability in the Alibaba T-Head C910 RISC-V-based processor, I noted that the authors of the paper had found more than just that one bug. The additional bugs are worth their own write-up, as they offer some more examples of what looks to be poor testing.

Ghostwrite – Now This is Weird

In August, a strange security vulnerability dubbed “Ghostwrite” was making the rounds in the press. Basically, a vector store instruction on an Alibaba T-Head C910 RISC-V-based processor would just write to a physical address without doing a virtual-to-physical translation or checking any kind of access rights. That is just totally weird. Just how could that be implemented and slip through testing???

The Quarterly Product and Feature Update

I think of myself to be a technical person. I like computers, simulators, code, things like that. And obviously interacting with people and helping them solve their technical problems using technology I know. However, it seems that one of the most impactful contributions made during my time at Intel was to start a meeting series. Maybe you can call it a process innovation.

Testing what the User will See (and why it might not work out)

The other day, I checked out the web interface to the database of in-patient care diagnoses run by Swedish Socialstyrelsen. On first opening the site, it looked broken and unusable – the text was basically unreadable, mixing giant numbers with strange- looking regular characters, lines of text overlapping each other, and a general sense of being totally chaotic. That is not what you or I would expect from an official site of a government entity. They have no reason to play games with the look of the site. So how come it was all that broken? Didn’t they test the site properly under all circumstances?

Blog: Grug Brained Developer Make Sense

A colleague pointed me at the grugbrain.dev website the other week. It is a very humorous set of observations on corporate life and advice to young developers – written in a silly “cave-man” style. It is obviously based on long experience. The text is also quite quote-worthy, with some passages even being quite poetic in their rough-hewn simplicity.

The ESA Schiaparelli Crash & Simulation

Back in 2016, the European Space Agency (ESA) lost the Schiaparelli Mars lander during its descent to the surface on Mars. From a software engineering and testing perspective, the story of why the landing failed (see for example the ESA final analysis, Space News, or the BBC) is instructive. It comes down to how software is written and tested to deal with unexpected inputs in unexpected circumstances. I published a blog post about this right after the event and before the final analysis was available. Thankfully, that has since been retired from its original location-it was a bit too full of speculation that turned out to be incorrect… So here is a mostly rewritten version of the post, quoting the final analysis and with new insights.

Intel Blog: The Right Mindset and Toolset for Testing

I have a two-part series (one, two) on testing posted on my Software Evangelist blog on the Intel Developer Zone. This is a long piece where I get back to the interesting question of how you test things and the fact that testing is not just the same as development. I call the posts Mindset and Toolset

Continue reading “Intel Blog: The Right Mindset and Toolset for Testing”

Intel Blog: Testing and the ESA Schiaparelli Lander

It is really sad that the European Space Agency (ESA) lost their Schiaparelli lander last year, as we will miss out on a lot of Mars science. From a software engineering and testing perspective, the story of why the landing failed rather instructive, though. It gets down to how software can be written and tested to deal with unexpected inputs in unexpected circumstances. ~~I wrote a piece about this on my blog at the~~ ~~Intel Developer Zone.~~ Update: read the 2022 take on Schiaparelli loss on this blog. The Intel blog post has been retired.

UndoDB Recording during Automatic Testing

UndoDB is an old player in the reverse debugging market, and have kept at it for ten years. Last year, they released the Live Recorder record-replay function. Most recently, they have showed an integration between the recorder function and Jenkins, where the idea is that you record failing runs in your CI system and replay them on the developer’s machine. Demo video is found on Youtube, see https://www.youtube.com/watch?v=ap8552P5vss.

Continue reading “UndoDB Recording during Automatic Testing”

Intel Blog: Wind River Using Simics to Test IoT at Scale

intel sw small This really happened last week, but I was in the US for the DAC then. I did another blog on Intel Software blog, about a white paper that Wind River put out about how they use Simics internally. The white paper is a really good set of examples of how Simics can be used for software development, test, and debug – regardless of how old or new the hardware is. It also touches my favorite topic of IoT simulation and scaling up – Wind River is actually using Simics for 1000+ node tests of IoT software! Read on at https://blogs.intel.com/evangelists/2016/06/06/wind-river-uses-simics-test-massive-iot-networks/

In Defense of Testing and Testers

opinion I have been thinking about the role and prestige of testing for the past several years. Many things I have read and things companies have done indicate that “testing” is something that is considered a bit passe and old-school. Testers are dead weight that get into the way of releases, and they are unproductive barnacles that slow development down. Testers can all be replaced by automatic testing put in place by brilliant developers. The creative developer types are the guys with the status anyway. I might be exaggerating, but there is an issue here. I think we need to be acknowledge that testers are a critical part of the software quality puzzle, and that testing is not just something developers can do with one hand tied behind their back.

Continue reading “In Defense of Testing and Testers”

Product Holes: Tesla Roadster & iPhone 4

Continuing on the thread from my previous post about the testing of products that fail to find problems that become obvious to (some) users after a very short time, I just read an article (in Swedish) about how the famed Tesla roadster cars behaved when they were confronted with Scandinavian winters.

Continue reading “Product Holes: Tesla Roadster & iPhone 4”