Back in the late 1980s, in what seems to have been a golden era of software innovation, there were a number of companies trying to sell graphical programming environments for the graphics-oriented Macintosh computer. Makes sense, right? I remember that I did not think so at the time, and the reason was that things tended to look more complex in a visual system than when expressed as plain text. I think one of the examples I saw was Prograph, where unwieldy flow charts replaced nice and dense if-then statements.

Today, twenty years later, it is hard to dispute the success of model-driven and graphical environments like MatLab, LabView, and others. Dataflow is very good paradigm for many things in signal processing, and it fits very nicely with graphical representations. UML statecharts do express state machines in a pretty readable manner which is far superior to any particular code encoding of the same information.

However, I think the key here is that the languages are domain-specific to a large extent. General-purpose programming is still not really feasible in a graphical way, and I doubt it will ever be. You also need to “go with the flow” in these systems to get full benefit from them. If you start to think in imperative coding terms, you tend to quickly get lost… Here is a personal example that a friend (who will remain anonymous to protect the guilty) and I cooked up in NXT, trying to code an algorithm to guide a robot along a line. The picture both contains the graphical representation and the approximately equivalent imperative-style code.

All the little red boxes are value reads, writes, and manipulations. It is kind of elegant that you can pass values on from one block to the next, but overall the graphical representation must be said to be totally unreadable. The logic was pretty good, actually, but it did not do things like you are supposed to do in Mindstorms. Rather than a reactive “act wait act” cycle, we tried to code complex loops.

I guess this is an example of the old saying that a good (or determined) programmer can write FORTRAN in any programming language. As an aside, I recall a Russian-schooled professor I had as an undergraduate who managed to abuse Prolog to create a deterministic loop from its backtracking system. Ugly indeed, but also quite an impressive feat of semantic wrestling.

The Mindstorm-style solution to following a line was found by a bit of Googling, and for the robot that I built a fairly (not entirely, though) robust program looked like this. Credit to http://www.nxtprograms.com/line_follower/steps.html for the original program that I had to modify just a bit to work with Mindstorms NXT 2.0.

In a sense, this program is bad since it obviously relies on non-functional properties of the NXT environment which are not semantically specified (i.e., the speed of sensing and how to drive the motors the fastest). However, the sense-act loop with no state saving is the right way.

Obviously, other graphical environments will have other strengths and weaknesses, but I think that this is a nice illustration that the typical constructs of general-purpose programming languages are poor fits for graphical visualization. Variables are simply not very visual in their nature…

For my parental leave, I have just bought myself a Lego Mindstorm NXT 2.0 kit. It is not much fun for our youngest, who mostly gets a bit scared by a piece of Lego driving around making noises, but I hope to be able to use it to teach my older child (almost five) to program. Let’s see how that turns out. It looks hard to make the NXT environment provide the kind of Roborally-style programming blocks that I had hoped to create, as I cannot for some reason get a sufficiently custom icon onto custom blocks.

It also presented me with an opportunity to try some domain-specific high-level graphical programming. The programming environment provided for the NXT series of Mindstorms kits is based on LabView from National Instruments, and it really does seem to work. It even features parallel tasks, which I tried to use…

It turned out that it is not that easy to get it right. I was trying to have a few different tasks look at different sensors and steer the robot in different directions based on the sensor readings. However, this quickly turned into a literal deadlock: the robot just sat there, doing nothing, after the first time that my ultrasound sensor task tried to steer it. Failure.

The manual is quite silent on the tasking semantics, and there are no (or I have not yet found) shared variables, locks, or message-passing mechanisms to synchronize the tasks.

Looking for an answer on the web, I came across a nice tutorial on tasking:

• http://www.ortop.org/NXT_Tutorial/tasks.html

It is a video, and it ends with the note that the only reliable way to multitask in the NXT environment is to NOT try to control the same thing from multiple tasks. If I want to listen to multiple sensors, the only way is to create a loop with switching on inputs. I.e., manual polling. So much for using the natural language of tasks to handle naturally concurrent activities. At some point, I might figure out if I can construct it from tasks and data value transfers, but for now, I will just go with the flow and write (or rather draw) some polling loops.

Here is an example from the manual. Note that the top “beam” controls motor A, and the bottom controls motors B and C:

That last part about data wires is intriguing… but I will need to find some more reference materials.

Anyway, the way to express parallel tasks here is really quite neat. At least as long as they are dealing with completely separate aspects of the control of the robot.

If this had been a more hard-core environment, it would have been fun to put different tasks on the ARM7 and AVR processors that are inside the NXT 2.0 brick. Yes, it is a true multiprocessor, if very limited in capabilities. At http://mindstorms.lego.com/en-us/whatisnxt/default.aspx you have the specs. ARM7, 256 kB of FLASH, 64 kB of RAM, and the AVR has 512 bytes of RAM and 4 kB of FLASH. A nice real embedded machine!