Essentially, they are voxel graphic representations that must be very hard to plan and execute.  The standard single-stud 1×1 Lego brick is their smallest unit, or maybe its 1/3 height flat version. Here are some examples that I photographed in Legoland during my visit this Summer.

Airbus A380, close to the entrance:

If we look up close at the nose, we see the stepping of voxels very clearly:

The wing roots show how the wings are constructed:

I also really liked this Volvo XC90 that I found in one of the town displays. It uses a few more types of elements compared to the big Airbus. I am not so sure about the way they do the windows, but maybe the murkiness is just due to its having been outdoors for a few years. It is interesting though that you immediately recognize this as a Volvo, which is a great testament to the Lego designers.

Another car is the Citroen found on the canalside in Amsterdam:

A typical example of the large-scale animals and people present all over Legoland:

Finally, a Danish flag in a very traditional Lego flat style:

The first movie is just the robot driving around using its MEPROM programming unit. I think this movie is slightly clearer than the old one I posted.

Next, here is how a program loop can be created in this system. By removing the white end marker from the program, the motor will just keep spinning the program until it spots a program step. This is a very mechanical infinite loop.

Finally, we have the emergency stop bumper that was added to the robot to make it stop when it hits something. It does not take effect immediately due to the latencies involved in the NXT programming environment, but at least the robot does not proceed to the next programming step.

It turned out that trying to show a group of ten impatient kids that the robot read instruction codes and moved accordingly failed. Using a chaser program, where the robot drove around waiting for something to show up close in front of it and then drove full speed forward to try to hit it, was much easier to appreciate.  Classics like driving around avoiding to hit walls was also just as much fun as ever. Like my previous attempts, the kids piled up on the robot and tried to make it turn the way they wanted.

Clearly, physical exercise is easier to sell than cerebral exercise.

That said, I did manage to get some more thoughtful kids interested in getting the robot to go particular places. The setup I used was to put the robot on a starting square marked on the floor, and then putting a red bucket out on the floor some distance away. The kids were asked to help me program the robot to either hit the bucket or drive around it. I explained what the robot did as taking steps forward and turning, and actually the approximately 50 cm long runs it did for each blue programming tile was fairly similar to a serious step by a five-year-old (some kids instead started tip-toeing forward, which did not quite provide the correct stride).

As a safety device and to retain a semblance on intelligence on the part of the robot, I equipped it with a bumper in front. As soon as the bumper is hit, the robot is supposed to say “stop” and stop all activities.

The problems we tried were:

• Put the bucket down in front and try to hit it, needing three forward to get there
• Drive around said bucket, which required a program like blue, green, blue, yellow, blue, blue, yellow, blue, green (forward, right, forward, left, forward, forward, left, forward, right).
• Leave the base and drive to the back instead of front.

I sat down with a group of three kids who really got into this part (even with ten other kids running around playing cops and robbers at a very high level of intensity). They did get the idea and did propose steps towards the solution. However, it is clear that at least initially; one cannot expect them to plan out an entire program in their heads.

Rather, we ended up in a style quite familiar to anyone learning a new programming language or taking their first programming course. Try one step, see where it takes us, add another step or two to the program, and try again. Very much how I myself solve programming and scripting problems where I interact with a complex and possibly unpredictable environment. In this way, we did manage to solve some problems, and I do think the kids understood what was going on.

To me, I feel the idea of the Lego-programmable robot as a preschool-level programming course is at least partially proven. As in most other educational ventures, what is needed is a good overall teaching environment. Next time, I will have to try this with a small group of kids, just after they have had a snack so that they are alert and fresh. I will also make sure to break up the hard work with some simple play, since that is needed to keep the kids from being bored.

Back in the late 1980s, in what seems to have been a golden era of software innovation, there were a number of companies trying to sell graphical programming environments for the graphics-oriented Macintosh computer. Makes sense, right? I remember that I did not think so at the time, and the reason was that things tended to look more complex in a visual system than when expressed as plain text. I think one of the examples I saw was Prograph, where unwieldy flow charts replaced nice and dense if-then statements.

Today, twenty years later, it is hard to dispute the success of model-driven and graphical environments like MatLab, LabView, and others. Dataflow is very good paradigm for many things in signal processing, and it fits very nicely with graphical representations. UML statecharts do express state machines in a pretty readable manner which is far superior to any particular code encoding of the same information.

However, I think the key here is that the languages are domain-specific to a large extent. General-purpose programming is still not really feasible in a graphical way, and I doubt it will ever be. You also need to “go with the flow” in these systems to get full benefit from them. If you start to think in imperative coding terms, you tend to quickly get lost… Here is a personal example that a friend (who will remain anonymous to protect the guilty) and I cooked up in NXT, trying to code an algorithm to guide a robot along a line. The picture both contains the graphical representation and the approximately equivalent imperative-style code.

All the little red boxes are value reads, writes, and manipulations. It is kind of elegant that you can pass values on from one block to the next, but overall the graphical representation must be said to be totally unreadable. The logic was pretty good, actually, but it did not do things like you are supposed to do in Mindstorms. Rather than a reactive “act wait act” cycle, we tried to code complex loops.

I guess this is an example of the old saying that a good (or determined) programmer can write FORTRAN in any programming language. As an aside, I recall a Russian-schooled professor I had as an undergraduate who managed to abuse Prolog to create a deterministic loop from its backtracking system. Ugly indeed, but also quite an impressive feat of semantic wrestling.

The Mindstorm-style solution to following a line was found by a bit of Googling, and for the robot that I built a fairly (not entirely, though) robust program looked like this. Credit to http://www.nxtprograms.com/line_follower/steps.html for the original program that I had to modify just a bit to work with Mindstorms NXT 2.0.

In a sense, this program is bad since it obviously relies on non-functional properties of the NXT environment which are not semantically specified (i.e., the speed of sensing and how to drive the motors the fastest). However, the sense-act loop with no state saving is the right way.

Obviously, other graphical environments will have other strengths and weaknesses, but I think that this is a nice illustration that the typical constructs of general-purpose programming languages are poor fits for graphical visualization. Variables are simply not very visual in their nature…

As discussed in my previous blog post about Kindergarten robots, I wanted to see if I can teach kids the core idea of programming. This project has now progressed to the point that I have a working prototype of a programmable robot.

Essentially, the robot is programmed by putting colored Lego bricks in a sequence on top of the robot. This should be accessible and direct enough to work with kids — and with no computer needed, just direct physical interaction with the system. For some reason, I think the extra level of abstraction from a screen to a robot is just an unnecessary obstacle at this level.

For an example run, see the Youtube video linked below:

The robot is shown in the picture below. Since I use the same base as a fast chaser robot, it uses the large wheels from a Technic kit I happened to have around, and a decent swiveling rear wheel. Using treads for the drive would probably have given me a bit more precision, but this is faster and simply feels more fun.

The inspiration for the programming method came from the boardgame Roborally, where you are dealt a set of card with meanings like “forward”, “back”, “turn right”, “turn around”, etc., and have to make your robot navigate a hazardous landscape to get to a destination. In my kindergarten robot, I simplified the programming to just forward, left, and right. And there are no conveyor belts, lasers, pits, or other hazards on the field.

The program steps available are the following (the color sensor can only distinguish a few primary colors):

• Black – start bit
• Blue – move forward
• Green – turn right
• Yellow – turn left
• Red – honk the horn (beep)
• White – stop bit

As a programming language, this is definitely not Turing-complete and nothing that you could program anything useful with. In particular, the programming model lacks decision statements, loops, and variables. However, it conveys what I feel is at the heart of programming: obtaining a goal by putting together a sequence of very small and simple steps. If you cannot grasp the idea of many small steps, you will never be able to program in any meaningful way.

User Testing

I have not tested this with a group of kids yet, but only with myself and my five-year-old son. So far, it seems to work. After some initial confusion, he is starting to get the idea of putting steps in order and having the robot execute them.

Here are some very simple missions that can be used to get the programming thinking going:

• Turn around a corner (of the fridge in this case)
• Move to hit a balloon lying on the floor some distance away
• Move in a square pattern to get back where it started

I am sure I will think of more as I go along. I will blog some more once I have more experience with the robot.

Details of the Program Memory

The major problem that had to be solved to build this robot was creating a working program memory. I have decided to call the program memory MEPROM, for “Mechanically Erasable Programmable Read-Only Memory”. It is a ROM, since the robot itself cannot change it.  Since you can both remove program steps and put in new steps using mechanical means (human hands, typically), it is definitely an M-EPROM.

The idea for the memory is simple: use Lego bricks to represent moves. It took a while to come up with the current form, as there are many ways to move colored bricks past a sensor.

Two of the program steps can be seen below: I used 3×1 standard lego bricks (which I borrowed from my son’s Legos, since  it seems very hard to find good kits containing this particular type of brick):

To move these programming blocks past the color sensor in order to read them, I used some classic Lego treads (which seems to have been in use since the late 1970s!). Today, these can be best obtained by buying the Mini Bulldozer kit (8259). The program steps are put on alternate tread elements:

The Mindstorms color sensor only sees six colors, as discussed above. Worse, it tends to see black in dark shadows. For this reason, it needs to read the bricks head-on to be reliable. I have tried in vain to make the system self-adjusting, so currently, the program start has to be manually aligning the start bit (black brick) in front of  the sensor. For the rest of the run, turning the driving motor 90 degrees reliably puts the next brick in place in front of the sensor.

The program turns out to be quite simple in the NXT programming environment. Turn program motor 90 degrees, sample the color sensor, and act accordingly using a “switch statement”:

When I was a PhD student in Uppsala back around 2000, we bought a pile of the Lego Mindstorms RCX kits, for use in real-time courses. Obviously, the students loved the opportunity to play with Lego (including the few females). What was less obvious and much more interesting was what happened when we brought in a bunch of children from a local kindergarten to visit — they really took a liking to our little yellow robots running around a classroom. They treated the robots as little animals, wondering what they were doing and why…

With that in mind, I decided to try to reprise this myself with my own son and his kindergarten friends. Last week, I took my robot kit with me and went to meet the kids.

My top-level goal with exercise was really to get the kids interested in technology as a future field of study and venture… you have to start early to counteract the prevalent tendency of people to want to be famous and work in media or something… Only time will tell if this had any effect at all (I doubt it).

In addition to the top-level goal, I wanted to communicate some about how things work. Make technology more understandable and less magical, and more accessible. In particular:

• Autonomy: the robot is not under remote control, it acts autonomously based on its programming.
• Sense-compute-actuate: the robot perceives the world and makes decisions that get sent to the motors.
• Limited sensors: the robot does not see like we do, it uses far simpler sensing.
• Programmability: the same physical setup can do a lot of different things just by switching to a different program.
• Concurrency: the robot can do several unrelated things at once.
• Stupidity: the robot is really dumb and just reacts very predictably to its environment.

I think these points can be brought across using an indirect approach. You cannot tell a four-year-old about programmability. But you can show that if you go to the robot’s control unit and press some buttons it does something different.

Note that some of the points above are artificial, in that you could build much smarter programs with more complex behaviors and less direct reactions. But that would make things more magical and “human-like”, which is not what I wanted to communicate.

Anyway, in the end I used two configurations of a driving base, with a few different programs.

The first configuration is shown below, using a color sensor pointing downwards and an ultrasound sensor pointing forward.It also had a shooter pointing forward (in this picture, the organic styling of the Lego Bionicle Zamor-derived shooter comes across very nicely, as well as the apparently quite special silver- and gold-colored balls I picked up on a sale).

First, I had this robot follow a black line on the floor. Putting the line down was fun, watching the robot follow the line did not work out as intended. Too hard to explain the algorithm for following the contrast between black and not-black. Adding the behavior to stop when the robot hit a red line was also too subtle.

Next, I tried to put the robot on a table with the program “stay within black lines”. Or rather “reverse and turn if you hit a black line”. Problem was that the table had a spotted pattern that included small specks of black, and the robot just kept doing avoidance maneuvers… so that failed. I still want to do a table-dancing robot, but I guess relying on an ultrasound sensor pointing downwards to detect the chasm at the edge will work better.

What was fun though was when I activated the ultrasound sensor and the shooter: the robot would crawl along the line, and if something got in the way, it would fire a ball. That was huge fun! All the kids crowded around to get the robot to shoot and collect the balls. Note for future attempts: dramatic actions are great!

It was even better when I set the robot to rove freely, using the ultrasound sensor to turn when something got in its way. This was an easy-to-understand behavior that the kids appreciated, putting feet and hands in the way of the robot to make it turn. The poor robot was often the center of a pile of kids that were all trying to make it turn.

I also tried a configuration using bumper sensors:

Here, the robot only had one program, to move forward until it hit something, and based on the side that hit, back up and turn to avoid hitting the same obstacle again. This was a very successful configuration. The kids started to play “don’t touch the robot” with it, having it drive between their legs or bending over the robot to make a bridge – but not touching it and making it turn.

Note that I did not bring any computer with me, all programs were preloaded on the NXT brick. Very handy, actually.

In summary, I had a great time doing this, and I will be back with new configurations and programs. Creating the programs was very quick and easy in the Mindstorms environment, proving the value of domain-specific programming.

The kids have hopefully learned that you can play with and control technology, and that you should not be too respectful. Maybe someone also picked up some of the ideas I presented at the start of this blog post.

Gender-Theoretical Notes

I do have to end with a slightly sad note.

It was surprising to me just how differently the boys and the girls reacted and behaved. I had assumed that kids would just be kids at this age (three to five years old), with no gender-related differences. It was very striking that this was not the case in reality. The boys just ran in and started playing, and very quite hard to get to listen to anything I had to say. The girls walked in and quietly waited for instructions, and took some warming-up before they would interact with the robot.

I cannot blame our kindergarten for this, they are definitely trying to avoid gender-based stereotypes.I guess it shows that fighting societal norms is just as hard as everyone says it is.

It was also interesting that the kids were surprised to see a parent build with Lego and have a great time. For some reason, that was not expected behavior from a dad. Sad too.

For my parental leave, I have just bought myself a Lego Mindstorm NXT 2.0 kit. It is not much fun for our youngest, who mostly gets a bit scared by a piece of Lego driving around making noises, but I hope to be able to use it to teach my older child (almost five) to program. Let’s see how that turns out. It looks hard to make the NXT environment provide the kind of Roborally-style programming blocks that I had hoped to create, as I cannot for some reason get a sufficiently custom icon onto custom blocks.

It also presented me with an opportunity to try some domain-specific high-level graphical programming. The programming environment provided for the NXT series of Mindstorms kits is based on LabView from National Instruments, and it really does seem to work. It even features parallel tasks, which I tried to use…

It turned out that it is not that easy to get it right. I was trying to have a few different tasks look at different sensors and steer the robot in different directions based on the sensor readings. However, this quickly turned into a literal deadlock: the robot just sat there, doing nothing, after the first time that my ultrasound sensor task tried to steer it. Failure.

The manual is quite silent on the tasking semantics, and there are no (or I have not yet found) shared variables, locks, or message-passing mechanisms to synchronize the tasks.

Looking for an answer on the web, I came across a nice tutorial on tasking:

• http://www.ortop.org/NXT_Tutorial/tasks.html

It is a video, and it ends with the note that the only reliable way to multitask in the NXT environment is to NOT try to control the same thing from multiple tasks. If I want to listen to multiple sensors, the only way is to create a loop with switching on inputs. I.e., manual polling. So much for using the natural language of tasks to handle naturally concurrent activities. At some point, I might figure out if I can construct it from tasks and data value transfers, but for now, I will just go with the flow and write (or rather draw) some polling loops.

Here is an example from the manual. Note that the top “beam” controls motor A, and the bottom controls motors B and C:

That last part about data wires is intriguing… but I will need to find some more reference materials.

Anyway, the way to express parallel tasks here is really quite neat. At least as long as they are dealing with completely separate aspects of the control of the robot.

If this had been a more hard-core environment, it would have been fun to put different tasks on the ARM7 and AVR processors that are inside the NXT 2.0 brick. Yes, it is a true multiprocessor, if very limited in capabilities. At http://mindstorms.lego.com/en-us/whatisnxt/default.aspx you have the specs. ARM7, 256 kB of FLASH, 64 kB of RAM, and the AVR has 512 bytes of RAM and 4 kB of FLASH. A nice real embedded machine!

In the picture below, we have the “Knight’s Castle” kit from 1984 (set #6073). I remember this as being a major kit at the time, one of the dream things to buy, and one of the biggest ones around. Compare it to the new Police Command Center (set #7743) from 2008. The small police station accompanying the large trailer is as high as the old castle, and the trailer as long as the castle is wide. It has about 25% more pieces that the old castle.

Also, note the old small vehicles in the front. Four Lego notches wide, unlike the six-wide police trailer. I remember the highway truck as being one of the magical models of my childhood.

To me, the most interesting aspect of comparing Lego sets is that it is a very good illustration of the economic growth that we have seen over the past 30 years. In relative terms, Lego sets have become far more affordable. Average income has certainly increased faster than the price of Lego, and my kids have quite a few more pieces to play with than I did at the same age. The nice thing with comparing Lego buying power with my childhood is that it is a product that is mostly “the same”, unlike comparisons involving electronics…

Lego Racers is a very buggy game. It takes almost no playing to get to a situation that is not covered by the rules, which would seem to indicate that play testing was not part of the design process. It seems that the designers made the same mistake as many programmers do: explore the obvious and primary path of execution, without thinking about what could go differently or go wrong. For something as simple as this games, that is simply sloppy. For something as complex as say, an operating system or telephone switch, it is more understandable.

This is where the age of software comes in: the more a particular piece of software has been used, the more different cases will have been explored, and the more errors, mistakes, and simple design holes will have been fixed. Steve Gibson at grc.com and Leo Laporte’s Security Now podcast often says that a completely new piece of software cannot be said to be secure, since there is no evidence to support that. It might well have been developed, in principle, with lots of security in mind. But until proven in the real world with real adversaries, there is no support for calling it secure.

It is also the case that no amount of internal testing can provide full coverage for all the cases that will appear in actual use at real customer and user sites. It is a matter of volume, but also a matter of sheer inspiration and creativity. Someone with a real problem to solve will use the tools they have in any way they can imagine… and your own developers cannot be expected have the same imagination as a user population many times their own size. That’s why beta testing and customer early access and iterative development are so important: only then will all the possible ways of using something be explored. It often turns out that users feel that your software can do something that you never quite thought it would — and that you sometimes have to insert specific limitations into the documention saying that “sorry, you cannot do that (for some not initially obvious but deep technical reason)”.

It also puts an interesting perspective on new creative software. Any new software entering a market will not have support for all possible users and all possible use cases. If there is an established older piece of software in the same domain, the new software will tend to solve fewer problems with fewer odd boundary conditions. The new software will typically be designed to solve some part of the problem better (or cheaper) than existing software (otherwise, its existence is hard to motivate), but initially it will not have the breadth and depth of coverage that a decade-old package will have. Simply from having been subject to users and their creativity and requirements for a long time.

That sounds like an awfully academic argument. One concrete example: the Linux operating system is now catching up to the old heavy-weights like Solaris and Aix in terms of scalability and robustness and features. Solaris still seems to scale better to really large number of cores and processes, but compared to where Linux used to be in the pre-2.6 kernel versions the situation is vastly improved. But doing multiprocessing like that well simply seems to take calendar time. More users does not help. You need the grind of having to transition through a few different generations of hardware of different types, and with different trends being judged important. Similary, the real-time operating systems now becoming SMP-aware will not scale as well as Linux does, at least not when judged from shared-memory flat designs. They do have their areas of merit, but they simply will not accumulate the same kind of shared-memory experience until a few years have passed. On the other hand, for the domains where predictability, control, and performance count, they are far superior to the general-purpose operating systems like Linux and Solaris, since they have been accumulating far more experience there. The same fact is evident in military history: time upon time, history shows how “seasoned troops” have a quality that no amount of quantity or training quality of fresh troops can match. Troops and equipment have to prove themselves in real battles with real enemies before their true quality can be assessed and their full potential realized.

To sum up, it seems to me that while cool new software is exciting to write and exciting to use, in heavy-duty real-world use, you want seasoned well-aged software that has been proven and tested in a wide variety of real-world caes over a significant period of time. Nothing beats experience, at least as long as the software system is maintained well so that it evolves in a way that keeps it open to future evolution. It does happen that old software gets worse with age… but there are many examples that are like fine wines and just tend to get better with time.

As a final aside,  we also own some other Lego-branded children’s games, and they, while not the most complex games in existence, at least are consistent and work without a problem. So the Lego brand itself needs not be avoided.