The first movie is just the robot driving around using its MEPROM programming unit. I think this movie is slightly clearer than the old one I posted.

Next, here is how a program loop can be created in this system. By removing the white end marker from the program, the motor will just keep spinning the program until it spots a program step. This is a very mechanical infinite loop.

Finally, we have the emergency stop bumper that was added to the robot to make it stop when it hits something. It does not take effect immediately due to the latencies involved in the NXT programming environment, but at least the robot does not proceed to the next programming step.

It turned out that trying to show a group of ten impatient kids that the robot read instruction codes and moved accordingly failed. Using a chaser program, where the robot drove around waiting for something to show up close in front of it and then drove full speed forward to try to hit it, was much easier to appreciate.  Classics like driving around avoiding to hit walls was also just as much fun as ever. Like my previous attempts, the kids piled up on the robot and tried to make it turn the way they wanted.

Clearly, physical exercise is easier to sell than cerebral exercise.

That said, I did manage to get some more thoughtful kids interested in getting the robot to go particular places. The setup I used was to put the robot on a starting square marked on the floor, and then putting a red bucket out on the floor some distance away. The kids were asked to help me program the robot to either hit the bucket or drive around it. I explained what the robot did as taking steps forward and turning, and actually the approximately 50 cm long runs it did for each blue programming tile was fairly similar to a serious step by a five-year-old (some kids instead started tip-toeing forward, which did not quite provide the correct stride).

As a safety device and to retain a semblance on intelligence on the part of the robot, I equipped it with a bumper in front. As soon as the bumper is hit, the robot is supposed to say “stop” and stop all activities.

The problems we tried were:

• Put the bucket down in front and try to hit it, needing three forward to get there
• Drive around said bucket, which required a program like blue, green, blue, yellow, blue, blue, yellow, blue, green (forward, right, forward, left, forward, forward, left, forward, right).
• Leave the base and drive to the back instead of front.

I sat down with a group of three kids who really got into this part (even with ten other kids running around playing cops and robbers at a very high level of intensity). They did get the idea and did propose steps towards the solution. However, it is clear that at least initially; one cannot expect them to plan out an entire program in their heads.

Rather, we ended up in a style quite familiar to anyone learning a new programming language or taking their first programming course. Try one step, see where it takes us, add another step or two to the program, and try again. Very much how I myself solve programming and scripting problems where I interact with a complex and possibly unpredictable environment. In this way, we did manage to solve some problems, and I do think the kids understood what was going on.

To me, I feel the idea of the Lego-programmable robot as a preschool-level programming course is at least partially proven. As in most other educational ventures, what is needed is a good overall teaching environment. Next time, I will have to try this with a small group of kids, just after they have had a snack so that they are alert and fresh. I will also make sure to break up the hard work with some simple play, since that is needed to keep the kids from being bored.

Back in the late 1980s, in what seems to have been a golden era of software innovation, there were a number of companies trying to sell graphical programming environments for the graphics-oriented Macintosh computer. Makes sense, right? I remember that I did not think so at the time, and the reason was that things tended to look more complex in a visual system than when expressed as plain text. I think one of the examples I saw was Prograph, where unwieldy flow charts replaced nice and dense if-then statements.

Today, twenty years later, it is hard to dispute the success of model-driven and graphical environments like MatLab, LabView, and others. Dataflow is very good paradigm for many things in signal processing, and it fits very nicely with graphical representations. UML statecharts do express state machines in a pretty readable manner which is far superior to any particular code encoding of the same information.

However, I think the key here is that the languages are domain-specific to a large extent. General-purpose programming is still not really feasible in a graphical way, and I doubt it will ever be. You also need to “go with the flow” in these systems to get full benefit from them. If you start to think in imperative coding terms, you tend to quickly get lost… Here is a personal example that a friend (who will remain anonymous to protect the guilty) and I cooked up in NXT, trying to code an algorithm to guide a robot along a line. The picture both contains the graphical representation and the approximately equivalent imperative-style code.

All the little red boxes are value reads, writes, and manipulations. It is kind of elegant that you can pass values on from one block to the next, but overall the graphical representation must be said to be totally unreadable. The logic was pretty good, actually, but it did not do things like you are supposed to do in Mindstorms. Rather than a reactive “act wait act” cycle, we tried to code complex loops.

I guess this is an example of the old saying that a good (or determined) programmer can write FORTRAN in any programming language. As an aside, I recall a Russian-schooled professor I had as an undergraduate who managed to abuse Prolog to create a deterministic loop from its backtracking system. Ugly indeed, but also quite an impressive feat of semantic wrestling.

The Mindstorm-style solution to following a line was found by a bit of Googling, and for the robot that I built a fairly (not entirely, though) robust program looked like this. Credit to http://www.nxtprograms.com/line_follower/steps.html for the original program that I had to modify just a bit to work with Mindstorms NXT 2.0.

In a sense, this program is bad since it obviously relies on non-functional properties of the NXT environment which are not semantically specified (i.e., the speed of sensing and how to drive the motors the fastest). However, the sense-act loop with no state saving is the right way.

Obviously, other graphical environments will have other strengths and weaknesses, but I think that this is a nice illustration that the typical constructs of general-purpose programming languages are poor fits for graphical visualization. Variables are simply not very visual in their nature…

As discussed in my previous blog post about Kindergarten robots, I wanted to see if I can teach kids the core idea of programming. This project has now progressed to the point that I have a working prototype of a programmable robot.

Essentially, the robot is programmed by putting colored Lego bricks in a sequence on top of the robot. This should be accessible and direct enough to work with kids — and with no computer needed, just direct physical interaction with the system. For some reason, I think the extra level of abstraction from a screen to a robot is just an unnecessary obstacle at this level.

For an example run, see the Youtube video linked below:

The robot is shown in the picture below. Since I use the same base as a fast chaser robot, it uses the large wheels from a Technic kit I happened to have around, and a decent swiveling rear wheel. Using treads for the drive would probably have given me a bit more precision, but this is faster and simply feels more fun.

The inspiration for the programming method came from the boardgame Roborally, where you are dealt a set of card with meanings like “forward”, “back”, “turn right”, “turn around”, etc., and have to make your robot navigate a hazardous landscape to get to a destination. In my kindergarten robot, I simplified the programming to just forward, left, and right. And there are no conveyor belts, lasers, pits, or other hazards on the field.

The program steps available are the following (the color sensor can only distinguish a few primary colors):

• Black – start bit
• Blue – move forward
• Green – turn right
• Yellow – turn left
• Red – honk the horn (beep)
• White – stop bit

As a programming language, this is definitely not Turing-complete and nothing that you could program anything useful with. In particular, the programming model lacks decision statements, loops, and variables. However, it conveys what I feel is at the heart of programming: obtaining a goal by putting together a sequence of very small and simple steps. If you cannot grasp the idea of many small steps, you will never be able to program in any meaningful way.

User Testing

I have not tested this with a group of kids yet, but only with myself and my five-year-old son. So far, it seems to work. After some initial confusion, he is starting to get the idea of putting steps in order and having the robot execute them.

Here are some very simple missions that can be used to get the programming thinking going:

• Turn around a corner (of the fridge in this case)
• Move to hit a balloon lying on the floor some distance away
• Move in a square pattern to get back where it started

I am sure I will think of more as I go along. I will blog some more once I have more experience with the robot.

Details of the Program Memory

The major problem that had to be solved to build this robot was creating a working program memory. I have decided to call the program memory MEPROM, for “Mechanically Erasable Programmable Read-Only Memory”. It is a ROM, since the robot itself cannot change it.  Since you can both remove program steps and put in new steps using mechanical means (human hands, typically), it is definitely an M-EPROM.

The idea for the memory is simple: use Lego bricks to represent moves. It took a while to come up with the current form, as there are many ways to move colored bricks past a sensor.

Two of the program steps can be seen below: I used 3×1 standard lego bricks (which I borrowed from my son’s Legos, since  it seems very hard to find good kits containing this particular type of brick):

To move these programming blocks past the color sensor in order to read them, I used some classic Lego treads (which seems to have been in use since the late 1970s!). Today, these can be best obtained by buying the Mini Bulldozer kit (8259). The program steps are put on alternate tread elements:

The Mindstorms color sensor only sees six colors, as discussed above. Worse, it tends to see black in dark shadows. For this reason, it needs to read the bricks head-on to be reliable. I have tried in vain to make the system self-adjusting, so currently, the program start has to be manually aligning the start bit (black brick) in front of  the sensor. For the rest of the run, turning the driving motor 90 degrees reliably puts the next brick in place in front of the sensor.

The program turns out to be quite simple in the NXT programming environment. Turn program motor 90 degrees, sample the color sensor, and act accordingly using a “switch statement”:

When I was a PhD student in Uppsala back around 2000, we bought a pile of the Lego Mindstorms RCX kits, for use in real-time courses. Obviously, the students loved the opportunity to play with Lego (including the few females). What was less obvious and much more interesting was what happened when we brought in a bunch of children from a local kindergarten to visit — they really took a liking to our little yellow robots running around a classroom. They treated the robots as little animals, wondering what they were doing and why…

With that in mind, I decided to try to reprise this myself with my own son and his kindergarten friends. Last week, I took my robot kit with me and went to meet the kids.

My top-level goal with exercise was really to get the kids interested in technology as a future field of study and venture… you have to start early to counteract the prevalent tendency of people to want to be famous and work in media or something… Only time will tell if this had any effect at all (I doubt it).

In addition to the top-level goal, I wanted to communicate some about how things work. Make technology more understandable and less magical, and more accessible. In particular:

• Autonomy: the robot is not under remote control, it acts autonomously based on its programming.
• Sense-compute-actuate: the robot perceives the world and makes decisions that get sent to the motors.
• Limited sensors: the robot does not see like we do, it uses far simpler sensing.
• Programmability: the same physical setup can do a lot of different things just by switching to a different program.
• Concurrency: the robot can do several unrelated things at once.
• Stupidity: the robot is really dumb and just reacts very predictably to its environment.

I think these points can be brought across using an indirect approach. You cannot tell a four-year-old about programmability. But you can show that if you go to the robot’s control unit and press some buttons it does something different.

Note that some of the points above are artificial, in that you could build much smarter programs with more complex behaviors and less direct reactions. But that would make things more magical and “human-like”, which is not what I wanted to communicate.

Anyway, in the end I used two configurations of a driving base, with a few different programs.

The first configuration is shown below, using a color sensor pointing downwards and an ultrasound sensor pointing forward.It also had a shooter pointing forward (in this picture, the organic styling of the Lego Bionicle Zamor-derived shooter comes across very nicely, as well as the apparently quite special silver- and gold-colored balls I picked up on a sale).

First, I had this robot follow a black line on the floor. Putting the line down was fun, watching the robot follow the line did not work out as intended. Too hard to explain the algorithm for following the contrast between black and not-black. Adding the behavior to stop when the robot hit a red line was also too subtle.

Next, I tried to put the robot on a table with the program “stay within black lines”. Or rather “reverse and turn if you hit a black line”. Problem was that the table had a spotted pattern that included small specks of black, and the robot just kept doing avoidance maneuvers… so that failed. I still want to do a table-dancing robot, but I guess relying on an ultrasound sensor pointing downwards to detect the chasm at the edge will work better.

What was fun though was when I activated the ultrasound sensor and the shooter: the robot would crawl along the line, and if something got in the way, it would fire a ball. That was huge fun! All the kids crowded around to get the robot to shoot and collect the balls. Note for future attempts: dramatic actions are great!

It was even better when I set the robot to rove freely, using the ultrasound sensor to turn when something got in its way. This was an easy-to-understand behavior that the kids appreciated, putting feet and hands in the way of the robot to make it turn. The poor robot was often the center of a pile of kids that were all trying to make it turn.

I also tried a configuration using bumper sensors:

Here, the robot only had one program, to move forward until it hit something, and based on the side that hit, back up and turn to avoid hitting the same obstacle again. This was a very successful configuration. The kids started to play “don’t touch the robot” with it, having it drive between their legs or bending over the robot to make a bridge – but not touching it and making it turn.

Note that I did not bring any computer with me, all programs were preloaded on the NXT brick. Very handy, actually.

In summary, I had a great time doing this, and I will be back with new configurations and programs. Creating the programs was very quick and easy in the Mindstorms environment, proving the value of domain-specific programming.

The kids have hopefully learned that you can play with and control technology, and that you should not be too respectful. Maybe someone also picked up some of the ideas I presented at the start of this blog post.

Gender-Theoretical Notes

I do have to end with a slightly sad note.

It was surprising to me just how differently the boys and the girls reacted and behaved. I had assumed that kids would just be kids at this age (three to five years old), with no gender-related differences. It was very striking that this was not the case in reality. The boys just ran in and started playing, and very quite hard to get to listen to anything I had to say. The girls walked in and quietly waited for instructions, and took some warming-up before they would interact with the robot.

I cannot blame our kindergarten for this, they are definitely trying to avoid gender-based stereotypes.I guess it shows that fighting societal norms is just as hard as everyone says it is.

It was also interesting that the kids were surprised to see a parent build with Lego and have a great time. For some reason, that was not expected behavior from a dad. Sad too.

For my parental leave, I have just bought myself a Lego Mindstorm NXT 2.0 kit. It is not much fun for our youngest, who mostly gets a bit scared by a piece of Lego driving around making noises, but I hope to be able to use it to teach my older child (almost five) to program. Let’s see how that turns out. It looks hard to make the NXT environment provide the kind of Roborally-style programming blocks that I had hoped to create, as I cannot for some reason get a sufficiently custom icon onto custom blocks.

It also presented me with an opportunity to try some domain-specific high-level graphical programming. The programming environment provided for the NXT series of Mindstorms kits is based on LabView from National Instruments, and it really does seem to work. It even features parallel tasks, which I tried to use…

It turned out that it is not that easy to get it right. I was trying to have a few different tasks look at different sensors and steer the robot in different directions based on the sensor readings. However, this quickly turned into a literal deadlock: the robot just sat there, doing nothing, after the first time that my ultrasound sensor task tried to steer it. Failure.

The manual is quite silent on the tasking semantics, and there are no (or I have not yet found) shared variables, locks, or message-passing mechanisms to synchronize the tasks.

Looking for an answer on the web, I came across a nice tutorial on tasking:

• http://www.ortop.org/NXT_Tutorial/tasks.html

It is a video, and it ends with the note that the only reliable way to multitask in the NXT environment is to NOT try to control the same thing from multiple tasks. If I want to listen to multiple sensors, the only way is to create a loop with switching on inputs. I.e., manual polling. So much for using the natural language of tasks to handle naturally concurrent activities. At some point, I might figure out if I can construct it from tasks and data value transfers, but for now, I will just go with the flow and write (or rather draw) some polling loops.

Here is an example from the manual. Note that the top “beam” controls motor A, and the bottom controls motors B and C:

That last part about data wires is intriguing… but I will need to find some more reference materials.

Anyway, the way to express parallel tasks here is really quite neat. At least as long as they are dealing with completely separate aspects of the control of the robot.

If this had been a more hard-core environment, it would have been fun to put different tasks on the ARM7 and AVR processors that are inside the NXT 2.0 brick. Yes, it is a true multiprocessor, if very limited in capabilities. At http://mindstorms.lego.com/en-us/whatisnxt/default.aspx you have the specs. ARM7, 256 kB of FLASH, 64 kB of RAM, and the AVR has 512 bytes of RAM and 4 kB of FLASH. A nice real embedded machine!