More on Programming

My blog post from last week, “Programming for Everyone, Everyone’s a Programmer?,” gave rise to some good comments from friends and colleagues on the matter of programming. Some of the points are worth addressing in a continuation of the discussion.  To get the starting point, please read the previous post first.

Maintenance is Important

One very important aspect of professional programming and software engineering that I glossed over in a few words is the matter of maintenance. Maintain and extending existing code is one of the most common tasks in the software world – very little software is written from scratch and thrown into the world and never changed again. Most software has a legacy, and needs to be maintained and updated and patched and bug fixed…

Such work is incredibly important and valuable, but also very hard to learn on your own, and not really supported by “easy programming” tools. Writing code is easy, reading code is hard and arguably a lot more important than being able to write it.  Writing code that is easy to read is the hardest of all, and is the hallmark of a true professional.

No matter the notation or graphical system used to write code, making it clear and well-structured is a skill that no tool can help you with. In the end, if a problem is sufficiently complex to be non-trivial, the solution is going to be non-trivial and involve explicit decisions on how to do things.

Note that a good language makes it harder to write difficult code and easier to write good code, and it is clearly the case there are many less-than-optimal languages being used today that definitely make the coding part harder than it needs to be (like the “write-only language” perl). On the other hand, a good language in hands of a bad coder/programmer can result in a mess as well. As a lecturer I once had said, “a good programmer can write FORTRAN in any language” – and proceeded to impose imperative loops on Prolog. Impressive, in some odd way.

Training and Practice

Programming no matter how you view it is a human task, and humans get better with practice and training. You need both – training from someone who already knows how to do something makes your learning a lot faster than trying to do it on your own. And just training without actual practice produces rather shallow results. Which means that in order to be a professional efficient developer, you need both the training and continued practice, which once again separates the hobbyist from the professional.

You need the training to learn from the past mistakes and successes from others. If we take the example of a captain of a ship. That person knows that they have to consider the stability of a ship when loading and ballasting it so that it keeps on the right keel. This is hopefully not something learnt from personal experience, but rather learnt from history and what happened before. Formal training is how we carry these learnt lessons down through the ages, and is a key part of why it pays to have a big brain rather than big muscles in an evolutionary setting.

Teaching Learning Machines

Another argument is that machine learning and artificial intelligence makes programming as we know it obsolete – who needs programmers when you can just teach the machine with some example data? I do not think that is true since the domain where ML and AI can be applied is limited. And even if we do assume this is true and everything can be replaced with ML, the outcome is simply that the nature of programming changes. From hand-crafting algorithms to process data, to building black-box functions from the data itself.

The programmer becomes a teacher for the machine.

As anyone who has worked as a teacher or trainer knows, it is not a trivial thing to do. Being a good teacher takes time to learn, and not everyone can do it. It is still a professional function that benefits from training and experience. You cannot just throw a learning algorithm at data and expect sane results, there is a lot of skill involved in knowing what to do and how to handle the data for good reliable results. Determining that a solution makes sense and its applicability and limits of applicability requires skill and a sense for what is going on.

A hobbyist can definitely put something together that looks like it is working, but whether that actually generalizes or makes sense in a bigger context is another question. Maintaining and improving an ML solution over time is a job that looks very similar to old-fashioned software engineering. When it comes time to clean up the data set and apply laws and regulations governing private data, things definitely are more complicated and requires specialist skills (and maybe an extra degree in law).

Summary

Programming, no matter how you do it, is definitely more than just throwing code together. It requires training and practice to do professionally, and it is a professional exercise just like all other specializations.

A teacher I once had stated that if you start with a working piece of code, making it into production-quality software takes 3x the effort. To make it into a software product, multiply by another 3x since you have to factor in documentation, support, road map plans, integrations, and all the other fun stuff that makes a software product work in the real world. This was in the 1990s, and maybe some of these multipliers have decreased a bit since then thanks to the Internet and more efficient tools and methods… but still, it is definitely a long way from “working” to “sellable”.

Better coding tools and easier languages make “working” easier, but adds very little to the real work of making quality software that users trust and pay money for. Which is what the game is about in the end.