CouchDB: A Parallel Program in a Parallel Language

I just listened to another Floss Weekly show, Number 36 where they interviewed Jan Lehnard of the CouchDB project. CouchDB is very interesting, in that it is a database designed for replication, redundancy, and thus massive parallelism. It was initially written by Damien Katz on his own, but now it is an Apache Foundation project sponsored by IBM. The most interesting thing is that Damien decided in 2006 to rewrite the C++ prototype he had in Erlang, and did so in just a few months if I understood my Erlang friends right. So here we have a really good parallel program written in a true parallel language.

It is clear that Erlang in this case is a very good choice, and Damien points out that he is a much productive programmer even after just a few months with the pretty different syntax. Even more interesting is the comment by Jan in the interview with FLOSS Weekly which can be summarized as “yeah, the syntax is a bit different, but once you understand it it is was really powerful”. It shows the importance people attach to how code looks (what I call the “tyranny of Syntax”), rather than the core semantics. But also validates that a good programmer can pick up any language quickly, and that good concise coding and semantics natural to the problem domain are worth the small effort of learning a new syntax.

The performance characteristic of using Erlang and its associated “put everything in a thread” style is also interesting: CouchDB is scales to hundreds of thousands of simultaneous requests with no real impact of reply times for any of the requests. The reply latency is fairly flat, which means that it is a bit longer than a typical solution when few requests are coming in, but there is also no “knee” above which latencies get intolerable. So we see the tradeoff of latency versus throughput again, and CouchDB is clearly on the throughput side.

I think FLOSS Weekly should really interview some of the people behind Erlang, like Joe Armstrong.