The Humane Programmer

A few days ago I had a conversation about the mobile space with a Romanian entrepreneur. Before that, I finally learned that Nokia E51 has a predictive text ability and I learned how to use it. Shortly after, I had an epiphany. This is the story of my surprise finding.

Read full story

Does Agile Work?

Short Answer:

The optimistic version: Yes. No.

The pessimistic version: No. Yes.

Phylosophical Answer:

If you become a Buddhist, will it end your suffering?

If you learn martial arts, will you be able to always defend yourself?

If you learn the rules of chess, will you become a grand master?

OpenAgile Romania 2010 - Register now

Read more

I have been involved in the last couple of years in a project that uses semantic technology, and I will try in this post to give a feeling about what it is and how it can be used.

I'm sure that we've all heard about semantics, in various contexts: language, parsers and compilers and lately related to web. However, until I've worked and thought of this concept, I didn't have a very good understanding of what semantics is in general.

According to wikipedia, semantics is "the study of meaning in communication", and actually you can substitute semantics with meaning in almost all contexts. Semantic analysis in grammar is looking at a sentence from the point of view of meaning. Semantic analysis in parsers and compilers is looking at the meaning of the programming language constructs put together. The idea of semantic web is to add machine-readable and inferable meaning.

The world we're living in contains semantics, a semantics built by three main characteristics: things, relations between them and actions that you can apply to a thing. Take the following example:

My(relation) name(thing) is(relation) Alex(thing) and(relation) I(thing) write(action) a blog entry(thing) while(relation) sitting(relation) on(relation) a chair(thing).

And that's basically what semantics is.

The problem with semantics is that we have a great tool for processing it, one that machines lack: our brains. The brain evolved along with us in this environment, and it developed special powers that allow it to process and understand the semantics of this world. Machines have no idea about things that to us are very simple to understand, like the fact that "Alex" is the name of a person, or what "writing" means. Therefore, in order to make machines process such information, you need to explain everything to them, or to allow them to evolve. The first solution is very time consuming, while the second may give very strange results as you cannot control the evolution but only the environment and the rules for it.

Despite these problems, people have found ways to build machine-readable and inferable semantics, some of them very familiar to any of us.
The web is the first example that comes to mind. It has a few types of objects (page, image, sound, object), a single type of relation - links to - and one action - follow link. A wiki is the same thing, only that there are rules for the free text content of the page - you know that in wikipedia the page http://en.wikipedia.org/wiki/Semantics discusses "Semantics" and you have additional actions: edit, create, delete.

Video games build their own semantics. In a FPS, you can move, rotate, shoot, and each object may have additional attributes: weight, health, stamina etc. A RPG builds upon a more complex semantics that describes not only your current state but also your possibilities of evolution - nothing more than a kind of metadata. This is only natural, since a game tries to define a world with meaning for the player.

The easiest built semantics that you can find in games is in MUDs. Due to them being text-based, there's plenty of freedom in defining objects, relations and actions very quickly, and obtaining a world where anybody can play and try. This advantage of quick prototyping allows for a lot of user generated content, even if the community is rather small.

Social networks also build semantics. The main object is a person, with various other collateral objects: company, group, awards, schools etc. The main relation is "know": "Person X knows person Y". The actions are add and remove. The problem is that they are limited, because other elements are missing from the picture: trusts, prefers, likes etc.


If we think about technology from this perspective, we realize that the tendency is to add and to enrich the semantics of the applications. However, this is no easy treat. Trying to do too much can only bring failure. It's better to start small, and that's why technologies like the web, wikis, social networks are so successful, despite having very limited semantics.

However, it is clear that adding semantics - new types of objects, of relations and of actions - can significantly enrich the user experience, if done properly. That's why I think that in the long run, the future is semantic.

So everyone will know how to read?
So everyone will know how to write?
So everyone will know how to compute?
So many people will read what's written by certain authors?
So everyone will be able to talk to everybody else?
So everyone will be able to see/hear movies/music made by some people?
So everyone will be able to tape the movies and music from radio/TV?
So everyone will be able to make movies?
So everyone will be able to make music?
So everyone will be able to publish text, images or sound?
So everyone will be able to transfer text, music, videos to a lot of people?
So everyone will be able to publish anything and everyone will be able to see/hear/read/get a copy?
So everyone will be able to do all that without anybody knowing it?

In my many long days (and a few nights) of professional software development, I've had quite a few WTF moments. I'm sure you all know what I'm talking about - that moment when you look to somebody's else piece of code and you realize that he/she/them used the worst possible solution to the problem in hand, one of such devious ingenuity that you could never have created it, and therefore builds into you feelings of both fear and nervous laughter at the same time.

Every time I shared such a solution with one of my fellow developers who have the appropriate stature to understand the point, we were both thinking: this is nonsense! This violates the most basic mechanics of software development! This violates our common sense!

But common sense is a very elusive concept. How do you express it, how do you explain it to those people violating it in such an unimaginable manner? How can they learn it, use it, teach others? How did you get into its possession?

Hard questions, with no answer. Until now, when I have finally developed a theory of common sense. It may be proven right or wrong, but at least it's a theory that is, I believe, worth exploring.

My theory can be summarized as following:

1. We start learning programming because we are attracted by it
2. We continue to learn and read more and more because it's more and more interesting
3. The knowledge pieces we gather connect to each other
4. The connections solidify in patterns
5. The patterns become the second nature and the common sense is born

The first empirical evidence of this theory was available to me a long time ago. I realized that the programmers that had common sense also touched a wide area of topics related to software and computers. I could discuss with them about all kinds of things, from the architecture of Linux kernel to functional languages, including topics from telecommunications, networking, assembly, electrical circuits or even unrelated topics like quantum physics, astronomy or genetics. One does not read such topics if forced, so they must have liked it.
On the other hand, the programmers that did those mistakes could not discuss any interesting topic and had almost zero culture in programming. Most of them had learned one programming language and one platform (.NET in my case) and were programming for money.

However, knowledge is not enough. I have also met people who knew quite some things about programming but had a crippled common sense. They weren't as bad as the WTF cases, but they had some repeated glitches. The brain plays a major factor here; the way it filters and organizes information can create very good systems or good enough systems of patterns.

My theory is also supported by some scientific research. A Scientific American article talks about The Expert Mind.


A chess master expert. Photo licensed under a CC license by mysza831

How did he play so well, so quickly? And how far ahead could he calculate under such constraints? "I see only one move ahead," Capablanca is said to have answered, "but it is always the correct one."

The article shows that the mind of an expert is formed by learning, practicing and working for at least 10 years. This helps forming a specialized type of memory that the expert uses for his work and that makes a very big difference between him or her and a beginner.

On the other hand, software development studies have shown that a good programmer is at least an order of magnitude better than a mediocre programmer. This isn't so unexpected when you know about the expert mind, is it?

In programming specifically, many studies have shown order of magnitude differences in the quality of the programs written, the sizes of the programs written, and the productivity of the programmers. The original study that showed huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968). They studied professional programmers with an average of 7 years' experience and found that the ratio of initial coding time between the best and worst programmers was about 20:1; the ratio of debugging times over 25:1; of program sizes 5:1; and of program execution speed about 10:1. They found no relationship between a programmer's amount of experience and code quality or productivity. (Code Complete, page 548)

And the final piece of evidence that I can provide at this point is a study on the ability of people to learn how to program. The study was initially called in a very suggestive manner The camel has two humps because it shows a large difference between the people who can learn programming and those who can't.


Expert pattern built by an expert. CC licensed work by Vilhelm Sjostrom.

Abstract: All teachers of programming find that their results display a 'double hump'. It is as if there are two populations: those who can, and those who cannot, each with its own independent bell curve. Almost all research into programming teaching and learning have concentrated on teaching: change the language, change the application area, use an IDE and work on motivation. None of it works, and the double hump persists. We have a test which picks out the population that can program, before the course begins. We can pick apart the double hump. You probably don't believe this, but you will after you hear the talk. We don't know exactly how/why it works, but we have some good theories.

In conclusion, my take on common sense in software development is:

• Some people like programming and some don't - the camel has two humps
• The people that like programming study a lot, forming their expert mind
• The expert mind makes an order of magnitude difference
• What we call common sense are the patterns found in the expert mind

Until next time, may the common sense become more common!