Software Development

In code reads #8, Scott Rosenberg discusses different styles of development. Mainly, the cathedral method where a select group of programmers develop software until the time is right to release it. Or the bazaar method where anyone may contribute (although the core may be either public or private). As Scott correctly points out, both methods can be found in both closed and open source projects. And most open source projects must start as a cathedral because people usually aren't interested in something that's likely to become vapourware. There are plenty of failed projects out there. If there's something to work with, then expanding on those ideas is so much easier.

This last part is what I want to discuss. I've noted before that programmers have a hard time creating something truly new. But what they do excel at is combining and tweaking existing tools, code and media. It's not just in programming either. It seems to be a natural human attribute.

So what do I mean by 'new' and 'combining'? Creating something new is rare. It hardly ever happens. But it does happen. Usually, it happens by accident. And for good reason. How can you think of something without anything to build on? Well, that's why we have imagination, dreams and intuition. It's where we don't have ALL the information, but have some inkling of where we want to go. This missing information sometimes reveals new sources of ideas and concepts just by sitting down and thinking about it.

If you want to know what 'new ideas' are like, then look at the work of Arthur C. Clarke, Stanley Kubrick, Isaac Asimov and Jules Verne. They all have the same thing in common. They all did work in science fiction. But just the fact of sitting down and thinking about how things would be different in remote or alien world, or even what's in store for humankind in the future, makes that you will likely come up with something different.

What I find is that too often, we start with a premise and we have to fulfill that premise. We have a film to shoot. We have a book to write. We have a piece of software to complete. Things like that. This narrows our concentration directly to what is available now. There aren't many occasions where we get the chance to explore new ideas. And the longer we deal with something, the more difficult it becomes to think in a new way.

Case in point, since I started programming, I've always used function calls, gotos, switch statements, while and for loops, if statements and all sorts of other hacks to tell the computer where to get its next instruction from. In my world, I could not imagine writing software without them. What I didn't realise at the time and had quite a problem accepting was that all these things were illusions. They were not necessary. I'd been programming in one specific universe where control statements were king. Yet, they are completely redundant when it comes to creating software. Although it's not new in the general sense, it was new for me. It wasn't based on anything I could build on. Where would one even begin to think that control statements are redundant? Most people would immediately think of what you would use to replace them. There's no need.

Again, I think there has to be something to trigger a revelation. A catalyst for ideas. And this is where accidental discoveries come into play. What I've found is that when looking for new ideas, the question is more important than the answer. Normally, when you have a problem or question to solve, you can try out different possible solutions. Yes, you can find new ideas this way by accident, but you're not looking for them. If you are looking for new ideas, you have to change the question when the solution you try doesn't work. I've found that the new idea and concept will come naturally if you have the proper question. And when a possible solution ends up not working, you gain insight as to why it doesn't work. This is new information that not very many people will have. In other words, you have to place yourself in a more likely position to discover new ideas.

Once you have a good idea, it's time to turn it into reality. In this case, it doesn't matter if you're going the closed or open source route, you have to start with the cathedral scenario. Only you and maybe a few others will understand and appreciate what you're doing. Top business people always tell me that ideas are a dime a dozen. What interest them are working concepts and products.

Having something tangible means that the public at large who is more comfortable in tweaking and modifying existing assets will now begin to see the promise of your ideas. A virtual limitless number of minds looking at your product means an exponential increase in the possible uses of your ideas. This is where you get into the bazaar model. And the more people who use your ideas, the likelier it will become for accidental discoveries. However, once those are tapped out, you again have to fight what people start accepting as absolutes.

Coming up with new and useful ideas is just one part of the battle in producing something successful (by whatever definition you use). Another part is giving programmers and the users a way to take part in something bigger than themselves, yet still feel like they have some control. Too often, I submit a contribution or try and discuss a useful feature and it gets torn down immediately. All these projects are now dead or inactive. It's not because they didn't accept my contributions. It's because they hardly accepted any contributions. This is no longer a bazaar. It's not even a cathedral. It's more like a hostage situation.

So just allowing people to contribute is a big plus. It's bad enough that some projects deny most contributions. If you don't shoot yourself in the foot like this, you're probably ahead of the game. But here's a situation where I think the open source community really sucks at. There are projects that allow contributions and this is fine and all. But there's hardly ever any interoperability standard or communication mechanism. In a bazaar, you need a way to interact. With programming, this applies not only to programmers, but also to software.

With all projects, no matter how difficult coming up with something new may be, you have to create the cathedral. Otherwise, you end up with nothing. It's a matter of necessity. But for the bazaar model to take place, you have to build the actual bazaar where both people and software can interact. While there are great tools such as forums, mailing lists and software repositories, I find that there's no equivalent for software interaction. While I can argue all I want that new and original ideas are really tough to come by, I'm betting that any project that enabled software interoperability will have a huge advantage over all other projects. Strangely enough, this is much easier than coming up with new ideas. It's what humankind is best at doing. I think it's because most like to buy and sell, but not build the actual marketplace or system of commerce.

Can I dare say that users are a dime a dozen, but architects are rare? Is this a valid assertion? I think it is and I further think it speaks volume as to what programmers actually are. Neither do I think this is a jab. I think it's a trait to be promoted. We need more users (as programmers) than architects. But we need at least one architect. Nowhere in computing can you learn to be an architect. Although their numbers are low, we do need them and there should be a way to learn to be one.