Software Development

The key to writing large software is really to write everything as a library. Applications are meant to be used. Its internals should be the same. That way, there is a clear hierarchy all the way from library calls to the main visual interface. The problem is how do you go from tight code to having an extensible framework? It's not easy.

First off, programming languages have shoddy facilities for communication. They really do. Messages in OO are most definitely NOT messages. Real messages are what you get over TCP/IP or over serial cable. OO messages are some weird concoction of looped back execution path, data and objects. This is not the best way to create software. To visualise this problem, imagine that you drove all over town to do all sorts of errands zigzagging to and fro. While you're driving about, also imagine that you are extending a rope from your car that will trace out exactly where you've driven. When you are done your errands, you must then retrace your path in the opposite direction until you've reclaimed all the rope left behind. The scenario I described at the beginning of this article, where we want to add extra functionality, is akin to splicing the rope at a certain point and making more stops until you return to the point where the splice took place. You'd have to do this in multiple locations. And some of the original destinations may also need to move around. That's essentially what programming is about when it comes to adding functionality. The only problem is that we don't consider the rope even though we often get hung by it. In programming, the rope (or execution path) is not visible and not considered at all in many cases.

The reason you want a custom framework is so that you can keep adding onto your code as you would a road network or a city. Over time, it gets larger, but the original functionality doesn't get destroyed. Ok, that analogy doesn't hold up in certain respects, but the point is still valid. We need something to build on rather than something we keep modifying. This notion of modifying existing code is really bad for the soul. I've done it that way for many years and it simply doesn't pay off.

Once you realise that you need a framework, you need to decide how it is to be built. I've talked at length about this before. But it's the challenge that will make you a good programmer into an exceptional one. Data is your friend. So I'll just point out a few tips.

1. Don't be afraid of helper classes. These are classes that operate on a certain kind of data. So you pass data around and NOT the class that operates on it. Well, you can pass that around too just as long as you keep track of who owns the helper class. Each helper class operates one ONE specific kind of data. So if the data format changes, you create a NEW helper class and leave the old one there. If the old data format isn't used anymore, then you can remove the old helper class.

2. Continuing on point #1, pass data around when it comes to interacting with external software. Even internally, use ONLY data when connecting major systems together. Data would be put into the other system's queue to be processed at a later time.

3. When your software gets large, split them up into systems where they each get a turn or where each one runs on its own core. Not only will it make your code easier to maintain and extensible, but it'll make it distributed if the need arises.

4. Use custom resource managers. Make these into a system if necessary. That way all your data and resources are taken care of when multiple events from different systems take place. You need one more thing to make this work which lead to point #5.

5. Allow for the use pluggable interfaces. So if you need a resource to be reset if a context is lost, then allow for an interface to be implemented for that use that can be plugged in.

6. Make sure you have a clear hierarchy for your objects. Objects at a higher level can only call objects at a lower level. They cannot call objects at the same or higher level. This will not only eliminate recursion problems that so often happen with event driven software, but will make your software extremely easy to maintain. If a lower level object needs to affect a change higher up, then you insert a command in the local system's queue. This will keep things always flowing from the top down and never bottom up. Bottom up execution flow is a nightmare to deal with. The problem arises when something is done while returning from other functions. You get into serious causality problems that are difficult to trace. Unfortunately, distinguishing from bottom up execution and returns is not so easy unless you have a clear object hierarchy.

Once you have a proper framework set up, it's really fun to work with. It does take a LOT of work to get there though. And that's the real shame of many programming languages. There are no serious tools for passing data around or for setting up these kinds of extensible facilities. Truly building on top of something without worrying that you'll break something is a breath of fresh air. Having it work the first time, every time, is a dream come true. I know people will disagree with what I have said here, but having set up these kinds of frameworks many times now and seeing how well they work, even in distributed environments, there is simply no comparison. I believe that at some point, we're going to have to move towards being able to BUILD instead of modifying all the time. Where if you need something changed, you swap parts, not change code. And finally, where the people who start projects aren't cursed at by the people who maintain them.