Software Development

Do you remember when you first learned to write software? You may have been taught to use pseudocode. Maybe it was flowchart. Maybe you just dove into it. Whatever the case may be, I bet it always revolved around listing the steps to be done. Why is this?

It may sound like a stupid question at first, but when you think about it, listing steps is not the best way to write software. Actually, I think it's one of the worst ways of programming. It took me a long time to resist that urge. Today, I do not do this. I haven't listed steps in a very, very long time. 99% of my programming time goes into design. Actual typing is quite minimal. Sitting in front of the computer screen actually slows me down. That's the reason I'm writing Project V. So that I can be productive in front of the computer for a change, but that's another story.

Let me put it this way. When someone tells you that they need a certain button to display a certain result, do you instantly think of how you would code that up or do you instead think of the information needed and how this will be coordinated with other parts that will also make use of this data? If you don't see the difference, then perhaps you never will. It's true. For many, many years, I could not have understood this. Over the years, I naturally evolved a different way of programming. One that requires very little time in front of the computer. Once I'm sitting in front of the computer, I'm pretty much done all the programming. It's just data entry at this point. Don't think that my time spent typing is short. Rather, I spend much time on non-programming tasks such as getting other people's code to work properly with mine. Only in programming do people think this is part of the job. It's not. At least, it shouldn't be. No other field thinks this is normal. In other fields, this is wasted time. It's only done because someone messed up.

I'm finding that many other fields work in the exact opposite way that programming does. Programming usually thinks of how something should be coded up BEFORE thinking of what information and how different parts interact. In other fields, people think of what the final picture is and then proceed to deal with the steps that will accomplish it. Everyone knows this is true. In fact, most people hate that clients don't know what they want. This is not true. They're just not speaking your language of defining steps. Instead, they're telling you the overall picture. It may not be complete. But so what?

In a house, I can remove a wall and put up another one somewhere else within hours. Even with electrical, plumbing and all that, it can be done relatively fast. The floor, walls and roof of a house can go up in two week with only two people. Heck, I'm doing that now and the floor will be done in under three days (with one of those days completely wasted). Another three days and the walls should be completed or close to it. But if there's anything wrong, I can fix it. If you want a deck, patio, or extension, no problem. Why isn't software that way? No, it's not because software isn't tangible.

It's because of many factors. Your computer is really dumb. That's the main reason. It knows how to do absolutely nothing on its own. Zero vocabulary. Once a piece of software terminates, all its know-how goes with it. There's no way to evolve. Next is that there's no way to define the overall picture in the tool that we use to create software. NONE! No other field works this way. Engineers have plans. From these plans, they build things. I've never seen a real design in software ever. No one has. It just doesn't exist. You don't believe me? Show me a design that indicates exactly in your code what it does and how you can change it to something else without breaking anything. Can't do it, can you? Real designs work that way. You can look at it and locate anything you want.

I say again that 99% of coding is design. Unfortunately, design in software isn't what I know as a design. And frankly, it's tough creating one the way we write software today. Can I locate anything I want in my code? Sure. But I write my code in a way that you can patch into it. I have a list of core services and then you can add your own services or make use of those services. Anything I want to track down, I just look at those services, or at what makes use of them. Services are things that handle certain types of data. So if you want to use data, you must assign a worker to handle it. Little did I know this is exactly how the real world handles physical structures. Workers are assigned to manipulate the actual "thing" you want modified.

I've mentioned in the past why I thought this worked so much better. I've also asked a few questions about why it worked so well. In programming terms, there are plenty of real reasons why this system works better. But this new way of looking at things where you assign workers to do the work on data just makes sense. I never really thought about it, but all the code that I have where I queue a worker to a service that handles the data works so much better. When a button must do some task, the activation code should not do the work itself. It should queue a worker to the service that handle that data and if it needs to wait for the results, then assign another worker for that part and return immediately. Doing the work directly from the activation routine is just asking for trouble. You're basically assuming you have control of everything when you probably don't. Am I off base here? What I like is that if anything goes wrong, you know what service it was, what worker it was and what activated it.

Every single time that I coordinated the data first and then assigned workers to those services, everything worked so much better and I could write MUCH larger software. The problem is that setting up these services along with resource handlers takes a long time. There are no built-in tools to let you program in this way. No really, there aren't any. Not where you can just click and it's done.

One thing that was missing in Project V that kind of annoyed me was what I was going to do with storage. You can't get away from it. All functional languages support storage nowadays too. In Project V, I was thinking too much in imperative terms of having components that retain data where you could request and update data (with lots of automatic management of this data). That has always bothered me. Now, I realise that a better way would be to assign workers to certain pieces of data depending on triggering information. This would allow yet another way to convert data by staying within the flow based methodology. So worker components can specify what data they need as inputs as well as the triggering information. When all the inputs are there, it can use and update the data. The outputs would also specify what services need updating. So coordinating data can be guaranteed to be consistent. The order in which the updating gets done is still the programmer's job to handle, but at least there's a way to get rid of most of the troublesome details. Plus, you can chain workers if you want. So you can guarantee order in that fashion. This means that certain workers can't be activated out of order. A protection that is lacking, well, pretty much everywhere unless you handle it manually.

An example of chained workers is when drawing a line. The initial component is triggered by a MouseDown event. Then the chain is passed on to the next two workers who are triggered by MouseMove which loops back and MouseUp which ends the chain. If you get a MouseUp while waiting for a MouseDown, the MouseUp component will not activate. In current languages, you must manually check for this. Also, events are triggered separately. You usually have a routine for each one. So something gets activated no matter what. This cannot happen in Project V with chained worker components. With the web, this would be of great value just to keep track of sessions between forms and page views. If they clicked back on their browser, you'll know instantly that they went back and can correct for this. How many web programmers take this into account? How many times did you get messed up data before you did? You can link up your web pages in the same manner. Just click and drag and you're done. Because dataflow software is concurrent, you can have as many sessions as you wish handled automatically. No more waiting on something. Instead you chain the next worker.

The more I look at this, the more it makes sense to work from the data first. It's no wonder that I used to hate web sessions and writing event handlers that must coordinate themselves. I was writing software backwards. It's still going on today and actively promoted. It's contrary to every known real world way of doing things. For me, I'm done banging my head against the wall. You can use these data-first techniques in your own software today. It takes a bit of setting up, but it's well worth it. You'll be glad you did.