Software Development

2. Routing

If your connection is not 100% severed, you can try and reach the external machine through an alternate route. I'm not sure how much this can be used directly from an application since routing is usually updated automatically.

3. Queuing

If there is no way to process the request right now, then queue it until the required resources become available once again. This is what you're email client should do when you compose an email and you're not currently connected to the Internet.


The above options are available if you know something went wrong. Before even getting to that point, you need to have a mechanism in place that can recognise that something went wrong in the first place. For example, if I send a message and never get a reply, I could be waiting forever. This is old school programming where the developer thinks that return-to-sender programming (imperative and functional) will work in a concurrent environment the same way it works on a single machine. You may know it as synchronous message passing. This is what all OO and functional programming languages do. For asynchronous message passing (like sockets), you need to use libraries or extensions. But the native message passing is always synchronous as is the case with method invocation.

Return to sender programming must really die a horrible death if we are to move into the concurrent world. Some people note that the web uses requests and replies and works great. Well, that's not entirely true. It works great because there's a human on the receiving end that can hit the refresh button when a request hangs. Web requests hang ALL THE TIME! That will simply not do if we start using concurrent programming where there is no human monitoring the situation.

Even if requests don't hang, here's another scenario that I've talked about a million times over. Suppose you have four live human users waiting for responses and their requests arrive at the server all at the same time. There are two ways to handle this. For now, let's assume that we only have a single processor. As to why we're doing this will become clear very shortly. On a single processor, we can handle the requests sequentially or we can handle them using threading. One final assumption is that the requests do not block. In such a scenario, we know that threading should not be used, but many programmers don't see any problem with it because all the work needs to be done anyways.

Before going further, let's tackle both scenarios. The first is handling the requests sequentially. For this example, let's say the requests each take 10 seconds to handle just to make things very clear. Four requests at 10 seconds each equals 40 seconds. But what is the average wait time for the users? The first request finishes after 10 seconds, the second after 20 seconds, etc. So 10+20+30+40 = 100 seconds total wait time. That's an average wait time of 100/4 = 25 seconds.

Now for the threading scenario. Each thread has equal rights to the processor, so requests will take N times as long to complete where N is the number of threads (which is 4). So each request will finish at exactly the same time of 10*4=40 seconds. Again, the total execution time is EXACTLY the same as the sequential code. But what is the average wait time for the users? It's 40+40+40+40=160 seconds. The average wait time is obviously 160/4 = 40 seconds. That's 60% longer than the sequential method.

So what's the big deal? Some would mention that we can just get a machine with multiple cores or use load balancing and be done with it. Sure, but that's possible when we're dealing with PARALLEL processing. Parallel processing is DIFFERENT from concurrent processing. Parallel processing is where there is ZERO coupling between processing tasks. Concurrent programming is where you can execute different tasks and where one processing task can be dependent on other tasks as is the case with pipelining.

Here's the TSN turning point. What if instead of four live users, we instead had four processing cores waiting for results where all five cores were part of a larger algorithm as is the case with true concurrent programming? All of a sudden, the way we view the problem changes drastically. The sequential processing of the main core yields an average wait time of 25 seconds vs. the threading model that yields a wait time of 40 seconds before the other cores can begin processing their data. The threading model is 60% slower even though its total execution time in the main core is EXACTLY the same as the sequential model. In the sequential model, one core only has to wait 10 seconds before starting work and only one core will wait 40 seconds. With the threading model, all of the other four cores will have to wait 40 seconds. Now think what would happen if those four cores also use the threading model and receive inputs from multiple sources. The lag will grow exponentially.

This isn't to say that threading won't have its uses in concurrent programming. It means that you must know when it's good to use them and when it's bad. Threads are usually good if they block at some point. While they are blocked, another thread can pick up the slack. That's how threads should be used although my personal opinion is that nothing should ever block. But this is a throwback to the old school, return to sender, style of programming where we assume everything will work just fine.

What's more is that concurrent programming must use some kind of timeout value before sending the request again, or canceling it all together. You don't want to use a style of programming that would lead to increased lag for no good reason which would result in lost and/or wasted computations.

The next topic concerning networks is the black box principle. Right now, programmers too often try to treat each processor as an actor that has a will of its own. In short, each processor is allowed to execute and perform whatever action it wants without regard for other processors. If those processors need to be coordinated, then the use of locking comes into play. For a few processors, this may work, but for concurrency over networks, remote locks are a sure fire way to create software backwards. Let me be more blunt. It's STUPID! Don't do it.

What I find particularly curious is that the black box principle has allowed pluggable components in EVERY SINGLE FIELD that has used it, except for the programming field. Actually, that's not true. It hasn't worked in contemporary programming languages because there is no such thing as a black box in those programming paradigms though certain languages have added extensions to allow the black box principle to take hold (like arrows and monads).

Unlike what Haskell programmers would tell you, concepts such as arrows can allow you to create anything you like. The limitations of arrows in Haskell is due to Haskell itself, not to the dataflow model.

To clear up a misconception... if you use functions, you cannot use the black box principle. It's impossible. What functional programmers have done to get around this restriction is to undefine exactly what a function really is. In functional programming, the usual way it works is that a function is the only way to change a state. This is done through the evaluation of that function where the function itself is substituted out in favour of the result. IOW, the function and the value (which can be another function) are equivalent. With arrows, this no longer holds. The function no longer uses the substitution model. But that does not stop functional programmers from saying that functions are still used for monads and arrows. Don't let them bamboozle you. It's not true. When used as an arrow, the "function" no longer uses the substitution model and you cannot interchange the "function" with the value.

The big difference is where the result goes and if the processing entity ("function") self destructs or not in favour of the value (substitution model). If you can replace the function with the value, then it's a function. If you cannot replace the function, but rather you must pass on the result to another processing entity, then you have data flow. Functional programmers often boast about Map and Reduce not knowing that they are in fact boasting about data flow and NOT functional programming. Also, these tools in functional programming are restricted forms of what they are truly capable.

This isn't a rant against functional programming. It's meant to clear up misconceptions so that the reader knows exactly what the differences are between seemingly similar entities. So? What's the deal with this black box principle? The black box principle allows you to connect things together in the form of a network. This is important because concurrent programming can only happen over a network.

I cannot stress enough how important that last point is. The physical platform will always determine what programming paradigm will work best on said platform. I often hear from functional programmers that the Von Neumann architecture is what is hurting the functional paradigm. Uhh.. yeah. That's my point. Still, we're now moving away from Von Neumann into multithreaded shader architecture. That's not going to help functional programming either. A shader is a usually small block of code that can be executed in parallel with other shaders. That's where we're headed. I think this is going backwards toward essentially having multicore programming, but on a single chip. In any case, multicore is here to stay, so we must have something that works well on a network.

And we finally get to the main point of this article as to why current languages fail at concurrent programming. And I had to learn this the very hardest way possible. I refused to believe it. So don't take my word for it. Go and try to solve the multicore crisis as countless others are trying to do. Multicore has been around for an extremely long time. It's not new. As I've been criticized in past, so I do unto you... Good luck trying to best the greatest minds in the field of the last 50 years. I hate that excuse, but I say it here because people keep trying to use the same tools hoping for different results.

The point is that there are TWO different systems at work here. The first is what we all understand. It's that we want to accomplish some task. Whether we want to accomplish it using imperative, functional or data flow doesn't really matter because anyone proficient in those languages can get it done. The real issue comes with the second system. The network. The infrastructure that the code runs on must be controlled. If you write a concurrent application, there has to be a way to say which machine is in charge of what tasks. Right now, I think 100% of programming languages that allow any control over this put that code together with the first system. Too much coupling. There needs to be TWO separate systems.

Take a road network. There are rules in place for everything that goes on them. Those rules do not care what the actual vehicle is, in the same way that networks do not care what kind of data is transferred. Those are the rules and nothing can break them. That's not the second system though. The second system has to do with determining which path to send the data through. When you go in town to buy something, the part about getting there is a secondary task, but it has to be done. See, what you want to buy and how you get there are two separate issues. The part about getting there should be replaceable. It should have NOTHING to do with the main task. You could take the bus, a taxi, drive, walk, take the subway, etc. The second system in charge of the network should be in charge of determining what is best for you. Also, if a new road (or connection) is built, the second system should be able to take that into account. IOW, an entirely new way of travel could popup and the original software need not change.

This is why threading is so difficult. We merge the network handling (locking) and the actual task all within the exact same code. In order to avoid this, Project V has two sections. I call them runtime and designtime networks. The runtime network is what you actually want to get done. The design time network can build networks. It can change, delete or add components. And it can reroute any connection. There is SOME level of interaction between the two. But these are mostly events that have no bearing on the actual task. For example, data arrival would be noticed by the designtime system which then activates a component if all inputs are available, but there is usually no actual logic that comes back down to the main network from the designtime network. The designtime network is there to assist the main network in completing its task.

Look at it this way. When a new road is added to a city, it's not the drivers that decide to add it. You don't drive down a road and when you think it'd be nice if there was an exit here, one magically appears. That's not how it works. No, it's another group of people that gets that done. Sure, they have input from the drivers. But not WHILE they are driving and not based on ONE person's opinion. In concurrent programming, we are not in control of what new wires are laid down either. But it does show how there is a separate system in place that controls the network. Software should also have its own separate system that controls where its data goes.

We should look at the consequences of not separating these two systems. If a task itself decides how to get data from point A to point B, it is stuck to that architecture. For example, when you use C++, you use method calls and returns. The software itself decides, and is enforced by the language, on how the data gets from one object to the next. This is why this code will not run without modification on a multicore system and make use of all the cores.

A clearer example is sending an email using functional programming. It can't be done because the functional paradigm restricts how data is transferred. In the functional paradigm, you may only substitute a function for a value. IOW, what goes up must come down. This means it's impossible to throw something up and keep it there, because eventually, it must come back down to the source. That's the substitution model. That's the paradigm itself deciding how the data travels between functions. At some point, one of those functions will have to be non-pure. Functional programmers will say that the external environment doesn't count, so the function is still pure. I always ask what they would do if the external environment also happened to be functional, would that not mean that the external environment would no longer be pure? Of course it would no longer be functional. So my point remains that you cannot send an email in a purely functional environment. You need something else and that alone shows that functional programming requires another system to make it work. Unfortunately, functional programmers wish to deny this truth and invent ways to wrap these things in constructs that look functional.

Then there is error correction. Error correction should be done in the second network if possible. Those three options I mentioned at the top of this article are implemented here. They do not go into the main software. The main network shows what you want done without any clutter. This means the second network is free to implement all sorts of recovery tactics. It can implement rollbacks by keeping tabs on what data came in at what connections. It can retry the operation. It can use several execution resources over the network in an attempt to get a correct result. It can even ask other machines if they know how to recover from this error. When it encounters an operation it knows nothing about, it can likewise query other machines if they know how to execute it. At that point, it can ask them to perform the action or it can request the code and perform it locally.

The drawback with current languages is that all this must be coded in directly with the use of checking return values, using exceptions or whatever other mechanism is in place. So a machine can have a way of fixing the problem, but if it's the main software itself that's in charge of handling errors, then too bad.

To sum up, the resources required to perform an action are secondary. Anything that is not directly part of the transformation from source to result should not be included in the primary network. Memory, processors, connections between operations, etc. should not be specified directly in the programming language.

Obviously, this isn't always feasible in every area, but if one were to do this most of the time, it would be a million times better than anything currently available. We can do so much better than the hardcoding going on today.

Imagine a world where your main software only has the task at hand without any resource allocations. No memory handling at all. Not even allocations. No processor allocations. That means no threads. You'll also have pure components that don't have the return to sender restriction of functional programming. So you can receive data, process it, and send the result to a different location than where it came and still be pure as is required by a true black box principle, thus allowing pluggable components. Think about what it would mean having a machine that learns and can update your software without user intervention.

There is so much more that is possible. I hope creators of development environments take all these points into account. I don't care if they agree or not just as long as they think about them and propose alternatives and solutions. How is it that concurrent programming is upon us, yet we're still trying to pound a square peg in a round hole. I think the corners have been chiseled off. It sometimes works, I suppose. Still, I'd rather have people accepting and discussing these topics openly. The situation right now in programming is no different than it was 30 years ago. Faster computers, better IDE's... but it's NOT different or more advanced. We basically have the programming equivalent of TV's with color.

Would you believe that my computer has no idea what a calculator is? It doesn't even have an interface where I could ask it such a question. My computer is 100% certifiably brain dead. It knows nothing. It has permanent amnesia. Every time I quit an application, it forgets everything it knew how to do just milliseconds ago. If humans operated that way, we wouldn't even have huts. I doubt we'd know how to walk. And we've been OK with this for over 50 years.

Don't let anyone fool you into thinking that there have been great advances in development in the last 30 years. Those are the same people that think Waterworld and Glitter are misunderstood and ahead of their time.

I believe that the near future will show great advances in development. Not because of what came before, but because of people who dared to see the world in a new light.

General impressions are never to be trusted. Unfortunately when they are of long standing they become fixed rules of life, and presume a prescriptive right not to be questioned. Consequently, those who are not accustomed to original inquiry entertain hatred and a horror of statistics. They cannot endure the idea of submitting their sacred impressions to cold-blooded verification.

- Sir Francis Galton

So beware the trap of generalizations. If one says that visual programming will never work, it may be worthy to note that this is only true of current programming styles. Visual programming is used quite successfully with dataflow with shaders in games, nodes in 3D renderers, codecs and other areas. So watch out for No Silver Bullet comments. They likely apply to specific cases that have been incorrectly attributed to the general case.

Then again, there's always this quote by one of my favourite actors.

Failure is unimportant. It takes courage to make a fool of oneself.

- Charlie Chaplin

Obviously, Charlie Chaplin wasn't around when they invented the Internet.