Software Development

Let's start with the network. (Click on image for larger view).



It should be obvious what this does. It splits up the input into their x and y coordinates. Then it adds the x's and y's together. The third set of components divides by 2. And finally, it puts everything back into a Point structure. As one can see, this is sequential in its layout. Structures are still the same as in any language.

One thing to note here is that the Splat component is generic. You may link any type to the source and all its members will automatically appear in the destination. Same thing for the Merge component, except that you would link a type on its output.

The above is very close to an actual screenshot of what Project V will look like. The lines will be splines and not arrows.

Let's look at what would happen during "execution". At each step, we can look and see what happens to the data. Look at the numbers in green. (As usual, click on the image for a larger view).



So we can already see parallelism at work. There are 7 components, yet it only takes 4 steps to complete the entire task (step 1 isn't actually a step). In the C++ example, we had 7 instructions. So already, we have a speed advantage if we have two cores (though to be realistic, you would need more complex operations to offset the lag in data transfer). One other thing to note is that the Splat and Bundle components don't actually do anything. They are there for the developer only. But for these examples, we'll continue to assume that they actually perform an operation in order to better visualize what is going on.

Project V also has the ability to create container components. These are components that contain other components inside of itself that will handle the actual task. So you could put all these components inside of another one and call the container "Midpoint" and this is in fact what would happen in Project V. The input would be two Points and the output would be the midpoint (also of type Point). This container component would appear as any other component like this.



The inside of the above component would be the network shown below.



There will be navigation icons available in the IDE (and on the component itself) to change levels of containment (either up or inside).

You could do the same thing with the Add and Divide components. You could put those inside a container called Average. In the IDE, you would simply draw a rectangle around them and select the "Create Component" icon. Average would have two inputs and one output, all of the same type. In fact, this component will exist by default and it too will be a generic component. Whatever you link to it, the component will use that type for all its inputs and outputs. If you connect all the inputs, another input will be created so that you can average any number of inputs. How this is done is for another discussion, but the IDE is under your full control. You can even have editors or full applications as configuration panels right in the IDE. Picking files, colors, etc. for input are common editors. You create these the same way you create any network. There are built-in components that give you access to the IDE, on-screen components and its events (as well as everything else the IDE has to offer including the compiler).

What about the lowest of levels? Surely, we can't keep going deeper infinitely. How is an addition component handled? Actually, this is where the internals of Project V come into play. You may consider these built-in components if you wish. What happens is that there is another set of components that map directly to the processor's opcodes. So the addition component's internals would be mapped to these components. And the compiler would be updated to handle how the data is handled (registers and addressing modes) as well as the ordering of opcodes. A lot of this is handled at runtime as well. The nice thing about this is that since components can "execute" on different cores, there's no need to have the same set of internal components for all the cores. That means you can mix and match processors if you wish. And since this is possible, then you can see how this would extend to the Internet. You can have different machines work together even if their processors are different. Also, you only need about 12 internal components to make all Project V software work. You may also simply write a library (DLL and whatnot) with conventional languages as a stopgap measure until a proper internal component set is created for that processor. OS components would also need to be created. That's a larger problem, and this too is best left for another discussion.

Ok, so far we've seen parallelism where the same operation or different sets of components can "execute" at the same time. But there's a more powerful kind of parallelism that you can use. Software doesn't usually tackle just ONE piece of data. Usually, you need to perform the same operations over and over. For example, take image or video processing. Not only do you have a stream of pixels that you must do the same operation over and over, but you must repeat this process for each frame in the case of video decompression.

So let's take a look at streaming or what is more commonly known as pipelining. Imagine we have a list of pairs of points that we want to find their midpoints. Look at the green numbers. This time we will have 4 pairs of coordinates. The inputs will feed from the bottom of the list. (As usual, click the image for a larger view).



We processed 4 sets of coordinates in 7 steps. This means that for each additional set of data, we only need one extra step. Compare that to the imperative method that would have required 7*4=28 steps. The speedup only gets better the more data we have. And the outputs wouldn't pile up like that either. They would be passed on to whatever the output is connected to and the pipeline would continue.

It is important to realise that for this to work in 7 steps, we need 7 cores (one for each component). Otherwise, the processors will have to alternate between components or group them together. For example, you could group one addition and one divide instruction together and "execute" this on the same processor. That will reduce the number of nodes needed, but will also slow down the entire process. In any case, this is how you can have this same network work on a single processor. It would simply alternate between all components that have all its inputs available.

This is the easiest example I could think of. There is ZERO chance of anyone actually building networks with such low level components. You'd never get anything done. However, I think it was good enough to explain the most basic aspect of Project V (that of parallelism and pipelining). This is only the tip of the iceberg and isn't what I like about this project. Well, the concurrency is nice. But it's the ability to mix and match as well as the possibility for editors. For example, you would never manually create a network as shown in these examples. Instead, the Midpoint component would be an expression component. You would launch its editor and type in "Output=(A+B)/2". The editor would then create the internal network for you. There will eventually be a scripting language as well to enable faster creation of complex low level networks. Over time, more and more components will be available and the less you'll need to create custom components.

One last thing. Mixing and matching components in this fashion is possible because looping is no longer necessary. We simply feed in all the data. No need to know how many items there are in the list. When you have loops, you seriously limit how code can be used. Not only is there the loop itself, but you can only use one item at a time and if you want to add an operation to the beginning or end, you must change the loop itself which lies INSIDE the function. With components, you just insert them and you're done. No need to worry about loops at all. Loops aren't abstracted away. They simply don't exist. Loops are an added unnecessary complexity that was created by the imperative von Neumann architecture. In conventional code, software always has a main loop that makes it non-resusable. With components, your application needs no loops. The application itself can be reused as a component. Functions also carry extra baggage in that the output must go to the same location from where the input came. Notice that this is no longer a limitation. You can send the output wherever you want. This reduction in complexity cannot be overestimated.

I hope this explains the fundamentals of data flow in a clearer fashion. I hope to one day talk about the more advanced stuff. On this blog, I haven't even been able to talk about even 1% of the power available in Project V. Progress is going well and once people can play around with it, I'm sure I'll be able to write about those topics.