Software Development

Just saw this short blog entry about not checking if you're out of memory. I don't normally check for this, but in some cases, I do check for it. I tend to do this in production code or in certain cases where I request large chunks of memory. If it's just allocating objects here and there, then I agree there's not much point considering that virtual memory will be in use. And if there's no virtual memory, then there are bigger problems at hand.

But this got me thinking about dataflow. Would you need to check for memory? In a dataflow environment, you don't allocate or deallocate memory. Current implementations do force you to do this. But in Project V, I've decided it's best to leave this up to the runtime engine. If there is no memory, the component won't execute. But more than that, the runtime engine can often know how much memory a component will use. For example, an addition component will take N inputs and produce one output. Executing these components can free up memory at a ratio of N to 1. This gives great flexibility in environments with limited memory. You don't need to be as careful.

I think this is another advantage of dataflow that might not be important on the surface, but not only can it play nice with other apps by being able to continue executing with less memory, it can execute faster because using less memory means better cache use. And that's a great way to speed up any application unless you use disk space.

One example I keep thinking about is when you have a LOT of data to process, it's easy to allocate huge buffers and work with that. But it's terribly slow because you're always working from RAM or worse, from the disk because of virtual memory. I often wish there was a way to easily stream my operations. Instead, it's much easier to allocate new buffers for intermediate results and use those. With dataflow, all these operations can be streamed and use very little memory and hence execute much faster than would normally be the case.

I'm not saying that Project V will be faster than existing software. Project V will have a runtime (with an option to compile directly), so there will be some unavoidable overhead. What I am saying is that I'll be able to produce software that will execute faster than I originally anticipated and will continue to get faster with new hardware like multi-core architectures and faster and larger caches (and new versions of the runtime engine).

I think it's a fascinating look at what could be a trend in the future where developers no longer need to have direct control of the execution point or of handling memory. Now, there are at least two layers in Project V. One layer is your application. But the other layer is what controls creation of components, linkage and all sorts of other things at runtime. So this part will still have direct access to execution, memory, resources and everything else. However, the need to access this layer will become less and less. The trend away from micro management without loss of control is what I think is a good move. My main objection to other languages and development platforms is that if they move in a certain direction, they tend to make it impossible to have access to those older features. They lock those features away. That's the wrong way to go. In Project V, I made certain that you can still implement custom allocation algorithms, execution profiles, etc. And this at any level of granularity where different parts of the network can have different resource management.

If we look at each item in Oleg's blog entry, we can see where dataflow has an advantage if implemented correctly.


- Because it increases your code size by 30%-40%

Not true in dataflow. In fact, the software will actually be smaller.


- You’re trying to be more catholic than the pope, since various
systems services you build on and interface with aren’t OOM-safe
anyway

While true when accessing native code, OOM-safe code will be automatic.


- You are trying to solve the wrong problem. Real OOM wil be signalled
via SIGKILL, not malloc() returning NULL.

With dataflow, executing components that have fewer outputs than inputs will effectively release memory. This is where letting the runtime engine control the execution point can be beneficial. Also, the program can simply halt until more memory is available if there is no choice. When your program has direct control of the execution point, this is simply not possible without the program itself checking for it where Oleg's points come into effect.


- You are trying to solve the wrong problem. Make sure your app never loses
data, not only when malloc() returns NULL

I'm not sure what this means. Dangling pointers? Buffer overruns? Even in older languages, there are techniques that take care of this.


- You can barely test the OOM codepaths

Not a problem with dataflow since there are no codepaths. All one needs to do is limit a network to use X bytes and you can test the stress levels (if it works or not) with different inputs.


It seems to me that many of the problems that occur in contemporary languages are due to the actor model. Some may think that the actor model can be used to implement a dataflow model, but this is only partially true. There are things in the actor model that have problems. First off is that the actor model has entities that can make decisions. This is the age old concurrency problem of synchronizing simultaneous actions. Not only that, but actors can create other actors. This has several issue similar to that of threading. In fact, actors are often created with this idea in mind.

I have to say that the "everything is X" mentality is what is flawed here. Take the wikipedia entry on the Actor model.

The Actor model adopts the philosophy that everything is an actor.

You may think this is eerily similar to everything is an object and you'd be right. Not saying they are the same, only that the "everything is X" mentality is pervasive.

"Everything is X" is impossible. You always need a duality. This is the way I'm building Project V. One layer is the application and another layer is where there are entities that have access to execution, memory, threads and other resources. This is very important. Without this separation of concerns, the same problems will keep surfacing. Neither OOP, functional or the actor model separate the data from the execution. Heck, functional binds them together so that they are indistinguishable. More than that though, it's possible to separate most, if not all, resource handling from the processing of those resources. So it's not just separating data from execution. It's possible to separate the managing of the resource from the processing of that resource. Duality is everywhere. Components vs. connections. Resource processing vs. resource management.

I will look into if the technique used for memory management can't be used for other resources. I think I will find more pleasant surprises as I move along. The IDE is presently what is slowing me down. I'm working on a command line at the present time with the results shown visually and will get that working first. Hopefully, this will speed up development. Will have another update when that is done.

edit: Oh, and I forgot one other advantage of dataflow with respect to OOM. In a distributed environment, a machine that is OOM can dump its tasks to another machine. And it would do this automatically.