Software Development

This CSS stuff is a pain. If something doesn't show up right, I'll try and fix it soon enough. I don't like the HTML they used. Oh well. I'll work around it eventually. Let me know if anything doesn't work. I know about the login box. I'll fix that later.

This is the second installment for the case against objects. This time, we'll get to the real issues surrounding objects and why it makes certain kind of tasks difficult, if not impossible to code.

There are two types of execution when it comes to a single thread. Vertical and horizontal. Or more commonly known as recursive vs. sequential. However, sequential in this case takes on a slightly different meaning.

In either procedural or object oriented code, you are programming in a vertical or rather recursive manner. Each function call can go ever deeper until it returns. But even after each return, it can again call more functions and continue deeper again. The reason I think plain procedural is better than OO is because you don't have global state information if you decide to call the same function again. Or at least I hope you're not using global data in a recursive function. In any case, if you don't have sharable state information, then your function is independent from other calls and there's no coupling of data between function calls. In OO code, the data in your object acts like global data and can cause the state of your object to become invalid if you're not careful with recursion even if you don't call the same function twice, but other functions in the class. These are known problems in procedural code. Yet we went and used a paradigm that makes these problems less obvious and in certain cases much much worse.

The other kind of execution is sequential or streaming code. This is where you pass along some data to be processed from one unit to another. At each stage, the data is used in some kind of computation and can even change format completely. At this point, it is passed along to the next stage for further computations if need be. The problem with this kind of streaming is that the data is bare. There is no protection of the data and each stage is free to modify the data as it sees fit. Also, if you are using the OO paradigm, leaving your data open to manipulation goes against the OO principles of encapsulation. If you were to try and implement this in an OO way, you would have to add each stage directly into the object. But this changes the object itself and introduces unnecessary coupling to these methods. It also makes it less than obvious what the chain of processing is supposed to be and how each stage processes the data. It also isn't conducive to modifying the format of the data. Not only that, but if the data can change, it makes externally accessible methods rather hard to define as each one will now have to check if the data is in the correct format before executing. Normally, the object would only contain data that ALL its methods could handle. And you would have a different object for each type of data. But this doesn't work for streaming data where it can change at any instant.

The thing is that these two paradigms can work very well together in small modules. Basically, we need clusters of classes (to borrow an Eiffel term). At each level, we need ever bigger clusters. Clusters of clusters. Super clusters. Groups of super clusters. And keep going bigger until you're done mapping your entire application.

This means that an object can only reside inside a cluster. This is vertical processing. Data goes in and starts the execution of that cluster. When the cluster's execution terminates, it should output the processed data if any. Each cluster will have its own thread during processing. Only raw data can be passed between clusters in a predefined format. So each cluster should define the kind of data they accept and what they output. You can use whatever you want, but there should be a standard container for different kind of things. If you want to use XML, that's fine, but there should also be a binary version.

This would be a normal app with a super cluster controlling the operation of all this. So the first super cluster is the application itself with as many clusters as needed inside. Each cluster can contain objects. So a single cluster is the way we write applications NOW. Trying to shove everything in there is ridiculous.

Now notice what problems we fix. Any shared library that is used by multiple clusters will have to be defined as re-entrant, be thread safe or copied directly into the cluster. Whatever way you decide is fine and you could even use a default to avoid confusion. This means there is no threading problem from within the cluster. There are no concurrency problems. And there are no synchronization problems. The super cluster is what manages all the recourses of all the clusters, so it also handles the creation, deletion and execution of threads. It's a grandiose thread manager. Notice also that the super cluster is responsible for chaining the subclusters together and for passing data between them. This passing of data can be done by yet more threads, but under the control of the super cluster. Notice also that each cluster can execute in parallel.

If you can't see the advantages of this, then you've probably never done large scale distributed or threaded applications. You always end up doing something like this anyhow. The funny thing is that there are very few people that actually code this stuff because it can get rather complicated very fast. If you've never done it, you will fail. That's all there is to it. I've spoken about this before so I won't go into it. But I think if the setup was more conducive for this kind of production, it would suit many types of applications. In fact, it would encompass the OO, procedural, hierarchal and streaming paradigms. Not to mention easy threading. You also don't need to chain clusters together. You can use them as independent processing units thus making it somewhat like a processing array.

Now super clusters are nothing more than ordinary clusters. They are simply at a higher level. So the higher up you go, the higher your concepts should be. If you start coding a network protocol in a super-super cluster for example, then you know you're doing something wrong. Processing of that kind should go into a cluster or even a library. This allows for very clear separation of abstractions. There is now a progression. A super-super cluster can be invasive in that it can replace lower clusters with replacement parts. An example of this could be to create a thread on another computer when an underlying cluster tries to create a thread. And to provide an identical interface that would link the two computers up. As far as the underlying cluster is concerned, it would not care or necessarily know that the thread it created is actually running on another computer. These replacement parts are coded at a lower level than the super-super cluster that actually uses them. All you do at this high a level is tell certain parts how they link up and where they should go. The details are left to lower level clusters. And because they are independent, there's no risk to any of the threads.

So now you can take regular applications and make them distributed with little effort. I can already see what kind of tools would be useful in these scenarios. It also means that we can go to infinite levels of distribution. So if in the future we get a super-Internet that makes our current Internet look like a regular hub, then there's no problem. We add a higher cluster that redirects the necessary parts and problem solved. No need to actually modify the main source. BTW, this kind of super-Internet would probably only happen if we colonized the moon for example. I think ahead and BIG!

Notice the usefulness of both vertical and horizontal processing. But a clear separation is critical. Without it, we get a jumble of classes all over the place. Even when we drive for example, we have different levels of roads. There are residential roads, main city arteries, inner-city express highways, inter-city state or provincial highways and national highways. There are different concentration and levels of roads. In any network, this is the case. Our software should be no different. To me, this is so obvious it's stupid.

I'm not going to spend much time on this item, but it should be mentioned. Yes, theoretically, you CAN do all this in OO just as you could in a procedural language. But I could also use gotos instead of control statements and I could use assembler for everything. The point is why would you not want to be organized in a more productive way? And that's all I'll say on that.

We all know that objects aren't very useful by themselves for the most part. There's always the odd RNG type class that can be reused, but if you're doing anything even remotely useful, you're not going to have these kinds of independent classes. The opposite is just as dangerous however. Having a huge amount of classes that all operate together is not very useful because it's a maintenance nightmare. And that's the problem most large applications have. So it should be obvious the need for independent groups of classes. A certain number of classes should be able to work together, but at some point there has to be a limit. This is the reasoning behind a cluster. It should have a specific functionality and be independent from other clusters. This doesn't mean the clusters can't interoperate, but it has to do so via the super cluster. So whenever there is an error, it doesn't backtrack through the call chain. No, it goes up to the cluster and it tries to fix the problem. If it can't it either cleans up the execution thread or passes it up to its super cluster.

I'm also going to make each cluster have a standard interface so that they can be easily replaceable. They'll be extensible of course, but you'll be able to access all its functionality through this single interface no matter how much it has been extended. You can also have direct access to this functionality if it is known ahead of time. In fact, this will be automatic in my development environment.

This system also clearly defines who owns what. Your so-called GC will work quite differently under the hood than what you're used to. Passing objects around will become very controlled and the need for a GC will not be as great. But if you wish to use one, it will be local to the cluster. The super cluster can of course monitor its activities. The GC will not be a GC in the normal sense. There'll be a memory manager for each cluster. When you allocate memory, you specify whether you want the memory manager to deallocate it automatically, to use the stack or scope for deallocation, or manual deallocation. Most people will use only use one form, but all allocation methods will be compatible with each other. Even if you allocate something that you want garbage collected, you can still manually deallocate it, and vice-versa. You can of course tell the memory manager to only accept certain kinds of allocations and notify you of any inconsistencies. This is mostly to enforce your organization's guidelines.

Notice too that you can automatically ignore manual deletions if you tell the memory manager to garbage collect everything no matter what. In this way, you control how things are done and the since the super cluster can invade its containing clusters to change the way certain parts work, you can fine-tune your application without any need to change a single line of your original code... even for how memory is handled. And since clusters are independent, this makes garbage collection much easier. Heck, I don't even see the reason for a GC if your clusters or modules are well written. This is why I'm against GC's. If your data is well organized, there's no need for it. And with clear boundaries and ownership, much of the code to handle memory management can be automatically generated without a GC. So yes, there are alternatives to GC. Dare to see in color and not in B&W.

So there are three types of objects. The first one is somewhat similar to what we currently use. Then there are clusters which can execute threads. The third type of object is raw data somewhat like a structure in C, but with more power. Also note that even though I use the word "object", these are quite different than what contemporary languages use. I hesitate to use that word, but a grouping is generally known as an object so I'll use that term. I also haven't showed what the implementation will be like for leaf objects inside clusters. I'm still working out the finer details, so that will have to wait. All I can say for now is that objects will not have inheritance as you know it in C++ or Java for example. Much of the confusion will be removed. The private keyword will not exist as you know it. Virtual functions will be unnecessary. In fact, there will only be two types of access. Internal and external to the class. The external part can be either commands or properties. Of these properties, they can be linked to both data and/or methods. After all these changes, I think I'm allowed some slack to say that they're not the same as what we use now.

Clusters will have a default container for sub-clusters and a link to its parent. If it has no parent, there will be a stub there instead that will have default actions. A cluster also has a main execution entry point with a channel for data. There is also an output channel. So with a common interface to exchange data, connecting clusters together will be much simpler than it has been in the past. These input and output channels can expose common protocols for even easier interconnectivity. With a common entry point and command/data channel, a super cluster can inspect its sub-clusters to see if they have the necessary functionality before connecting them. In this way, dynamic updates will become much easier. And I already mentioned that clusters will be responsible for their own internal resource management. Any resources it can't manage will be passed up the hierarchy in the same way errors are passed up.

This brings up an obvious advantage when it comes to fault tolerance. Instead of generating exceptions that can collapse your whole system, the cluster is responsible for handling errors that the object who caused the error can't handle. It doesn't backtrack along the execution path. This is important. Now the cluster can try and fix the problem and resume execution or it can pass it up to the super cluster for it to fix by either trying another cluster or whatever else is in place to handle the error. It could just as well signal the cluster to clean up and terminate. In each case, the rest of the system can continue operating and finding a solution is not dependant on the caller. It is the responsibility of what controls the resources. This is a major departure from the way we currently do things.

Anyhow, the list goes on and on as to how this model of BOTH horizontal and vertical processing fixes current problems. Exceptions sucked, but are now fixed. Threading sucked, but is now fixed. Concurrency sucked, but is now fixed. Replacing components used to be virtually impossible at runtime, now it's simple. Objects sucked hard, but they too have been drastically updated. Fault tolerance sucked, but is now much improved.

I am fully aware that at this point, they are just ideas. And I haven't gotten into the details of what's involved inside a cluster. The thing to keep in mind is that I've put forward alternatives. I'm hoping that some people will see how some of this is different and how they could be beneficial. I do know that on the surface, they look similar to what you use or have seen. I'm hoping that soon enough I'll be able to have something in motion so that you can appreciate the differences. There are still many details to work out at the lowest level, so that will take time. And words cannot really do it justice.

I suppose a simple example would not hurt. Take for example opening a file. Currently, we usually check the error code or trap an exception. If the exception isn't caught then it backtracks along the execution chain. Now consider if the open file command passed the error automatically to its parent super cluster along with the filename and other associated information. The super cluster can now pass this information to the GUI sub cluster to display an error message and some kind of option like choosing an alternate file for example. Then the super cluster can replace the filename and retry the open file command. The key here is that there is a centralized location for resource management and handling of subcomponents. So you can have standard recovery practices for the same types of errors. It should also be noted that you can fine tune the error recovery. If you wanted the leaf object itself to try and correct this error, you can do that too. Sometimes, the fact that the file doesn't exist is perfectly OK. But if you run out of memory or hard drive space for example, it can be automatically recover by notifying the user without putting this directly into the main flow of your source code.

There are some weird things you could do like create backups of all data in clusters so that if any of them crashes, you can easily restart them without loss of information. It'd use up a lot of space, but you could theoretically keep the system running long enough until you can fix the bug and update the cluster without ever shutting down the entire application. I'm also planning that you could move clusters to other machines at runtime with the click of a button. The other machine can be completely different from the source. Heck, it could be alien technology and it would work just by implementing a few minimal details. The ability to port to an unknown system with almost no effort will open up a whole new area of support for all types of hardware that never existed before.

Everything in this entry is the tip of the iceberg. While there may seem to be an abundance of parts to this system, it's time we move into the future. Most of the parts will be automatically generated anyhow. I doubt if you'll ever see any of the high level parts if you just write a normal application. The fun part is that other applications will appear as regular clusters with specific functionality. So add a super cluster to your computer, link this application as a sub cluster and you can add a different GUI, resources or anything you like without ever modifying the original application. You want two users over the Internet to use the same application? Click of a button. You wrote a game, but didn't make it networkable? Add a filter to the input devices so that it can merge commands from a remote computer. Done. Could you imagine writing an application as if it only ran on one computer and have it automatically distributed over many machines without changing a single line in your main application? Imagine that we could just add a remote cluster to our application that would link up to a search engine. You could pick and choose which search engine you want. All this with the click of a button. This feature could be used for many things. Web browsers could link up to sites in a completely new and infinitely more powerful manner. There's just too many possibilities.

This is how I think things should be. And I see no reason why we're not there now other than false hype and reluctance to admit that maybe there is a better way. At the same time I know that many of my ideas will change over time as the quirks get worked out. Yet I can't help thinking we should be programming with much more ease and power than we are now. I've already got the most basic part of my environment working. The rest is creating some standard interfaces for commonly used features and working out the details of the compiler for what I've talked about in this article. It's all gravy after that.

Today, I thought I'd give a shout out to everyone's that quoted me online in their blogs or website. And a big thank you to those who said I was wasting my time and insinuated I was bonkers in some fashion. To me, this is an indication that I am either actually crazy or I'm actually on the right track. I'll place my bets on the latter.

I've been criticized for my use of terms and rightly so. I am very direct. I'd rather build than use fancy terms. That's one of the reasons this blog was created. Because I could no longer build. Or rather I could no longer build as fast as my capacity would enable me if I had the right tools. After looking around, I found that there was nothing out there.

Here's a few things to note about the way I look at things and the way I think:

1. I don't care about notations or what it's called as long as it works.

2. When I code, I see it all in assembler no matter what language I use.

3. I don't like seeing power go to waste. Whether it be 3D or CPU instructions or any kind of hardware that goes to waste.

4. When I have a discussion, I'd much prefer to talk to *you* than a quote.

5. I actually do know what I'm talking about.

That last one may not be obvious though. I've been criticised for my use of "object" for example. And as a commenter said, my definition of encapsulation is not the norm. For 99.99% of us, when we use an object, there's usually both data and methods inside. So that's my definition. Nothing more, nothing less. Remember that I see things in assembly. In assembly, it'd be a bunch of disconnected data and methods. Nothing more, nothing less.

So I do tend to use certain terms in their most basic meaning. If I say "class", I probably mean class as you would use it in C++ or Java. I may even use them interchangably because although in "research" terminology, there is quite a difference, when you actually want to do something, classes don't really exist, only the object or instance.

Ok, enough about that. In almost all blogs and other web sites that quote me, they seem to revel in the fact that they can point out something somewhere that seems to suit my needs. If this were so, I wouldn't even talk about it. They seem to miss the bigger point that these things they point out don't suit ALL my needs. Just because Java does this and SmallTalk does something else, this doesn't fix my problems. It's like someone prior to 1900 saying he wants a flying carriage and everybody else exclaiming that it's old technology because birds already fly and we already have carriages. That's all nice, but putting those technologies together isn't so simple. So you bloggers that say I'm reinventing the wheel, I say back that you don't even know what a wheel is.

I've come to the conclusion a while back that the only way I can really explain what I want is by creating it. So that's what I'm trying to do. I've written plenty of parsers, parser generators of every kind (some are on sourceforge), compilers and all that. But it takes a whole different magnitude when it comes to actually designing a new platform. So I have to make sure I get things right. Oddly enough, I've agreed with 99% of what people have commented on. The 1% that I disagree on is how they're used, implemented, designed and whatnot. The ideas themselves are not necessarily bad. This has caused many to not understand where I stand. Well, it's easy. Just look at the tool and tell me who's in control. If it's the programmer, then I'm for it. If it's the other way around, I'll ridicule it as much as I can. Hey, there's got to be some entertainment, no?

Then there's the Java fanatics. My goodness, these people are adamant aren't they? Nothing can pierce their shield of hype. Yes, Java can compile to native code. Good for you. I could compile BASIC to native code too. Bunch of good that did me. They miss the entire point. Why would I want to go to native code? What's the purpose behind it? An assembler programmer knows. It's the flexibility, power, unbelievable speed, and the fact that any new hardware becomes readily accessible. I want to bring that power to the masses. And then when I hear someone say that Java does all this back in '95? Heck, C didn't even do this back in 1969-73. Must be fun to live in a dreamworld though. Ignorance is bliss I suppose.

This is something else I have to mention. Assembler speed is so much faster than anything else you can use. People pick out this algorithm or that. Well, that's fine, but overall the speed difference is so great that most people just don't understand that their computer could be running at least 10 times as fast and be that much more responsive. There's no reason why this speed should be wasted.

So I'm not looking for half solutions. I'm looking for the whole package. Anyone can point things out. Putting them together into a seamless and consistant solution is not so easy. And there's nothing out there that suits my needs.

Now, to those who have quoted me in a good light, I thank you. You know who you are. Unfortunately, he's as real as Java portability.

That brings up another question. Why would I want to write software to run on top of another piece of software? For games, the objective is clear. But when it comes to programming in general, it doesn't make much sense. The Java fanatics claim it works well. Ok, but why the resistance to something better? I'd rather write code for the machine instead of a virtual platform. Virtual platforms have the exact same problems as other real (hardware) platforms and bring even more problems with them. What we need is a way to program where it doesn't matter the platform, virtual or not. Now do you see where I'm getting at? I don't make a distinction between a hardware platform and a virtual one. That's why I don't see the point in virtual platforms at all since they don't physically exist. I think the problem is that people think this VM platform is somehow special. Well, no. It's like yet another computer. A limited one at that. Not to mention that it doesn't even exist.

I've also been told that I'm contradicting myself with bytecode, especially with my last entry. What people don't realise is that bytecode is just some form of intermediate representation. Is an AST bytecode? Is an RTL bytecode? Is an object file bytecode? Heck, is source code bytecode? Where do you make the distinction? It's not just that. But the Java proponents think that bytecode was invented by Sun specifically for Java. So I try to avoid having to explain this all over again. It seems like if Java does something right in its users eyes, then the arms go flailing in the air and they start running about in circles. No, even if Java did actually do something right, it isn't the whole package that I'm looking for. This falls under the pointing things out category. I can point to the moon and then point to my car. Doesn't mean I can fly to the moon with my car. That's what the Java fanatics don't get. Nevermind that by the time the car starts, the moon will have crashed into the Earth.

I will use an intermediate language. I've said this from the get-go. Now watch the floodgates open and the Java fanatics come pooring in pointing their fingers. I don't want to use bytecode in the sense that I don't want another instruction set. I want to leave enough information that it can be optimized properly on the destination platform. But this will live side by side with native code for distributed code. Applications compiled for only one platform will not even use the intermediate language at this stage. So is bytecode the answer? Obviously not. There are other issues as well that I won't go into now. But it comes back to control.

I guess my saying that I don't want to use 40 year old technology sparked a few comments. Perhaps I should have qualified that as technology that did not take off 40 years ago and are being brought back from the dead. I can only assume I'm the only one looking to the future. edit: I suppose I should also mention that I'm trying to get a rise out of some people. What I'm really after are alternatives out there if any. I want to see what's out there and decide what's the best solution for my environment.

Another thing I want to bring up is the lack of context when people quote me. If I'm talking about C, then I'm talking about C. It doesn't matter that there's something else out there if that's not what I'm talking about. For example, saying exceptions will solve a problem doesn't really help a C programmer. Never mind that the solution is not viable anyways. Lisp proponents are especially bad for this. This is how a conversation goes with a Lisp programmer:

User: "I need something good with lists."
Lisper: "Use Lisp. That's what it's great at."
User: "Ok, I'll use Lisp, but I also want something extra."
Lisper: "Use Lisp. It'll do everything you need."
User: "I am using Lisp. But I want more."
Lisper: "No no. Lisp is the solution. Everything else is just a variation of Lisp anyhow."
User: "I already said I'm using Lisp, but not everyone likes Lisp. So I want something else as front-end."
Lisper: "You have to use Lisp. It's great. You want lists, it's built for it!"
User: "I am using Lisp."
Lisper: "Use Lisp."
User: "I am."
Lisper: "Use it."
User: "Can you hear me?"
Lisper: "Lisp!"
User: "Hello?"
Lisper: "Everything is Lisp!"
User: "..."
Lisper: "Use it."

Now on to a bigger problem. The problem of writing applications. I'm talking about business applications and large distributed systems. All over, I've been criticised about reinventing the wheel and that you can't be bothered with low level minutia if you're going to be writing a huge application used by multiple corporations for example. Well, that's like saying you can't be bothered with rivets when building a cruise liner. In both cases, I'm pretty sure your ship will sink. My whole reason for repeatedly mentioning low level constructs is because you need a good foundation before being able to build larger applications. This is the whole problem nowadays of not being able to build and maintain systems larger than a certain size. My whole point is that this foundation is flawed. If the low level "abstractions" are flawed, where does that leave us? With a ship that has no chance of floating.

I'm not saying to write things in that low level. Quite the contrary. I think we shouldn't even have to deal with them. What I want is the option to be able to fix it if it is flawed and to add to the foundation if new features and hardware comes out. More than that, if you understand how the foundation is built, then you can build more stable support and platforms on top. Get it? I'm all for large scale infrastructures with the least possible effort.

I want to talk about abstractions a little too. What I've seen so far is mediocre at best. Objects? Messages? All these different paradigms? I thought I was bad with analogies, but the computing world has topped me. Objects are nothing more than a bastardisation of the procedural paradigm by giving it accessible state information. It's spagetti code without an explicit goto. If you don't like goto's, you should hate objects too. They're just as dangerous.

Another thing people forget is that as a programming environment developer, I have to be able to convert these abstractions into concrete concepts. It's all nice that the regular programmer doesn't have to deal with this, but I do. And the way these abstractions can match the underlying hardware will have a lot to do with how well the language will work on that system. By "match", I mean converts or can be transformed to be used on the said hardware. Some people think that these abstractions should not reflect the underlying hardware. It should be a way to better put down our ideas. While I agree to some extent, there should be different layers. At the very top, a completely disconnected abstraction would be ok, but there should be many layers until you can reach that level of abstraction. In today's world of programming, there's only two levels. Abstraction and reality. There's no in-between. There's no progression. And that causes a lot of frustration.

So when you built a distributed system, you should program at a higher level than you would programming a function or a class. You should be dealing with connecting different parts of your system together without worrying about small details. That's what I've been trying to get across. There should be a layer for each level of complexity. I know I'm bad with analogies, but to me, it's like putting a faster engine in a car to go faster. At some point, you're going to reach a limit. So you have to use something different like a plane with jet engines. In programming, these paradigms are fine for small pieces of software, but they'll only get you so far. At some point, you have to use bigger concepts that encapsulates all these lower paradigms. By "encapsulate", I'm using the traditional dictionary meaning.

Note that as you go to higher levels of abstractions, you're dependent on lowel level abstractions. That's why I say it's a bad idea to abstract away primitive types completely unless you're programming at a really high level of abstraction. This would require a little more than just using objects for example. I cannot stress enough the importance of a good foundation.

Finally, I'd just like to say to all bloggers that have quoted me... all four of you... I hope you'll one day understand the world I see for the future. The things you point out do not work. They haven't worked for a very long time. And the failure to recognise these failures is very dangerous. I implore you to look at the real reasons why software development sucks and to seek out answers. I've made my case and I want something better. But keep making noise and telling me where I'm wrong. I can't learn if no one tells me.

I have a problem about portability that I can't seem to resolve. I need to be able to transmit code/software over a wire.

While this may not seem like much of a problem, this is concerning a new programming environment I'm working on. Is there still a need for source code protection? In the open source community, of course not. So no problem there. In the commercial world, things are very different. You can't just give away code that the company has paid you to write. Well, you could I suppose. But don't be surprised if someone shows up at your door with some legal documents or even the police.

My problem lies in the fact that different computers may have different CPU's and hardware. Let's take this simple example:

const int limit = 8;
int a[limit] = {2,3,4,5,6,7,8,9};
int b[limit] = {18,17,16,15,14,13,12,11};

for(int i=0;i<limit;i++)
{
  a[ i ] += b[ i ];
}

Simple enough, no? Now, if I compile this in any way to CPU instructions, I have to compile for the lowest common denominator. Otherwise known as backwards compatibility. Not exactly true, but you're making your software run on old technology. While this is acceptable for some applications, it'd be nice if it could take advantage of features available on more recent CPU's such as vector processing and threads with multi cores.

In the example above, we could compile to bytecode for example, but then we need a VM (and I don't want to get into 40 year old technology) and more importantly, it doesn't take advantage of vector processing if it is available. With MMX or ISSE on the Pentium IV for example, you can process the arrays 2 or 4 items at a time respectively doubling or quadrupling the execution speed. If it's compiled down, the source code abstractions may be lost and these optimizations may become more difficult.

In statically compiled code, oftentimes the computer is queried for its features and all possible code for different versions of the CPU comes prepackaged. The problem with this is that you have redundant code, so the executable is bigger. Not only that, but usually a check is done before executing these routines to make sure the most optimized version is used, oftentime offsetting the performance advantage.

A bigger problem is that of portability across platforms and CPU's. A different CPU cannot use a different instruction set unless there is a mapping between the two. Possible, but this just becomes another kind of bytecode. Again, I don't want 40 year old technology.

Oddly enough for data, this isn't a problem. Data always comes down to atoms. There are only 3 fundamental atoms and only one is really necessary. Integers and floating point are the most common. You could also have fixed point notation if the precision needed is limited. Integer and fixed point notation are done with the same instructions. Floating point requires a different set of operations. In effect, data always comes down to 3 properties:

1. Integer or float
   
2. Size in bytes
   
3. big or little endien
   

That's all you need to know. And usually an entire program will store all numbers with the same endien-ness. That means there's only size and floating point vs. integer. Of course, other primitive types may exist, but these are the most common. Therefore, data is easily transferred. However, the problem of reverse-engineering remains for companies that wish to keep their code secret.

So this comes back down to the fact that I need to transmit code over a wire. Encryption is not an option as the computer has to be able to read the code. If the computer can read it, then so can its user. As a side-note, I think people who write copy-protection schemes don't get this fact.

In my programming environment, the above code would be stored a little different in a format not unlike that of Lisp, but with certain extensions. While it would look cryptic enough, it would not be undecipherable as you can convert it to a higher level language.

Short of using the actual source, is there any alternative? I don't see any way of supporting commercial endeavors that wish to keep their code secret. If I just use source code, or the intermediate representation of this source code, all my problems disapear. But so does all closed source corporate interest.

Catch-22 if I've ever seen it. Maybe this is why we're still using 40 year old technology. Perhaps Richard Stallman was onto something when he said all software should be free, as in freedom to view and change the source. However, I doubt he had this in mind.

Anyone see a way out of this? If not, I'll just use some form of intermediate code.

No article today. I had one written, but after checking another page, I clicked on the tab with the article so that I could submit it, but there was a delay and I clicked the tab again just as the X showed up. Bye bye article.