Software Development

There are those who call themselves hackers. Then there are those who write hacks. Oddly enough, these are two different categories of hacking, but there is some overlap. A hacker would usually take pride in their work and admire the clever tricks and techniques they used. Today, that's not seen so much. What is still commonplace is the creation of hacks. A hack is usually something quick and dirty to do a simple job or a temporary one. Converting a text file into a better formatted file is still quite common. There are tons and tons of this kind of legacy data still hanging around. What about testing? Most of these are throwaways because software never stays the same. These are hacks too. There will always be a need for hacks even in the most professional software company, though they will not call it as such.

So what do you write your hacks in? Usually, it's in the same language as the main software if there is one. Sometimes, it's just the favourite language of the person paying you. Or it can be the favourite interpreted language of the programmer. My personal experience is that hacks are usually written in interpreted languages. The reason is that interpreted languages don't bother you with a lot of useless details that just get in the way of writing hacks. You don't care about types. Just give me strings and integers. The rest be damned. Give me a way to open and close sockets, read files and the rest can be... well... hacked.

I used to use BASIC a lot for quick hacks. Then with the web, Perl was like a treasure trove. It really was write once and throw away because the mess was unreadable afterwards. But it's great for creating software from scratch. I haven't use Ruby or Python much, but I wonder if these languages don't suffer from the same thing. I'm sure this will anger a few people. Of course, I could be completely wrong. It's just a feeling I get from what I hear.

Creating software from scratch is great and all. It allows one to get started awfully quickly and for most beginners, this is what turns them away the most if they don't get rewards for their efforts fast enough. It's like manual memory management. It's frustrating at first. Tracking other resources is also annoying. Syntax errors. And the grand daddy of frustrations, irreproducible or erratic runtime errors.

What's happening now with programming languages is a trend where they try to remove the most annoyances possible. While I agree this gets you started faster, I've found that software collapses quite quickly after a certain size. If you have automatic resource management, your software will be bloated and slow. Realise that if your software is small, the bloated piece of software is not of any concern. But as in budgets and compound interest, every little bit counts in the overall prognosis. It adds up FAST!

All this matters little though when productivity is up and costs are down. The thing is that I'm not sure programmers today realise what's going on. Anyone coding in the 80's and 90's knew instantly that if you used something that got you going fast, it was a hack in an interpreted language. It wasn't going to be used for anything large scale. Over time, small outfits had to create software fast. So things like Visual Basic made a huge impact. Then I remember Java starting to take over the masses even though it took much more time to get started in.

At some point, hack languages crossed over into the generally accepted notion of regular programming languages made for large scale software. What I find fascinating is that instead of having two different sets of functionality for different tasks, it seems everything must fall under one language in an organisation. I kind of see the reasons why, but if a programmer cannot learn two languages, I wonder where the computer industry is heading.

Case in point, games often have scripting languages built in. But the main software isn't built using this scripting language. A mod is basically a hack. It allows you to change things in the game rather quickly. But no one would think of writing the core engine with it. Games have real requirements of handling resources and user input with optimal performance. General programming doesn't seem to worry about this and it's curious as to why.

Another case in point. On sourceforge, the top software is Azureus. A java based P2P client. It loads up and runs fairly well. It's one of the pieces of software that I thought might change my mind about Java in general. Then I saw uTorrent. The software is a whopping 173K total. It's much faster than Azureus and uses under 6MB of RAM under normal circumstances. I can't even load the JVM with 6MB. The responsiveness is also far superior. Not only that, but it uses almost zero CPU time. I cannot say the same for Azureus as much as I like the app. In fact, this is what made me look for alternatives. One time, it was the JVM itself that caused it to use 100% CPU time because of version incompatibilities.

My point is this. What if software development kits were a bit more like how games are developed? Where you have a main language as well as a scripting language that can plug into resources of the main app. The web is the worst example of not doing this. Client pages can only use JavaScript. And I don't know what it is lately, but its usage of complex and slow code has taken up an exponential leap. The MSNBC, ABC News and technorati sites are just horrendous now. So maybe I have a slow machine. Maybe I don't. But one of the strengths of the web was to have easily accessible information on even the slowest of machines. When you need 2Ghz just to view web pages, it makes all those notions about the demise of desktop apps fall flat on their face.

Desktop apps made with compiled languages have a solid core, but it's inaccessible from the outside. The web has some accessible features, but no solid core that you can upgrade or modify.

The art of hacking is only as great as the core that makes it possible. Hacking, no matter what kind, has always been about modifying something greater to suit your needs. If you find yourself programming and you cannot identify the core of what you're controlling, then your code is the core. And if you're using a language that has a low setup requirement, then your core is a hack. That's not what you want. With interpreted languages, the core is always the interpreter or the VM no matter what. Your code changes the interpreter's behaviour and gives you access to its core services. Again, that's not what you want. You want a scenario like games where your use of its scripting language can affect the actual software. An interpreter or VM acts as a really poor example of a "core" software.

What it comes down to is that hacking is fun. Everyone wants and loves to do it. Humans have always been able to take something that exists and tweak it to suit their needs. We're creative creatures. Rarely can one come up with something entirely new on their own. So maybe this is my best explanation of what Project V aims to be. It is to be the core that will allow the resurgence of humanity's natural ability to hack. To newcomers, this may be a scary thought. In actuality, it will be a much cleaner process than how software is created today. The ability to change, remove, insert, tweak, convert or update any part of your software is hacking. Strangely enough, these are the things that the software industry is struggling with.

There's a real reason why plugins, scripting languages and widgets (as in Opera) exist. They're an extension of the hacking concept brought to a cleaner conclusion. But most of these are independent of each other. I intend to provide "plugins" and other things that allow the interconnection of multiple "cores". So providing extra functionality won't be about writing a function. It'll be about adding a module that connects different things together, sorts the data and provides it at the appropriate location. User interfaces does exactly that for example. It connect *you* as a "core" with your own set of functionality to the "core" provided in the software. This will allow one to hack the human experience a little easier than it's been in the past.

Hope this has provided a little different perspective on where I'm coming from. The ability to add onto existing software is something too great to pass up. And with interconnectivity comes the power of distributed computing. So that's how all these pieces start to fall together. You no longer need actual code to run on the same machine anymore, or at all, thus decoupling the core from its plugins. This will effectively remove the need for the Web as we know it though you can still make use of it as a (HTTP) plugin. For the first time, the Internet will have a core that you can plug into and shape it into any way you wish.

(As a final side-note, contrast what I spoke about with Web 2.0 and you can start to see obvious problem areas. There is no core and no interoperability. The Web itself is not a core. It's not tangible. Project V will be.)