Code Rewrite - Yes
Tuesday, September 25, 2007 1:44:15 AM
So should I backpeddle? Should I recant my statement about rewrites? I mean, I've seen my fair share of bad rewrites. No, I think rewrites are fucking fantastic. If your software is in its first incarnation, I say a rewrite is a good thing. But it's a good thing only as it relates to your code base. You'll have to decide if it's actually worth it. Is the time and money spent justified? That's another discussion. I want to talk about the code itself.
I've seen bad rewrites, but have NEVER, not once been involved in a bad one. Frankly, it's because I rewrite things when the original is so bad, that it can no longer be improved upon. Sounds like a contradiction, but any programmer has seen code so bad that any modification will result in a reduction in functionality no matter what you do.
The problems that arise in a rewrite are many. First, funding is a problem. I don't want to talk about this too much, but when you do a rewrite, managers and higher ups feel like money is going down the drain. The pressure on programmers makes them uncomfortable and this is the #1 reason why software fails. Programmers don't feel compelled to test, do trial runs, write unit test or do any of the things that are worthwhile because they don't want the finger pointed at them for wasting more time and money. If you have this kind of scenario, get out now. JUST ONE NEGATIVE THOUGHT WILL FAIL THE PROJECT! ONE! I'm not joking. The absolute worst thing a project can have is one single negative thought. After that, it snowballs and game over.
You can and will have obstacles. But everyone should always think that it's possible to succeed. The moment you start to doubt this, the project is over. Obstacles need not be negative. A setback is just that, a setback. You can keep going. A negative thought is one where you move backwards. Where you don't think you can advance, ever. Those are the ones that kill projects.
Every single rewrite that succeeds is WAY superior to the original. Not by a little, but by incredible amounts. That sounds like an obvious statement. Of course, if a rewrite fails, it'll suck. And a successful one will rock. Well, here's the secret to a successful rewrite. You need the original programmers doing the rewrite. If they're gone, kiss the rewrite goodbye. It's as simple as that. If you use new programmers, they will hit the same problems as before, only this time, they won't be tested and won't work.
There's only one kind of situation where I've seen a proper rewrite happen successfully with new programmers. It's where all inputs, transformations and outputs are documented and all special cases are likewise written down in documents. With this amount of documentation, you need to sort it. So thus begins the organisation and the new programmers are forced to think about how this all fits together and also to think about a framework. Another thing. These new programmers CANNOT have access to the old code. They cannot see it, look at it, poke it, or even touch the back of a screen where it is displayed, even if the person is blindfolded. The reason is what I just said. You don't want the new software to be a recreation. Recreations mean a reproduction of not only its features, but of its problems. So you'll just end up with a duplicate of the original, just without all the bug fixes.
Another way to make a rewrite successful is to do it incrementally. In my experience, these have the most chance of success and are the easiest to implement. What you do is start with the original software, but basically butcher it. You rip out everything you don't need. Only leave the core and rewrite the protocol that all the modules in your system will use. Write conversion routines that can translate back and forth between the two protocols. This means you can no longer call a module directly. You'll be amazed at the modularity you'll get just by doing this. It'll force you to rethink your implementation. Then start replacing the modules that you need for the new core to work properly. Then start rewriting the core systems. Some modules may be merged, changed locations or have a different set of functionality. Just as long as the overall operation remains the same. I once took a Java server and made all modules callable by sockets. I rewrote the modules in C++ and eventually left certain parts in its own process. I could then run different parts on different machines when this wasn't a requirement. What it did was give me more flexibility later on.
The biggest problem I've seen with software is that it's monolithic. There are no languages out there that are pluggable. Functions are the anti-thesis of modularity. Ironically, it's being incorrectly promoted as the best tool for this. Unless you're writing a library, no such luck. Most software start out small. You don't need plugins and a framework. You just need a few features. But at this point, all functions call all others. There's no way to insert more code without modifying the existing source. That's where rewrites should concentrate their efforts. If you rewrite just for the sake of a rewrite, don't bother.
And here's the weird part. If you want to do a rewrite, you should want to rewrite the core systems. In short, you want a new framework. You can't write a new framework in languages that force you to use theirs. This includes ALL languages that have an interpreter or VM. C, C++, Pascal, etc. are still the best tools to do a rewrite. It simply makes no sense to rewrite something on top of the same framework. What are you getting? Nothing.
Think of successful rewrites. One example is Linux. Linus used Minix until he had enough pieces in place that he no longer needed it. uTorrent is also a good example of a successful rewrite. Again, there's a framework on which to build on. It may not be obvious, but there are plenty of existing P2P files being transferred right now.
Rewrites are absolutely fantastic. But you have to know what you're doing. If you get someone that has never done one and knows nothing about the original software, then good luck. You'll need it. Rewrites are in the realm of optimizations. They make your software work better. Most programmers have been told not to even look in that direction. It's not that rewrites are bad. It's that it's not a practice that is taught in schools, Universities or Colleges. And there's the bigger point. The main point. The crucial point why people would rather avoid doing a rewrite. It's that normally success is just getting something to work. With a rewrite, the bar is set SO much higher. So high in fact, that you have a concrete measure to compare with. All of a sudden, there's accountability. That's the real reason you shouldn't do one.
Rewrites are actually very easy. Everything is already there. You have MORE data and test cases at your disposal than you could dream for. The reality is that it exposes just how good a programmer you really are and if you can do what you say you can. And about surprises and undocumented features or side-effects. I don't want to hear it. That's your job as a programmer to assess these things. That's how every other field works. There are risks. It's your job to ascertain them. If you decide to start one, you best make sure you have everything you need. Just like any job, there are some that are too risky. But saying that you should avoid all rewrites is a cop out.
After the initial rewrite, you should actually continue rewrites. While you can think up many reasons to do a rewrite and probably more for not doing one, it comes down to this. You rewrite code because you're stuck modifying existing source code. What you want is a system where you NEVER touch the original code except for bug fixes. You don't touch it for adding functionality. You don't touch it to add a module. You don't touch it to refactor. You don't touch it for ANY reason except to fix what should have been working in the first place. If your code doesn't do that, then this is a good reason for a rewrite. It should be only one reason out of two for a rewrite. The other being that the original didn't work.
So this should make it clear what you do with a second rewrite. You don't do a complete rewrite. You basically rip out all the parts that have been added since the last rewrite and make it work in a more consistent way with what's already there, rewriting much of that too if need be. Normally, a second rewrite would only be 20% of the total source tops. A third rewrite is 5% or less. In fact, everything I do is made of rewrites. Every iteration gets better. It is my contention that rewrites are done too late. They wait until they must change everything.
I say yes to rewrites. I rewrite until I no longer need to touch the source code. If I'm touching the source code, then I've done something wrong and it should be rewritten. And if I'm the only one who thinks this way, then so be it. I don't miss not having big balls of mud.