Skip navigation.

exploreopera

| Help

Sign up | Help

There is no spoon

It is not the spoon that bends, it is only yourself

Posts tagged with "software"

BadProgrammingException or On The Issue of Developer Mistakes Exceptions

, , ,

Sometimes, in one of my Pai Mei moments, I think about introducing an exception called BadProgrammingException. Ideally, such an exception would also call svn blame and record the name of the developer who did the last modification in the code. (This might actually work from the technical point of view, if the project doesn't use branches and merges; or if you use a distributed vcs like bazaar or git).

The BadProgramming exception would be used in some of the following cases:
- on stack overflow
- on null pointer or reference
- on invalid casts
- on any exception thrown from the code that is not documented
- on value out of bounds
- on assert failures

and I'm sure you can think of some examples, too. Ideally, the introduction of such an exception would create more responsibility for the developers, and make them look twice at the code they're writing.

Of course, this is not possible and it won't work because of the human factor. But this idea raises an interesting question:

Should we try to eliminate "developer exceptions" like the above from our frameworks and products? Couldn't we avoid them in some way?

The answer is: Sure we could, and there's one simple way: design by contract. But, for some reason I can't understand, no mainstream programming language has design by contract built-in. But the possiblity exists; check for example the Nice programming language.

Instead, we are condemned to look at the code and search for all kinds of possible developer mistakes, like this one (C# + LINQ):

User u = db.User.Where(p=>p.ID=150).FirstOrDefault();
System.Out.WriteLine(u.Name);


or this one (C#):

Entity entity = someObject as Entity;
System.Out.WriteLine(entity.ID);


or, even worse, this one (C#):

Entity entity = (Entity)someObject;


In languages like C# or Java, the nullable pattern might work much better. Nullable was introduced for value types that should be allowed to be null. For example:

DateTime? dt = someObject as DateTime?;
if(dt.HasValue)
{
    System.Out.WriteLine(dt.Value);
}


(There are many other versions of syntax, including shorter ones.)

The idea is that the compiler will tell you if you try to convert someObject to DateTime instead of DateTime? (shortcut for NullableType<DateTime>). So why couldn't we use the same pattern for all types? Disallow them to be null, and only allow them when the programmer says so.

But this is just the beginning of a would-be-very-lengthy-discussion. Any ideas?


Personal advertisement: I am PassionIT and I help teams develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help. Alexandru Bolboaca-Diaconu
Add to Technorati Favorites

Cheering vs. Getting Involved in an Open Source Project

, , , ...

I know many developers, including myself, who show an undeterred belief and a continuous fascination in the open source development model, but have never participated to any open source project. Some of them, including myself, even work on commercial applications based on Microsoft technologies. Some of them, including myself, have never edited a Wikipedia page.

This is at best paradoxical. How comes that people who are such believers and supporters of open source are only users and cheerleaders instead of getting involved?

Maybe it's because we are Romanian, and we are used with such paradoxical behaviors. Or because the difference between cheering and doing is very very large. Or because programming actually is hard. Or because we like the idea in principle, but we don't have the courage to start living it. And I can think of a couple of other reasons.

Regarding myself, I tried to choose an open source project to participate in. First I tried Mozilla then Open Office and I found out that they are really complex projects and that the time to enter such a project is quite high (incidentally, Open Office recently became much more contributor-friendly, with its new website). Then I looked at smaller projects that I was using: ArgoUML and PostSharp. Still no luck - they didn't manage to capture my attention, although both are great pieces of work. I even tried to open my own open source project, something I think would have been a cool application - a "send to"-like tool allowing not only to send data to an application, but also a "send as" option allowing to transform the data into something else before sending it (e.g. "send to thunderbird as attachment", "send to thunderbird as zipped attachment" etc.). I didn't even find the time to add the document describing the mechanism and the prototype code to its sourceforge account.

There are many reasons why this happens. First, I wasn't very familiar with Linux until recently, and Linux is the best open source development tool because all the libraries you need for development and their documentation are a command line away (be it tar, apt-get, slapt-get or any other command). I very much miss the installation super-powers when I work on Windows.

Second, my time is fragmented, no matter how much I try to organize it. I have development tasks to finish, then I work on some training materials, then I think about my website, then I think about my blog, then I work with my wife on projects coming through her company and so on. And I find that many times the learning curve for an open source project is too steep, and I don't have the time to invest in it.

Third, I got used with giving my time on demand, instead of donating my time. If somebody asks me to help at a project, I would jump in the bandwagon as soon as I think it's interesting. But that's not the open source way. Rather, it's me who should find a project that interests me and to invest in it.

Maybe that's just me, but I think times have changed since The cathedral and the bazaar. The number of open source projects has exploded, and the large majority of them fail. There's nothing wrong with it; it's only the selection of the fittest, but who likes to jump in a probable failure? And, even more difficult, who has the time to look at all new open source projects and choose one that's interesting?

I'm not saying it can't be done. I'm just saying that I don't know how to do it, and that I'd very much love to participate to an open source project that fits my knowledge, my skills, and who can capture my attention. So, if you work on an open source project, please let me know how you solve those problems.


Personal advertisement: I am PassionIT and I help teams develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help. Alexandru Bolboaca-Diaconu
Add to Technorati Favorites

The Tester

, , , ...

The Tester is one of the most important member of a software development team. Unfortunately, in many projects I've worked, his/her role was severely underestimated, resulting in low product quality and high costs.

I believe this happens because testing is a more difficult task than the common sense tells us. Testing software is hard because it hits the same essential difficulty as all other activities in software development: software is constructed from ideas. It's even worse for a tester, because how can one check that the representation of an idea corresponds to the idea?

Because of this fact, a tester has to have a few key skills:
  • Smart, in order to understand the problem
  • Attention to details, in order to be able to find even the issues burried deep in the functionality and in order to be able to fill the missing details from the functional requirements (there always are)
  • Rigour, because he has to follow a process that allows him to check all functionalities, to recheck them periodically, to fill in the missing pieces and to improve the process
  • Empathy, because he needs to understand the users
  • Effective communicator, because he has to communicate both in writing (through test documents, bug reports etc.) and in speaking (with the developers, the business analyst, the project manager etc.)


If we add to those skills things like automatic testing, managing virtual machines and virtual networks, testing for security, stress testing and, the cream on the top, writing unit tests, we understand that a tester is a really complex character.

And a really important one, too. He has to work side by side with the development team right from the start of the project, because of simple economical reasons: studies have shown that bugs caught earlier are much less expensive than bugs caught late. Even a week difference between the moment when the bug is found may translate in a significant increase in costs: the developer locates the bug in a longer time, it's more difficult to fix it, it has to be re-tested and so on.

Therefore, if you want quality in your project, use a good tester and use him/her early. The price you pay for testing will be an investment, not an expense.


Personal advertisement: I am PassionIT and I help teams develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help. Alexandru Bolboaca-Diaconu
Add to Technorati Favorites  Digg!

Advice for the passionate and young software developers

, , , ...

I have gathered some advice for young and passionate software developers. It comes mainly from my experience and from my peers'. I hope it helps.

1. Be aware that programming is hard

Despite what Microsoft, Sun and other software development tools vendors want you to think, programming is hard. It looks easy though. Start an IDE, run a wizard, do some work in a designer, drag and drop database tables, automatically generate code and everything seems to work just fine.
Well, in 99% of the cases, it won't. Something will happen that will make your application fail, and because you used an IDE, a wizard, a designer and because you don't know what's going on in the backstage, you're in trouble.

2. You're not done when you've stopped writing the code

You get a task from your project manager, write the code, compile and run the application once to see if it's working. If it's working, it means that you're done.
No, you're not. You need to test it thoroughly before you're actually done. Yes, there is a tester in the team, but bugs caught early are much less expensive than those caught in the testing phase. The project manager will insist on this topic.

3. A programming job means continuous learning

The only constant in our lives is change. You feel this even more in IT. New hardware, new software, new programming languages, new tools come up every year. There are good news however: the basic principles haven't changed in 25 years. Understand them and you will be less surprised. The bad news is that it takes time to understand them.

4. The technology used is not essential

C# is good. Java is good. Perl is good. Python is good. Hell, even Cobol is good. (Only VB and old ASP are evil :-D.) Each of them for their own domain. Learn them all. The more languages you learn the easier will be to learn more, and you will find innovative and effective solutions for your projects.


5. Solve the problem

Be pragmatic. You work for a project because somebody has a problem and they want it fixed. Find the best way to solve it. Yes, there are limitations. Yes, you can't always use the ideal solution. This is why you are asked to solve the problem and not a computer. There are no recipes. You have the capability to adapt to the situation. Solving the problem is the fun part, too.

I hope that helps!


Personal advertisement: I am PassionIT and help teams that develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help.
Alexandru Bolboaca-Diaconu



Add to Technorati Favorites  Digg!

Defining an error management policy

, , , ...

This week I managed to define the first logical and robust error management policy from my 8 years career. It may sound like I haven't tried before, and it's not true.

The truth is that the right error management policy is very difficult to define.

The error that I did before was to underestimate the time needed for such a policy, and not to think of a method for it. I finally got smarter, and first tried to define such a method. That's what this blog entry is about ;-).

I realized, with some help, a simple fact:

The errors that may appear in software derive from the disruption of data flows and use cases.

So, here's the method:

  1. Take a module of the application
  2. Define the data flows
  3. Define the use cases
  4. Brainstorm on the errors that may appear
  5. Categorize the errors
  6. Define a method to treat each category
  7. Review and incrementally improve the policy

We notice here a few things.

First, error management is not generic, but is tailored for the needs of each module and/or application. There are fundamental differences between distributed systems, desktop applications, mobile applications and so on, even if we don't take into account the differences in technology.

Second, exceptions are not mentioned in the method. Even if modern languages use exceptions for error management, exceptions are not fit for all cases. If you are throwing exceptions over the boundary of a Web Service, for example, then you could get in trouble. (Been there, done that). Sometimes it's better to use error codes or compensation instead of exceptions.

Third, the response to an error deoends on the context. Valid responses are:
  • Do nothing
  • Compensate and retry
  • Retry
  • Notify the user


When choosing the appropriate response, non-functional requirements play an important role. For example, what is more important for a web service: to retry in case of an error or to reply immediately saying that an error occured? Well, it depends on the specified response time: is it more important to answer quickly or to give back results? This kind of questions is application specific.

So, to summarize, what did I find out about error management policies?
  1. They are NOT easy to define
  2. They depend on the context (application, module, non-functional requirements etc.)
  3. Exceptions are NOT the ultimate answer


You also have a tried and tested method for defining exception handling policies. If you find this useful, I would like to know ;-).


Personal advertisement: I am PassionIT and help teams that develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help.
Alexandru Bolboaca-Diaconu



Add to Technorati Favorites  Digg!

Software design: basic principles

, , ,

6 months ago I delivered a training named "Object Oriented Design for Developers". To be true, it wasn't only OOD, but also contained design ideas from Functional Programming; it should probably be called "Software Design for Developers". Its purpose was to help developers make the leap to software designers. It tried to give them the principles, mindset and techniques used in designing software. The training should have been followed by practicing in a real-life project, under supervision.

To be fair, I have to say that it was too theoretical, missed laboratories and examples and was probably too difficult to digest for most attendees. That's why I plan to completely rewrite the training materials and to add the missing pieces. But I digress.

There are a few basic ideas that need to be understood by anyone who wants to be a good software designer.

Any person's brain is too small to be able to design complex systems.

This doesn't apply only to software, but to all areas of engineering. What does it mean? We shouldn't design complex systems? Certainly not. We should continue designing complex systems, but keeping in mind the second idea:

Because our brains are too small to design complex systems, we need to manage complexity.

In the pure spirit of this blog, I can only notice that this is related to the human factor:

All design methodologies we're using are driven by limitations in the human brain.

Structured Programming, Object Oriented Programming, Functional Programming, Service Oriented Programming and so on are just methods we're using to manage the complexity, because our brains are too small to easily and completely design complex systems. If you want to design software, it's very important to understand those ideas.

Starting from those three ideas, people have derived rules for software design and development. Let me enumerate ten of them:

  1. Rule of Modularity: Write simple parts connected by clean interfaces.
  2. Rule of Clarity: Clarity is better than cleverness.
  3. Rule of Composition: Design programs to be connected to other programs.
  4. Rule of Separation: Separate policy from mechanism; separate interfaces from engines.
  5. Rule of Simplicity: Design for simplicity; add complexity only where you must.
  6. Rule of Robustness: Robustness is the child of transparency and simplicity.
  7. Rule of Least Surprise: In interface design, always do the least surprising thing.
  8. Rule of Silence: When a program has nothing surprising to say, it should say nothing.
  9. Rule of Repair: When you must fail, fail noisily and as soon as possible.
  10. Rule of Extensibility: Design for the future, because it will be here sooner than you think.


If you are interested about learning more on the subject, read this. For some people, it may be surprising ;-).

I'll talk about it soon.


Personal advertisement: I am PassionIT and help teams that develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help.
Alexandru Bolboaca-Diaconu



Add to Technorati Favorites  Digg!

The Ego in Software Development

, , , ...

I have recently stressed the importance of having a team formed from very good developers in order to obtain software quality. I received interesting feedback regarding this statement, including one issue related to the ego of programmers.

The problem goes like this: you gather your best developers and ask them to work on a project. Instead of producing much more than the average programmers, they fight over design, libraries and principles.

I believe that there are a few key problems when this happens. First of all, design by committee has proved to be an anti-pattern. Second of all, because software development is about ideas, all projects should start with prototype implementations that are refined during the project's consequent phases. Third of all, there's the Ego Problem.

It is easy to see that developers often have an oversized Ego. That aspect can be very good or very bad for a project.

Let's take for example Linus Torvalds. Whenever he talks in public, he says "I'm always right, you're wrong. If I don't like what you say, I may say you're ugly and stupid". That's certainly an oversized ego.
On the other hand, nobody can say that Linux is not a succesful project. More than that, Linux is organized in such a way that Linus Torvalds doesn't control all its phases. And there's even more to the story: Linux might not have been so succesful if Linus Torvalds had a smaller ego.

I've also met the other type of developer. The one who thinks that knows everything, the one that doesn't agree with older and smarter people from the industry. The one who thinks his ideas are very bright when they actually are really bad mistakes. The one that tries to impose his solutions to everybody else.
I guess that this phenomenon takes place in Romania because the IT industry doesn't have tradition, doesn't have values, and it's not settled yet (actually, like the whole society). People still try to be original when they should just listen to the voice of the experience.

But what is different between Linus Torvalds and John Developer My-Solution-Is-Always-The-Best?
You can probably name a lot of differences. Knowledge and understanding are a few of them.

I believe that the main factor differentiating a Good Ego from a Bad Ego is the intellectual honesty.

If you read what Linus Torvalds says, you will see that he has no problem to point out his mistakes and his limitations. That's normal, because he works in an open environment, where all mistakes are noticed. If he doesn't talk about them, the others will.
At the other end of the spectrum, John Know-It-All will never accept he's mistaken, even if you prove it with all the resources you can. He will find out all sorts of ways to prove them wrong: "Those people don't do what we do", "This is theory", "I haven't seen this work" etc.

So there's one point to look for:

A person with a Good Ego is his/her first critic.

Intellectual honesty also includes the willingness to learn. Not only the technology you work with, but everything related to software development. Yes, that's hard, takes a lot of time and it's frustrating because you always find out you made mistakes. But software development is hard.

Continuous learning is the best way to transform a Bad Ego into a Good Ego.

My experience includes many moments of enlightment and of realization of my mistakes. I've come to understand that mistakes are part of life and that we should take maximum advantage from them. Understanding our mistakes and finding ways to correct them is how we evolve intellectually. During this process, a Bad Ego is transformed into a Good Ego.

The answer? Look for people who are intellectualy honest. This is an absolute requirement for what I call "good programmer".


Personal advertisement: I am PassionIT and help teams that develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help.
Alexandru Bolboaca-Diaconu



Add to Technorati Favorites  Digg!

3 Hidden Costs in Software Projects

, , , ...

One of the core reasons for which software project fail is that managers and/or customers don't know the hidden costs of software development. Here are three of them.

Hidden cost #1: Unneeded complexity

Complexity is the enemy of a software project. Not only it undermines the productivity when developing the "too complex" feature, it also undermines the productivity of developing related features, of debugging, and of fixing issues. It undermines the moral of the team, produces headaches for developers and managers, and can also result in loss of personnel.

Complexity is fought with a variety of means, but all seem costly to an untrained eye: using good developers, good design, design reviews, code reviews, refactoring.

When complexity is well kept under control, the final costs of the project go way down. First of all, developers are happier, they are working better and get better results. Second of all, managers don't have to fight with them to get them motivated.


Hidden cost #2: Security

Only recently security has become a major ocupation for the commercial software developers. This only happened because of painful experiences and because it became clear that not taking security into account results in major losses.

Security is difficult to implement in a software. The only way is to use a holistic approach, to think of it right from the start and all throughout the process. It also implies additional development costs: training the developers on security issues, continuously monitor the code from a security perspective, implement specific security-related mechanisms that wouldn't be needed for an application deployed in the void.

Hidden cost #3: Choosing cheap but bad developers over good but expensive ones

Choosing cheap developers seems a very good financial decision. But there's a catch.

Good developers are often 10 times more productive than bad developers, and they are never paid 10 times more. On the other hand, if a team is formed of a lot of good developers, other will want to join it. If a team has only a few bad developers, either the bad developers or the good developers need to leave.

I've seen projects where good developers were fixing during the night what bad developers were implementing during the day, and they were not always paid for it. This is one reason for which this cost doesn't reach the financial balance. So, stop doing it!

So there you are: 3 hidden costs in software projects.

UPDATE: Martin Fowler talks about this third hidden cost in his blog.


Personal advertisement: I am PassionIT and help teams that develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help.
Alexandru Bolboaca-Diaconu



Add to Technorati Favorites  Digg!

Entropy in software development

, , , ...

Quite often I hear developers or managers acting very surprised about the fact that an application "doesn't work".

I am never surprised when an application doesn't work. Probably because of my engineering background, and probably because I can still remember the second law of thermodynamics:

The entropy (=randomness, disorder) of an isolated system which is not in equilibrium will tend to increase over time, approaching a maximum value at equilibrium.

This law states something that we all know intuitively: it takes effort to move things in the desired direction. It doesn't matter whether we're talking about a carpenter trying to build furniture, a cleaning person trying to keep a room clean or a software developer trying to develop an application.

This means that the normal state of things is for an application not to work according to specifications. That's part of the reason why programming is hard: you have to fight entropy.

The reason why people are surprised with an application that doesn't work is that they don't realize just how hard it is to fight entropy in a software project, avoiding in the same time the other big enemy: complexity. Many people don't understand that programming actually means working with ideas.

What comes as a natural consequence of the above law is that I am always nervous when an application we develop seems to work fine. That's just not natural.
When I say "seems to work fine", I mean that there is no reasonable proof of the fact that the application is working fine. Of course, we can't prove that an application just works (no matter what marketing tries to tell us). But there are many examples of projects that were driven back because they lacked this proof and because they discovered at some point that inside the seemingly good working box there were many lurking bugs.

The first rule that results from this reasoning is:

Never trust your developers when they say the application is working fine.

Only trust them when they prove it to you with: unit testing, functional testing, beta testing (with real users) etc. Developers' mindset is about making the program work, not making it fail. That's why not many of them are good at testing their own modules.

In the end, remember that it's normal for an application not to work. It's unnatural when the people working on it say that it works without investing too much effort into it. The entropy is hard to beat.


Personal advertisement: I am PassionIT and help teams that develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help.
Alexandru Bolboaca-Diaconu



Add to Technorati Favorites  Digg!

The essential difficulty of software development...

, , , ...

... is that we are working with ideas, not with material items.

The relation between people and ideas is fundamentally different from the relation between people and material items. For one thing, material items are perceived in the same way by different people (unless some of the participants are under the influence of some perception-altering substances). Material items can be described completely by diagrams and numbers.

The IT industry has gone to a great length of defining ways that would allow completely describing ideas. We have UML diagrams, templates for writing technical documentation and so on. We have ways to describe data and the transformation it suffers. We also have ways to describe the user interface.

The problem with the description methods we're using to describe ideas in IT is that:
  • It's really difficult and time consuming to completely describe an idea (try to read Kant or Hegel to realize how difficult it is)
  • Different people understand the description differently
  • It's difficult for people to understand the description completely, to take into account all the details

Besides this, ideas evolve, and the people who need the application expect them to be easily included in the implementation. It's very difficult to dissociate an idea from its programmatic representation. Ideas are easy to modify; the implementation is itself immaterial; so why is the implementation so much difficult to change than the idea itself?

In order to prevent misunderstandings from happening, we are testing the "materialization" of the ideas; in other words, the implementation. But how do you test an idea? It's not a matter of fitting a mechanical piece in a mechanism. It's about comparing the mental image of the idea with the actual implementation. But you can't actually touch it, move it, look at it from all angles; at least not physically.

And then, there's the relation between users and their applications. The computer is different from any other tool known to man because of a few key characteristics:
  • It processes information and the result is information
  • It gives good results only when given good input
  • It gives wrong results in any other situation
  • The user cannot touch an application

Therefore, the user has to form a mental model of the application and of how it works. So we are back to the same problem of perception of ideas. The user's mental model about the application is not bound to be the same as the one of the developer, of the producer and so on.

This essential difficulty is responsible for the most problems we face in software development. And, because it's an essential difficulty, there are no ways to avoid it, only to alleviate its effects.


The first solution that comes into mind is to minimize the communication chain. If the mental model gets more corrupted the more it travels then make it travel less.

Open source is a succesful model because the people writing the software are the same people who need it, and they can take decisions regarding it. In other words, they are bound to have a similar mental model because they need the same thing, and they are free to adapt the ideas.

Agile methodologies are succesful because the communication chain is the shortest possible. The customer is part of the team and he is the one explaining the mental model directly to the people responsible with materializing it.

All the other methodologies have proven in time to be impractical. Completely documenting an application in UML is less productive than writing the application and adapting it as it goes. In fact, completely defining an application means that we are creating a complete representation of the application in another language, and isn't that exactly what programming does? Except that you can run a program and you can, with some extent, use your senses with a program. That's impossible with another abstract representation of the same idea.

The other way to alleviate the difficulty is to use the right people for the job. Since defining a mental model for the application is a human problem, it's dependent on the individual. People with the right mindsets and with a certain mental agility that allows them to adapt to exotic mental models manage to form a mental image that's much closer to the one that's transmitted to them. And, with experience, they come to the point where the fill in the missing parts with the correct pieces of the puzzle (for a very interesting discussion on the expert mind, see this article).

To summarize, if you want your software project to be succesful, do the following:
  • Minimize the communication chain between the vision and the implementer
  • Use the right people in your team


I believe this to be the basis for any succesful software project.


Personal advertisement: I am PassionIT and help teams that develop high quality software. I train, I help improving and I work hands-on, depending on the needs. Contact me if you need help.
Alexandru Bolboaca-Diaconu



Add to Technorati Favorites  Digg!