Thursday, 10. September 2009, 22:56:09
This is from J. B. Rainsbergers talk
Integrations Tests Are a Scam, which by the way is a great talk that is very worth watching.
Rainsberger defines three kinds of tests:
- Collaboration tests
- Contract tests
- State-based tests
I would like to add a fourth kind to that list, called the Concurrency tests.
Let me try to explain what each of those are about, in turn.
Collaboration tests: I talk to a bunch of other guys and, assuming they work correctly, am I working correctly? In essence, we are testing the correct behaviors of the individual units in our system, under the assumption that all other units work correctly in accord with their contract.
Contract tests: someone uses this thing in a certain way and expects a certain behavior from that, do you live up to that contract? In essence, we are testing that the units use each other correctly. In other words, that we have gotten the links between the collaborators right.
For example, if we have ThisThing that expects ThatThing to return an empty list under certain conditions, then we would mock the world as such in our collaboration tests but we will make sure that this assumption holds, in our contract test. In other words, we must have a test that sets up ThatThing in those certain conditions and verifies that it really does return and empty list, and not something else, like null or a list with something in it.
State-based tests: doing someAction to someThing must produce thisObservableEffect. These are the good old super simple tests for pure functions, or very simple state mutations, or side-effects to the outer world.
Concurrency tests: like state-based tests, except the observable effect is some specific behavior of the unit under concurrent use.
Concurrency tests makes sense to separate from state-based tests because a) things that deal with concurrency should deal with nothing else, and b) testing for specific concurrent behaviors is typically harder and more complex than single-threaded state-based tests
and if they are not then you are doing something horribly wrong with your state-based tests.
If you do these tests, and design your system this way with all these things ind mind, then, Rainsberger postulates, you will only need integration tests as a debugging tool for figuring out which contract or collaboration tests are either missing or wrong. And once
that issue has been fixed, you will have a focused object test that covers the same exact case as the integration test did and so you remove the latter to get rid of the duplication.
I must say that that sounds like a very nice place to be.