About them automated tests
In the past I tended to ask interviewers if they had automated tests for their code and I was completely satisfied with a positive answer to that question. Now after growing up, I ask 8 more questions (as a dialogue, not as a numbered list). This is because just having tests is not enough, they need to have certain properties.
Read on to find out what I think are the most important qualities of the tests you write.
About their speed
Let me get this straight. You want your tests to be blazing fast. Why? Because when they are slow you got no incentive to run them. And trust me, you want to run them as often as possible. You can think of it as running them on each ctrl+s. OK OK, it was an exaggeration, but now you get what frequency I am talking about.
How slow is too slow? Every dev has their own standards. My standards are the following: I start frowning when a test takes 50ms to run; I create a task and start looking for ways to speed it up when it reaches a 100ms of slowness.
Sometimes your code is time dependent — when artificial delays are used for example. Do NOT use real time in your test — this will make them slow af. Rather use fake time. Every worthy testing framework will provide that feature.
About their granularity
The extremities are the following: too coarse is one test class for the whole application, too fine is one test class per production class.
The “too coarse” situation is rather easy to discard, since the test class will be huge, you won’t know what exactly isn’t working when one of them fails, etc. Easily no good.
The “too fine” situation however is seen by some as the ideal case. They call these unit tests. I don’t particularly like them, because they tend to pin down a portion of the source code that is too small: ideally your production classes would be 30–50 LOC, but no more than 100. You can imagine what a burden it is to break those tests with every change that you introduce, having to adapt them each time. Such a fine granularity also hinders refactorings such as move method, inline class, etc. And refactoring is one of the most important disciplines to engage at.
The ideal granularity is to group several related classes into a class cluster and test them all with one test class. Usually the cluster borders match the borders of DDD’s aggregates. This way you keep some room for inter-class refactoring and also possibly decouple the tests from the implementation details better. I used to call these cluster tests, because of the clusters of classes, however it seems there are already a couple of somewhat ubiquitous terms for such kinds of tests — developer tests and programmer tests.
When you absolutely need a fine-grained test for some edge case, it is OK to write it, but remember to comment it out or delete it after you see everything is fine.
About their isolation
Make them absolutely independent. Don’t share any mocks they might be using, or (god forbid) the system under test — create a new instance of the SUT for each test. Some juniors start thinking if it would be a problem to share the resources, even try to engage me in that endeavor. I always tell them that I don’t even want to have to think about that and I just shut the conversation (politely, of course). Just isolate the tests completely and move on with your life without wasting any emotional energy on this.
About their length (in terms of LOC)
The ideal test is three LOC: one line for setup, one line for executing the tested method, and one line to assert the result of the test. This is almost never achievable — in fact I can’t tell you if I ever managed to pull this off during my career.
What you don’t want is a 30 line monstrosity with heavy setup, act, assert, or all of the above, sections. How to achieve this? Use test driven development. When you write the test just before the production code, you would never write a monster test, would you?
About their execution order
Tests requiring a certain execution order are trash. It means there is some interference between them. The best way to waste 2 or 3 days on a test class is to try to figure out why tests run only in the order they are written; why some run alone but not together with others or vice versa. Pure hell. To avoid this, don’t let yourself slack and randomize their execution order since the start.
About mocks
Use as little as possible — to avoid doing I/O and at the class cluster boundary I mentioned above. I have seen and created some pretty horrible messes with mocks. When used extensively, especially in fine-grained tests, you create a lot of APIs/contracts at the boundaries and so now you have no guarantee that the mocked behaviour is correct and matches the actual implementations. You start spending a huge amount of time fixing and maintaining the mocks. At the end the tests become once again pure trash that you would be better off if you just deleted them.
About putting logic inside of them
Put as little logic as possible inside of your tests. Conditionals and loops are generally a bad idea — there is a chance that you write a buggy test. And having secondary tests for testing your primary tests is overkill, right?
About testing privates
Just don’t do this. If you find it hard to target that obscure code path through a public method, it is a symptom of an iceberg class which does too much. You need to break it down into several smaller ones.
About their coverage
You thought this would be on top, didn’t you? Well, this is an overly abused metric that tells you almost nothing about your tests and production code. While no coverage is obviously bad, full coverage means nothing. Consider this — I can easily write tests that do 100% coverage without testing anything — just exercise all the execution paths and assert nothing at the end. Should it give me any confidence in my code: absolutely not. Does it give my manager any confidence about my code: yes, a lot. In short, don’t rely on code coverage too much and never have a coverage threshold in CI/CD. Actually when I think about it, it seems that code coverage is so popular because it is the only metric that is measurable. [Sad noises.]
Test code is often overlooked in favor of production code, but this is a mistake I think. If you neglect your tests, your life will be closer to a nightmare. You have to take good care of them if you want them to serve you right. Neglected tests are more of a burden than relief.
