Sometimes, no matter how much you test something, no matter how preemptive and careful you are during code reviews, no matter how much your test coverage goes up, no matter how many people look at an MR, there are the inevitable cases of logical bugs slipping through unchecked past all the walls and guardrails that are put in place, ready to wreak havoc in production against all the expectations.

Part of the problem is that during development time, during reviews, and during the usual back and forth between addressing review comments and adding the corresponding logic and tests, it's easy to become too embedded into the environment in such a way that some obvious inconsistencies can only be spotted later in the process, when the work is being used in production, which is just about the worst possible time to find issues.

End-to-end testing before hitting merge

A great way to overcome this is to do extra end-to-end testing before the code gets into production but, after the pipelines have been executed and coverage has been measured and review comments have been addressed.

Once this particular stage is reached, there is a mutual agreement between at least two people (author and reviewer) that the code about to go live is of good quality and solves the problem at hand in a correct and efficient way, so a big hurdle is seemingly crossed.

However, performing an e2e test with yet another person, exercising the code in the same way as it is intended to be used in production can reveal a lot of details that are buried underneath all the green checks in the pipelines: insert a specific entry in the DB fails, a consistency check that is filtering too much should be removed, etc.

Only after these details have been fully ironed out and experienced in an e2e testing scenario should we have the full confidence to hit that merge button.

Extra testing is awesome. Do a lot of it!