Introduction to Flaky Tests by Example

Source: giphy
source: giphy

Storytime!

Story 1: 5 days a month isn’t a big deal

Synthetic code representing the computation and the flaky test
Modified code patching the current time

Story 2: We use that configuration!

Synthetic code representing the problematic situation
Problematic tests cases

Bonus story: A good flakiness

Conclusion

  • Specific execution time. If the code is dependent on time, there may be failing tests at specific dates.
  • Randomness. Using random values in the main code or in the tests needs extra care or the behaviour may vary depending on those random values.
  • Modified global state. Using a global state in a project can create inconsistencies in tests if the state is not managed correctly.
  • Control your test execution environment to keep reproducible execution.
  • Avoid global states to minimize the side effects of environment settings.
  • Randomize the test execution order to determine dependencies between tests.

Continue on the subject :

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Antoine Veuiller

Antoine Veuiller

Software engineer focused on Backend and DevOps. All my blog posts are available on https://aveuiller.github.io/