📝 SummarXiv

Abstract

We present JS-TOD (JavaScript Test Order-dependency Detector), a tool that can extract, reorder, and rerun Jest tests to reveal possible order-dependent test flakiness. Test order dependency is one of the leading causes of test flakiness. Ideally, each test should operate in isolation and yield consistent results no matter the sequence in which tests are run. However, in practice, test outcomes can vary depending on their execution order. JS-TOD employed a systematic approach to randomising tests, test suites, and describe blocks. The tool is highly customisable, as one can set the number of orders and reruns required (the default setting is 10 reorder and 10 reruns for each test and test suite). Our evaluation using JS-TOD reveals two main causes of test order dependency flakiness: shared files and shared mocking state between tests.