This paper introduces and assesses the effects of continuous testing as a tool to aid students in introductory computer science programming courses. Continuous testing is the practice of automatically executing tests in the background of the editing application. Programmers’ tests are automatically run by a tool in real time while they edit their code. Visual feedback of test results is provided within the programming environment, which increases awareness of the tests’ states. This process alleviates the responsibility and effort of manually running the tests from the programmer, in addition to running tests more frequently and increasing developer confidence in code correctness.

Racketeer is a continuous testing plugin for the Racket programming language in the DrRacket development environment. We report on the implementation of Racketeer and a controlled experiment to evaluate its effects on novice programmers in an introductory computing course. The experiment indicates that study subjects perceived a positive effect, although there were no significant effects on development time.