Abstract
In this lightning talk, we explore the impact of adding reflective debugging to a web-based problem-solving IDE, Spinoza, that we created to support teaching programming with Python. Spinoza allows the instructor to create (or select from a library) Python problems with automatic unit tests. Each time a student attempts a new problem, the system randomly decides if reflective debugging will be required; in which case each time the student runs their code and the code does not pass the unit tests, the student will be required to classify the type of error (syntax, logic or runtime error), provide a description of the bug, and explain how they plan to fix it before they are allowed to revise and run the code again. Our main result from this pilot study is that the number of debugging steps to reach a correct solution was statistically significantly less when students were required to use reflective debugging. Our hope was that by being required to analyze each error for some problems (about one out of three), students would see the benefits and develop the habit of reflective debugging even when it was not required.