In the readings for my Evaluation course, a concept struck me: when something fails, do you blame the theory or the implementation? I think that in education, great concepts often get abandoned because of poor execution and a lack of understanding about the implementation. Researchers Leviton and Lipsey describe this as not knowing the inner workings of the “black box.”
I first read about the notion of “the black box” in the context of the economics and education last fall. Economic researchers frequently write about inputs and outputs. They examine what goes into the black box of schools in terms of time, money, or resources and then measure what comes out – student test scores, teacher evaluations, etc. However, economists rarely open that box.
The same thing happens with schools when they look to implement a new program. In their 2012 article, Theoretical Frameworks to Guide School Improvement, authors Evans, Thornton, and Usinger explain that most educational reform efforts occur absent a defined theory of change. Instead, schools and districts implement new plans or programs without defining why they believe that these change efforts should work; what they hope to achieve; and how they might measure success. In other words, they often take a black box approach.
For my dissertation work, I have to define the theory of treatment for my intervention study so that if something goes wrong, I will know if the problem lies in the theory or the execution of my design. To do this, I have to define the logic behind my assumptions, the causal or correlational relationship between variables, and the literature to support my claims. This latest blog post at least helped me to wrap my head around the concept and the process.