The recent release of Teacher Data Reports (TDR) in New York City made the problems with evaluations based on tests scores visible for many people. The idea of evaluating teachers based on how well their students do is a simple and wonderful idea. Using a statistical model that can factor in the student’s background and what they might be expected to learn – the value-added model (VAM) – is a great even better. The problem is while these are great ideas, the data that come from them often stinks. It isn’t always awful, but almost always imprecise and in some cases as the teacher who teaches English Language Learners (ELLs) in New York who was recently ranked at the bottom because the students she served were unable to score well on the tests for reasons outside of her control, the problems are grave. The TDR debacle has no doubt given many proponents of VAMs some second thoughts.
Some may be asking whether this is the end for the use of data for educational decisions, the death of VAMs and the reform horses they rode in on. I don’t think so. I believe that despite the many problems with educational data generally, including with test scores and anything based on them that the field is still in the early stages of using data. One simple reason is that the alternative is less attractive. Before the current rigorous evaluation approaches, teacher evaluation was based on wisps of air. Credentials have been shown to have little impact. Most districts didn’t take the quality of their teaching corps seriously enough to make changes. And, we are seeing cases where VAM data can be put to good use without the type of problems seen recently in NY and elsewhere when they are tied too closely to individual teachers AND where the underlying data systems have significant problems.
The reality I believe, and that I discuss in my book forthcoming from Teachers College Press, is that the data movement is here to stay. Data quality, once an issue for technicians and those who run reports is going to become everyone’s business pretty soon. This isn’t just because the Data Quality Campaign will be keeping it in our minds, but because without attention to the underlying data that feeds VAMS – basic stuff like rosters and teacher-student assignments and mobility – there will be problems with using the data for many useful purposes.
