Many of the proponents of educational data will use the phrase that “technology is the easy part” and then explain why changing cultures and behaviors of people using the data is where the real challenges lie. On the face of it, this proposition makes a lot of sense. After all, technology and technologists should not hold the keys to progress. Many information solutions that are needed can be engineered. This is how industry after industry has seen technological progress and why should education be different? I argue in my forthcoming book that viewing technology as just subordinate to the social and organizational issues is incomplete. In practice there is often a complex relationship between the social and the technical. Here are three reasons that this sociotechnical view makes sense in the area of educational data use.
1. Technology can be an enabler of new ways of seeing things and new practices
2. Technology choices can have big implications for practice
3. Some education technology problems are proving not so easy
Technology can enable of new ways of seeing and new practices
The field of education has many examples of where a particular type of technological innovation made other things possible. One example of this is the development around 1980 of multileveled statistical models that helped refute some of the erroneous findings of the Coleman report that schools made little difference in overcoming student circumstances. Once these models came along, researchers were able to use large datasets to find that in fact different schools working with the same kinds of students could produce different results. There were after all school effects that could be shown and then different aspects of leadership and culture could be related to those effects.
Today, linked longitudinal data that shows not only student growth, but the teachers involved in that growth is making other realities of educational practice apparent. It has shown that teacher credentials do not predict their performance in classrooms, which helped undermine an important part of NCLB and provides alternative preparation programs with conceptual validation. Researchers using this type of data have also been able to show the common practice of assigning new teachers to the hardest students. This has been one of education’s dirty little secrets and one that carries with it some troubling ways of viewing teaching that with experience comes the right to avoid educational challenges. Without the longitudinal data sets that linked teachers to students these insights would not be possible.
Technology choices can have big implications for practice
As the debates about value-added models show, some technology choices can actually make a difference in practice. The linking of student and teacher data and selection of one or another type of value-added model to evaluate teacher performance makes a difference in the lives of some teachers. Some can be dismissed and some can be rewarded. Further, it seems that using different models can lead to different types of results so that rather than there being one true teacher effectiveness value, there are several depending on the models chosen.
In a related area we see the mechanism for linking teachers and students in data systems, also called the Teacher Student Data Link (TSDL) presents some important choices regarding which teacher(s) are given responsibility for the growth of students. In cases of special education where more than one teacher routinely interact with a student, which teacher is assigned the student for the purposes of measurement? Some states are choosing to pick one teacher. Some states are assigning both 100% of the students’ scores. Some states are choosing to have a partial score assigned to each teacher that will total up to 100%. These choices are not neutral. They have the potential to impact how teachers respond to being accountable for student growth and are examples of the technology being able to shape practice.
Some education technology problems are proving not so easy
While It would be nice to put technology in the backseat to issues of policy and political will, the reality is that data quality and the reliability of educational data have continued to be difficult problems. The biggest of all of these is the educational measurement. When we consider other fields such as medicine and finance the kinds of error and variation they have in their foundational data is much less than what occurs in education. A thermometer may measure slightly different temperatures on each use, but that variation with modern technology is very low and the variation is usually independent of the patient. The same for blood pressure, etc. In finance, the details of a financial transaction are almost always invariant. Now, in education a foundational measurement is student learning. These measurements under the best of circumstances have much greater variation and in many cases they vary by population.
Looking at educational data, there are hosts of quality and reliability issues that affect how the data can be used. These include problems with class rosters, assignments, and any data that requires or allows for human input. There are cases of cheating and misrepresentation. Also, there are many issues of comparability in education where things such as districts, schools, and teachers seem comparable, but in reality vary in ways that challenge the technologies we have. These variation and comparability issues impact what the technology can provide.
Summary
While I would not argue for technological determinism, the idea that the technology will lead to certain results, I do believe that the technology can’t be taken for granted. Anyone on the front lines of working with educational data knows this is the case. There are many barriers to using information to improve educational performance that depend on the technology.
