In a recent post in EducationNext, Eric Hanushek of the Hoover Institution asked why not have open tests? An open test would mean publishing online and in advance items that might be on an accountability test. IS HE CRAZY? This goes against all of the standard accepted protocols for test administration, protocols that require strict security over the tests in advance to prevent teachers drilling students on the questions they will receive thereby compromising the tests’ validity as instruments to independently assess student knowledge. Well, it turns out he is not crazy and others (the SBAC folks I know have been discussing this also) have been suggesting similar approaches recently. There is a lesson in this.
Fundamental to the recommendation that Hanushek puts forward involves technology and volume. The technology to make test items available involves basic Internet features that anyone who has taken an online survey would easily recognize. Even adding in more complex performance based assessment items would be easy with currently available technology. Add to this ability to deliver items is the ability to easily mass large volumes of test items in a common test bank. There is nothing difficult about this. Anyone who has been involved with state test development (I studied it in my dissertation) knows that states develop far more items than most students see. They develop many knowing that only a few will be of the highest quality. They also develop more than students need because extra test versions (forms) are needed for make-up tests and other purposes. Test developers spend a lot of time and money ensuring that these items and these forms are comparable. Add in computer adaptive testing and we could be looking at lots of test items in a given state.
Since each state traditionally develops their own items, across the different states are thousands and thousands of items for each topical area. Pulling these items together and making all of them available for teachers to use is the key. Teachers could pull from the large pool for use in classrooms to give kids practice with the types of tasks they will encounter when tested. The items could also be used for in-class discussions. Anyone who has watched teachers discuss items (as I did for my dissertation) quickly realizes that all the work test developers put into even the simple multiple choice items can be used to discuss instructionally relevant topics as teachers unpack the very specific wording of test items and possible responses. The chances the students might encounter an item they had already seen could be so small it would be like statistical error. Add in a few bells and whistles and this scheme could maybe account for which items a given kid or class had seen further reducing issue of corruption of test scores.
Without getting into the substantial research benefits from all of the data that could be collected by the thousands of students interacting with different items (a fun topic to be discussed later), one important point here is how technology allows us not only to do the same things better, but to do different things to accomplish the same goal. This is a lesson from business where many companies have been able to redesign the ways that they work in addition to improving the things they had already been doing. This is an example of using technology in education accomplish the same goals (and more) in entirely different ways. Of course there are some details to be worked out such as the digital version of the content standards that allow teachers to find the items. That is for a different post.
Leave a Reply