Monday, January 31, 2011

What if Tests Don't Test What You Want to Test?

When my daughter was in elementary school, she came home with a couple of quizzes on which she'd scored zero or one out of ten. The topic was "greater than / less than". There were two columns of numbers and she had to draw a greater than or less than sign between them. (e.g. 5 less than 9).

I created similar quiz sheets. I cut out a bunch of little squares with the mathematical signs on them, and gave her a glue stick. Instead of drawing the sign, she could glue it in. She put in all the signs correctly, every time. Ten out of ten.

So my daughter's dismal marks on the quizzes did not indicate whether she knew the difference between less than and greater than. They did not indicate whether she knew which sign to use. They indicated whether she had the capability of drawing the sign.

As you might guess, my daughter has a learning disability. But the lesson here goes beyond that. Teachers of all grades should be aware that many times the questions on quizzes and exams are testing something other than what the examiner intended to test.

I remembered the episode with my daughter when I made a summative exam for my Workplace Math class last week. (I had taken over the class from a teacher midway through the course.) The exam consisted of 30 questions down the left of a large sheet of paper with answer boxes on the right. I gave the students all the answers on slips of paper, and a glue stick. They were to arrange the slips in the proper places and glue them into the boxes. (Students could write the answers if they wanted.) The student with the lowest mark in the course to that point (38%), scored a 70%, the class average on the exam.

I wonder what the earlier evaluations, the ones that rated him dead last, were actually testing.