Following up on its award-winning reporting on a cheating scandal in Atlanta’s public schools – which spurred the retirement of Beverly Hall, the district’s superintendent. Subsequently, 36 people were indicted on criminal charges related to the cheating, including Hall. the AJC is now looking at testing errors that occur before and after the questions are put to the students. Veteran reporter Heather Vogell of the AJC’s investigations team spoke with EWA. We also did a Google Hangout on this topic with Vogell, which you can watch above:
- You did the bulk of your reporting during your year as a Spencer Education Fellow at Columbia University’s journalism school. Could you have tackled a project of this scope while on your regular newsroom assignment?
The paper’s been very supportive of the testing reporting we’ve done over the past few years. But like all papers it’s very hard to step away from the daily grind for that long. The research time spent on this series was deep and technical. I spent a lot of time talking to a psychometrician – we actually had weekly meetings. The fellowship really allowed me to wear the researcher’s hat a little more easily than most reporters in a newsroom today, even if they have the level of editorial support. - The series will likely be used as ammunition by those who contend standardized testing in schools is out of control. What’s your reaction to that?
I wanted to write the story from a perspective that would be concerning and engaging for people whatever side of the testing debate they were on. People who want tests used for accountability purposes want them to be good tests – well-designed, well-implemented and well-funded. There are people fighting within the system to make these tests better. At the same time, if you’re going to implement testing policies that are, in some cases, still so draconian, you need to be aware of the impact and consequences for kids. But of course I knew people who oppose these tests would use this as more evidence of their flaws. People on the ground don’t say they never want their kids tested. They’re concerned about the quality and implementation. That makes me think there’s some common ground that can be reached on the use of measurement in schools. - What was the surprise for you in your reporting?
It’s always surprising when you start digging deeper and realize how even with a well-intended and seemingly well-designed system of quality control checks, things can just go terribly wrong. I saw that again and again. These testing companies have processes in place and certainly make an effort to make these tests error-free. Perhaps if you looked at their policies you’d say “That looks pretty good.” But when that meets reality it’s amazing how often that can go wrong. Some of the problems stem from the time constraints and the money pressures. But you also have to have a deeper understanding of the subjectivity that’s built into testing. It’s sold as a scientific enterprise. But we have all these human actors making judgment calls all the time. - Your story focuses on the impact testing errors had on actual students. How difficult was it to find them?
It was really hard to find people. I made about 100 phone calls in Mississippi alone, and I never got to talk to a kid. What was distressing to me is that many of the students who were affected were already on the bubble — kids who were close to the cutline (for a passing score). They’re the kids who didn’t have as much confidence to come forward and talk about it. The ones who would talk to me were academically confident already. They’re probably not the most serious victims here. - Can you give us an example?
I had a mom email me about their son who had an “A” grade in his high school science class, took the end-of-course exam and failed it. They tracked the answer sheet down and there was a problem in how it was scored. This was a student where there was a history of academic confidence and an involved parent, which gave them a lot more confidence in challenging the result. Not every kid has those advantages.
