Pearson...s SAT Scoring Errors Affect 4,000 Students
By Karen W. Arenson and Diana B. Henriques
THE NEW YORK TIMES
The scoring errors disclosed this week on thousands of the College Board’s SAT tests were made by a company that is one of the largest players in the exploding standardized testing business, handling millions of tests each year.
The mistakes, which the company, Pearson Educational Measurement, acknowledged Thursday, raised fresh questions about the reliability of the kinds of high-stakes tests that increasingly dominate education at all levels. Neither Pearson, which handles state testing across the country, nor the College Board detected the scoring problems until two students came forward with complaints.
“The story here is not that they made a mistake in the scanning and scoring but that they seem to have no fail-safe to alert them directly and immediately of a mistake,” said Dean of Admissions Marilee Jones. “To depend on test-takers who challenge the scores to learn about system failure is not good.”
These were not the first major scoring problems that Pearson has experienced. The company agreed in 2002 to settle a large lawsuit over errors in scoring 8,000 tests in Minnesota that prevented several hundred high-school seniors from graduating. It also has made significant scoring errors in Washington and Virginia.
After those problems, company officials had assured clients that they had vastly improved their quality control. But the new problems on the October SAT turned out to be the most significant scoring errors that the College Board had experienced.
Pearson said Thursday that the SAT errors, which affected 4,000 students out of 495,000 who took the October test, arose partly because of excessive moisture that caused the answer sheets to expand before they were scanned at the company’s large test-processing site in Austin, Texas.
Another factor, the company said, was that its scanners did not pick up some lightly marked answers.
The company said in a statement that it was taking steps to make sure that “this unfortunate situation will not happen again.” Chiara Coletti, the College Board’s vice president for public affairs, said Thursday that the College Board has continuing confidence in Pearson. “Pearson says they now understand the technical issues fully and we know they can control for those issues now,” she said. “We are confident of that because our operations people have been talking to their operations people steadily.”
The College Board has said that most of the students affected actually had higher scores than were reported to colleges. In some cases the scores were off by as many as 400 points out of a possible 2,400 on the three-part exam covering mathematics, reading and writing.
Pearson said Thursday that it had examined the scoring of all the subsequent SATs, which were administered in November, December and January, and found no further problems.
But some critics were not reassured. Shawn Raider, the lawyer who represented the Minnesota families who successfully sued Pearson, questioned whether the company had made good on its promise to improve its procedures.
“They certainly said in the course of our lawsuit that they not only were going to, but already had, implemented new quality-control measures,” he said.
The Pearson testing unit, the largest subsidiary of Pearson PLC, the giant publishing company which also owns The Financial Times, won the sizable contract for scoring the SAT exams in 2003, taking over some functions previously performed for the College Board by the Educational Testing Service.
It was one of many contracts that have helped make Pearson a giant in a field that has grown enormously since President Bush signed the No Child Left Behind Law in 2002, spurring demand for state testing. Indeed, for 20 years, Pearson has worked on the Texas testing program that was the template for Bush’s national testing initiative. Nationally, it scored more than 300 million answer sheets last year.
Even as the company explained what went wrong Thursday, new complaints emerged from students and educators who questioned how they could continue to have confidence in the nation’s testing apparatus.
Joe Giglio, director of admission at St. Peter’s College in Jersey City, N.J., said, “It seems that there is a need for some sort of outside auditing of their processes to insure the integrity of the testing from this point forward.”
Philip Benoit, a spokesman for Franklin & Marshall College in Pennsylvania, said Thursday that at least one applicant, whose SAT score was revised upward by 110 points, now qualified for the school’s merit-based Marshall Scholarship of $12,500.
Beatrice Bradley, a senior at the Williams School in Connecticut, who found out that her reported score on the writing section of the SAT exam should have been 700 rather than 690, said one of her friends had also had an Advanced Placement score raised last year.
“You have to wonder how many things go unchecked,” she said.
The SAT errors, which the College Board started to investigate only after two students questioned the scores they received in late December, were not unprecedented.
As testing has expanded dramatically in the last decade, many more errors have occurred and almost all of them have been detected by students, parents or school officials challenging the accuracy of scores. Pearson said Thursday that it did not learn of the SAT problems until early February.
Some testing industry executives acknowledged Thursday that the highly-visible SAT errors will add to the pressures the industry is already facing.
“There’s no question that the testing industry is challenged,” said Stuart R. Kahl, president and chief executive of Measured Progress, a nonprofit testing publisher in Dover, N.H., that provides testing services to 24 states. “But with the growth in business, most companies are implementing systems to make this job doable, so I don’t get a sense that there is likely to be an exponential growth in errors.”
But Kahl noted that standardized tests at all educational levels are constantly being revised. “The SATs have been undergoing a lot of changes,” he said Thursday.
For now, college officials, who were caught by surprise by the mistakes at the height of the admission season, said they were working to take the revised scores into account so that students are not disadvantaged by the scoring errors, almost all of which lowered student scores.
The College Board said that 83 percent of the score errors were between 10 and 40 points.