Backwards Report on Advanced Study
Jay Mathews has been on the staff of The Washington Post for decades. He is author of several books on secondary level schools, and has his own column dedicated solely to education. So when his formula for ranking the best public high schools -- the Challenge Index -- was Newsweek’s cover story, one would think that despite the magazine’s sometimes sensationalist bent, Mathews knows what he is doing. One would be wrong.
Rather, it’s quite startling that a metric as simple and ridiculous as Mathews’ could be considered a significant gauge of school quality. He merely takes the total number of individual Advanced Placement (AP) and International Baccalaureate (IB) tests taken by students at a school and divides it by the size of the school’s graduating class. Those schools where the quotient ends up above one are then ranked on the scale. Schools which select a majority of their students through “academic criteria” are not included because Mathews feels that would confound the quality of the school and the motivation of the students.
Yet he seems to be ignoring the similar stratification that occurs within high schools. New Trier High School, for example, received a 2.041 on the Challenge Index for 1859 APs taken over its 911 seniors. According to New Trier’s Web site, only 776 of its 3829 students actually took an AP exam. Even if only the seniors were taking APs, that 85 percent who do might be further differentiated by the number of APs from given individuals, or the difficulty of the tests themselves (Literature or Language, Macro or Micro, AB or BC?). If Mathews expects that all AP students are alike, let alone comparable to IB students, he is mistaken.
Then again, he isn’t really measuring IB or AP students as much as he is IB or AP test-takers. Mathews knows that schools act as gatekeepers by barring certain students from more rigorous subjects. He feels any effects may be mitigated by ignoring passing rates, which he believes are made “artificially high.” While the IB program is too expensive for schools and students to encourage standing for examination without first taking the courses, raise your hand if you never met a kid who took an AP exam but not the class. In the end, he’s probably still measuring student motivation to a significant degree.
Furthermore, it is a motivation amongst an already-established elite. Having any AP program - let alone an extensive one or IB options - is a privilege which varies from school to school. 8901 schools offered AP U.S. History in 2002, but only 1317 offered the E&M portion of Physics C. Mathews still lauds IBs and APs so much because he believes they are a both more egalitarian and effective measures than the SATs or ACTs. It is elitists, he believes, who decry their ever-growing popularity because it is a threat to their reputations when inner-city youths are pulling down top scores alongside their pampered pupils.
Mathews cites, for example, the growth in numbers of minority students taking APs, and how it has outpaced that of the general population since 1998. Still, according to the College Board, Latinos and “African-Americans” now constitute no larger a slice of the AP pie than they did in 1999 (down from 10 and 5 percent [with rounding] to 9.7 and 4.4 percent, respectively). It seems the disproportionate increase in absolute numbers of minority participation was driven mostly by an increase in Asian participation, and even their percentage has changed little in a decade (13 percent [with rounding] in 1991, 14 percent in 2002). Of course, Blacks and Latinos are more likely to go to poor schools than the general population, so those data doesn’t necessarily undermine the basic premise of the Challenge Index. Not all of the underrepresentation is a byproduct of marginalized schools, however. The Center for Education, in its 2002 report on advanced study in American high schools, noted that Blacks and Latinos are much less likely than the general population to take AP classes even when they are available. A similar lag occurs amongst low-income students of all demographics, in no small part because even the AP’s exam fee -- which is around a tenth that of the IB and may be subsidized -- can be economically prohibitive.
Even assuming that APs and IBs are indeed mind-enriching experiences fundamental to a motivational school atmosphere and not overextended teleological test preparation courses which inherently degrade the high school experience, the Challenge Index proves insensitive to critical variations. It conflates a school’s options for its elite with overall quality, and mistakes college-minded students for an encouraging educational environment. Ludicrously, it ignores non-graduating seniors and other unchallenged quality measures like retention rates. Those allowances and exclusions Mathews did consider -- highly selective schools and actual exam scores -- serve to highlight further complexities a robust Index would have to consider. Finally, for all its supposed superiority to the SATs as a predictor of later success, AP participation and SAT scores share similar institutional biases, while IBs and ACTs are both second-class citizens to their more widespread College Board counterparts. It would considered especially foolhardy to equate the IB and AP, if it were not for the fact that the entire Challenge Index concept is an oversimplified and overblown popularity contest -- in other words, a Newsweek cover story.