Impricision, Irrelevence Plague Rankings
U.S. News & World Report's annual rankings of the best schools use faulty reasoning and evaluation methods that lead to inaccurate and misleading results. Despite MIT's recent high scores in the magazine's rankings of graduate schools, The Tech continues to believe that the details of the rankings are arbitrary and should be ignored.
Recently, 162 of the nation's law schools, including seven of the top 10 in the U.S. News rankings, sent a letter to 93,000 prospective applicants. The letter, entitled "Law School Rankings May Be Hazardous to Your Health," encouraged recipients to reject the rankings and to use their own judgement. The law schools justified their action by stating that the rankings ignore students' individual needs and had little bearing on reality. The schools emphasized that each student has his or her own preferences about what is important in a school.
MIT would do well to emulate its law school colleagues. While it may help MIT in the short run to advertise its high score in the ratings this year, in the long run MIT and other schools are all hurt by the ratings because they encourage prospective students to make poor decisions. Opposition by a strong, analytical institution that has traditionally performed well in the U.S. News rankings would go a long way towards discrediting them.
The data the magazine gathers are not precise enough for it to reach its conclusions. The weights U.S. News assigns to each category are essentially arbitrary and a slight adjustment changes the rankings drastically. In addition, some categories seem to be created purely for the sake of furthering a false sense of precision. For example, in the undergraduate rankings, the specious "value-added" category compares a college's predicted graduation rate with its actual graduation rate to measure the "school's involvement in the academic success of students."
Better ranking systems nearly suggest themselves. For example, U.S. News could divide colleges into several tiers. Under the current system colleges move up or down several positions from year to year, reflecting little more than random noise. A tier-based system would better reflect the lack of precision in these ratings. Another solution would be for the magazine to publish only its raw data and resist the dangerous urge to perform simplistic analyses that paint a distorted picture. With such a system, applicants could choose the college that best suits them by examining the data most important to them.
Despite the rankings' shortcomings, the Institute seems to embrace its first-place ranking in nine graduate fields. MIT's Web site openly advertises its first place in engineering for the ninth straight year. The contrast to MIT's dismissive reaction when the Institute fell to sixth in the undergraduate rankings last fall suggests that the rankings are being embraced only when it is convenient.
Instead of promoting the U.S. News ratings, MIT should cooperate with other schools in denouncing them. Ideally, MIT would send a letter to prospective applicants encouraging them to ignore the rankings. MIT could even take a more active role in discrediting the rankings. Rather than congratulating itself for looking good today, MIT should stand up for sound reasoning and publicly reject the ratings.