College Rankings Are Not Accurate

This fall U.S. News & World Report released its annual college rankings.  As usual, the occasion generated an avalanche of publicity and millions of hits on US News’ website. This isn’t surprising since many parents and students believe that these rankings represent the final word on college quality.

Students in some foreign countries actually assume that the federal government generates U.S. News’ rankings, which makes them equally valued overseas.

I understand why families place so much faith in these rankings, but here’s a little known secret about this evaluative tool: Rankings are largely meaningless.

A school could do an excellent job of graduating highly employable young graduates who can write well and think critically and it wouldn’t budge a school’s rankings even one spot. People don’t appreciate that U.S. News doesn’t even attempt to measure the type of learning going on at the nation’s colleges and universities.

While the rankings ignore what happens in the classrooms, the rankings methodology does care deeply about whether an institution is prestigious, elite and rich. Let me explain…

Reputation, Reputation

The biggest factor that determines a school’s ranking is something called its reputational score. This measures what others connected to the higher-ED universe think of the institution.

US News (I can’t call it a magazine because it died, but the name lives on as an Internet site that makes money by grading all sorts of things such as hospitals, doctors, cars, nursing homes and even diets!) sends out three reputational surveys to every school it ranks. Each survey contains the names of the institution’s peers. The hundreds of schools in the “national university” category, for instance, are sent a scorecard that lists all these institutions including such disparate universities as Harvard, Ball State, Georgia Tech, Brandeis, Notre Dame, Texas Christian, UCLA, North Dakota State and Colorado School of Mines.

Someone in the offices of the president, provost and admissions is supposed to grade each peer institution on a one-to-five scale.  If this seems absurd, it is. What will administrators at the University of Michigan know about what’s going on at San Diego State, Brown or Oral Roberts?

Administrators in the other prestige category , “national liberal arts colleges, ” face the same dilemma. What would the people stuck with completing the surveys at Vassar, Amherst and Pomona colleges know about the quality of education at Gettysburg, Kalamazoo or Beloit colleges?

Completing these surveys becomes a guessing game that is heavily influenced by prestige and brand appeal.  The schools that are the most highly ranked remain that way because they enjoy excellent reputations whether or not they are actually deserved. These schools happen to be the nation’s wealthiest and oldest institutions and are the subject of the most press veneration.

When people completing the survey don’t know anything about some of their peer institutions guess where they turn to form their opinion?  When they are trying to figure out how to grade Tougaloo College or the University of Minnesota-Morris or Southern Illinois University, you guessed it! They consult previous US News rankings. The rankings perpetuate themselves.

As if they needed any further advantage, US News hands the nation’s most elite schools an even greater advantage by giving weight in the methodology to the size of an institution’s endowment and the percentage of graduates who donate to their alma mater. That’s one reason why you will see few public universities among the most highly ranked universities. UCLA, University of Virginia and the University of California, Berkeley, tend to be the only ones that crack the list of the top 25 national universities.

Dubious Methodology, but Revered Just the Same

Despite the dubious methodology, schools take the rankings very seriously.  Higher rankings can translate into more applications that allow schools to reject more students, which make them look more exclusive. Being among the elite schools, as judged by U.S. News, will also trigger more interest from affluent students, which is what colleges, and universities crave.

The stakes are so high that some schools cheat to get ahead. In August, for instance, Emory University in Atlanta admitted that it had intentionally misreported SAT and ACT scores to U.S. News. The highly ranked university sent along the higher test score tallies of admitted students rather than those who actually ended up attending the school.

A top administrator at Claremont McKenna College in California was fired after it was revealed last year that he had been inflating SAT test scores for years. Meanwhile the U.S. Naval Academy was embarrassed when it became public that its acceptance rate wasn’t nearly as impressive as it had been reporting.

Here’s the bottom line:  The best way that parents and students can use the rankings is as a tip sheet to generate college ideas. They should not assume that a school’s numerical rating means anything at all.