

What college rankings really tell us - boh
http://www.newyorker.com/reporting/2011/02/14/110214fa_fact_gladwell?currentPage=all

======
Alex3917
What this article doesn't mention is how much effort the colleges at the top
put into gaming the rankings. For example, Harvard spends millions of dollars
every year sending direct mail to graduating seniors who they know have no
chance of getting in, encouraging them to apply. Why? To increase their
rejection rate.

Similarly, Stanford was considered a relatively mediocre school until one year
when they rejected every single valedictorian across the country as a
marketing stunt, and suddenly jumped in the rankings after guidance counselors
started encouraging the kids with the highest grades to apply there.

Similarly, the admissions departments get an update twice a day telling them
their projected US News rank based on the grades and SAT scores of the kids
who they are admitting. Thus your chances of getting in can vary wildly based
on whether your application is read in the morning or afternoon. If the
projected rank drops too low then they stop even reading the essays and just
admit solely based on grades and SATs until the rankings are back up.

~~~
austenallred
Exactly. Brigham Young University (my university) is an absolute slave to the
rankings. They intentionally limit the acceptance rates into certain programs
even when there's open space for numbers' sake. If these rankings disappeared
and universities operated only for the sake of its students, I think the
overall quality of education would actually improve.

~~~
gliese1337
That just further enforces the point that you can't decide on what the best
college for you would be by looking at any kind of ranking for the university
as a whole. Different colleges and different departments within the university
can be run so radically differently that it hardly matters that they're
officially part of the same institution.

------
kenjackson
The USNWR rankings are especially insidious because they're also not
objective.

I wish I could find the citation for this, but I recall reading about how when
it first started the person who created the ranking formula showed USNWR the
list of top 10 schools and the Ivy League schools didn't do as well as
expected. If HYP weren't at the top there was a flaw with the methodology, so
they had to go back and get a new methodology that would conform with
conventional wisdom.

Also back in '99 Caltech took the top spot. Again this was viewed with some
disdain from the East Coast elite. They changed their methodology and
added/emphasized a metric they call "value add", which is the difference
between actual and expected graduation rates. Because CalTech is difficult to
graduate from, they were seriously penalized.

IOW, rankings are often simply a way to perpetuate the status quo, but make it
look objective.

~~~
ajross
I think your broad point about "elite" schools being a matter of reputation
rather than objective measurement is basically right. But some of your
analysis seems weird.

On the graduation rate: why _shouldn't_ a school be penalized for a low
graduation rate? The point of those rankings (beyond selling magazines, of
course) is to give prospective students an idea of how each school will help
them in life. If I have a choice of going to school A from which I'm 90%
likely to graduate and an otherwise equally ranked school B which is 50%
likely to kick me out without a diploma, which am I going to choose?

And the bit about CalTech seems a little like sour grapes. I was at one of
those "East Coast elite" schools in the 90's and don't remember anyone
thinking anything but good things about CalTech (Pasadena, on the other
hand...). Basically within its fairly narrow realm of focus CT is absolutely
one of the "elite" schools everyone looks to.

~~~
gms7777
> On the graduation rate: why shouldn't a school be penalized for a low
> graduation rate? The point of those rankings (beyond selling magazines, of
> course) is to give prospective students an idea of how each school will help
> them in life.

Its easy to look at this from the other side though.

I think a lot of people look at these rankings as the value of a degree earned
from that institution. That is, they are looking at it from the side of the
alums as opposed to the prospects. Suppose two schools have the same
admissions criteria and everything else, but the graduation rate of one is
much higher. I would think the school with the lower graduation rate would be
viewed as more rigorous and therefore "better".

~~~
mchusma
Even in high school, I never thought differently. >10 years later this is the
first time I ever even thought that someone would consider the probability of
their graduating (totally within their control). It was always 100% which
would be the best school to be from...or which one would be the most fun.
Graduation rate has nothing to do with either.

------
prezjordan
I attend a small tech school outside of NYC, and it's not terribly well-known,
and not terribly well-ranked. I was a good student in high school, and got
into schools ranked much higher than the one I attend now. Why? I love the
location here, and I love the "personal" factor that comes with a small
school. I've met lots of interesting people, I live in a beautiful town with
beautiful views, and I really enjoy what I'm learning. It just "felt right"
and it still does today.

Some days I get a little upset that I don't attend a prestigious university,
and I'm not really sure why. There's always grad school, though. Anyway, just
a personal experience.

~~~
kevhsu
RPI? RIT? Those are my guesses...

~~~
prezjordan
Those were the schools I got into that are ranked higher than where I go now
(Stevens Institute - Hoboken). More people know them by name, but even beyond
that lies CMU (I was waitlisted, and I'm hoping to maybe pursue a masters
there).

~~~
larrys
Did financial considerations come into play in your decision to choose Stevens
vs. the other schools?

------
justaddwater57
Interesting that Tulane is called out here as an example:

"One common statistic used to evaluate colleges, for example, is called
“graduation rate performance,” which compares a school’s actual graduation
rate with its predicted graduation rate given the socioeconomic status and the
test scores of its incoming freshman class.... Tulane, given the
qualifications of the students that it admits, ought to have a graduation rate
of eighty-seven per cent; its actual 2009 graduation rate was seventy-three
per cent. That shortfall suggests that something is amiss at Tulane."

Anyone else think that those numbers for the 2009 graduating class has
anything to do with the fact that Hurricane Katrina hit during the first few
weeks of '09's freshman year, forcing students at Tulane (which is located in
New Orleans) to take leaves of absences or transfers?

Just goes to show, in addition to inherent biases, rankings also don't capture
extenuating circumstances that can have drastic short term effects that go
unnoticed.

~~~
sfny
Have you read Gladwell's book "Outliers"? Basically deals with the situational
factors that create outliers like Bill Gates. Circumstances have incredible
life-determining effect that go totally unnoticed too.

------
bane
I worked for a great guy who also happened to be on the board of a reasonably
well respected tech university in the South Western U.S. Every year they have
a meeting about doing the kinds of things that boost their rankings in US News
and every year they've decided to _not_ do those things because of how it
would hurt what they feel to be their academic mission - educating students.

He said that almost everything that you can do to boost your ratings are
gimmick events like rejecting more applicants, or increasing spending on
getting applicants.

One particularly greusome idea was to hire national survey companies to hang
outside of popular teen events and get students to fill out a preliminary
application that they would then reject to get boost their rejection percent
and appear more "competitive".

They keep rejecting these kinds of meaningless shows of status.

But the lure of prestige keeps this meeting on the agenda anyways.

------
appleflaxen
I liked malcolm gladwell a lot more before I found out he is a big tobacco
shill.

[http://exiledonline.com/malcolm-gladwell-tobacco-industry-
sh...](http://exiledonline.com/malcolm-gladwell-tobacco-industry-shill/)

------
mnemonicsloth

        (sort-by  
           (fn [u]
             (* (:age u)
                (:tuition u)
                (if (rhymes-with (:name u) 'Warvard)
                   1
                   (+ 0.9 (rand 0.1))))
           univs)

------
rickdale
This article brings up some really good points. Ultimately schools are
obsessed with rankings because alum and prospective students are also obsessed
with rankings. The rankings do a really good job of enthralling the consumer
and making it competitive to get into college.

I graduated in 2009. In that same year we got a new president. He pledged to
move the school up in the rankings from 22 in a welcome speech. The crowd gave
a standing ovation.

------
enraged_camel
I graduated from University of Washington, a school that ranks 41st on that
list. My roommate in my freshman year of college was Chinese. His dad, a
college professor in Hong Kong, always gave him shit for not attending a
university that placed higher in the rankings. This is despite the fact that
my roommate was a computer science major, and UW has one of the best Comp-Sci
programs in the nation. It always seemed to me that his dad cared more about
the prestige factor than the actual quality of education his son received. It
wouldn't surprise me to find out he has dreams about bragging to his friends
that his son went to Harvard or Princeton.

What a shame.

~~~
CountHackulus
Speaking specifically of computer science programs, in eastern Canada (mostly
Quebec and Ontario), there's a computer science contest called the CS Games.
It's mostly just for fun and networking, but it's interesting to see where
different universities ranked in different events. Relatively unknown
universities like Carleton (my alma mater) placed excellently in extreme
programming and algorithms, but not so well in debugging and shell scripting.

But to draw a parallel to your comment, both Sherbrooke and Harvard attended.
Sherbrooke being relatively unknown outside of computer science circles. Well
that year, Sherbrooke absolutely dominated every competition placing 4th or
better in every event, and 1st overall. Harvard however placed last nearly
every event (except AI where they were in the middle of the pack).

Looking at the overall reputations of both of these universities, Harvard is
by far the more well known and respected. Just like the article was saying,
overall rankings aren't very useful, and that if you're serious about getting
a good education, you'll have to dig deeper.

~~~
kevhsu
Well to be fair Harvard isn't ranked that highly for CS, and this single data
point doesn't say a whole lot about it either. You'll find CS undergrads that
are willing to enter, but would place last in this competition at MIT,
Stanford, Caltech, Carnegie Mellon.... every school.

Just playing devil's advocate here. Rankings are a crapshoot, and I don't love
the Ivies at all.

------
sigil
Watch for recommender systems to disrupt and replace many of these rating
systems.

As Gladwell pointed out in the Car & Driver example, the "scores" assigned in
these rating systems are arbitrary linear combinations of factors chosen by
"experts." You the car buyer will personally care more or less about certain
factors than they do, but they've already chosen the coefficients for you, so
too bad. Same for the college ratings assigned by the USNWR.

I think we're asking the wrong questions with college rankings. Prospective
students should be asking, what education will maximize the expected value to
_me personally_ in the future? It's not a heterogenous and inexplicable
"score" they're looking for, it's the probability that the match between
student and college will be a productive one. Given data about individual
students, colleges, and historical outcomes, surely a better predictive model
is possible.

------
bentlegen
On the subjectiveness of reputation:

I once met a recruiter for the University of Toronto (Canada) who focuses
exclusively on visiting US high schools. She told me how U of T was behind
other "top" Canadian schools in advertising to potential students in the US.
She mentioned how McGill University (in Montreal) started going after US
students aggressively beginning in the 70s, and as a result, has a far better
reputation there today. Even The Simpsons once joked that McGill is "the
harvard of Canada", even though most Canadians would probably say otherwise.

Clip: <http://www.youtube.com/watch?v=cc5vN2XReWs>

~~~
tikhonj
I suspect that getting US students is important because they pay significantly
more than students from Canada itself. I'm not sure exactly how it works in
Canada, but at least at my US public university out-of-state and international
students pay far more than instate students which certainly doesn't hurt the
budget. Since the US is really close, has expensive education domestically and
has a large population, it's probably relatively easy to convince high-paying
US students to come to universities in Canada.

------
geebee
The Washington Monthly had an interesting angle on college rankings, and how
different their results are from US News:

[http://www.washingtonmonthly.com/magazine/septemberoctober_2...](http://www.washingtonmonthly.com/magazine/septemberoctober_2011/features/introduction_a_different_kind031630.php)

------
jmduke
[http://en.wikipedia.org/wiki/List_of_colleges_and_universiti...](http://en.wikipedia.org/wiki/List_of_colleges_and_universities_in_the_United_States_by_endowment)

Sort by total endowment per student. Cross-reference this list with the USNWR
rankings. The overlaps are striking.

------
RedwoodCity
Thank You to who ever posted this article. It points out how stupid
college/car/etc... rankings are. I think people have a natural tendency to
want to rank things, in the end it just makes us unhappy.

------
api
I love how everyone responsible for ranking colleges lives along the North-
Eastern seaboard. That area is such a bubble of regional self-importance.

~~~
jmduke
True, but yesterday a post titled 'How Coffee Meetings Power Silicon Valley'
hit the front page.

Every region has people who are self-important and people who are not.

------
mullr
I wonder where the data for these rankings comes from. Is it publicly
available? It would be nice to have a dynamic ranker which prioritizes the
things you care about.

------
crusso
If you're a slave to the school rankings in choosing a school, you get what
you deserve.

------
tkahn6
I predict the obsession with college and therefore collegiate rankings will
dissipate in the next 50 years, maybe even by the close of this decade.

Consider what a college experience offers more readily over what you can
accomplish with a high-speed internet connection.

\- a community of people around the same age with no liabilities or
responsibilities

\- guaranteed access to an expert on the subject who can answer direct
questions

\- verification of knowledge

Anything else?

Why do we need thousands of college professors giving lectures on linear
algebra each semester? We have the bandwidth to distribute a complete video
lecture series from MIT to all college-going people in the US.

The collegiate model is predicated on the inaccessibility and uneven
distribution of information. That's why the top colleges have the biggest
libraries. That's how information was primarily stored and accessed and
catalogued.

The developed world is a completely different place now.

~~~
Locke1689
The importance of college won't change until hiring practices do. Most
employers don't actually test most of their candidates on their skills. Until
they do they need a way to distinguish their candidates. Hiring from Harvard
is a good way to do it since you can outsource your requirements to the
Harvard acceptance department.

~~~
aswanson
CS and engineering jobs must exist in a different universe than the rest of
the working world. I have yet to go through an interview process that did not
require a verbal/written series of technical questions being answered.

~~~
xanados
It is not much of an exaggeration to say that they do. In many (most?) other
professional fields decisions are made based solely on interviews, including
finance, accounting, sales, middle management, etc.

