
Going to elite Indian colleges improves earnings, but not test scores - blueblisters
https://www.economist.com/graphic-detail/2020/09/09/going-to-elite-indian-colleges-improves-earnings-but-not-test-scores
======
zwaps
Important caveat: The test in question seems to be the exit score of junior
colleges, equivalent to 12th grade education.

Which suggests that the discontinuity found might very well be due to the
educational quality of the "regular" college. It would, likely, only get rid
of general intelligence as confounder. This, however, would just make
educational quality more pertinent rather than less.

Edit: Well I found a working paper from 2011. The matter is more complex.

Apparently, in India, both public and private colleges exist under the heading
of a single local university. The students at private and the more prestigious
public colleges then take the same exams in Bachelor.

The authors have collected data from one university and a couple of colleges.
For that sample, they control for ability with the entrance scores, but then
match the regression discontinuity on exit scores.

That is: Public colleges have on average higher exit scores. The paper shows
that this is due to having stronger students (entry exams). Controlling for
this, they find no difference in the "educational trajectory" of public vs.
private colleges, however, public colleges then have better job outcomes.

So my earlier caveat should read: The study shows results for one Indian
university (with several colleges) in the field of liberal arts.

Whether the study is externally valid for IITs I'll leave up to the reader.

~~~
searchableguy
Get it from [https://sci-hub.tw/10.1257/app.20140105](https://sci-
hub.tw/10.1257/app.20140105) :)

------
satya71
I went to an "elite" Indian college. Anecdotally, I can say the quality of
education was quite a bit higher than what could be had at non-elite colleges.
I can also say with reasonable certainty that the large salary difference is
mostly due to signalling. Even the graduates from a top college has much to
learn before they can contribute to their job.

Also, elite college admissions are based on test score on one day in the
student's life in high school, while their brain is still developing. I have
personally seen many who didn't qualify, but have gone on to do great things.
Their initial salary was definitely low.

~~~
austincheney
To add to this with numbers the US military imposes a standardized test on all
service members and many high school students who never become service
members. The test, the ASVAB, is composed of several sections but all anybody
really looks at is the GT score.

[https://en.wikipedia.org/wiki/Armed_Services_Vocational_Apti...](https://en.wikipedia.org/wiki/Armed_Services_Vocational_Aptitude_Battery)

The first time I took the test as a dumb 17 year old who was at risk of
failing out of high school my GT score was only 107 (out of 130) and my
average performance placement compared to all other test participants that
year was 79 (on a scale of 0 - 99). Generally, everybody who entered the
military as a teenager notices dramatic improvement on that test when retaking
it later life even with no preparation.

When I wanted to become a warrant officer I had to retake the test because of
my low scores. The minimum GT score to become a warrant officer is 110. I
retook the test at the age of 35 after 8 years of being a full time software
developer in the corporate world. My new GT score was 129 (out of 130) and my
placement amongst other test takers that year (on a scale of 0 - 99) was 98.

I think research generally bears this out across a variety of metrics and
meaures including violent crime data. The human brain is still developing
until around age 24 give or take two years. This is why applaud California's
recent decision to halt standardized tests, such as SAT and ACT, as a
mandatory placement criteria for entering university. All those tests measure,
given the youth and maturity of traditional high school students, is extreme
preparation and not performance or potential. Preparation in that regard is a
socio-economic stratifier as it takes time and money outside of the student's
own initiative and state provided education.

~~~
legolas2412
> This is why applaud California's recent decision to halt standardized tests,
> such as SAT and ACT, as a mandatory placement criteria for entering
> university. All those tests measure, given the youth and maturity of
> traditional high school students, is extreme preparation and not performance
> or potential.

Unless you think that universities should pick their students by lotteries,
you must answer how whatever that replaces SAT is not worse. Because whatever
replaces it, for example identifying well rounded candidates with extra
curricular achievements is even more socio-economic stratifying.

It largely feels like an excuse to lower the competitive bar to manage the
demographics, allowing both more legacy hires and more representative
demographics. Such actions however will come at the cost of merit.

~~~
shajznnckfke
> It largely feels like an excuse to lower the competitive bar to manage the
> demographics, allowing both more legacy hires and more representative
> demographics.

It doesn’t seem like a good solution to that problem. How will removing test
scores as a factor help smart kids of underrepresented minority groups against
dumb rich kids (with influential parents) who are too lazy to study for the
SAT? It seems much better to keep the test but apply a corrective factor if
you believe it’s biased against some group. It’s the only way I can think of
to keep the dumb rich kids out.

~~~
bonoboTP
The point was that they don't want to keep dumb rich kids out. The parents are
an important money source and you don't want to upset the ruling elite by
withholding from them the prestige they are used to.

~~~
srtjstjsj
Public schools aren't for the "ruling elite"

------
addicted
I’m not sure I understand. This study doesn’t really seem to distinguish
between what the economist says it’s distinguishing.

The economist appears to be arguing that this study indicates that elite
colleges lead to higher wages due to signaling and not the quality of
education.

However, the study basically compares the wages of people who scored about the
same in the standardized exams after separating those who made it into the
elite colleges and those who didn’t. The ones who made it into the elite
colleges appeared to make more money than those who didn’t.

So basically, this study tells us that going to an elite college helped
someone make more money than someone else who had about the same intelligence
(as defined by their standardized test scores to enter the college) but went
to a non elite college.

However, unlike what the economist is claiming, this does not tell us whether
this effect is because of signaling, or because the quality of education is
better. The economist claims that the study indicates its signaling, but the
higher earnings could very well be explained by better instruction in the
elite colleges as well.

I also suspect those who went to elite colleges feel far more pressure to
inflate their reported earnings relative to those who didn’t.

~~~
sgillen
The study being referenced compares college _exit_ exams as well as mean
salaries. They found that the exit exams were not significantly different
between people who went to elite vs non elite colleges. This doesn’t prove but
might suggest that a better education is not what explains the higher
salaries.

~~~
sfifs
There ain't no such thing as standardized exit exams for colleges - especially
for the elite rung of the colleges. My college department's professors set
their exam papers as did every other faculty in similar colleges - so it's
really puzzling what this article is arguing.

Lower rung universities do have papers set by the university for all colleges
affiliated with that university but again every university sets their own
exams.

There are also entrance exams for MBA schools which are entire different beast
and not really connected to what you study in Bachelor's

(Source: studied in the said elite Indian colleges)

------
sumgame
There is a caveat that this article doesn't consider.

In India, if you are studying for elite colleges, in this case the IIT's, you
write the entrance exam designed specifcally for that, which is IIT-JEE.

Most people go for separate training camps to study for JEE. The 12th grade
exams are a cakewalk compared to these exams, and the stuff you study for the
JEE is sometimes super advanced, like calculus in super advanced levels.

Most kids studying for JEE ( I was one of them), don't optimize for 12th grade
exams because the JEE exam rank is all that matters to get into these
"prestigious" institutions.

So a bad 12th grade score is by design more than anything.

Though there is definitely a lot of signaling, the research has ignored a very
important behaviour.

------
screye
The difference in quality of preparation of students who study to get into the
top universities (IITs, NITs, BITS, handful of private/local universities =
ELITE) so far outstrips the preparation of your average engineering candidate,
that it is not even funny.

To study computer science at an IIT, you have to be in the 0.01 percentile (1
in 1000). Most of my cousins with sufficient wealth, opted to go study Math/CS
at Stanford, MIT, etc. instead of going through the grind of getting into an
IIT. That's how difficult it can be.

High school exit examinations are a joke. I studied for less than a week for
them. I studied for 2 years (8+ hours per day) for JEE, and I was considered
one of the 'lazy ones who started late'.

> In short, elite colleges probably do offer their students more value than
> their competitors—just not in their lecture halls.

This is an outright lie.

The difference in education between the ELITE universities I mentioned above
and your average engineering college is that of Harvard vs a Community
College. My brother goes to a well regarded sub-elite university, and the drop
of quality from the ELITE and his is mind-boggling. Most IIT professors have
PHDs at US top 20 universities. Most sub-elite professors are teaching because
they couldn't find a job in CS and have never published a paper in their
lives. (The delta is 10x worse for any core engineering branches such as
Mech,EEE,Civil,etc)

I would have loved for it to be different. It is entirely unfair that the
state of a person from age 15-18 leads to an opportunity gap that's this
massive. But, that is the truth of how things are in India.

~~~
0xffff2
>The difference in education between the ELITE universities I mentioned above
and your average engineering college is that of Harvard vs a Community
College.

A lot of people (myself included) would say that the difference between
Harvard and a community college also resides entirely outside the lecture
halls. I can't speak to Harvard, but my experience with community college and
a well regarded University of California campus was that instruction was much
better at the community college. No one takes a position at an elite US
university to teach. They go there to do research. Conversely, the professors
at community college don't generally do research, so they have much more time
for their students.

~~~
newen
I have taken classes in Physics, Math, and CS in a top 20 ranked US university
and a university ranked in the 200s. The difference in teaching skill and the
difficulty in the materials taught for the same course is massive and very
noticeable.

This is not even to go into the research that the professors do and the
academic levels that the students are in. The difference is huge.

~~~
0xffff2
>The difference is huge.

Which way? My experience was that my (not quite top 20, but close) university
professors were an extremely mixed bag. None of them were half as good as my
best community college professors, and some of them were downright awful. I
would be truly shocked if you told me that lower ranked universities were
worse.

~~~
newen
For one, courses were significantly more difficult and the homework was harder
in the top 20 university. For example, the junior level CS algorithms course I
took at both were so different that I wouldn't even call the one I took in the
200s ranked university an actual algorithms course. This was just the worst
case though; the other classes were better than this class but the differences
were significant.

I agree though that it can be a mixed bag in that there were good and bad
teachers in both universities but I found the top 20 university had in general
professors who consistently taught better, the courses were better structured,
and the professors seemed more motivated to perform well than the 200s ranked
university. There were a number of exceptions on both sides but the exceptions
favored the top 20 university.

------
canistel
The so called exit exams(tests) in Indian colleges may not be what Westerners
expect it to be. In Indian exams, challenging problems are not touched upon at
all. Exams are about how much ground can be foraged rather than how deep you
can wade in. You can easily get by through cramming and rote learning. I am
sure university/college/school exams still lack questions that force students
to solve problems, applying learned concepts along with their analytical
skills.

There are two types of exams in India:- Entrance exams - where the grasp of
concepts, analytical ability etc are tested. School/college exams - where it
is all about regurgitation of information.

With latter, the only aptitude that can possibly be assessed is the tolerance
for a monotonous and brainless desk job.

~~~
srtjstjsj
It's similar in the US, where we have the SAT IQ test and the GRE (SAT repeat
plus basic knowledge test in a subject area).

Neither has the sort of deep knowledge you'd see in a US Graduate Degree
"qualifying exam" which tests depth of understanding of undergraduate material
(for which many students use remedial graduate school classes to prepare)

~~~
ska
This is broadly true, but qualifying exams can be wildly different even within
faculties, let alone across schools - enough so that I can't imagine a
meaningful comparison of "results". They are also often pass/fail.

------
blocked_again
The article talks about how students who went to elite Indian collegs perform
similiar to students who went to low tier colleges in standardized exit tests
wich is apparently a thing in India.

The only issue is, students don't take any standardized exit tests in India.
Which makes the whole paper pointless.

I went some digging to see what data this paper is using. Apparently this is
what I found out.

> All students in colleges (private or public) affiliated with the same
> university take the same exit exams. These exams vary by stream of
> education, but conditional on the stream, private and public college
> students study the same curriculum and take the same exit tests.

> I obtained administrative data from colleges in a district in North India
> and the university with which they were affiliated.

The issue with using this data is that almost none of elite colleges in India
are affliated to some University. They are their own University. So apparently
the data of the paper doesn't include any elite colleges! Kind of ironic given
the title of the paper "Prestige Matters: Wage Premiumand Value Addition in
Elite Colleges"

~~~
square_usual
> students don't take any standardized exit tests in India

From the article:

> There, pupils in their final year of _secondary school_ sit a leaving exam
> known as the Senior Secondary School Examination.

(Emphasis mine.)

There is a bit of a mistake here. There isn't any _one_ Senior Secondary
School Examination, but many. Some of them are standardized at the state
level, and I know of two which are standardized across the country, the
largest of which is the CBSE. I assume this mistake is from the Economist,
since the abstract of the paper itself says "Admission to the elite public
colleges is based on the scores obtained in the Senior Secondary School
_Examinations_ " (emphasis mine). Going through the "data" archive linked on
the paper's page, I see references to the CBSE, which makes it likely that
that was the standardized test the author considered. The CBSE is, by
definition, a standardized exit test for secondary school students in India.

> am actually amused this article was based on an research paper published by
> Universitry of Virginia

The author of this paper is of Indian origin.

~~~
blocked_again
I was referring to the exit test scores that is used by the author for
comparing students who goto elite vs normal college. See my updated comment
above.

------
mikorym
This is an interesting topic and has many analogies.

If you gave mathematicians some general exam and have all universities
participate, I would also not expect top universities to score the most.
However, if you asked instead what the _money value_ is of their research,
then MIT, Oxford, et al. would score highly.

I don't want to extrapolate on the economist.com article, and whether it's
true or not, but rather just illustrate that this happens in other contexts.

Another example is salaries. Are the best programmers in the areas that pay
the most? I think no, but the ones whose work make the most money for their
employers probably are. A case in point is the alpine Emacs coder that
featured in a comment a few days ago, Thierry.

The main issue is that if you focus on the money part, then you focus on ideas
that have settled. Many small places or small universities are still at the
distillation step and the distillate tends to move to where the money is. I
use this argument as one of the main arguments for fundamental research, and
also for why it has cognitive dissonance with earning potential.

~~~
augustt
Well there's the Putnam competition, which MIT has been completely dominating
recently ([https://kskedlaya.org/putnam-
archive/AnnouncementOfWinners20...](https://kskedlaya.org/putnam-
archive/AnnouncementOfWinners2019.pdf)).

------
starpilot
Are these tests like the SAT? I studied pretty specifically for it before
college. I imagine I'd do worse after college because my degree didn't have
much to do with the tricky algebra questions or weird vocabulary that it
seemed to focus on.

~~~
srtjstjsj
I don't know. One difference between k-12 and college is that we are much
better at long-term remembering things we learned earlier in life.

------
baba_ramdev
Some details from the paper: \- The paper studies colleges that offer a three-
year undergraduate course in arts and science (no engineering or medicine).
So, extrapolating this for IIT/NIT vs non-IITs is just argumentative with no
data points.

    
    
      - The statistical data did not survey students who managed to secure a seat in an elite public college using reservation quota.
    
      - The sample data is from admission cohorts a couple of decades ago (1999 - 2002).
    
      - The detailed in-person survey was done only in urban areas during 2011-2012 time frame.
    
      - Data sample:
        * Initial survey request sent - 1981
        * Actual surveyed - 1506 (76%)
        * Either self-employed or salaried - 748 (49% of surveyed)
        * Final data sample - 439 (58% of employed)
          # public colleges - 190 observations
          # private colleges - 249 observations
    
      - The entire analysis (and the claim) is made comparing the outcomes of students at the margin of admission (around admission cut-offs). So, if a student secured top score and entered into a public college and gets a top dollar salary, the author weakly attributes it to better learning environment.

------
sb2nov
Can the exam be the confounder here, where the exam is just not good at
differentiating the different in educational experience between the
universities.

Second the job interview and exam might just be looking for skills on
different dimensions ?

~~~
zwaps
Yes that is a possibility in this sort of research design. The exit-test
scores are obviously taken as proxy for educational quality and the study
cannot test for any "secret sauce" beyond that.

On the other hand, there's only so much one can do with observational data
doing these sort of causal analyses. I'd say its still an interesting outcome.

------
tbenst
It’s also possible that tests are simply a very poor way to assess
knowledge/aptitude. I certainly felt this way when taking the SAT/GRE in the
US at least.

------
Mortiffer
Am I the only one that finds their primary data plot at the top terrible?
Journalists should have a look at the Grammar OF Graphics .

------
jonplackett
Who you know, not what you know...

------
nym3r0s
Disclaimer: I graduated in CS from one of these "elite" colleges. I also took
the JEE in 2013 when it underwent significant changes.

Back in 2011, the government decided to revamp the educational grading system
by introducing "Continuous and Comprehensive Evaluation" (CCE) for class X. We
were the first batch to go through it and every assignment/internal exam had
some weightage in your final grade. Fast forward to 2013, there was a repeated
attempt to bring similar change to this system. The central government merged
the AIEEE and the IIT-JEE into two sets of exams - the JEE Mains and the JEE
advanced. They also introduced weightage to your XII grade marks in the final
rankings - meaning you had to do well in your boards in addition to the one-
day exam.

Admittedly, this was less radical than year-round performance, but it was
still significant enough to tell everyone - "Hey, you need to perform all
across our evaluations, not just in one exam". This meant that you could have
gone through all the folks who took the CBSE XII exams in my class - and they
would have been in the top 5% (maybe even top 1%).

Having said that, at the end of the day - apart from signalling, the main
reasons to get into the IIT/NITs were infrastructure, the environment and the
network. The infrastructure was way better than a smaller, private university
and your entire class is filled with the top people from your batch.

Even if we conceded that the curriculum and the faculty were not at par with
the top universities in the world, the talent surely is. The extra-curriculars
and the competition within ensured that you constantly honed your skills to
remain relevant - we took MIT OCW courses, participated in global hackathons
etc.

TL;DR - High-school test scores were relevant when it mattered in college
entrance exams. The moment the weightage was lost, the relevancy was lost.

------
ssivark
There are no standardized “exit exams” for Indian universities, unless one is
talking about those colleges affiliated to a specific university system (but
those are not referred to as “elite” institutions in common parlance; just
terminology, I’m not making value judgements). Many states in India have a
10+2 education system where the “+2” (last two years of high school) are
referred to as “junior college”. And admission to “elite” institutions is
through specialty purpose entrance exams which are typically considered to be
_much harder_ than high school / junior college exit exams. Evaluation in the
latter can also be a crapshoot, so scored in the high school / junior college
exit exams might not have a very strong correlation with scores in the
university entrance exams.

I don’t understand what’s actually being analyzed or commented on (both the
Economist article and the source journal article are paywalled for me)

------
j45
Like most things, it’s not what your education makes of you, but what you make
of your education, specifically your ongoing self-education.

------
muktabh
I dont disagree with the premise that credential based tertiary education
credits for jobs are pointless and a genius can come out of any university.
You can do well despite of where you studied. There are people who succeed in
life due to no talent and just connections.

However, this analysis is just based on outright flawed data. The author has
no idea about how education system works in India, or deliberately chose to
ignore it because this is what could be done on the data they had accumulated.
And I am not talking about the disclaimer about data sample being small, the
entire model is wrong for majority of Indian students.

Couple of things I want to comment on when you look at the article and the
original paper :

1\. The study treats few public institutions as "elite" and private
institution as for non-elite people. That is wrong. There are private
institutions which are elite too. My alma-mater is one of the elite ones and
it was not a public university. This is the least problematic error in
analysis.

2\. The assumption is a common academic score being used to score university
students. For college entrants in early 2000s (the time period when the survey
respondents applied to colleges), people had to literally give 50 different
entrance exams to get into universities. The score which is used to evaluate
who was marginally better to get into elite university and who was rejected
(the entire premise of the article), IS NOT what is used to decide entrance
into universities in India. Its getting slightly standardized now, but was
really chaotic 15-20 years back. There is a common exam that people give at
secondary / senior secondary level but very few universities entertain them at
entrance. Actually there are many "boards" to give Senior Secondary exams as
well. Net net, there is very large variety of ways by which universities in
India take entrance. Even the elite ones. I think, given the author is from
the field of economics, they are overgeneralizing. A few (not all) economics
elite universities do treat senior secondary tests as entrance criterion.

3\. Exit exams used to compare academic credentials of students to see if they
are academically better than the other DO NOT EXIST ! They dont even exist now
and there is no point of them existing. Every public/private university in
India (at least the elite ones), have their own grading/curriculum. So the
academic credentials cannot be compared directly. With different universities
having different grading method, people having somewhat similar academic
distribution is a known thing (I dont recollect the name of the phenomenon
exactly, but its studied and published). That said, it doesn't go with how the
data has been modeled here.

Overall, the author's model is wrong for many (50%+) of Indian students. They
might be referring to economics colleges (given they come from field of
economics) Delhi University, Mumbai University and Kolkata University type of
institutions where their model of tertiary education fits correctly, but in
that case her analysis doesn't work well for most of "India". Totally wrong
messaging.

------
bambam24
Please, please Stop sharing articles which requires paid subscription.

~~~
DanBC
[https://news.ycombinator.com/item?id=23597733](https://news.ycombinator.com/item?id=23597733)

------
zhang_sage
Why should we expect it to? Education does not increase fluid intelligence.

~~~
ajkjk
Uh. Other things probably factor into test scores than that?

------
contingencies
_< various babble>_

This just means "Nepotism is real". Results should carry to all countries,
though I doubt that large and present studies are available.

------
aaron695
Nothing to see here.

You don't graduate university for test scores. Test scores only work in low
level learning like high school.

For this study they used -

"These exams test for language competencies (English and regional language)
and stream-specific competencies; for example, commerce students take tests in
accounting, taxation, and so forth."

Basically IQ tests.

Maybe they learned more. Maybe it's about cohorts. Maybe elite universities
are used to signal. You can't tell from this study.

DOI: 10.1257/app.20140105

