
The Lesson to Unlearn - adunk
http://paulgraham.com/lesson.html
======
knzhou
As somebody who has designed some of the tests Paul Graham is complaining
about, it really is hard from the other side!

For example, it's long been known in the physics education research community
that students come away from introductory courses with very little physical
understanding, even if they can do the plug and chug problems on typical tests
just fine. Students can all recite Newton's third law, but immediately
afterward claim that when a truck hits a car, the truck exerts a bigger force.
They know the law for the gravitational force, but can't explain what kept
astronauts from falling off the moon, since "there's no gravity in space".
Another common claim is that a table exerts no force on something sitting on
it -- instead of "exerting a force" it's just "getting in the way".

For research purposes, we measure physical understanding using a battery of
tests, such as the Force Concept Inventory, containing only simple conceptual
questions with unambiguous answers. So then everybody asks: if ordinary tests
are so hackable, why not just switch to these conceptual ones? But that
wouldn't work. There are less than ~100 distinct FCI-style questions. If these
conceptual tests were the norm, students would just memorize the answers and
parrot them back, with a flimsy understanding that crumples the second any
follow-up question is asked. It would be just the same problem as before,
except they would be worse computationally, too. The FCI only works as long as
it doesn't count for a grade.

The problem isn't tests, it's scale. If the people aren't motivated, any
standardized measure will miss the mark -- even the entrepreneurship Paul
Graham advocates for. God knows I've seen a lot of bullshit in that direction.

~~~
taeric
It is amusing how many of my peers still hold to the notion that heavier
things fall faster. Not a higher terminal velocity, but the intuition that
they accelerate faster. Hilariously tough for folks to shake.

~~~
GCA10
Add air resistance into the mix, and denser materials do fall faster. Coins
vs. napkins. Plums vs. crumpled balls of paper

I'd actually take the lesson the other way. It's hilariously tough for model-
makers to acknowledge real-world noise that prevents their theoretically
precise models from delivering reliable results.

~~~
taeric
I covered that with terminal velocity. And it isn't just falling, but going
down a hill. Folks commonly think heavier bikes will go faster down a hill.
That isn't how that works.

~~~
marcosdumay
Faster bikes will go downhill faster. Both the acceleration of the bikes and
their terminal velocity are determined by aerodynamics (on very steep hills)
and rolling friction (on less steep ones), and those do not change much with
bike weight.

------
ken
I'm glad he made the leap from school tests to funding tests, as it seemed to
be a thinly veiled analogy from the start. I'm disappointed, though, that he
didn't take the final step and admit that money itself is also a poor,
hackable test.

Funding, growth, usage -- these are all still one level removed from something
worthwhile or beneficial. Cigarettes have amazing usage numbers even in 2019.

I'm waiting for the entrepreneur who will say he doesn't care how much money
he makes or how many customers he has, only that it's worthwhile and needed
doing.

~~~
jtbayly
Every free (as in speech) program is written by somebody that said that. But
they are generally not entrepreneurs.

The entrepreneur is by definition somebody that cares about money.

~~~
pbhjpbhj
An entrepreneur to my mind (en-gb native) is "a person who takes on the risk
and reward of a new business venture, particularly a novel business (in the
geographically area they are in)".

~~~
jacquesm
No idea why this is downvoted, that's an excellent definition.

~~~
jtbayly
Agreed. And as he says, it's a business venture at heart, meaning that money
is involved.

------
daly
I got into "computer science" before there was a CS degree. Our CS classes
were taught in the math dept (fortran), business dept (cobol), or engineering
(assembler).

The profs were literally 1 chapter ahead of the students.

I, however, was in love with the subject. I was the "student advisor" in our
"machine room". We had 5 teletypes connected to a remote mainframe at Rutgers.
I lived and breathed CS.

I wrote tests for the profs (even thought I was also in the class) and I
answered questions for students about the tests when they came for computer
help.

Pick a subject you love so deeply that you're always at the leading edge
(which essentially means a new subject the school wants to teach). For
example, CMU introduced an AI curiculum. Learn how to do NNs, DNNs, GANs, etc.
Read the latest papers. Write working code. Chat with the profs. Hang around
the dept.

Proof systems are a hot topic. Learn LEAN, COQ, AGDA. Learn to write proofs by
machine. Read the papers. Get ahead of the curve. It is hard to "fail" a
student who knows a lot more than you, especially when they help craft the
tests.

Intel stuck an FPGA inside its CPU. Learn Verilog and learn to hack new CPU
instructions (e.g. read Gustufson's book "The End of Error" that introduces a
new kind of floating point arithmetic). Write code that outperforms BLAS code.

Learn BPF (Berkeley Packet Filter) so you can do impossible things about the
kernel in User mode. Make your networking class look like it is stone-age.

I could go on but you either "get it" or you don't.

~~~
kaymanb
I think I fall into the "don't get it" category. Could you explain what I
should be taking away from this?

I understand that being at the leading edge has benefits, but it's not
feasible to understand the state-of-the-art across an entire CS curriculum.

~~~
cjp
Of course, no one can understand everything, but he said, "pick a subject".
Just one, not everything. I'm reminded of this description of a PhD:

[http://matt.might.net/articles/phd-school-in-
pictures/](http://matt.might.net/articles/phd-school-in-pictures/)

~~~
neltnerb
I disagree with this cartoon in that getting to the edge is not easier by
focusing on only one thing. Not knowing other things is just as likely to hold
you back from some insight as not knowing your specialty well enough. It's so
common that people unaware of progress in other fields spin their wheels
solving problems with known solutions that I think the circle should have been
more of a confounded sphere where knowing a bit of EE and ChemE makes your
work on BioEng 10x closer to the edge. Knowing more than one field reinforces
the strength of your innovation. Simple example, how much easier is it to make
progress in applied physics if you know enough EE and machining to design and
build the best test apparatus in the world?

Pick a focus and whenever you get to the point that you're a strong expert (or
whatever level you feel is _useful_ ) and can do the job you want to do you
should reach a bit and pick up something you're curious about to add more
diversity to your skillset.

Being a generalist isn't bad, I basically have a PhD in being a generalist and
tons of people want to work with me because someone who only understands
optics will be lost when they need to use a photodiode and a lock in amplifier
or someone who wants to test magnetic properties will be lost when they don't
know how to actually construct a custom magnet. Among many obvious examples.

Unless you're at a place so big that you can just glide on assuming other
people will fill in the gaps in your knowledge, you need both general and
specific skills. A PhD doesn't have to mean you are only capable of one thing
unless you let it, even if that was the common meaning 20 years ago. There
just aren't the jobs in academia to support that kind of singular focus
anyway.

We're going to live long enough that it's a terrible career decision to pigeon
hole yourself so badly 50 years before you're even theoretically going to
retire. Especially with online classes it's only going to get more common for
people to have multiple significant skills and major changes in their field
over the course of their career. Required even. Very few people are going to
go for 50 years with a single technical expertise, certainly jobs for such
experts are far harder to find than jobs for people who can do a few things at
a more modest level of expertise.

I know a ton about materials science and physics, have genetically engineered
viruses, have built chemical reactors, have designed catalysts, have done
machine learning, have designed lots and lots of electronics, have programmed
as required (not my favorite kind of work, but I've done 8051 assembly up to
ARM firmware, Matlab, Scheme, Python, and _shudder_ LabView), am more than
half decent at machining, teach karate, perform music, and make art... trying
to be the best at just one thing to the exclusion of all else is unnecessary
and limiting both professionally and personally.

I was terrible at all of these things when I started. The article echoes my
philosophy as well -- you have to learn to learn because you want to and not
because you want a grade. I got a C- in differential equations. Two years
later in stat mech I was the only one of my friends who recognized LaGrangian
multipliers being used (by name even) because when I took the course my goal
was to learn differential equations regardless of what grade I got, so I
didn't forget it all the second the final was done. If you are the kind of
person who just won't do homework unless it's graded, you should work on that
because it will hold you back and you're training yourself to rapidly forget
information that isn't required to pass a test.

Don't be afraid to try new things that interest you. Be okay with skilled
people saying you're bad at something you did your best at, grow a thicker
skin since those people are the ones likely to be able to give you advice to
become good at it. Professors _want_ you to come to office hours, especially
if you're struggling.

You have to be okay with getting that C-, who cares what grade they assign you
as long as ten years later you still remember convolutions or whatever. Be
solid at a bunch of stuff rather than extremely good at, I don't know,
something super specific like the spin states in cobalt platinum alloys or
matlab modeling of alloy surface energy same as you shouldn't refuse to learn
anything but Python if you want to still be a developer in 2069.

------
manmal
My wife and I are contemplating unschooling our kids. The longer our son is in
primary school, the more we see the deterioration of his willingness to learn
anything school-related. He hates homework (as do most kids), and this is even
more painful to see given that it has been shown that homework is essentially
useless for learning performance. He still likes to write and do math, but as
long as he is in school, we have to take care that his interest doesn’t go
south.

During school holidays, he actually starts to do school exercises for fun;
those are the very same exercises he would fight against doing for hours on a
regular school day. Go figure.

Btw, Gates, Page, Brin - all were not in a conventional school. The capacity
to just build things without fear of being judged is invaluable.

~~~
anonytrary
> Btw, Gates, Page, Brin - all were not in a conventional school.

This is Anecdata^3, you probably shouldn't make rash decisions about your
children based on 3 successful technical entrepreneurs.

~~~
shantly
Especially since going to any kind of private school tracks closely with
“parents have lots of money” which probably has a lot more to do with it.

~~~
manmal
Not necessarily, there are lots of private schools which do things very
normally.

~~~
shantly
And many of those turn out tons of successful people you’ve heard of, too. Bet
the ones that do so at the highest rate tend to be among the most expensive.

~~~
manmal
Yes it has been shown that having successful friends will make yourself more
successful. Putting your kid into a school for rich people will more likely
lead to rich friends later on.

------
ZeljkoS
Yes, most tests are bad, but there is a quick advice that improves them
immensely: never ask questions that are not at least Level 3 in Bloom's
taxonomy of knowledge: [https://blog.testdome.com/blooms-
taxonomy/](https://blog.testdome.com/blooms-taxonomy/)

I cofounded a technical screening startup, and despite our efforts to educate
our customers that enter their questions on our platform, 95% of their custom
questions are bad questions. They tend to ask trivia questions, that can
quickly be googled, instead of asking work-sample questions (which we at
TestDome.com prefer). I think they learned that from years of paper tests in
school, it is very hard to unlearn.

Just to give you an idea, for testing web programmers we suggest questions
where you need to find bugs in HTML code
([https://www.testdome.com/questions/html-
css/inspector/17629](https://www.testdome.com/questions/html-
css/inspector/17629)), while our customers would ask questions like "What does
CSS stand for?"

~~~
appleflaxen
can anybody comment on the difference between "understanding" and "applying"
in that taxonomy. The examples provided don't seem very good to me.

is it the difference between understanding a language and not speaking it,
perhaps? following a conversation, but not being able to participate in it?

~~~
abdullahkhalids
I have learned Bloom's taxonomy with comprehension instead of understanding.
So if in an algorithm's class, you were taught that algorithm X was
exponential time, then the exam question at the comprehension level can be,
"describe why does it take exponential time to run algorithm X".

An applying level question could be, "Find the running time of algorithm Y
(which was never discussed in class)."

------
gnicholas
I usually find PG’s writing to be very illuminating, but I largely disagree
with this piece.

I think the reason founders want introductions to influential people is not
because they think that this is what you need to be successful. It’s because
they have seen many cases where influential people talked up startups that
seemed to be pretty mediocre, and it helped them succeed. And they think “well
my startup is at least as good as those other ones, so publicity by famous
people will accelerate our path to success.”

They’ve seen press coverage of lousy startups propel them into fundraising
successes and growing revenues, even if they never became profitable. And they
think “I’d succeed faster if my legit startup had that kind of exposure.”

It’s not that they think these things are a substitute for having a good
product. They think (correctly) that having these things will accelerate their
growth and somewhat lower the bar to success (particularly if there are
network effects involved).

~~~
ajju
> They think _(correctly)_ that having these things will accelerate their
> growth and somewhat lower the bar to success

Incorrectly, unless you define success as a local maxima. You usually can’t
influence your way to broad user adoption

~~~
gnicholas
What makes you say that? For early stage startups, it seems especially likely
that increased awareness can (though usually will not) create a material
amount of additional success.

I know that a few fortunate tweets my startup received drove tons of traffic
our way, and likely resulted in the press coverage we received soon after. And
that press coverage brought more users, and also the proof points of having
been covered by major media. That makes fundraising and partnerships easier,
etc.

------
hooande
I know that learning how to hack systems is valuable because that's how I got
into YCombinator. I set out specifically to hack the application process and
succeeded. It wasn't even that difficult to do.

If pg believes what he's saying in this essay, I have two questions:

1\. why did he design a testing system that was easily hacked?

2\. why does YCombinator itself focus so much on hacking the venture capital
system, as opposed to making great products? [1]

I think the answer to both is that the ideas presented in this essay don't
scale. If you can't create a 200 person per year startup incubator that
focuses on true learning instead of process hacking, then how could someone do
it for a 20,000 person university? Things get boiled down to metrics like
grades and valuations because they are the only way to measure/manage the
productive output of a large number of people.

The insight that I've had is that as long as large civilizations and
organizations exist, there will be easily hackable systems. There is a direct
relationship between the number of humans involved in a thing, and the number
of opportunities to hack that thing. Understanding and exploiting this seems
like a valuable ability.

[1] I was personally struck by how much of YCombinator was (in 2008) oriented
around learning what magic words to say to investors and exactly when to time
techcrunch launch articles, etc. There seemed to be no time given to "how to
make a great product", likely because this is something that can't be taught

~~~
coffeemug
Companies are this way too. The best companies are all run by OKRs and
metrics. But if you peek under the hood, all of it is insanely hackable; you
have to be naive to try and actually meet your explicitly stated objectives
because that's not how you get promoted.

It reminds me of a quote from The Elephant in the Brain. I don't remember the
wording exactly, but it's something like "if there is a behavior in a large
group of people that doesn't make sense to you, not only is it by design, but
it's also probably the whole point."

Incidentally, I was also struck by how much of YC was (in 2009) about hacking
venture capital. But lots of people from that era now say this behavior is
unethical disavow all knowledge that it was ever actively promoted. It's like
we lived through different realities.

If I were to steelman The Lesson to Unlearn I think I'd say this. On an
individual level there is enormous benefit to learning how to operate without
hacking a test. That's how you do great work; that's what original thinking
_is_. On a societal level this seems unshakable. But societies _have_ made
dramatic shifts in behavior and social organization before. From hunter
gatherers to farmers to city dwellers, for example. So while this is hard to
imagine it might just be that-- lack of imagination. The social return to
abandoning this system might indeed be enormous.

~~~
iamwil
I really like the steelman interpretation. I suspect that often times, PG's
trying to express some sort of core idea, but readers bring their own context
with it, and apply it to something else, in this case, YC. And in a different
thread, interviews.

But if one were to find oneself in a situation where there are no gatekeeper
tests, then how would you go about doing great work? And by being able to see
it as an idealism, at least we have a target to shoot for, even if we might
fall short in practice.

------
pavlov
The tech industry has unfortunately adopted the methodology of centralized
hackable tests as the canonical gatekeeping method in the form of programming
interviews.

Most big tech companies don't care about how good you have been at delivering
some value through creating software: they want to see you deliver a very
specific type of performance at a whiteboard. Interviewers are given specific
math puzzle questions to ask. Interviewees are explicitly told by the same
companies' hiring departments that they should aim to hack the system by
studying books like "Cracking the Code Interview".

This is an industry that prides itself on supposedly making data-driven
decisions through A/B testing. When it comes to hiring people to make those
decisions, everybody just plays along to a decades-old script.

~~~
burlesona
Being a hiring manager at a big company, I can tell you this is just as
frustrating for me as it is for candidates. I hate “leet code” and frankly
find algorithmic interviews to be very low signal compared to more practical,
open-ended, domain-specific problems.

I will say though, the problem is one of “standardization” across an
organization where it’s too big for everyone to fit in a room.

Suppose you give each team high autonomy to hire whoever they like using
whatever “good” process they come up with. 90% of the time this results in
good hires. But as you grow, that ten percent of underperforming people
becomes large in absolute numbers, and is very painful to deal with.

It becomes a real problem when relatively lower performing people end up
concentrated on a team, and then start being the hiring gatekeepers for that
team, thus multiplying the number of lower performing hires.

Later you start having institutional problems when everyone starts to perceive
that the engineers in Department A are generally better than the engineers in
Department B. Engineers in Department A are more likely to leave if they
perceive the company is getting worse at engineering - it becomes a self-
fulfilling prophecy.

Then you get enormous pressure to come up with standardized testing - aka
algorithms on the whiteboard, or some other academic inspired exercise -
imposed by higher level leadership that wants to address a genuine problem
(skill disparity across the org) but does not know any better way to do it.

I think, as PG points out, there may be a real opportunity to innovate here,
and probably a big financial opportunity if anyone can figure out how to
productize and scale a solution.

I struggle to see an easy answer, though. In a utopian universe (for a hiring
manager) I’d do something like pay candidates to come on site and work for a
week, then make a hire/no-hire decision based on that. But I think that is far
too onerous for candidates (and a big company) to have legs.

~~~
OnlineGladiator
> But as you grow, that ten percent of underperforming people becomes large in
> absolute numbers, and is very painful to deal with.

I think you could reduce the 10% to 1% simply by actively firing bad
employees, and the easiest way to do that is with a probationary hiring period
(let's say 6 months, maybe a year).

I am fully aware of how systemic rot can be, and how hard it can be to remove.
The _real_ problem is when you hire a bad manager, because they will not only
let the rot fester but encourage it to grow.

I think you could wildly relax hiring standards for ICs and be incredibly
successful. What we need are better practices around hiring managers more than
anything.

~~~
johnm
Alas, for all of the focus on the interview processes to "prevent" false
positive hires, those same companies suck as actually getting rid of the bad
hires. Some is rational risk aversion to lawsuits and the like but that
clearly bs when you see just how fast people that aren't liked by someone high
enough up the food chain are shown the door.

The effect on morale and real productivity of getting rid of the toxic and
worst performers is amazing.

~~~
OnlineGladiator
The first job I had there was a probationary period - it was good and bad. It
was good because you knew everybody there was extremely talented and you could
rely on your coworkers. There was a bit of an understanding about the cut
throat nature of the place though, and obviously that's not for everyone. But
at the end of the day, when it came time to just get shit done, there was
never any problem.

I can't name that place without doxing myself, but it sounds like Netflix has
a similar culture. Some people love it. Some people hate it. But I don't think
many people would say they have an issue with employing less-than-stellar
employees.

Of course this is only "solving" (it's certainly not perfect) the problem of
hiring for engineers, which most of us are on HN (at least I assume, but
certainly more engineers than engineering managers). It does nothing to
address the issue of hiring bad managers. I have no solution for that, other
than to make sure the first manager is excellent. A players hire A players
because they want to do the best work they can, and will go out of their way
to hire people _better_ than themselves. B players hire C players because they
just want to make themselves look good. Once you have a bad manager, the
assumption is everyone beneath him will be as bad or worse (obviously there
are exceptions).

~~~
johnm
As I mentioned in a sibling comment, the toxic (and incompetent) manager
problem is fundamentally a leadership problem. The single biggest
responsibility of the founders is the culture of the organization --
everything stems from that, one way or another.

Leadership, from the top down, is absolutely responsible for the mentoring,
training, and environment created by the management. None of the systemic
problems with & induced by management is new. It comes down to: does
leadership actually care enough to do anything about it? Companies institute
all sorts of formal stuff like internal surveys and NPS scoring and so forth
but it always comes down to trust & communication and then follow through. And
the follow through is the clearest way to create trust. All of the things that
even well intentioned leaders spout, if not followed through, quickly erodes
and ultimately destroys trust. So, just like with kids, don't make promises
you won't keep.

~~~
OnlineGladiator
Just out of curiosity, why do you make multiple replies instead of one larger
reply?

~~~
johnm
Because you made a variety of specific points and it's easier to address them
specifically.

------
jsherwani
We (Screenhero, W13) may well have been one of the many YC companies PG’s
referring to! But instead of hoping there was some test to hack, we were
worried that there was some unknown hack out there that we didn’t know. It was
quite a huge relief when PG told us something to the effect of “all the
successful startups have found their initial grown through just one thing:
delighting their users with a great product”.

That realization / confirmation freed us four engineering cofounders from
worrying about some “growth hack” we never learnt, and helped us focus on
prioritizing our product development through talking to users. We then used
Sean Ellis’ “very disappointed” survey methodology (which Rahul @ Superhman
recently wrote about) and used it as our North Star to find the few features
to focus on polishing.

We got acquired by Slack in 2015, and built out Slack Calls. I left last year,
and I’m now in the final stages of launching a new product around super-
charged meetings for remote teams (picking up where Screenhero / Slack Calls /
Zoom left off), and am using the same principles again. No hacking of tests,
just building a product that people want, and using their feedback as the only
valuable bit of data in determining priorities. Thanks PG and YC (and Sean
Ellis!) for startup principles and methodologies that have stood the test of
time.

------
mattbee
This spends too many words justifying the stale "real hacker" tradition of
dunking on education, and kindof ignores the problem of how to measure
something so subjective as learning.

So people who are good at tests get ahead. We get that you're not interested
in grades. We all know people who play the game and win, not sincerely engage
with their job or education or whatever.

But I'd like to hear from pg / other people in YC how they think they are
"hacked". What do their successful applicants actually optimise for when
"delight" and even "growth" are just as subjective as "learning"? After all
they fund lots and lots of companies and founders, many of whom make no
returns. How do founders keep their funding "success" long past the point it
was deserved?

~~~
dkyc
"Growth" in the YC definition ("revenue growth with positive unit economics",
or the closest proxy to that) seems a pretty objective test to me.

~~~
killerstorm
It can take a lot of resources to get to that point, no?

So there's still a question what companies should do until they reach this
growth possibility.

------
fhennig
This is a very interesting thing to think about, I thought about it a bunch of
times already, and have a couple of thoughts about it too.

First of all, once you're 55, it's easy to say stuff like that, because you
won't be tested anymore. And I think there is a good chunk of survivorship
bias: "Hey, I made it without worrying about tests, so you can, too!" Although
that is probably not true for a big chunk of the population, especially once
we look at non CS people.

Then I think, he is fundamentally right. Grades shouldn't matter so much, it
is about what you learn. And I like to approach things that way too, but in
the end, I always have to study for the tests as well.

The problem is exactly how deeply it is ingrained in everything. If I just
stop caring, it doesn't really help. I will just end up in a worse position,
since everyone around me still cares about test results. The people that need
to change their mind are the people that use tests as measures of qualities
that they are not a good measure for. I think a lot of people are falling
thought the cracks because they don't fit the expectation of a HR person close
enough.

In the end, we will always have to have tests. A completely individualized
assessment of peoples qualities or fit for certain roles just doesn't scale.
It would be great though to come up with new ways of testing that are maybe
more in line with actual learning. For example, it would be nice to have tests
at university where I can google things, just like in the real world.

I am lucky enough to be able to work in the booming tech sector where there
are so many jobs that it is fine if I don't work towards a test.

~~~
vntx
> For example, it would be nice to have tests at university where I can google
> things, just like in the real world.

One of the best professors I had at university for a Linux course always said:
“If you don’t know something, google it!”

He backed up his words by allowing googling on the actual coding part of his
exams. You had to understand the underlying concepts but you never had to
memorize syntax. One time, he had us write a small program in C that involved
threading during a proctored timed exam. Everyone ran out of time for that
part.

His course and exams were _rigorous_. You could never cram for his class and
expect to pass and indeed I know of one student repeatedly failing his course.

This was one of the courses in university where I gained the bulk of my useful
knowledge and practical skills I use to this day. I can count the courses that
were of similar quality I took at university on one(1) hand. The rest of my
degree involved hackable tests, apathetic professors and were massive time-
wasters. I learned nothing from them, obviously.

The good professors made me realize how bad the rest of my degree was and how
much of a racket higher education could be.

I look back at my undergrad days and wonder how much more I would have learned
if I had not focused so much on grades. I’m still actively trying to unschool
myself.

~~~
jacques_chester
> _He backed up his words by allowing googling on the actual coding part of
> his exams. You had to understand the underlying concepts but you never had
> to memorize syntax. ... His course and exams were _rigorous_. You could
> never cram for his class and expect to pass and indeed I know of one student
> repeatedly failing his course._

When I studied law, all our exams were open book. Bring whatever you want. I
brought casebooks and printouts of legislation and important cases. Others
also brought headnotes and volumes of _Halsbury 's_ (a legal encyclopaedia).

None of this helped if you didn't know the material well enough to identify
the legal issues in each question.

Which is why I failed Torts.

~~~
vntx
> None of this helped if you didn’t know the material well enough to identify
> the legal issues in each question.

My machine learning professor who was from CMU would allow us to put anything
we wanted on both sides of one piece of printer paper to bring to exams.

No amount of notes helped if you had no understanding of the algorithms and
formulas.

I almost failed that course too, but it was also one of the cornerstones of my
university career. It made me realize how much I had been optimizing for the
wrong thing: grades instead of understanding.

------
daly
I took a class from John White at UCONN. He gave us a couple dozen CS papers
at the start of the year. Every class had 3 paper presentations, 20 minutes
each. A random student was picked for each presentation.

So you had to read and understand each of the upcoming 3 papers because there
was a random chance you would have to present one of the three.

The result is that you eventually read all of the papers well enough that you
could give a 20 minute presentation.

There were no grades. It was pass/fail.

It was one of the best classes I ever took.

~~~
bonoboTP
But the person who presented the very first paper won't have to care about the
rest of the semester right? Or can you get randomly picked multiple times?

~~~
daly
There were only 10 students allowed in the class. So you got picked about
every 3rd class (sometimes more than once per class).

~~~
jventura
I wish I could do that to my students, but I have an average of 120 students
on my classes each semester.. :/

------
choppaface
“But you can't blame teachers if their tests are hackable. Their job is to
teach, not to create unhackable tests. The real problem is grades, or more
precisely, that grades have been overloaded.”

Two important things here: * Grade inflation makes for happier students and
counterbalances the value of test hacking. Today there are very few schools
that don’t do grade inflation. Without considering grade inflation, much of
this essay is simply Graham entertaining his own nostalgia. (I’m not
necessarily a proponent of inflation but it’s a key phenomena missing from the
essay).

* Good teachers hold lots of office hours and give good feedback through those offerings. Email support is also a lot more popular today. You can also play games like test corrections, rough drafts, etc. Students don’t learn well without feedback, and most teachers who don’t invest in it won’t succeed today. This focus towards feedback in modern teaching is at odds with the education community profiled in this essay.

If there’s one lesson to un-learn, it’s to forget about giving attention to
non-constructive papers that lack evidence. While this essay gives a thorough
criticism of tests, it offers no concrete alternative. And it offers no
grounding for the claimed ‘good’ student who ‘focuses on valuable learning.’
Graham, like any other VC, seeks to upwell sentiment in the interest of
controlling it (e.g. creating an investment asset out of it). Reading this
essay doesn’t teach you anything about hacking unless you recognize that the
author is trying to hack you himself.

~~~
pmichaud
The suggested alternative I got from the essay was "try to cause a real,
specific outcome in the world. If the action you took caused the outcome, you
passed the test."

~~~
choppaface
Did Sam Altman pass the test with Loopt?

Just because you made waves somehow or filled your pockets with money doesn’t
mean your time was ‘better spent’ than trying to make A’s in organized
education. School can very well teach you how to make better outcomes.

But, yes, school steers youthful energy away from making change... err making
money for VCs.

------
japhyr
My son is 8, and yesterday he came home from school and said, "I'm not above
grade level in math." He sounded a little disappointed, but not too much; he
also seemed to be asking if that really means anything.

We had a really interesting conversation. I've been a middle and high school
math teacher most of my life, so this is quite familiar ground for me. Over
dinner I asked him how he would go about measuring whether a kid is "above
grade level". He got really animated and started to tell me what he thought
second graders should know and be able to do, and how third graders should
expand on that, and what he thought he'd start to learn in fourth and fifth
grade.

I told him I could start to teach him some of the operations and skills he'll
learn in the coming years, but that it probably wouldn't mean a whole lot.
Instead I asked him what ideas he's already familiar with that other kids his
age don't know about, because we talk about math almost every day. "I know
what phi is, and pi!" Yes, and we've also talked about limits, calculus,
negative numbers, fractions, decimals, and he's looked at pages of math that's
well beyond my understanding as well.

I told him that when he gets to middle and high school classes, he's going to
see higher level math and it's going to sound familiar because we've already
been talking about the ideas for years. He's going to watch other kids fail
and drop out of those classes because the ideas and the skills will be too new
and too difficult for them to take in. I said we're probably much better off
just continuing to talk about fun and interesting areas of math for the time
being instead of worrying about "grade level".

------
adwn
> _The way you get lots of users is to make the product really great._

That is wrong – or, at the very least, incomplete. "Build it and they will
come" is the dream and misconception of every programmer who's talented at
software development but not at sales. Hell, this is the number one advice of
every business how-to book ever: You can't rely on people finding out about
your great product by chance, you need to put in the work and sell it!

> _Then people will not only use it but recommend it to their friends [...]_

That's not how it works for B2B software, and only rarely for B2C. Usually, if
a product is successful, then as a result of good marketing (a good product
ist somewhat necessary, but not sufficient).

~~~
TeMPOraL
100% right, and the corollary of this is: you don't actually need to make a
great product, if your sales&marketing game is good enough.

Or, in other words, a lot of successful business - including startups - is
exactly the "test hacking" PG urges founders to unlearn.

I'm honestly surprised by this essay arguing that "hacking the test" is the
wrong approach in startups/business. "Hacking the test" is essentially what
your marketing is supposed to do. These "non-authoritarian" tests like selling
things are just as hackable as school tests; you just have to discover how the
system really operates.

The whole authoritarian/non-authoritarian split doesn't carve reality at the
joints, IMO. Football match is really an authoritarian test - it tests who
wins under the game rules, which are given from the top. It may be hard to
hack, but that's because passing that test is usually synonymous with the
terminal goals of the test takers - i.e. they want to win the game. You'll
note however that, once money gets involved, people are sometimes made to
deliberately lose games; in these scenarios, there are usually other people
involved who are absolutely hacking the test.

The best way I've found to carve the reality at the joints is to talk about
terminal and instrumental goals. Learning useful things isn't the terminal
goal for most students, getting a good career is (and/or not pissing off
parents by getting bad grades).

So I'm ultimately surprised that PG argues that building good things, not
hacking the test, is how you win the startup game - I would think most of the
startups have products as instrumental goals; the _exit_ is the terminal goal.
And building a great product isn't the best way to achieve that goal.

~~~
derefr
> you just have to discover how the system really operates.

One might say that, by analogy, this is like saying you cheated on a test by
studying. “Learning how the system really operates” isn’t _hacking_ ; it’s the
thing you’re formally setting out to do as a business!

~~~
TeMPOraL
The hacking starts once you learn that building great products isn't the only,
or even the best, way to create a successful business.

------
esotericn
Not sure I agree with this.

The real world is very much about optimizing for and beating tests.

It might be producing a specific CV or preparing for an interview whiteboard
session to land a job.

It might be socializing and networking in a specific way in order to land
funding.

The real world does not commonly reward just being really good at arbitrary
things. It's almost always focused on meeting a need that someone else has
defined, much like a test.

~~~
xorcist
That is only true if your life ambition is securing a job with that successful
tech company.

If your ambition is being successful on your own, as an academic, an
entrepreneur or as an artist, there are no standardized tests to beat.

~~~
watwut
Being successful academic is all about optimizing for various tests and
metrics and beating tests. It would be hard to find more institutionalized
occupation.

------
grappler
I think there are some good insights to be gleaned from this essay, as with a
lot of pg's essays. The perspective of the guy who founded and ran Y
Combinator is always an interesting one to hear from.

However for this essay in particular, had I been among the set of draft
readers, I would have suggested toning down the “I hadn't realized X until
now, so now I think other people haven't realized X either” message.

Back in college I remember noticing choices of “better learning vs better
grade” and deliberately choosing the former with a proudly “screw the
authorities” attitude. I'm sure plenty of other Tara Ploughmans could say the
same.

------
bkohlmann
An important corollary is that school and tests teach you not to be wrong.
They teach you that incorrect answers will be punished.

Yet, most interesting questions in life don’t yet have defined answers. Thus
you need to have and test a hypothesis, which will very often be “incorrect”
the first time around. But that doesn’t actually matter - the mere act of
thinking about and defining what an answer could be sets you up to iterate and
test. Eventually, you may find product market fit (a - not the - right
answer).

You have to be willing to be wrong at first to learn...and win.

~~~
robocat
Even worse, most tests have a well defined question and a correct answer.

In real life you don’t know the question, there isn’t a single canonical
answer, and there is no score-keeper to tell you if you are passing.

------
AlchemistCamp
This is great seeing so many PG essays! I've really missed his writing.

> _And at elite universities, that means nearly everyone, since someone who
> didn 't care about getting good grades probably wouldn't be there in the
> first place. The result is that students compete to maximize the difference
> between learning and getting good grades._

> _When I started advising startup founders at Y Combinator, especially young
> ones, I was puzzled by the way they always seemed to make things
> overcomplicated._

I suspect part of the problem (of early founders being focused on hacking
tests) was related to how heavily YC funded people from elite
universities—which select people who do exactly that.

~~~
anonytrary
> This is great seeing so many PG essays! I've really missed his writing.

I love reading his tweets! They're all so down to earth and illuminating. That
said, I'm not a fan of his blog writing as much. I find it full of overly-
sparse nuggets of wisdom. It needs to be more concise, to the point, with
fewer clever metaphors. This article could've been compressed 70% and still
have been human readable.

~~~
pyb
pg seems to have a backlog of essay ideas that he's catching up on. This is
all good, but also means that they get written a bit too quickly. It's the
classic "had I had more time to spend on this text, it would have been
shorter" syndrome.

~~~
anonytrary
Indeed, time is of the essence; playing tug-of-war with quality.

------
wrinklytidbits
I'm a non degree student at an ivy league and I took a course this semester
because I wanted to learn the basic fundamentals of electrical engineering.
Because I'm learning because I want to, I put in effort into things I find
interesting and difficult without the pressure of deadlines and grades. I love
grades because it gives me the feedback I need (otherwise I would "learn" from
youtube videos and pdfs) to assess how well I know the subject.

I can take breaks from truly hard problems until I have enough clarity of mind
to approach the problem with a new perspective. I can't do that with a
deadline.

I don't care if the answers "are out there" meaning the professor has returned
graded tests and posted the answers; I do the work because I want to learn.
That paradox of students wanting to review graded homeworks and students who
want to learn by doing the homework is resolved practically by those that are
more vocal about their wants.

I'm happy to take my time, doing my best (with the understanding that my best
takes time) and observing my grade afterwards. Sometimes my learning takes me
down paths that harder and more rewarding with the consequence that I take
longer than others: I am a slow learner.

------
astatine
I don't think students are the blameless victim of this "system". I know of a
continuous evaluation system adopted in a college, where the final tests had a
less than 50% weightage for the grade. Thr students rebelled. It was far
easier for a majority to burn the midnight oil for a few weeks than be
diligent through the course. The test model has stuck because it's convenient
for everyone, regardless of its clear weaknesses.

~~~
matwood
I'm one of those weird outliers I guess, but even when there were only 1-2
exams for the whole course I stayed diligent throughout. The night before a
test I just did a bit of review, and got a good nights sleep. I never
understood those who tried to cram because it's not something that would ever
work for how I learn and understand.

------
bmahmood
Likely unpopular opinion here, but it seems a little unfair to fault YC
founders for believing "that the way to win was to hack the test", when the YC
application itself would seemingly select for founders that exhibit this
behavior with questions like "When have you most successfully hacked a non-
computer system to your advantage?"

I'd posit the YC application in and of itself shares some of the facets that
the article is critiquing in a "test". The fact there are paid services
popping up to review YC applications reminds me of SAT Prep services.

------
Hendrikto
> In theory you shouldn't have to prepare for a test in a class any more than
> you have to prepare for a blood test.

The key here is the “in theory” part. It is very hard to design exams that
actually test your understanding of some topic as opposed to your memorization
skills.

As somebody who recently got his bachelor’s CS degree (currently working on my
master’s degree), and who also puts more emphasis on actually learning rather
that getting good grades, I can tell you that understanding a subject might be
enough to pass, but is seldom enough to get good grades.

~~~
asdfasgasdgasdg
This is especially true in a domain that has no practical purpose. In computer
science, the "exam" could be to go solve some hard problem, or write a program
with a particular effect. In engineering, you can have students build a bridge
and then stress test it. In athletics, the exam is competition. In business,
it's making money. In dating, it's finding a mate (or whatever your personal
goals are in that space). In farming, it's making food come out of the ground.
For writers, it's a compelling or profitable story or book. For painters, it's
a painting. Woodworkers can build a chair.

All those domains are testable. But what is the practical work product of a
deep knowledge of medieval history? It's extremely difficult to test the past,
so it's certainly not predictions about what already happened. Nobody needs
knowledge of medieval history for any practical purpose in the modern day.
There is no possible way to test for this knowledge in a practical scenario
because there is no practical outlet for the knowledge. Literary criticism is
the same, along with much that is called "liberal arts" today. So for subjects
like this, exams and essays are the only possible work product, and it's true
that at that point you're going to have to use contrived tests, since real
tests don't exist.

~~~
spats1990
>But what is the practical work product of a deep knowledge of medieval
history? It's extremely difficult to test the past, so it's certainly not
predictions about what already happened.

This strikes me as a really sad way to see the world.

Would you then agree that say, archaeology or palaeontology are things that
nobody needs knowledge of for any practical purpose in the modern day? If you
disagree, why?

~~~
asdfasgasdgasdg
I didn't say we shouldn't study those things. I just said that there is not a
much of a practical work product, and especially not one that a student could
produce.

But, I also think that probably more people are studying them than need to be.
If that is a student's interest, then well enough. Otherwise, I'm not sure
that, e.g., a middle school medieval history class is a good use of a
student's time.

~~~
spats1990
There is a practical work product: research, formation of theories, etc. Edit:
Possible new ways to understand the modern world. Just like archaeology or
palaeontology.

In my opinion, the issue in liberal arts education is with grade inflation,
along with the scalability of assessing learning discussed in the essay/these
comments.

I suspect that really rigorous liberal arts education is much harder to do at
scale than really rigorous science/engineering education because to be blunt,
there is less room for argument in scientific concepts.

For example, properly grading student essays and research is more difficult at
scale in the liberal arts and the level of the whole thing gets dumbed down as
a result.

------
_hardwaregeek
Hackathons are a great example of this. Ostensibly they should just be about
the project that is the coolest or most interesting. But participants started
to realize that it wasn't _really_ about the most fascinating project. It was
about the project that got the usually non-technical, naive judges to go
"ooooh". People started hacking hackathons by building projects that were
totally infeasible and completely fake in terms of technical implementation,
but had some sort of wow factor through buzzwords (AI/ML/CV/VR/etc) or "making
the world a better place".

It's no coincidence that some of the largest hackathons are run and attended
by students from the top universities pg mentions in the essay.

Which is why YC's hackathon was so satisfying. It was great seeing people
pitch the normal hackathon-y ideas, then getting simple but direct questions
such as "Why would someone use this app?" or "Why wouldn't I use <alternative>
instead?".

------
bavcyc
Interesting essay, however it is written from a perspective of having basic
needs met and having the resources to spend time 'hacking' a system. When you
are struggling to pay the bills for the basics, you don't focus on the hacking
just on the surviving.

The question I have is how do you hack the system such that you help those who
are struggling to survive?

------
smiley1437
I love Paul Graham's articles, but isn't this Goodhart's Law but with more
words? (When a measure becomes a target, it ceases to be a good measure)

~~~
acidburnNSA
Searched the page for Goodhart and found this. There's added value here as
this is a discussion of why Goodhart's Law is bad and how it's so prevalent in
our education and business systems. I knew about the law but still got a lot
out of this essay.

~~~
dccoolgai
I did a Ctrl+f "goodhart", too. Glad to see that there is growing awareness of
this principle... I think it underlies a lot of basic problems with
society/tech/government in general. There is a related real-life parable I
like about the ww2 plane designer who put more armor where returning bombers
_weren't_ shot up, but I forget his name.

------
guptaneil
The problem is not that fundraising _feels_ like a test, it's that fundraising
_is_ a test posed by authority figures, and by pg's own definition, is
probably hackable. How hackable is proportional to the quality of the VC.

Luckily startups themselves are indeed not a test. So you can avoid any test
by making enough revenue to not need external funding, but the second best
strategy is take the VC's test without trying to hack it as your own test of
the investor. Assuming you're a good investment, you'll beat those trying to
hack the test unless the investor's test is susceptible to hacking. If you're
not a good investment (which is by definition not obvious in the early days),
maybe you should just try to learn to hack the test? Ironically, one of YC's
biggest value adds is that it helps you hack those tests by lending you their
name.

~~~
ignoramous
Paul, though he doesn't reference his previous essay [0] (which has a section
on _gaming the system_ ) in the current one, did write then that _fooling the
investors_ and treating it like a test is only going to delay the inevitable.

[0] [http://paulgraham.com/before.html](http://paulgraham.com/before.html)

------
aex
I've found about how to hack the tests quite early on. I've had straight A's
until I no longer cared about the grades. I've also realized that entering and
pursuing a career at a tech corporations is a game as well.

What has been driving me to startups is the fact that creating a profitable
business has no place for hacks.

Fundraising is a game that could be easily hacked though. I thought it was
only possible to hack the fundraising game for a couple of rounds but Adam
Neumann et al. changed my mind that hacks could carry a business even to post-
IPO.

~~~
geofft
> _What has been driving me to startups is the fact that creating a profitable
> business has no place for hacks._

Not sure that's true. Marketing is a hack, for one. If you have the best
product but nobody knows about it, it's no use.

More generally, there's a _reason_ that the tech career ladder is hackable:
performance reviews are both important and also too expensive to do right. If
you want a real evaluation of what I've done over the past year, what you
really need is someone following me 40 hours a week, noting how I contributed
(positively or negatively) at small meetings, keeping track of whether my
projects are late because I helped someone with something truly important or I
spent my time on HN, etc. But you can't assign one reviewer per employee, so
you make an approximate process where employees self-report what they did and
managers report the fraction they've seen. You also can't get rid of the
process, because a simple profit motive demands you incentivize employees for
actually delivering business value. So you have a process that's vulnerable to
exploits like flashy launches that will wither in a year.

If you as a business owner aren't evaluating your employees, you won't be
profitable. If you're watching each moment of your employees, you're wasting
time. If you spend your time developing a fairer review process, you're not
working on your actual business. If you don't hire employees and just put
yourself out there in the market, your customers certainly aren't evaluating
you fairly. (And of course _your employees_ and potential hires are evaluating
_you_ on partial data, and it's in your interest to hire and retain good
employees.) So whatever you do, it's hackable, and if you don't play the game
you'll forfeit it.

------
asveikau
> If getting into college were merely a matter of having the quality of one's
> mind measured by admissions officers the way scientists measure the mass of
> an object, we could tell teenage kids "learn a lot" and leave it at that.

It is equally naive to think some idea of "quality of mind" or what someone
has already learned in their first ~17 years is what they should go by. This
is just shifting from one bias to another.

Evaluating a human being, reducing them to a number or a boolean, is hard, and
people tasked with doing this, no matter how confident or how much thought and
intelligence they put into it, will get it wrong, often with bad consequences.
So perhaps we should take such systems lightly, stay humble, and not get
overconfident in our judgements.

------
empath75
And yet I’ve interviewed with 2 or 3 different y combinator companies that
hire based on coding tests that test nothing other than that you’ve memorized
“cracking the coding interview”.

------
BenoitEssiambre
This is one of the reasons I quit academia and am skeptical of much of the
research coming out of universities. As a grad student, at some point, it
became pretty clear that I would have to choose between doing good science or
doing career advancing work (by ignoring biases in my research, p-hacking
etc.). Universities have self selected for people who do bad but publishable
science.

------
jakobmi
It's exactly the same with hiring at FANG/McKinsey/BCG/... :). You optimize
for the (well known) interview process. You study 10 hours/day for 2 weeks.
And you get the job with 99% accuracy.

~~~
avindroth
The artificiality (?) is what gets to me tbh

------
marccarre
The problem might be a lot bigger.

Possibly most of one's life is spent (over?) fitting to some "test" in a
system, or "expectation" someone else has, instead of doing what they truly
want: trying to pass a test at school, trying to land that job (you think) you
want, trying to be socially acceptable in the schoolyard, at a party, or in
the office, trying to please a parent, a friend, a partner, trying to conform
to some unconsciously shared vision of "success", trying to conform to your
own vision of "success", etc.

The list is long.

Fairer, unhackable systems would definitely help in reducing the amount of
such "tests" we artificially created as a society. We should strive for this.

However is it possible to solve human nature? To solve "affinity bias",
"confirmation bias", "psychological needs", etc.? Is it possible to create a
society in which everyone is truly free? Or at least as free as possible from
all this? Is it possible to solve freedom & wisdom? :-)

------
raldi
Hacking tests is a lesson much older than modern schools. It's a fundamental
part of being human, or indeed, an animal.

When a peacock is being judged for his feathers or song or mating dance, these
are all being used as proxies for the actually important thing, the quality of
his genes. And it's not just a male thing; women do all kinds of work to
appeal to arbitrary male preferences.

On the other hand, survival of the fittest is an honest judge, so maybe these
peacock tests work better than we think?

------
dnprock
I feel like Paul Graham is unlearning his own experience. Maybe, he's at the
top of the game and tries to figure out why it's messed up. Life is full of
hacks. People hack stuff: school, test, career. The site we're on now called
itself Hacker News. I suppose Paul Graham gave it the name.

When I was in school, I did all kinds of hacks. I didn't graduate high school
in the US. I took the GED to go to college. I did challenge tests to skip
classes and save money. I got to know people who collected old tests. They
trade old tests with me for homework help. Everyone graduated, got jobs, and
continued their life hacks.

The focus on learning and doing is important. As long as we're delivering, we
can hack some stuff. The philosophy debate between Kant and Locke will never
end. There's a balance for each person. We need to discover it ourselves.

~~~
GavinMcG
It seems like you're responding to the idea that "hacking is bad" and that's
not at all what the essay got across to me. In fact, it outright agrees that
"the focus on learning and doing is important."

Instead of "hacking is bad", the essay was saying "incentivizing young people
with a single easily-hacked methodology is bad".

------
Wowfunhappy
> If you merely read good books on medieval history, most of the stuff you
> learned wouldn't be on the test. It's not good books you want to read, but
> the lecture notes and assigned reading in this class.

It is the professor's job to decide _which_ parts of medieval history are the
most important to learn. Therefore, those are what I need to study.

Maybe not all professors do this effectively—but as long as we're talking
about ideals, this is how it should work.

~~~
thepete2
> And even most of that you can ignore, because you only have to worry about
> the sort of thing that could turn up as a test question.

~~~
Wowfunhappy
That part just did not describe my experience in college. Maybe I was bad at
knowing what "could turn up as a test question."

My college also did not have any old tests floating around, at least as far as
I was aware.

------
r3trohack3r
I personally don’t think this is _the_ problem with education. Education has a
lot of shortcomings but, as some siblings point out, having a standardized
measure of learning is a _hard_ problem.

The thing that had always left me a bit uneasy after my rocky road through
education was just how damned certain everyone is about everything. Our
education system, at least up through undergraduate unless you dip your toes
into undergraduate research (and even sometimes still then!), teaches you
things we “know” about the world we live in. Humans have discovered an
_incredible_ amount about the world around us, but we have a _tonne_ left to
figure out. We don’t lean into the unknown with our education system until
late stage. The masses miss out on the beauty of the unknown.

------
brador
Good grades = Passing well paying jobs resume filter = startup initial capital
for 90% of the population without access to the bank of mom and dad or a large
inheritance.

PG starting to lose touch with regular joe.

~~~
0x445442
Yeah, I had a similar take. A lot of what he writes seems to come back to
keeping the YC pipeline full.

Half way through the essay I was thinking, surely this can't be the first time
PG has realized the idea of "playing the game", which is what I've known it as
all my life. But then at the end of the essay he uses that exact same phrase
to hit home the point that "playing the game" is antiquated now if one desires
to become rich.

While I'd agree there are more opportunities for individuals or small groups
to get rich now than in 1960, for the vast majority of us, the best bet to
become rich is to "play the game".

~~~
dennis_jeeves
Had to chime in with a nod of agreement to both brador and 0x445442 .

I'm not being normally someone to comment on such things ( why bother with the
effort?). I am making concrete efforts to get noticed by identifying
groups/people who might align with my own thoughts/beliefs for my own vested
interests.

------
jhatemyjob
Paul Graham is one of the most powerful people on the on the planet. Of course
he sees "hacking bad tests" (aka politics) as something that will become
obsolete. He has the privilege of ejecting from any political situation he
finds unpleasant.

Meanwhile, the other 7B population with substantially less power, we have to
deal with each other. Politics are, and always will be, a thing. As long as
people exist.

------
auggierose
Love it. It has been a while since I read a P.G essay, although this was what
first attracted me to HN. It also means that it is OK to work on your idea
without entering YC. After all, being accepted into YC is just another test by
authority at this point.

------
ynniv
Artificial "hackable" tests are opportunities to see progress at scale. While
I appreciate the recognition that bespoke, perhaps Montessori, education might
better prepare people for the open ended ness of life, there aren't enough
unicorn startup opportunities in the world to use that as a primary means of
knowing how well you know things. Evaluation through essays is better than
multiple choice, the Socratic method is better than essays, but we don't know
how to scale those things yet. Perhaps some day we will have a Young Ladies
Illustrated Primer, but until then we have hackable tests.

------
crocal
I think the elephant in the room is the question of purpose. What is the
purpose guiding your actions? Getting good grades is a mean that /can/ be
necessary to achieve the purpose you have chosen in life. But it’s not
sufficient for most purposes. On the other hand, once you have a purpose, you
truly make the most of education.

------
njacobs5074
That is a great essay. Thank you for sharing.

I've worked in tech for quite a while now. I used to buy into the meritocracy
cant but as the years wore on, I realized how much bullshit it was.

I think the biggest moment came when I started to think about how to explain
to people in my new home, South Africa, that working in tech would free you
economically. There's some truth to that. But that's not the whole truth.

The truth is that all those years ago when I got my first coding job in
financial services, I ticked 3 important boxes for the interview: white, male,
and university-educated.

------
keithyjohnson
As an engineer, this makes me wish there were a better path to FANG employment
than hacking their whiteboarding interviews by leetcoding for weeks. They
probably do this because it's more objective and simpler than something like
the "long conversation with a professor" test that pg suggests.

------
QueensGambit
This is a great essay. But, I find it odd that YC's famous "cohorts" are
invented by PG. Cohorts work because startups in that batch compare against
each other, which in itself is a grading system. Getting into YC, topping the
batch and raising money are inevitable symptoms of "cohort" thinking. We could
argue YC uses cohorts to make startups compete for meaningful metrics instead
of vanity metrics. But still, YC is based on this innate human nature to
compete and get social acceptance.

------
jimduk
I think at various points in life, you worry about the whole system and its
effects on people, not just maximizing your local niche. This seems good, both
morally and from a systems perspective.

For me when younger this was about Ivan Illich (deschooling society), Feynman
(the Brazilian light polarization story) etc. Now I'm older it's reading the
anger in 'Excellent Sheep', watching the pain in low-trust workplaces, and
telling my kids I value real learning over test-passing (they listen, but huge
pressure to conform).

It seems true that a new field is more intellectually stimulating and
liberating than a codified one (I've talked to older female analyst
programmers from the 70s/80s who had a blast with computers and left when it
got rigid and boring, I worked in Gurgaon early 00s when those interested in
IT were self-educating geeks doing it for love, who later got moved aside +
down when the money arrived). Maybe this is why IT people chase the new shiny,
as a defensive strategy against codification, a red queen game to keep the
game interesting ( though inefficient).

Anyway, glad Paul is publishing these viewpoints, even if the actions to be
taken are not clear. Also, society is over-geeked currently. It's not obvious
that hackers ( or intellectuals or logistics experts or technocrats) should
have too much power instead of History students, or politicians or sports-
stars or whoever. Life is both a game and not a game, short term gaming has
long-term consequences. History and Biology and Literature teach us this
clearly.

------
anonytrary
> No one was pulling all-nighters two weeks into the semester.

As a physics student, I can say we definitely pulled 2am-ers 2 weeks into the
semester. Our Condensed Matter professor gave us some of the most random
questions that no one could anticipate. To be honest, I was one of those
"diligent" students who got good grades. I'll tell you right now his quizzes
were _not_ hackable. The following held true:

    
    
      |quiz subject matter| >> |lecture matter|
    
    

He tested on the former, so we had no way of memorizing or hacking the quiz.
You had to really understand the concepts to do well on his quizzes. Ditto for
my grad Quantum Mechanics class.

On the other hand, I've also had professors who give exams pulled from the
internet and allow use of the internet while taking those exams. I hacked that
immediately during the exam, then almost got in trouble for it.

> ...if the professor tells you that there were three underlying causes of the
> Schism of 13... you'd better know them.

A professor like my Condensed Matter Physics professor would go on and on
about 3 fundamental causes and then quiz students about the 5 fundamental
causes, giving a zero to anyone who couldn't think of all the base 3, then
grading the remaining students who thought creatively of 2 additional causes
against each other on a bell curve. :)

~~~
majos
> A professor like my Condensed Matter Physics professor would go on and on
> about 3 fundamental causes and then quiz students about the 5 fundamental
> causes, giving a zero to anyone who couldn't think of all the base 3, then
> grading the remaining students who thought creatively of 2 additional causes
> against each other on a bell curve. :)

To me, this sounds like a plausible argument _against_ such tests.

The issue of creatively grading responses to an ill-defined question often
pops up in discussion here about interview practices. In those discussions,
typically someone will say “I don’t do a generic whiteboard interview. Instead
I do [idiosyncratic thing x]. It really gives me amazing insight into the
candidate.”

Then someone else says “Yeah right, a weird test with unclear metrics just
gives you a big empty space to fill in with all your biases and pick someone
who answers the way you would”.

Of course, both sides are exaggerated here. But it’s not clear to me that
“creative” tests are necessarily any better.

------
daly
Forget the grade. Impress your profs. Let them learn from you.

For example, you could work on analog computers (look it up). The early AI
systems were all analog, using motors on "volume knobs" to adjust the weights.

Analog is MUCH faster than digital. You could probably train an analog neural
net "at movie speeds" because it all happens in parallel. Analog weights are
trivial (capacitors), backprop is just differentiation which analog does
trivially. Function inversion just involves putting the function in the
feedback loop of an op-amp. A board with 100x100 mini-caps would be dirt cheap
and would have 10,000 "weights".

Besides being "up to speed on AI NNs" you would also be breaking new ground
(well, really old ground but nobody remembers anything before last year).

Imagine talking the department into an "analog AI lab" in cooperation with the
EE dept and you get to be "the person". Heck, now you've nearly "joined the
department" rather than being a student.

Think of college as a big company that lets you do ANYTHING. Profs have to
prove themselves to get tensure. Students have to "get good grades" and "learn
to hack the FAANG interviews so they can be a 'good cultural fit (aka grade
hackers)". But YOU are at college as a "free agent" and can use all the
resources to excel at what you love. Imagine having 4 years at a company with
the resources of a college that lets you do ANYTHING.

You will never have it this good again. Forget the grade.

------
eloff
What if we moved to pass/fail grading?

Every assignment and test is just marked pass or fail. Realistically tempered
by what percentage you're allowed to fail for the class, e.g. 40% for
university calculus - no different than grading on a curve today.

Fail any N assignments/tests and fail the course. Maybe some are weighted to
count extra, e.g. Final project, midterm, final exam.

This is really not much different than the current system, but if you stop
caring about the difference between C and A, students can either slack off or
focus more on learning (if the subject is interesting).

Nobody cares about your grades after your first job anyway. I had nearly a
perfect GPA in college, and it was a terrible waste of time in hindsight. If
I'd spent more time socializing maybe I would have made some lifelong contacts
that would have benefited my career or become co-founders with me.

I've always thought to tell my hypothetical kids to not sweat the grades and
remember that the benefit of university is really the friends (and sometimes
life partner) who you meet there. It's total shit when it comes to learning
stuff. A self motivated person can learn more in a year than a 4 year degree.
Somewhat depends on the subject though - I wouldn't want a self taught doctor
who didn't do the practical parts of the education.

------
irjustin
So while I agree with what he is saying directly, to me, testing is a function
of filtering at large scale. To a secondary degree, measuring as feedback for
that system.

Testing and grades in general are a society level filtering mechanism and is a
measuring proxy for how the work force filters for candidates at the entry
level. As most know, once you've got about 10 years, most of that is largely
ignored at the bottom of your resume.

Entrepreneurs, in the eyes of this system, are deviants of sorts. Hacks who
couldn't/didn't want to make it in the system, but turns out that system is
not the only way. A non-western example of this are a lot of the families that
came up in money in China the past few decades. They didn't make it in the
rigor of Chinese school system, but when they said fuck-it, I'll survive by
starting something, they were wildly successful.

Controversially there's evidence pointing to education doesn't help people at
scale[1]. So that whole structure, debatably, has diminishing returns.

So, I agree with Paul Graham 100%, schooling isn't the only way, and I mean
that by - if my children decide they do not want to attend school anymore
because they have a plan that, while unconventional, can do great things;
Support all the way.

------
aardvarks
There is a parallel with one of pg's other essays, 'Why nerds are
unpopular'[1]. He says (I paraphrase): 'nerds care more about being smart than
being popular'. But that's another way of saying nerds think the tests you
have to pass to be popular are (on balance) a waste of time, ie bad tests.

[1] [http://paulgraham.com/nerds.html](http://paulgraham.com/nerds.html)

------
buboard
I can't find whose quote it is that "the most difficult thing is to teach
people to think simply".

Unfortunately education cannot test that, because tests are by definition
testing only a fragment of a system. Knowing and understanding one thing end-
to-end is meaningful knowledge, and qualitatively better than knowing
fragments of 100 things. Perhaps you can assess the former by grading a
diploma thesis on a very specific subject. Tests may have an important role in
motivating students to delve more deeply in subjects but they don't go beyond
that.

There's similarly more value to knowing some technology "end-to-end" rather
than specializing in only one level and never understanding what's below.

It's also interesting to read this:

> There are now ways to get rich by doing good work, and that's part of the
> reason people are so much more excited about getting rich than they used to
> be.

Which may reflect how things were 15 years ago. There were 2 threads in "Ask
HN" last week which basically concluded that these days you're much better off
(money wise) working for an uncreative role in FANG.

~~~
avindroth
[https://www.wikiwand.com/en/Gordian_Knot](https://www.wikiwand.com/en/Gordian_Knot)

~~~
buboard
the gordian knot is more of a brutish non-solution to a hard problem.
"thinking simply" is more about developing intuitive understanding to the
point where you can also explain it simply

------
lettergram
Interesting enough, I have the same philosophy - I never cared about grades,
not even a degree really. I took what interested me and what I felt would help
me. For me I always took 16-20 credit hours Per semester, including summers
and averaged a GPA if 2.8-3.4 per semester or something. I also took enough
classes in 5 years to get two bachelors.

Unfortunately, during job interviews many companies ask for GPA on graduation.
I always had to explain “I learned a lot! I just pushed myself to the limit
and when you do that you get B’s”

It’s always an interesting discussion and occasionally will lose my potential
interviews. The truth is that school is really about certifying what you know,
not learning. Personally, I think that should change, because building the
skill of curiosity and The ability to research and understand a topic is far
more valuable. But to learn to be curious you also have to be able to explore
- can’t do that craming for test.

------
eBombzor
Wonder what this guy thinks of Asian schools, where you literally have to
dedicate yourself to studying 9 hours a day, 6 days a week, through middle and
high school (where you have to dedicate 11+ hours regularly) for one test that
literally determines your entire career path, unless you want to move to a
different country that is.

------
mentos
"When a measure becomes a target, it ceases to be a good measure" -Charles
Goodhart

I've read most of PG's letters but I think this is now my favorite because
while I've noticed each of these issues separately I've never connected them
together through this common flaw in our education system. I find it really
profound.

------
krosaen
I actually have found that, at least for math and science classes, focusing on
learning was a less stressful way to do well on tests, even if cramming was
potentially a workable way to get the grade too. The remaining problem is that
often times the course load dictated that some classes had to be undertaken in
hack mode.

------
jmull
I think the worst thing school teaches you -- especially STEM -- is that 99%+
questions have a certain right answer.

IRL most interesting and important questions are not anywhere close to that.

------
tcj
For anyone who enjoyed this, a lot of these ideas were really well discussed a
few years back in Neil Postman's "Teaching as a Subversive Activity" (1969)
and John Dewey's "Democracy & Education" (1916). As you can see these books
are over 50 years old. These ideas aren't new, which goes to say how long
we've known this and what little have we done.

The real question Graham poses here is: what is the point of schooling? At the
moment, it teaches and conditions people to "hack tests" as Graham puts it. So
people continue to do that throughout their lives.

It also conditions people to think there is a "correct" answer to problems.
Which in the open-ended nature of the real questions we have to grapple with
in life is a complete lie. A teacher will rarely say: "It depends."

What passes for "education" today is really just "training" and very far
removed from real learning. People only learn about things that interest them.
And you learn by asking questions, which most of the time will lead to further
questions to which you will only ever find tentative, temporary answers. Not
that different from how scientists treat their "solutions" to big questions as
tentative.

What can we do to change that? Change schools. It's the one place where we
have kids kept for 15 years by mandate! It's the biggest lever we have to
change perceptions and attitudes. Remove tests and grading. Learning will free
up when that happens. It's remarkable what happens when at the beginning of
the year the teacher gives everyone an A. Let go of the idea that permeates
American society that everything needs to be measured. Some things do, but
things like intelligence can't and shouldn't.

I know the immediate response is that this is unpractical. And to that I ask
why? Because we need a way to control? Control what? What district is
"performing" better? When was a score on a math test a true measure of human
potentiality?

We need to go back to core questions as to what is the point of schools and
what is worth teaching and learning in today's world. The world has changed
immensely, but our "curriculums" are the same. The only difference is we use
tablets and Microsoft Word to hand in assignments.

I agree with Graham's conclusion. When we liberate ourselves from these
useless idealogical metaphors we will free people and society up to stop
hacking tests and start living.

------
throwawwy
Several schools of economic thought claims "Money" also operates as a test for
"value for society". It's also as porous as a colander.

> I suspect many people implicitly assume that doing working in a field with
> bad tests is the price of making lots of money.

> Now you can make lots of money by making cool things.

An analogous set of incentive structures, which compels people in elite
universities to play the tests applies to intra & inter-company interactions
as well: companies who play for the tests will enjoy structural advantages
over companies which do not; and this lever allows winner-takes-most
mechanisms via buyouts, consolidations, private equity.

While founders, individually, may make out of this wealthy, companies, and
society as a whole suffers, and playing for the tests, as a behavior,
saturates.

------
gerbilly
>For me, as for most students, the measurement of what I was learning
completely dominated actual learning in college.

Well I never had this problem in school.

I remember a high school calculus exam where I got the answer to the
derivatives of the fundamental functions wrong (simple memorization - boring!)
but I got the delta epsilon part of the test perfect.

I got a C- on the exam, but the teacher wrote a comment that always stuck with
me: "You understood the most complicated part , as usual!"

Same thing in University, I _never_ did homework in groups. I wasn't looking
for the answer to the questions on my problem sets, I wanted a unique answer,
or a more elegant answer.

I would outdo the professors sometimes. One time I saved a ton of busywork on
a problem by setting up a homomorphism with another vector space, proof done
in 5 steps!

------
mettamage
You need a metric to measure performance. It's hard to get such a metric
right. You need this metric because the government institutionalized schooling
and need to have aggregate data on performance.

However, for the student, it'd be better if they'd focus on the intersection
of what they're interested in and what is societally relevant and not to be
graded at all.

I know a friend of mine who went to a school for 1 year where he didn't get
graded, all he learned was soft business skills. I can clearly see that he
grew a lot from it. But I can also see that it is tough for anyone else to see
that he learned anything at all. Partially this is because he isn't graded and
the behaviors he picked up are now natural to him and it just looks like it
was always there.

------
donmatito
The most interesting class I took in my entire studies (Soft Matter physics)
was also the one with the most interesting exams.

Our year's 2-hour exam was the following. The professor gave us a small cup of
water, a piece of paper, and a piece of toilet paper. There was only 2
questions \- place one end of the paper in the water. Describe what happens.
Explain \- place one end of the toilet paper in the water. Describe what
happens. Explain

The year before us was with a small piece of glass and a small piece of
potatoe \- breathe on the glass. Describe. Explain \- rub the potatoe on the
glass. Breathe on it. Describe. Explain

You REALLY needed to understand the class material to succeed in these
seemingly simple tests

------
meterplech
As a new parent I think about this a lot in terms of how to educate my son.

I find it especially relevant as my first job was starting my own SAT tutoring
company. Anecdotally I found it interesting that immigrant parents & most of
the wealthiest US-born parents shared a recognition that this was a game. They
were direct and explicit to their kids that this was a system that they must
participate in, so they might as well win at it.

I think that is the key: make sure your kids can recognize when something is a
bs system they need to hack (like selling enterprise software) and when
something is actually about craft and the best will win (like building great
products that end-users want to use).

------
wglb
It took me several years to learn how to get good grades. I was not that great
a student in the beginning, and was motivated by things that were interesting
to me. The two eventually converged, and I ended up with good grades by the
end.

One test I had was a counterexample to the hackable tests that Paul talked
about. And it was the only one of that kind that was of that nature. It was in
the EE Static Fields course. The final exam was one question, and you could
hear the groans as the test was passed out:

 _Derive Maxwell 's Equations_

This test was a great indicator of how well you got the basic facts of the
course.

------
CivBase
One of my favorite college courses was a computer science course that taught
C++. It only had 4 graded components, each worth 25% of your final grade.
There were two tests (programming tests with paper and pencil only) and two
projects.

It wasn't hard, but it was the fifth most failed course in the university
because the tests were extremely difficult to "hack" and the projects were big
enough that a student could not rush them at the last minute. There was no
homework or attendance to pad grades.

It was one of the few times in my college career that I felt like a professor
actually cared whether or not students actually learned about the subject.

------
sirspacey
This is the difference between project-based learning and standardized
testing.

When you learn through projects your ability to apply concepts is tested by
the results you generate, connecting to our intrinsic motivations.

Standardized testing is, functionally a disease. Check out “DIY University” to
see the effect it has on society long-term.

If you think projects can’t be the path to teaching highly abstract
disciplines, such as physics, check out “Surely You’re Joking Mr. Feynman.”

“Most Likely to Succeed” is probably the best book written on this topic.

I spent 10 years as an edtech entrepreneur. It is hard to overstate the degree
to which we have built a Testing system at the expense of an educational one.

------
burntoutfire
The solution seems to be - avoid people with degrees from top universities?
Or, in more extreme form, all college graduates.

Less extreme variant of it would be - OK, you went to college - but show me
what you did on your own (personal projects, small attempts at startups etc.)
during that time. That's what the game dev industry is doing. If the answer to
this question is "nothing", then you're probably facing a person who's happy
to take directions and land himself a cushy job in some large organization,
where he will use his intelligence to navigate towards maximum pay and minimum
workload.

------
JesseAldridge
> Why did founders tie themselves in knots doing the wrong things when the
> answer was right in front of them? Because that was what they'd been trained
> to do.

Maybe I'm overly cynical, but I see another possibility: They want to create
the appearance of running a hot startup without actually having to do the
work. Why make something people want when you can just dazzle some investors
and get them to dump a million dollars in your lap, no strings attached?

I wonder if that also might explain why such a tiny fraction of startups
generate essentially all the wealth -- most of them aren't really trying.

------
xrd
A corollary: software engineering interviews are tests. When you are young you
can memorize more things. When you are older you've experienced many different
ways of building software. But, interviews, most of them, generally just ask
you to regurgitate concise facts, just like PG points out tests do. And, our
brains are tuned to pick out mistakes in those facts and make decisions based
on that. I'm not saying hiring for engineers is biased against older
engineers, but it makes me think after reading this essay.

------
gcc_programmer
TL; DR; Einstein said sth along the lines of "Don't let your education get in
the way of your knowledge." Finish uni with high grades then spend the rest of
your life learning.

~~~
username90
> Finish uni with high grades then spend the rest of your life learning.

I'd rather not waste 30% of my life on that nonsense. Better to use uni for
what it is meant for: Learn whatever seems interesting and don't mind the
grades.

------
rsp1984
I completely agree with the the theory, however what 90% or more of us do when
they finish university is to go work for an established (i.e. large) company,
where winning means hacking bad tests (promotion cycles).

So in some perverse sense school _does_ prepare most of us well for working
life, as most of the time doing well in BigCo isn't to actually perform well
or to actually make a difference, it's to leave the impression of it at the
right time with the people that matter for your next promotion.

~~~
throwawaytemp1
This isn't fair to tests. They are actually far more fair and objective than
BigCo promo processes. You can't fake doing a variety of hard math questions
the way you can move up the ladder due things like nepotism.

------
grappler
Coming back to this discussion a few days later as I've just started an online
course on Kubernetes from the Linux Foundation and I came across this section
in the introduction materials that seemed very relevant here. Quoting from it:

    
    
      Training/Certification Firewall
    
      The Linux Foundation has two separate training divisions:
      Course Delivery and Certification. These two divisions are
      separated by a firewall. 
    
      The curriculum development and maintenance division of the
      Linux Foundation Training department has no direct role in
      developing, administering, or grading certification exams. 
    
      Enforcing this self-imposed firewall ensures that
      independent organizations and companies can develop third
      party training material, geared towards helping test
      takers pass their certification exams. 
    
      Furthermore, it ensures that there are no secret "tips"
      (or secrets in general) that one needs to be familiar with
      in order to succeed. 
    
      It also permits the Linux Foundation to develop a very
      robust set of courses that do far more than teach the
      test, but rather equip attendees with a broad knowledge
      of the many areas they may be required to master to have a
      successful career in open source system administration.
    
    

Running with this idea a bit, suppose schools were organized this way, with a
similar "teaching/testing firewall" across everything they do. This doesn't
get us all the way to solving problems around "teaching to the test" but it
does put in place separate entities with the explicit mandate of making tests
less 'hackable'.

------
vezycash
>The real problem is that most tests don't come close to measuring what
they're supposed to

This is what happens when the teacher's hidden goal is to make marking/grading
easier.

~~~
TeMPOraL
Couple that with students' hidden goal being "not ruining my career prospects
and/or angering my parents by getting bad grades", and you arrive at modern
school.

~~~
vezycash
>angering my parents by getting bad grades

At secondary school level, it's no longer an issue. Parents just change
schools. A few months ago, an uber unserious secondary girl altered her end of
term results. The father brought the slip to school to complain about the
unprofessional look of the slip due to multiple alterations.

Found out what his daughter did and change schools.

------
mncolinlee
Perhaps one of the most noticeably hackable tests I've participated in is
democratic elections. I ran for office a few times and learned a lot about how
the system actually works. I believe this is why politics has such a negative
connotation. Ultimately, voters want politicians to do what's right and
politicians learn they have to gain the genuine love of one group and appease
a large enough cross-section of groups without offending an important one.

If there's one thing I learned in politics, the biggest hack is in having a
great narrative. Ultimately, brands, companies, and politicians are characters
in our consciousness that only achieve meaning through the stories they tell.

Essentially, the story must be believable and ring true to the audience being
targeted according to their past experience. For example, evangelicals love a
good redemption and hard work story. Each group has their own highest morals.
I'd recommend "Don't Think of An Elephant" by George Lakoff, which explains
how politicians hack the human mind by crafting an entire language that plays
into their hopes and fears.

I'd love to have every test in life be genuine and unhackable. But when tests
are clearly so hackable, it doesn't help to not call out how the hack works.
Call it responsible cultural exploit disclosure.

------
yotamoron
Ivan Illichlready said it 50 years ago. Read 'Deschooling Society', its all
there. We are schooled to confuse process with essence, and the bad results
are apparent.

------
anonytrary
> The real problem is grades

Bingo! The real problem is the using incorrect metrics. If you judge a fish by
its ability to climb a tree, it will spend its entire life thinking it is
stupid!

------
hogFeast
What is funny about this is that he doesn't realise most medieval historians
have actually recognised this and solved it...answer: no exams. Just get
students to write essays. Most history classes do exams but, where it is
taught well (i.e. not the US), they give you the questions that will be on the
exams...these aren't exams like in maths. This tests for understanding, and
actual ability to conduct research yourself (you still get bias from
professors, I had a Chinese politics professor who had some fairly inflexible
views about the Chinese economy...but it does average out).

I still have no idea why this isn't used more generally in life. Want to see
if someone can do this job...hm, rather than actually seeing them do the
job...have you ever thought about how many windows there are in New York? All
you are testing is whether someone has either been asked that kind of question
before or, more likely, whether they went online and looked at what you would
ask them.

It is truly stunning how stupid people are about this. And, tbh, I think it
largely reflects what most students about the world at university where
certain professors seem to revel in the utter banality of pointless tests.

------
DFXLuna
This article reminds me of my interview for workstations at HPi. I came away
from it knowing that I had to work there. The interview was 5 hours: 4 one
hour sessions and lunch with the team. I didn't write a single line of code
the whole day. Every question came with the disclaimer that the answer was
less important than the thought process.

Disclaimer: I work there now and my opinions don't necessarily represent my
employer's etc.

------
noobermin
As someone who's not entirely convinced VC funding start-ups isn't the best
model for how society can allocate resources, I'm always impressed with how pg
thinks, but I find he misses the boundaries of the world he lives in. He's
right about artificial tests, but he fails when he says the current situation
really allows "people doing good things" to get ahead. I mean, just look
WeWork. It's hard to claim WeWork won at an artificial test, unless you count
convincing others with your confidence as your test. I'm not sure that's what
he thinks it is though.

You can claim non-artificial tests are "better," but humans (VC's) still hold
the purse-strings so to speak. If you can convince _them_ you're winning the
non-artificial tests say of the market, then you can get money and do coke and
party for a while, whilst sure your company will crash and burn but you escape
unscathed with millions of sweet free money. My point is the problem is deeper
than just "artificial" vs. "non-artificial", although it is an important
aspect.

------
halfway
It seems that hackability of tests, along with their purpose can be
generalized. Tests, like big company politics, are signalling mechanisms. The
totality of all tests, culminating in a degree from an institution of a
certain prestige, is a signal of potential to contribute.

Companies and other big institutions need very strong assurance that
supplicants will indeed "work out" by comporting themselves to the culture and
making valuable contributions to their goals. And the reason such absurdly
strong signals of assurance are needed is the ridiculous level of friction
inherent in admission to and ejection from these institutions and their
hierarchy of stations.

So, to get into Big Company X, you need a prestigious degree, and to advance
once you're there, you must identify and play their internal signalling games.
Such is the attachment to signalling, that you'd often do much better to lead
the holiday decoration committee than to create actual value by solving some
important problem.

The relative lack of friction in open source is why, given the ability and
inclination, even a dog can contribute at a high level.

------
sinameraji
Love it. I wanna put this in the context of innovation in education:

IMHO, a radical innovation in education is one that can address at least two
of the following flaws and failures in the current education system, at any
growable scale:

–Duration and cost –Content –Delivery

If I were to satisfy a VC, I could also add a 4th bullet point called
"Supply/Demand", but since I'm not writing for a VC, then I'll write what
actually matters. In the case of education, that is, it's okay to ignore the
existing supply and demand, and focus on the demand that must exist that
doesn't yet.

Wrote a memo on these and elaborated more
[https://www.linkedin.com/pulse/q4-2019-what-i-see-current-
st...](https://www.linkedin.com/pulse/q4-2019-what-i-see-current-state-
innovation-education-sina-meraji/)

P.S: I think I got lucky that my mom was a teacher, so I grew up having a
definition of right and wrong different from what was expected of me, and that
stayed with me throughout my education. I didn't do what was asked, but what I
believe was right for my growth.

~~~
BlueTemplar
Considering that "education" is being used for daycare and conformity first,
it might not be "fixable" when these goals are still requirements...

------
gliese1337

      In theory, tests are merely what their name implies:
      tests of what you've learned in the class. In theory
      you shouldn't have to prepare for a test in a class any
      more than you have to prepare for a blood test. In
      theory you learn from taking the class, from going to
      the lectures and doing the reading and/or assignments,
      and the test that comes afterward merely measures how
      well you learned.
    

That was always how I approached tests in college. While everyone around me
was stressing out, not once did I actively study for finals. Finals week is
for relaxing, sleeping, watching movies, reading books, and having parties in
between tests. Because, y'know, learning the stuff on the test was what _the
whole frickin ' semester was for_, and if you weren't paying attention up to
now, what makes you think you can learn it all over a weekend?

I was constantly amazed at how many of my peers, who were paying good money to
_learn stuff at school_ , were utterly shocked by that philosophy. I really
wish more students understood it.

------
songzme
> I would explain that what makes a startup promising, not just in the eyes of
> investors but in fact, is growth.

Growth is like cancer. Eventually a company will get big enough to bully
others for the sake of growth.

Last night I had dinner with a friend who is building a (yet another) language
learning app. Simple fact about language learning, or learning in general: You
must practice almost every single day. Even if it is only 5-10 minutes.

PG himself said, if you don't die (aka give up), you will eventually succeed:
[http://www.paulgraham.com/die.html](http://www.paulgraham.com/die.html)

To me, the best, most sensical way for learning applications to measure
success is to delete user accounts with > 3 days of inactivity. This way you
can measure the absolute effectiveness of your program. Students who stick
through with your application should 100% become fluent. Target the users who
actually want to learn, and measure your success by how many users you made
successful.

"NO no no", he says. You'll have no users if you do that.

~~~
mattmaroon
There's a lot of evidence that taking breaks helps with language learning. And
there are several other ways to learn languages. I'll sometimes stop using duo
lingo but listen to Spanish language podcasts instead for a couple weeks

------
Terretta
> _No, I would explain, that is not how to get lots of users. The way you get
> lots of users is to make the product really great. Then people will not only
> use it but recommend it to their friends, so your growth will be exponential
> once you get it started._

> _At this point I 've told the founders something you'd think would be
> completely obvious: that they should make a good company by making a good
> product. And yet their reaction would be something like the reaction many
> physicists must have had when they first heard about the theory of
> relativity: a mixture of astonishment at its apparent genius, combined with
> a suspicion that anything so weird couldn't possibly be right._

The essay context is students and startup founders, but it turns out most
multi-billion dollar enterprises have forgotten this as well.

This notion does not come embedded in the heads of most “senior management”
from either world.

~~~
BlueTemplar
The point is that big companies have other, easier roads to success.

------
whack
> _How does one get lots of users? They had all kinds of ideas about that.
> They needed to do a big launch that would get them "exposure." They needed
> influential people to talk about them. They even knew they needed to launch
> on a tuesday, because that's when one gets the most attention._

> _No, I would explain, that is not how to get lots of users. The way you get
> lots of users is to make the product really great. Then people will not only
> use it but recommend it to their friends, so your growth will be exponential
> once you get it started._

Ironically enough, if there is one thing I've heard from HN, it is to not
believe in the _" if you build it, they will come"_ myth. That no matter how
good a product you build as an engineer, it is all pointless if you aren't
willing to hit the ground and start aggressively marketing and selling it.
That a mediocre product with great marketing/sales, will win over a better
product with poor marketing/sales.

I've heard this directly from YC partners themselves, when they keep telling
early-stage founders to market themselves via _" do things that don't scale"_.
At Reddit, one of YC's early successes, the founders were literally spending
their time spamming their own site with fake posts and comments using sock-
puppets, in order to create the illusion of activity. As someone who loves
building products, I hate the idea of spending time and energy on tactics like
that. I'd love to spend all that time and energy on making my product better.
But I've learnt grudgingly from YC that product development is pointless
unless I'm using guerilla tactics to hawk my product in front of users.

Maybe I'm in the minority, but I've actually had the opposite experience as
what PG describes in his essay. Back in school and University, I did study
very hard to ace tests. But the way I studied was to genuinely learn and
understand the material as well as possible. Not to "hack" it in some way.
Whereas once I graduated, and especially once I tried out entrepreneurship, I
realized that just building a great product was insufficient. I now had to _"
hustle"_ and use _" street smarts"_ and _" growth hacking"_ in order to get
people to notice what I'm doing. The lesson I had to unlearn from school was
that the quality of your work will speak for itself, and will win the day. I
never needed marketing and sales in school, but it seems indispensable in the
real world.

~~~
BlueTemplar
> That no matter how good a product you build as an engineer, it is all
> pointless if you aren't willing to hit the ground and start aggressively
> marketing and selling it.

You have pretty much summarized another one of his talks :
[http://paulgraham.com/ds.html](http://paulgraham.com/ds.html)

(Or even two :
[http://paulgraham.com/schlep.html](http://paulgraham.com/schlep.html) )

------
dunkelheit
In my opinion the most insidious feature of the education system is not tests.
After all a proper hard STEM program is not that easily hackable - the easiest
way to complete it may be to actually study!

What is worse (and although it is related to grades it is not the same), you
only do what you are told to do. Your goals are set for you - read this
chapter, solve this problem, answer this question. Thus your inherent
creativity is slowly but surely gets sapped away until you become a
"professional" \- someone who solves problems of others without questioning
them much.

Not sure what to do with this though - every way to overcome this problem
seems inherently not scalable. Also not sure if this is really a bad thing - a
society where everybody suddenly becomes more creative can suddenly collapse
because of all the excess creativity!

------
hirundo
> When I was a kid, you could either become an engineer and make cool things,
> or make lots of money by becoming an "executive." Now you can make lots of
> money by making cool things.

Software is outrageously accessible. The threshold cost for make-lots-of-
money-by-making-cool-things has fallen to a used laptop and a wifi password.
The prerequisite talent is mostly the ability to focus relentlessly while
illegitimi non carborundum.

But since those are rare talents the laptop can only elevate rare people. It's
a path to relative wealth for only a pareto distribution of people, like
pretty much anything else. But it's a path to absolute wealth for most of the
rest of us. Cool things that make money tend to make our lives better.

Make-lots-of-money-by-making-cool-things is a recipe for both more absolute
and unequal wealth.

------
Mauricio_
I'm too late to comment, but my 2 cents: As long as there's something
important at stake people will try to hack the test. There's no way around
that.

The best that can be done is to try to keep the correlation between the "test"
and what it's trying to measure add high as possible and to teach everyone how
to hack it, that way everyone is on a level play field. If everyone hacks it
then the difference in scores will be mostly about what is being measured and
not about who hacked it and who didn't.

That's why big companies actually advice interview candidates to practice on
leetcode and give them example interview questions.

But I agree with the main point that being good only at "hacking tests" in
life instead of being good at your profession is very bad.

------
yosefzeev
I actually did what this article suggests in college. It didn't go well grade
wise. However, I had money and time to study and learn about things I wanted
to learn. I found that my path often was interwoven with the path of the
class, but that the perspective was different and sometimes professors are
only grading on perspective--specifically their own. The biggest dividing line
is, I suspect, that college does not, at this point teach one how to think.
How to think and good grades are not necessarily the same thing at all. I was
busy learning how to think, and deciding what could be relegated to "It's in a
book, I can find it if I need it" vs "I should memorize this because it will
be useful in life for me later".

------
adyer07
I’m returning to school for a BFA in art, and I’m constantly surprised by
exactly how un-hackable the assignments are, compared to my experience
studying engineering. It’s finals season, and many people (myself included)
take it as an excuse to push and make work that is harder and more complex
than they could during the rest of the semester. Most classes are pass/fail,
so the only reward is having better work (and a better portfolio). You could
duct tape a banana to the wall, but that’s on you to live with.

Maybe there aren’t a lot of fine artists starting YC companies, but how about
designers who went to art school? Do they treat these problems differently? Do
they have less to unlearn, or different bad habits?

------
Ericson2314
The managerial class will keep on embarrassing themselves sweating bullets
over facile things like tests, college admissions, and school districts until
they stop feeling precarious and downwardly mobile.

There's no point looking at one of their idiosyncrasies in isolation.

But then Paul Graham plays himself because all the startups are looking to get
or softbanked. Absorbed into the monopolies or put in permanent luxery lift
support. What a joke. Playing the VCs is more important than playing the
increasingly poorer consumer hoards of the decreasing yield suburban consumer
farms. We're years deep into the startup fad, and blinded test takers
appealing to their blinded test taker money holders is all that's left.

------
arh68
I think "hackability" is a spectrum [0]. The idea is that VC funding is less
hackable than college admissions, and that market growth/acceptance is very
far on the un-hackable end, along with fairly-referee'd sporting events.

He doesn't admit any ways to actually hack market growth (as if there were
none), which is weird. Seems like Use Dark Patterns, Pad Your Numbers, Fake It
'Til You Make It, Regulatory Capture, &c would all make the list.

I agree with the general idea, though: I think the least hackable tests are
either Wimbledon/tennis or GSL/StarCraft tournaments; you simply cannot
schmooze your way to even the lower brackets.

[0] like most things people put in 2 categories

------
kresten
Nothing said here about how entrepreneurs think who have not been to
university.

It would have been an interesting contrast.

I wonder is that because almost all people who do YC are university educated.

Perhaps Paul graham doesn’t have much experience of entrepreneurs who are not
university people.

------
jpm_sd
> At this point I've told the founders something you'd think would be
> completely obvious: that they should make a good company by making a good
> product.

Sure, ok. How do you make a good product? That definitely is not "completely
obvious"

~~~
BlueTemplar
At least one user is going to love it.

------
vegiraghav
So i have been looking at architecture and security lately. Is it a thing that
you automatically start noticing what is not right in a sight? For instances
first thing i saw in that link was that the site runs on http not https.

------
artemisyna
I'm not sure if the correct takeaway here is "unlearn how to hack systems" but
something more akin to "prioritize the _right_ hacks you need to do".

Making a good product (with whatever shortcuts/hacks come with it) is
necessary for doing well. Getting exposure and launching on a Tuesday are
neither necessary nor sufficient, but they will give a marginal edge.

Both the former and the latter are 'hacks', the only difference is degree of
relevance. pg throws the baby out with the bathwater a bit by saying don't do
both types of hacks.

------
ve55
This post reminded me of Eliezer Yudkowsky's Guessing the Teacher's Password
[https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-t...](https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-
the-teacher-s-password), which focuses more on the need to unlearn answering
questions in the style that school expects you to answer them in, since that
style has been optimized to make testing easy, rather than demonstrate in-
depth understanding

------
randomsearch
A related observation: what is the point of a “B” grade? It means “this person
didn’t learn everything.” If you’re going to say “oh ok they got a B so we
need to teach them some more”, that’s really useful.

But if you just given someone a “B” and send them up a year at
school/university, then presumably they find it harder and harder to do
subsequent courses. And over time they learn they’re a B student. Which is
demoralising, and possibly also insulting as it ascribes failure of the
learning process to the student and not the teacher.

------
rajacombinator
This is actually wrong and surprising to hear from PG. For the most part, life
is hackable. Sure, being the “real deal,” whatever that is, is great. But most
of “success” in life (again, whatever that is) comes from hacking. (And luck.)
Look at WeWork, for instance, one of the greatest and most successful hacks of
our time. YC itself became a hack as soon as it acquired useful signal. If
you’re optimizing for success, you should teach students to hack _more_ not
less.

------
codingdave
I went to a small liberal arts school, and tests were rare. Papers were
common, presentations as well. Projects, and experiments in the science
courses. But not tests.

Most of the time, our grades were based on the work we produced, and our
participation in discussions. Even in Art History, when the job was to study
and memorize slides and identify them, the test was not to just say what they
were, but to write a paragraph about how they fit into the cultural context
from which they came.

------
slipmagic
Great and inspirational article that’s pretty practical.

That said, the website is a pain to use on mobile safari and difficult to
properly capture when sharing it to Notion or Bear. Notion doesn’t capture it
at all, and Bear can’t capture the hyphens or line/paragraph breaks. It’s
probably because of the read more button. It also has some jaggy scrolling
issue where if you switch between apps it scrolls up and sometimes collapses
the article or jumps to the bottom of the page.

------
1k
Studying, getting hired, career promotions, raising funds... It’s all about
beating the system. You’re evaluated based on certain criteria and you
optimize for those.

------
darkkindness
The following paragraph is a good summary, I think:

> Anyone who cares about getting good grades has to play this game, or they'll
> be surpassed by those who do. And at elite universities, that means nearly
> everyone, since someone who didn't care about getting good grades probably
> wouldn't be there in the first place. The result is that students compete to
> maximize the difference between learning and getting good grades.

It's a race to the bottom.

------
mekazu
I wonder why pg didn’t launch this article on a Tuesday.

------
renlo
School grades are more a measure of conformity and ability to adapt a wage-
work environment. Performing well in both wage-work and school requires an
individual to jump through arbitrary hoops (performance reviews, office
politics requiring certain procedures, etc). Teachers are just as capricious
as bosses and upper management with their directives.

Perhaps if we move away from wage work then school will change too.

------
k__
I never cared about grades, but I was never very good or very bad. Just
slightly above average

In academia, having good grades is sometimes helpful to climb the ladder. Not
all people can study medicine. Not all people can get a master degree.

Often these gigs are given to the best students only, but sometimes you can
circumvent the grading issue by waiting longer, going to the military, be
friends with a prof. or working for free.

~~~
robocat
> Not all people can study medicine

I think you are falling for the fallacy that the essay is trying to expose?

Surely those that pass medicine are those that are good at medicine exams.
Plenty of people would make great doctors that don’t due to the exam system,
not due to a lack of capability.

------
analog31
I've got one kid in college, the other is finishing high school. Here's what
I'm telling them, although not in these words: If you get something of real
lasting value out of your education, then you're the one who is hacking the
system.

Figuring out the tricks to pass tests is not hacking, it's going with the
flow. Or at best, it's hacking in the "script kiddie" fashion.

------
correlator
It sounds a lot like raising venture funding. Make sure there's a slide that
says "$10 billion" because that's the new billion. Talk about your "AI" which
was formerly "blockchain" or "big data" and make sure you fit the mold of
whatever the funding is looking for. Leave creativity at the door please.

------
scscsc
As a teacher, the way to fix this is to give "aligned tests". That is, where
the test tests for _exactly_ what you want the student to learn. See
[https://en.m.wikipedia.org/wiki/Constructive_alignment](https://en.m.wikipedia.org/wiki/Constructive_alignment).

------
avindroth
I am surprised this book is not mentioned. Talks about how optimization
doesn't lead to optimal goals in complex environments (e.g. life).

[https://www.amazon.com/Why-Greatness-Cannot-Planned-
Objectiv...](https://www.amazon.com/Why-Greatness-Cannot-Planned-
Objective/dp/3319155237)

------
zopine
TLDR: The way to succeed in school is by gaming the system, but the way to
succeed in business is by making a great product.

But PG is wrong. Startups can, and do, game the system. By showing
unsustainable, artificial growth with no real value behind it, they can fool
investors long enough to make a lucrative exit by IPO or acquisition.

~~~
buboard
an IPO or acquisition without proving their market value should count as
startup _failure_.

------
ph2082
And Lesson to Learn is when real life problems come, people will ask can you
solve the problem not what grade you got.

------
mtreis86
I wonder if there will be any changes made to the YC application process now
that Paul is thinking about these things.

------
qwerty456127
Good point, but there have been countless amazing products that failed to
raise money and/or to attract enough users. Many have even succeeded just to
be bought and terminated. Doing the actual job is not enough, in real the real
world you are doomed to hack users, investors, laws and many other things
anyway.

~~~
BlueTemplar
Did he ever pretend otherwise?

------
paulsutter
Getting tested accelerates learning. That’s why we learn more from actual
work, every action is tested by reality. Or why we learn language by using it,
every interaction is a test.

> I was genuinely interested in most of the classes I took, and I worked hard.
> And yet I worked by far the hardest when I was studying for a test.

------
ZguideZ
Tsunebaro Makiguchi, an early 20th century Japanese educator, was striving
towards deconstruction of rote education but his work was disrupted by
authorities who worried that his educational revolution would disrupt orderly
Japanese society. Careful, PG, you might be rocking the boat too much with
this. ;)

------
NPMaxwell
"how to raise money ... read as the test. It came at the end of YC. It had
numbers attached to it, and higher numbers seemed to be better. It must be the
test."

This is a great sharing of internal mental processes. The same process evolves
around investment reports. "Oh look! I got a good grade!"

------
pvg
_No, no, no, experienced students are saying to themselves._

    
    
        Oh, no no no, it was too cold always   
        (Still the dead one lay moaning)   
        I was much too far out all my life   
        And not waving but drowning.
    

Somehow captures the mood of reading pg's later oeuvre.

------
lixtra
Paul should talk to Seth about education. He seems to have had the insights
before him[1].

[1] [https://seths.blog/2018/09/education-needs-to-be-
inconvenien...](https://seths.blog/2018/09/education-needs-to-be-
inconvenient/)

------
shrubble
This makes me wonder if pg has ever read any of the writings of John Taylor
Gatto?

Various quotes:
[https://www.goodreads.com/author/quotes/41319.John_Taylor_Ga...](https://www.goodreads.com/author/quotes/41319.John_Taylor_Gatto)

~~~
avindroth
I feel like he definitely has, he seems like he reads a lot and Gatto is not
unpopular for people with thoughts on school

------
andreyk
This post glosses over something important, which is that many college classes
don't assign most of your grade based on tests. In Stanford's CS 229 Machine
Learning class (which I am TAing right now), 20% of the grade is the midterm,
40% is HW, and 40% is a big project that students do throughout the quarter
(and for which they get to decide the problem, approach, and most anything
with just feedback and guidance from TAs). In my experience, "In practice, the
phrase "studying for a test" was almost redundant, because that was when one
really studied." is absolutely false, because studying also happened when
doing assignments and projects. Of course one could cheat if they just cared
about the grade, but I did not because I don't think most smart people really
only care about grades, it feels bad to cheat.

So TLDR is, I dont think the situation is so bad, because grades are not only
a function of tests but also assignments and projects. I think the deeper
issue is that people learn to be told how to learn (which assignments or
projects to do), and not taught how to learn by themselves. There are
exceptions (at Stanford a ton of CS classes have projects where the students
map out the problem and solution themselves), but I think this is pretty
important too.

------
Jgrubb
There's a passage in Zen and the Art of Motorcycle Maintenance about exactly
this.

\---

Phaedrus' argument for the abolition of the degree and grading system produced
a nonplussed or negative reaction in all but a few students at first, since it
seemed, on first judgment, to destroy the whole University system. One student
laid it wide open when she said with complete candor, "Of course you can't
eliminate the degree and grading system. After all, that's what we're here
for."

She spoke the complete truth. The idea that the majority of students attend a
university for an education independent of the degree and grades is a little
hypocrisy everyone is happier not to expose. Occasionally some students do
arrive for an education but rote and the mechanical nature of the institution
soon converts them to a less idealistic attitude.

The demonstrator was an argument that elimination of grades and degrees would
destroy this hypocrisy. Rather than deal with generalities it dealt with the
specific career of an imaginary student who more or less typified what was
found in the classroom, a student completely conditioned to work for a grade
rather than the knowledge the grade was supposed to represent.

Such a student, the demonstrator hypothesized, would go to his first class,
get his assignment and probably do it out of habit. He might go to his second
and third as well. But eventually the novelty of the course would wear off
and, because his academic life was not his only life, the pressure of other
obligations or desires would create circumstances in where he just would not
be able to get an assignment in.

Since there was no degree or grading system he would incur no penalty for
this. Subsequent lectures which presumed he'd completed the assignment might
be a little more difficult to understand, however, and this difficulty, in
turn, might weaken his interest to a point where the next assignment, which he
would find quite hard, would also be dropped. Again no penalty.

In time his weaker and weaker understanding of what the lectures were about
would make it more and more difficult for him to pay attention in class.
Eventually he would see that he wasn't learning much; and facing the continual
pressure of outside obligations, he would stop studying, feel guilty about
this and stop attending class. Again, no penalty would be attached.

But what had happened? The student, with no hard feelings on anybody's part,
would have flunked himself out. Good! This is what should have happened. He
wasn't there for a real education in the first place and he had no real
business there at all. A large amount of money and effort had been saved and
there would be no stigma of failure and ruin to haunt him the rest of his
life. No bridges had been burned.

The student's biggest problem was a slave mentality which had been built into
him by years of carrot-and-whip grading, a mule mentality which said, "If you
won't whip me, I won't work." He didn't get whipped. He didn't work. And the
cart of civilization, which he supposedly was being trained to pull, was just
going to have to creak along a little slower without him.

This is a tragedy, however, only if you presume that the cart of civilization,
"the system," is pulled by mules. This is a common, vocational, "location"
point of view, but it's not the Church attitude.

The Church attitude is that civilization, or "the system" or "society" or
whatever you want to call it, is best served not by mules but by free men. The
purpose of abolishing grades and degrees is not to punish mules or get rid of
them but to provide an environment in which that mule can turn into a free
man.

The hypothetical student, still a mule, would drift around for a while. He
would get another kind of education quite as valuable as the one he'd
abandoned, in what used to be called the "school of hard knocks." Instead of
wasting money and time as a high-status mule, he would now have to get a job
as a low-status mule, maybe as a mechanic. Actually his real status would go
up. He would be making a contribution for a change. Maybe that's what he would
do for the rest of his life. Maybe he'd found his level. But don't count on
it.

In time - six months; five years, perhaps - a change could easily begin to
take place. He would become less and less satisfied with a kind of dumb, day-
to-day shop-work. His creative intelligence, stifled by too much theory and
too many grades in college, would now become reawakened by the boredom of the
shop. Thousands of hours of frustrating mechanical problems would have made
him more interested in machine design. He would like to design machinery
himself. He'd think he could do a better job. He would try modifying a few
engines, meet with success, look for more success, but feel blocked because he
didn't have the theoretical information. He would discover that when before he
felt stupid because of his lack of interest in theoretical information, he'd
now find a brand of theoretical information which he'd have a lot of respect
for, namely, mechanical engineering.

So he would come back to our degreeless and gradeless school, but with a
difference. He'd no longer be a grade-motivated person. He'd be a knowledge
motivated person. He would need no external pushing to learn. His push would
come from inside. He'd be a free man. He wouldn't need a lot of discipline to
shape him up. In fact, if the instructors assigned him were slacking on the
job he would be likely to shape them up by asking rude questions. He'd be
there to learn something, would be paying to learn something and they'd better
come up with it.

Motivation of this sort, once it catches hold, is a ferocious force, and in
the gradeless, degreeless institution where our student would find himself, he
wouldn't stop with rote engineering information. Physics and mathematics would
come within his sphere of interest because he'd see he needed them. Metallurgy
and electrical engineering would come up for attention. And, in the process of
intellectual maturing that these abstract studies gave him, he would be likely
to branch out into other theoretical areas that weren't directly related to
machines but had become part of a larger goal. This larger goal wouldn't be
the imitation of an education in Universities today, glossed over and
concealed by grades and degrees that gave the appearance of something
happening when, in fact, almost nothing is going on. It would be the real
thing.

~~~
keithyjohnson
Is it ironic that I'm going to quote this passage in my grad school admissions
essay?

------
edanm
Is anyone else not seeing the footnotes inline? I'm only seeing the notes
section at the end.

------
flyGuyOnTheSly
Funny to read and enjoy this article this morning...

Open up HN again this afternoon for a quick work break...

And find my eyes IMMEDIATELY glancing at my karma score in the top right
before anything else.

Perhaps PG should practice what he preaches and rip that out of the UI?

I would encourage it... damnit... I just looked again.

~~~
darkkindness
Just inject:

td:nth-child(3) > span.pagetop { color: transparent; }

Hides karma, leaves user and logout intact (as they're links)

------
d--b
How is the YC interview not hackable? It seems to me that it consists of
ticking boxes mostly...

------
hodder
I actually had to drop out of Spanish courses in university as they were going
to crush my GPA if I didn’t drop them. I was learning a ton and enjoying them,
but for my targeted profession my resume would be tossed if my GPA didn’t meet
an arbitrary cutoff.

------
StacyRoberts
>In the mid-twentieth century, when the economy was composed of oligopolies,
the only way to the top was by playing their game. But it's not true now.

It is still true. A FAANG job is the easiest path for a engineer to
millionaire status in about 5 - 10 years.

The interview process is eminently hackable.

Or perhaps you want to do a startup, get funding. You build a this for that
.com. Do some social media, find a few users who love it. Read pg's advice,
pitch deck advice, go to some demo days.

Hack the test. Get funding. The incubators, the vc fiends are themselves,
oligopolic. A handful of firms with common expectations. Mmr. Hockey stick
user growth. It's the team! Here's your A good chap.

Those are the tests here. But there are other ways to get an A+. You don't
have to start a business the way software people start a business. You can do
it like the cabinet maker, or the electrician, or the store owner.

You can buy one increasingly easier today because the baby boomers who started
than decades ago want to retire. Imagine how you can take that business into
the future with the software you can use to automate it.

Business brokers abound. Go talk to one.

Stop thinking go big or go home.

Go talk to a business broker. Go ask about owner absent business options.
Residual income and you didn't even have to be there. Use automation to make
the employees lives better and more prodictive. And your customers' lives.

Package that software you write into a SaaS and sell it to you competitors.
Profit more.

You are your own case study.

------
spion
To use a fun analogy, the teachers write the unit tests and you pass them by
writing a mock or fake implementation of knowledge.

edit: and now I got to a point in the article where that's pretty much already
said. Well, too late to delete this comment now.

------
projektfu
I suspect that some of the things people do to get good grades, like proofread
their essays and edit them down to reduce redundancy, are good things to teach
in school, and are broadly applicable outside of the university.

------
caro_douglos
Recently I have been charged with helping someone setup paperwork for their
startup. I am not a lawyer so I simply have the paperwork, standard
agreements, and general company structure that any lawyer would (I assume)
give away for free. Avodocs has pretty much all that paperwork covered, carta
has the captable, and pilot has the books under wrap. After reading this
article I'm somewhat torn about mentioning things like c corps and filing in
Delaware to my friend since in the end it really is only the product that
matters. Corporate structure steeped in law/history is a hackable test which
everyone must take...resulting in either getting bitten or by running to
attorneys and tax advisors (because they were bitten in the past).

TLDR opinion. The hackable c corp structure Y pushes has payed off well for
Ycombinator yet it is the very thing that can distract from building a good
product.

Then again I'm sure someone might make the claim that c corps are also a way
to easily share equity with your employees.

------
softwaredoug
One solution might be to silo the people who evaluate the students from those
that teach the materials. Treat the test more about evaluating the
effectiveness of instructors and less about the students.

------
thatthatis
I always shot for a “B+” in college. My logic was that in 5 years I’d remember
the content at a B+ level, and any incremental effort was better spent
socializing or building student organizations.

------
ryacko
I disagree with this essay, I have no knowledge in the field that would allow
me to argue against this essay, I think this essay lacks brevity, and the
moral of the story seems weak.

------
stornetn
The point is summarized well by one of my favorite aphorisms (often attributed
incorrectly to Mark Twain): "I have never let my schooling interfere with my
education."

------
souterrain
The free market itself is the ultimate hackable test, no? Our society has
examples where superb work goes unrewarded financially.

Does pg argue that they simply should ignore this test grade?

~~~
erikerikson
He doesn't offer help there but...

Yes and the job interview.

------
wwarner
Makes me hopeful, and why I am so excited about the Recurse Center.
[https://www.recurse.com](https://www.recurse.com)

------
remotecool
I think this is the difference between education in many countries In Asia and
the US: they care only about the grades, not what you actually learn.

------
Xelaz
Is the latest essays the in-text links to notes are missing.

(My favorite from this one: "Learning is the naive algorithm for getting good
grades.")

------
Rerarom
Yeah, but what you learned can be changed in the future (by learning more when
you're in a more comfortable position), grades can't.

------
parr0t
As someone who got very high grades in my CS degree I agree, transitioning to
the real world with my first job was 12 months of pain.

------
z3t4
Once in a while there will be a unicorn that will spread from mouth to mouth
simply because its better then everything out there.

------
viburnum
The hoop-jumping that most shapes society is the need to maximize profit, at
the expensive of all other goals.

------
dennis_jeeves
It gets worse:

In my view schools/universities are institutions for mass indoctrination. Noam
Chomsky has said something along similar lines. I borrowed the words from him.

A bunch of my other observations:

[https://realminority.wordpress.com/observations-of-the-
world...](https://realminority.wordpress.com/observations-of-the-world-2/)

------
ViralBShah
Thus the famous quote - Don't let your schooling get in the way of your
education.

------
woodandsteel
I think one solution to the problem PG describes is self-directed study.

------
DataWorker
Unit tests are unit tests. Sometimes the unit is the worker.

------
adebelov
I never let my schooling get in a way of my education :)

------
thallukrish
The way we filter candidates into any stream be it college or companies need
to change. People are inherently pressurized by capitalistic mindsets and
competition that makes them to not question things but just follow the herd
mindset of setting up papers and interviews mechanically. We always are
chasing the next than solving the now. Some related thoughts I wrote a while
ago on the state of Education [https://medium.com/@thallukrish/why-the-why-
matters-d7b8170b...](https://medium.com/@thallukrish/why-the-why-
matters-d7b8170b5a2c)

------
listle
Listen to this essay here:
[https://listle.app.link/v0haf9qve2](https://listle.app.link/v0haf9qve2)

------
flint
"A"s are for cowards.

------
louma12
I think it is characterized by using the ideal solution

------
Bost
Is there any tl;dr ? Thank you in advance.

------
graycat
When I was teaching math in college, it was easy enough to give tests that (i)
really tested knowledge of the material and (ii) were not _hackable_. Here's
how I did that: Just announce at the beginning of the class that (i) the
homework assignments were just to work enough of the exercises in the book to
be good at working the exercises and (ii) the tests would be just some of the
exercises in the book with just minor or no modifications. Worked fine.

In D. Knuth's _The TeXBook_ is:

"The traditional way is to put off all creative aspects until the last part of
graduate school. For seventeen or more years, a student is taught
_examsmanship,_ then suddenly after passing enough exams in graduate school
he's told to do something original."

In particular, there can be some _comeuppance_ to too much course hacking and
not enough learning: Without enough actual learning, may have too much trouble
doing "something original".

On the course hacking Paul describes:

(1) Dad had his Master's in education and to me emphasized just learning and
never grades. So, I emphasized learning. He was a bit extreme in ignoring my
grades: A few times my neglect of grades cost me, never much but a little.

And for Paul's remarks on studying for tests, that's what I did: I never
studied for tests. Essentially as Paul explained, all along in the course I'd
been trying to learn the material so no way would just _cramming_ and losing
sleep help me.

Broadly Dad was correct: Most of what I got out of education that actually
helped me was just the learning. And one of the best things I did learn was
how to learn, i.e., pick good subjects, topics, and sources and learn
independently from the sources.

E.g., the college I went to as a freshman was selected because it was nearly
for free and I could walk to it. But the college was so much like just a grade
13 that they didn't let me start with calculus. But I'd been a math major at a
relatively good college prep high school, e.g., that the year before me had
sent three students to Princeton and my year sent one to MIT. And in my year,
of #1, #2, #3 on the Math SAT, the MIT guy was #3 and I was #2. Similarly on
the CEEB Math knowledge test. Sooooo, I SHOULD have been in calculus, and the
course they had me in was 90% lower than what I'd already learned in high
school and for the other 10% I just taught that to myself in a few days. But I
wanted to be a college math major and not fall behind so got a good calculus
book and dug in and did okay. The next year I went to a college with a quite
good math department, continued with their sophomore calculus from the same
text Harvard was using, pretty good text, made As, and did fine. So, I never
took freshman calculus; my ability to teach myself saved my tail feathers.

On Paul's remarks on grades, there was a cute story: When the SAT scores came
back, one of the teachers, I'd had in the sixth grade, was appointed the
_counselor_ for giving the scores to the students; so, she knew me well and
knew my _reputation_ among the teachers. She read my Verbal SAT score and had
a nice smile, said it was "Very good". It wasn't so good, but she was happy
because the score was MUCH better than my reputation! Then she read my Math
SAT score and became afraid, upset, confused, speechless. When she recovered
she said "There must be some mistake." Yup, there had been, hers and the other
teachers for 12 painful years.

E.g., there was one really good day for learning: In plane geometry, my
favorite subject, the teacher was competent but really nasty. No way did I
want her to have any credit for my learning; so no way would I ever admit to
doing any homework. Each day she assigned three not very difficult exercises.
I outlined solutions in the margin of the book for the most difficult
exercises, maybe some of hers, and then turned to the more difficult exercises
in the back and did them ALL, never once missing one. One of them took me from
Friday afternoon to Sunday evening. In class on Monday, she worked at the
board an easy exercise with the same figure I'd worked on all weekend. So, I
woke up from having my head down resting as usual and for the first and last
time spoke in class: "There's an exercise in the back with the same figure."
She took the bait! About 15 minutes later, with frustration, she was exhorting
the class "Think, class, think. Think about the given ....". Heck, I didn't
know that I was the only one doing ALL the homework and that even the teacher
had not done all the homework! I didn't want to be accused of ruining the
class so blurted out "Why don't we ..." and in a few more words, with great
anger, she interrupted me. NO WAY did she want my solution! With anger she
said "You knew how to do it all along!". Of COURSE I did: No way would I ask
HER for help in math! This story might make a good scene in a movie!

There was one more with her: On the state test, the last problem was how to
inscribe a square in a semi-circle. After the test I kept thinking about a
good solution so asked to see her after school. She didn't like that. I
mentioned the problem, and she asked "Why are you working on that?". Well, it
was curious, and I'd already solved everything else in sight! So, I went off
on the side and constructed a square, circumscribed the semi-circle,
constructed a fourth proportional, so had the crucial length in the original
figure and did that construction. She said it was not legal to construct the
other figure. In college, a student in a geometry for teachers class showed
me, given triangle ABC, construct D on AB and E on BC so that the lengths of
AD = DE = EC. So, off on the side I started with the angle BAC, picked any
length for the AD for that figure, finished constructing the figure on the
side similar to the given figure, found the crucial fourth proportional, and
completed the construction in the original figure. She said, "How'd you know
to do that? That is smilimitude, an advanced technique we are studying." I
said "I rediscovered it in high school, and the teacher said can't do that.".

(2) I started my career in applied math, computing, and physics for mostly
problems in US national security near DC. There no one cared what course
grades I'd made but cared a LOT what I could DO that was important and based
on what I'd learned or could learn quickly. And I was learning like drinking
from a fire hose. Net, what I knew and learned got me a good career, at one
time an annual salary six times what a new high end Camaro cost.

(3) I did well in freshman physics, led the class. One of the guys I'd beaten
got some respect for me so contacted me when FedEx was struggling with
scheduling the fleet. I'd been teaching computer science at Georgetown
University (had never taken a course in computing but had been learning from
that fire hose) so joined FedEx, typed software furiously, got a schedule for
the fleet, thrilled the Founder Smith, pleased the BoD, enabled some crucial
funding, and saved the company, all based on what I'd learned, in physics
class and in computing. Later did a little calculus for some revenue
projections, pleased two crucial BoD Members, and saved the company again.

(4) In grad school, what I'd learned before entering helped a LOT: On the five
Ph.D. qualifying exams, I did the best in the class on four of the exams, and
for three of them what I did was all from what I'd learned in class or taught
myself before entering. Later my interest in learning and neglect of
formalities torqued off my department Chair. So, in a course in optimization,
I saw a question with no answer in the course. I found no answer in the
library. I got some rough ideas for how to get an answer so signed up for a
_reading course_ to address the problem, not necessarily solve it. In two
weeks I had a nice solution with a nice new general theorem and also a nice
solution to a related problem stated but not solved in the famous paper of
Arrow, Hurwicz, and Uzawa in mathematical economics. My work looked
publishable (it was; I did publish it later in JOTA). Word of my solution
spread in the department. Then I had a halo: No one cared about formalities
and let me just LEARN and, then, do my Ph.D. research, with no faculty
direction. Again, what I had LEARNED saved my tail feathers. And for Knuth's
remark about something "original", the learning let me do that. I did find
that the thinking techniques I'd used to do the more difficult exercises in
Rudin, Royden, Neveu, Loeve, Halmos, Fleming, etc. were sufficient to do
publishable original research.

So, net, just learn and do that by doing the more difficult exercises! E.g.,
back to high school a challenging test question got me to reinvent similitude!

------
ronilan
Q: > _You 're investing time in the same company you're asking them to invest
money in. If it's not a good investment, why are you even doing it?_

A: Because you and them don’t have the same circumstances, and thus don’t have
the same utility function. You have different values.

The whole (long) thing is a mess of obvious misses. Too bad.

------
tmsh
Fwiw I responded vi Twitter thread
[https://twitter.com/p10q/status/1203348614338113536](https://twitter.com/p10q/status/1203348614338113536)

------
daly
Are you in EECS? Most big data centers use FPGAs "on the wire" to do things
like encryption on the fly or compression so that they get "wire speedups"
without burning CPU cycles.

So FPGA solutions that "live on the wire" and can be "plugged into the LAN
port" are a really useful area you can hack in your dorm room.

I worked in the security area. One of the key problems is "exfiltrations"
where someone tries to copy valuable files off a server. I created a startup
to attack this "on the wire" with hardware so it can't be hacked. (It failed
because nobody knew what an FPGA was, nor how TCP packets worked. Sigh).

The idea is to create crypto-hashes of the valuable files. Hand-install the
hashes into an FPGA using a read-only micro card. The FPGA sits on the wire,
hashing files being sent. If the hash matches, scream.

THe same idea of "unhackable" malware virus scans can be done on an FPGA. Only
someone with physical access to the FPGA card can modify it. It operates "at
wire speeds".

The hard part is finding people who can spell FPGA and know what malware
means. Be that person.

You could do this in your dorm room.

The side effects are that you learn a LOT very quickly, it is grounded in real
code, and you could actually convince YC that your "homework" is worth a
passing grade.

Besides, FPGAs are just plain fun. Make your own hardware Neural Nets. Hack
your bike with a camera and an NN to warn of approaching cars.

~~~
rbinv
Wrong thread?

~~~
klik99
or a very overtrained neural net machine commenter

