
How to Catch a Cheater - bkrausz
http://nerdlife.net/how-to-catch-a-cheater/
======
WillyF
During my Freshman year of college (2002), I was taking Econ 101. It was a
class of probably 500 students, and our tests were multiple choice. By the
time I got back to my dorm room, the answers to the test were always up on the
course website (I'm not sure when they actually were posted).

After the first two tests, the professor must have noticed that something
wasn't right. I finished the third test quickly, and was pretty confident that
I had done extremely well. We used scantron sheets for our answers, so we got
to take the paper with the questions home after the test. I had marked all of
my answers on both sheets, so I loaded the course website and started checking
my answers.

My heart sank as I went through the answers. Somehow I had managed to get
every single question wrong. I figured that there must have been something
wrong with the answers, and when I matched the answers (not the letters), I
realized that I had actually gotten every question right.

The prof swapped the letters, so the kids whose friends were texting them the
"answers" got screwed. It was pretty funny, even if it increased my blood
pressure for a little bit.

~~~
lotharbot
For freshman year math courses, we wrote two versions of each test. They'd be
nearly identical, but with a few of the numbers and variable names modified.
We'd alternate passing them out so that people sitting next to each other had
different tests. We'd also go around the room and write down who was sitting
where. This made it really easy to catch the "look at your neighbor's paper"
sort of cheating. (It's really hard to text answers to a calculus test, and
taking a photo of your page would be _really_ conspicuous.)

Other departments had their own anti-cheating methods. Kids coming straight
out of high school, who were used to less sophisticated cheat-detection
methods, tended to get burned pretty bad their first semester.

------
wwortiz
> We were explicitly told that searching online for answers to homeworks was
> forbidden.

I might be in the minority but I am of the opinion that if you are grading
students on something that can be easily obtained through a google search you
probably shouldn't be grading them on that.

~~~
pohl
I suspect you're actually in the majority. I bet that, rather, the minority
position is that the instructor should create whatever conditions they want to
so that they can exercise the students as they see fit - and that the students
should extend enough respect to consider why those conditions are being
created. This is my view.

Replace the teacher with a coach in a weightlifting class. One day he wants
you to lift some freeweights, but he wants you to lift them in a particular
way that does not use your legs. (Let's say he's having you do "military
presses".) "Nonsense!" says the modern student, who already knows that it's
best to lift heavy things with their legs. "Why shouldn't I be able to use
this awesome resource (my legs) that's right here at my disposal!?"

Well maybe - just maybe - the coach wants to target your deltoids. Hmmm.

Enter Professor Luis von Ahn, who wants you to do homework without using
google. Perhaps that is because he doesn't want you to target and improve your
research abilities. It could be that, rather, he would like you to exercise
and sculpt your ability to work problems from first principles. The professor
may be aware that practice makes permanent, and homework is intended to get
students to strengthen specific things, and that taking the easy way out robs
the student of the intended benefit.

That's how I approach being a student, and so I grant that when sensei says
wax-on, paint-uh-fence, and sand-uh-flo that he's got some idea of what I
should be working on. But that's an old, old attitude that exists only in a
handful of Hollywood movies these days. So that's why I'm betting you're
actually in the majority.

By the way, the instructor here is an awesome guy. Anybody who hasn't heard of
Luis von Ahn should do themselves a favor and watch his presentation on Human
Computation:

<http://www.youtube.com/watch?v=dtFroEJN1nI>

~~~
xenophanes
Many math teachers ban calculators on math tests. I'm pretty sure this has
majority support, not minority.

Google does simple math, so that's basically an equivalent issue.

IMO it's dumb -- in real life you have access to Google. But I mostly wanted
to say I think there's clearly majority support for artificial grading
conditions, since the no-calculators thing is so popular.

~~~
tzs
> IMO it's dumb -- in real life you have access to Google.

If one is taking a class merely to learn how to Google for the answers, then
one should not be in a university. One should be in a trade school which only
teaches you enough of your fields of study to be able to figure out the right
terms to Google for.

When you take university classes, the goal is to learn to be the people that
provide the solutions that the trade schoolers will Google for.

------
dkarl
My AP (Advanced Placement) Chemistry teacher noticed some cheating on a major
chemistry test we took. The cheating was pretty obvious; some kids just
decided they were going to fail the test so they might as well have some fun.
A week later, after a week of lecturing on a new section of the chemistry book
and no mention of our scores, a student asked. No, he hadn't graded the tests.
He was sorry, he had been really busy. After that, students started asking him
every day if he had graded the tests.

Two weeks after we had taken the test, he said, "I have to come clean. The
reason I have not graded your tests is that I lost them and have been trying
to find them for the last two weeks. I'm very, very sorry, it was entirely my
fault, and as a result" (he produced stack of paper from behind his back) "you
all have to retake the test right now. Put your books away, take a test and
two sheets of scratch paper, and start working."

We didn't have as many students sign up for the second semester of AP
Chemistry. Not many people failed the first semester, and scores in AP classes
were given a full letter grade boost compared to regular classes, but students
knew he wasn't messing around. As a result, it was one of the best AP classes
we had, and lots of kids got good scores on the AP test.

~~~
Semiapies
I wonder how many perfectly honest students did worse on the impromptu re-take
two weeks later, with entirely new material on their minds.

~~~
tomjen3
Agreed it seems that it is always the honest students who gets the short end
of the stick.

Thats a good thing though - it prepares the students for the real world.

~~~
skinnymuch
How is doing this preparing the students for the real world (I'm being serious
and not condescending or anything)? I don't see the real world being
particularly more difficult than normal high school. Example - didn't file or
pay taxes on time? No problemo - just take a penalty and finish up your work
late.

~~~
tomjen3
Because it causes them to loose (hopefully) any idea that they get anything
out of being honest, except shafted.

~~~
dkarl
You've got it backwards: the students who studied honestly had much better
retention after two weeks than the kids who wasted time trying to cheat or
game the test. It was the teacher's intention to reward honest work by giving
the hard-working kids a big advantage over the cheaters, and that's exactly
what he accomplished. If only life was always so fair!

~~~
skinnymuch
Your explanation is for a fantasy-land. Studying honestly in many AP classes
comes down to cramming. Here's how Semiapies summed it up in a comment in this
thread:

"In high school and particularly AP, those courses are blatant cycles of 1)
cram, 2) test, and 3) forget all but the core of what that section was about.

(If you want to say, "You should learn it all for the final" keep in mind
nobody does, nor are final exams built around remembering everything from the
whole course.)"

\--

It's not like the crammers are even bad kids. A lot of them have no choice
because if they didn't study that way they wouldn't have enough time to study
and will end up getting worse grades.

This would mainly show people that the world screws you over and people who
are part of crappy systems will go ahead and screw you over on top of the
crappy system. I see little fairness in this situation.

~~~
dkarl
Semiapies' description of AP classes is really bizarre and can only reflect a
situation in which the teachers did not give a rat's ass about their students'
AP scores. The nice thing about an AP class versus a non-AP class is that the
AP test gives teachers an excuse to care whether their students are actually
learning or not. Naturally that means the kids end up working a lot harder,
but teachers who care at all about the scores at the end of the year (and mine
did) will do everything they can to discourage cramming and other fake
learning.

Kids who can't pass the class without cramming and who aren't going to retain
the material shouldn't be in an AP class in the first place. They're going to
run into a concrete wall at the end and waste their money (or the school's
money) taking a test they aren't prepared for. Some students can handle it,
and that's who the class is for. In my graduating class, there were maybe a
dozen kids who got a 5 on at least one AP test, and I think there were two
dozen who got a 3 on at least one AP test. That's out of a graduating class of
a little over 300. (The entering class was over 400, so you can tell I didn't
go to an especially good school.)

------
aplusbi
That sounds terrible. If I saw something like that, I'd probably do the
problem, then look up more information on it. I'd probably start with
wikipedia but move onto google (well, ddg these days) if I didn't find
anything.

The sort of policy that does not allow one to do basic research is pretty
anti-academic if you ask me.

~~~
bkrausz
The early homeworks in this class (Great Theoretical Ideas in Computer
Science) were basically puzzles, where if you've seen the style of problem
before you more or less knew the answer. The goal was to expose us to these
types of problems and teach us how to think them through properly.

It was definitely one of the most fun courses I've taken.

~~~
flogic
That's one of those critical to include details. Otherwise the story makes the
prof sound like a douche.

~~~
bkrausz
Absolutely right, I updated the post.

------
Semiapies
This is a neat little sting, but looked at honestly, it's entrapment.

 _"Commonly referred to as The Glorblar Problem"_

That phrasing is a cue that the student is _expected_ to either look something
up or to refer back to notes or memory of lectures or discussion of the
Glorblar Problem in class. Why else mention it, if it were an honest homework
problem?

We know they didn't hear about the Glorblar Problem in class - it doesn't
exist. For the same reason, we know that hitting the books, either back at the
dorms or in the library, wouldn't have turned up anything.

Remember that this is _homework_ , not a test. Yes, presumably the problem is
doable without any reference material. The students don't necessarily know
that, though. They do know that they've been _prodded_ in that question to at
least check their thinking or work against external information....which isn't
available to them through the channels they've been permitted to use.

So, the professor deliberately tempted people to look up something online,
then amused himself by busting the people who did. Cute method, but it proves
nothing other than "schmuck bait works".

~~~
rick888
"So, the professor deliberately tempted people to look up something online,
then amused himself by busting the people who did. Cute method, but it proves
nothing other than "schmuck bait works"."

The way I see it, the people that are actually doing their homework wouldn't
even be tempted by it..because they are actually doing their homework.

"Remember that this is homework, not a test. Yes, presumably the problem is
doable without any reference material. The students don't necessarily know
that, though. They do know that they've been prodded in that question to at
least check their thinking or work against external information....which isn't
available to them through the channels they've been permitted to use."

Checking your work against the correct answer and Googling the answer (and
just writing it down) are two different things.

~~~
Semiapies
Thank you - this is the most vapid thing I've seen today not written by a TSA
official.

Especially:

 _"Checking your work against the correct answer and Googling the answer (and
just writing it down) are two different things."_

And you can tell the difference based on a server log...exactly how?

~~~
rick888
"Thank you - this is the most vapid thing I've seen today not written by a TSA
official."

Are you here all week? Do you always get this many laughs?

"And you can tell the difference based on a server log...exactly how?"

If you can't see that googling answers isn't cheating, I'm sorry.

------
ecaradec
I remember professors doing that kind of things when I was a student then
declaring that students didn't studied well, didn't followed their rules,
cheated, or anything else.

I don't think it help students to progress it any way. It just feel like you
are fighting against a bureaucraty which has no rules, or which has rules that
will only be revealed until it's too late. What they are creating is traps,
and traps don't teach anything because they are random.

Please teachers, read Richard Feynman stuff, he had such a joy to teach his
students and stop bullying your students.

------
PostOnce
If the point of college is to learn rather than to simply get a degree, then
after you do you work, you should check it, to see if you got it right, and if
you failed, try again.

So, he doesn't want students to check their work, eh? After that first
assignment, I'd cheat by every method I could possibly think of at every
opportunity, because at this point, he has made it about grades, rather than
about education.

The only time you can't go back and re-work and check the internet for help is
during a test. You're SUPPOSED to look things up with homework, homework is
for learning before a test. After all, the whole point of homework is to
prepare yourself for a test, if you don't do the homework, if you just copy
the answers, you won't be ready for the test. So what exactly is he achieving
by not letting students get the maximum value out of homework?

------
Semiapies
Here's a funny thought - what if someone did the assignment, put it away, then
wondered, "Huh, so why is the Glorblar Problem notable?"

Dirty, dishonest intellectual curiosity - you have to _catch_ those people,
right?

------
pg7
You guys in US do it all the wrong way. I remember how examiners fought
cheating when I was a student at Warsaw University, Poland (1990 - 1995). They
simply allowed cheating. So, at the beginning of every exam an examiner
announced something like: "Ladies and gentlemen. You are allowed to cheat. You
can browse your books and notes, no questions asked. But I must warn you that
exam tasks are designed such a way, that they require thinking - and thinking
is not compatible with cheating, because thinking is fast and cheating is
quite slow. So, you are allowed to cheat, but if you will it is very likely
that you will not complete minimum number of tasks required to pass the exam.
I wish you good luck."

------
xenophanes
He should have put the wrong answer on the website. Anyone who just copies it
will get a 0 for being wrong. Anyone who actually thinks about it and notices
the difference shouldn't fail, imo, since that's appropriate, thoughtful use
of the internet.

------
cousin_it
Why not just put an incorrect solution online?

~~~
ultrasaurus
Agreed, that's what I thought the punchline would be. A sample solution that
only works with the sample data provided. (something along the lines of x*y =
x+y, with the sample data being (0,0) and (2,2))

------
daimyoyo
The true lesson here is simple. If you're going to do something online you
shouldn't, get behind a proxy(or 7).

------
lazyant
It seems this professor is more interested in entrapment and showing how smart
he is rather than in his students learning (OK, maybe this was the lesson)

~~~
rick888
..or catching cheaters. The students that cheated obviously don't care about
learning or education, so why should he?

------
sachitgupta
Wow - this seemed strangely familiar, until I realized I was in the class!
What sucks is that a lot of people found the solution and showed it to other
classmates, but only the 1 person who found it got into trouble. (No, I wasn't
one of the students who found it).

------
nohat
So he was going to subpoena their isps? Only the people who used lab computers
needed to worry.

~~~
bkrausz
If you're in CMU's network your computer is registered with them, so unless
you live off-campus (which Freshmen aren't allowed to do) you can easily look
up which student registered which machine (plus it's usually obvious from the
hostname).

------
torme
This seems to severely punish those who would even attempt to follow the
rules. I've had instances where I can't find the answer to a problem and will
spend hours hunting it down. If I had a professor give me a problem with that
constraint, I'd probably scour a book for hours. What are you other options?
Put "I don't know."?

Seems like this just forces everyone to the same result: those who break the
rules will and save themselves time, those that try and follow the rules will
probably break after hours, and end up googling anyways.

Maybe I'm missing something but this seems totally absurd.

------
enjo
General cheating story:

My wife is a professor (ya that again:>). As an accounting professor they do a
lot of their homework/projects in excel. Excel, of course, saves the
information of the document creator in it's meta-data which is trivial to
look-up. She'll routinely get 5 or 6 of the exact same file (with the same
origination data) for the same project. Overall, she'll catch 10%-20% of her
class cheating in a given semester without even really trying.

------
gnubardt
It really bothers me when people cheat in engineering classes, or really
anything where your competence (or lack of) could affect someone's wellbeing.

------
hdctambien
I think some people are missing a few critical points of this exercise:

1) the homework was supposed to work out your thinking muscles. Getting the
"right" answer wasn't the point of the assignment as much as trying to
discover how to obtain an answer. (much better described in these comments by
pohl)

2) On crying "entrapment": See Orpheus and Eurydice
(<http://en.wikipedia.org/wiki/Orpheus#Death_of_Eurydice>). There were rules
clearly put in place and whether they made sense or not, if broken carried
with them penalties that the perpetrators were well aware of. If that doesn't
smell like the "real world" I don't know what does. With that said...

3) Universities are NOT the real world. They barely resemble the real world.
The point of university has never been to "prepare you for the real world."

~~~
Semiapies
As for 1, I think you're missing that a lot of people responded before the
blogger updated the post (twice) in order to explain those points, including
that he'd misremembered how the question was phrased.

As for 2, heaven forfend anyone apply ethics to the actions of those in
authority.

As for 3, well, you've just cancelled out your second point.

------
jchonphoenix
My year the puzzle was called "Giramacristo's puzzle."

And Luis ended up telling everyone he was going to give them 0's and they'd
take a huge hit in their grade, but he ended up not actually factoring that
into the grades.

------
dot
run a proxy server company? this would be perfect material for a viral
campaign.

~~~
protomyth
I was actually wondering how he would get the access records for the school's
PCs? It would seem that it would be a big bluff as most Sys Admins would give
such records to a prof.

~~~
enjo
At least at the two schools I'm fairly familiar with, when investigating a
cheating allegation those are routinely handed over.

------
kbatten
I guess the lesson is that real life is not at all the same as university
life.

------
phalien
Or you could just watch Lie To Me and learn to interpret microexpressions :)

<http://en.wikipedia.org/wiki/Microexpressions>

