My institution is using software called honorlock. It is a MESS. your laptop/device doesnt' have a webcam? you cant' take a test. Do the computers in computer labs have cameras? No. Need to take a test right after an in person class? your SOL because you will get flagged for having a mask on if you take the test in a public space...so you have to go back to your room. Have a roommate who needs to be in the same space while you take a test? Flags all over the place. Wear a facial or head covering? Flags all over the place....and then we have a serious issue.
Add in that faculty have lazily started to just use the flags as 'you cheated' without reviewing them meaningfully. I tried it once, one student, for no discernable reason, generated over 200 flags.
These software are useless, not because they don't have their place but because they people implementing them are trying to solve their own fear rather than a meaningful problem in a meaningful way. Tests are minimally useful as an educational assessment to begin with. Some, like licensing exams, sure fine I'll give you some grace there. But if you are going to use the tool...you need to make at least a minimal effort to use the tool well. If you don't you make the tool useless. That lack of give a damn by users is driving a lack of give a damn by vendors.
I'm sorry but reading all this, at a certain point I have to ask: Why are people (students, faculty) playing along with that? Is there not a point where people are willing to stand up for themselves and say: Enough is enough?
Imagine, students would be asked to appear in the nude to write the exams and undergo invasive checks to prove that they haven't got any miniature transmission devices hidden on their body. Would they just agree to this as well? Would the faculty that's tasked to check the students' orifices play along happily?
I wonder what a degree from an institution that treats students (and faculty forced to use this) with such a level of contempt, treats them as a priori cheaters and crooks instead of teaching them honor and responsibility, is actually worth? The name "honorlock" alone is beyond ridiculous for obvious reasons.
In my days (at one of the Ivies), admittedly a long time ago, I seem to remember that we had to complete a lot of coursework at home and most if not all our exams were open-book. I even remember a test or two where we could bring all our notes from class (not that it did you much good at all, the skills they asked for just weren't in the notes). Not one exam was proctored in that sense, I think, although I remember often longing for one as I imagined them to be much easier: Just regurgitate some knowledge onto the page and go home. No days and weeks refining your arguments and making a case for yourself.
Professors also always knew if someone hadn't done all the work by himself. I remember a girl breaking out in tears during a presentation because she couldn't explain the meaning of a rather boastful technical term she had used in context (and apparently just copied from somewhere).
I think, if I had been asked to install this kind of malware on my PC back then so that some incompetent perv could watch me answering stupid multiple choice questions, I would've laughed in their faces and just quit then and there (I did eventually quit anyway but that's because the offer was just too enticing). Maybe I would've set up an elaborate sound stage with green screen etc. and spliced it into the camera feed to fool the proctors first before turning on some holograms or dirty movies, I don't know (this stuff wasn't so advanced and readily available in those days).
The point is, these setups can always be fooled. Always. You should therefore create an environment where you don't need them. Where people are proud of the work they do themselves, where the skills taught and asked aren't "replayed" in a manner that needs such proctoring, and where people adhere to an honor code that doesn't require them to cheat. If you can't teach people that nowadays, then what are you good for?
How broken must the current educational system be that a) such software exists and is widely used and b) its vendors aren't simply laughed out of the room straight away and c) both students and faculty don't complain at least as loudly about that stuff as they complain about certain other things...
> Why are people (students, faculty) playing along with that?
My theory: Since the early 1970's the number of college administrators has gone up radically. Here is one example but it seems to be the same everywhere[0]. All things being equal one can assume that more administrators = more policy, more policy = more bad policy. As organizations grow they often turn inward and focus on their own propagation more so than their original mandate. That tends to end up in a scenario that, the bad policy that inevitably results from policy creation is defended as arguing against it is seen as a attack on the organization itself. A organization like that is calcified, it's members stop asking questions as they have seen others that have been punished for that insolence and they simply tow the line in order to make it out the other end (graduation or retirement respectively).
Students don't have a BATNA or the self worth to be able to say no, which makes sense, given how much they've invested in their degree already.
Suppose you're a 3rd year student, and you're asked to do this. You've probably already spent at least ~20K and 2 years of your life on your degree so far (if not more). At that point, it's a hard sell to ask them to just quit at that point and throw away the time and money invested. Might as well just suck it up and deal with some spying for the next year.
> Why are people (students, faculty) playing along with that?
From the faculty perspective: lack of time to do anything else. Writing good test questions (questions that require synthesis of multiple topics and not just regurgitation) is hard, takes a lot of time, and may not even be possible in some introductory classes. Looking for cheaters after the fact is time consuming, too. I have to pay for a Chegg account out of my own pocket, and then keep it open whenever I do my grading, to look for solutions to my own questions.
That said, I steadfastly refuse to use Lockdown or Protorio or any of the other "solutions" my school offers; the downsides are simply too great. If I end up passing a few people who cheated their way to an A, that's a price I'm willing to pay.
>In my days (at one of the Ivies), admittedly a long time ago, I seem to remember that we had to complete a lot of coursework at home and most if not all our exams were open-book. I even remember a test or two where we could bring all our notes from class (not that it did you much good at all, the skills they asked for just weren't in the notes). Not one exam was proctored in that sense, I think, although I remember often longing for one as I imagined them to be much easier: Just regurgitate some knowledge onto the page and go home. No days and weeks refining your arguments and making a case for yourself.
Right! All my tests are open notes, open book, use real data, use measurements taken from oneself (its biomedical engineering statistics)...its not that you can't cheat its that cheating is harder than learning. I basically tell them as much and the students seem to appreciate it. I always have one or two numpties a year, and they get smacked, learn a lesson (or don't) and I move on with my life.
Cheating being an effective approach to a class is more of an indictment of the class than the student in my eyes.
The problem is inline with what venture capitalists seek for: a technological solution for something (so it can be repeated ad nauseum).
And the root problem is that some things cannot and should not be automated. There definitely is a need - but the software itself is racist, sexist, and more because it is badly trained with data whose origins are racist and sexist... But the key is that 'its software so it doesnt have a bias and therefore we trust it'.
And what do we get when we have this software subjected against us? We end up with black and darker skin people not being identified, we end up with software that discriminates against those who wear face coverings, and more... And when those maleficent softwares 'flag' people, the bias is that they have no bias and are to be completely trusted. And then the professors fall in line.
Whats the solution here? Well, it's ugly. Most universities are just paths to a degree for a better job - and those uni's know that. People demand a return on investment and will cheat so they don't lose their money. So cheating will happen. But the ProctorAI and ilk are setting themselves up for company-breaking lawsuits. And until we have enough people vaccinated, in-person is going to be pretty discouraged (for good reason!).
Personally I think the root of the problem is how much value is placed on the outcome of exams. Apparently the deal is that companies, in addition to their own invasive and broken interview methodologies, want proof from extremely expensive external institutions that you can cram a lot of information and retain it for at least one day.
Maybe we should switch to the old Chinese system where you had to write a nice poem and then were automatically qualified to do pretty much anything.
Those folks are obviously beyond having any sense of irony even if it slaps them in the face ("Honorlock" was mentioned as another piece of such software above), let alone an etymological understanding of their company name.
I completely agree with you that any form of exam in which such a system would even theoretically add any value is broken, meaningless and should not be used in the first place.
Ironically, nowadays Chinese schools (don't know about unis) are also among those who employ extremely invasive methods against cheating. Competitiveness for coveted spots at some institutions is orders of magnitudes higher than in the west, the tests are all completely standardized and students have shown remarkable inventiveness in finding ways to cheat.
Don't say: "Undress to impress & show your proctos to your proctor."
Say rather:
The cloakroom pegs are empty now, /
And locked the classroom door, /
The hollow desks are lined with dust, /
The students here no more.
They sit at home, get perved upon /
by proctors via cam. /
I raise my glass and praise the fact /
At school I no more am.
> In the United States and some other countries, the word "proctor" is frequently used to describe someone who oversees an examination (i.e. a supervisor or invigilator) or dormitory.
The most malware-y test-related piece of software I've had to install on my computer is Lockdown Browser, which requires to be run as root on macOS or Windows(!) (no Linux obviously), detects if its running in a VM, has a blacklist of processes that it forcibly kills and disables use of external displays and keyboards. I've heard that there's also a camera/microphone option but my professor didn't enable it.
And it's a difficult situation to be in because on the one hand you feel a moral imperative to say something but on the other you just want to pass the class. I have no idea how effective it was when students could just use their phone to look things up.
It doesn't have to be this way. I especially like UC Berkeley's guidelines for remote exams[0], some of which have been followed by other professors. e.g. if you limit submission time it's going to be near impossible to learn how to convert a state machine into a synchronous logical circuit and perform it within 15 minutes (plus other questions).
Oh yes, the webcam feature definitely exists and sucks. A lot.
I have to use LockDown + Webcam for my physics exams, which are very heavy on solving equations on your own paper. It works mostly fine until it notices that I'm looking at the paper, then slaps a blue splash page over my screen saying "hey we can't see your face you filthy cheater, show us your face." Happened at least 10 times on one exam.
With webcam exams I feel almost obligated to find a private space to take it outside of my room because you're not 'supposed' to have anything in the background of your video and that would be an unfair imposition on my roommate since he has to deal with the same crap. Fortunately my residence hall has study rooms I can use.
Everyone hates LockDown. When I just had one laptop (a MacBook), there was a 50% chance it would lock up the entire laptop after launching the software, requiring a hard reset. I currently run it in a separate user account on a separate (Windows) machine. I have an iPad, and I'm honestly 100% ok running it on there but turns out some professors disable the iPad app for no reason and sometimes it just doesn't work.
I definitely appreciate the exam guidelines you posted and agree with them. Funny enough, that physics class I mentioned also does the "set time exams" where everyone starts at 4pm. You're entirely SOL if you can't procure a working laptop that will comply with LockDown if yours isn't working.
Fortunately not all instructors use LockDown here, some ask you to join a Zoom call while you're taking the exam (which I think is mostly fair) and some just post the exam as a regular Canvas quiz with varying security settings.
Personally, I think the only form of "cheating" that should be actively seeked out is that students work alone and are submitting their own exam (that is, I should not be paying someone else to take my exam). There are few situations in which you do not have access to the internet and exams should honestly be written in such a way that internet access shouldn't completely break them.
I haven't gotten anywhere making the moral argument to teachers or administration at my university. However, making the technical argument that Lockdown doesn't run on my system because I have Linux on my laptop worked. Occasionally teachers will ask to video chat with me during the exam, but I think that's acceptable.
Most of Berkeley's own classes don't follow the official guidelines or best practices. I just took a high-stakes test today that was curved, closed note/book/everything, and required live proctoring over zoom with hands in view at all times. And that isn't cherry picking, because all of my classes this semester with assessments have similarly invasive policies. I am thankful, however, that the pushback on proctoring software has been strong enough that the university doesn't use it. Most faculty (4/5 of mine. I dropped the 5th person's class) will also accept students with technical incapabilities or objections to being recorded live, probably because they understand that if a student escalates privacy concerns, the department will side with them thanks to the university's "official" stance on proctoring.
> The most malware-y test-related piece of software I've had to install on my computer is Lockdown Browser, which requires to be run as root on macOS or Windows(!) (no Linux obviously), detects if its running in a VM, has a blacklist of processes that it forcibly kills and disables use of external displays and keyboards. I've heard that there's also a camera/microphone option but my professor didn't enable it.
That's one of the reasons I'm not opposed to app stores and "closed garden" ecosystems.
Want to run as root? Denied. Want to install your special snowflake driver? Nope. Kill processes? Never. Get a list of other running processes not started by your app? Absolutely not.
> how to convert a state machine into a synchronous logical circuit and perform it within 15 minutes
You speak in riddles.
We implemented similar recommendations at the start of Covid when our university got closed and they slow down the cheating process but it's not enough to stop students fully when they're coordinating over facebook. I'm hoping more strategies come out of the lockdown.
Well, hopefully the big scandalous hack happens early on and stops this practice before the harm it will cause is too widespread. Future headline: “Millions of student computers fall victim to massive ransomware attack after testing software bug exploited. Vendor suing students who first notified journalists about it”
> has since deleted the posts and apologised, saying that he and Proctorio “take privacy very seriously”.
There's an inherent irony in this that presumably he didn't realise at the time he made those posts...
Actions speak louder than corporate mouthpiece words. Companies that care about privacy don't do this sort of thing, and don't build privacy infringing products.
I don’t understand why people don’t switch to open book exams. I have been doing it since COVID-19 situation. Despite open book my students struggle to to get an A.
Another aspect: How much inadvertent child pornography has Proctorio captured and stored within their system? What are their controls around handling and removing this data? Is anyone with access to this data a registered sex offender?
How many professors and/or Proctorio employees have masturbated to images of students captured by this software? Do students with large breasts or cleavage showing have their videos reviewed more frequently?
That's assuming "cutting edge machine vision" isn't offshored to third party contractors in foreign countries where the students and parents have no recourse.
Yeah, are nude images of children being sent to foreign countries? Does proctorio have controls to prevent this and/or provably delete this data when this occurs?
Are their third party contractors who have access to image data in jurisdictions where US courts can compel them to delete nude images that were sent to them by Proctorio?
My university's: solution give just enough time, ramdomize question order, no backtracking. Some profs did the work and you get the questions from a set, instead of the same ones as everyone else. For oral exams webcam, microphone is required(usually done trought ms teams). For written exams webbrowser and rarely a vm. No webcam, microphone, malware required. They literally banned the usage of microphone, webcam, malware for spotting cheaters during written exams.
This is similar to what Indian education startup WhiteHatJr was trying to do & lost badly after being exposed for coming up with fake children who received magical job offers from FAANG cos, random applications being submitted to app stores, fake profiles for underage users boasting of their work in generic tweets.
Hah just tape a clipboard to the back of your laptop and suddenly all notes are available. Expecting this software to actually prevent unauthorized use of notes/references is foolish.
Add in that faculty have lazily started to just use the flags as 'you cheated' without reviewing them meaningfully. I tried it once, one student, for no discernable reason, generated over 200 flags.
These software are useless, not because they don't have their place but because they people implementing them are trying to solve their own fear rather than a meaningful problem in a meaningful way. Tests are minimally useful as an educational assessment to begin with. Some, like licensing exams, sure fine I'll give you some grace there. But if you are going to use the tool...you need to make at least a minimal effort to use the tool well. If you don't you make the tool useless. That lack of give a damn by users is driving a lack of give a damn by vendors.