The professor gets exactly what they want here, no?
"We experimented on the linux kernel team to see what would happen. Our non-double-blind test of 1 FOSS maintenance group has produced the following result: We get banned and our entire university gets dragged through the muck 100% of the time".
That'll be a fun paper to write, no doubt.
Additional context:
* One of the committers of these faulty patches, Aditya Pakki, writes a reply taking offense at the 'slander' and indicating that the commit was in good faith[1].
Greg KH then immediately calls bullshit on this, and then proceeds to ban the entire university from making commits [2].
The thread then gets down to business and starts coordinating revert patches for everything committed by University of Minnesota email addresses.
As was noted, this obviously has a bunch of collateral damage, but such drastic measures seem like a balanced response, considering that this university decided to _experiment_ on the kernel team and then lie about it when confronted (presumably, that lie is simply continuing their experiment of 'what would someone intentionally trying to add malicious code to the kernel do')?
* Abhi Shelat also chimes in with links to UMN's Institutional Review Board along with documentation on the UMN policies for ethical review. [3]
[1]: Message has since been deleted, so I'm going by the content of it as quoted in Greg KH's followup, see footnote 2
As an OSS maintainer (Node.js and a bunch of popular JS libs with millions of weekly downloads) - I feel how _tempting_ it is to trust people and assume good faith. Often since people took the time to contribute you want to be "on their side" and help them "make it".
Identifying and then standing up to bad-faith actors is extremely important and thankless work. Especially ones that apparently seem to think it's fine to experiment on humans without consent.
Tell someone upstream - in this case Greg KH - what you want to do and agree on a protocol. Inform him of each patch you submit. He's then the backstop against anything in the experiment actually causing harm.
Yes, employers often send out fake phishing e-mails to test resilience and organizational penetration testing is done on the field with unsuspecting people.
Ah. I never replied to the e-mails sent out by my employer about registering for a training in phishing detection. I just assumed those e-mails were phishing e-mails.
Apologies in advance if my questions are off the mark, but what does this mean in practice?
1. If UNM hadn't brought any attention to these, would they have been caught, or would they have eventually wound up in distros? 'stable' is the "production" branch?
2. What are the implications of this? Is it possible that other malicious actors have done things like this without being caught?
3. Will there be a post-mortem for this attack/attempted attack?
I don't think the attack described in the paper actually succeeded at all, and in fact the paper doesn't seem to claim that it did.
Specifically, I think the three malicious patches described in the paper are:
- UAF case 1, Fig. 11 => crypto: cavium/nitrox: add an error message to explain the failure of pci_request_mem_regions, https://lore.kernel.org/lkml/20200821031209.21279-1-acostag.... The day after this patch was merged into a driver tree, the author suggested calling dev_err() before pci_disable_device(), which presumably was their attempt at maintainer notification; however, the code as merged doesn't actually appear to constitute a vulnerability because pci_disable_device() doesn't appear to free the struct pci_dev.
This is not to say that open-source security is not a concern, but IMO the paper is deliberately misleading in an attempt to overstate its contributions.
The ethics in big-lab science are as dire as you say, but I've generally got the impression that the publication imperative has not been driving so much unethical behaviour in computer science. I regard this as particularly cynical behaviour by the standards of the field and I think the chances are good that the article will get retracted.
FWIW, Qiushi Wu's USENIX speaker page links to a presentation with Aditya Pakki (and Kangjie Lu), but has no talk with the same set of authors as the paper above.
It's good to call out bad incentive structures, but by feigning surprise you're implying that we shouldn't imagine a world where people behave morally when faced with an incentive/temptation.
I dislike feigned surprise as much as you do, but I don't see it in GP's comment. My read is that it was a slightly satirical checklist of how academic incentives can lead to immoral behavior and sometimes do.
I don't think it's fair to say "by feigning surprise you're implying..." That seems to be putting words in GP's mouth. Specifically, they didn't say that we shouldn't imagine a better world. They were only describing one unfortunate aspect of today's academic world.
Here is a personal example of feigned surprise. In November 2012 I spent a week at the Google DC office getting my election results map ready for the US general election. A few Google engineers wandered by to help fix last-minute bugs.
Google's coding standards for most languages including JavaScript (and even Python!) mandate two-space indents. This map was sponsored by Google and featured on their site, but it was my open source project and I followed my own standards.
One young engineer was not pleased when he found out about this. He took a long slow look at my name badge, sighed, and looked me in the eye: "Michael... Geary... ... You... use... TABS?"
That's feigned surprise.
(Coda: I told him I was grateful for his assistance, and to feel free to indent his code changes any way he wanted. We got along fine after that, and he ended up making some nice contributions.)
Why should we imagine this world? We have no reason to believe it can exist. People are basically chimps, but just past a tipping point or two that enable civilization.
We'd also have to agree on what "behave morally" means, and this is impossible even at the most basic level.
Usually "behave morally" means "behave in a way the system ruling over you deems best to indoctrinate into you so you perpetuate it". No, seriously, that's all there is to morality once you invent agriculture.
Hypothetically if these patches were accepted and was exploited in the wild; If one could prove that they were exploited due to the vulnerabilities caused by these patches can the University/ Prof. be sued for damages and won in an U.S. court (or) Would they get away under Education/Research/Academia cover if any?
Not an attorney but the kernal is likely shielded from liability by it's license. maybe the kernal could sue the contributers for damaging the project but I don't think the end user could.
Malicious intent or personal gain negate that sort of thing in civil torts.
Also US code 1030(a)5 A does not care about software license. Any intentional vulnerability added to code counts. Federal cybercrime laws are not known for being terribly understanding…
To me, seems to indicate that nation state supported evil hacker org (maybe posing as an individual) could place their own exploits in the kernel. Let's say they contribute 99.9% useful code, solve real problems, build trust over some years, and only rarely write an evil hard to notice exploit bug. And then, everyone thinks that obviously it was just an ordinary bug.
Maybe they can pose as 10 different people, in case some of them gets banned.
"As U.S. intelligence agencies accelerate efforts to acquire new technology and fund research on cybersecurity, they have invested in start-up companies, encouraged firms to put more military and intelligence veterans on company boards, and nurtured a broad network of personal relationships with top technology executives."
Foreign countries do the same thing. There are numerous public accounts of Chinese nationals or folks with vulnerable family in China engaging in espionage.
The fundamental tension is between efficiency and security. Trust permits efficiency, at the cost of security (if that trust is found to be misplaced).
A perfectly security system is only realized by a perfectly inefficient development process.
We can get better at lessening the efficiency tax of a given security level (through tooling, tests, audits, etc), but for a given state of tooling, there's still a trade-off.
Different release trains seem the sanest solution to this problem.
If you want bleeding-edge, you're going to pull in less-tested (and also less-audited) code. If you want maximum security, you're going to have to deal with 4.4.
I have the same questions. So far we have focused on how bad these "guys" are. Sure, they could have done it differently, etc. However, they proved a big point: how "easy" it is to manipulate the most used piece of software on the planet.
How to solve this "issue" without putting too much process around it? That's the challenge.
What's next, will they prove how easy it is to break into kernel developers' houses and rob them? Or prove how easy it is to physically assault kernel developers by punching them in the face at conferences? Or prove how easy it is to manipulate kernel developers to lose their life savings investing in cryptocurrency? You can count me out of those...
Sarcasm aside, pentesting/redteaming is only ethical if the target consents to it! Please don't try to prove your point the way these researchers have.
Just playing devil advocate here, the surprising factor does play into it. No bad actor will ever give you heads-up.
If the researcher has sent these patches under a different identity, that would be just like how malice contributions appear. The maintainers won't assume malice, waste a bunch of time communicating with the bad actor, and may NOT revert their previous potentially harmful contribution.
> the surprising factor does play into it. No bad actor will ever give you heads-up.
I too thought like this till yesterday. Then someone made me realize thats not how getting consent works in these situations. You take consent from higher up the chain, not the people doing the work. So Greg Kroah-Hartmancould could have been consulted, as he would not be personally reviewing this stuff. This would also give you a chance to understand how the release process works. You also have an advantage over the bad actors because they would be studying the process from outside.
I see what you are saying. But he is also sort of the director to this whole thing. The research question itself is worthwhile and I don't think if it was done properly this much time would be wasted. All they have to prove is that it will pass few code reviews. That's a few man hours and I really don't think people will be mad about that. This whole fiasco is about the scale of man hours wasted both because they repeatedly made these "attacks" and because this thing slipped into stable code. Both would be avoided in this scheme.
But I would like to put in a disclaimer that before getting to that point they could have done so many other things. Review the publicly available review processes, see how security bugs get introduced by accident and see if that can be easily done by a bad actor, etc.
the way it should work imho is contributors to be asked for consent (up-front, retroactively), that stealthy experiments would happen at some point. Given the vital role of the linux kernel, maybe they'll understand. And if they turn out to be under-resourced to be wasted on such things, then it would highlight the need for funding additional head count, factoring in that kind of experiments/attacks.
Yes, and if you do it without a heads-up as well that makes you a bad actor. This university is a disgrace and that's what the problem is and should remain.
C'est la vie. There are many things that it would be interesting to know, but the ethics of it wouldn't play out. It would be interesting to see how well Greg Kroah-Hartman resists under torture, but that does not mean it is acceptable to torture him to see if he would commit malicious patches that way.
To take a more realistic example, we could quickly learn a lot more than today about language acquisition if we could separate a few children from any human contact to study how they learn from controlled stimuli. Still, we don't do this research and look for much more complicated and lossy, but more humane, methods to study the same.
They proved nothing that wasn't already obvious. A malicious actor can get in vulnerabilities the same way a careless programmer can. Quick, call the press!
And as for the solutions, their contribution is nil. No suggestions that haven't been suggested, tried and done or rejected a thousand times over.
Agreed. So many security vulnerabilities have been created not by malicious actors, but by people who just weren't up to task. Buggy software and exhausted maintainers is nothing new.
What this proves to me is that perhaps lightweight contributions to the kernel should be done in safe languages that prevent memory leaks and with tooling that actively highlights memory safety issues like use after free. Broader rust adoption in the kernel cant come soon enough.
I also consider Greg’s response just as much a test of UMN’s internal processes as the researcher’s attempt at testing kernel development processes. Hopefully there will be lessons learned on both sides and this benign incident makes the world better. Nobody was hurt here.
I understand where you are coming from, and I agree that it's good that we are paying more attention to memory safety, but how would a memory safe language protect you from an intentionally malicious code commit? In order to enact their agenda they would need to have found a vulnerability in your logic (which isn't hard to do, usually). Memory safety does not prevent logic errors.
> Nobody was hurt here.
This is where you got me, because while it's clear to me that short-term damage has been done, in the long term I believe you are correct. I believe this event has made the world a safer place.
One could argue that when a safe language eliminates memory safety bugs (intentional or unintentional), it makes it easier for the reviewer to check for logic errors. Because you don't have to worry about memory safety, you can focus completely on logic errors.
I would agree that it does, and I do agree that we should try to reach that point. I just want to point out that I think it's dangerous to assume safety in general because one thing is assumed to be safe.
This is for me unrelated and even a little bit minimizing the issue here.
The purpose of the research was probably to show how easy it is to manipulate the Linux kernel in bad faith. And they did it. What are they gonna do about it besides banning the university?
I believe it comes down to having more eyes on the code.
If a corporation relies upon open sourced code that has historically been written by unpaid developers, if I was that corportion, I would start paying people to vet that code.
So you are just fine knowing that any random guy can sneak any code in the Linux kernel? Honestly, I was personally expecting a higher level of review and attention to such things, considering how used the product is. I don't want to look like the guy that doesn't appreciate what the OSS and FSF communities do everyday even unpaid. However this is unrelated. And probably this is what the researchers tried to prove (with unethical and wrong behavior).
I'm not fine with it. But those researchers are not helping at all.
And also, if I had to pick between a somewhat inclusive mode of work where some rando can get code included at the slightly increased risk of including malicious code, and a tightly knit cabal of developers mistrusting all outsiders per default: I would pick the more open community.
If you want more paranoia, go with OpenBSD. But even there some rando can get code submitted at times.
If you've ever done code review on a complex product, it should be quite obvious that the options are either to accept that sometimes bugs will make it in, or to commit once per week or so (not per person, one commit per week to the Linux kernel overall), once every possible test has been run on that commit.
I am not sure if these are the only options we have here. Did you see the list of commits that this bunch of guys sneaked in? It's quite big, it's not just 1-2. A smart attacker could have done 1 commit per month and would have been totally fine. All they needed apparently was a "good" domain name in their email. This is what I think is the root of the problem.
> So you are just fine knowing that any random guy can sneak any code in the Linux kernel?
I mean, it is no surprise. It is even worse with proprietary software, because you are much less likely to be aware of your own college/employee.
Hell, seeing that the actual impact is overblown in the paper, I think it is a really great percentage caught to be honest, assuming good faith from the contributor.
> However, they proved a big point: how "easy" it is to manipulate the most used piece of software on the planet.
What? Are you actually trying to argue that "researchers" proved that code reviews don't have a 100% success rate in picking up bugs and errors?
Specially when code is pushed in bad faith?
I mean, think about that for a minute. There are official competitive events to sneak malicious code that are already decades old and going strong[1]. Sneaking vulnerabilities through code reviews is a competitive sport. Are we supposed to feign surprise now?
Bug bounties are a different beast. Here we are talking about a bunch of guys who deliberately put stuff into your next kernel release because they come from an important university, or whatever other reason. One of the reviewers in the thread admitted that they need to pay more attention to code reviews. That sounds to me like a good first step towards solving this issue. Is that enough, though? It's an unsolvable problem, but is the current solution enough?
What would be the security implications of these things:
* a black hat writes malware that proves to be capable of taking out a nation's electrical grid. We know that such malware is feasible.
* a group of teenagers is observed to drop heavy stones from a bridge onto a motorway.
* another teenager pointing a relatively powerful laser at the cockpit of a passenger jet which is about to land at night.
* an organic chemist is demonstrating that you can poison 100,000 people by throwing certain chemicals into a drinking water reservoir.
* a secret service subverting software of a big industrial automation company in order to destroy uranium enrichment plants in another country.
* somebody hacking a car's control software in order to kill its driver
What are the security implications of this? That more money should be spent on security? That we should stop to drive on motorways? That we should spent more money on war gear? Are you aware how vulnerable all modern infrastructure is?
And would demonstrating that any of these can practically be done be worth an academic paper? Aren't several of these really a kind of military research?
The Linux kernel community does spend a lot of effort on security and correctness of the kernel. They have a policy of maximum transparency which is good, and known to enhance security. But their project is neither a lab in order to experiment with humans, nor a computer war game. I guess if companies want to have even more security, for running things like nuclear power plants or trains on Linux, they should pay for the (legally required) audits by experts.
I agree with the sentiment. For a project of this magnitude maybe it comes to develop some kind of static analysis along with refactoring the code to make the former possible.
As per the attack surface described in the paper (section IV). Because (III, the acceptance process) is a manpower issue.
It's possible that they are developing a static analysis tool that is designed to find places where vulnerabilities can be inserted without looking suspicious. That's kind of scary.
Have they submitted patches to any projects other than the kernel?
As an Alumni of the University of Minnesota's program I am appalled this was even greenlit. It reflects poorly on all graduates of the program, even those uninvolved. I am planning to email the department head with my disapproval as an alumni, and I am deeply sorry for the harm this caused.
> Leadership in the University of Minnesota Department of Computer Science & Engineering learned today about the details of research being conducted by one of its faculty members and graduate students into the security of the Linux Kernel.
- Signed by “Loren Terveen, Associate Department Head”, who was a co-author on numerous papers about experimenting on Wikipedia, as pointed out by: https://news.ycombinator.com/item?id=26895969
It should. Ethics begins at the top, and if the university has shown itself to be this untrustworthy then no trust can be had on them or any students they implicitly endorse.
As far as I'm concerned this university and all of its alumni are radioactive.
It's not about guilt, it's about trust. They were trained for years in an institution that violates trust as a matter of course. That makes them suspect and the judgement completely fair.
Lots of universities have had scandals. I could probably dig one up from your alma mater. They're big places with long histories. Collective punishment achieves little productive and should be avoided.
Collective punishment is a clear and unilateral signal that something is extremely wrong and not within the punisher's power to unwind properly (or prevent in the future). Until it's clear that this university can be trusted, they should not be. I would feel the same about any schools that I attended, and I would not have issues with blanket bans for them either if this was the kind of activity they got up to.
Their graduates might not have been directly involved, but it's not possible to ig ore that those graduates were the product of an academic environment where this kind of behavior was not only sanctioned from the top but also defended as an adequate use of resources.
Adequate use of resources seems like a bizarre reasoning. Do you also evaluate how a candidates alma mater compensates its football staff before you hire?
It makes perfect sense once you realize that universities are in the business of selling reputation.
When someone graduates from the university, that is the same as the university saying "This person is up to our standards in terms of knowledge, ethics and experience."
If those standards for ethics are very low, then it naturally taints that reputation they sold.
Why is the university where you put the line? You could as well say every commit coming from Minnesota is radioactive or, why not, from the US.
It is unfair to judge a whole university for the behavior of a professor or a department. Although I'm far from having all the details, it looks to me like the university is taking the right measures to solve the problem, which they acknowledge. I would understand your position if they tried to hide this or negated it, but as far as I understood that's not the case at all. Did I miss something?
Linux kernel is blocking contributions from the university mail addresses, as this attack has been conducted by sending patches from there.
It doesn't block patch submissions from students of professors using their private email, since that assumes they are contributing as individuals, and not as employees or students.
It's as close as practically possible to blocking an institution and not the individuals.
I think that is a reasonable measure by the LK team. In my opinion, it is the right solution in the short term, and the decision can be revised if in the future some student or someone else have problems to submit non-malicious patches. But I was specifically referring to this comment:
> As far as I'm concerned this university and all of its alumni are radioactive.
That is not a practical issue, but a too broad generalization (although, I repeat, I may have missed something).
I don't read it like this. Alumnis and students are not banned from contributing, as long as they use their private emails. It's the university email domain that is "radioactive". The Assumption here is that someone who uses university email is submitting a patch on behalf of the said university, and that may be in a bad faith. It's up to the said university to show they have controls in place and that they are trustworthy.
It's the same as with employees. If I get a patch request from xyz@ibm.com I'll assume that it comes from IBM, and that person is submitting a patch on behalf of IBM, while for a patch coming from xyz@gmail.com I would not assume any IBM affiliation or bias, but assume person contributing as an individual.
> Alumnis and students are not banned from contributing, as long as they use their private emails. It's the university email domain that is "radioactive".
That's not what the comment I was responding to said. It was very clear: "As far as I'm concerned this university and all of its alumni are radioactive". It does not say every kernel patch coming from this domain is radioactive, it clearly says "all of its alumni are radioactive".
You said before that alumni from the university could submit patches with their private emails, but according to what djbebs said, he would not. Do we agree that this would be wrong?
I think current context of the world as it is is full of unjustified and unjust generalization.
And as unfortunate as it sound it look like all victim of such generalization, the alumni would have to fight the prejudice associated to their choice of university.
That's a ridiculously broad assertion to make about the large number of staff and students who've graduated or are currently there, that is unwarranted and unnecessarily damaging to people who've done nothing wrong.
That's a witch hunt, and is not productive. A bad apple does not spoil the bunch, as it were. It does reflect badly on their graduate program to have retained an advisor with such poor judgement, but that isn't the fault of thousands of other excellent graduates.
Both variations are common. "It was just a few bad apples" is the one you more often see today. But it only became common after refrigeration made it so that few people now experience what is required to successfully pack apples for the winter.
Undoubtedly I am in the minority here, but I think it's less a question of ethics, and more a question of bad judgement. You just don't submit vulnerabilities into the kernel and then say "hey, I just deliberately submitted a security vulnerability".
The chief problem here is not that it bruises the egos of the Linux developers for being psyched, but that it was a dick move whereby people now have to spend time sorting this shit out.
Prof Liu miscalculated. The Linux developers are not some randos off the street where you can pay them a few bucks for a day in the lab, and then they go away and get on with whatever they were doing beforehand. It's a whole community. And he just pissed them off.
It is right that Linux developers impose a social sanction on the perpetrators.
It has quite possibly ruined the student's chances of ever getting a PhD, and earned Liu a rocket up the arse.
> it's less a question of ethics, and more a question of bad judgement.
I disagree. I think it's easier to excuse bad judgment, in part because we all sometimes make mistakes in complicated situations.
But this is an example of experimenting on humans without their consent. Greg KH specifically noted that the developers did not appreciate being experimented on. That is a huge chasm of a line to cross. You are generally required to get consent before experimenting on humans, and that did not happen. That's not just bad judgment. The whole point of the IRB system is to prevent stuff like that.
Ah, so people do actually use the expression backwards like that. I had seen many people complain about other people saying “just a few bad apples”, but I couldn’t remember actually seeing anyone use the “one/few bad apple(s)” phrase as saying that it doesn’t cause or indicate a larger problem.
Based on my time in a university department you might want to cc whoever chairs the IRB or at least oversees its decisions for the CS department. Seems like multiple incentives and controls failed here, good on you for applying the leverage available to you.
I'm genuinely curious how this was positioned to the IRB and if they were clear that what they were actually trying to accomplish was social engineering/manipulation.
Being a public university, I hope at some point they address this publicly as well as list the steps they are (hopefully) taking to ensure something like this doesn't happen again. I'm also not sure how they can continue to employ the prof in question and expect the open source community to ever trust them to act in good faith going forward.
As far as I can tell, the papers he co-authored on Wikipedia were unlike the abuse of the kernel contribution process that started last year in that they did not involve active experiment, but passive analysis of contribution history.
Doesn't mean there aren't ethical issues related to editors being human subjects, but you may want to be more specific.
You realise that the GP went through the trouble to point out that research on people should involve consent, and that they [wikipedia] needed to release a statement saying this. What does that tell you about the situation that gave rise to that statement?
Well, the university of Minnesota managed to escape responsibility after multiple suicides and coercion of subjects of psychiatric research. From one regent: “[this] has not risen to the level of our concern”.
Wow, very interesting read (not finished yet though), thank you. To me, this seems like it should be considered as part of UNM's trustworthiness as a whole and completely validates GKH's decision (not that any was needed).
The way I've seen Harvard, Stanford, and a few other university researchers dodge IRB review is by doing research in "private" time in collaboration with a private entity.
There is no effective oversight over IRBs, so they really range quite a bit. Some are really stringent and some allow anything.
It was a real world penetration test that showed some serious security holes in the code analysis/review process. Penetration tests are always only as valuable as your response to them. If they chose to do nothing about their code review/analysis process, with these vulnerabilities that made it in (intentional or not), then yes, the exercise probably wasn't valuable.
Personally, I think all contributors should be considered "bad actors" in open source software. NSA, some university mail address, etc. I consider myself a bad actor, whenever I write code with security in mind. This is why I use fuzzing and code analysis tools.
Banning them was probably the correct action, but not finding value requires intentionally ignoring the very real result of the exercise.
I agree. They should take this as a learning opportunity and see what can be done to improve security and detect malicious code being introduced into the project. What's done is done, all that matters is how you proceed from here. Banning all future commits from UMN was the right call. I mean it seems like they're still currently running follow up studies on the topic.
However I'd also like to note that in a real world penetration test on an unwitting and non-consensual company, you also get sent to jail.
Everybody wins! The team get valuable insight on the security of the current system and unethical researchers get punished!
A non-consensual pentest is called a "breach". At that point it's no longer testing, just like smashing a window and entering your neighbour's house is not a test of their home security system but just breaking and entering.
Yeah - and usually stops short of causing actual damage.
You don't get to rob a bank and then when caught say "you should thank us for showing your security weaknesses".
In this case they merged actual bugs and now they have to revert that stuff which depending on how connected those commits are to other things could cost a lot of time.
If they were doing this in good faith, they could have stopped short of actually letting the PRs merge (even then it's rude to waste their time this way).
This just comes across to me as an unethical academic with no real valuable work to do.
> You don't get to rob a bank and then when caught say "you should thank us for showing your security weaknesses".
Yeah, there’s a reason the US response to 9/11 wasn’t to name Osama bin Laden “Airline security researcher of the Millenium”, and it isn’t that “2001 was too early to make that judgement”.
But bad people don’t follow some mythical ethical framework and announce they’re going to rob the bank prior to doing it. There absolutely are pen tests conducted where only a single person out of hundreds is looped in. Is it unethical for supervisors to subject their employees and possibly users to those such environments? Since you can’t prevent this behavior at large, I take solace that it happened in a relatively benign way rather than having been done by a truly malicious actor. No civilians were harmed in the demonstration of the vulnerability. Security community doesn't get to have their cake and eat it too. All this responsible disclosure “ethics” is nonsense. This is full disclosure, it’s how the world actually works. The response from the maintainers to me indicates they are frustrated at the perceived waste of their time, but to me this seems like a justified use of human resources to draw attention to a real problem that high profile open source projects face. If you break my trust I’m not going to be happy either and will justifiably not trust you in the future, but trying to apply some ethical framework to how “good bad actors” are supposed to behave is just silly IMO. And the “ban the institution” feels more like an “I don't have time for this” retaliation than an “I want to effectively prevent this behavior in the future” response that addresses the reality. For all we know Linus and Greg could have and still might be onboard with the research and we’re just seeing the social elements of the system now tested. My main point is maybe do a little more observing and less condemning. I find the whole event to be a fascinating test of one of the known vulnerabilities large open source efforts face.
We live in a society, to operate open communities there are trade-offs.
If you want to live in a miserable security state where no action is allowed, refunds are never accepted, and every actor is assumed hostile until proven otherwise, then you can - but it comes at a serious cost.
This doesn't mean people shouldn't consider the security implications of new PRs, but it's better to not act like assholes with the goal being a high-trust society, this leads to a better non-zero-sum outcome for everyone. Banning these people was the right call they don't deserve any thanks.
In some ways their bullshit was worse than a real bad actor actually pursuing some other goal, at least the bad actor has some reason outside of some dumb 'research' article.
The academics abused this good-will towards them.
What did they show here that you can sneak bugs into an open source project? Is that a surprise? Bugs get in even when people are not intentionally trying to get them in.
Of course everyone knows bugs make it in software. That’s not the point and I find it a little concerning that there’s a camp of people who are only interested in the zzz I already knew software had bugs assessment. Yes the academics abused their goodwill. And in doing so they raised awareness around around something that sure many people know is possible. The point is demonstrating the vuln and forcing people to confront reality.
I strive for a high trust society too. Totally agree. And acknowledging that people can exploit trust and use it to push poor code through review does not dismantle a high trust operation or perspective. Trust systems fail when people abuse trust so the reality is that there must be safeguards built in both technically and socially in order to achieve a suitable level of resilience to keep things sustainable.
Just look at TLS, data validation, cryptographic identity, etc. None of this would need to exist in a high trust society. We could just tell people who we are, trust other not to steal our network traffic, never worry about intentionally invalid input. Nobody would overdraft their accounts at the ATM, etc. I find it hard to argue for absolute removal of the verify step from a trust but verify mentality. This incident demonstrated a failure in the verify step for kernel code review. Cool.
This is how security people undermine their own message. My entire job is being "tge trust but verify" stick in the mud, but everyone knows it when I walk in the room. I don't waste peoples time, and I stop short of actually causing damage by educating and forcing an active reckoning with reality.
You can have your verify-lite process, but you must write down that that was your decision, and if appropriate, revisit and reaffirm it over time. You must implement controls, measures and processes in such a way as to minimize the deleterious consequences to your endeavor. It's the entire reason Quality Assurance is a pain in the ass. When you're doing a stellar job, everyone wonders why you're there at all. Nobody counts the problems that didn't happen or that you've managed to corral through culture changes in your favor, but they will jump on whatever you do that drags the group down. Security is the same. You are an anchor by nature, the easiest way to make you go away is to ignore you.
You must help, first and foremost. No points for groups that just add more filth to wallow through.
I agree, but I would say the better result, most likely unachievable now, would be to fix the holes that required a humans feelings to ensure security. Maybe some shift towards that direction could result from this.
I would implore you to maintain the ban, no matter how hard the university tries to make ammends. You sent a very clear message that this type of behavior will not be tolerated, and organizations should take serious measures to prevent malicious activities taking place under their purview. I commend you for that. Thanks for your hard work and diligence.
I'd disagree. Organizations are collections of actors, some of which may have malicious intents. As long as the organization itself does not condone this type of behavior, has mechanisms in place to prevent such behavior, and has actual consequences for malicious actors, then the blame should be placed on the individual, not the organization.
In the case of research, universities are required to have an ethics board that reviews research proposals before actual research is conducted. Conducting research without an approval or misrepresenting the research project to the ethics board are pretty serious offenses.
Typically for research that involves people, participants in the research require having a consent form that is signed by participants, alongside a reminder for participants that they can withdraw that consent at any time without any penalties. It's pretty interesting that in this case, there seemed to have been no real consent required, and it would be interesting to know whether there has been an oversight by the ethics board or a misrepresentation of the research by the researchers.
It will be interesting to see whether the university applies a penalty to the professor (removal of tenure, termination, suspension, etc.) or not. The latter would imply that they're okay with unethical or misrepresented research being associated with their university, which would be pretty surprising.
In any case, it's a good thing that the Linux kernel maintainers decided that experimenting on them isn't acceptable and disrespectful of their contributions. Subjecting participants to experiments without their consent is a severe breach of ethical duty, and I hope that the university will apply the correct sanctions to the researchers and instigators.
Good points. I should have qualified my statement by saying that IMO the ban should stay in place for at least five years. A prison sentence, if you will, for the offense that was committed by their organization. I completely agree with you though that no organization can have absolute control over the humans working for them, especially your point about misrepresenting intentions. However, I believe that by handing out heavy penalties like this, not only will it make organizations think twice before approving questionable research, it will also help prevent malicious researchers from engaging in this type of activity. I don't imagine it's going to look great being the person who got an entire university banned from committing to the Linux kernel.
Of course, in a few years this will all be forgotten. It begs the question... how effective is it to ban entire organizations due to the actions of a few people? Part of me thinks that it would very good to have something like this happen every five years (because it puts the maintainers on guard), but another part of me recognizes that these maintainers are working for free, and they didn't sign up to be gaslighted, they signed up to make the world a better place. It's not an easy problem.
I agree. I don't think any of the kernel developers ever signed up for reviewing malicious patches done by people who managed to sneak their research project past the ethics board, and it's not really fair to them to have to deal with that. I'm pretty sure they have enough work to do already without having to deal with additional nonsense.
I don't think it's unreasonable for maintainers of software to ignore or outright ban problematic users/contributors. It's up to them to manage their software project the way they want, and if banning organizations with malicious actors is the way to do it, the more power to them.
It turns out that the Associate Department Head was engaged in similar "research" on Wikipedia over a dozen years ago, and that also caused problems. The fact that they are here again suggests a broader institutional problem.
Looks like the authors have Chinese names [1]. Should they ban anyone with Chinese names, too, for good measure? Or maybe collective punishment is not such a good idea?
I'm currently wondering how much of these patches could've been flagged in an automated manner, in the sense of fuzzing specific parts that have been modified (and a fuzzer that is memory/binary aware).
Would a project like this be unfeasible due to the sheer amount of commits/day?
That’s not likely to work after a high profile incident like this, in the short term or the long term. Publication is, by design, a de-anonymizing process.
Putting the ethical question of the researcher aside, the fact you want to "properly review them at a later point in time" seems to suggest a lack of confidence in the kernel review process.
Since this researcher is apparently not an established figure in the kernel community, my expectation is the patches have gone through the most rigorous review process. If you think the risk of malicious patches from this person have got in is high, it means that an unknown attacker deliberately concerting complex kernel loop hole would have a even higher chance got patches in.
While I think the researcher's actions are out of line for sure. This "I will ban you and revert all your stuff" retaliation seems emotional overaction.
> This "I will ban you and revert all your stuff" retaliation seems emotional overaction.
Fool me once. Why should they waste their time with extra scrutiny next time? Somebody deliberately misled them, so that's it, banned from the playground. It's just a no-nonsense attitude, without which you'd get nothing done.
If you had a party in your house and some guest you don't know and whom you invited in assuming good faith, turned out to deliberately poop on the rug in your spare guest room while nobody was looking .. next time you have a party, what do you do? Let them in but keep an eye on them? Ask your friends to never let this guest alone? Or just simply to deny entrance, so that you can focus on having fun with people you trust and newcomers who have not shown any malicious intent?
> Why should they waste their time with extra scrutiny next time?
Because well funded malicious actors (government agencies, large corporations, etc) exist and aren't so polite as to use email addresses that conveniently link different individuals from the group together. Such actors don't publicize their results, aren't subject to IRB approval, and their exploits likely don't have such benign end goals.
As far as I'm concerned the University of Minnesota did a public service here by facilitating a mildly sophisticated and ultimately benign attack against the process surrounding an absolutely critical piece of software. We ought to have more such unannounced penetration tests.
we don't have the full communication and I understand that the intention is to be stealthy (why use an university email that can be linked to the previous research then?). However the researcher's response seems to be disingenuous:
> I sent patches on the hopes to get feedback. We are not experts in the Linux kernel and repeatedly making these statements is disgusting to hear.
this is after they're caught, why continue lying instead of apologizing and explain? Is the lying also part of the experiments?
On top of that, they played cards, you can see why people would be triggered by this level of dishonesty:
> I will not be sending any more patches due to the attitude that is not only unwelcome but also intimidating to newbies
From reading other comments about the context surrounding these events, it sounds to me like this probably was an actual newbie who made an honest (if lazy) mistake and was then caught up in the controversy surrounding his advisor's past research.
Or perhaps it really is a second attempt by his advisor at an evil plot to sneak more buggy patches into the kernel for research purposes? Either way, the response by the maintainers seems rather disproportionate to me. And either way, I'm ultimately grateful for the (apparently unwanted?) attention being drawn to the (apparent lack of) security surrounding the Linux kernel patch review process.
They should not have experimented on human subjects without consent, regardless of whether the result is considered benign.
Yes, malicious actors have a head start, because they don't care about the rules. It doesn't mean that we should all kick the rules, and compete with malicious actors on this race to the bottom.
I'm not aware of any law requiring consent in cases such as this, only conventions enforced by IRBs and journal submission requirements.
I also don't view unannounced penetration testing of an open source project as immoral, provided it doesn't consume an inordinate amount of resources or actually result in any breakage (ie it's absolutely essential that such attempts not result in defects making it into production).
When the Matrix servers were (repeatedly) breached and the details published, I viewed it as a Good Thing. Similarly, I view non-consensual and unannounced penetration testing of the Linux kernel as a Good Thing given how widely deployed it is. Frankly I don't care about the sensibilities of you or anyone else - at the end of the day I want my devices to be secure and at this point they are all running Linux.
I don’t see where I claim that this is a legal matter. There are many things which are not prohibited by law that you can do to a fellow human being that are immoral and might result in them blacklisting you forever.
That you care about something or not also seems to be irrelevant, unless you are part of either the research or the kernel maintainers. It’s not about your or my emotional inclination.
Acquiring consent before experimenting in human subject is an ethical requirement for research, regardless of whether is a hurdle for the researchers. There is a reason that IRB exists.
Not to mention that they literally proved nothing, other than that vulnerable patches can be merged into the kernel. But did anybody that such a threat is impossible anyway? The kernel has vulnerabilities and it will continue to have them. We already knew that.
>I view non-consensual and unannounced penetration testing of the Linux kernel as a Good Thing...
So what other things do you think appropriate to not engage in acquiring consent to do based on some perceived justification of ubiquity? It's a slippery slope all the way down, and there is a reason for all the ceremony and hoopla involved in this type of thing. If you cannot demonstrate mastery of doing research on human subjects and processes the right way, and show you've done your footwork to consider the impact of not doing it that way (i.e. IRB fully engaged, you've gone out of your way to make sure they understand, and at least reached out to one person in the group under test to give a surreptitious heads up (like Linus)), you have no business playing it fast and loose, and you absolutely deserve censure.
No points awarded for half-assing. Asking forgiveness may oft times be easier than asking permission, but in many areas, the impact to doing so goes far beyond mere inconvenience to the researcher in the costs it can extract.
>at the end of the day I want my devices to be secure and at this point they are all running Linux.
That is orthogonal to the outcome of the research that was being done, as by definition running Linux would include running with a new vulnerability injected. What you really want is to know your device is doing what you want it to, and none of what you don't. Screwing with kernel developers does precious little to accomplish that. Same logic applies with any other type of bug injection or intentioned software breakage.
> As far as I'm concerned the University of Minnesota did a public service here by facilitating a mildly sophisticated and ultimately benign attack against the process surrounding an absolutely critical piece of software. We ought to have more such unannounced penetration tests.
This "attack" did not reveal anything interesting. It's not like any of this was unknown. Of course you can get backdoors in if you try hard enough. That does not surprise anybody.
Imagine somebody goes with an axe, breaks your garage door, poops on your Harley, leaves, and then calls you and tells you "Oh, btw, it was me. I did you a service by facilitating a mildly sophisticated and ultimately benign attack against the process surrounding an absolutely critical piece of your property. Thank me later." And then they expect you to get let in when you have a party.
It doesn't work that way. Of course the garage door can be broken with an axe. You don't need a "mildly sophisticated attack" to illustrate that while wasting everybody's time.
By keeping the paper, UMN is benefiting (in citations and research result count). Universities are supposed to have processes for punishing unethical research. Unless the University retracts the paper and fires the researcher involved, they have not made amends.
"It was my brother on my unsecured computer" is an excuse I've heard a few times by people trying to shirk responsibility for their ban-worthy actions.
Geographic proximity to bad actors is sometimes enough to get caught in the crossfire. While it might be unfair, it might also be seen as holding a community and it's leadership responsible for failing to hold members of their community responsible and in check with their actions. And, fair or not, it might also be seen as a pragmatic option in the face of limited moderation tools and time. If you have a magic wand to ban only the bad-faith contributions by the students influenced by the professor in question, I imagine the kernel devs will be more than happy to put it to use.
To continue the analogy, it would be like finding out that the offender’s friends knew they were going to do that and were planning on recording the results. Banning all involved parties is reasonable.
"... planning on recording the event to show it on YouTube for ad revenue and Internet fame."
In this case, the offender's friends are benefiting from the research. I think that needs to be made important. The university benefits from this paper being published, or at least expected to. That should not be overlooked.
The fact you want to "properly review them at a later point in time" seems to suggest a lack of confidence in the kernel review process.
Basically, yes. The kernel review process does not catch 100% of intentionally introduced security flaws. It isn't perfect, and I don't think anyone is claiming that it is perfect. Whenever there's an indication that a group has been intentionally introducing security flaws, it is just common sense to go back and put a higher bar on reviewing it for security.
Not all kernel reviewers are being paid by their employer to review patches. Kernel reviews are "free" to the contributor because everyone operates on the assumption that every contributor wants to make Linux better by contributing high-quality patches. In this case, multiple people from the University have decided that reviewers' time isn't valuable (so it's acceptable to waste it) and that the quality of the Kernel isn't important (so it's acceptable to make it worse on purpose). A ban is a completely appropriate response to this, and reverting until you can review all the commits is an appropriate safety measure.
Whether or not this indicates flaws in the review process is a separate issue, but I don't know how you can justify not reverting all the commits. It'd be highly irresponsible to leave them in.
I guess what I am trying to get at is that this researcher's action does have its merit. This event does rise awareness of what sophisticated attacker group might try to do to kernel community. Admitting this would be the first step to hardening the kernel review process to prevent this kind of harm from happening again.
What I strongly disapprove of the researcher is that apparently no steps are taken to prevent real world consequences of malicious patches getting into kernel, I think the researcher should:
- Notify the kernel community promptly once malicious patches got past all review processes.
- Time these actions well such that malicious patches won't not get into a stable branch before they could be reverted.
----------------
Edit: reading the paper provided above, it seems that they did do both actions above. From the paper:
> Ensuring the safety of the experiment. In the experiment, we aim to demonstrate the practicality of stealthily introducing vulnerabilities through hypocrite commits. Our goal is not to introduce vulnerabilities to harm OSS. Therefore, we safely conduct the experiment to make sure that the introduced UAF bugs will not be merged into the actual Linux code. In addition to the minor patches that introduce UAF conditions, we also prepare the correct patches for fixing the minor issues. We send the minor patches to the Linux community through email to seek their feedback. Fortunately, there is a time window between the confirmation of a patch and the merging of the patch. Once a maintainer confirmed our patches, e.g., an email reply indicating “looks good”, we immediately notify the maintainers of the introduced UAF and request them to not go ahead to apply the patch. At the same time, we point out the correct fixing of the bug and provide our correct patch. In all the three cases, maintainers explicitly acknowledged and confirmed to not move forward with the incorrect patches.
All the UAF-introducing patches stayed only in the email exchanges, without even becoming a Git commit in Linux branches. Therefore, we ensured that none of our introduced UAF bugs was ever merged into any branch of the Linux kernel, and none of the Linux users would be affected.
So, unless the kernel maintenance team have another side of the story. The questions of ethics could only go as far as "wasting kernel community's time" rather than creating real world loop holes.
That paper came out a year ago, and they got a lot of negative feedback about it, as you might expect. Now they appear to be doing it again. It’s a different PHD student with the same advisor as last time.
This time two reviewers noticed that the patch was useless, and then Greg stepped in (three weeks later) saying that this was a repetition of the same bad behavior from the first study. This got a response from the author of the patch, who said that this and other statements were “wild accusations that are bordering on slander”.
> Now they appear to be doing it again. It’s a different PHD student with the same advisor as last time.
I'd hate to be the PhD student that wastes away half a dozen years of his/her life writing a document on how to sneak buggy code through a code review.
More than being pointless and boring, it's a total CV black hole. It's the worst of both worlds: zero professional experience to show for, and zero academic portfolio to show for.
Strangers submitting patches to the kernel is completely normal, where throwing people off is not. A better analogy would involve decades of examples of bad actors throwing people off the bridge, then being surprised when someone who appears friendly does it.
Your analogy also isn't the best because it heavily suggests the nefarious behavior is easy to identify (throwing people off a bridge). This is more akin to people helping those in need to cross a street. At first, it is just people helping people. Then, someone comes along and starts to "help" so that they can steal money (introduce vulnerabilities) to the unsuspecting targets. Now, the street-crossing community needs to introduce processes (code review) to look out for these bad actors. Then, someone who works for the city and is wearing the city uniform (University of Minnesota CS department) comes along saying there here to help and the community is a bit more trustful as they have dealt with other city workers before. The city worker then steals from the people in need and then proclaims "Aha, see how easy it is!" No one is surprised and just thinks they are assholes.
Sometimes, complex situations don't have simple analogies. I'm not even sure mine is 100% correct.
While submitting patches is normal submitting malicious patches is abnormal and antisocial. Certainly bad actors will do it, but by that logic these researchers are bad actors.
Just like bumping into somebody on the roof is normal, but you should always be aware that there’s a chance they might try to throw you off. A researcher highlighting this fact by doing it isn’t helping, even if they mitigate their damage.
A much better way to show what they are attempting to is to review historic commits and try to find places where malicious code slipped through, and how the community responded. Or to solicit experimenters to follow normal processes on a fake code base for a few weeks.
> Strangers submitting patches to the kernel is completely normal, where throwing people off is not.
Strangers submitting patches might be completely normal.
Malicious strangers trying to sneak vulnerabilities by submitting malicious patches devised to exploit the code review process is not normal. At all.
There are far more news reports of deranged people throwing strangers under traffic, subways, and trains, than there are reports of malicious actors trying to sneak vulnerable patches.
> Malicious strangers trying to sneak vulnerabilities by submitting malicious patches devised to exploit the code review process is not normal.
How could you possibly know that? In fact, I would suggest that you are completely and obviously wrong. Government intelligence agencies exist (among other things) and presumably engage in such behavior constantly. The reward for succeeding is far too high to assume that no one is trying.
We damaged the brake cables mechanics were installing into people's cars to find out if they were really inspecting them properly prior to installation!
To add... Ideally, they should have looped in Linus, or someone high-up in the chain of maintainers before running an experiment like this. Their actions might have been in good faith, but the approach they undertook (including the email claiming slander) is seriously irresponsible and a sure shot way to wreck relations.
> This event does rise awareness of what sophisticated attacker group might try to do to kernel community.
The limits of code review are quite well known, so it appears very questionable what scientific knowledge is actually gained here. (Indeed, especially because of the known limits, you could very likely show them without misleading people, because even people knowing to be suspicious are likely to miss problems, if you really wanted to run a formal study on some specific aspect. You could also study the history of in-the-wild bugs to learn about the review process)
That's factually incorrect. The arguments over what constitutes proper code reviews continues to this day with few comprehensive studies about syntax, much less code reviews - not "do you have them" or "how many people" but methodology.
> it appears very questionable what scientific knowledge is actually gained here
The knowledge isn't from the study existing, but the analysis of the data collected.
I'm not even sure why people are upset at this, since it's a very modern approach to investigating how many projects are structured to this day. This was a daring and practical effort.
> The questions of ethics could only go as far as "wasting kernel community's time" rather than creating real world loop holes.
Under that logic, it's ok for me to run a pen test against your computers, right? ...because I'm only wasting your time.... Or maybe to hack your bank account, but return the money before you notice.
Ethics aside, warning someone that a targeted penetration test is coming will change their behavior.
> Under that logic, it's ok for me to run a pen test against your computers, right?
I think the standard for an individual user should be different than that for the organization who is, in the end, responsible for the security of millions of those individual users. One annoys one person, one prevents millions from being annoyed.
> Ethics aside, warning someone that a targeted penetration test is coming will change their behavior.
They could discuss the idea and then perform the test months later? With the amount of patches that had to be reverted as precaution the test would have been well hidden in the usual workload even if the maintainers knew that someone at some point in the past mentioned the possibility of a pen test. How long can the average human stay vigilant if you tell them they will be robbed some day this year?
I think the question may have been rhetorical, and the intended answer the opposite of yours: No, it doesn't pose a question, since it obviously shouldn't be done.
I wouldn't put it past them to have a second unpublished paper, for the "we didn't get caught" timeline.
It would give the University some notoriety to be able to claim "We introduced vulnerabilities in Linux". It would put them in good terms with possible propietary software sponsors, and the military.
> the fact you want to "properly review them at a later point in time" seems to suggest a lack of confidence in the kernel review process.
I don't think this necessarily follows. Rather it is fundamentally a resource allocation issue.
The kernel team obviously doesn't have sufficient resources to conclusively verify that every patch is bug-free, particularly if the bugs are intentionally obfuscated. Instead it's a more nebulous standard of "reasonable assurance", where "reasonable" is a variable function of what must be sacrificed to perform a more thorough review, how critical the patch appears at first impression, and information relating to provenance of the patch.
By assimilating new information about the provenance of the patch (that it's coming from a group of people known to add obfuscated bugs), that standard rises, as it should.
Alternatively stated, there is some desired probability that an approved patch is bug-free (or at least free of any bugs that would threaten security). Presumably, the review process applied to a patch from an anonymous source (meaning the process you are implying suffers from a lack of confidence) is sufficient such that the Bayesian prior for a hypothetical "average anonymous" reviewed patch reaches the desired probability. But the provenance raises the likelihood that the source is malicious, which drops the probability such that the typical review for an untrusted source is not sufficient, and so a "proper review" is warranted.
> it means that an unknown attacker deliberately concerting complex kernel loop hole would have a even higher chance got patches in.
That's hard to argue with, and ironically the point of the research at issue. It does imply that there's a need for some kind of "trust network" or interpersonal vetting to take the load off of code review.
> The kernel team obviously doesn't have sufficient resources to conclusively verify that every patch is bug-free, particularly if the bugs are intentionally obfuscated.
In a perfect world, I would agree that the work of a researcher who's not an established figure in the kernel community would be met with a relatively high level of scrutiny in review.
But realistically, when you find out a submitter had malicious intent, I think it's 100% correct to revisit any and all associated submissions since it's quite a different thing to inspect code for correctness, style, etc. as you would in a typical code review process versus trying to find some intentionally obfuscated security hole.
And, frankly, who has time to pick the good from the bad in a case like this? I don't think it's an overreaction at all. IMO, it's a simplification to assume that all associated contributions may be tainted.
Why? Linux is not the state. There is no entitlement to rights or presumption of innocence.
Linux is set up to benefit the linux development community. If UMinn has basically no positive contributions, a bunch of neutral ones and some negative ones banning seems the right call.
Its not about fairness, its about if the hurts outweigh the benefits.
Not only that, good faith actors who are associated with UMN can still contribute, just not in their official capacity as UMN associates (staff, students, researchers, etc).
> Since this researcher is apparently not an established figure in the kernel community, my expectation is the patches have gone through the most rigorous review process
I think the best way to make this expectation reality is putting in the work. The second best way is paying. Doing neither and holding the expectation is a way to exist certainly, but has no impact on the outcome.
I mean, it's the linux kernel. Think about what it's powering and how much risk there is involved with these patches. Review processes obviously aren't perfect, but usually patches aren't constructed to sneak sketchy code though. You'd usually approach a review in good faith.
Given that some patches may have made it though with holes, you pull them and re-approach them with a different mindset.
Doesn't this basically prove the original point that if someone or an organization wished to compromise linux, they could do so with crafted bugs in patches?
This might not be on purpose. If you look at their article they're studying how to introduce bugs that are hard to detect not ones that are easy to detect.
Well, you or whoever was the responsible maintainer completely failed in reviewing these patches, which is your whole job as a maintainer.
Just reverting those patches (which may well be correct) makes no sense, you and/or other maintainers need to properly review them after your previous abject failure at doing so, and properly determine whether they are correct or not, and if they aren't how they got merged anyway and how you will stop this happening again.
Or I suppose step down as maintainers, which may be appropriate after a fiasco of this magnitude.
On the contrary, it would be the easy, lazy way out for a maintainer to say “well this incident was a shame now let’s forget about it.” The extra work the kernel devs are putting in here should be commended.
In general, it is the wrong attitude to say, oh we had a security problem. What a fiasco! Everyone involved should be fired! With a culture like that, all you guarantee is that people cover up the security issues that inevitably occur.
Perhaps this incident actually does indicate that kernel code review procedures should be changed in some way. I don’t know, I’m not a kernel expert. But the right way to do that is with a calm postmortem after appropriate immediate actions are taken. Rolling back changes made by malicious actors is a very reasonable immediate action to take. After emotions have cooled, then it’s the right time to figure out if any processes should be changed in the future. And kernel devs putting in extra work to handle security incidents should be appreciated, not criticized for their imperfection.
Greg explicitly stated "Because of this, all submissions from this group must be reverted from the kernel tree and will need to be re-reviewed again to determine if they actually are a valid fix....I will be working with some other kernel developers to determine if any of these reverts were actually valid changes, were actually valid, and if so, will resubmit them properly later. For now, it's better to be safe."
If the IRB is any good the professor doesn't get that. Universities are publish or perish, and the IRB should force the withdrawal of all papers they submitted. This is might be enough to fire the professor with cause - including remove any tenure protection they might have - which means they get a bad reference.
I hope we hear from the IRB in about a year stating exactly what happened. Real investigations of bad conduct should take time to complete correctly and I want them to do their job correctly so I'll give them that time. (there is the possibility that these are good faith patches and someone in the linux community just hates this person - seems unlikely but until a proper independent investigation is done I'll leave that open.)
> We send the emails to the Linux communityand seek their feedback. The experiment is not to blame any maintainers but to reveal issues in the process. The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research. We obtained a formal IRB-exempt letter. The experiment will not collect any personal data, individual behaviors, or personal opinions. It is limited to studying the patching process OSS communities follow, instead of individuals.
> The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research.
I'm not sure how it affects things, but I think it's important to clarify that they did not obtain the IRB-exempt letter in advance of doing the research, but after the ethically questionable actions had already been taken:
The IRB of UMN reviewed the study and determined that this is not human research (a formal IRB exempt letter was obtained). Throughout the study, we honestly did not think this is human research, so we did not apply for an IRB approval in the beginning. ... We would like to thank the people who suggested us to talk to IRB after seeing the paper abstract.
I'm a bit shocked that the IRB gave an exemption letter - are they hoping that the kernel maintainers won't take the (very reasonable) step towards legal action?
I'd guess they may not have understood what actually happened, or were leaning heavily on the IEEE reviewers having no issues with the paper, as at that point it'd already been excepted.
> We send the emails to the Linux communityand seek their feedback.
That's not really what they did.
They sent the patches, the patches where either merged or rejected.
And they never let anybody knew that they had introduced security vulnerabilities on the kernel on purpose until they got caught and people started reverting all the patches from their university and banned the whole university.
> (4). Once any maintainer of the community responds to the email, indicating “looks good”, we immediately point out the introduced bug and request them to not go ahead to apply the patch. At the same time, we point out the correct fixing of the bug and provide our proper patch. In all the three cases, maintainers explicitly acknowledged and confirmed to not move forward with the incorrect patches. This way, we ensure that the incorrect patches will not be adopted or committed into the Git tree of Linux.
It'd be great if they pointed to those "please don't merge" messages on the mailing list or anywhere.
Seems like there are some patches already on stable trees [1], so they're either lying, or they didn't care if those "don't merge" messages made anybody react to them.
The paper doesn't cite specific commits used. It's possible that any of the commits in stable are actually good commits and not part of the experiment. I support the ban/revert, I'm just pointing out there's a 3rd option you didn't touch on.
We have 4 people, with the students Quishu Wu and Aditya Pakki intruducing the faulty patches, and the 2 others, Prof Kangjie Lu and Ass.Prof Wengwen Wang patching vulnerabilities in the same area. Banning the leader seems ok to me, even if he produced some good fixes and SW to detect it. The only question is Wang who is now in Georgia, and was never caught. Maybe he left Lu at umn because of his questionable ethics.
Also, they are talking of three cases. However, the list of patches to be reverted by gregkh is far longer than three, more than a hundred. Most of the first batch look sufficiently similar that I would guess all of them are part of this "research". So the difference in numbers alone points to them most probably lying.
I was more ambivalent about their "research" until I read that "clarification." It's weaselly bullshit.
>> The work taints the relationship between academia and industry
> We are very sorry to hear this concern. This is really not what we expected, and we strongly believe it is caused by misunderstandings
Yeah, misunderstandings by the university that anyone, ever, in any line of endeavor would be happy to be purposely fucked with as long as the perpetrator eventually claims it's for a good cause. In this case the cause isn't even good, they're proving the jaw-droppingly obvious.
The first step of an apology is admitting the misdeed. Here they are explicitly not acknowledging that what they did was wrong, they are still asserting that this was a misunderstanding.
Even their choice of wording ("We are very sorry to hear this concern.") is the blend of word fuckery that conveys the idea they care nothing about what they did or why it negatively affected others.
..."Because if we're lucky tomorrow, we won't have to deal with questions like yours ever again." --Firesign Theater, "I Think We're All Bozos on the Bus"
Yet we do nothing about it? I wouldn't call that jaw-droppingly obvious, if anything, without this, I'm pretty sure that anyone would argue that it would be caught way before making it way into stable.
I've literally never come across an open source project that was thought to have a bullet proof review process or had a lack of people making criticisms.
What they do almost universally lack is enough people making positive contributions (in time, money, or both).
This "research" falls squarely into the former category and burns resources that could have been spent on the latter.
This is zero percent different from a bad actor and hopefully criminal. I think a lot of maintainers work for large corporations like Microsoft, Oracle, Ubuntu, Red Hat, etc... I think these guys really stepped in it.
> And they never let anybody knew that they had introduced security vulnerabilities on the kernel on purpose...
Yes, that's the whole point! The real malicious actors aren't going to notify anyone that they're injecting vulnerabilities either. They may be plants at reputable companies, and they'll make it look like an "honest mistake".
Had this not been caught, it would've exposed a major flaw in the process.
> ...until they got caught and people started reverting all the patches from their university and banned the whole university.
Either these patches are valid fixes, in which case they should remain, or they are intentional vulnerabilities, in which case they should've already been reviewed and rejected.
Reverting and reviewing them "at a later date" just makes me question the process. If they haven't been reviewed properly yet, it's better to do it now instead of messing around with reverts.
This reminds me of that story about Go Daddy sending everyone "training phishing emails" announcing that they had received a company bonus - with the explanation that this is ok because it is a realistic pretext that real phishing may use.
While true, it's simply not acceptable to abuse trust in this way. It causes real emotional harm to real humans, and while it also may produce some benefits, those do not outweigh the harms. Just because malicious actors don't care about the harms shouldn't mean that ethical people shouldn't either.
This isn't some employer-employee trust relationship. The whole point of the test is that you can't trust a patch just because it's from some university or some major company.
The vast majority of patches are not malicious. Sending a malicious patch (one that is known to introduce a vulnerability) is a malicious action. Sending a buggy patch that creates a vulnerability by accident is not a malicious action.
Given the completely unavoidable limitations of the review and bug testing process, a maintainer has to react very differently when they have determined that a patch is malicious - all previous patches past from that same source (person or even organization) have to be either re-reviewed at a much higher standard or reverted indiscriminately; and any future patches have to be rejected outright.
This puts a heavy burden on a maintainer, so intentionally creating this type of burden is a malicious action regardless of intent. Especially given that the intent was useless in the first place - everyone knows that patches can introduce vulnerabilities, either maliciously or by accident.
The vast majority of drunk drivers never kill anyone.
> Sending a malicious patch (one that is known to introduce a vulnerability) is a malicious action.
I disagree that it's malicious in this context, but that's irrelevant really. If the patch gets through, then that proves one of the most critical pieces of software could relatively easily be infiltrated by a malicious actor, which means the review process is broken. That's what we're trying to figure out here, and there's no better way to do it than replicate the same conditions under which such patches would ordinarily be reviewed.
> Especially given that the intent was useless in the first place - everyone knows that patches can introduce vulnerabilities, either maliciously or by accident.
Yes, everyone knows that patches can introduce vulnerabilities if they are not found. We want to know whether they are found! If they are not found, we need to figure out how they slipped by and how to prevent that from happening in the future.
> If the patch gets through, then that proves one of the most critical pieces of software could relatively easily be infiltrated by a malicious actor, which means the review process is broken.
That is a complete misunderstanding of the Linux dev process. No one expects the first reviewer of a patch (the person that the researchers were experimenting on) to catch any bug. The dev process has many safeguards - several reviewers, testing, static analysis tools, security research, distribution testing, beta testers, early adopters - that are expected to catch bugs in the kernel at various stages.
Trying to deceive early reviewers into accepting malicious patches for research purposes is both useless research and hurtful to the developers.
Open source products rely on trust. There is no way to build a trust-less open source product. Of course, the old mantra of trust, but verify is very important as well.
But the linux kernel is NOT a security product - it is a kernel. It can be used in entirely disconnected devices that couldn't care less about security, as well as in highly secure infrastructure that powers the world. The ultimate responsibility of delivering a secure product based on Linux lies with the people delivering a secure product based on the kernel. The kernel is a essentially library, not a product. If someone is assuming they can build a secure product by trusting Linux to be "secure" than they are simply wrong, and no amount of change in the Linux dev process will fix their assumption.
Of course, you want the kernel to be as secure as possible, but you also want many other things from the kernel as well (it should be featureful, it should be backwards compatible with userspace, it should run on as many architectures as needed, it should be fast and efficient, it should be easy to read and maintain etc).
> Yes, that's the whole point! The real malicious actors aren't going to notify anyone that they're injecting vulnerabilities either. They may be plants at reputable companies, and they'll make it look like an "honest mistake".
This just turns the researchers into black hats. They are just making it look like "a research paper."
Not sure why you are so obsessed with this. Yes this process does involve humans, but the process has aspects can be examined as independent of humans.
This study does not care about the reviewers, it cares about the process. For example, you can certainly improve the process without replacing any reviewers. It is just blatantly false to claim the process is all about humans.
Another example, the review process can even be totally conducted by AIs. See? The process is not all about humans, or human behavior.
To make this even more understandable, considering the process of building a LEGO, you need human to build a LEGO, but you can examine the process of building the LEGO without examine the humans who build the LEGO.
This study does not care about the reviewers, it cares about the process. For example, you can certainly improve the process without replacing any reviewers. It is just blatantly false to claim the process is all about humans.
This was all about the reaction of humans. They sent in text with a deceptive description and tried to get a positive answer even though the text was not wholly what was described. It was a psych study in an uncontrolled environment with people who did not know they were participating in a study.
No. This is not all about the reaction of humans. This is not a psych study. I have explained this clearly in previous comments. If you believe the process of doing something is all about humans, I have nothing to add.
People are obsessed because you're trying to excuse the researchers behavior as ethical.
"Process" in this case is just another word for people because ultimately, the process being evaluated here is the human interaction with the malicious code being submitted.
Put another way, let's just take out the human reviewer, pretend the maintainers didn't exist. Does the patch get reviewed? No. Does the patch get merged into a stable branch? No. Does the patch get evaluated at all? No. The whole research paper breaks down and becomes worthless if you remove the human factor. The human reviewer is _necessary_ for this research, so this research should be deemed as having human participants.
You did. You just won't accept it because you don't want to. Every time you try to draw the focus of the conversation to "it's a process study" you're trying to diminish the severity of what the researchers did here.
How was this study conducted? For every patch that the researchers sent, what process did it go through?
The answer is, it was reviewed and accepted by a human. That's it. Full stop. There's your human subject right there in the middle of your research work. It's not possible to conduct this research without that human subject interacting with your research materials. You do not get to discount that human participation because "Oh well we COULD replace them with an AI in the future". Well your study didn't, which means it needs to go through the human subjects review process.
When you claim that this study was about a process, you're literally taking the researchers side. That's what they've been insisting on as the reason why this study is ethical and they did not need to inform or obtain consent from the kernel development team. That's the excuse they used to get out of IRB's review process so they can be considered "not a human subjects research". That's the excuse they needed so they can proceed without having to get a signed consent form. They did all of this so they could conduct a penetration test without the organization they were attacking knowing about it.
You don't seem to be able to comprehend why or how the maintainers feel deceived here, or that their feelings are legitimate. If you did, you wouldn't keep banging on about "oh this is just a process study, the people don't matter, it's all isolated from humans". Funny enough, the people who DID interact with this research DID feel they mattered and DID feel deceived. The whole point of IRB was to prevent exactly this; researchers conducting unethical research which would only come to light after the study concluded and the injured parties complained (and deceit IS a form of harm). For research which is supposed to be isolated from humans and thus didn't see the need in obtaining a signed consent form, that's not really the outcome you expect to see if everything was on the up and up. Another form of harm from this study, the maintainers now have to go over everything they submitted again to ensure there's nothing else to be worried about. That's a lot of wasted man hours and definitely constitutes harm as well. All of University of Minnesota now has less access to the project after getting banned, even more collateral damage and harm caused to their own institution.
Let's be honest. If the researchers were able to sneak their code into a stable, or distribution version of the kernel, they'd be praising themselves to high heaven. Look at how significant our results were, we fucked up all of Linux! Only reason they didn't is because at least they can recognize that would be going a step too far. They're just looking for excuses to not get punished at this point. Same with the IRB. The IRB is now trying to wiggle out of the situation by insisting everything is ok. The IRB is also made up of professors who have a reputation to maintain! They know they let something through that should never have been approved in it's current form. Most human subject research NEVER get this kind of blowback and the fact that this one did means they screwed up and they know it.
No ethics review board considers a multi page, multi forum, lengthy discussion on the ethics of a study they approved as a good sign. Honestly, any study that gets this much attention would be considered a huge success in any other situation.
"The answer is, it was reviewed and accepted by a human. That's it. Full stop. There's your human subject right there in the middle of your research work. "
Thats not the correct or relevant criteria. If you were correct, testing airport security and testing AntiMoney Laundering checks at a bank would amount to human experiments. In fact its hard to think of any study of the real world that would not become a "human experiment".
"When you claim that this study was about a process, you're literally taking the researchers side."
Thats some seriously screwed up logic right here.
"Weinstein was a Nazi and a serial killer, if you disagree with me you are taking his side"
Um, academics aren't allowed to assemble bombs and then try and sneak them onto planes with the excuse that it's not a human trial. That'd be absurd.
It's easy to think of studies that don't involve humans so that statement is just wilful obfuscation. Physics, chemistry, heck lots of biology, and of course computer science are primarily made of studies on objects rather than people. Of those that are done on people they are almost always done on people who know they are the subject of an experiment. Very few studies are like this one.
I am sorry, you arument is all over the place. What on earth are you arguing? That human trial does not excuse what would otherwise be a crime? That airport security is not tested with real bombs? That every study outside of natural sciences is a human expriment?
Studies of airport security are done all the time, thats how we know its terribly ineffective. The staff of the airport are not told about them, they are not human experiments.
The experiments on people have a spesific definition that goes beyond "a human is present"
Airport security staff consent to this type of testing at hiring time so testing can be random, and not just anyone can try to sneak a weapon through security to see if it's caught as "a test".
Perhaps a similar approach that allows randomness with some sort of agreement with the maintainers could have prevented this issue while preserving the integrity of the study.
Unfortunately “no” doesn’t constitute a rebuttal, and the responding commenter makes many valid points.
It is self-evident that this study tangibly involved people in the scope, those people did not provide consent prior, and now openly state their grievances. It is nothing short of arguing in bad faith to claim otherwise.
Repeating the same thing over and over does not make it a fact.
Maybe the stated aim of the research was to study the process. But what they actually did was study how the people involved implemented it.
Being publicly manipulated into merging buggy patches, and wasting hours of people's time are two pretty obvious effects this study had that could cause some amount of distress and thus it cannot be dismissed as simply "studying the process".
This is exactly what I would have said: this sort of research isn't 'human subjects research' and therefore is not covered by an IRB (whose job it is to prevent the university from legal risk, not to identify ethically dubious studies).
It is likely the professor involved here will be fired if they are pre-tenure, or sanctioned if post-tensure.
How in the world is conducting behavioral research on kernel maintainers to see how they respond to subtly-malicious patches not "human subject research"?
Of course, there are other ethical and legal requirements that you're bound to, not just this one. I'm not sure which requirements IRBs in the US look into though, it's a pretty murky situation.
It seems to qualify per §46.102(e)(1)(i) ("Human subject means a living individual about whom an investigator [..] conducting research: (i) Obtains information [...] through [...] interaction with the individual, and uses, studies, or analyzes the information [...]")
I don't think it'd qualify for any of the exemptions in 46.104(d): 1 requires an educational setting, 2 requires standard tests, 3 requires pre-consent and interactions must be "benign", 4 is only about the use of PII with no interactions, 5 is only about public programs, 6 is only about food, 7 is about storing PII and not applicable and 8 requires "broad" pre-consent and documentation of a waiver.
rather than arguing about the technical details of the law, let me just clarify: IRBs would actively reject a request to review this. It's not in their (perceived) purview.
It's not worth arguing about this; if you care, you can try to change the law. In the meantime, IRBs will do what IRBs do.
If the law, as written, does actually classify this as human research, it seems like the correct response is to sue the University for damages under that law.
Since IRBs exist to minimize liability, it seems like that would be that fastest route towards change (assuming you have legal standing )
Woah woah woah, no need to whip out the litigation here. You could try that, but I am fairly certain you would be unsuccessful. You would be thrown out with "this does not qualify under the law" before it made it to court and it wouldn't have much bearing except to bolster the university.
It obviously qualifies and the guy just quoted the law at you to prove it.
Frankly universities and academics need to be taken to court far more often. Our society routinely turns a blind eye to all sorts of fraudulent and unethical practices inside academia and it has to stop.
I had a look at section §46.104 https://www.hhs.gov/ohrp/regulations-and-policy/regulations/... since it mentioned exemptions, and at (d) (3) inside that. It still doesn't apply: there's no agreement to participate, it's not benign, it's not anonymous.
If there's some deeply legalistic answer explaining how the IRB correctly interpreted their rules to arrive at the exemption decision, I believe it. It'll just go to show the rules are broken.
IRBs are like the TSA. Imposing annoyance and red tape on the honest vast-majority while failing to actually filter the 0.0001% of things they ostensibly exist to filter.
are you expecting that science and institutions are rational? If I was on the IRB, I wouldn't have considered this since it's not a sociological experiment on kernel maintainers, it's an experiment to inject vulnerabilities in a source code. That's not what IRBs are qualified to evaluate.
> it's an experiment to inject vulnerabilities in a source code
I'm guessing it passed for similar reasoning, along with the reviewers being unfamiliar with how "vulnerabilities are injected." To get the bad code in, the researcher needed to have the code reviewed by a human.
So if you rephrase "inject vulnerability" as "sneak my way past a human checkpoint", you might have a better idea of what they were actually doing, and might be better equipped to judge its ethical merit -- and if it qualifies as research on human subjects.
To my thinking, it is quite clearly human experimentation, even if the subject is the process rather than a human individual. Ultimately, the process must be performed by a human, and it doesn't make sense to me that you would distinguish between the two.
And the maintainers themselves express feeling that they were the subject of the research, so there's that.
Testing airport security by putting dangerous goods in your luggage is not human experimentation. Testing a Bank's security is not human experimentation. Testing border securiry is not.
What makes people revieing linux kernel more 'human' than any of the above?
It's not an experiment in computer science; these guys aren't typing code into an editor and testing what the code does after they've compiled it. They're contributing their vulnerabilities to a community of developers and testing whether these people accept it. It is absolutely nothing else than a sociological experiment.
This reminds me of a few passages in the SSC post on IRBs[0].
Main point is that IRBs were created in response to some highly unethical and harmful "studies" being carried out by institutions thought of as top-tier. Now they are considered to be a mandatory part of carrying out ethical research. But if you think about it, isn't outsourcing all sense of ethics to an organization external to the actual researchers kind of the opposite of what we want to do?
All institutions tend to be corruptible. Many tend to respond to their actual incentives rather than high-minded statements about what they're supposed to be about. Seems to me that promoting the attitude of "well an IRB approved it, so it must be all right, let's go!" is the exact opposite of what we really want.
All things considered, it's probably better to have something there than nothing. But you still have to be responsible for your own decisions. I bamboozled our lazy IRB into approving our study, so I'm not responsible for it being obviously a bad idea, just isn't good enough.
If you think about it, it's actually kind of meta to the code review process they were "studying". Just like IRBs, Code review is good, but no code review process will ever be good enough to stop every malicious actor every time. It will always be necessary to track the reputation of contributors and be able to mass-revert contributions from contributors later determined to be actively malicious.
I guess I have a different perspective. I know a fair number of world class scientists; like, the sort of people you end up reading about as having changed the textbook. One of these people, a well-known bacteriologist, brought his intended study to the IRB for his institution (UC Boulder), who said he couldn't do it because of various risks due to studying pathogenic bacteria. The bacteriologist, who knew far more about the science than the IRB, explained everything in extreme detail and batted away each attempt to shut him down.
Eventually, the IRB, unhappy at his behavior, said he couldn't do the experiment. He left for another institution (UC San Diego) immediately, having made a deal with the new dean to go through expedited review. It was a big loss for Boulder and TBH, the IRB's reasoning was not sound.
They weren't studying the community, they were studying the patching process used by that community, which a normal IRB would and should consider to be research on a process and therefore not human Research. That's how they presented it to the IRB so it got passed even if what they were claiming was clearly bullshit.
This research had the potential to cause harm to people despite not being human research and was therefore ethically questionable at best. Because they presented the research as not posing potential harm to real people that means they lied to the IRB, which is grounds for dismissal and potential discreditation of all participants (their post-graduate degrees could be revoked by their original school or simply treated as invalid by the educational community at large). Discreditation is unlikely, but loss of tenure for something like this is not out of the question, which would effectively end the professor's career anyway.
At a minimum, is needlessly increasing the workload of an unwitting third party considered a harm? I ask, because I’d be pretty fucking mad if someone came along and added potentially hundreds of man-hours of work in the form of code review to my life.
Considering that the number of patches submitted was quite limited I don't think the original research paper would qualify as a DoS attack. The workload imposed by the original research appears to have been negligible compared to the kernel effort as a whole, no more than any drive by patch submission might result in. So no, I wouldn't personally view that as harmful.
As to the backdated review now being undertaken, as far as I'm concerned that decision is squarely on the maintainers. (Honestly it comes across as an emotional outburst to me.)
Wasting time is nor considered stealing. If it were, there is a long queue to collect money from: all the add agencies, telephone menues where you have to go 10 level deep before you speak to a person, anyone who is bothering people on the street with questions, anyone making your possessions dirty would be a criminal. Anyone going on a date that doesnt work out would be a criminal.
Sure, but I'm still going to be pretty annoyed with you. And if you've wasted my time by messing with a system or process under my control then I'm probably going to block you from that system or process.
As a really prosaic example, I've blocked dozens - if not hundreds - of recruiter email addresses on my work email account.
In my experience in university research, the correct portrayal of the ethical impact is the burden of the researchers unfortunately, and the most plausible explanation in my view given their lack of documentation of the request for IRB exemption would be that they misconstrued the impact of the research.
It seems very possible to me that an IRB wouldn't have accepted their proposed methodology if they hadn't received an exemption.
> The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research. We obtained a formal IRB-exempt letter.
Is there anyone on hand who could explain how what looks very much like a social engineering attack is not "human research"?
This is, at the very least, worth an investigation from an ethics committee.
First of all, this is completely irresponsible, what if the patches would've made their way into a real-life device? The paper does mention a process through which they tried to ensure that doesn't happen, but it's pretty finicky. It's one missed email or one bad timezone mismatch away from releasing the kraken.
Then playing the slander victim card is outright stupid, it hurts the credibility of actual victims.
The mandate of IRBs in the US is pretty weird but the debate about whether this was "human subject research" or not is silly, there are many other ethical and legal requirements to academic research besides Title 45.
> there are many other ethical and legal requirements to academic research besides Title 45.
Right. It's not just human subjects research. IRBs vet all kinds of research: polling, surveys, animal subjects research, genetics/embryo research (potentially even if not human/mammal), anything which could be remotely interpreted as ethically marginal.
If we took the case into the real world and it became "we decided to research how many supports we could remove from this major road bridge before someone noticed", I'd hope the IRB wouldn't just write it off as "not human research so we don't care".
I agree. I personally don't care if it meets the official definition of human subject research. It was unethical, regardless of whether it met the definition or not. I think the ban is appropriate and wouldn't lose any sleep if the ban also enacted by other open-source projects and communities.
It's a real shame because the university probably has good, experienced people who could contribute to various OSS projects. But how can you trust any of them when the next guy might also be running an IRB exempt security study.
There are a lot of people who in fact do consider “research” that comes out of social media companies to be both ethically and, in many cases, procedurally tainted, and thus unusable and unpublishable as-is.
>It's one missed email or one bad timezone mismatch away from releasing the kraken.
I don't think code commits to the Linux kernel make it to live systems that fast?
I do agree with the sentiment, though. It's grossly irresponsible to do that without asking at least someone in the kernel developer's group. People don't dig being used as lab rats, and now the whole uni is blocked. Well, tough shit.
No, but they're very high-traffic and if the "this was a deliberately bad patch" message is sent off-list, only to the maintainer, things can go south pretty easily. Off-list messages are easy to miss on inboxes whose email is in MAINTAINERS and receive a lot of spam, you can email someone right as they're going on vacation and so on. That's one of the reasons why a lot of development happens on a mailing list.
> I hope we hear from the IRB in about a year stating exactly what happened. Real investigations of bad conduct should take time to complete correctly and I want them to do their job correctly so I'll give them that time
That'd be great, yup. And the linux kernel team should then strongly consider undoing the blanket ban, but not until this investigation occurs.
Interestingly, if all that happens, that _would_ be an intriguing data point in research on how FOSS teams deal with malicious intent, heh.
We presented students with an education protocol designed to make a blind subset of them fail tests. Then measured if they failed the test to see if they independently learned the true meaning of the information.
Under any sane IRB you would need consent of the students. This is failure on so many levels.
I'm really not sure what the motive to lie is. You got caught with your hand in the cookie jar, time to explain what happened before they continue to treat you like a common criminal. Doing a pentest and refusing to state it was a pentest is mind boggling.
Has anyone from the "research" team commented and confirmed this was even them or a part of their research? It seems like the only defense is from people who did google-fu for a potentially outdated paper. At this point we can't even be sure if this isn't a genuinely malicious actor using comprimised credentials to introduce vulnerabilities.
It's also not a pen test. Pen testing is explicitly authorized, where you play the role as an attacker, with consent from your victim, in order to report security issues to your victim. This is just straight-up malicious behavior, where the "researchers" play the role as an attacker, without consent from their victim, for personal gain (in this case, publishing a paper).
Because of the nature of the research an argument can be made that it was like a bug bounty (not defending them just putting my argument) but they should have come clean when the patched was merged and told the community about the research or at least submitted the right patch.
Intentionally having bugs in kernel only you know about is very bad.
Hearing how you phrased it reminds me of a study that showed how parachutes do not in fact save lives (the study was more to show the consequences of extrapolating data, so the result should not be taken seriously):
Conclusions: As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials.Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.
With the footnote: Contributors: GCSS had the original idea. JPP tried to talk him out of it. JPP did the first literature search but GCSS lost it. GCSS drafted the manuscript but JPP deleted all the best jokes. GCSS is the guarantor, and JPP says it serves him right
I liked this bit, from the footnotes: "Contributors: RWY had the original idea but was reluctant to say it out loud for years. In a moment of weakness, he shared it with MWY and BKN, both of whom immediately recognized this as the best idea RWY will ever have."
"Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon: An Argument For Proper Multiple Comparisons Correction" (2010, in Journal of Serendipitous and Unexpected Results)
We had the good fortune to have discussion of the study with comments from the author a few years back:
Well part of the experiment is to see how deliberate malicious commits are handled. Banning is the result. They got what they wanted. Play stupid game. Win stupid pri[z]e.
"The University of Minnesota Department of Computer Science & Engineering takes this situation extremely seriously. We have immediately suspended this line of research."
But this raises an obvious question: Doesn't Linux need better protection against someone intentionally introducing security vulnerabilities? If we have learned anything from the SolarWinds hack, it is that if there is a way to introduce a vulnerability then someone will do it, sooner or later. And they won't publish a paper about it, so that shouldn't be the only way to detect it!
So, it turns out that sometimes programmers introduce bugs into software. Sometimes intentionally, but much more commonly accidentally.
If you've got a suggestion of a way to catch those bugs, please be more specific about it. Just telling people that they need "better protection" isn't really useful or actionable advice, or anything that they weren't already aware of.
That question has been obvious for quite some time. It is always possible to introduce subtle vulnerabilities. Research has tried for decades to come up with a solution, to no real avail.
Does nobody here even understand what actually happened?
Really losing my faith in the accuracy of HN if such a huge thread is full of misinformation.
Basically (as I understand it, feel free to correct me) this is what happened:
Researcher emailed maintained with flawed code, maintainer LGTMed it, researcher told maintainer that the code is buggy and not to merge it. The researchers confirmed that the code was not merged or commited anywhere. Paper gets published. Nothing of note happens.
Now, one of the researchers grad students has submitted stuff to linux oh his own volition- he does not appear to be associated with the previous research. These commits are "obviously bad" according to linux maintainers and claim that the grad student is just continuing the "merge bad shit" research. These commits do not appear to be intentionally flawed but rather newbie mistakes (so claims the student)- which is why he feels the linux community is unwelcoming to newcomers.
Now how on earth did that warp to whatever everyone here is smoking?
You too have missed some of the details, but then so have many others.
The paper you’re referring to was from last year. Two of the three patches that they emailed in under fake author names were rejected; they wrote a paper about the experience. All that happened as a result was that everybody told them that it was a terrible idea, and they tweaked the wording of the paper a bit.
Now _this_ year, a different PHD student with the same advisor posted a really dubious patch which would introduce one or more use–after–free bugs. This patch was also rejected by the maintainers. Greg noticed that it looks like another attempt to do the same kind of experiment again. Nobody but them know if that’s true or not, but the student reacted by calling it “slander”, which was not very advisable.
The methodology in the original paper had one redeeming feature; after any patch was accepted, they would immediately email back withdrawing the patch. That doesn’t appear to have happened in this case, but then this patch was rejected.
As a result of this, all future contributions from people affiliated with UMN are being rejected, and all past contributions (about 250) are being reviewed. Most of those are simply being backed out wholesale, unless someone speaks up for individual changes. A handful of those changes have already been vouched for.
That is pretty drastic, because there will certainly be acceptable patches that will need to be re–reviewed and possibly recommitted. On the other hand, if you discover a malicious actor, wouldn’t you want to investigate everything they’ve been involved with? On the gripping hand, there are such things as autoimmune diseases.
> Now how on earth did that warp to whatever everyone here is smoking?
There's no other option when someone on the same research team later sends them 4 diffs, 3 of which have security holes, than to assume they're still doing research in the same area.
This is what happens when you do a social experiment without at least informing someone in the organization beforehand. There's no way to verify whether it was well intentioned diffs or not. So you must assume it's not.
Its not someone on the same team. Its someone working underneath one of the research members- a grad student who likely had no knowledge of what his supervisor did.
> These are two different projects. The one published at IEEE S&P 2021 has
> completely finished in November 2020. My student Aditya is working on a new
> project that is to find bugs introduced by bad patches. Please do not link
> these two projects together. I am sorry that his new patches are not
> correct either. He did not intentionally make the mistake.
There's a reply to the LKML from the researcher in question admitting that the new student is also working under him doing research. He claims it's not related, but it's not clear how much his word is worth now...
I read everything I could find, in short, researchers did not give any options for maintainers not to participate in research.
The best analogy I could come with so far is; someone offered you compelling job offer, and when you where ready to sign up they would be yeah, that was research project, sorry - would you be ok with such behavior ?
This not ok, because you did not consent to waste your time on someone else's research project.
They addressed this in the paper by making the change small (5 lines). Obvsiously time is still wasted, but the team felt that the research warranted it. This is up to debate and should not be used solely as a reason to crucify them.
> Really losing my faith in the accuracy of HN if such a huge thread is full of misinformation.
What is the point of dubbing yourself the arbiter of the moral high ground and spreading mis-information in the very next breath?
I am less puzzled by you spreading misinformation than I am by the fact you have this outrage at the very thing you are doing and don't hesitate to attack the character of people you disagree with.
> A number of these patches they submitted to the kernel were indeed successfully merged to the Linux kernel tree.
It turns out the researchers DID allow the bad faith commits to be merged and that is a big problem that is still being unwound.
This seems like exactly what happened to me. I do still think the researcher should have gotten approval from maintainers or the foundation before going ahead, and the way he did the research was pretty shitty.
But you also forgot the part where Greg throws a hissy fit and decides to revert every commit from umn emails, including 3+year old commits that legitimately fix security vulns.[0] Great job keeping mainline bug free with your paranoid witchhunt!
The problem with such experiment is that it can be a front. If you are a big entity, gov, or whatever, and you need to insert a vulnerability in the kernel, you can start a "research project". Then you try to inject it with this pretense, and if it fails, you can always say "my bad, it was for science".
I had a uni teacher who thought she was a genius because her research team powdered wikipedia with fake information while timing how long it took to remove them.
"Earth is center of universe" took 1000 years to remove from books, I'm not sure what her point was :D
Joke's on you - this was really sociology research on anger response levels of open source communities when confronted with things that look like bad faith.
Setting aside the ethical aspects which others have covered pretty thoroughly, they may have violated 18 U.S.C. §1030(a)(5) or (b). This law is infamously broad and intent is easier to "prove" than most people think, but #notalawyer #notlegaladvice. Please don't misinterpret this as a suggestion that they should or should not be prosecuted.
So, the patch was about a possible double-free, detected presumably from a bad static analyzer. Couldn't this patch have been done in good faith? That's not at all impossible.
However, the prior activity of submitting bad-faith code is indeed pretty shameful.
I'm not a linux kernel maintainer but it seems like the maintainers all agree it's extremely unlikely a static analyzer could be so wrong in so many different ways.
I think this hasn't gone far enough. The university has shown that it is willing to allow its members to act in bad faith for their own interests, under the auspices of acting ethically for scientific reasons. The university itself cannot be trusted _ever again_.
Black list the whole lot from everything, everywhere. Black hole that place and nuke it from orbit.
What would be the point? Of course people can miss things in code review. Yet the Linux developer base and user base has decided that generally an open submission policy has benefits that outweigh the risks.
Should every city park with a "no alcohol" policy conduct red teams on whether it's possible to smuggle alcohol in? Should police departments conduct red teams to see if people can get away with speeding?
Let's say that no one has ever seen someone speeding or drinking the park. But then someone announces that they just did it, got away with it, and the system isn't effective at catching folks that violate the policies. It might make sense to figure out how you could change the way the system works to stop people from violating the policy. One way to do that is to replicate the violation and see what measures could be introduced to decrease the likely-hood. I would say it is very much akin to the companies that test to see if your employees can be phished or the pen testers to see if you can be hacked. Other important things that people want to protect have these teams to make them a harder target and I think in the case of something as important as the Linux Kernel it might pay dividends.
Not that I approve of the methods, but why would an IRB be involved in a computer security study? IRBs are for human subjects research. If we have to run everything that looks like any kind of research through IRBs, the Western gambit on technical advantage is going to run into some very hard times.
The subjects were the kernel team. They should have had consent to be part of this study. It's like red team testing, someone somewhere has to know about it and consent to it.
It wasn’t a real experiment, it was a legitimate attempt to insert bugs into the code base and this professor was going to go on speaking tours to self promote and talk about how easy it was to crack Linux. If it looks like grift it’s probably grift. This was never about science.
Yet another reason to absolutely despise the culture within academia. The US Federal government is subsidizing a collection of pathologically toxic institutions, and this is one of many results, along with HR departments increasingly mimicking the campus tribalism.
Who do you think subsidizes and guarantees student loan debt that allowed academic institutions to raise their prices at 4 times the rate of inflation?
Nothing. However if they can't claim ownership of the drama they have caused it's not useful for research that's publishable so it does nix these idiots from causing further drama while working at this institution. For now.
They don't need to claim ownership of the drama to write the paper, in fact, my first thought was that they would specifically try to avoid taking ownership and instead write a paper "discovering" the vulnerability(ies).
> What's preventing those bad actors from not using a UMN email address?
Technically none, but by banning UMN submissions, the kernel team have sent an unambiguous message that their original behaviour is not cool. UMN's name has also been dragged through the mud, as it should be.
Prof Lu exercised poor judgement by getting people to submit malicious patches. To use further subterfuge knowing that you've been already been called out on it would be monumentally bad.
I don't know how far Greg has taken this issue up with the university, but I would expect that any reasonable university would give Lu a strong talking-to.
Exactly. Users contributing from the University addresses were borrowing against the reputation of the institution. That reputation is now destroyed and each individual contributor must build their own reputation to earn trust.
Nothing. I think the idea is 60% deterrence via collective punishment - "if we punish the whole university, people will be less likely to do this in future" - and 40% "we must do something, and this is something, therefore we must do it".
If they just want to be jerks, yes. But they can't then use that type of "hiding" to get away with claiming it was done for a University research project as that's even more unethical than what they are doing now.
The point is to make it very obviously not worth it to conduct this kind of unethical research. I don't think UMN is going to be eager to have this kind of attention again. People could always submit bogus patches from random email addresses - this removes the ability to do it under the auspices of a university.
> this removes the ability to do it under the auspices of a university
It really doesn't though. You can claim ownership of that email address in the published manuscript. For that matter, you could even publish the academic article under a pen name if you wanted to. But after seeing how the maintainers responded here, you'd better make sure that any "real" contributions you make aren't associated with the activity in any way.
I think you're getting heavily downvoted with your comments on this submission because you seem to be missing a critical sociological dimension of assumed trust. If you submit a patch from a real name email, you get an extra dimension of human trust and likewise an extra dimension of human repercussions if your actions are deemed to be malicious.
You're criticizing the process, but the truth is that without a real name email and an actual human being's "social credit" to be burned, there's no proof these researchers would have achieved the same findings. The more interesting question to me is if they had used anonymous emails, would they have achieved the same results? If so, there might be some substance to your contrarian views that the process itself is flawed. But as it stands, I'm not sure that's the case.
Why? Well, look at what happened. The maintainers found out and blanket banned bad actors. Going to be a little hard to reproduce that research now, isn't it? Arbitraging societal trust for research doesn't just bring ethical challenges but /practical/ ones involving US law and standards for academic research.
Additionally some universities use a subdomain for student addresses, only making top level email addresses available to staff and a small selection of PhD students who needs it for their research.
Again, we're entering the territory of fraud and cybercrime, whether its white collar crime or not. Nothing wrong with early detection and prevention against that. But as it pertains to malicious actors inside the country, the high risk of getting caught, prosecuted, and earning semi-permanent permanent blackball on your record that would come up in any subsequent reference check (and likely blackball you from further employment) is a deterrent. Which is exactly what these researchers are finding out the hard way.
Anonymity is de facto, not de jure. It's also a privilege for many collaboration networks and not a right. If abused, it will simply be removed.
Given what the Linux kernel runs these days, that would probably be advisable. (I'm a strong proponent of anonymity, but I also have a preference that my devices not be actively sabotaged.)
> we're entering the territory of fraud and cybercrime
So what? The fact that it's illegal doesn't nullify the threat. For that matter, it's not even a crime if a state agency is the perpetrator. These researchers drew attention to a huge (IMO) security issue. They should be thanked and the attack vector carefully examined.
I think you're focusing too much on the literal specifics of the "attack vector" and not enough on the surrounding context, or the real world utility/threat. You're not accurately putting yourself in the shoes of someone who would be using it and asking whether it has a sufficient cost benefit ratio to merit being used. Isn't that what you mean by "carefully examined?"
If you want to talk about a state level actor, I hate to break it to you, but they have significantly more powerful and stealthier 0-day exploits that are a lot easier to exploit than a tactic like this. Guess what's the last thing you want to have happen when you commit cybercrime? Do it in public with where there's an immutable record that can be traced back to you, and cause a giant public hubbub, maybe? So, I can't imagine how someone could think there's anything noteworthy about this unless they were unaware of that.
That's somewhat the unintentional humor and irony of this situation -- all the researchers accomplished was proving that they were not just unethical but incompetent.
Upon further reflection, I think what I wrote regarding anonymity specifically was in error. I don't think removing it would serve much (if any) practical purpose from a security standpoint.
However, I don't agree that what happened was abuse or that it should be deterred. Responding in a hostile manner to an isolated demonstration of a vulnerability isn't constructive. People rightfully get angry when large companies try to bully security researchers.
You question if this vulnerability is worth worrying about due to the logistics of exploiting it in practice. Regardless of whether it's worth the effort to exploit I'd still rather it wasn't there (that goes for all vulnerabilities).
I think it would be much easier to exploit than you're letting on though. Modern attacks frequently chain many subtle bugs together. Being able to land a single, seemingly inconsequential bug in a key location could enable an otherwise impossible approach.
It seems unlikely to me that the immutable record you mention would ever lead back to a competent entity that didn't want to be found. There's no need for anything more than an ephemeral identity that successfully lands one or two patches and then disappears. The patches also seem unlikely to draw suspicion in the first place, even after the exploit itself becomes known.
In fact it occurs to me that a skilled and amoral developer could likely land multiple patches with strategic (and very subtle) bugs from different identities. These could then be "discovered" and sold on the black market. I see no convincing reason to discount the possibility of this already being a common occurrence.
The only sensible response I can think of is a focus on static analysis coupled with CTF challenges to beat those analysis methods.
Indeed the situation is bad, nothing can be done. At the very least as long as they can make unintentional vulnerabilities, they are defenseless against intentional ones, and fixing only the former is already a very big deal.
I saw at least one developer lamenting that they were going to potentially bring up mechanisms for having to treat every committer as malicious by default instead of not at the next kernel summit, so it's quite possible that's going to take place.
> lamenting that they were going to potentially bring up mechanisms for having to treat every committer as malicious by default
I think "lamenting" is very much the wrong attitude here. Given all the things that make use of Linux today that seems like the only sane approach to me.
Well, it seems unlikely that any other universities will fund or support copy cat studies. And I don't mean in the top-down institutional sense I mean in the self-selecting sense. Students will not see messing with the linux kernel as being a viable research opportunity and will not do it. That doesn't seem to be 'feel-good without any actual benefit to the kernel's security'. Sounds like it could function as an effective deterent.
Isn't this reaction a bit like the emperor banishing anyone who tells him that his new clothes are fake? Are the maintainers upset that someone showed how easy it is to subvert kernel security?
It’s more like the emperor banning a group of people who put the citizens in danger just so they could show that it could be done. The researchers did something unethical and acted in a self-serving manner. It’s no surprise that someone would get kicked out of a community after seriously breaking the trust of that community.
The middle ground would be if the Emperor jailed the tailors of the New Clothes after he had shown off the clothes at the Parade, in front of the whole city.
> Because of this, I will now have to ban all future contributions from
your University.
Understandable from gkh, but I feel sorry for any unrelated research happening at University of Minnesota.
EDIT: Searching through the source code[1] reveals contributions to the kernel from umn.edu emails in the form of an AppleTalk driver and support for the kernel on PowerPC architectures.
In the commit traffic[2], I think all patches have come from people currently being advised by Kangjie Liu[3] or Liu himself dating back to Dec 2018. In 2018, Wenwen Wang was submitting patches; during this time he was a postdoc at UMN and co-authored a paper with Liu[4].
Prior to 2018, commits involving UMN folks appeared in 2014, 2013, and 2008. None of these people appear to be associated with Liu in any significant way.
> I think all patches have come from people currently being advised by Kangjie Liu[3] or Liu himself dating back to Dec 2018
New plan: Show up at Liu's house with a lock picking kit while he's away at work, pick the front door and open it, but don't enter. Send him a photo, "hey, just testing, bro! Legitimate security research!"
If they wanted to do security research, they could have done so in the form of asking the reviewers to help; send them a patch and ask 'Is this something you would accept?', instead of intentionally sending malicious commits and causing static on the commit tree and mailing lists.
Notify someone up the chain that you want to submit malicious patches, and ask them if they want to collaborate.
If your patches make it through, treat it as though they essentially just got red teamed, everyone who reviewed it and let it slip gets to have a nervous laugh and the commit gets rejected, everyone having learned something.
I wonder if informing anyone of the experiment would be frowned upon as it might affect the outcome? However, this research doesn’t appear to be fastidious about scientific integrity so maybe you are right.
Wouldn't that draw more attention to the research patches, compared to a "normal" lkml patch? If you (as a maintainer) expected the patch to be malicious, wouldn't you be extra careful in reviewing it?
You probably can learn more and faster about new drugs by testing them in humans rather than rats. However, science is not above ethics. That is a lesson history has taught us in the most unpleasant of ways.
You don't have to say you are studying the security implications, you could be say you are studying something else like turn around time for patches, or level of critique, or any number of things.
Yeah, so an analogy would be to put human feces into food and then see if the waiter is going to actually give it to the dinning customer. And then if they do, just put a checkmark on a piece of paper and then leave without warning someone that they're about to eat poop.
This is funny, but not at all a good analogy. There's obviously not remotely as much public interest or value in testing the security of this professor's private home to justify invading his privacy for the public interest. On the other hand, if he kept dangerous things at home (say, BSL-4 material), then his house would need 24/7 security and you'd probably be able to justify testing it regularly for the public's sake. So the argument here comes down to which extreme you believe the Linux kernel is closer to.
Yeah, for one thing, to be a good analogy, rather than lockpicking without entering when he’s not home and leaving a note, you’d need to be an actual service worker for a trusted home service business and use that trust to enter when he is home, conduct sabotage, and not say anything until the sabotage is detected and traced back to you and cited in his cancelling the contract with the firm for which you work, and then cite the “research” rationale.
Of course, if you did that you would be both unemployed and facing criminal charges in short order.
Everyone has been saying "This affects software that runs on billions of machines and could cause untold amounts of damage and even loss of human life! What were the researchers thinking?!" and I guess a follow-up thought, which is that "Maintainers for software that runs on billions of machines, where bugs could cause untold amounts of damage and even loss of human life didn't have a robust enough system to prevent this?" never occurs to anyone. I don't understand why.
It's occurred to absolutely everyone. What doesn't seem to have occurred to many people is that there is no such thing as a review process robust enough to prevent malicious contributions. Have you ever done code review for code written by mediocre developers? It's impossible to find all of the bugs without spending 10x more time than it would take to just rewrite it from scratch yourself. The only real alternative is to not be open source at all and only allow contributions from people who have passed much more stringent qualifications.
There is no such thing as a process that can compensate for trust mechanisms. Or if you want to view it that way, ignoring the university's protests and blanket-banning all contributions made by anybody there with no further investigation is part of the process.
People are well aware of theoretical risk of bad commits by malicious actors. They are justifiably extremely upset that someone is intentionally changing this from a theoretical attack to a real life issue.
I'm not confused about why people are upset at the researchers that introduced bugs and did it irresponsibly. I'm confused about why people aren't upset that an organization managing critical infrastructure is so under prepared at dealing with risks posed by rank amateurs, which they should've known about and had a mechanism of dealing with for years.
What this means is that anyone who could hijack a university email account, or could be a student at a state university for a semester or so, or work at a FAANG corporation could pretty much insert backdoors without a lot of scrutiny in a way that no one detects, because there aren't robust safeguards in place to actually verify that commits don't do anything sneaky beyond trusting that everyone is acting in good faith because of how they act in a code review process. I have trouble understanding the thought process that ends up basically ignoring the maintainers' duty to make sure that the code being committed doesn't endanger security or lives because they assumed that everything was 'cool'. The security posture in this critical infrastructure is deficient and no one wants to actually address it.
> I have trouble understanding the thought process that ends up basically ignoring the maintainers' duty to make sure that the code being committed doesn't endanger security or lives because they assumed that everything was 'cool'. The security posture in this critical infrastructure is deficient and no one wants to actually address it.
They're banning a group known to be bad actors. And proactively tearing out the history of commits related to those known actors, before reviewing each commit.
That seems like the kernel team are taking a proactive stance on the security side of this. The LKML thread also talks about more stringent requirements that they're going to bring in, which was already going to be brought up at the next kernel conference.
None of these things seem like ignoring any of the security issues.
After absorbing what the researchers did, I believe it's time to skip right over the second part and just concentrate on why so many critical systems are run on unforked Linux.
I remember a true story (forget by whom) where the narrator set up a simple website for some local community activity. A stranger hacked and defaced the website, admitted to doing so without revealing his identity. His position in contacting the author of the website was, "I did you a favor (by revealing how vulnerable it was)." The person telling the story reacted, "yes, but... you were the threat you're warning me of." It didn't result in the author recreating the site on a more secure platform, it only resulted in him deciding it was not worth the trouble to provide this free service any longer.
It wasn't intended to be serious. But on the other hand, he has now quite openly and publicly declared himself to be part of a group of people who mess around with security related things as a "test".
He shouldn't be surprised if it has some unexpected consequences to his own personal security, like some unknown third parties porting away his phone number(s) as a social engineering test, pen testing his office, or similar.
I wouldn't be surprised if the good, conscientious members of the UMN community showed up at his office (or home) door to explain, in vivid detail, the consequences of doing unethical research.
The actual equivalent would be to steal his computer, wait a couple days to see his reaction, get a paper published, then offer to return the computer.
If this experience doesn't change not only the behavior of U of M's IRB but inform the behavior of every other IRB, then nothing at all is learned from this experience.
Unless both the professors and leadership from the IRB aren't having an uncomfortable lecture in the chancellor's office then nothing at all changes.
The main thing you want here is a demonstration that they realize they fucked up, realize the magnitude of the fuckup, and have done something reasonable to lower the risk of it happening again, hopefully very low.
Given that the professor appears to be a frequent flyer with this, the kernel folks banning him and the university prohibiting him from using Uni resources for anything kernel related seems reasonable and gets the point across.
The students do need a bit of punishment - they are adults who chose to act this way. In this context though, switching their advisor and requiring a different research track would be sufficient - that's a lot of work down the drain and a hard lesson. I agree that expulsion would be unfair - (assuming good faith scholarship) the advisor/student relationship is set up so that the students can learn to research effectively (which includes ethically) with guidance from a trusted researcher at a trusted institution. If the professor suggests or OKs a plan, it is reasonable for the students to believe it is a acceptable course of action.
1. What the student code at umn says and what i think the student deserves are vastly different things.
2. Something being grounds for expulsion and what a reasonable response would be are vastly different things.
3. The rules saying "up to and including" (aka grounds for) and the full range of punishment are not the same - the max of a range is not the entirety of the range.
You are mixing up two students. The one who complained about "bordering on slander" had nothing to do with the research paper at issue, other than having the same advisor as the author.
I agree in this case the driver of the behavior seems to be the professor, but graduate researchers are informed about ethical research and there many ways students alone can cause harm through research potentially beyond the supervision of the university and even professor. It's usually much more neglible than this, but everyone has a responsibility in abiding by ethical norms.
The suggestion about the IRB was made by a third party. Look at the follow up comment from kernel developer Leon Romanovsky.
> ... we don't need to do all the above and waste our time to fill some bureaucratic forms with unclear timelines and results. Our solution to ignore all @umn.edu contributions is much more reliable to us who are suffering from these researchers.
To follow up on my comment here, I think Greg KH's later responses were more reasonable.
> ... we have the ability to easily go back and rip the changes out and we can slowly add them back if they are actually something we want to do.
> I will be working with some other kernel developers to determine if any of these reverts were actually valid changes, were actually valid, and if so, will resubmit them properly later. ... future submissions from anyone with a umn.edu address should be by default-rejected unless otherwise determined to actually be a valid fix
Well, yes? Seems like recourse in their case would be to make a convincing plea or plan to rectify the problem that satisfies decision makers in the linux project.
This is not responsible research. This is similar to initiating fluid mechanics experiments on the wings of a Lufthansa A320 in flight to Frankfurt with a load of Austrians.
There are a lot of people to feel bad for, but none is at the University of Minnesota. Think of the Austrians.
No, it's totally okay to feel sorry for good, conscientious researchers and students at the University of Minnesota who have been working on the kernel in good faith. It's sad that the actions of irresponsible researchers and associated review boards affect people who had nothing to do with professor Lu's research.
It's not wrong for the kernel community to decide to blanket ban contributions from the university. It obviously makes sense to ban contributions from institutions which are known to send intentionally buggy commits disguised as fixes. That doesn't mean you can't feel bad for the innocent students and professors.
To be clear, Linux kernel patches from good UMN researchers and students are rare. We have plenty of great people at the University of Minnesota, they just don't work on the Linux kernel.
It's justifiable and natural for our name to be dragged in the mud here, but as a run of the mill software engineer who graduated from UMN, I hope our reputation isn't soured too much.
Sure, I hope it was clear from my original comment I only question whether the UMN contributors to the kernel are acting in good faith. I have identified other questionable patches personally, out of curiosity. Naturally I tend to attribute them to ignorance rather than malice... except, that bad actors intentionally pushing bad patches to an OSS project will inevitably rely on people assuming ignorance rather than malice. This has been well-understood for decades.
Someone in this HN thread found kernel patches (at a guess, not among those now reverted?) from UMN dating back to 2008, -09, and -13 (IIRC). Probably by totally unrelated people.
> This is similar to initiating fluid mechanics experiments on the wings of a Lufthansa A320 in flight to Frankfurt with a load of Austrians.
This analogy is invalid, because:
1. The experiment is not on live, deployed, versions of the kernel.
2. There are mechanisms in place for preventing actual merging of the faulty patches.
3. Even if a patch is merged by mistake, it can be easily backed out or replaced with another patch, and the updates pushed anywhere relevant.
All of the above is not true for the in-flight airline.
However - I'm not claiming the experiment was not ethically faulty. Certainly, the U Minnesota IRB needs to issue a report and an explanation on its involvement in this matter.
> 1. The experiment is not on live, deployed, versions of the kernel.
The patches were merged and the email thread discusses that the patches made it to the stable tree. Some (many?) distributions of Linux have and run from stable.
> 2. There are mechanisms in place for preventing actual merging of the faulty patches.
Those mechanisms failed.
> 3. Even if a patch is merged by mistake, it can be easily backed out or replaced with another patch, and the updates pushed anywhere relevant.
The approved methodology - described in the linked paper - was that when a patch with the introduced vulnerabilities is accepted by its reviewer, the patch submitter indicates that the patch introduces a vulnerability exists, and sends a no-vulnerability version. That's what the paper describes.
If the researchers did something other than what the methodology called for (and what the IRB approved), then perhaps the analogy may be valid.
There are literally mails in that list pointing out that commits made it to stable. At least read the damn thing before repeating the professor's/student's nonsense lies.
The mails in the pointed-to threads indicate that commits by those UMin people made it to stable; it does not say that commits which introduce bugs made it to stable - it is following a decision/suggestion to back out all patches by these people to the kernel.
There is further indication that the patches to revert are not mostly/not at all vulnerability-introducing patches in a message by "Steve" which says:
> The one patch from Greg's reverts that affects my code was actually a legitimate fix
So, again, while it is still theoretically possible that vulnerabilities were introduced into stable, that is not known to be the case.
You seem to think this experiment was performed on the Linux kernel itself. It was not. This research was performed on human beings.
It's irrelevant whether any bugs were ultimately introduced into the kernel. The fact is the researchers deliberately abused the trust of other human beings in order to experiment on them. A ban on further contributions is a very light punishment for such behavior.
How would you feel about researchers delivering known-faulty-under-some-conditions AoA sensors to Boeing, just to see if Boeing's QA process would catch those errors before final assembly?
You are ignoring the problematic experiment's methodology, which explicitly prevents the problematic patches from making it in by drawing attention to the vulnerability after they are accepted.
Now, if this protocol were not followed, that's a different story, but we do not know this to be the case.
I would feel that I'm wasting time that I could be using to find out why Boeing makes this possible (or any other corporate or government body with a critical system).
It's important to note that they used temporary emails for the patches in this research. It's detailed in the paper.
The main problem is that they have (so far) refused to explain in detail how the patches where reviewed and how. I have not gotten any links to any lkml post even after Kangjie Lu personally emailed me to address any concerns.
Seems like a bit of a strong response. Universities are large places with lots of professors and people with different ideas, opinions, views, and they don't work in concert, quite the opposite. They're not some corporation with some unified goal or incentives.
I like that. That's what makes universities interesting to me.
I don't like the standard here of of penalizing or lumping everyone there together, regardless of they contribute in the past, now, in the future or not.
The goal is not penalizing or lumping everyone together. The goal is to have the issue fixed in the most effective manner. It's not the Linux team's responsibility to allow contributions from some specific university, it's the university's. This measure enforces that responsibility. If they want access, they should rectify.
If a company that sold static analysis products did this as part of a marketing campaign, would you likewise have so many reservations about blacklisting contributions from that company, or would you still be insisting on picking out individual employees?
It's pretty obvious what would happen if a firm tried this: they'd be taken to court and probably imprisoned, as this is a clear violation of the law (which is pretty broadly to capture any attempted interference with the correct operation of a computer program).
One way to get everyone in a university on the same page is to punish them all for the bad actions of a few. It appears like this won't work here because nobody else is contributing and so they won't notice.
It's not the number of people directly affected that will matter, it's the reputational problems of "umn.edu's CS department got the entire UMN system banned from submitting to the Linux kernel and probably some other open source projects."
This was approved by the university ethics board so if trust of the university is by part because the actions of the students need to pass an ethics bar it makes sense to remove that trust until the ethics committee has shown that they have improved.
The ethics board is most likely not at fault here. They were simply lied to, if we take Lu's paper serious. I would just expell the 3 malicious actors here, the 2 students and the Prof who approved it. I don't see any fault in Wang yet.
The damage is not that big. Only 4 committers to linux in the last decade, 2 of them, the students, with malicious backdoors, the Prof not with bad code but bad ethics, and the 4th, the Ass Prof did good patches and already left them.
So the pen-test on the ethics board showed that they had not institutionalized proper safeguards regarding malicious actors? (And not even a paper on this… ;-) )
I'd concur: the university is the wrong unit-of-ban.
For example: what happens when the students graduate- does the ban follow them to any potential employers? Or if the professor leaves for another university to continue this research?
Does the ban stay with UMN, even after everyone involved left? Or does it follow the researcher(s) to a new university, even if the new employer had no responsibility for them?
It's the university that allowed the research to take place. It's the university's responsibility to fix their own organisation's issues. The kernel has enough on their plate than to have to figure out who at the university is trustworthy and who isn't considering their IRB is clearly flying blind.
that is completely irrelevant. they are acting under the university, and their "Research" is backed by university and approved by university's department.
if university has a problem, then they should first look into managing this issue at their end, or force people to use personal email ids for such purposes
I don't feel sorry at all. If you want to contribute from there, show that the rogue professor and their students have been prevented from doing further malicious contributions (that is probably at least: from doing any contribution at all during a quite long period -- and that is fair against repeated infractions), and I'm sure that you will be able to contribute back again under the University umbrella.
If you don't manage to reach that goal, too bad, but you can contribute on a personal capacity, and/or go work elsewhere.
How could a single student or professor possibly achieve that? Under the banner of "academic freedom" it is very hard to get someone fired because you don't like their research.
It sounds like you're making impossible demands of unrelated people, while doing nothing to solve the actual problem because the perpetrators now know to just create throwaway emails when submitting patches.
It definitely would suck to be someone at UMN doing legitimate work, but I don't think it's reasonable to ask maintainers to also do a background check on who the contributor is and who they're advised by.
How thorough is IRB review? My gut feeling is that these are not necessarily the most conscientious or informed bodies. Add into the mix a proposal that conceals the true nature of what's happening.
(All of this ASSUMING that the intent was as described in the thread.)
It varies a lot. A professor I worked for was previously at a large company in an R&D setting. He dealt with 15-20 different IRB's through various research partnerships, and noted Iowa State (our university) as having the most stringent requirements he had encountered. In other universities, it was pretty simple to submit and get approval without notable changes to the research plan. If they were unsure on something, they would ask a lot of questions.
I worked on a number of studies through undergrad and grad school, mostly involving having people test software. The work to get a study approved was easily 20 hours for a simple "we want to see how well people perform tasks in the custom software we developed. They'll come to the university and use our computer to avoid security concerns about software security bugs". You needed a script of everything you would say, every question you would ask, how the data would be collected, analyzed, and stored securely. Data retention and destruction policies had to be noted. The key linking a person's name and their participant ID had to be stored separately. How would you recruit participants, the exact poster or email you intend to send out. The reading level of the instructions and the aptitude of audience were considered (so academic mumbo jumbo didn't confuse participants).
If you check the box that you'll be deceiving participants, there was another entire section to fill out detailing how they'd be deceived, why it was needed for the study, etc. Because of past unethical experiments in the academic world, there is a lot of scrutiny and you typically have to reveal the deception in a debriefing after the completion of the study.
Once a study was accepted (in practice, a multiple month process), you could make modifications with an order of magnitude less effort. Adding questions that don't involve personal information of the participant is a quick form and an approval some number of days later.
If you remotely thought you'd need IRB approval, you started a conversation with the office and filled out some preliminary paperwork. If it didn't require approval, you'd get documentation stating such. This protects the participants, university, and professor from issues.
--
They took it really seriously. I'm familiar with one study where participants would operate a robot outside. An IRB committee member asked what would happen if a bee stung the participant? If I remember right, the resolution was an epipen and someone trained in how to use it had to be present during the session.
They are probably more familiar with medical research and the types of things that go wrong there. Bad ethics in medical situations is well understood, including psychology. However it is hard to figure out how a mechanical engineer could violate ethics.
I had to do human subjects research training in grad school, just to be able to handle test score data for a math education project. I literally never saw an actual student the whole time I was working on it.
Perhaps more dire than what actually happened, but, can you imagine the consequences if any of those malicious patches had actually stuck around in the kernel? Keep in mind when you think about this that Android, which has an 87% market share globally in smartphones[0] runs on top of a modified Linux kernel.
seems extreme. one unethical researcher blocks work for others just because they happen to work at the same employer? they might not even know the author of the paper...
The university reviewed the "study" and said it was acceptable. From the email chain, it looks like they've already complained to the university multiple times, and have apparently been ignored. Banning anyone at the university from contributing seems like the only way to handle it since they can't trust the institution to ensure its students are doing unethical experiments.
Well, the decision can always be reversed, but on the outset I would say banning the entire university and publicly naming them is a good start. I don't think this kind of "research" is ethical, and the issue needs to be raised. Banning them is a good opener to engage the instiution in a dialogue.
It seems fair enough to me. They were curious to see what happens, this happens. Giving them a free pass because they're a university would be artificially skewing the results of the research.
Low trust and negative trust should be fairly obvious costs to messing with a trust model - you could easily argue this is working as intended.
It is an extreme response to an extreme problem. If the other researchers don't like the situation? They are free to raise the problem to the university and have the university clean up the mess they obviously have.
Well, shit happens. Imaging doctors working in organ transplants, and one of them damages trust of people by selling access to organs to rich patients. Of course that damages the field for everyone. And to deal with such issues, doctors have some ethics code, and in many countries associations which will sanction bad eggs. Perhaps scientists need something like that, too?
Which just demonstrates that these guys are actual bad actors, so blocking everyone at the university seems like a reasonable attempt at stopping them.
It's objectionable because of severe consequences beyond just annoying people. If there was a malicious purpose, not just research, you could bring criminal charges against them.
In typical grey hat research you get pre-approval from target company leadership (engineers don't know) to avoid charges once discovered.
That's not really how it works. Nobody's out there 'approving' research (well, not seemingly small projects like this), especially at the university level. Professors (all the way down to PhD students!) are usually left to do what they like, unless there are specific ethical concerns that should be put before a review panel. I suppose you could argue that this work should have been brought before the ethics committee, but it probably wasn't, and in CS there isn't a stringent process like there is in e.g. psychology or biology.
If you read the research paper linked in the lkml post, the authors at UMN state that they submitted their research plan to the University of Minnesota Institutional Research Board and received a human subjects exempt waiver.
for q1, the study collects data like observations of behavior, so we must answer yes.
for q2, none of the exemptions apply - it's not an educational setting, they're not sending a survey, it's not an observation of the public - they're interacting, and it's clear that these interactions are not benign - they have clear impact on the community. None of these exemptions apply.
Based on this flow, it's clear the study involves "human research".
The emails suggest this work has been reported in the past. A review by the ethics committee after the fact seems appropriate, and it should’ve stopped a repeat offence.
> Not a big loss: these professors likely hate open source.
> They are conducting research to demonstrate that it is easy to introduce bugs in open source...
That's a very dangerous thought pattern. "They try to find flaws in a thing I find precious, therefore they must hate that thing." No, they may just as well be trying to identify flaws to make them visible and therefore easier to fix. Sunlight being the best disinfectant, and all that.
(Conversely, people trying to destroy open source would not publicly identify themselves as researchers and reveal what they're doing.)
> whereas we know that the strength of open source is its auditability, thus such bugs are quickly discovered and fixed afterwards
How do we know that? We know things by regularly testing them. That's literally what this research is - checking how likely it is that intentional vulnerabilities are caught during review process.
Ascribing a salutary motive to sabotage is just as dangerous as assuming a pernicious motive. Suggesting that people "would" likely follow one course of action or another is also dangerous: it is the oldest form of sophistry, the eikos argument of Corax and Tisias. After all, if publishing research rules out pernicious motives, academia suddenly becomes the best possible cover for espionage and state-sanctioned sabotage designed to undermine security.
The important thing is not to hunt for motives but to identify and quarantine the saboteurs to prevent further sabotage. Complaining to the University's research ethics board might help, because, regardless of intent, sabotage is still sabotage, and that is unethical.
"Dear GK-H: I would like to have my students test the security of the kernel development process. Here is my first stab at a protocol, can we work on this?"
and
"We're going to see if we can introduce bugs into the Linux kernel, and probably tell them afterwards"
is the difference between white-hat and black-hat.
It should probably be a private email to Linus Torvalds (or someone in his near chain of patch acceptance), that way some easy to scan for key can be introduced in all patches. Then the top levels can see what actually made it through review, and in turn figure out who isn't reviewing as well as they should.
Yes, someone like Greg K-H. I'm not up to date on the details, but he should be one of most important 5 people caring for the kernel tree, this would've been the exact person to seek approval.
Auditability is at the core of its advantage over closed development.
Submitting bugs is not really testing auditability, which happens over a longer timeframe and involves an order of magnitude more eyeballs.
To adress your first critic: benevolence, and assuming everyone wants the best for the project, is very important in these models, because the resources are limited and dependent on enthusiasm. Blacklisting bad actors (even if they have "good reasons" to be bad) is very well justified.
> Auditability is at the core of its advantage over closed development.
That's an assertion. A hypothesis is verified through observing the real world. You can do that in many ways, giving you different confidence levels in validity of the hypothesis. Research such as the one we're discussing here is one of the ways to produce evidence for or against this hypothesis.
> Submitting bugs is not really testing auditability, which happens over a longer timeframe and involves an order of magnitude more eyeballs.
It is if there's a review process. Auditability itself is really most interesting before a patch is accepted. Sure, it's nice if vulnerabilities are found eventually, but the longer that takes, the more likely it is they were already exploited. In case of an intentionally bad patch in particular, the window for reverting it before it does most of its damage is very small.
In other words, the experiment wasn't testing the entire auditability hypothesis. Just the important part.
> benevolence, and assuming everyone wants the best for the project, is very important in these models, because the resources are limited and dependent on enthusiasm
Sure. But the project scope matters. Linux kernel isn't some random OSS library on Github. It's core infrastructure of the planet. Assumption of benevolence works as long as the interested community is small and has little interest in being evil. With infrastructure-level OSS projects, the interested community is very large and contains a lot of malicious actors.
> Blacklisting bad actors (even if they have "good reasons" to be bad) is very well justified.
I agree, and in my books, if a legitimate researcher gets banned for such "undercover" research, it's just the flip side of doing such experiment.
Before a patch is accepted, "auditability" is the same in OSS vs in proprietary, because both pools of engineers in the review groups have similar qualifications and approximatively the same number of people are involved.
So, the real advantage of OSS is on the auditability after the patch is integrated.
> So, the real advantage of OSS is on the auditability after the patch is integrated.
If that's the claim, then the research work discussed here is indeed not relevant to it.
But also, if that's the claim, then it's easy to point out that the "advantage" here is hypothetical, and not too important in practice. Most people and companies using OSS rely on release versions to be stable and tested, and don't bother doing their own audit. On the other hand, intentional vulnerability submission is an unique threat vector that OSS has, and which proprietary software doesn't.
It is therefore the window between patch submission and its inclusion in a stable release (which may involve accepting the patch to a development/pre-release tree), that's of critical importance for OSS - if vulnerabilities that are already known to some parties (whether the malicious authors or evil onlookers) are not caught in that window, the threat vector here becomes real, and from a risk analysis perspective, negates some of the other benefits of using OSS components.
Nowhere here I'm implying OSS is worse/better than proprietary. As a community/industry, we want to have an accurate, multi-dimensional understanding of the risks and benefits of various development models (especially when applied to core infrastructure project that the whole modern economy runs on). That kind of research definitely helps here.
> On the other hand, intentional vulnerability submission is an unique threat vector that OSS has, and which proprietary software doesn't.
On this specific point, it only holds if you restrict the assertion to 'intentional submission of vulnerabilities by outsiders'. I don't work in fintech, but I've read allegations that insider-created vulnerabilities and backdoors are a very real risk.
> That human society exists is proof that people are, or act, mostly, "good", over the long term.
That's very true. It's worth noting that various legal and security tools deployed by the society help us understand what are the real limits to "mostly".
So for example, the cryptocurrency crowd is very misguided in their pursuit of replacing trust with math - trust is the trick, the big performance hack, that allowed us to form functioning societies without burning ridiculous amounts of energy to achieve consensus. On the other hand, projects like Linux kernel, which play a core role in modern economy, cannot rely on assumption of benevolence alone - incentives for malicious parties to try and mess with them are too great to ignore.
> It's likely a university with professors that hate open source.
This is a ridiculous conclusion. I do agree with the kernel maintainers here, but there is no way to conclude that the researchers in question "hate open source", and certainly not that such an attitude is shared by the university at large.
[Edit: they seem to truly love OSS. See child comments. Sorry for my erroneous judgement. It reminded too much of anti-opensource FUD, I'm probably having PTSD of that time...]
I fixed my sentence.
I still think that these professors, either genuinely or by lack of willingness, do not understand the mechanism by which free software warrants its greater quality compared to proprietary ones (which is a fact).
They just remind me the good old days of FUD against open source by Microsoft and its minions...
> In the past several years, we devote most of our time to improving the Linux kernel, and
we have found and fixed more than one thousand kernel bugs; the extensive bug finding and fixing
experience also allowed us to observe issues with the patching process and motivated us to improve it. Thus, we consider ourselves security researchers as well as OSS contributors. We respect OSS volunteers and honor their efforts.
My analysis of that exact same quote was that it was insincere cover to allow them to continue operating an anti-OSS agenda which was made clear in the paper itself.
> They introduce kernel bugs on purpose. Yesterday, I took a look on 4 accepted patches from Aditya and 3 of them added various severity security
"holes".
Interestingly, that paper states that they introduced 3 patches with bugs, but after acceptance, they immediately notified the maintainers and replaced the patches with correct, bug-free ones. So they claim the bugs never hit any git tree. They also state that their research had passed the university IRB. I don't know if that research relates to what they are doing now, though.
> the strength of open source is its auditability, thus such bugs are quickly discovered and fixed afterwards
That's not true at all. There are many internet-critical projects with tons of holes that are not found for decades, because nobody except the core team ever looks at the code. You have to actually write tests, do fuzzing, static/memory analysis, etc to find bugs/security holes. Most open source projects don't even have tests.
Assuming people are always looking for bugs in FOSS projects is like assuming people are always looking for code violations in skyscrapers, just because a lot of people walk around them.
> (whereas we know that the strength of open source is its auditability, thus such bugs are quickly discovered and fixed afterwards)
Which is why there have never been multi-year critical security vulnerabilities in FOSS software.... right?
Sarcasm aside, because of how FOSS software is packaged on Linux we've seen critical security bugs introduced by package maintainers into software that didn't have them!
You need to compare what happens with vulnerabilities in OSS vs in proprietary.
A maintainer pakage is just one more open source software (thus also in need of reviews and audits)... which is why some people prefer upstream-source-based distribs, such as Gentoo, Arch when you use git-based AUR packages, or LFS for the hardcore fans.
> You need to compare what happens with vulnerabilities in OSS vs in proprietary.
Yes, you do need to make that comparison. Taking it as a given without analysis is the same as trusting the proprietary software vendors who claim to have robust QA on everything.
Security is hard work and different from normal review. The number of people who hypothetically could do it is much greater than the number who actually do, especially if there isn’t an active effort to support that type of analysis.
I’m not a huge fan of this professor’s research tactic but I would ask what the odds are that, say, an intelligence agency isn’t doing the same thing but with better concealment. Thinking about how to catch that without shutting down open-source contributions seems like an important problem.
Some clarifications since they are unclear in the original report.
- Aditya Pakki (the author who sent the new round of seemingly bogus patches) is not involved in the S&P 2021 research. This means Aditya is likely to have nothing to do with the prior round of patching attempts that led to the S&P 2021 paper.
- According to the authors' clarification [1], the S&P 2021 paper did not introduce any bugs into Linux kernel. The three attempts did not even become Git commits.
Greg has all reasons to be unhappy since they were unknowingly experimented on and used as lab rats. However, the round of patches that triggered his anger *are very likely* to have nothing to do with the three intentionally incorrect patch attempts leading to the paper. Many people on HN do not seem to know this.
There is no doubt that Kangjie is involved in Aditya's research work, which leads to bogus patches sent to Linux devs. However, based on my understanding of how CS research groups usually function, I do not think Kangjie knew the exact patches that Aditya sent out. In this specific case, I feel Aditya is more likely the one to blame: He should have examined these automatically generated patches more carefully before sending them in for reviewing.
Kangjie should not have approved any research plan involving kernel patches knowing that he had already set that relationship on fire in order to get a prior paper published.
>based on my understanding of how CS research groups usually function
If you mean supervisors adding their names on to publications without having contributed any work, than this is not only limited to CS research groups. Authorship misrepresentation is widespread in academia and unfortunately mostly being ignored. Those who speak up are being singled out and isolated instead.
I would say it's less authorship misrepresentation and more an established convention that's well-known to people within the field. Lead authors go first, then supporting contributors, and finally advisors at the end.
Even if they didn't write or perform part of the research, they did act in an advisory or consulting fashion, and therefore could significantly shape the research. Maybe there should be a different way to credit that, but right now convention is to put them in a low position in the list of authors.
Where I went, it was so widespread that it was considered normal and nobody would even think about speaking out about it. Only with hindsight did I realize how despicable it was.
At my last job a low quality paper was written about "my work" despite me saying that it's too early. It was based on my idea and resulted in our research groups grant participation being extended, despite our professor saying that the grant won't be extended since he switched universities and continent.
It was written while I was working on the software and my name was then put in third position on the list of authors. Only way I was able to defend myself was, asking them to remove my name from the paper which resulted in the paper not being published.
Aditya's story about the new patches is that he was writing a static analysis tool and was testing it by... submitting PRs to the Linux kernel? He's either exploiting the Linux maintainers to test his new tool, or that story's bullshit. Even taking his story at face value is justification to at least ban him personally IMO.
To be clear: asking Linux maintainers to verify the results of static analysis tools they wrote themselves, without admitting to it until they're accused of being malicious?
Asking Linux maintainers to apply patches or fix “bug” resulting from home grown static analysis tools, which usually flag all kinds of things that aren’t bugs. This happens regularly.
As someone who used to maintain a large C++ codebase, people usually bug-dump static analysis results rather than actually submitting fixes, but blindly "fixing" code that a static analysis tool claims to have issue with is not surprising to see either.
If the patches were accepted, the person could have used those fixes to justify the benefits of the static analysis tool they wrote.
They generally state that this was found with so-and-so static analysis tool. And, as GKH pointed out in the thread, the resulting patches generally make changes that match a certain pattern, not the random uselessness that Aditya's patches were.
Sounds like these commits aren't related to that paper, they're related to the next paper he's working on, and the next one is making the same category error about human subjects in his study.
he claims "Hi there! My name is Aditya and I’m a second year Ph.D student in Computer Science & Engineering at the University of Minnesota. My research interests are in the areas of computer security, operating systems, and machine learning. I’m fortunate to be advised by Prof. Kangjie Lu."
so he in no uncertain terms is claiming that he is being advised in his research by Kangjie Lu. So it's incorrect to say his patches have nothing to do with the paper.
I would encourage you not to post people's contact information publicly, specially in a thread as volatile as this one. Writing "He claims in his personal website" would bring the point across fine.
This being the internet, I'm sure the guy is getting plenty of hate mail as it is. No need to make it worse.
> So it's incorrect to say his patches have nothing to do with the paper.
Professors usually work on multiple projects, which involve different grad students, at the same time. Aditya Pakki could be working on a different project with Kangjie Lu, and not be involved with the problematic paper.
> S&P 2021 paper did not introduce any bugs into Linux kernel.
I used to work as an auditor. We were expected to conduct our audits to neither expect nor not expect instances of impropriety to exist. However, once we had grounds to suspect malfeasance, we were "on alert", and conduct tests accordingly.
This is a good principle that could be applied here. We could bat backwards and forwards about whether the other submissions were bogus, but the presumption must now be one of guilt rather than innocence.
Personally, I would have been furious and said, in no uncertain terms, that the university keep a low profile and STFU lest I be sufficiently provoked to taking actions that lead to someone's balls being handed to me on a plate.
What sort of lawsuit might they bring against a university whose researchers deliberately inserted malicious code into software that literally runs a good portion of the world?
I'm no lawyer, but it seems like there'd be something actionable.
On a side note, this brings into question any research written by any of the participating authors, ever. No more presumption of good faith.
>What sort of lawsuit might they bring against a university whose researchers deliberately inserted malicious code into software that literally runs a good portion of the world?
I am also not a lawyer, but aside from any civil action, the conduct looks like it might be considered criminal under the Computer Fraud and Abuse Act:
"Whoever knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer;"
> According to the authors' clarification [1], the S&P 2021 paper did not introduce any bugs into Linux kernel. The three attempts did not even become Git commits.
Except that at least one of those three, did [0]. The author is incorrect that none of their attempts became git commits. Whatever process that they used to "check different versions of Linux and further confirmed that none of the incorrect patches was adopted" was insufficient.
> The author is incorrect that none of their attempts became git commits
That doesn't appear to be one of the three patches from the "hypocrite commits" paper, which were reportedly submitted from pseudononymous gmail addresses. There are hundreds of other patches from UMN, many from Pakki[0], and some of those did contain bugs or were invalid[1], but there's currently no hard evidence that Pakki was deliberately making bad-faith commits--just the association of his advisor being one of the authors of the "hypocrite" paper.
But Kanjie Lu, Pakki’s advisor, was one of the authors. The claim that “ You, and your group, have publicly admitted to sending known-buggy patches” may not be totally accurate (or it might be—Pakki could be on other papers I’m not aware of), but it’s not totally inaccurate either. Most academic work is variations on a theme, so it’s reasonable to be suspect of things from Lu’s group.
As Greg KH notes, he has no time to deal with such BS, when suggested to write a formal complain. He has no time to play detectives: you are involved in a group that does BS and this smell like BS again, banned.
It shouldn’t be up to the victim to sort that out. The only thing that could perhaps have changed here is for the university wide ban to have been announced earlier. Perhaps the kernel devs assumed that no one would be so shameless as to continue to send students back to someone they had already abused.
The person in power here is Greg KH. It seems like he can accept/reject/ban anyone for any reason with little recourse for the counter-party. I'm willing to withhold judgement on these allegations until the truth comes out. Seems like many here want retribution before any investigation.
You realise that the evidence is already there plain for everyone to see. It's even laid out in the email thread we're commenting on. This sort of linguistic weaseling doesn't help anyone, least of all folks who may not understand the entire context of what went down here (which is much worse than it might seem on the face of it).
In an evolving situation, I hesitate to immediately accept something as fact just because its in a comment. I don't believe in online "mob justice", nor in guilt by association.
You seem convinced, I am not. And as such, we just have a disagreement, no biggie. :)
>This sort of linguistic weaseling doesn't help anyone, least of all folks who may not understand the entire context of what went down here (which is much worse than it might seem on the face of it).
There's only one way the kernel dev team can afford to look at this: A bad actor tried to submit malicious code to the kernel using accounts on the U of M campus. They can't afford to assume that the researchers weren't malicious, because they didn't follow the standards of security research and did not lay out rules of engagement for the pentest. Because that trust was violated, and because nobody in the research team made the effort to contact the appropriate members of the dev team (in this case, they really shoulda taken it to Torvalds), the kernel dev team can't risk taking another patch from U of M because it might have hidden vulns in it. For all we know, Aditya Pakki is a pseudonym. For all we know, the researchers broke into Aditya's email account as part of their experiment--they've already shown that they have a habit of ignoring best practices in infosec and 'forgetting' to ask permission before conducting a pentest.
From his message, the ones that triggered his anger were patches he believed to be obviously useless and therefore either incompetently submitted or submitted as some other form experimentation. After the intentionally incorrect patches, he could no longer allow the presumption of good faith.
It doesn't matter. I think this is totally appropriate. A group of students are submitting purposely buggy patches? It isn't the kernels team to sift through and distinguish they come down and nuke the entire university. This sends a message to any other University thinking of a similar stunt you try this bull hockey you and your entire university are going to get caught in the blast
https://news.ycombinator.com/item?id=26887670&p=2
https://news.ycombinator.com/item?id=26887670&p=3
https://news.ycombinator.com/item?id=26887670&p=4
https://news.ycombinator.com/item?id=26887670&p=5
https://news.ycombinator.com/item?id=26887670&p=6
https://news.ycombinator.com/item?id=26887670&p=7
(Posts like this will go away once we turn off pagination. It's a workaround for performance, which we're working on fixing.)
Also, https://www.neowin.net/news/linux-bans-university-of-minneso... gives a bit of an overview. (It was posted at https://news.ycombinator.com/item?id=26889677, but we've merged that thread hither.)
Edit: related ongoing thread: UMN CS&E Statement on Linux Kernel Research - https://news.ycombinator.com/item?id=26895510 - April 2021 (205 comments and counting)