Hacker News new | past | comments | ask | show | jobs | submit login
Only positive reinforcement for researchers in some fields (columbia.edu)
77 points by thetan on Nov 23, 2022 | hide | past | favorite | 52 comments

> It’s like people assume that critique cannot be helpful unless its somehow balanced with positives or provided in the context of some anonymous format or at a time when authors hace prepared themselves to hear comments and will therefore not be surprised if someone says something critical.

I think this is spot on. This is the norm, and it’s seeping in everywhere. You see it in the corporate environment too.

Perhaps it can even be generalized one more step: People increasingly expect to be able to react completely instinctively to stimuli, outside of a few clearly defined contexts such as the Q&A after a scientific talk where they will “brace themselves”.

>People increasingly expect to be able to react completely instinctively to stimuli

This is a good framing, and one that fits a cluster of related behaviors and emerging cultural norms I've witnessed. One of these is hardly original: the tendency to speak about what one "feels" versus what one "thinks". We should be careful, because "I feel" is often used as a synonym of "I think", but I am increasingly under the impression that American culture has become more accepting of unarticulated, unreasoned, stream-of-consciousness thought.

But this is now ratcheting up a notch. Not only is this kind of impulsive speech acceptable in the workplace (no big deal, in and of itself), there is now an emerging social norm such that we may no longer expect more. That, I think, is detrimental.

> I am increasingly under the impression that American culture has become more accepting of unarticulated, unreasoned, stream-of-consciousness thought.

Politicians speaking about grave matters will spout absolute gibberish with a straight face, and nobody calls them out or asks them to explain what they mean. I think it is because the individual words and phrases used, and the intonation they are spoken with convey the right feelings and emotions.

What seems to matter is that they are feeling appropriately happy, sad, angry, about something. After that, people don't care too much or maybe don't even realize that what is being said does not actually mean anything or provide them with any real information.

It's bizarre, but it must work. Lots of politicians do it on all sides of politics, I'm not talking about the thing where they say a lot of words without answering a difficult question, and it's not just occasional mis-speaking or conversation taken out of context, because they do this in their speeches too.

It may seep in in the corporate environment. But I cannot see it in academia.

A relevant presentation will be followed by a critical discussion. If there is no discussion or only positive comments, the presentation was is very likely irrelevant, or shallow at best.

I would consider an academic environment without critical discussion as broken.

I sometimes wonder if people have become more resistant to critique or if uninvited critique has become more common. But early twentieth century books like How to Win Friends and Influence People emphasise that people respond well to positive reinforcement, which makes me suspect that people have long given unsolicited criticism and long been annoyed by it.

Sure if your goal is to get something from a person then you should use another tactic. People have always had egos and taken criticism personally. Often it becomes an argument. But those arguments can still be very good for everyone else in the field to hear and for research progress.

In this article the problem was the new "weapons" people have created by which the person being criticized can automatically win that argument based on emotion basically.

> People increasingly expect to be able to react completely instinctively to stimuli

I think this is true. If we're doing hypothesis about human behavior, you might be able to argue that our methods of communication, always forever online, have spawned this approach as an adaptation to stress of having to embody more personas for more numerous interaction. ( You can take it a step further and question the origin of 'acting' out a persona but i belief that in itself is extremely valuable )

On a further tangent, i still sometimes think about an Asian person commenting on the traits of US society to be obsessed with behaving as 'your true self' everywhere and all the time, whereas their culture it was noticeably less so.

There probably is some reasonable etiquette that needs to be established. I'd probably feel blindsided if someone criticized my work in public w/o first talking to me about it. The counterargument is that -- it's a public paper, we should be able to criticize it in any venue we like. But in that world it's very easy for those with the loudest microphone to effectively kill research they don't (or the research of people they don't like).

In a small enough community, it seems reasonable to go to the researcher and say, "we're going to make this statement about that research -- do you have a comment". Almost like a newspaper -- both mediums are trying to present truth, rather than create a narrative, so that's probably why it's so similar.

> I'd probably feel blindsided if someone criticized my work in public w/o first talking to me about it.

Why, though? If you present your work in front of audience, it is extremely reasonable to expect the audience offer questions or criticism. If the mere fact of people criticizing your work after your presentation makes you feel blindsided, it’s simply a matter of adjusting expectations: you simply should expect criticism.

Then, if you expect criticism, but feel blindsided by the contents of the criticism, this means that you failed your job as a researcher. If someone can come up with a point after listening to your presentation for a few minutes, that you didn’t have thought of in advance and prepared answer for it, it means that you’re doing bad job, and should do better. You should actually thank the critic for bringing the issue to your attention, because in doing so, he helped you.

To be clear, I wasn’t talking about during a presentation. I was thinking after, for example on Twitter or in a magazine editorial.

The one place I’d be more OK with it is in another paper. But even then if the criticism is rather scathing, I think there is a courtesy to them reaching out and saying — “we’re planning on showing how your method was wrong and your results misleading for this reason in our forthcoming paper”.

Because once the argument goes public people tend to dig in their heels and true progress is not likely to be made.

Progress is not made because one person was defensive about using the wrong method? Wouldn’t the other 99% just ignore the person who “digs in their heels”?

The other 99% probably can’t tell who is who. And now rather than the researchers working together, they’ve both dug in to a public dispute rather than doing science. It’s human nature.

> You should actually thank the critic for bringing the issue to your attention, because in doing so, he helped you.

Heckling is not "helping." Nobody wants to be humiliated by public criticism they weren't prepared for, which may or may not itself be valid-- but a poor response from the presenter makes one look incompetent in front of their peers nonetheless.

Callouts are sadistic and unprofessional. That anyone should thank someone for punching them in the face as criticism for not seeing it coming is ludicrous. It's bully logic.

You very quickly went from "criticism" in the parent post to "heckling", "humiliated", "callouts", "sadistic", "punching them in the face", "bully". Criticism doesn't have to be any of those things.

I’ll go further and say that criticism is _none_ of those things and that conflating is tantamount to scaremongering with a straw man.

An academic must be prepared to defend his research. An academic must be willing to change their mind when information warrants it. A researcher must expect and even welcome queries, challenges, and deep questions. A member of the academic community must politely but rigorously criticise presented research or theories because science is based on cordial mutual doubt, that’s why we insist on independent replication and verification.

Play fair but don’t be deferential. And don’t tarry every critic with the epithets of heckling. It’s called academic debate, without it you have progress, just polite mutually-reinforcing deference that advances human knowledge and technology one iota.

I agree with you completely. What makes me sad, however, is that you describe the ideal of academia I had in my mind when I was entering academia, which simply no longer exists, if it ever had. Academia is not focused on the pursuit of the truth anymore, and we get to watch its value and reputation decay and die in front of our eyes.

Doesn't praising people create the same issue?

It's easy for those with the loudest microphone to promote the work of the researchers they like or are connected to, or research that confirms their preexisting beliefs. This risks making the field insular by excluding new voices with new perspectives and without such preexisting interpersonal ties.

I think that you need criticism to be able to overcome this issue: to allow disinterested parties to push back on praise given by biased sources, and to ensure that current researchers aren't always elevated above newer ones.

It doesn’t create quite the same issue because being criticized can make the outside voices not only unheard, but people may want to actively avoid even association.

One environment discourages research in certain fields while the other encourages it in certain fields. It’s similar, but not the same. And I think the former is worse.

Praise tends to cause people to seek association with those being praised, and those who are praised the most tend to be insiders with lots of friends in the field. This effectively does the same thing, causing people to shun outsiders and instead try to attach themselves to those who are already the most prominent members of the field.

I don't think that it really encourages research to create an environment that makes people uncomfortable voicing criticism of those with the greatest power and influence.

They won’t be shunned. That’s the point. In the other model they would be. But in this model people will actively seek out breakthroughs to get praise.

Everyone says they want constructive feedback. But most people actively avoid it. If I told you as a young scientist you can go into a field where you will never get praised, just criticized or Vice versa where do you think even most of the top young scientists would go?

No one wants to have their career be “I think I did good work, but pretty much everyone just found issues so it never really made traction”.

If I asked someone whether they wanted to go into a field where they would never feel able to criticize people, do you think they would want to join?

Suppose, for instance, that somebody in HCI proposed a new system that would be horribly inaccessible to people with disabilities, to the point that widespread adoption would cause actual harm. Wouldn't it be best for society if the rest of the field felt empowered to criticize their work?

Furthermore, people often feel pressured to praise the works of their friends and colleagues; in doing so, they elevate the most connected members of the field above those doing breakthrough research.

The etiquette is if someone catches you in a real mistake, you should thank them (yes it can be hard, etiquette isn't always the same as reacting naturally). After all they could have ignored your existence but bothered to read your work.

If you think they are wrong you are expected and encouraged to respond publicly with why. Perhaps many people misread your work just same as that person so would benefit from the debate.

EDIT: and by the way, if you think they are wrong you should thank them for the same reason.

Mistakes are common in doing any non-trivial work. On HN coding is a great example. We have code reviews and it is standard practice to have issues raised there. It’s less common for someone to find and issue at CR and send it to the CEO to call an all hands to have him point out your bug to the company. In fact it would seem absurd to even raise this as a hypothetical. That is more the equivalent of what we’re discussing here.

Would you thank your colleague for raising your bug to the CEO or posting a LinkedIn article about your bug before talking to you about it?

Coding is not a great example. It can be fixed silently and the error disappears, unlike published claims. And I'd say you're making an analogy that is extreme. The corrections we are talking about are shared in the same forum the error was shared. If you had presented the mistake to the CEO or on linkedin then it would be appropriate for the error to be pointed out there too. Otherwise the audience will remain misled.

If it's some minor code bug from a typo or something, your embarrassment should be correspondingly tiny. And still the right thing to do is say "thanks" or "nice catch".

My biggest learning opportunities in science have come from watching two intellectual heavyweights engaged in robust debates. Unfortunately, it seems increasingly common for people to infer mental or emotional states of their fellow debater, which derails good debate.

Reasonable etiquette in science is critiquing the expressed ideas and methods rather than the person (or what has been inferred).

I definitely agree this is an etiquette rules discussion - and that maybe this change has to do with how loud the loudest microphones can get. Maybe pre-internet, the only people who would see criticism would be other experts in the field. But now maybe even things offline are more public. I've heard many academics use twitter - which means a critique in public could easily become a tweet, and then viral in the community, etc.

I do really like that newspaper comparison, especially because it could prevent misunderstanding what an author was trying to say.

Twitter is poison for academic debate. It is already impossible to discuss a complex matter over email—most nuance gets lost. Twitter is much worse, a) because of the character limit, b) because of the crossfire that is a wild mix of relevant and irrelevant comments, plus pure trolling.

It’s got to the point where if I find myself in cordial debate with so,embody whose thesis I disagree with (in 280 character chunks at the time) I thank them for the debate.

And that’s the thing: if you go into a conversation only to ‘right’ the other person’s views and are unwilling to revise/question yours then you’re not debating, you’re preaching… and there’s way too much of that already. Furthermore some of people are very ‘polite’ (in that they let you expound your views) but I’ve noticed many just wait for an option to lunch into their monologue. That isn’t debate either. It’s socially less jarring but in a way intellectually dishonest.

On the other hand, perhaps we could sit through half a book of refutation of refutation (cf. Conjectures and Refutations, literally the entire second half of the book), sometimes with not quite an impersonal tone.

This is very unlike my experience (PhD Computational optimization) where aggressive criticism was too be expected and anticipated during a presentation. I remember running "pre-mortems" before talks with my supervisor to prepare for as much criticism as we could possibly predict.

Engaging with critique is an invaluable skill that is relevant way beyond academia. They key is to never take it personal, otherwise one has lost the debate already.

Understanding the main angle of critique on the spot is hard enough, coming up with a rebuttal even harder, especially when the critique comes from a senior researcher.

Accepting points of critique that are valid as opportunities to improve the research, or at least as limitations to the generality of the findings reported is highly valued.

I always tended to think that a PhD is awarded for the ability to accept and counter critique towards work that one has spent years of their life with.

It seems like academic criticism can be binned into a few categories:

1. legit criticism, e.g. you missed something or did something incorrectly

2. reveals an opportunity to communicate better on your part

3. self-promotion by the criticizer

1 and 2 are directly valuable. Distinguishing between 1, 2, and 3 is a skill, especially since 3 is so common.

I never encountered the mentality described in the article during my physics PhD, either. Though it was stressful at the time, having committee members who were known for asking tough questions was invaluable as a motivator to learn more about underlying theory. It's of course very important that the process be conducted respectfully and constructively, and I had the fortune of being in a department with a healthy culture in that regard.

All the people I know who were not so fortunate experienced unduly harsh, personalized, and non-constructive criticism. I'm not sure whether or not that's worse than never being critical, but they are kind of equivalent in that they create barriers to students engaging with the science.

What is described in the article is not at all ubiquitous as some commenters are implying, and afaik is essentially unknown in the hard sciences and math.

lots of places still do hazing, even at the high levels.. honestly

but I found this approach quite fitting to the environment. The math didn't care about my emotions, or how much effort I put into the work. The criticism was (most of the time) mathematically valid, and was delivered extremely efficiently in its brutality. We could argue (politely) but without worrying about hurting egos. It forced me to learn to become emotionally detached from my specific work, allowing me to accept mistakes and take corrective actions.

There is a difference between hazing and aggressive questioning. Hazing is done entirely to maintain power structures through humiliation, while aggressive questioning is done to improve the field by reducing the impact of emotion on rationality.

This seems worrisome. Sure, don't destroy a new PhD student giving their first talk in front of everyone for sport, but open and constructive criticism is the long conversation that moves science forward.

"HCI" as in "human-computer interface"? Or something else.

"not possible to be both an inclusive field and a field that embraces criticism."

Hm. Over time, will the field will stop producing useful new results?

A comment from an actor, years ago: "In New York, they tell you if you suck. In LA, they don't call you back if you suck. In SF, nobody tells you that you suck. So you suck forever."

> "HCI" as in "human-computer interface"? Or something else.

Human Computer Interaction: https://en.wikipedia.org/wiki/Human%E2%80%93computer_interac...

i.e., the sort of research that gets presented at CHI: https://chi2022.acm.org/

Tough love is sometimes the best thing you can be given, like a five foot tall player told he might want to stop trying to play basketball professionally.

Some journals encourage the submissions of comments to published papers. After all, a little controversy attracts traffic and gives both the critique and the critiqued paper more visibility.

But not all authors like being critiqued. This is even worse when competing interests are involved, e.g. patents, or an author is employed at a research-active company that sponsored that paper.

This is particularly bad when a paper is accompanied by press releases that dramatically oversell the findings. Unfortunately many researchers seem to read press releases and confuse their content with the actual science reported in the paper.

In that case it must be possible to critique a paper and set things right, if only to have a citable record for the next time a reviewer tries to shoot down a paper or grant based on a skewed understanding of prior research.

In my time in academia (math/cs), public criticism of other people's work was extremely rare. Not for the (somewhat strange) reasons listed in the article, but because it never ever reflects well on anyone involved.

I knew a researcher who publicly challenged some obviously bullshit research, and basically got absolutely nothing out of it, apart from a certain reputation. The correct thing to do is just ignore the bullshit. If you want to challenge good work with some gaps, there certainly are ways to do it without appearing critical.

Self awareness in criticism on both sides of the equation is really hard.

Some people derive ego out of criticizing, some are angry, spiteful, ideological, personal - and frankly, I think all criticism have a least small hints of all of these things.

But as someone suggested 'etiquette' with a hint of self-awareness, even as simple as 'Do I need to do this?' 'How can I make this about the work and not about either of us?' etc. might help.

Wrapped in a bit of 'tone shifting' to make it feel a bit nicer ... then debate ought to be encouraged.

On the receiving end, I really think that a bit of 'adversity training' would go a long way. I think people are incredibly resilient, and you can 'pretend expose' people to all sorts of stuff (even some personal attacks) and they will get used to it very quickly. In sports, even in the military, you see people develop certain kinds of resiliency that I think comes from doing challenging things, camaraderie, teammates chirping you when you get to big for yourself (or for any other reason). Confident people don't perceive questions as threats, even if they are reactionary in the moment.

Doesn't this likely sound like an unintentional Goodhart's Law? Our heroes are those who have never erred, and anyone who ever errs can never be great. Given that we have created this ideal, it is unsurprising that no one ever wants to set that "have erred" bit. And given that, it is unsurprising that anyone would want someone they like or respect to have the "have erred" bit set on them. And given that, it is best to avoid any discussion of error at all.

Perhaps, and I speculate, it is our culture of dunking on targets. My personal favourite is that I disregard anything regarding Colin Powell. Someone might say "Well, he was quite the X" or "he would have made a good Y" and I cannot resist but say "But the mindless clown will just repeat nonsense if you give him nonsense". Others might decide that errors I consider insignificant in others are the equivalent of Colin Powell's.

Consequently, it is important to never state that another has erred.

Famous example: Ragnar Granit studied color vision in the cat for 10 years. It turned out cats are colorblind. He carried on and later was awarded the Nobel Prize for related research.

Cats aren't really colorblind, they just have reduced color perception compared to humans, but still are sensitive to many hues. In fact that's exactly what his paper in 1943 on cat vision said: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1748-1716....

If the paper is public, and the critique is based in data or standard procedure, then just make the critique anonymously.

... Unless of course someone is critiquing it to gain some sort of notoriety.

If it's about the fundamental science, then make the critique on fundamental grounds and do it anonymously.

There’s a way to give criticism that is based on understanding and context, which can be kind a generous; many don’t master this approach and fall into fail modes of glib positivity or scouring negativity.

Suggest changing the title to "Only positive reinforcement [is permitted,] for researchers in some fields" for clarity.

“This is sometimes called “white knighting.” I discussed one such example a few years ago. The white knight in question (a senior faculty member, whom I’d never met, who taught at a different university) sent me some very angry emails.

It seemed that, from his point of view, I was breaking the rules by publicly criticizing published work. The whole thing seemed so ridiculous to me, but to him it was all about good guys and bad guys. So frustrating.”

We have a state sponsored religion now. That the priesthood of this religion does not tolerate critique is bog-standard throughout human history. So is the hysteria in response to heresy and public apostasy (Rogan, Gabbard, etc).

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact