Imagine that shouting has no actual effect on performance, but it is traditional to shout at underlings when they do something particularly poorly. When your trainees screw up, you berate them - and afterwards they actually do tend to do better. Unfortunately, this is because the screwup is more often than not a random variation, and the improvement is due to the mean regression, not the treatment. Conversely, praising them when they do well (again, assuming no underlying effect) actually seems to worsen their performance.
Now, flip all your coins. Yell at the ones that got tails, praise the ones that got heads. Now, repeat your test. Wow, 50% of the ones you yelled at 'improved' and 50% of the ones you 'praised' got worse!
It is clear we need to yell at coins more.
The point of the article was not to say that exits are bad, only that to develop a cultural milieu of technology in a particular region, there is a need for at least some companies to stick it out and remain determined to become long-lived and predicated upon a chosen workplace culture.
If companies never do that, then a given region can't generate enough inertia to remain a competitive place, and this has all sorts of bad effects on the employment market in that area.
I thought Michael O. Church had a great extension of that article in his recent post . He argues that the broader short-term goals of VCs in general are misaligned with the fundamental value creation premise that undergirds the idea of start-up culture (or at least what was start-up culture when the term was new).
Still seems like a bad idea. Destroyed morale for the ones that do well but get scolded anyways, and encouraged laziness for the ones that do poorly.
Yet coaches yelling at youth league basketball players who just so happened to miss their first few free throws, but then regressed to some respectable mean by making an average number of free throws after that, will pat themselves on the back and believe that yelling was a form of constructive feedback.
The problem is that accidents are randomly spaced (assuming some level of road design and driver training) and putting in a teepee beside the road would probably show the same effect. Particularly so if the speed camera is not obvious, so is unlikely to have an effect on a driver unless they travel the route frequently and get fined.
Why would shouting have no effect on performance? People generally don't like being shouted at and often modify their behavior as a result. At the very least, a person being shouted at has to process the input.
Think about it, the shout interrupts whatever context existed before the shout. Future actions are now based on a context where there was an interruption instead of a context where there was no interruption.
The shouting has absolutely no information value ever.
What you describe might work for low motivation individuals. But they are hopelessly used to the shouting at the end of basic training. So the officers have to use threats instead.
>we could simply test if shouting has any effect on performance
Maybe we are using different definitions of performance, but what you want to test is essentially untestable. You'd have to be able to observe the exact same situation twice, once with shouting and once without. This is impossible.
But this isn't generally required to show close to certain probability.
If the same is large enough (doesn't have to be very large) and controlled for possible bias, then the test should reveal the effect of the difference between sets.
If performance cannot be measured then the sergeant shouldn't be yelling in the first place. Performance is set at a very clear standard, if you underperform consistently and you then perform at the standard or above, you've improved your performance.
And I agree it may have other effects, but what we're interested in is the desired result of the sergeant.
And yelling may be effective, but without testing it we shouldn't advocate either way.
90%+ of published outcomes can be invalidated by simply looking at the published data. If you want some chuckles, read blog posts by Al Lewis ripping on research publised by companies touting their own performance. He's acerbic and condescending but also, mostly, correct.
I told a friend of mine who's into alt.med once that yes I think a lot of alt.med is bunk but percentage wise it's probably not that much more bunk than "mainstream" medicine.
It's easy for emergency medicine. Customer comes in with bullet wound, leaves alive and healthy and without bullet wound. But outside of domains like that it's crushingly hard to tell the difference between treatment and noise. Compounding the fact is that humans aren't a standardized item that can be compared against reference performance metrics. We're all different and our genome and environment are constantly changing.
Oh yes it is!
First of all, the actual effort is there in real medicine, while alternative medicine just doesn't care about evidence.
Also, most real drugs work as intended. The fringe cases get emphasized a lot, but still, the basic things, like antibiotics, painkillers etc. work very well. Medicine has advanced A LOT in the last 100-150 years.
Fake medicine, like homeopathy, is nothing like this. Don't mislead people.
Throw in a bit of pharma companies funding their own studies and I don't think the situation is as black and white as you say.
Preachy overconfidence in the clear superiority of enlightened scientific medicine actually bolsters public support for alt.med as a reactionary movement.
You're not comparing the level of scientific rigor in mainstream medicine to the level of scientific rigor in alternative medicine. You're comparing it to the level of scientific rigor that we wish existed in a perfect world. Are you also willing to measure alternative medicine against the same standard so that they can be compared on an equal footing?
P-value fishing is hardly the exclusive province of mainstream medicine. And in the "funding their own studies" department the most notable difference between mainstream pharma and alt med is that the mainstream pharma companies are legally required to disclose this fact (and preregister their studies in order to discourage fishing, too), while in alt med it's standard practice to use cute little financial engineering tricks to try and hide where the funding came from.
* But I don't hate lying on occasion.
Is fat good for your or bad for you this week?
What's more suspicious is medicine that people use just because they're in a culture of taking medicine for everything. Reliefs for colds, coughs, mild pain, sore throat, psychological problems, etc. These illnesses have a strong get-better-anyway effect so it's very easy for people to believe that whatever they took cured them. Don't believe me? Go to China and you'll see people taking herb drinks for the same illnesses and having just as much faith in their effectiveness. Other popular cures include drinking warm water and "growing a pair".
What's curious with your list is that one is a disease ("cold"; the rhinovirus) while the others are symptoms ("cough, mild pain, sore throat"), and the other is an entire class ("psychological problems"). It's hard to respond to what you said because you called them all "illnesses".
I think you are correct in that many people confuse palliative treatment/medicines (eg, cough drops, warm water, gargling with salt water) with curative treatment/medicines. However, in doing so I think you've changed the topic from the differences in medical vs. alt. med., as measured in patient outcomes, to the differences in how patients subjectively view the different forms of treatment.
Here's an excerpt:
>We included randomised placebo trials with a no-treatment control group investigating any health problem
That article's more recent (2010) link supports its conclusion, which wouldn't need to be stated unless it contrasted with what it calls “pharmacological folklore” from the original publication, which the author says he “took literally” for many decades after.
The point being that the 1955 version of the idea is currently much more widely known than modern no-treatment comparisons, and most people don't realize that its research basis is so flimsy.
This is just wrong, the following procedure would be much more convincing than a RCT:
Decide what you are measuring, collect group of people A, measure it, give people placebo, measure it again, record results. Then repeat under different circumstances with a different group of people B. Maybe even go back and do A again. Are you getting consistent measurements? Good.
Now come up with an explanation for why the results have that distribution, sources of variation, etc. Use that to quantitatively predict what should be seen a new group of people C. Now go check group C. Did it match the predictions? If so, good, keep at it.
Make a prediction for group D. Do the group D results match? If so, you are probably interpreting the results right.
The get-better-anyway effect is too vague. Say theory A predicts value x is in the range [3.1,3.3] and you observe x=3.36 +/-.1. Also, there is a theory B that predicts x>0. Theory A has been much more severely tested so is supported by the evidence. Theory B, meh.
I don't understand how more studies testing theory A and getting 25%, while excluding theory B from testing, can show that A should be preferred over B. The rationale that A has been tested more thoroughly and therefore has a smaller confidence interval sounds circular.
That is not a theory. You have to come up with an explanation (a theory) for why it should be that value. Then from that theory you deduce predictions for what it should be under other circumstances. If the predictions are consistently close to the observations it indicates you are onto something.
I expect it to be much easier to come up with a statistical prediction for the get-better-anyway effect (regression to the mean), with more specificity than x>0. It would be needed regardless, in order to exclude the null hypothesis.
There is no reason to ignore RCT results, but that is not the only way to get the necessary data (which is what I disagreed with). An RCT is also not sufficient on its own, you need the theory that explains the results as well.
Perhaps when you say in the first post "give people placebo", you mean "give the placebo to some subset of the individuals leaving the remainder untreated"? I agree that you can often draw useful conclusions from an observational study as long as some comparable individuals receive each treatment, and as long as you can control for the initial differences between the treated and untreated. But if you are saying that you don't need to look at the results for the untreated group at all, then I think you are mistaken.
I meant just placebo. I am using predictive skill of the theory behind the explanations to distinguish between them, this is common in physics/astronomy/etc. As noted by asherer, placebo effect has been studied for a long time in the manner of seeing how two groups differ. There is nothing wrong with RCTs, but does near sole reliance on them (and their observational analogue) appear to have lead to cumulative knowledge and growth of understanding?
A really good source for all of us to follow about the latest research on placebos in human medicine is the group-edited blog Science-Based Medicine, which is edited by several active medical researchers and also includes lawyers, pharmacists, and even a reformed chiropractor among its contributors. The blog includes many informative posts about the placebo effects observed in human medical research that help illuminate issues we all discuss a lot here on Hacker News.
Some of my favorite recent articles on placebo effects from Science-Based Medicine include "Are Placebos Getting Stronger?" (21 October 2015) by Steven Novella, a neurologist; "Placebo by Conditioning" (29 July 2015), also by Dr. Novella; "Should placebos be used in randomized controlled trials of surgical interventions?" (25 May 2015) by David Gorski, a surgeon and cancer researcher; and "Placebo, Are You There?" (24 February 2015) a translation by a French-language article, translated by Harriet Hall. All go a long way toward explaining just what has been shown, and just what has not been shown, by previous research on placebo effects in human medicine.
"Placebo medicine" has so far only been shown to have any effect at all on self-reported subjective patient symptoms such as pain and nausea that ebb and flow in the natural course of untreated disease. If you broke your arm, you wouldn't seek placebo treatment from your doctor, but actual effective treatment. If you have an injury that causes chronic pain (as I do), you are best off looking for the best available medically verified standard treatment, and not looking for any kind of placebo treatment.