Real world systems are complicated. In theory, you could do belief propagation to update your beliefs through the whole network, if your brain worked something like a Bayesian network.
Natural selection didn't wire our brains to work like a Bayesian network. If it had, wouldn't it be easier to make converts to the Church of Reverend Bayes? /s
Alternatively, brains ARE Bayesian networks with hard coded priors that cannot be changed without CRISPR.
Isn't that Edwin T. Jaynes example just p-hacking? If only 1 out of 100 experiments produces a statistically significant result, and you only report the one, I would intuitively consider that evidence to be worth less. Can someone more versed in Bayesian statistics better explain the example?
> One who thinks that the important question is: "Which quantities are random?" is then in this situation. For the first researcher, n was a fixed constant, r was a random variable with a certain sampling distribution. For the second researcher, r/n was a fixed constant (approximately), and n was the random variable, with a very different sampling distribution. Orthodox practice will then analyze the two experiments in different ways, and will in general draw different conclusions about the efficacy of the treatment from them.
But so then the data _are_ different between the two experiments, because they were observing different random variables -- so why is it concerning if they arrive at different conclusions? In fact, the _fact that the 2nd experiment finished_ is also an observation on its own (e.g. if the treatment was in fact a dangerous poison, perhaps it would have been infeasible for the 2nd researcher to reach their stopping criteria).
I think the point is that the different planned stopping rules of each researcher--their subjective thoughts--should not affect what we consider the objective or mathematical significance of their otherwise-identical process and results. (Not unless humans have psychic powers.)
It's illogical to deride one of those two result-sets as telling us less about the objective universe just because the researcher had a different private intent (e.g. "p-hacking") for stopping at n=100.
_________________
> According to old-fashioned statistical procedure [...] It’s quite possible that the first experiment will be “statistically significant,” the second not. [...]
> But the likelihood of a given state of Nature producing the data we have seen, has nothing to do with the researcher’s private intentions. So whatever our hypotheses about Nature, the likelihood ratio is the same, and the evidential impact is the same, and the posterior belief should be the same, between the two experiments. At least one of the two Old Style methods must discard relevant information—or simply do the wrong calculation—for the two methods to arrive at different answers.
If you have two researchers, and one is "trying" to p-hack by repeating an experiment with different parameters, and one is trying to avoid p-hacking by preregistering their parameters, you might expect the paper published by the latter one to be more reliable.
However, if you know that the first researcher just happened to get a positive result on their first try (and therefore didn't actually have to modify parameters), Bayesian math says that their intentions didn't matter, only their result. If, however, they did 100 experiments and chose the best one, then their intentions... still don't matter! but their behavior does matter, and so we can discount their paper.
Now, if you _only_ know their intentions but not their final behavior (because they didn't say how many experiments they did before publishing), then their intentions matter because we can predict their behavior based on their intentions. But once you know their behavior (how many experiments they attempted), you no longer care about their intentions; the data speaks for itself.
Well no because it’s talking about either a fixed sample size or stopping when a % total is reached. Neither imply a favourable p-value necessarily.
I think the author means to say that it’s two methods incidentally equivalent in the data they collect that may draw different conclusions based on their initial assumptions. Question is how do you make coherent sense of it.
At level 1 depth it’s insightful.
At level 2 depth it’s a straw man.
At level 3 depth, just keep drinking until you’re back at level 1 depth.
> The other ... decided he would not stop until he had data indicating a rate of cures definitely greater than 60%
I believe that "definitely greater than 60%" is supposed to imply that the researcher is stopping when the p-value of their HA (theta>=60%) is below alpha, so an optional stopping (ie. "p-hacking") situation.
Yes, the possible programs are enumerable, and you can start searching with the least complex programs and work your way up in complexity. Once you find a program that explains the available data, you cannot guarantee it will continue to explain possible future data, unless, like you mention, you constrain the program space to a finite set. What you're describing is generally how people make models of the external world.
Just for the sake of argument, I can name some an entire field of science that was invalidated in light of genetic & neuroscience evidence: phrenology. At the time, it was the newest advancement in the gleaming era of scientific Enlightenment. It just happened to justify colonial policies of that time. Now, a couple hundred years later, we're walking back on a widely supported but misguided "scientific" field.
Are those two events connected? They both happened in the same decade and received some press. That's all I can come up with. And both things you mention -- LGBT rights and class consciousness -- have been around for a while.
These arew for racism, but similar hockeysticks with the exact same timing exist for other social justice topics. Media, as one, started harping on identity politics and never let their foot off the gas.
If you're just saying that rainbow capitalism is in alignment with capitalists' interests, and anti-capitalism movements are not, then that makes sense.
Yes. Social justice movements are perfect channels for champagne socialism. The activists get to fight and fight and fight and the problem never goes away. They get to feel like they're such good people and have no need to concern themselves with whether their methods work. (Their allegiance is to their methods first, ostensible causes decidedly second)
I don’t know if it’s fair to say that revolutionary wealth redistribution movements, like Occupy, are failing because everybody is distracted with social justice movements. The Left is a coalition for social justice and also for wealth redistribution, to varying degrees. That’s why I’m saying that I’m not seeing a direct connection where a success in one area subtracts from progress in another.
If you think about it, those lives actually were saved. People either didn't play the video game, or they spent a moment in the loading screen in quiet contemplation.
To play devil's advocate we should balance those out with the impatient gamers who acted out aggressively (e.g. the 100th time proceeded to kick a PC, smaller family member, throwing a mouse across the room, etc) or consumed more unhealthy snacks/drinks :(
I personally was often frustrated waiting several minutes, thankfully I don't have any violent tendencies, snacks on the other hand... :/
The tweet is in response to a preliminary paper [1] [2] studying text found in images generated by, e.g., "Two whales talking about food, with
subtitles." DALL-E doesn't generate meaningful text strings in the images, but if you feed the gibberish text it produces -- "Wa ch zod ahaakes rea." back into the system as a prompt, you would get semantically meaningful images, e.g., pictures of fish and shrimp.
I think the tweeter is being a bit too pedantic. Personally I spent some time thinking about embeddings, manifolds, the structure of language, scientific naming, and what the decoding of the points near the center of clusters in embedding spaces look like (archetypes), after seeing this paper. I think making networks and asking them to explain themselves using their own capabilities is a wonderful idea that will turn out to be a fruitful area of research in its own right.
If DALL-E had a choice to output "Command not understood", maybe we wouldn't be discussing this.
Like those AIs that guess what you draw, and recognize random doodling as "clouds", DALL-E is probably using the least unlikely route. That a gibberish word is drawn as a bird is maybe because it was "bird (2%), goat (1%), radish (1%)".
That's extremely optimisic. When faced with gibberish, the "confidences" are routinely 90%+ as with "meaningful" input.
It's almost as-if its an illusion designed to fool, we, the users.. by only providing inputs meaningful to us, we come to the foolish idea that it understands these inputs.
This is a good point. The fact that DALL-E will try to render something, no matter how meaningless the input, is a trait it has in common with many neural networks. If you want to use them for actual work, they should be able to fail rather than freestyle.
Especially since his results confirm most of what the original thread claimed. A couple of the inputs did not reliably replicate, but "for the most part, they're not true" seems straightforwardly false. He even seems to deliberately ignore this sometimes, such as when he says "I don't see any bugs" when there is very obviously a bug in the beak of all but two or three of the birds.
When I zoomed in, I felt only four in ten birds clearly had anything in their beaks, and in each case it looked like vegetable matter. In the original set, only one clearly has an insect in its beak.
Not really, he afterwards says that he was more trying to inject some humility. He really doesn't think this is measuring anything of interest. For the birds result in particular, see https://twitter.com/BarneyFlames/status/1531736708903051265.
If I read what that tweet says properly, the system ended up outputting things that were almost scientific nomenclature for the general class of items it was being asked to draw. There are probably many examples of "bird is an instance of class X" in the text but they are not consistent, and the resulting token vector is a point near the center of "birdspace".
Yes. Indeed, it seems to interpret a lot of nonsense tokens it doesn't recognize as though it's probably the Latin / scientific term for some sort of species it doesn't remember very well (keeping in mind that all these systems are attempting to compress a large corpus into a relatively small space). I think https://twitter.com/realmeatyhuman/status/153173904648934195... is best illustrative of this phenomenon.
So, it's certainly an "interesting" result in the sense that it shows how these kinds of systems work, but it's definitely not a language.
Why is it important if it's "a language" or not? What we're talking about are concept representations (nouns), not languages. But I think most people who read "DALL-E has a secret language" probably picked up on that because we're accustomed to the hype in machine learning naming things to sound like they are more profound and powerful than they really are.
It's important if it's a "language" because the original thread claimed that it was one (and indeed, a number of comments in responses to this article are still making that claim). You may argue that discovering how DALL-E tries to map nonsense words to nouns is independently interesting, and that's fine (I don't find it interesting personally though--considering it has to pick something, and the evidence that these spaces are not particularly robust when confronted with far out of sample input, I don't even think calling it a secret vocabulary would be accurate), but the authors should reasonably expect some pushback if they argue that this is linguistics.
> asking [neural networks] to explain themselves using their own capabilities
Exactly. This could be profound. I'm looking forward to further work here. Sure, the examples here are daft, but developing this approach could be like understanding a talking lion [0] only this time it's a lion of our making.
I think it’s more likely we can train two neural networks, one to make the decision and one to take the same inputs (or the same inputs plus the output from the first one) and generate plausible language to explain the first. This seems to correspond to what we dimwits consciousness and frankly I would doubt one system can accurately explain its own mechanism. People surely can’t.
It’s a fruitful area of research for sure, but there is a huge gap between “it invented pig Latin” and “it invented Esperanto/Lojban”. Referring to the first as inventing a language is very misleading.
