Yes, you're right. Esp your third paragraph -- having someone demonstrate that they can repeatedly succeed is the surest proof. On the other hand, you can try to analyze process, and guess at if that process can reasonably produce results -- but that's not really an easier problem.
Let me paraphrase a story from the book that addresses Buffet. Taleb compares people playing the market to people flipping coins.
If everyone in the US flipped a coin once a morning, after about four weeks, you'd expect to have maybe a thousand people who'd flipped all heads. You could go out and interview these people, find out what their 7 highly effective habits are, whatever, but you couldn't actually be sure to be learning anything about what's "really going on."
On the other hand, if after four weeks, you found that all the people who flipped straight heads were clustered around Omaha, and all reported taking their coin-flipping-lessons from one guy in the town, then it might behoove one to look a bit closer.
In both cases, you can study "process", but without an understanding of the phenomena, you might just be spinning your wheels, coming up with correlations rather than causations.
Let me paraphrase a story from the book that addresses Buffet. Taleb compares people playing the market to people flipping coins.
If everyone in the US flipped a coin once a morning, after about four weeks, you'd expect to have maybe a thousand people who'd flipped all heads. You could go out and interview these people, find out what their 7 highly effective habits are, whatever, but you couldn't actually be sure to be learning anything about what's "really going on."
On the other hand, if after four weeks, you found that all the people who flipped straight heads were clustered around Omaha, and all reported taking their coin-flipping-lessons from one guy in the town, then it might behoove one to look a bit closer.
In both cases, you can study "process", but without an understanding of the phenomena, you might just be spinning your wheels, coming up with correlations rather than causations.
TL;DR -- causal mechanisms are hard to find.
You should read the book :)