Can you comment on the notion that Turnstile's primary goal isn't to keep bots out 100% but instead to slow them down to "human" speeds.
Asking because as a dev I hate when sites don't allow bots... however can appreciate that automation should be rate-limited. IOW, isn't preventing bot access actually an anti-pattern since rate-limiting is sufficient?
I see a lot of marketing which bashes Turnstile [detection] rates and tries to leverage this misunderstood nuance. And, it seems to be a dishonest point of contention but am willing to hear opposing arguments.
Cloudflare is really good at network bot detection. Rate-limiting is super helpful here, for example during DDoS attacks.
Our customers are a little different. They sometimes struggle with high-volume bot attacks (e.g. SMS toll fraud in ticketing marketplaces), but we specifically focus on online platforms that want to verify a human is on the other side of the screen. For example, survey pollsters and labor marketplaces want to stop a slow agent that can complete traditional CAPTCHA even if it's solving it a human speed
I see. I'll have to read the marketing more closely next time, lol. The cynic in me only notices the detection rate comparisons, which I'm sure the marketing folks don't mind much ;-)
It doesn't catch OpenAI even though the mouse/click behavior is clearly pretty botlike. One hypothesis is that Google reCAPTCHA is overindexing on browser patterns rather than behavioral movement
I think about this as a startup founder building a 'proof-of-human' layer on the Internet.
One of the hard parts in this space is what level of transparency should you have. We're advancing the thesis that behavioral biometrics offers robust continuous authentication that helps with bot/human and good/bad, but people are obviously skeptical to trust black-box models for accuracy and/or privacy reasons.
We've defaulted to a lot of transparency in terms of publishing research online (and hopefully in scientific journals), but we've seen the downside: competitors fake claims about their own best in-house behavioral tools that is behind their company walls in addition to investors constantly worried about an arms race.
As someone genuinely interested (and incentivized!) to build a great solution in this space, what are good protocols/examples to follow?
Great question! One of the core results of this paper was to explain this discrepancy. Basically, we found a 'mixture of theories' - a hybrid of prospect theory and expected utility theory, where people essentially arbitrate between one of the two decision-making mechanisms depending on the complexity of the gamble.
> Curious that you can "mix" PT & EU functionals (with perceptron) but not the corresponding "decision-making mechanisms"..?
Great push. We actually can't make any mechanistic claims from the data/math in this paper. From an ML prediction standpoint, we're mixing a PT and EU theory together. But to what extent that is the actual cognitive process we have to remain agnostic about. That being said, a reason this arbitration between EU and PT is intriguing is because there's a lot of work about arbitration between dual process models in psychology (System 1 and 2; model-free and model-based; labor versus leisure; etc.)
You can generalize theories of decision-making into broad functional forms and then apply gradient descent to find the best parameters for that functional form. For example, prospect theory is multiply a utility weighting function U(x) with a probability weighting function p(x). Kahneman and Tversky proposed one specific set of U(x) and p(x), but we can use autodiff to generate all.
Can you explain what a “differentiable” decision theory is? I understand, for instance, maximizing expected value (and taking a derivative to get a maximum), but I don’t understand how the concept of maximizing expected value could itself be made into a derivative.
Edit: Seems like a “differentiable theory” is just one that can be framed in terms of an optimization problem that can be solved by gradient descent. Is that right?
I think a common misconception of Moneyball is that it's about analytics. The broader lesson is that people need to systematically evaluate undervalued assets in sports/business etc.
One of the interesting 'post-Moneyball' stories is when old-school scouting methods came back onto the scene. People started overvaluing the new popularized statistics, and the market advantage was to combine the analytics and traditional approach in a cost-efficient manner.
The 2014/2015 Royals capitalized on this to some degree, picking up players who didn't strike out or walk much, at a time when players who walked a lot were at a super premium.
Some of the smarter teams in the NFL seem to be figuring out that maybe running backs aren't completely fungible, as has been the mantra for a while.
I think this will be a positive effect of the rise of AI agents. We’re going to have a much different distribution of automated vs human traffic and authentication/methods will have to be more robust than they are now
Happy to talk more details about PoH (disclaimer: I'm a cofounder and this is my YC S23 company)