Hello HN, I made this silly project over the long weekend. It’s pretty basic right now and the captchas are very easy. I plan to add captcha difficulty levels for link creators soon.
Unfortunately the example problems are simple enough to be solved with AI. As a test I ran two of them by CoPilot, and it solved them instantly. I like the idea, but would want something more difficult as a captcha since it is easy for bots but hard for a human.
Maybe a better approach would be to have a prompt at the top with unclear specifications, or some kind of riddle instead of a function name. It would also be good not to have a bank of problems, since someone could just pattern match on them, but to generate them automatically somehow.
This is a lot more interesting than finding traffic lights though, and the website looks well designed. Thank you for sharing!
Sure, but if you're capable of running CoPilot to write an isEven or reverseString function in JS, it's probably less effort to just write the functions then and there. And either way you're clearly the sort of person this captcha would be intended to allow through, I think.
If you could wrangle AI to solve this problem for you, I'm sure you wouldn't have any issues solving the captcha manually. Hence the CodeCaptcha still works!
You can have the function requirements in the instruction text. So instead of isNumberEven, have "write a function that returns whether a number is evenly divisible by two."
Copilot could absolutely solve the task given the instructions as a comment that's stated above. Unfortunately the gap between AI's capabilities and a task humans can solve quickly is super thin. You also have to constantly evade advancements in computer vision for the current type of captchas, such as FunCaptcha implementing swirls and animals in certain rotations.
Reminds me of one of my favorite quotes about trash in Yosemite.. "There is considerable overlap between the intelligence of the smartest bears and the dumbest tourists."
> Back in 1980s, Yosemite National Park was having a serious problem with bears: They would wander into campgrounds and break into the garbage bins. This put both bears and people at risk. So the Park Service started installing armored garbage cans that were tricky to open—you had to swing a latch, align two bits of a handle, that sort of thing. But it turns out it's actually quite tricky to get the design of these cans just right. Make it too complex, and people can't get them open to put away their garbage in the first place. Said one park ranger, "There is considerable overlap between the intelligence of the smartest bears and the dumbest tourists."
If the function name is an issue, call it “captchaTestFunction” and include a comment to the effect that “this function should act like a function called theRealFunctionName”. Though that would be easy to automate around if anyone cared to.
I noticed one of the challenges is "reverse a string." Can I just rant a little about how much I hate that as an interview question?
It's meaningless to reverse a string. Not just in the "there's no purpose to doing it" sense (very true) but genuinely in the "it literally isn't a defined operation" sense. If you've only lived in the nice insulated world of ASCII or a mostly-ASCII-like language, you might scratch your head - just put the letters in backwards order, right?
Well, what do you when you hit a Unicode joiner? Or a multi-byte emoji? Maybe your reversing scheme is clever and looks at "whole codepoints" or whatever. But then what happens when you normalize the "reversed" string? Or what about the modifier characters that affect the previous/next character - how to treat those? I've never been satisfied with anyone's answer to these questions, because the problem is invalid from the start. You can't "reverse" an arbitrary string, it's not a well-defined operation.
The fact that it most people think it's a well-defined operation is what makes it a useful question. A total newbie who doesn't know how to code will fumble. A junior engineer will get to the simple solution. A senior engineer will ask you to define it better. Etc... It's an effective tool for determining level because there isn't a limit on how much depth you can go into about it.
I'm jaded enough to imagine that if I ask someone to define it better and try to give a "senior" response that covered codepoints, an interviewer will write down "struggled with simple question".
Edit: That seems consistent with a statement made in another response.
There is a (slight, but important) difference between the syntactic meaning, and the semantic meaning.
you are correct that reversing a string gives back irrelevant things. (and you don't need to go that far: what does a reversed word mean?)
however, in the sense of a list of characters, the content is irrelevant.
> I noticed one of the challenges is "reverse a string." Can I just rant a little about how much I hate that as an interview question?
I find "reverse a string" a good interview question then! If the applicant got lost in considering all possible interpretations instead of just solving it how 99% of humans/engineers would understand it, then they will likely be unfit for working in a team and/or have poor communication skills.
You are weeding out all candidates who understand Unicode. This is exactly the sort of problem that a good engineer would keep an eye out for because it's almost certainly going to explode with edge conditions if you try to do it the "obvious" way.
Unless you're giving that problem and then hitting the input with a string that includes directional formatting characters. Because that's exactly what is going to happen in real life.
The only good thing about that question is at least you didn't ask them to casefold the string.
No, not all, probably not even many. There are lots of candidates who know Unicode who would just say “I’ll assume that we have just ASCII characters unless you think I should consider Unicode” and if you say the latter, then they can say “Well, I could reverse it by just ensuring the text appears right to left, or perhaps by reordering each grapheme cluster to the end, or something like that. Let me know if you want one of those, otherwise I’ll just write it assuming you mean A”.
The magic of engineering isn’t doing what you’re told. It’s understanding and solving a problem someone has told you about.
My org doesn’t need the guys who need exhaustive instructions to take the next step. It needs those who can make good Bayesian inferences, reason inductively, and figure out problems.
So yes, it does filter out people with lots of knowledge who nonetheless lack these skills.
Hey, listen, the way I see it is that you don't want to work with a company like us and we don't want to work with you. It's an eminently compatible situation.
Depending on the use case, though, you could just say that anyone who can use the debugger to figure out how to hack around the captcha passes the test :)
Edit: oh, I see, it submits the code to be evaluated on the server after it passes on your browser (but the above causes the server to 500 at you, so it just says "Passed! Submitting..." and gets stuck in that state). Seems a bit dangerous to trust the client to control what code runs on your server, but I suppose platforms like leetcode manage it so in principle it should be possible to do safely.
I hate all of these. In the current use case I don't care because for me all of the problems are trivial. But I worry about people abusing this idea with harder problems
This is a fun idea. I tried the demo a few times. msft copilot solved them all immediately. This won't be effective keeping bots out, but it may be good for turning away non-technical humans.
I think I stumbled across the smallest amount of characters needed to pass the first test. I placed my cursor just after 'return', began typing, and almost immediately the page unlocked. Baffled, I went back to see what I'd typed, and realized I'd gotten as far as
Years ago at green tech college hackathon my team built a captcha that requires users to correctly sort trash into recycling, compost and non-recyclable bins. Anything little bit more fun is better than mind numbing selecting traffic lights, boats and trains.
I would be surprised if CoPilot can reason numerically like this (understand "seventeenth bit" and "sixth bit" and generate the right code for that combination).
For python it seems to generate a more reasonable result at least:
# Swap the sixth bit and seventeenth bit of a 32-bit signed integer.
def swap_bits(x):
# Get the bit at position 6 and 17.
bit6 = x & (1 << 6)
bit17 = x & (1 << 17)
# Swap the bits.
x = x ^ (bit6 << 17)
x = x ^ (bit17 << 6)
return x
The function doesn't do the comment says it does.
The code to "Swap the bits." just turns the bits on.
>>> def swap_bits(x):
# Get the bit at position 6 and 17.
bit6 = x & (1 << 6)
bit17 = x & (1 << 17)
# Swap the bits.
x = x ^ (bit6 << 17)
x = x ^ (bit17 << 6)
return x
>>> x6 = (1 << 6)
>>> f"{x6:b}"
'1000000'
>>> s6 = swap_bits(x6)
>>> f"{s6:b}"
'100000000000000001000000'
Here's one that correctly swap bits. It could be made more concise.
>>> def swap_specific(x,i,j):
def get(x,p): return 1 if x & (1 << p) else 0
def set(x,p): return x ^ (1 << p)
def clr(x,p): return x & ~(1 << p)
bi, bj = get(x,i), get(x,j)
x = set(x,j) if bi else clr(x,j)
x = set(x,i) if bj else clr(x,i)
return x
>>> f"{x6:b}"
'1000000'
>>> b6 = swap_specific(x6,6,17)
>>> f"{b6:b}"
'100000000000000000'
I guess CoPilot has seen bit swapping in its Python training input but not in its Go training input.
The Python code is wrong because the 17th bit is shifted up, not down. Also, the bits are shifted by the wrong amount, not up/down by 11 (= 17 minus 6), but up by 6 and up by 17. What a joke.
Not only that, even if the shifts were correct, it's simply xoring the bits. The swap is completely wrong.
Yeah, it seems to be pretty heavily trained on Python. It's honestly still (and should be used as) a glorified autocomplete, which is pretty useful from time to time.
With just that prompt, Copilot keeps writing a comment about the function but never actually writes the function. Prompting to actually write the function by starting it with `func` gives:
// A Go function to swap the sixth bit and seventeenth bit of a 32-bit signed integer.
func swapBits(x int32) int32 {
return ((x & 0x0F) << 28) | ((x & 0xF0000000) >> 28)
}
And there you have it, the difference between real intelligence and regurgitation.
This is the kind of numerically specific coding that could be the basis of a CAPTCHA that CoPilot can't solve. Sixth bit, sixth byte, seventeenth bit, seventeenth byte, etc.
I think it’s great! I think most people either ignore your statement or decided to interpret it in the wrong way.
For everyone else:
The author made it clear that the purpose is weed out non-engineers. Practicaly there may not really be a use case there but it was never designed to replace captcha, most people wouldn’t be able to access the link, and anyone using or purchasing the use of a bot farm already meets the captcha requirements (albeit with extremely unnecessary additional steps).
I wonder if you could use advent of code or Project Euler style challenges that have a multitude of problem/solution pairs to bootstrap support for languages besides JS? The difficulty would be perhaps a bit high, but not a bad starting place.
This is actually brilliant. Reminds of how Headlands Technologies solicits applications: they ask for a simple C++ program printing a number. There's a C way and C++ way.
If you want to limit something to programmers, I think one idea would be to ask them to run a command, giving a linux and a windows possibility, and copy/pasting the result.
Maybe a better approach would be to have a prompt at the top with unclear specifications, or some kind of riddle instead of a function name. It would also be good not to have a bank of problems, since someone could just pattern match on them, but to generate them automatically somehow.
This is a lot more interesting than finding traffic lights though, and the website looks well designed. Thank you for sharing!