Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I don't trust Claude as personal assistant yet because too many websites block it from reading their contents, leaving Claude to guess and rely on very indirect sources. Performing actions online is even worse, with CAPTCHAs and whatnot.

It has made enough mistakes in this area that I now have a standing instruction (the only one) to always loudly warn whenever the official source for something couldn't be fetched.

 help



I have it use `playwright-cli` which runs a headless Chrome session, with relatively structural representations of websites without the HTML bloat, plus image viewing options. The builin web fetch tool indeed seems blocked. Headless Chrome is just Chrome, you can't block it. I think even the "Are you human" checkboxes are okay, since LLMs are good at coordinate guessing and initiating clicks.

You should consider using something like Exa or Firecrawl.

How do they get around blocks? Residential proxies?



Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: