Hacker News new | past | comments | ask | show | jobs | submit login

I read it as: Your stack is so complex that:

- you will not have it running 10 years from now

- I am not able to make a snapshot of the content using tools like curl




Weird how the conclusion is that the sites are wrong, not that curl is an insufficient tool for indexing the modern web.

Google search hasn't died because sites use JS.


> Google search hasn't died because sites use JS.

I believe it's the other way around. Websites use just as much JS as they can get away with, without sinking in Google results. If Google never ran JS in the first place, most websites wouldn't depend on it.

And now every crawler has to keep up with Googlebot's computing power if they want to archive the web.


But it shouldn’t require such a complex js execution engine when the end result is something that is equivalent to what used to be retrievable with a simple curl request.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: