Hacker News new | comments | show | ask | jobs | submit login

Is all that HTML? Spiders generally only care about HTML -- they're not going to download the js and css and images and flash and whatever.



8 years ago, maybe. Today all search engine spiders pull down and execute javascript & css as well just so that they can index a real user's experience.


The better ones may, but perhaps not for all pages. Generally, when viewing logs I see search engine spiders make a single request, for only the html content and not js or css.


duh, I didn't think about that :) HTML is way smaller




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: