

Could you make a crawler that crawled S3? Does it already exist? - niels_olson

We are aiming to handle academic lecture video, slide decks, handouts, class notes, etc. The professors aren't terribly worried about it, but they want to know. I really don't know what S3's security policies are. Any experience with this? Anyone?<p>On a related note: do use use one-time-use URLs to privatize S3 content? If so, how? Do you reach in and expire the links at the same time as the session cookie expires?
======
wmf
S3 isn't special; it's just another part of the Web. If you are crawling, obey
robots.txt.

