Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Where can I find a text file with a set of representative URLs?
1 point by brad0 on Sept 21, 2018 | hide | past | favorite | 2 comments
My google-fu is failing me today.

I'm looking for something similar to an a list of phrases but for URLs.

A phrase list looks something like this: https://www.phrases.org.uk/meanings/phrases-and-sayings-list.html

Is there a standard text file with various URLs of random web sites?

Ideally some URLs would have the path, query and fragment components.




With the chrome extension Link Klipper you can export all the links on a page to csv, then make whatever kind of list you want.


I'm looking for more of a large set of links (say 10,000) that are representative of a wide range of URLs used over the web. Theoretically I could use a web scraper but it's better to see if there's already something out there first.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: