Ask HN: Are there people who make a living from just web scraping?

[chirau]

I'm curious to know if there are people who lie strictly on the scraping part of data pipelines. Is this an actual possible career specialty or it is just a skill that is needed across certain roles but not standalone?

I know it's part of many other roles, but on it's own, are there people who live off of this? Would be really interested in hearing their job descriptions

[dave_sid]

Google makes a decent income from it

[benjaminjosephw]

There are a number of companies who use scraping as a core data source for their services. Try looking at the job descriptions for companies like that but be aware that job titles and responsibilities can vary massively.

Two UK based examples I found: Skyscanner has a role for a Senior Data Engineer[0] and OpenCorporates has a role for a Data Integration Developer[1].

[0] -https://www.skyscanner.net/jobs/job/2768508/

[1] - https://jobs.opencorporates.com/o/data-integration-developer...

[rl1987]

Well there are entire companies that primarily work on web scraping, e.g. Zyte (formerly Scrapinghub) and Scrape Hero.

For example job description see: https://apply.workable.com/zyte/j/6F4A3254BF/

Also it is a perfectly valid niche for specialisation as a freelancer.

[arkitaip]

If you can scrape together useful lead data you can sell it for marketing purposes. There are also people who scrape and sell sales data for sellers on e-commerce platforms like Amazon or Ebay.