I'm in the job search right now but I wrote my own little tool to scrape linkedIn.
It's a simple automation which searches through 10 pages of job postings and stores the results in a log file.
This alone is helpful because it helps me avoid getting distracted by all the non-job related aspects of linkedIn; click-bait headlines, cringe posters etc.
In addition it also filters out jobs that it has seen before. I found that the exact same job can be posted again with a different ID so I use a hash of the job description as the ID.
I also have filters for certain words like "7+" which I assume means 7+ years of experience which I don't have. It's not perfect but it works well enough. I hate reading a job posting which looks decent only to find hidden near the end that they want someone with over a decade of experience.
After a few days of using this tool consistently, I'll reach a saturation point which means that after going through the first 10 pages it will not find any new job.
That's not a bad thing. That's actually the goal. It means that when an actual new job is posted, my tool will help me see it through all the clutter.
The saturation point gives me a goal to reach every day. Instead of some arbitrary goal like sending out 10 applications or spending 1 hour every day, my goal is always to keep going until I reach the saturation point. This encourages me to be consistent and gives me a reasonable stopping point.
It's a simple automation which searches through 10 pages of job postings and stores the results in a log file.
This alone is helpful because it helps me avoid getting distracted by all the non-job related aspects of linkedIn; click-bait headlines, cringe posters etc.
In addition it also filters out jobs that it has seen before. I found that the exact same job can be posted again with a different ID so I use a hash of the job description as the ID.
I also have filters for certain words like "7+" which I assume means 7+ years of experience which I don't have. It's not perfect but it works well enough. I hate reading a job posting which looks decent only to find hidden near the end that they want someone with over a decade of experience.
After a few days of using this tool consistently, I'll reach a saturation point which means that after going through the first 10 pages it will not find any new job.
That's not a bad thing. That's actually the goal. It means that when an actual new job is posted, my tool will help me see it through all the clutter.
The saturation point gives me a goal to reach every day. Instead of some arbitrary goal like sending out 10 applications or spending 1 hour every day, my goal is always to keep going until I reach the saturation point. This encourages me to be consistent and gives me a reasonable stopping point.