
Show HN: Where is “Who is hiring?” hiring? - manlio
http://whereis-whoishiring-hiring.me
======
zedpm
Nice name and clean interface!

I'm glad to see Remote as a location, but due to the free-form writing in the
original posts, there are errors. For example, "Haskell dev at Standard
Chartered Bank" is listed under Remote, but the post itself says "Remote work
isn’t an option". The post for Button similarly doesn't allow remote, but uses
"Remote - no" to convey that.

I've been planning on building some filtering for the Who is Hiring threads,
and I've pretty much determined that some degree of manual review will be
needed. In the most recent thread, I found a huge number of posts containing
"remote" which don't actually allow remote working. "No remote" is fairly
common and easy to filter out, but there are any number of variations that you
can't anticipate a priori.

~~~
manlio
> I've pretty much determined that some degree of manual review will be needed

You're spot on with everything. I did a lot of manual review and the site
already filters out "NO REMOTE", "REMOTE no", "Remote not" and "No Remote"
entries. I did spot the "Remote work isn’t an option" post, but I decided I'm
not going to write _that kind_ of completely ad-hoc filtering rules, it's just
ugly.

~~~
louthy
You could break the text up into sentences [1] and do sentiment analysis [2]
on the sentences with 'remote' in. Then flag based on that.

[1]
[https://opennlp.apache.org/documentation/1.5.3/manual/opennl...](https://opennlp.apache.org/documentation/1.5.3/manual/opennlp.html#tools.sentdetect)

[2] [http://nlp.stanford.edu/sentiment/](http://nlp.stanford.edu/sentiment/)

~~~
davidw
Or you could hire people to do it via oDesk or Mechanical Turk. Not so
interesting technically, but it's a job people are good at.

~~~
fudged71
Hire people for cheap to help people be hired for $$$, with no reward for the
upsell. Brilliant! :)

------
artursapek
This is awesome. Only suggestion is to add a backlink to the post and/or user
who posted it.

~~~
getsat
Yes, this please. Sometimes there are comments or you want to investigate the
profile/history of the poster.

~~~
manlio
Aaaand, feature is up ;)

~~~
artursapek
Amazing. Thanks so much!

------
majke
There is no way to view the data on the HN website. Please add a link to a
source, or at least
[https://news.ycombinator.com/user?id=whoishiring](https://news.ycombinator.com/user?id=whoishiring)

~~~
manlio
Fixed that.

------
jlhonora
The only way to deal with the unstructured nature of the "Who is hiring" posts
is to have some sort of schema that can be processed. I also wanted to do
something similar (imagine it with dc.js, for example!), but the data is too
diverse.

A sample entry could be:

    
    
      company: 'Some Company',
      jobs: [
        {
          dev_type: 'Web/Mobile/Data',
          dev_sub_type: 'Frontend/Backend/DevOps/Android/iOS',
          visa: 'Required/Not required/Transfer only/Sponsored',
          remote: 'Yes/No/Maybe'
          locations: [
            'SF',
            'London',
            ...
          ]
        },
        ...
      ]
    

All posts could have a METADATA: compressed_json entry that can be processed
by the site and displayed/filtered accordingly. Perhaps it could be built
manually at the beginning until it catches up.

~~~
mojuba
dev_type: 'Web/Mobile/Data/Design/Hardware' at least.

~~~
jlhonora
Sure, there are many things that could be improved.

Anyhow, I'm surprised. I'm a Hardware Engineer myself, so how could I miss
that!

------
mountaineer
This is cool, nice to look at other projects analyzing this data. I publish HN
Hiring Trends, [http://www.ryan-williams.net/hacker-news-hiring-
trends/](http://www.ryan-williams.net/hacker-news-hiring-trends/) , that
watches the various technology terms being mentioned in the postings.

Once thing I changed to was just including top level comments and no
replies/discussion of the posting. Do you handle similarly?

~~~
manlio
HN Hiring Trends is really sweet, I didn't know about it. I do include only
top level comments; and incidentally, digging through HN's HTML code was...
uhm... let's say a bit _messy_.

~~~
mountaineer
Yeah, I can imagine. HTML is really tricky. Why not use the API? With a few
requests, you can query for and pull all of the whoishiring threads.

Here's how I pull them all down:
[https://github.com/ryanwi/hiringtrends/blob/master/lib/hirin...](https://github.com/ryanwi/hiringtrends/blob/master/lib/hiringtrends/program.rb#L48)

------
jeromegv
Great project, however I've noticed some listings are missing. For example,
from April,
[https://news.ycombinator.com/item?id=9303396](https://news.ycombinator.com/item?id=9303396)
there was a posting from Questrade. It's missing on this website.

~~~
manlio
Thanks for the feedback! That particular post is missing because the city is
written in all caps (TORONTO). I used a bunch of tricks to be able to catch as
many cities as possible (say NY and NYC and Manhattan and even NEW YORK all go
under "New York City"), but not _every_ city is tested against its "all caps"
equivalent atm because I wanted to be able to rebuild the db as fast as
possible during development phase. I should probably fix that now.

~~~
_s
I used to fall for this a lot so now most string comparisons I do something
like

    
    
      // string contains string against lower case / no whitespace
      cityFromList.indexOf(cityFromPost.trim.toLower()) 
    

Might help in your case too?

~~~
manlio
Unfortunately no, 1) you can't trim the whitespace if you're matching cities,
2) matching the cities lowercase opens up too many false positives. At the
same time, matching every city with its uppercase equivalent doubles the time
requested to build the db but only adds a tiny handful of posts. That's why
(for now) I settled with a tradeoff where I catch the uppercase equivalent for
the biggest cities only.

------
venantius
Sorting by language would be a great feature for this as well, particularly
for those of us who work in more obscure / esoteric languages.

~~~
Trufa
Yes, some sort of filtering is missing, I would like to filter by frontend for
example. It would be very practical that way.

------
nstart
Love this. I was able to get more useful information about remote working than
I was able to when reading the original who is hiring April post. This is
probably a good time to suggest changing the format of the who is hiring
posts. Would be great if the use of a standard form template is encouraged.
That would make an effort to parse the data much easier, and would make the
reading of the original post easier, and would probably make it easier for
companies to create their posts too. Win for all?

------
bhayden
Worth noting April Houston listings have a false positive because an investor
was named Drew Houston.

------
Shadow6363
I'm surprised nobody else has requested this, but any chance for a state
category? If you really want to impress me, perhaps a warm and cold climate
section or maybe have to shovel snow vs. unlikely to shovel snow. :P Gotta set
my priorities straight…

~~~
loblollyboy
also alphabetize the cities when there is a tie

~~~
manlio
Love the alphabetize idea! I just fixed that, thanks for the suggestion :)

------
dataker
Great work!

As a sugestion for next feature, I'd recommend a selection for visa
sponsorship or not.

~~~
manlio
I actually thought about it, but it's a total mess to parse :( Think of posts
like "We're sorry we can't sponsor H1B at the moment, but we might get you a
VISA for our London office".

On the other hand, once you're already browsing New York City, good ol' ctrl-f
for VISA will probably serve you well enough.

~~~
c4n4rd
Agreed on the parsing. It would be nice if there was a standard "form" to be
posted. Something line.

Description: .... Company: keyworks: python, startup, collstuff Visas
Sponsered: YES

~~~
pbhjpbhj
Back when the web looked (to me) like it was going to move to XHTML2 rather
than HTML5 data encoding using Microformats looked promising,
[http://microformats.org/wiki/job-listing](http://microformats.org/wiki/job-
listing). One doesn't hear "semantic web" much nowadays though.

------
stephengoodwin
Although I'm sure companies won't complain about the additional publicity, are
there any concerns of copyright issues with scraping and republishing the text
from other HN posts?

~~~
necrodawg
If everyone cared that much about copyright I think no one would ever make
anything. There's always a way to sue someone over something, especially
copyright.

~~~
stephengoodwin
I asked out of curiosity, not criticism.

------
yitchelle
Maybe an error but the entry for
[https://news.ycombinator.com/item?id=9305360](https://news.ycombinator.com/item?id=9305360)
is getting confused with Melbourne, Australia. I think is should be under
Melbourne, Florida.

I am from Melbourne which made me looked at the Australian entries.

------
yodsanklai
General question about remote work: as a european, can you work remotely for
an american company without an H1B visa?

~~~
bjelkeman-again
Is it hard to find remote work in Europe at the moment? Just curious, I
haven't had to look, but was under the impression there ought to be a
reasonable amount available.

~~~
emp_
It might have to do with the USD being close to the EURO lately, so it opens
more options.

------
chatmasta
Very cool idea, fun name, practical interface. I like it.

Perhaps add a simple tagging system where users can add tags to hiring posts.
That way you don't need to comb through every post and hopefully you
crowdsource some helpful taxonomic data.

------
ryen
Useful, but falls short for surrounding areas of Los Angeles like Venice
(neighborhood of LA) & Santa Monica (adjacent and much a part of LA). I
imagine there are issues like this for other cities and regions too.

~~~
manlio
Quite the opposite, both Venice and Santa Monica had so many posts I decided
to treat them as independent cities, e.g. [http://whereis-whoishiring-
hiring.me/city/2015/4/Santa%20Mon...](http://whereis-whoishiring-
hiring.me/city/2015/4/Santa%20Monica)

~~~
ugh123
i believe its not its own city. 2 posts for venice should be part of los
angeles. you can still have venice its own section if u want, but excluding it
from los angeles doesn't make sense.

what u have is too simplistic.

~~~
CalRobert
I realize that you may be using a limited input device, but surely you can
afford a few spare grams of pressure for the shift key, or y and o? Anyway, I
can see what you mean re: Venice/LA (Venice is not its own city), but as
someone who used to work in Santa Monica there was a time when the ONLY places
I would consider were SM, Venice, and just maybe El Segundo (the beach bike
path did make for an amazing commute). Having WeHo or Northridge jobs mixed in
there could be a pain.

~~~
ugh123
having the problem of a few more posts to sift through is much better than the
problem of missing something that might have been categorized as "Los Angeles"
where the job is actually in venice or santa monica <\- happens all the time.

these things are just a starting point - further investigation on the
company's site and actual location are always necessary.

------
blt
I was surprised at how few are hiring in Los Angeles. I'm thinking about going
to grad school there. Can anyone in LA comment on the state of your tech
economy?

~~~
yarone
LA-area tech community has been growing incredibly over the last few years.
Lots of early stage startups; some are now maturing like Dollar Shave Club and
Lynda.com

~~~
blt
what about non-web stuff? I'm interested in robotics, sensors, embedded,
manufacturing tech, etc.

------
zo1
A bit of a side note, it seems someone created an "alternate" who is hiring
bot or account:

[https://news.ycombinator.com/user?id=_whoishiring](https://news.ycombinator.com/user?id=_whoishiring)

~~~
dang
Only because the standard "bot" slept in last time.

~~~
JshWright
I assume that was just a DST issue. The bot posted it an hour later than
usual, and the DST transition (for the US) took place between the March and
April posts.

~~~
dang
Ah so that's why. Good catch.

------
asherdavidson
Can you add an option to sort by companies that are offering internships as
well?

~~~
andrewjkerr
I would love to see this as well. If I have some time this weekend, I'll look
into submitting a PR :)

~~~
manlio
Done! Better late than never ;)

------
necrodawg
Really nicely done. I did something similar as a blog post a good while ago
and it was quite popular. The results haven't changed much it seems.

One comment I got was that I had just mapped where HN users are in the world.

------
xb
Very cool, and interesting to see the results. I immediately looked for a
'trends' feature to see how cities rank change over time, or maybe this could
be plotted?

------
27182818284
This is really cool. Thanks for creating this. I'm not looking at the moment,
but I'm always super interested by the Who's Hiring threads.

------
ubercow
As someone who lives in Washington State, not Washington DC. It always
frustrates me trying to sift through the DCs looking for something.

------
untilHellbanned
Boston is gaining and should probably be gaining more if lumped together with
Cambridge.

Less sure about lumping San Francisco and Palo Alto. Thoughts?

~~~
ScottBurson
I would suggest three buckets: San Francisco, including San Bruno, Millbrae,
and Burlingame; Mid-Peninsula, covering San Mateo, Foster City, Belmont, San
Carlos, Redwood City, Menlo Park, and Palo Alto; and South Bay, covering
Mountain View, Sunnyvale, Santa Clara, San Jose, Cupertino, Campbell, Los
Gatos, and Milpitas (and maybe Fremont?).

------
strathmeyer
...apparently all the companies I am being rejected from are being acquired by
all the companies too good to speak to me...

------
lordlarm
Just a small note about "Cambridge, MA" not being the same place as
"Cambridge, UK".

~~~
manlio
Yep, plain Cambridge is "Cambridge, MA", the other one is listed as
"Cambridge, UK". In the same fashion, "Venice" is actually "Venice, CA". It
hurt a bit but it was the right thing to do ;)

~~~
jn
Great work. Looks like Dublin, Ohio is getting categorised as Ireland though:
[http://whereis-whoishiring-hiring.me/country/2015/3/Ireland](http://whereis-
whoishiring-hiring.me/country/2015/3/Ireland)

------
cauterized
Awesome, awesome idea! Any plans to do the same with "Who's looking for work"?

~~~
manlio
Not really, but if _you_ want to do it all it takes is to replace the URLs to
wget (i.e. you want to wget all the "Who's looking for work" pages instead of
the "Who is hiring?").

You can easily build your local Sqlite database like that. I wrote some more
instructions about it on the README.md on Github.

~~~
cauterized
Ah, cool. I'll check that out. Thank you for open-sourcing this (and for
documenting it)!

------
smoovej
This is fantastic. Thank you!!

------
meritt
This is really cool. I'd love to see this by technology composition too.

------
cmpb
This is pretty excellent.

------
ryan-allen
This is great! Thanks for making it :)

------
zuck9
Typo: "[brose by country]"

~~~
chrisbennet
Should it have been "bros" by country? :-)

------
NxSandro
Awesome work, bookmarking this !

------
sabman83
Nice work. Makes it easier for me to keep an eye on for jobs in Portland.

------
pknerd
can you please also provide filter of Technology?

------
poslathian
cool! you should lump cambridge MA in with boston

------
mydpy
Best URL ever.

------
theviajerock
Great!

------
616c


------
larsberg
I should mention that I and other hiring managers I've talked with are moving
away from posting on the "Who is hiring?" post.

It was pretty useful ~6 months ago. But, the amount of spam generated from
recruiting and sourcing firms, various startups trying to push their
revolutionary new online coding tools, etc. is pretty ridiculous and many of
them, especially the SV-area startups, have been quite aggressive (e.g., phone
calls and switching to my personal e-mail address after I told them I was not
interested).

Posting jobs on twitter has been a far more effective sourcing tool than HN
"Who is hiring" has become recently, at least in the free space.

~~~
dang
Ruh roh, that sounds bad. What should we do?

~~~
brandnewlow
Should "Who Is Hiring" posts require a certain age of account or level of HN
Karma to post in?

~~~
slayed0
How about instead of requiring a certain amount of karma to post in the
thread, require a certain amount of karma just to view it?

~~~
benologist
\+ exclude karma earned from submitting popular sites.

~~~
minimaxir
There is little correlation between the domain of a submission and the amount
of points it receives on average. (The exception is the more niche posts by
more renowned programmers)

~~~
benologist
The stories don't have to be popular to generate karma, as long as the domains
are some articles will get karma from other people submitting the same links
and manual upvotes too. Yesterday someone autosubmitted everything a dozen big
tech sites published and got 100 - 200 karma without hitting the front page.

------
WaxProlix
Show HN: Ask HN: Where is “Ask HN: Who is hiring?” hiring?

