
Our Hybrid lda2vec Algorithm - haraball
http://multithreaded.stitchfix.com/blog/2016/05/27/lda2vec/
======
stephanheijl
So this is basically a repost of the Slideshare on this topic
([https://news.ycombinator.com/item?id=10919744](https://news.ycombinator.com/item?id=10919744)
< previous HN thread.) without the pretty landing page around it. It does look
like some 5000 new LOCs went into it since then, so that 's positive.

I also like the honesty of this report, mentioning different methods that are
similar (even another project called lda2vec) and the disclaimer that this is
probably not for everyone. The Python code does make it more accessible
however, so I could see myself at least reusing concepts that are implemented
here. On the whole this looks like an interesting algorithm for the stated
purposes.

------
willvarfar
> Maybe the folks behind Hacker News want to evolve the site by encouraging
> more job hiring posts, or maybe they’d like to devote special site features
> to housing or more security features. That’s the kind of analysis we need to
> effectively steer our companies: more insight, less black box.

I guess that was written in gest, but I do hope some kind of reddit or SO-
style siloing isn't on the cards to 'make HN grow'!

I rather hope that HN grows in the collaborative filtering direction instead.
Instead of most interesting topics never making it high enough up the first
page to actually gain any traction, it would be nice if collaborative
filtering was used to make it so people can easily get more of what they like
and easily find something completely different. Of course I've blogged and
petitioned for this kind of approach in the past ;)
[http://williamedwardscoder.tumblr.com/post/15581427232/self-...](http://williamedwardscoder.tumblr.com/post/15581427232/self-
organizing-reddit)

Then HN can stay the HN we all like, as many of us like slightly different
things, and collaborative filtering can evolve with each of us letting us find
like-minded others to share our filter-bubble with :)

~~~
dang
I like the phrase "collaborative filtering" and it fits what we have in mind
for HN nicely. (The term we use internally is "story review game", because we
want to make a game of it.) But I think there's value in having HN's front
page be the same for everybody, even though that guarantees that it's optimal
for nobody, and produces a daily crop of "why is this on HN" protest comments.

------
minimaxir
A clarification of the "job postings" mechanic: only YC startups can make job
postings, and they start at #6 and undergo constant rank decay (this behavior
has been stated by the mods multiple times). The fact that % job postings has
increased implies that % increase in YC startups > % increase in HN
submissions over time, which is not unexpected.

If HN were profit-optimizing and let anyone put ads, it would break the site
completely as the density of ads makes it more difficult to read content
(something _many many_ blogs like Forbes fail to realize).

