

Ask HN: Why are a large proportion of legit posts being marked [dead]? - ekpyrotic


======
matt1
I had the same thing happen twice earlier today. I hit a huge milestone on a
product I launched on HackerNews about two months ago, spent a few hours
writing a blog post about it and lessons learned, and posted it on HackerNews
around 11am EST.

Within 50 minutes it had 17 points and had climbed up to about #13 on the
front page when all of the sudden it disappeared [1]. I signed out of my HN
account and checked the comments link and sure enough the page was blank,
indicating that it had been killed.

I was talking to a friend on GChat at the same time this was going on. He
reposted it, thinking that it was killed because of an algorithmic fluke
(which was probably true) [2]. The new post gained 9 points in 10 minutes and
then was killed as well.

The only thing I can think of is that because that friend upvoted the original
post (and he's upvoted some of my previous posts), combined with how quickly
it shot up the front page, somehow caused it to be flagged and automatically
killed.

I'd still love to repost it both to share my product's milestone and to get
feedback from the community, but I'm afraid it will be killed again. Any
recommendations?

I'm all for stopping spam and voting rings, but it shouldn't be at the expense
of legitimate posts.

[1] <http://news.ycombinator.com/item?id=3788402>

[2] <http://news.ycombinator.com/item?id=3788806>

Edited to add: I noticed a lot of the new posts around the same time had
several points within a few minutes of being posted. Almost none had 1 point,
which I thought was odd. I think someone might have written a script to upvote
articles from multiple fake accounts, thereby causing HackerNews's voting-ring
algorithm to mistakenly identify the posts as spam.

~~~
jacquesm
Wow, what a waste. That's a really good post and it is a pity that it became
the subject of this nastiness. Maybe you could petition PG for a re-run? I'm
sure lots of people would like to read what you wrote.

(If you're interested and can't find the link, the article is here:
<http://www.leandomainsearch.com/2000> ).

~~~
dfc
I realize its not the best thing to happen to HN but do we really think its
fair/appropriate/considerate to _"petition"_ pg every time someone's
submission is flummoxed?

~~~
pbhjpbhj
> _do we really think its fair/appropriate/considerate to "petition" pg every
> time someone's submission is flummoxed_ //

As it appears this is the only way to appeal against what appears to be a bad
algorithmic story rejection then IMO yes this is fair and appropriate.

If there were moderators or a user moderation process then these sorts of
issues would get picked up there.

Of course PG is perfectly within his rights to ignore any such submission,
depends on the purpose of the site really.

------
mikecane
I think we're not being let in on a silent war happening here. It seems bogus
points are being assigned to submissions and they are then being killed. It
started several hours ago.

See first hints here: <http://news.ycombinator.com/item?id=3788069>

And someone else noticed: <http://news.ycombinator.com/item?id=3788740>

~~~
acangiano
This is most likely what's happening. I submitted an interesting book review.
The kind that would get routinely ignored on HN. It immediately got 8 points.
Very odd. I expect it to die any time now.

~~~
mikecane
I've rarely had submissions killed. But I was surprised when mine got ten
points within moments of posting it. That made me suspicious and since then
it's the likely reason for submissions being killed: Someone is giving
everything bogus points.

~~~
gruseom
You guys are more on the ball than I am. When my post got instant upvotes I
thought, "How pleasant that so many people are interested in dataflow
computation".

Probably HN's software is detecting the suspicious voting and killing the
posts.

~~~
jacquesm
I've seen the same thing happen on this article that I wrote last friday:

<http://news.ycombinator.com/item?id=3775365>

It went up like a rocket and disappeared just as fast. I didn't submit it so I
have no clue whether or not the submitter gamed the system but it sure looked
weird.

Given the subject of this thread that's actually an on-topic link :)

------
untog
There is- or has been- something very strange going on lately. At the height
of the GeekList mini-controversy last week the HN thread discussing it[1],
which was in third place on the front page, was flagged and killed. The second
discussion that was created[2] was also killed. Then I posted a blog entry[3]
which also got to the front page, then was also flagged and killed. At that
point I just gave up, because what can you do? It seems like the flagging
mechanism is far too powerful- just a few reports and the entry gets shunted
back seven pages. A few more and it's gone entirely.

It's worrying for two reasons: one, that original Geeklist discussion had some
great points being made, and a lot of users were obviously engaging with it
before a minority decided to dispose of it. Two, there's a clear minority on
here that would like to see discussions about sexism in tech removed from HN.

[1] <http://news.ycombinator.com/item?id=3739913>

[2] <http://news.ycombinator.com/item?id=3740378>

[3] [http://blog.untogether.co.uk/post/19740556298/why-are-
posts-...](http://blog.untogether.co.uk/post/19740556298/why-are-posts-about-
the-geeklist-controversy-being)

~~~
Tichy
If you think about it, the relationship of the GeekList incident to HN was
pure coincidence. Some company had a video with a scantily clad girl in it.
That's the content of that news item. Would you have considered it related to
HN if the company would have been making beer, or hovers, or cars, or
whatever? The only assumed relationship was that the company carried the word
"geek" in it's name. It's 2012, so I wouldn't even assume that every HN reader
is automatically a geek anymore - in fact, does geek even still have much of a
meaning?

But anyway, the point is: that story really was not HN material at all. The
policies for HN clearly state to refrain from political discussions.

~~~
untog
_Would you have considered it related to HN if the company would have been
making beer, or hovers, or cars, or whatever?_

Of course not. But it wasn't about any of those things, it was about a tech
startup where "all bad-ass code monkeys around the globe can communicate,
brag, build their street cred and get found". It's a site both made and used
by exactly the type of people that make up the HN audience. How could it _not_
be relevant?

And I'm confused by your comment about political discussions- the GeekList
discussion certainly wasn't political.

~~~
Tichy
Don't blame me or others for what one company does. That is just ridiculous.
Even if they claim they represent "us". They don't represent me, so I felt
entitled to flag their story. It certainly doesn't provide any statistics on
the sexism of geeks. It is just a bad anecdote (a company claiming to cater to
geeks behaves in a sexist way -> all geeks must be sexist).

~~~
untog
Why on earth do you think I am blaming you? I'm not. I'm not suggesting that
their actions represent the actions of "geeks", either. All I am saying is
that GeekList is a company made by techies and used by techies- and article
about them might be of interest to techies. I don't think that's a giant leap
in thinking. Aside from anything else it was an instructive tale on how (not)
to deal with customer complaints.

~~~
Tichy
Fair enough. Perhaps that article only had the bad luck to be washed in with a
whole wave of sexism discussions.

------
dclaysmith
This seems to go beyond individuals killing story. A few hours ago ALL stories
were killed...

This is a screenshot of the /newest (page 2 now) that shows the gap in
submissions: <http://i.imgur.com/hLkwa.png>

------
unimpressive
<http://news.ycombinator.com/user?id=michaelkscott>

Notice his karma is lower than his highest ranked post right now. In a now
deleted post he claims he lost 900+ karma overnight.

Be warned guys. If you post here the chances of you getting hellbanned are
non-zero.

EDIT: And if anything like that happens on a mass-scale and PG doesn't have a
transaction rollback in the wings...

~~~
tomjen3
PG can't roll back, because he isn't using a database backend.

As for being hellbanned, well I have 3 after my name for a reason.

~~~
unimpressive
>PG can't roll back, because he isn't using a database backend.

I was afraid of that...

May I ask why? Too much resource usage? Arc doesn't have a DB API?

~~~
abtinf
Via email, I asked PG about this 7 years ago.

me: "I've read in your essays that you (and yahoo) chose to store all your
information in files rather than a database. I am curious if you would make
the same decision now that excellent oss databases are available (mysql,
postgresql, etc)."

PG: "I'd do the same thing. It was a huge win to use alists (a lisp construct)
and just write them to files. Then if anything went wrong, or data formats
changed, I could just munge the raw files. Nothing was wired in."

~~~
unimpressive
Thank you for answering. At the same time however, I'm not really sure what to
think about that. On the one hand future proof is a massive win. On the other
no rollbacks seems like a risky way to operate. I hope he keeps regular
backups of the data files.

------
DanBC
Some people just submit a lot of posts.

Not pointing fingers, but here's one:

(<http://news.ycombinator.com/submitted?id=daegloe>)

30 posts in less than 3 hours.

~~~
mikecane
Wow. Maybe he needs a blog or linkblog. I think one day I did 5 and I felt bad
but I thought all were germane to hacker interests. Generally I am not
submitting everyday.

------
benohear
I had a similar thing happening to a submission I made the other day. It
quickly climbed up the ranks, then it dropped to 27 then it vanished.

However, about 20 minutes later it got re-instated and spent most of the
afternoon on the homepage. No idea what went on.

------
mikecane
What worries me now is this. Recently some people showed how points could be
assigned to posts by clicking on a link. Now I wonder if maybe some of us have
been somehow infected and are somehow contributing these points without
knowing? Is that even possible?

~~~
Estragon
Do you mean a XSRF-style attack on upvotes? Could be, but I can't imagine
why...

~~~
dfc
You can't imagine why Eve would want to trick Alice and Bob into upvoting one
of her submissions she had an interest in?

Or you can not imagine why Eve would want to trick Alice and Bob into upvoting
other submissions so that they got killed and her submission received more
attention?

~~~
Estragon
I can't imagine how it would be worthwhile to go to that much technical effort
for the sake of ranking on HN.

~~~
dfc
Really? Think about the exposure a fledgling starup can get from being on page
1 for an hour or two. I think a post from malda said HNed was the new /.ed...

------
mikecane
Now I'm beginning to think that perhaps some accounts here might have been
compromised:

<http://news.ycombinator.com/item?id=3789224>

Those who run HN might want to check out all those fake poll submissions today
for likely suspects.

------
waterlesscloud
I read with showdead on, so I always see the comments from the hellbanned.

If I think the comment is worthwhile, I upvote it anyway. I don't know if it
does any good, or if it maybe eventually leads to them being unbanned, but I
try to help out if they're contributing.

------
mikecane
OK, look at this: <http://news.ycombinator.com/item?id=3788970>

That's like the tenth frikkin poll that's shown up here today. Are individual
accounts hacked too?

------
mikecane
And now we get _another_ poll submission:

<http://news.ycombinator.com/item?id=3789065>

Something is happening here and it's very bad.

------
mikecane
I thought things were calming down. As of 2:08PM EDST, I'm seeing almost every
submission on New immediately get points again.

------
z_
They will be resurrected Monday coming.

------
danso
I submitted this NYT article, "Does the iPad Have One Button Too Many?"
<http://news.ycombinator.com/item?id=3788937>

It got five votes in 3 minutes -- I know HN'ers love Apple talk, but that's a
little fast. And the submission is not in the top 100 of submissions. It could
also be that I'm just autobanned for some reason.

edit __: FWIW, about half an hour ago, the submission was revived. Thanks(?)

~~~
Alex3917
It sounds like people are using bots to upvote other peoples' stories so that
they get autokilled, thus making it more likely that their own stories reach
the front page.

~~~
unimpressive
That's...brilliant. Sort of reminds me of my concern about bots artificially
inflating google rankings for competitors to get them banned. (Which
apparently turned out to be unwarranted.)

EDIT: By "brilliant" I mean brilliant in that twisted security breaker kind of
thinking that slightly creeps most people out.

EDIT2: As the post below points out, in retrospect it's pretty weak. But then,
most things of this nature seem pretty weak in retrospect. In my opinion the
fact that it's so weak is the whole beauty of it. It's a simple oversight that
wasn't obvious until pointed out.

Regardless the perpetrator is an asshat.

~~~
Alex3917
Meh, it's weak at best. Because HN follows zipf's law, you can fix 85% of
problem just by making it more difficult to flag submissions by the top 250 or
so users.

Plus it's just annoying. The yc partners are supposed to be reading
applications this week, and now they have to deal with this crap.

~~~
klapinat0r
Interesting. There was a post about different data analyzed towards Benford's
law which caught my attention - about a year ago I think. Would you mind
elaborating what you mean by 85% of the problem can be fixed in relations to
zipf's law (I thought I was aware of the law, but perhabs I'm missing some
intuition since I don't quite get your point?).

~~~
Alex3917
The #1 HN user has 10x as much karma as the #10 user. The #10 user has 10x
more karma than the 100th user. The #100 user has 10x more than the 1000th,
etc. Essentially the vast majority of highly upvoted submissions come from a
tiny group of already trusted people, so by fixing the problem for just 1% of
users you are actually fixing the problem for ~85% of content. It's almost
impossible for a bot to make it into this group, and if they do then it's
trivially easy to ban them.

~~~
tedunangst
Your fix is self perpetuating in a bad way. If only the top 100 users can get
on the front page, only the top 100 users are going to get upvotes. It's
impossible for real people to make it into this group.

~~~
Alex3917
Yeah it definitely wouldn't be good if that were the only thing done, but as
one of several things it would make a lot of sense.

------
daegloe
It appears that duplicate submissions are getting through as well.

------
Steko
Maybe the algorithm was modified to give some initial karma based on the users
past submissions and this is running into the spam filter settings?

------
pcvarmint
Wake me up when I can downvote. Until then this discussion is moot.

------
richf
I lost 4 karma points off a dead post.

