
Salesforce hackathon comes under attack from devs - coloneltcb
http://venturebeat.com/2013/11/22/salesforce-hackathon-comes-under-attack-from-devs/
======
breck
I participated in this Hackathon.

It was a good hackathon.

I am happy for the winners.

The truth of the matter is, there were so many entries, that it was a complete
lottery who ended up winning. I personally thought our app was great, and had
a decent shot at winning, but I went to bed assuming 99% odds we wouldn't get
picked. With that many entries the odds were not that great.

As lame as this sounds, I've learned the way to really win at Hackathons is to
build something you wanted to build anyway. That way, if you win, it's a nice
bonus, but you'll come out ahead no matter what.

The judging processes in these things is always suboptimal. I knew it was
going to be a nightmare to try and narrow down that many entries to 5 in just
a few hours. They were very optimistic in thinking they could pull that off. I
think ChallengePost has a great opportunity to improve here by innovating so
that if you enter a Hackathon, you get direct feedback from the judges (either
quantitative numerical feedback or qualitative comments), so you get some
acknowledgement for the hard work put in. The judging was indeed very light--
as another data point I don't think we got a single hit on our app-- but I
thought they tried to do something unprecedented and did a decent enough job
and can learn going forward.

It was a very cool event, I'm happy for the winners, and hopefully some of
these Hackathon software tools like ChallengePost and HackerLeague can
continue to get better and help alleviate some of these transparency concerns
for future Hackathons.

I would definitely recommend participating in future Salesforce Hackathons.

~~~
zaguios
Paying $500 for a 0% chance to win; that doesn't sound like the best deal to
me. You can get a lot more value from a regular hackathon that you can
actually get some attention and feedback from and that isn't going to charge
you a crazy amount of money. I don't understand how you can justify the
competition when your app wasn't even viewed.

Edit: Apparently the ticket price was $100 and then reimbursed. That makes it
better, but I would still argue you get more benefit from a standard
hackathon.

~~~
jorde
Tickets for the Hackathon were $99 a head but they quickly started giving them
out for free. I also heard stories that people got reimbursed fro the ticket
price if they had payed.

------
minimaxir
This article doesn't mention the allegation that the judges didn't even look
at some of the other applicants:
[http://salesforce1million.challengepost.com/forum_topics/264...](http://salesforce1million.challengepost.com/forum_topics/2644)

------
gwf
I work at salesforce but I am not a spokesperson for salesforce nor do I have
any authority to speak for the company. What I write here, I write as a member
of HNN only.

I did not have any part in the hackathon process, but I had and still do have
quite a bit of visibility on how the process was run. I can assure you that an
enormous amount of effort went into the judging and that it was entirely
unbiased. There were literally dozens of the most senior executives and
engineers within the company that worked on the earlier judging rounds behind
the scenes. I actually declined to be part of that process because it was too
much of a commitment for me to make. I know, for example, that the next-to-
final round judges were up until the early AM reviewing the submissions and
that each judge looked at on the order of 20 different submissions per rounds.

Sadly, whenever something like $1M is on the line, there is always going to be
someone that is unhappy with the outcome. People will speculate and make
claims that are outright false. But please keep an open mind as you read more
about this event. There will soon be some official communication on this topic
from salesforce. I suspect that the more actual facts you learn about the
process, the more reasonable it will seem.

~~~
willchilcutt
Are you saying that the dozens of people claiming that no one tested their
apps are all lying?

Like others have said previously, Salesforce must have forgotten about
analytics.

~~~
gwf
I am saying that there is a misunderstanding. The earliest rounds of judging
looked at the videos of the apps that contestants produced as well as a code
review as a sanity check. This was done most likely so that each app could be
viewed in the best possible light as well as to help the judges scale to the
number of entries.

So, no, the people complaining are not lying but they and anyone else would be
incorrect to conclude that their app was not evaluated. Every app was
evaluated. All by videos. Some by code review. And a few through every
reasonable means.

~~~
teraflop
What about the people who checked the analytics on their videos and determined
that they hadn't been viewed?

~~~
gwf
I honestly haven't seen a single complaint like that. Can you point me to one?
FWIW, I am pretty sure that each entry was reviewed at least twice.

~~~
shuzchen
This hn comment links to a few places where people complained:
[https://news.ycombinator.com/item?id=6779036](https://news.ycombinator.com/item?id=6779036)

The link that specifically shows people complaining that nobody viewed their
app or looked at their video is this:
[http://salesforce1million.challengepost.com/forum_topics/264...](http://salesforce1million.challengepost.com/forum_topics/2644)

~~~
gwf
Your first link doesn't contain a single complaint claiming that people's
videos were not viewed. It does, however, link to the second link, and the
other two links it contains make no references to this sort of complaint.

The second link has exactly one person clearly stating that they've received
zero launches and zero video plays. There is a second person that states
"Yeah, same here. We had a workflow setup to send us notifications if any new
used the app and received none." Notice that he/she doesn't mention video
plays.

So, in total, we have exactly one person that says that they've received zero
launches and zero video plays. If I've unfairly characterized the data behind
your links, please let me know. But that's all that I can find.

In trying to find out if they have a valid complaint, I've discovered that the
person behind that complaint is responsible for a significant portion of all
of the discussions and complaints at the challenge site.

One thing is clear: their submission is in the same area as the winning entry.
In their shoes, I might be disappointed for that reason alone. But for the
sake of transparency, here are the two entries:

The winning entry (which is admittedly being evaluated to see if they
conformed to all rules):

    
    
        http://salesforce1million.challengepost.com/submissions/18552-healthcare-love
    

The entry for the person or group that claimed that their video was never
viewed:

    
    
        http://vimeo.com/79921465
    

In any case, I encourage you to look at the two videos and tell me if you
think there was an injustice with the rank ordering.

~~~
willchilcutt
Thanks for giving more feedback. Would you be kind to respond to the other
child of your OP pointing out that the winner had been working and actually
demoed their app way before the starting period set out in the rules? Thanks!

~~~
gwf
No problem. Will do in a second...

~~~
colabi
gwf. if you, and salesforce, really believe it was an improbable series of
poor decisions and not malicious intent, tell your bosses it is time to build
the redemption story. and we can speak about how to reboot this together, the
company and the contestants.

------
nzk1
Hah Salesforce - you were better off having no hackathon at all. This is
disgusting. Preselecting winners possibly?, not looking at everyone's app /
video, selecting a winner that started way before hackathon was announced. yep
total mess.

------
codeoclock
If developers are incentives by the possibility of winning $1m, they would
start working months in advance. That's why it's not ok to offer $1m as a
hackathon prize, as it's not in the spirit of hacking.

~~~
theboss
I think you could offer 1million but it has to be a competition where the
teams that enter solve a single problem. That way accurate scoring metrics can
be developed and used.

It doesn't make sense to say "Hackathon, 1 million dollars, Go!".

edit: To add. The problem should be announced the day of!

~~~
vdaniuk
I guess that it's better to allow much more time for bigger prizes. Like the
Netflix prize. That was a great mix of marketing and feeding the recommender
system.

------
sek
This is a general problem for hackathons, the idea of limiting something like
development is impossible.

Even if you don't code, you could plan the whole concept, user your super
familiar framework and there is a huge difference to the "spirit" of just
developing the concept on the spot.

I don't want to say I dislike hackathons in general, but not for huge prices.
Just meeting interesting people and trying to build something together is the
really valuable part.

~~~
resu_nimda
There are a number of ways to combat this.

\- Have a theme, a dataset, or some other component that is not announced
until the day it starts (and to be eligible an entry must demonstrate
meaningful incorporation of that component).

\- Have a manageable organizer-to-participant ratio so they can do a decent
job of verifying that teams aren't using a developed concept.

\- Don't allow premade teams, have individuals pitch concepts and form teams.

Each of those can be gamed to some degree, but they can go a long way, and to
game them requires serious intent to deceive.

They didn't even try, at all. The winner was openly a fully formed startup
that _demoed their app at a Salesforce meetup_ weeks before the hackathon.
That's just, mind-blowing.

~~~
saryant
There's another good way to eliminate these concerns: don't have big prizes.

When I hosted a hackathon last summer, I put no limits on projects but the max
value of the prizes was $25. With prizes so small, there's no incentive to
game the system nor is it grossly unfair to let people work on something
they'd already started on.

The goal was purely to work on something cool for a day with like-minded
people. One guy brought on a voice-controlled Nest lamp he'd been working on,
another built an Arduino step-counter, a third took a crack at writing a basic
chess AI.

We provided pizza and beer and just let everyone loose. It went great and
there were no hard feelings.

------
justinsb
I participated also, and enjoyed the hackathon itself. I wasn't expecting to
win. I expected the winner to be someone that had a suitable product that was
ready to launch around the same time as the hackathon was announced, and
decided to seize the moment for some PR. This is more or less what happened. I
think what is leaving a bad taste is that I didn't expect that it would done
so brazenly.

The problems in my mind aren't about unfair judging (all judging is biased),
but rather that:

* The winning company had clearly been started long before the permitted start date.

* It took less than an hour for this to be 'proved'; Salesforce obviously can't check compliance on every team, but not double-checking the five finalists seems foolish.

* The smoking gun was a demo Upshot gave at a _Salesforce meetup_. One of the two company founders had worked at Salesforce for _9 years_ until earlier this year. There's no way any sensible verification could have missed that they'd been working on this for a while. Asking a few of his former colleagues, for example, would likely have revealed the truth.

* Salesforce haven't yet acted to remedy this; which makes them appear complicit in it. I attribute their behavior to incompetence rather than malice. If it was a total fix from the start, there's no reason to exclude projects started before the hackathon in the rules.

The Upshot product itself looks fairly good, so I feel sorry for them in that
they could do even better financially if they hadn't won. But not that sorry,
given they have a million dollars cash-in-hand :-)

To be fair to Salesforce, their million dollar prize seems to have backfired
rather spectacularly with the very audience they were trying to win over. They
are probably more annoyed than anyone, and are probably having to tread very
carefully for legal reasons. Even if they reclaimed the prize and gave it to
#2, it seems that many of the top 5 may also not be entirely above suspicion,
and they would want to be doubly-sure of those entries. They'd also presumably
have to figure out who was #6 and give them the (new) 5th place prize.

All in all, a major mess.

------
dirkdk
hackathons should go back to what they used to be: great events to try out new
technology, and meet up with fellow engineers while having fun

------
danso
The best defense for Salesforce, in light of the circumstances, is to argue
that if they had really intended to scam everyone, would they be this dumb
about it?

~~~
couchand
Yes. It is, after all, Salesforce.

------
yeukhon
Where are the winners? I am sure they are on HN. Why are they so silent on
this matter?

~~~
argonaut
Uh... Not everyone is on HN. It's presumptuous to assume that because someone
hasn't posted here, they're being "silent" on the matter.

~~~
yeukhon
I will be fair and yes not everyone. But they are the winners and we are
poking at this matter for a day and I would be surprised they haven't checked
HN (I am sure their friends are telling them about what readers think of them
and Salesforce).

~~~
argonaut
... That's still presumptuous of you. For example, I don't check Reddit. Ever.
Is it so hard to believe that some other people who no doubt have something
better to do with their time don't check HN? [1]

[1] Rhetorical question.

~~~
yeukhon
Sorry, I beg to differ in this story. Are you saying these two engineers have
no idea what the hell is going on and that they have not been told about what
is being discussed here? These guys are software engineers and they sure have
friends and co-workers who are HN readers. And they being the main party
involved you think they haven't visited the site is one big fat red
presumptuous of you in your own word.

If I accuse you being a cheater you sure won't check HN. I am pretty sure.

------
devanti
This is why I don't attend hackathons anymore. At the bigger events, there's
always bound to be controversy, cheaters, questionable criteria, etc.

~~~
jskopek
Perhaps you are attending hackathons for the wrong reason? I didn't personally
attend, but I'm sure anyone who went in order to build an interesting indea
with smart hackers probably had a decent time.

Allegations of cheating are serious, but they shouldn't affect your enjoyment
of the event unless your main purpose was the win the prize.

~~~
devanti
For the wrong reason? What's with all these smartie pants on hackernews? A big
hackathon always has some sort of incentive, and aiming for that is wrong?

I don't know about you, but I enjoy winning, and I've won hackathons before.
If I just wanted to build something with my hacker friends, I can do it any
weekend I want.

------
msoad
My first reaction to this was "it's too good to be true". Now I won't wonder
if it wasn't true

------
leeoniya
the article is stuffed with salesforce.com backlinks, including a nice
salesforce profile at the bottom.

any publicity is good publicity.

~~~
johnpowell
The summary of the company is the best part.

With more than 100,000 customers, salesforce.com is the enterprise cloud
computing company that is leading the shift to the social enterprise. Social
enterprises leverage social, mobile and open cloud technologies to put
customers at the heart of their business.

I don't know what this means.

~~~
CalRobert
It means some obnoxious twit telling you to start using Chatter instead of
email, and spend half your day using one of the worst UI's known to man
instead of getting work done.

------
philthesong
Marc Benioff has no credibility

