

Ask HN: I am in a David vs. Goliath fight and I need some advice - __mtb__

I participated in the GE FlightQuest competition hosted by Kaggle.com. I had the 4th best score, but because of what other competitors and myself view as a software bug on Kaggle.com's platform [1], I was not recognized as a winner when GE and Kaggle announced the prize winners. For reference, the 4th place prize winner would have earned 30,000 USD.<p>I notified Kaggle.com within ~1hr of their announcement of the mistake and they did not get back with me for 2 days. 8 days after the announcement was made I was finally able to get a call with a Kaggle executive. The executive offered me partial prize money and kaggle competition points to resolve the matter unofficially. I said I wasn't interested. Kaggle then opened a new forum discussion [1] which a number of other competitors chime in and explain the limitations/bugs of the Kaggle.com platform specifically as it related to the FlightQuest competition and discuss my specific situation.<p>Finally, an executive from GE contacted me this morning over email and let me know that he supports Kaggle.com's decision of not recognizing me as the 4th place finisher.<p>I feel like Kaggle/GE have not handled this properly and I would like to get some opinions on what, if anything I should do.<p>[1] https://www.gequest.com/c/flight/forums/t/4284/acknowledging-two-more-great-competitors
======
runjake
Get a lawyer is the best advice you'll get -- just like every other time
someone Asks HN what to do with (potential) legal matters. Apparently, most
lawyers will do some sort of initial consult for free. Also, might be a good
idea to delete this post ASAP, if you're seriously considering legal action.

~~~
__mtb__
I have not consulted a lawyer yet, the reason being the competition rules
state the following:

"By participating in the Competition, each Entrant agrees to release,
indemnify and hold harmless GE, Kaggle Inc., and their respective affiliates,
subsidiaries, advertising and promotions agencies, as applicable, and each of
their respective agents, representatives, officers, directors, shareholders,
and employees from and against any injuries, losses, damages, claims, actions
and any liability of any kind resulting from or arising out of your
participation in or association with the Competition. GE is not responsible
for any miscommunications such as technical failures related to computer,
telephone, cable, and unavailable network or server connections, related
technical failures, or other failures related to hardware, software or virus,
or incomplete, late or misdirected Entries. GE reserves the right to cancel,
modify or suspend the Competition should any computer virus, bug or other
technical difficulty or other causes beyond the control of GE corrupt the
administration, security or proper play of the Competition, and to determine
winners from among Entries not affected by the corruption, if any, in its sole
discretion."

I interpret that as saying: GE can pick the winners using its sole discretion.
So I don't believe this is a legal issue I can win. I have a lawyer in the
family, I will run it by her and see what her opinion is.

~~~
lifeisstillgood
Please, get a lawyer outside the family with experience of competition law -
maybe your sister/aunt can tell you the area of law that might apply and point
to some local firms

You need objective advice, not coloured by "what if my advi e leads to not
speaking to half the family" - don't put that on her.

Secondly, consult a lawyer because they have - if according to the common-
sense rules you should get 30k (and I don't at all understand the situation)
then no amount of disclaimers or indemnification deprives you - statute on
competitions run for marketing purposes are widespread.

Maybe you have a case, maybe not, but without a lawyer you won't know and
without a lawyer they (GE?Kaggle) won't take you seriously - which kind of
screws the only other reason you seem to have for not getting a lawyer - the
idea that maybe you will pick up some work later on. If they see you as a push
over on this, why will they respect you next time?

This does not mean drop a writ for 10 million in the morning - get advice, if
there is a case, ask if the lawyer can directly contact the organisers lawyers
for clarification or discovery. If you have a case they will have to do
something - push now, so that their decision has not been set in concrete and
them moved on.

------
__mtb__
Also, does anyone else have a problem with Kaggle offering money and points
for resolving the matter unofficially?

When the offer was made, I was really surprised because it seemed unethical
and unfair - both the the other competitors as well as the sponsor.

Just curious what other, non-Kaggle competitor opinions are on this topic.

~~~
bcoates
You have a dispute with them. They're running a business and want the dispute
(and you) to go away. Eventually, you will go away, either because you gave up
or lost in court; or because they paid you to do so. If you're ethically
entitled to compensation, this is a perfectly decent thing for them to do.

------
frankacter
tl;dr: contestant submitted 23 entries to a contest, rules state you have to
select one for official calculation of your score, contestant changed his mind
a few times and ultimately selected an entry that did not have enough points
to win despite having another entry that did have enough points.

\--

In reading Anthony response, this seems less about a "bug" but more about you
not being happy with the results and blaming the complexity of the submission
and selection process.

This part of his response seems fairly clear:

"The submission page stated: “Note: You can select up to 1 submission that
will be used to calculate your final leaderboard score. If you do not select
them, up to 1 entry will be chosen for you based on your most recent
submissions. Your final score will not be based on the same exact subset data
as the public leaderboard, but rather a different private data subset of your
full submission. Your public score is only a rough indication of what your
final score might be. You should choose an entry that will most likely be best
overall, and not necessarily just on the public subset.”

During the course of the competition, you made 23 submissions of which 3 were
selected for leaderboard scoring. First you selected submission 242748 on
January 7, 2013 (all times UTC). You unselected that submission on January 24,
2013 and selected submission 250852. You unselected that submission on January
26, 2013 and selected submission 252143. You did not revise that selection at
the time of your final submission. It seems logical to conclude that you
understood the process of selecting submissions. Three of the other winners
selected their final submission for scoring. The two that did not select the
final submission had not previously made a manual selection or had unchecked
prior submissions and the software worked as designed and selected the final
submission. One participant (but not a winner) made 3 final submissions and
selected one for scoring - which was the case the software for selecting a
response was intended to handle. There was no bug in the software. "

While unfortunate for you, it is worth noting that all of the competitors had
to follow the same set of submission guidelines to be considered. Your
response becomes a debate on how you “feel” the submission process should
work, as opposed to indicating a “bug” in the software.

“What on earth is a checkbox doing on the submission screen of a two phase
competition? There is no ambiguity - there is only a single _file type_ that
is valid for the final leaderboard file. Why on earth is your software not
smart enough to figure that out?

Also - why didn't the default selection logic kick in? Because only final
leaderboard files are valid for the final submission, why didn't your software
pick the _only_ valid option?”

~~~
__mtb__
That is a good summary, but it is a little bit more complex than what you have
stated. But your comment is helpful because this is exactly the understanding
that Goldbloom has and it is incorrect in an important way.

The 23 entries that were submitted are for the public leaderboard set. The
purpose of this set is so over the duration of the competition, competitors
can see how they are performing relative to each other. These 23 submissions
have no impact on the final prize winners.

A different data set, called the final data set is what actually determines
the final prize winners. I submitted this file exactly one time. It was
processed and scored by Kaggle and I followed the directions exactly as the
competition admin stated. I did not check a checkbox because I did not believe
it applied - I only submitted a single file and I recieved the 'all zeros'
response from Kaggle's system as it said it would.

Also, the text Anthony quoted is not applicable to the Flight Quest
competition either. This is the text that is used for the single phase
competitions, not a two phase competition like Flight Quest.

If you haven't competed before I know the difference might not be clear. But
if you read the response from the other competitors in the forum, you can see
there is general agreement that the submission page is faulty for a two phase
competition.

~~~
frankacter
Thanks for your reply and clarification.

1\. So your stance is that Anthony, the CEO and Founder of Kaggle, is mistaken
about the rules and execution of the 2 phase competition that his company is
running, even after 8 days of his company communicating back and forth in
email correspondence with you?

2\. In the forum you initially state:

"As I have been saying over email repeatedly over the past 8 days, and
yesterday on the phone with you and the Sorkin, there is no checkbox to
check!"

..then later state:

"What on earth is a checkbox doing on the submission screen of a two phase
competition? There is no ambiguity - there is only a single _file type_ that
is valid for the final leaderboard file. Why on earth is your software not
smart enough to figure that out?"

and later:

"I submitted exactly one final submission. At this point the checkboxes should
not matter"

and in your reply above state:

"I did not check a checkbox because I did not believe it applied"

While I'm not familiar with all of the intricacies of the competition,
conflicting statements like those have me a bit confused over this whole
thing.

3\. $30k is a lot of money, so I sympathize with you for missing out on that
given that your algorithm is better than what the final results suggest. That
being said, the other entrants that won competed using the same submission
process that you did, was it just bad luck that your entry was not accepted in
the manner you had expected or why did they not also encounter this submission
bug?

4\. Is this your first two phase competition with Kaggle? Was this competition
executed different than other competitions? I ask as you seem
surprised/confused about this "checkbox" that appears to be at the center of
the controversy.

~~~
__mtb__
Thank you for reading the forum trying to better understand my POV.

re: #1. Yes, my stance with Anthony is that he doesn't think there is a bug in
the software because he thinks this is a typical one phase competition. Flight
Quest was more complicated than the typical Kaggle competition.

re: #2. I have done a poor job explaining the checkbox. I understand the
confusion, as my statements are contradictory. In a single phase competition,
the checkbox is important because over the course of the competition, numerous
submissions are scored and you ultimately need to pick the one you want to
apply as your final submission. In the case you do not make a final selection,
the system selects the most recent submission.

The same UI is used for the two phase competition, including the same text
that Anthony posted in his response to me, however the submissions work
slightly differently as I mentioned.

> "As I have been saying over email repeatedly over the past 8 days, and
> yesterday on the phone with you and the Sorkin, there is no checkbox to
> check!"

I said this because I don't believe there should have been a checkbox on the
screen. That is what I meant by saying there is no checkbox to check.

------
ScottWhigham
IANAL but here's what I think an attorney who took the time to look the whole
thing over would say:

The language in the contract (that you posted) seems pretty cut and dried, but
I don't know whether that's actually going to hold up in a court of law. You
entered a contest expecting the contest holders to hold the contest correctly.
When the flaw was exposed, they did not honor it. That means your work was in
vain. That's not in the spirit of the contest, thus the written words are only
part of the story. A reasonable person expects that the company running the
contest does so fairly. You seem to have not been treated fairly thus I think
you have a case. I would imagine that, if you posted up a $500,000 lawsuit
with $30,000 of reward and $470,000 of punitive damages, you'd hear from a
different GE exec who has a very different POV.

~~~
lifeisstillgood
In the UK at least there are quite stringent rules on how you can run a
competition - give away a car? Abide by the rules.

~~~
__mtb__
I did not realize this might be a gray area.

I will do some research on how I can find some representation. Any tips, or
experience with who I might get in contact with? Am I looking for some sort of
technology or competition specialist?

~~~
lifeisstillgood
[http://www.socialmediaexaminer.com/social-media-
promotions-a...](http://www.socialmediaexaminer.com/social-media-promotions-
and-the-law-what-you-need-to-know/)

I suggest you google for lawyer sweepstake New York

And lawyer contest law New York - talk to a couple till you get a feel for it.
It about exploring your options not going to court tomorrow, so the gung-ho
ones might not be best. But that's a Brit talking :-)

------
brudgers
Sounds like they tried to mitigate the situation in a reasonable manner. If
they name you as fourth, then the formerly fourth place finisher gets upset.

Some things are like futbol - there's no instant replay. Lawyering up isn't
going to impress anyone.

~~~
__mtb__
I get the instant replay analogy, but I have a problem with a bug in their
software impacting the results of the competition. I work in software, so I
get living with bugs and understanding they are always going to be there. But
I am unhappy that Kaggle/GE announced the winners via press release at the
same the rest of the competition field was notified. It makes it impossible
for a competitor to dispute the final results.

From the response I have received from Kaggle/GE I get the impression, under
no circumstances what so ever would they go back and change the published
results of this competition.

~~~
brudgers
> _"It makes it impossible for a competitor to dispute the final results."_

Yes. That's the purpose.

The teleology of gamified competition is toward declaring a winner. The rules
are structured with that in mind. Those rules are different from the
marketplace where protesting and litigation are allowed and consequently,
declared winners are simply a matter of opinion, not fact.

Accepting mistakes as part of the game is sportsmanship. And it was a game.
Best to move on to the next one: <http://www.kaggle.com/competitions>

~~~
__mtb__
>'... declared winners are simply a matter of opinion, not fact'

I certainly agree with that part of your statement.

I have a hard time thinking about moving on to a new competition, this one has
left such a bad taste in my mouth. Maybe time will change my mind, but
certainly not happening any time soon.

~~~
brudgers
I understand where you are coming from.

A bit of my background - I referee futbol (or as we call it, soccer). Every
game, I try to give my best performance to the players. Sometimes, I make bad
calls. The score is still the score.

Obviously, you're pretty damn good. Your competitors deserve to compete
against you. Cheating is unfair. Mistakes aren't. Play long enough and it all
evens out. At least statistically.

Good luck.

------
ActVen
If you are interested in working with/through Kaggle on a longer-term basis,
finding some solution outside of a lawsuit would be something to think about.
You are getting a chance to interact with them and maybe build contacts that
could be useful down the road. If the offer was insulting and/or you don't
want to work with them down the road, that is another matter. When you start
bringing in lawyers on both sides it becomes an entirely different matter than
what it "right". It becomes a costly legalese/terms/courtroom issue. Good luck
either way.

~~~
__mtb__
Ironically, that has been the best part about this ordeal - a number of the
top competitors have shown support through the forum as well as private
emails. I compete alone, so this issue has given me the opportunity to get to
know some of the other folks.

I am not certain I want to work on anymore Kaggle competitions. I put (just a
guess) somewhere between 100 and 200 hours into FlightQuest. It is hard for me
to rationalize putting this amount of effort into another Kaggle hosted
competition after going through the experiences I went through for this
competition.

