

Scribd says: Beat our programmers at a coding challenge - jennylin
http://coding.scribd.com/2012/05/24/compete-against-our-developers/

======
tazzy531
For anyone that is doing CS at Columbia University or considering CU, if you
like these types of coding challenges, definitely look into Professor Ross's
Programming and Problem Solving class. It was one of the best CS classes I
took at Columbia.

Here is his website: <http://www.cs.columbia.edu/~kar/teaching.html>

Each semester consists of 4 different open ended programming problems like
these. You work as a team to compete against other members of the class.
There's no tests and each class is run as an open seminar where people talk
about their strategy and implementation and consider the best approach to
solving these problems.

This was my favorite from my year:
<http://www.cs.columbia.edu/~kar/4444f02/node18.html>

~~~
pcopley
It always blows my mind how many CS academics have these god-awful mid-90s era
webpages.

~~~
biot
It's worse than that. This webpage presents the information in a clear,
readable text format with no unnecessary clutter. Why, it's such an old design
that it's similar to what Gutenberg was cranking out on his printing press and
to what you can buy today on your Kindle. Imagine that!

------
qbrass
On the game rules page: <http://www.scribd.com/jobs/botrace_rules>

The game starts off with Apples Bananas and Oranges, but ends with Apples,
Bananas and Melons.

Robots are a wise choice for safety's sake when dealing with quantum fruit.

~~~
matthiaskramm
Fixed, thanks :) We'll deploy a new page tomorrow.

------
Kototama
I've got a coding challenge for the scribd team: make documents downloads one-
click away from the document page and without mandatory login.

------
a1k0n
You may need to consider a better way to rank players - percentage of games
won is problematic (as was discovered during the Tron Google AI challenge) and
something like TrueSkill or bayeselo would be more robust.

~~~
matthiaskramm
Yeah, we're seeing the first problems with that right now- newly uploaded bots
progress faster than they should because they play against a more "diluted"
pool of opponents than the oldtimers did. We'll think of something.

~~~
a1k0n
I also think your alpha-beta bot is objectively better than the (currently) #1
bot.

Easiest thing to throw together now is a script which just feeds the whole
game history into bayeselo as a bunch of fake chess matches, with the white
move advantage set to 0. That's what was done for Tron.

~~~
berdon
:(

------
maayank
The more I think about it, the more I like it as a recruiting venue[1]:

* A good way to measure the potential recruit skills

* Gives a good fun experience to go over later at the interview, both verifying it is the recruit who coded it and setting common background beforehand

* You get people who showed some interest in your company, seeing the contest

* You get people who are interested in hacking around puzzles

[1] (I don't know if scribd sees it that way)

~~~
backspace
Adding to what you said, someone who discovers, attempts and solves this
solution is already better than 70% of the programmers out there. They show
promise of being motivated to crack a puzzle that robs them of their sleep,
and take the time to learn and code against the Scribd API.

Even if they don't "win", they are greatly superior than a normal resume
pusher.

------
tjsnyder
I'm pretty sure I was the guinea pig for this system and I have to say, it was
one of the more enjoyable interview tasks I have had to perform.

There is just something extremely satisfying about watching your little robot
run across a field collecting little fruit.

~~~
rorrr
If we started asking this as an interview question, NOBODY would pass. Time
after time we have "senior" developers who fail to write very simple
functions, like to count the number of occurrences of each letter in a string.

For me, personally, it sounds like an awesome question, but it's not clear how
you would judge it. Let interviewees' bots fight?

~~~
kwamenum86
Wait...when you say fail are you saying they fail to produce something optimal
or fail to produce working code?

~~~
rorrr
They fail to produce working code. I've written about this a number of times.

Most candidates CANNOT write a string reverse function without the off-by-one
error.

------
cfqycwz
Smart that they implemented a 10-second time limit, because my first thought
was to write a bot that would pass the current game state to Mechanical Turk
and have a person solve it. It would be interesting to see how a human fared
against Scribd's developers' bots.

------
gridspy
...Must keep working on Gridspy.... Must not enter fun contest...

 _twitch_

------
matthiaskramm
Btw. here's a game, played between our two best bots so far:

<http://www.scribd.com/job_game/match/263907>

~~~
icefox
Can we use nacl?

~~~
matthiaskramm
We _do_ plan to add some additional languages. Next on our list are: Haskell,
Lua, Python.

~~~
icefox
Sorry to be specific is the sandbox chrome or just a js engine? If it is
chrome I could compile and run a nacl module to improve speed.

~~~
matthiaskramm
It's using SpiderMonkey right now. Using nacl as sandbox is a neat idea
though.

------
bravura
Given that each move must be made in 10 seconds, here's my approach. I'm
proposing that you use machine learning to evaluate moves, to approximate the
exact (search-based) scoring. I'm not implementing this, but feel free to ask
me for more details:

Training phase:

Create a random board configuration. Exhaustively explore the search space to
find whether this is a win +1, lose -1, or draw 0 for Player 1. You now have a
training example: (board configuration, game outcome)

Now, train a neural network (or other non-linear ML model, e.g. SVM) to
predict the expected outcome based upon the board configuration.

Deployment phase:

Port the neural network to Javascript. For each possible move, use the neural
network to predict the outcome of that move. Pick the move with highest
expected outcome. The neural network will run in constant time, most likely
well under 10 seconds per move.

~~~
alxv
> Create a random board configuration. Exhaustively explore the search space
> to find whether this is a win +1, lose -1, or draw 0 for Player 1

Given that the search space can grow O(4^mn), this can be done only for
endgame configurations. Further, not knowing any bounds on the board size
makes the input representation difficult to define for a such machine learning
approach. And, your target should probably be the weights of an evaluation
function, rather than the exact game outcome.

As for the learning algorithm, I know TD-learning was found to be a good
approach in various chess programs.

> For each possible move, use the neural network to predict the outcome of
> that move. Pick the move with highest expected outcome.

You would likely still want to run an alpha-beta search to pick the move to
minimize the prediction error.

------
lacksconfidence
For people that havn't studied algorithms much, where is a good start on game
playing algorithms? Pointers to papers, or names of data structures to look
up? Perhaps <http://erikdemaine.org/papers/AlgGameTheory_GONC3/paper.pdf> ?

~~~
Jach
Combinatorial game theory would be the wrong tool for this, I believe, since
this isn't a strictly finite turn-based combinatorial game. (And in my opinion
combinatorial game theory is more useful for analysis and proof techniques
than for creating AI algorithms, but I've only had one class on it.) The blog
mentioned "fruitwalk-ab" uses minimax (with the alpha-beta optimization of
course), which is the bread-and-butter algorithm for these kinds of games and
I expected it to be in 1st place. Sure enough, it is at the moment. (Edit2: No
longer.)

In an intro machine learning course you'd learn about minimax and others, but
skip paying any money and just read the basic algorithms here and look up the
wiki pages for even more examples:
<http://www.stanford.edu/~msirota/soco/blind.html> (The introduction has some
term definitions.)

Edit: Also, the obligatory plug for _Artificial Intelligence: A Modern
Approach_ [http://www.amazon.com/Artificial-Intelligence-Modern-
Approac...](http://www.amazon.com/Artificial-Intelligence-Modern-Approach-
Edition/dp/0136042597)

~~~
lacksconfidence
Thanks for the info, interesting stuff. I've got more first-gen minimax going
right now, sorta works but gets stuck in a repeating move pattern. Interesting
stuff to work on though, i really appreciate the pointers. Hopefully this
weekend i can work out a better heuristic valuation of a game state, for a
first run i'm just counting my objects vs theirs(very simple), but i need to
somehow weight it such that it prefers picking objects up sooner than later.
And of course it would better match if it also took into account groupings,
but thats for after i get basic board-clearing working.

------
epaik
Hey, I'm writing a bot for this.

One thing though: I think resetting the board state should call new_game()
again. This likely only matters when testing your bot, but it's nice for
variables that need to be instantiated per each game.

It's pretty fun! My current greedy solution is called: SoGreedy. :)

~~~
matthiaskramm
Fixed, thanks!

Your bot is doing pretty well already. :)

------
Trufa
Hey, honest question here, is it impossible to brute force this and consider
(almost) every possible move and get to a non possible to loose point? Given
that the board is restricted to 15x15 and the fruits to 81... I'm not saying
this is easy, I'm asking if the game AI has an upper limit so to say...

~~~
egowaffle25
The instructions say you need to make a move in 10 seconds, but that's the
only ceiling I can imagine for this approach.

~~~
Trufa
Ohh, I missed that, that makes more sense now, thought ten seconds is quite a
lot, you will very probably not be able to brute force every possibilities
before every move. Thanks!

------
michaelvillar
[Sorry for bug report here, didn't find another way] I think there is a bug
with get_my_item_count() and get_opponent_item_count(). These two functions
don't return decimal. When players have 0.5, they return 0. This works locally
though.

Example of failed match : <http://www.scribd.com/job_game/match/295703> This
line for example 1 / 1 / 0 / 0 / 1 \- first number is the fruit type \- second
number is the total fruits \- third number is my fruits \- fourth number is
opponent fruit \- fifth number is irrelevant (You'll never see 0.5 in the
third or fourth number..)

------
JesseAldridge
Bot compilation seems to be stuck on this guy:
<http://www.scribd.com/jobs/botrace_bot/58>

~~~
matthiaskramm
Fixed.

~~~
pcopley
I'm getting a 404 when going to any bot page.

~~~
matthiaskramm
Log into www.scribd.com, that'll make viewing of the bot pages work. We'll
deploy a fix for that later today.

~~~
pcopley
Thanks!

------
michaelvillar
I think there is some kind of bug where a bot is stopping. Example :
<http://www.scribd.com/job_game/match/275573> I simulated two boards where
this bug happens online, and it doesn't happen locally.

Also, the API is wrong here : "return the index of that fruit (starting with
0)" => It starts at 1

~~~
matthiaskramm
Thanks! We'll fix the API docs.

About the bug: Try to trace() out a few messages in your make_move() function,
that might tell you whether something went wrong (the logs will appear at the
bottom of the game replays.)

~~~
michaelvillar
OK. Will do that thanks !

------
evanlong
Scribd has Kramm. You have already lost.

------
davidjhamp
I think they need to reevaluate how they score a tie, out of 58 games I have 4
losses and 30 wins and the rest are ties, but my percentage of games won is
only 50%ish. Seems like a tie shouldn't be put in the same category as a loss.

------
neilparikh
I see there's a limit on how long the moving function can run. Is there a
similar limit on how long the new_game() function can run?

~~~
matthiaskramm
10 seconds as well.

------
steve8918
If would probably make it easier if they had an interactive version, so that
we could learn the game before starting to program.

~~~
egonschiele
The API is _really_ simple. The simplest version of a bot is:

    
    
        function make_move() { return EAST; }
    

We also provide a downloadable environment so you can test your bot locally.

------
donall
Why are the fruit counts represented as floats?

~~~
epaik
They didn't really have a choice in the matter! In JavaScript, all numbers are
floats.

~~~
pyrotechnick
Incorrect: <http://www.khronos.org/registry/typedarray/specs/latest/>

Besides, there's Number.prototype.toFixed().

------
pantaloons
What are the grid-size and fruit-count limits like on the server -- the same
as in the standalone client?

~~~
matthiaskramm
Yep! Width and height are between 5 and 15, and there are up to five different
types of fruits on the grid.

------
MrVitaliy
This looks like a perfect application for using a genetic algorithm solution
with two competitive agents.

------
pantaloons
Looks like there are permissions problems when emailing botrace@scribd.com

------
ak0s
Stingy v1 James Ojaste 1200.0%

What's happened with the ranking?

------
swang
How long will this contest be up? Or will this be ongoing?

~~~
matthiaskramm
We're planning to keep this running indefinitely.

We may also add additional languages in the future, like Haskell and Lua, and
possibly some advanced playing modes.

~~~
eridius
Might I suggest Go? It's already sandboxable, demonstrated by the "Try Go" box
right on the front page of golang.org.

------
shuaib
Seems like the upload a bot link is down...

~~~
shuaib
It should be mentioned that you need to register in order to submit. That was
my issue.

However, while my bot was compiling, I moved away from the page. Later on
trying to resubmit, I kept on getting errors, until I changed the name of my
bot submission.

More helpful error messages would greatly improve the whole process. :)

------
bosky101
this is cool. submitted mine, cross-posted to a few places mine's are prefixed
with bosky101 :) ~B

------
duked
can't upload a bot seriously ....

~~~
matthiaskramm
Aw, that sucks. We're still fixing glitches in the system, but uploading
actually should work.

What kind of error are you seeing?

~~~
PurplePanda
I'm getting 504 Gateway Time-out The server didn't respond in time. on
<http://www.scribd.com/job_game/process_uploaded_bot>

~~~
christiangenco
Same here when trying to upload my bot the first time.

I WANT HIM TO LIIIIVE AND PLAY WITH OTHER BOTS IN THE REAL WORLD ;_;

~~~
matthiaskramm
Uploading bot revisions is broken right now- you can try to upload a new bot,
or wait for our fix which should go live in a couple of hours. :)

~~~
matthiaskramm
Also, feel free to contact us at botrace@scribd.com if you see any more
issues.

------
fozzle
This was so much fun! Thanks scribd. :) Humble Hodor and Greebly Wartfinger
gooooo.

------
danso
I was going to post how they should do a clearer job of describing the
challenge (i.e. using bullet points, headers) and main rules, but I guess
there's a separate webpage devoted to the challenge:

<http://www.scribd.com/jobs/botrace_rules>

------
AjJi
compiling indefinitely...

------
dclowd9901
Ah, the new generation of the "design contest."

Thanks, but no thanks. Hire better developers if you need your algorithm.

~~~
feral
It seems unlikely to me that scribd are actually looking for a solution to
this problem, or an isomorphic one. Have you any basis for thinking that?

There are many standard approaches to the general class of two player board
games (they are clearly aware of these, with an alpha-beta based bot).

I would guess that what is going to differentiate winners from losers in this
specific game are the exact specifics of the setup. So if they are looking for
a solution to an isomorphic problem, it must be a _very_ similar problem - as
the general type of challenge is well understood.

It just looks like a neat game to me; I doubt its a cleverly disguised problem
from 'social reading and publishing' that they are trying to get a cheap
solution to..?

~~~
dgabriel
I'm with you. It just looks like a lot of fun. My ten year old is really
interested in just playing for the sake of play. So we will!

