
Designing and Building Stockfighter, Our Programming Game - jsnell
http://www.kalzumeus.com/2015/08/20/designing-and-building-stockfighter-our-programming-game/
======
Chirael
Maybe it's from the length of the message or the general impressiveness of the
project, but the first thought that came to mind was, "Wow, I'd have to be
unemployed to have enough time to work on this - there's no way I could tackle
these challenges in the real world otherwise."

Worse, if this is something that's designed to showcase one's technical skills
to potential employers, I don't imagine there's a way for the system to
distinguish between "took a long time solving challenges because just not
smart enough" vs. "took a long time solving challenges because have a day job
and a life outside of work/tech".

Since this is designed, essentially, to showcase how smart someone is to
potential employers, assuming solving time is a factor, it's almost better not
to compete until you know you'll definitely have lots of time to devote to it,
because otherwise there is the risk of taking too long and looking not-good.

I'll add it to the long list of "things I really, really wish I had time for"
\- or I'll assign it to clone #1023 once that becomes viable :-/

~~~
patio11
We want it to be playable by people who have demanding commitments like
families, jobs, etc. This affects our level design (the first six will
actually time out after 30 minutes; they're beatable with the right script in
seconds or minutes after you get it working or by hand in ~10 with play at
least as good as mine) and how we go about executing on the business.

Ultimately the mechanics of our matchmaking come down to "Starfighter spits a
heck of a lot of signal about you at Thomas/Erin/Patrick; we try to decide if
it makes sense to introduce to to clients and, if so, to which." All of us are
working parents. We will certainly not penalize people for prioritizing piano
lessons over being one of the first 5 people to crack a new level.

I won't commit to "I'll never look at time between a level is first opened and
when it is solved" but, as somebody who has fed his family with code for as
long as I've had a family, that strikes me as being among the least
interesting things I could possibly learn about an engineer's ability.

~~~
jscottmiller
After para 1, I was going to ask "what's to stop someone from tossing a
solution in a pastebin and sharing it around?" Related, "what's to stop
someone from sharing the questions ahead of time?"

From the second, it sounds like you guys will seek out the top performers and
weed out the bozos. I like it! Are you looking at a traditional recruiter fee
model or something else? Apologies if this is posted somewhere else.

~~~
yaur
The big problem isn't some bozo posting it to pastebin, it's shady
"recruiters" that are set up to make their candidates look awesome. This is
extremely difficult to detect without on-site validation.

------
quot66555
I'm very interested in Starfighter and am looking forward to the release.
However, there's something from your original announcement that still worries
me. In this post you said:

 _The designed operation of our trading levels is “Play them, lose horribly,
write a program, reset the level, run the program, have it crash with a bug,
fix and re-run, lose modestly, adjust your approach, re-start the level, run
the program, win.” (Folks have asked whether we’ll hold this against you. No,
no, of course not. We log everything under the sun, but losing /restarting
levels is planned. That capability is an advantage of doing this in a rich
simulation as opposed to real life — you can go Knight Capital as many times
as you want and no one loses their job!)_

This feels good. But earlier, this was said [1]:

 _We will track player behaviors and skill at incredible levels of granular
detail, instrumenting them like they were built by the Orwellian MiniPeace...
We can tell you exactly what happened when your candidates tried to implement
a REST API. We can compare their performance against hundreds of other
talented engineers (including your current employees) on the same task._

Can you please elaborate on what exactly is going to be tracked? Is any of
this configurable by the player? Personally, I'm going to have a hard time
enjoying and learning if I feel like every mistake I make, or every extra
piece of time something takes is going to be judged and count against me later
on.

[1] [http://www.kalzumeus.com/2015/03/09/announcing-
starfighter/](http://www.kalzumeus.com/2015/03/09/announcing-starfighter/)

~~~
patio11
I feel like this is largely an issue about expectations and disclosure more
than it is about logging per se, right? We have a privacy settings page, which
I'll show you in a minute, but here's the public view of myself on the staging
server. I have it configured to Maximally Public (and will have that on in
prod).

[https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442...](https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442RUB/image.png)

The redactions are just to avoid spoiling levels. The "replay" link is a UI
templating bug. (It isn't for _showing_ a replay, it is a convenience method
for replaying the level, and it's now supposed to show unless you're logged in
and able to play the level yourself. Fixing this in the next 5 minutes.)

Now, if I (as a player) log into and view _my own_ profile, I get a better
sense of some stuff the system knows about me:

[https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-
ammvZA_3FD...](https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-
ammvZA_3FDfYB/image.png)

That is obviously not all the information the system has about me. If a
sparrow yawned in a world simulation that I took part in, that fact is dumped
onto an S3 bucket, somewhere. To be maximally explicit: every order I sent in.
Every execution I got back. Snapshots of the GM's memory state as I passed
through the levels. etc. We grab all that. That isn't an exhaustive list.
Assume if it is amenable to being discovered by a computer that it will be
discovered and persisted until the end of time.

This is not exposed publicly.

Here's the preferences UI. Verbose but it gets the point across, right.
Defaults are "Anonymous" and "No Public Profile", respectively.

[https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKX...](https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKXl4B/image.png)

Now to a question which may be implied: "OK, I now understand that my info is
locked up pretty tightly within Starfighter. But you want to tell people that
eventually, right? That's how you justify your service to clients." At the
point I'm on the phone with the CTO of $CLIENT I am a _contigency recruiter_.
That means I _only get paid_ if that CTO decides to interview you and
ultimately hire you. You can bet that I will be doing my best Enterprise Sales
work and showcasing your performance in its most positive truthful light. If I
did not feel like that would be possible, that phone call never happens. If
the CTO had previously mentioned to be "Be on the lookout, specifically, for
people who are good at analyzing mountains of data. We need them for our fraud
team.", and I had relevant signal on that question from level X, you can be
reasonably assured that if you did really well on it I'd open the call with
"The reason I'm recommending Foobar to you is because you told me you wanted
folks who are really good at analyzing data. Let me walk you through the
solution Foobar came up with -- it's a hoot."

(Worth re-iterating explicitly -- that that point, we've talked on the phone
and you've explicitly given me the go-ahead on placing you.)

To the question: "OK, so I get that people outside the system don't get
arbitrary read on my history, and I understand that you're essentially a
firewall which is incentivized to only allow the egress of the best possible
true packets: what about _you_? Aren't _you_ going to be influenced by, you
know, taking a look directly at my stream of HTTP requests and seeing that I
fumble fingered a URL the first time I played the level?" To which I say: I
don't care and don't have time for that nonsense -- I'd much rather
concentrate on things which _actually matter to an engineer_.

~~~
quot66555
Thanks for the thorough response. Yeah, pretty much. It's good that you all
have thought about this stuff. There's always a PVP aspect if you're ever
going to put yourself public or on the leaderboards, but that's just part of
the game. I can always start out anonymous and go public if I turn out to be
any good...

~~~
j_s
account1: practice.

account2: 'for-realzies'.

Succeeding by whatever metric because of this? Let the true games begin!

~~~
tptacek
That "account2" isn't going to do you any good that figuring things in
"account1" didn't already do for you, but knock yourself out. ;)

------
EFruit
I have to say, this is both impressive and _infuriating_ ; they beat me to it!

My effort was started in Nov. 2014, and is similarly written in Go. Clocking
in at ~1600 LoC, it is certainly nothing special. I have no background in
finance and only a cursory understanding of the technical aspects of the
exchanges. It's mostly working, but I've really stalled because I can find
neither a real purpose to continue, nor any resources on building interesting
models. All it does is make financial noise (random trades between similarly
irrational parties)

All in all, I'm happy for the guys working on this for doing what I apparently
couldn't.

If anyone's interested, I'd be happy to post it to Github.

~~~
mattchew
I'm interested.

~~~
EFruit
Alright, I'll clean it up and post it ASAP. Beware, it's (probably) awful,
terribly unidiomatic code; I'm not a professional.

~~~
mattchew
Thanks for sharing.

I'm only a Go novice myself, but it doesn't look bad to me. Doesn't need to be
perfect for me to learn something from it.

------
pjungwir
Patrick, you should just admit it that the reason you are starting off with a
financial theme is so you can include a level about breaking into a Bitcoin
exchange. :-)

More seriously, to somewhat address the "8000 words?" folks, the runup to
Starfighter reminds me a lot of StackOverflow. Both seem like projects that
would have flopped if not for the creators' existing audience and great care
in cultivating excitement by explaining in depth the need, the goals, and the
philosophy of the project. It'd be interesting to revisit the early
StackOverflow marketing and compare the two more closely.

------
ctz
I do wonder about the logic of collecting the email addresses of interested
parties, then sending them 2000 and 8000 word (respectively) essays. Do people
actually read and value such long emails? Do you not lose a bunch of on-the-
fence users each time you send them an email which isn't immediately useful or
actionable?

Then again, I know very little about running a startup; patio11 has a track
record. Maybe this works as a strategy.

~~~
patio11
_Do you not lose a bunch of on-the-fence users each time you send them an
email which isn 't immediately useful or actionable?_

That's a number which is trivially calculable. I don't know what it will be
yet, but if you asked me to pick the over/under, I'm in for 80 out of 16k
people on our list at present. An unsubscribe is not "I hate you guys and wish
you would die in a fire"; it's the email equivalent of a browser's back
button. "Not for me, not right now." Many startups let each unsubscribe punch
them in the gut. Don't do that. The only way to avoid unsubscribes is to avoid
email. You _profoundly_ don't want to avoid email.

We've sent ~2 emails since... March? If I were a consulting client of myself,
I'd have written somewhere between one and two dozen.

Is this email going to produce business results for the company? Hard to say.
Wild guess? I get a contact in the next hour from one of the companies who has
a stalled LOI saying "Oh great you're shipping let's get that signed."
Depending on which company that is, the economics are worth hundreds of
thousands of dollars.

Many professional email marketers strongly disagree with writing long emails.
I will chalk that up to a respectable professional disagreement. They work
_very_ well for me, by any trackable metric and by most of the intangibles.
I've never felt like I created much value with the little 125-word-and-a-
picture squibs that the industry blasts out by the hundred million; I know
(because people tell me) that some people like my writing rather a lot and are
disappointed that I don't hit them with 8k words every Friday. As it happens,
a portion of Starfighter's list is folks who are in exactly that bucket.

Does that answer the question? Not trying to be very defensive about this,
just trying to help other startups out. The modal startup sucks at email
marketing. It is, far and away, the highest ROI channel you have and probably
the easiest to use well. You should send more email. You should send better
email.

~~~
jc4p
Just anecdotal evidence: The two e-mails y'all have sent since I signed up
have caused a pavlovian response, I literally left my office and sat down on a
couch to read your last e-mail the second I saw it in my inbox.

~~~
gknoy
I do that whenever I see anything from Patrick posted on HN, and getting it
delivered in my email box has a similar effect. I would say that I have an
irrational enjoyment of reading things that he writes, but I actually think
it's pretty rational -- I've never read anything by him that didn't exceed my
expectations of information, humor, and valuable content.

------
MichaelGG
This is awesome. I still don't see how this isn't easily cheatable. What's
stopping me from writing up a little tool that can win these levels and
providing minimal instructions?

At one point, it even says they will award people that write OSS for it.
Doesn't that just further reduce the effort needed to pass levels?

Not to be negative. I'm very, very much looking forward to this.

~~~
patio11
I have some ideas jumbling in my head about this, but it's getting late and
I'm not sure they're maximally coherent. Briefly:

a) What's the worst thing that possibly happens? Someone scams a row out of
our database? It has more rows. 10,000 someone scams a row out of our
database? I will write a for loop which scams _a million_ rows out of our
database. What about the actual world or Starfighter's capability to
positively impact our players, our customers, the industry, or our own
situations just got worse as a result of that for loop?

2) I think you will find that a solution which beats level six also sails over
the Turing Test, so if you have a tiny script which does so, I recommend not
burning it on Starfighter, but would be honored if you decided to try.

3) At the end of the day, the model is not "Someone pays us for a PDF
automatically spat out of our database", the trivialization of which PDF would
be dangerous to the business model. It is "We become sufficiently enthusiastic
about someone's performance that we successfully introduce them to a company
which is not us and then _that company hires them._ " Is the threat model "You
use a script which trivializes our levels to win a free conversation with
Patrick about your career?" Anyone can get that! At any time! Save yourself a
git clone; just send me an email!

The more interesting question is then, given that you've learned absolutely
nothing from Starfighter because you've git cloned your way to the finish
line, whether you can (in thirty minutes with me, or Erin, or Thomas) convince
us "Yes, this person is _totally_ someone I should burn my stack of
accumulated karma with the CTO of $CLIENT with to suggest they interview
them."

~~~
mbesto
> _The more interesting question is then, given that you 've learned
> absolutely nothing from Starfighter because you've git cloned your way to
> the finish line, whether you can (in thirty minutes with me, or Erin, or
> Thomas) convince us "Yes, this person is totally someone I should burn my
> stack of accumulated karma with the CTO of $CLIENT with to suggest they
> interview them."_

So, how is this any different from $CODER burning their accumulated karma
directly with CTO of respective $CLIENT ? Why should we expect them to behave
any differently because it's you, Erin or Thomas?

~~~
graeme
Not the coder's karma. Patrick's karma.

Patrick is saying that if someone is incompetent gets through with scripts,
they'll still have to pass a chat with Patrick before Patrick puts his
reputation on the line with a CTO.

~~~
mbesto
That's my point. Why can't all of the IT recruiters that exist today simply
just do the same thing? I'm failing to see how this is any different than the
current funneling/filtering system?

~~~
tptacek
They totally could do the same thing. They can come up with their own outreach
programs to start meeting lots of developers at scale on the Internet. They
can invent systems that reveal programming aptitude that isn't on the resumes
of people who have been stuck doing line-of-business J2EE apps at insurance
companies, or QA for enterprise products. They can hire people that can
competently talk to developers on the phone or in person and actually learn
things about them.

Maybe, with companies like ours and TripleByte in the mix, that's what they'll
start doing. Won't that be interesting?

We're a bootstrapped startup founded by 3 veteran consultants. Consultancies
always compete for gigs; that's why we have to write proposals. This isn't a
consultancy, but we've learned our lessons. We sincerely do not give a shit
about what other incumbent recruiting companies do. Our approach will work or
it won't, and if it works, we're going to grind on it and continuously improve
on it, and if other people do the same thing, eventually we'll start having
meetups and sharing beers with them.

------
radmuzom
Interesting sentence in the article - "Trading systems touch low-level coding,
networking (all seven layers of hello OSI Model), APIs, end-user UI,
databases, embedded systems, enterprise web frameworks, cutting-edge
programming language research, Big Data, state management, etc etc.".

However, whenever the topics of programming in finance come up in HN, I see a
general tone of disdain somewhere along the lines of "why do you want to waste
your life in finance when you could be starting a start-up in Silicon
Valley?". Of course I am generalizing a bit here, but I have always felt that
finance provides incredible challenge to technical people - especially those
with a background in mathematics or physics and interested in programming.

~~~
lifeisstillgood
I have only a years perspective on the sector now, but I think finance has
started down a road all sectors must tread over the next decade or two. We
will not escape software eating our world, and while most banks have had too
much money and so not confronted many of these upcoming realities, they have
had twenty years of faster and faster software-isation- and I can reasonably
state that finance are mostly software houses that think they are banks.

If you can't code, your future there is bleak. If you can its not that much
better. And this is where we are all heading.

Yes there are challenges - but it's not often technical but organisational,
project, people and process challenges that are the main issues. Those
interesting challenges for good STEM people will be in transport, farming and
pharmacy too by 2020.

Wages might be lower :-)

------
bliti
@patio11:

Why should I "play" this and not blog about programming? I still have to go
through the usual technical interview when applying for a job. What's my
incentive? I'm trying to find a reason to spend time learning this, because it
doesn't translate into marketable skills. I can't put "Level 65 player in
StockFighter" (because this might turn out to be a niche product that no one
outside HN will know about), but I will definitely include "XSS research on
Java Spring".

Note: I'm not trolling or trying to put the project down. I'm interested in it
but unsure.

~~~
no_wave
You would play instead of blog if you have high inhibition... very common in
this industry

~~~
bliti
Great point! I hadn't considered that since I'm not shy at all.

------
jsingleton
This sounds really fun but that is one long post/email. Almost 8000 words!
Will read it later though.

Looks like the schedule has already slipped. The email said "2~3 weeks of
today" but the blog post says "3 weeks of today"! :P

~~~
tptacek
Imagine that! An ambitious development project with a constantly slipping
schedule! Maybe we're the Duke Nuke'Em Forever of CTFs! :)

~~~
jsingleton
I know how it is. At least you didn't compare it to Daikatana. :)

~~~
tptacek
I swear to god, and this is terrible, but the Daikatana Effect is one of the
reasons we've been so quiet about this.

Since we're going to keep rolling out new level sets continuously, we should
probably have been a lot more chatty. But I keep telling Erin and Patrick that
I want to control expectations, at least until we know exactly what we're
shipping in chapter 1.

Of course, now we know what chapter 1 is, so I've lost my excuses to keep them
from writing!

Erin & I have an emulator post ("Everything Thomas Got Wrong About Emulators
This Time That He Didn't Get Wrong In Microcorruption") brewing, too.

~~~
vezzy-fnord
I actually think this is the right way of approaching lots of projects.
There's a strong culture which encourages people to release things as early as
they have a skeleton, but I've always preferred "Be discreet until you have a
fully functional 0.1." There are certainly some initiatives that benefit from
immediate fanfare as soon as you have a web page and a 200-line shim (if even
that), but not all that many.

------
Pyxl101
> We do not intentionally expose you to the speed of light as a limiting
> factor early in Stockfighter… but we can’t remove latency as being an actual
> limitation distributed systems have to overcome.

Couldn't you remove latency as a factor, at least in a game world? To play
devil's advocate, it seems like you could build a game where time advances
according to the player's control by API.

Imagine if each API call optionally took a parameter describing by how much
time should have advanced since the last call, upon the arrival of the current
call. If the parameter is omitted, the server advances the world by the
corresponding amount of real clock time. The game simulates that time passing,
and simulates receiving the call at the time specified. Perhaps an API to stop
and start the flow of game time, or run the game for a certain time and stop.

Players can interact with the game naturally, and solve problems without
controlling time, but if they run into latency issues they can orchestrate the
advance of time on the server precisely. I understand this will not be simple
to implement, especially if the game consists of multiple communicating
agents, but it could allow you to remove the uncertainty of players
interacting with the game over a network, and more easily set up complex race
conditions or timing attacks. It will also more gracefully handle any slowness
that occurs in the game or bot simulations, since while game calls might take
longer real time to return, they will presumably not take more game time. Last
but not least, it could allow you to design test cases that are deterministic
in ways that involve time, i.e., this event occurs exactly 100ms after the
game world begins, every time. Players can exactly reproduce situations that
would otherwise occur nondeterministically, while you retain your ability to
simulate interactions that you _want_ to be nondeterministic.

(I'm just exploring the idea because it seemed interesting, not making the
case that this is especially important in the scheme of things.)

------
lifeisstillgood
Well apart from "there goes October, dang I had real work planned", I am
really welcoming this game. I would normally have jumped all over their
compiler "tree", but I have been in my first Finance contract this year and I
have not yet understood HFT.

Edit: this post got longer than planned - apologies - but this HFT / algo
thing has been bugging at me for ages and this seems a likely thread for
knowledgeable folks.

The references to Flash Boys worry me slightly - Lewis sold a false scenario
there - it's pretty clear that no fool is going to see Bob buying a million
Apple shares in London, then zip over their speed of light towers and front
run Bob in Frankfurt for another million Apple shares. Who knows if Bob wants
a million? Flash boys it seemed relied too much on a protected type of trader
(market maker) getting roasted by the emergence of real competition - the
"algo" traders.

It seems incontrovertible- equity market spreads have fallen by about 5/6ths
in the past decade (no wonder Lewis' hero trader thought someone was stealing
from him) but ... How and Why?

what algo traders actually _do_ is a bit of a mystery to me.

The basics I kind of understand :

There are correlations between shares - spot those and you can make money.
Let's say oil goes up, car manufacturers will go down by some amount. Short
them and make money. The first time people did this they probably thought they
had found a money printer. But then someone else starts playing the same
algorithm - so you have to get your orders in taste than that guy - pretty
soon you are worrying about the speed of light over glass.

But other correlations exist and get spotted and tried out and ... Well it
seems a bit of a weak business model to me to live in P of .95.

So the business model of HFT is a bit weird, and the way their actions feed
into reduced spread / liquidity is a bit unclear - but anyway, I look forward
to totally screwing up my social life in a few weeks.

Anyone who can enlighten me, please shout.

~~~
tptacek
Trust me, Patrick agrees with you about Flash Boys. If you haven't yet, read
the followup/critique book he cited, even if you haven't read Flash Boys. It's
fantastic and chock full of interesting technical details. It's way nerdier
than Lewis' book.

~~~
jndsn402
I have read Flash Boys but not the books written in response.

How do the other viewpoints explain why the IEX exchange was created and
designed specifically to stop HFT? That is the central narrative of the book,
and in Lewis's telling it makes complete sense as a way to save the common man
from the evil HFT traders. But if Lewis got it all wrong and HFT is perfectly
fine, why would a completely new exchange be set up to stop it?

~~~
tptacek
There are hundreds, probably thousands, of little exchanges nobody has ever
heard of. Why set one up to "combat HFT"? Because if you can convince
companies to trade on it based on nebulous "anti-HFT" features, you can
collect fees without having to compete with the major exchanges, who both bid
prices down and also have a 100:1 advantage in engineering and support staff.

Believe it or not, that's not even the "cynical" answer.

~~~
minimax
There are about 40 active registered ATSs for trading US equities. It's a high
number but it's not thousands.

~~~
tptacek
There are many multiples more private exchanges, trading all sorts of
electronically tradable instruments. I wasn't referring simply to stock
exchanges. I was making a point about the return on investment you can get
from developing a trading platform.

------
lewisl9029
On the topic of automated trading:

I strongly encourage anyone interested in real-world algorithmic trading to
take a look at Quantopian.

[https://www.quantopian.com/home](https://www.quantopian.com/home)

It's not HFT by any measure, but it manages to make algorithmic trading
extremely accessible by providing a platform where users can develop their
algorithms, backtest them against 13 years of market data, live trade them
with paper money, or live trade with real money by linking a brokerage
account.

They have also open-sourced the algorithmic trading engine used to run their
platform:

[https://github.com/quantopian/zipline](https://github.com/quantopian/zipline)

~~~
tptacek
Yep. There is like, zero overlap between Quantopian and what we're doing;
they're both good things to play with.

~~~
jbredeche
(I'm the CTO and cofounder of Quantopian)

I am incredibly excited to try my hand at Stockfighter. Hopefully some of the
things I've learned over the last few years at Quantopian will be useful, but
I expect I'll learn quite a few new things as well.

~~~
tptacek
but like, no pressure or anything

#dayslefttorelease

------
minimax
_My order came back saying that I had bought 0 shares. And the quote from the
exchange now read “500 shares available at $120.”_

Maybe I don't get the full picture, but it sounds like you have to poll via
REST API to query the book? Modern exchanges just give you a feed of updates
(adds, deletes, trades), so you don't run into situations like this. You can
just check the feed to see what happened.

~~~
patio11
The feed is hooked up via a websocket to the UI, but not persisted anywhere on
the client side. From my eye's perspective, the cancel never registered. If I
had had wireshark open I would have seen it. (I encourage folks who understand
why this is not desirable to build their own trading UIs using our provided
e.g. websockets.)

------
davidw
I'll admit to not finishing that - it was cool to see Patrick geeking out
about the impressive tech he's been working on, rather than marketing, though.

Good luck, guys!

------
mwcampbell
Looking forward to figuring out the REST API for the GM server, then writing a
command-line client for that and the exchange itself. Then, presumably, we can
play the trading tree of Stockfighter mostly or entirely from the terminal.

------
lmorris84
_We solved this in a roughly similar way to a real stock exchange: the order
book is persisted entirely in memory, in Go._

What happens if your process crashes? how do you rebuild the order book?

Great work btw guys, can't wait to get started!

~~~
patio11
At that moment, we can't recover from that and players connected to one of the
crashed exchanges (1 server process: N exchanges) are told to restart levels,
which is not an ideal outcome but it's essentially free. I hope to eventually
be able to do something like use our persisted-out-of-memory (e.g. to NSQ)
tapes of the orders, bring up the crashed exchanges, and fire every order back
into it in order, which (since it's a stock exchange) should result in the
same orderbook it crashed with minus whatever we lost during the crash.

After we can do that reliably, then I will strongly consider making an
advanced mode level where a crash / market reset / rollback of trades is a
planned feature of the level (they are absolutely a thing which happens in
real life) and dealing with it from the player's perspective is the main task.
(e.g. What would you do if you got a message from the stock exchange saying,
basically, "We lost data. How much? Umm, not sure, the last orderId we have is
X. Orders after X are no longer on our books but, um, I guess there exists the
possibility that they will nonetheless settle successfully if they matched
before we went down. ... Good luck to you!")

~~~
inconshreveable
I imagine you've read it, but if you haven't, you may want to read about the
LMAX architecture (from which they created the well-known Disruptor) because
they dealt with this exact same problem:
[http://martinfowler.com/articles/lmax.html](http://martinfowler.com/articles/lmax.html)

------
sireat
Two anecdotal data points on somewhat similar sites from an employed
programmer and father.

Codehunt (from Microsoft Research), very addictive yet nice concise format,
polished interface, allowing Java or C# a plus.

Minuses - Requires some unnatural code golf for maximum stars. Needs new
problems and difficulty is too variable. I got in Top 5 but realized it was a
completely worthless achievement as it just was not hard enough, somehow I
hoped for a short note from MS saying GJ here is the next challenge level.

Bonus: Did not feel bad about playing this at work on breaks.

Codecombat - program in javascript, using classical MMORPG terminology, this
should have been more addictive but something was off, maybe it was the
latency or browser glitches.

My bot using simple strategy got in top, but I realized that I would rather go
back to writing my scripts for real MMORPGs where my bots make actual money
from RMT.

I still get Codecombat monthly e-mail reminders of new features but somehow
the appeal of fake MMORPG is not there.

Also, felt very guilty about firing Codecombat up at work...

So you guys not going for the standard DnD type of setting might be a good
idea.

------
listic
There's a lot of things going on here. But what is the point of Arduino?

~~~
tptacek
AVR. Not Arduino.

Starfighter is the successor to Microcorruption, a microcontroller-based
memory corruption CTF.

We wrote Microcorruption to provide hands-on learning for memory corruption
exploits.

We used microcontrollers instead of full-featured CPUs so we could (a) lose
most of the irrelevant details of operating systems and (b) fit the whole
memories of thousands of emulated CPUs into a single server process. We also
liked the idea of leveling the playing field, so that pros with large
libraries of X86 tools didn't have too much of an advantage.

Microcorruption used MSP430. We picked MSP430 because it's the simplest, most
elegant MCU instruction set for which GCC support was trivial.

Starfighter uses AVR because we didn't want to use MSP430 twice, and AVR is
the most popular MCU instruction set among hobbyists. It turns out AVR does
other interesting things for a security CTF, because unlike MSP430, it's a
Harvard architecture.

The AVR levels are intertwined with the trading levels, so that the objectives
for the AVR levels are more interesting than those of Microcorruption. You won
Microcorruption mostly by figuring out how to exploit simple memory corruption
bugs. My hope is that Starfighter requires a bit more in-sim programming.

------
donkeyd
I've just started programming professionally. I've noticed I'm a quick learner
and I'm pretty good at problem solving. However, because I just started I lack
quite a lot of knowledge. Will I still be able to play this game, or is 5
years of full-time work on multiple languages a prerequisite?

~~~
patio11
The first few levels are specifically designed to be solvable by someone near
your skill level, and after you get through them, you might surprise yourself
on the "hard" ones.

------
hawkice
Really simple question: do the privacy settings have any impact on the
recruiting stuff? I would love for you guys to poke around my data, I trust
you guys, and would love for you to be able to get my name, but I would
otherwise prefer to be anonymous. Ideally it would be a privacy-orthogonal
checkbox, maybe?

~~~
tptacek
YES. Reminder: most people who play with Starfighter CTFs will never be
interested in a new job. We're designing this stuff for those players just as
much as job seekers; we're competitive about CTFs, and trying to make a mark.

Look at the leaderboard on Microcorruption.com and notice how few real names
are there. That's what CTFs are like. If you want to have a profile page,
we'll give you one, if you want to link up your Twitter account, you can do
that, but if you want to be secretsquirrel1235813, you can do that too. From
experience with CTFs: I think a slight majority of people will be anonymous.

There is nothing we will do with your data that won't be opt-in.

------
pja
So, do I prioritise Stockfighter or TIS-100?

Decisions, decisions :)

------
ExploitsforFun
Out of curiosity, have you gotten any press inquires about your launch from
major tech publications? I know that your company is outside of the VC funding
publicity bubble but I am curious if anyone has reached out?

------
runevault
If nothing else reading this has me interested in trying Go. Kind of wish the
new book by Kernighan was out before November, as that book feels like a great
excuse t do a deep dive on the language.

------
galois198
Looks great - I get that this was built using Rails/Go, however what
language(s) will players be coding in?

~~~
patio11
For trading? Anything which can drive a REST API. For the compiler/emulator
levels, a subset of C and assembly.

------
FLUX-YOU
Do you guys model regulation through world simulation (e.g. the twitter feeds)
and is there a Federal Reserve/Mint with related data?

