

How NYC startup Tapad uses Scala and NoSQL to process 25k requests/sec (for now) - petesoder
http://g33ktalk.com/post/10782213265/scala-how-tapad-uses-it-why-they-chose-it-dag

======
sreque
The fact that they use Spring to wire their application together is telling
that Scala in its current state isn't meeting everyone's wants or needs at the
architectural level. If you follow the Scala mailing lists you'll find a lot
of anti-Spring sentiment, and I totally agree with them. I've hated Spring
since I first saw it years ago. Why use a statically typed language if you're
going to throw away type safety for the important task of wiring your
application together? Why use a modern and powerful language if you're going
to, again for important parts of your application, rely on a horribly slow and
gimped XML DSL?

One of the reasons I think people still use Spring is that XML configuration
files don't require recompilation. I think that to address the Scala
developers or community need to make it incredibly easy for you to run Scala
scripts as part of your application. This will then allow people to use Scala
like a dynamic language and allow them to deploy Scala configuration without
needing to recompile.

Also, I think work needs to be done in the community to document accepted
design patterns one can employ to wire together your application in Scala in a
way that solves all of the use cases Spring configuration is currently used
for. This, combined with making it easier to run dynamic Scala, and people
will never feel the need to turn to Spring again and the world will be a
better place.

~~~
daggerrz
You make some very good points here. Although we started from scratch with
Scala, we decided to use Spring for several reasons. I mention one in the talk
- we initially adopted Scala for the language and syntax itself, and it made
sense at the time to mitigate some technical risks by sticking to a well-known
and familiar architecture / wiring / configuration model.

As we have become more familiar with the language, FP and the Scala idioms, we
have definitely become less comfortable with our Spring "legacy". Initially,
we felt that @BeanProperty annotations and setter injection saved us some
boiler plate, but now most will shudder with having uninitialized vars.
Constructor injection is somewhat better, but the XML DSL of Spring is still
ridiculously verbose. Luckily, this doesn't bother us much in the day-to-day
work, we spend most of our writing idiomatic Scala.

As our platform and the entire Scala ecosystem and libraries have evolved, the
Spring and JEE "legacy" has become much less prevalent in our system than it
was originally. We use a pimped version of Spring JDBC, Spring contexts for
wiring, a Servlet container, but that's about it. In fact, we could probably
easily rewrite the whole Spring config in Scala and drop most of the Spring
deps.

I couldn't agree more with you in that the us in the Scala community really
should come up with an accepted way of handling configuration and wiring.
Having worked with more than a few Scala libraries, I can definitely attest
that the plethora of configuration styles used can be a pain to manage (and
dependencies -> configgy-nologgy, anyone?).

We regularly talk about this topic in the NYC Scala community, but we haven't
come up anything good yet. Personally, I strongly believe that dependency
injection is the way to go and that singletons and factories (which are fairly
common in Scala libs) are evil. Most long-time Java devs will agree that we've
been down that road before (gotta love JNDI)... That being said, we did not
adopt Scala to fight for status quo, so we'll keep an open mind.

Dag

~~~
dmk23
Have you looked at graph databases and how would you compare them to your with
Citrusleaf document store?

A similar-sized ad network, CPX Interactive is running ad requests on
InfiniGraph:
[http://www.youtube.com/watch?v=88HCKQ73nFU&feature=relat...](http://www.youtube.com/watch?v=88HCKQ73nFU&feature=related)

Seems like graphing engine would give you a lot of native search / lookup
functionality for free.

~~~
daggerrz
We don't need to do any advanced querying or search during the real-time
bidding request process - it's just raw lookup based on "cookie ids" (we're
not always using cookies). Simplicity of operations, horizontal scaleability
and consistent, predictable performance are the key characteristics required.

Other parts of the system have very different requirements and yes, we are
considering graph oriented databases for our cross-device analytics product.

------
noelwelsh
Great stuff there, but you didn't ask about the most interesting stuff: how
they determine what to bid on an ad! Of course, I'm biased as a machine
learning guy :)

Having the transcript is a really nice touch. It's much faster to read than
listen. I hope you can keep that up, though I realise transcription takes a
lot of time.

~~~
daggerrz
Maybe you should come talk to us? :)

~~~
noelwelsh
I've exchanged a few emails with Toby about this topic. He knows what we do,
but he's very cagey about revealing details of what you do. :)

------
peteforde
Great content, and I love the length — I get overwhelmed when I see a half
hour or hour long tech video. I literally have to schedule time for those... 7
minutes is the perfect depth into most topics.

------
kanwisher
Scala scene is heating up in NYC, Tumblr seems to be the biggest in the area,
some of the other startups like FourSquare and ideeli are using it also. Word
is some of bloomberg.com is on it also

~~~
clojurerocks
Tumblr is using scala? Any idea for what? I know ideeli was using jruby. Did
they start using scala as well?

~~~
kanwisher
Tumblr is doing their new backend services in scala, they are starting to ramp
up hiring, they want to do a lot of stuff like twitter so your seeing them do
a lot of hbase also. Ideeli told me they were considering using it for backend
services. Obviously I was interviewing lately so I got to hear all the recent
gossip ;)

------
samstokes
_insert boilerplate request for skimmable, searchable transcript or slides to
accompany video_

~~~
petesoder
Sure - in case it's more convenient, you can get a transcript of the entire
interview here: [http://g33ktalk.com/post/10782097232/dag-tapad-interview-
tra...](http://g33ktalk.com/post/10782097232/dag-tapad-interview-transcript)

~~~
samstokes
Awesome, thank you!

Reading it again, my comment while intended to be flippant just came out as
churlish - apologies for that, and thanks for sharing this information!

------
nyorker2011
Good content, well edited.

------
nathanjones
7 minutes is the perfect length. Even though I have no idea what the fuck
these nerds are talking about.

