
Building a Bank with Kubernetes [slides] - renaudg
https://community.monzo.com/t/building-a-bank-with-kubernetes-presentation-tv/6434
======
obeattie
Author of the talk here. Happy to answer any questions anyone has. This post
also contains more info on how we build our systems:
[https://monzo.com/blog/2016/09/19/building-a-modern-bank-
bac...](https://monzo.com/blog/2016/09/19/building-a-modern-bank-backend/)

And as an aside, I'll also be giving a longer talk at Kubecon going into more
detail on some of this stuff :-)

~~~
KirinDave
Not that I don't appreciate the subject matter, I wave the flag for K8S all
the time and I've shipped bank product presentation stuff on it... But...

The fundamental challenges of building a bank are almost entirely orthogonal
to things like distributed system uptime and resiliency (unless, I suppose,
you could lose consistency during the types service loss Kubernetes makes easy
to ameliorate). Evidence for this abounds: nearly every major bank out there
is at least 10 years behind the tech we're discussing here. Sure, banks are
getting savvy to modern techniques for their presentation and API layers (my
employer, Capital One, for example).

But the actual challenges are data consistency and liveliness and audit-
ability are preserved. I'm really curious if you're using a novel technique to
achieve this that Docker and self-managed micro-app swarms can deliver on
better.

Because what really limits most financial institutions from embracing a lot
more modern tech is their core systems of record AND the acceptance of said
systems by their governing agencies.

So when we talk about basic datacenter ops, that's all great. But I don't
think they're the things that make K8S great for a bank. I think most people
in a position to evaluate a financial institution roadmap would be unmoved by
this deck.

Now, if you talk about other things we both know K8S is great for... Things
like discovering the genesis of a database action by preserving it throughout
the chain of responders, or being able to rapidly respond to site exploits
with rolling restarts and a built in mechanism for feature flagging, or having
a really great way to offer data scientists the environments they need without
risk for data theft, or being able to use traditional CI/CD methodologies but
end up with a single deployable unit that is amenable to both automated an
manual review and mechanical deployment in spite of the tooling used within.

Not that I think selling K8S is your job. But I thought I'd mention the
perspective of someone doing inf modernization and product work at a major
bank.

And of course, as always: the opinions above are my own and not those of my
employer or co-workers.

~~~
obeattie
>what really limits most financial institutions from embracing a lot more
modern tech is their core systems of record AND the acceptance of said systems
by their governing agencies.

I just want to say that I think there is a huge amount of FUD about how you
can and cannot build your technology as a regulated entity – and in particular
as a bank. In reality, close to 100% of requirements from a regulator will
tell you _what_ you must build, not _how_ you must build it. Even then,
especially in terms of resilience and security, they are almost always a
subset of our own requirements.

What can be more of a challenge is convincing an auditor that what you have
done is acceptable, since it can be so different from what they may have seen
before. Again, I don't think this is a reason to compromise. We see technology
as a major competitive advantage, so it is worth the effort to find open-
minded auditors, and spend time to explain and demonstrate how (and why) our
software meets the requirements.

I don't think there's any way we could build a secure, resilient bank with the
kind of product experience we want, AND do it on the budget of a startup if we
approach technology through the same lens as existing banks.

~~~
tluyben2
> I just want to say that I think there is a huge amount of FUD about how you
> can and cannot build your technology as a regulated entity – and in
> particular as a bank. In reality, close to 100% of requirements from a
> regulator will tell you _what_ you must build, not _how_ you must build it.
> Even then, especially in terms of resilience and security, they are almost
> always a subset of our own requirements.

Having worked with banks and insurers solely in Java and connecting to legacy
(mostly Cobol) before, I was surprised, in my current position, to see some
companies doing their complete banking back end in PHP & MySQL. I knew the how
is not part of the regulations, but I did expect the CTOs to pick the 'no one
ever got fired for choosing' choice.

~~~
webscaleizfun
It seems common knowledge these days among the slightly but not too
technically inclined that any new major project should use a LAMP stack as its
base.

People see Facebook, Amazon, and many others running PHP & MySQL on Linux at
scale and they know it works reliably, so while it may not have the support of
Cisco or Oracle, it is pretty close on the 'no one ever got fired for
choosing' X scale, since you can point to every other major company using
these building blocks reliably if your investors, CEO, board or auditors asked
why you chose to use PHP & MySQL.

In summary, PHP & MySQL have become the modern equivalent of a "safe" choice
for your stack to be built on. Its not necessarily a bad choice either, you
get access to a large community of skilled people who can write PHP & write
SQL statements, and while everyone likes to hate on PHP, it isn't about to up
and disappear any time in the next decade either (unlike COBOL).

~~~
lgieron
I'd say the reliability the bank is looking for is much higher than what's
acceptable for web companies (i.e. web companies are fine with eventual
consistency, which would is obviously unacceptable in a banking system outside
of trivial non-core features).

~~~
jerf
I'd also suggest that it's not just scale; the kind of reliability Facebook
needs is fundamentally different than what a bank needs. Broadly speaking,
Facebook needs the site to keep working as well as possible even if some
subservice fails, and a bank needs a subservice not to fail. I'm summarizing
here and I know it; clearly neither of them is actually on the absolute
extreme end, as Facebook needs authentication to work and a bank may not care
if the interest rate display widget on their customer banking app fails to
load a couple of times. But I'd still suggest there's enough difference
between the requirements to be a fundamentally different domain.

Even in "the cloud" things differ between services. A social media app has
very different reliability requirements than a backup cloud.

~~~
hueving
Well actually there are many sub services in a bank that can go down without
major impacts. The two major banks I use have weekly planned outages of
features like old statement retrieval, person to person payments, ACH
transfers, etc. Basically everything in the web interface could experience
outages without any major crisis.

As long as ATM requests always work, nobody really seems to care.

------
coleca
Here is the video from the presentation. Love to see real world use cases for
K8S.

[https://skillsmatter.com/skillscasts/9146-building-a-
microse...](https://skillsmatter.com/skillscasts/9146-building-a-
microservices-with-kubernetes)

------
akramhussein
As an aside, as a Monzo beta tester I can't say enough good things. Currently
travelling around Asia and the card works almost everywhere and I pay 0 fees
and get the exact Master Card rate. Twice now I've landed with 0 cash in the
airport and been fine. I've used the card in over 10 countries outside UK with
<1% issues, and often this is better than my Barclays or Amex.

This is just a small benefit but when you put the whole product/experience
together from the app + in-app customer service + the card + etc, it just
works so well and really comes into a category of it's own. A lot of people
say "yeah buy my bank does X too" and while true, they way I look at this is
Monzo is like the iPod - other MP3 players had same functionality but this one
just works and works damn well.

~~~
ed_blackburn
Simply using it as a bog standard card in the UK it is head and shoulders
above other high street banks.

Kudos for having the brass balls to actually execute a conversation that has
probably happened a million times in every City pub.

------
avitzurel
It can be called: "How we built 'x' with Kubernetes".

Really the only thing that is specific to a bank (as I see it) is that they
use separate linkerd in order to do the secure stuff. Which is essentially
what banks have been doing for ages.

I commented before on how Kube has just taken over and beat mesos/marathon
stack. This talk is an example to that. You can see how many people jumped on
the Kube stack and running successful deployments on it.

~~~
agibsonccc
Disclosure: I mainly use DC/OS mesos myself. I've evaluated k8s for our use
case and didn't find it was quite what we were looking for. Our customers and
stack are mainly JVM based. We do on prem deployments not cloud where GCE is
already doing pretty well. We also mainly work with the microsoft side of
things (azure,enterprise stuff)

Not convinced of this. Direct mesos and yarn integration with spark (not to
mention a lot of the software already being built on top) is going to keep
mesos/marathon relevant for a long time. It's definitely better for big data
workloads.

I think a lot of startups will jump to this for sure. Many startups don't
actually have big data stacks and prefer to use go based stuff (mainly because
it's simpler). In that case k8s makes sense for that.

A lot of these companies will likely prefer DC/OS and mesos/marathon because
they have in house zookeeper expertise already. ZK is a dependency for much of
the big data ecosystem as well a kafka and mesos.

The synergy is a lot better.

That being said: Many here will disagree. I definitely think k8s is winning
developer mindshare overall, but I don't think it will have 100% of the
market.

~~~
alexandre_m
What about running K8s on top of DC/OS?

I find the idea of using Mesos as the resources scheduler much more
interesting, especially for multi-tenancy where each tenant launch their own
k8s cluster on shared infrastructure.

~~~
agibsonccc
Yeah that is a great poin and something I want to look into if k8s is
supported by our customers. Hence why I said I would be interested in seeing
it evolve.

------
kokey
Well, it's not a bank, yet. Their banking license restrictions could be lifted
next year. That said if they are successful it will probably go a long way
towards disrupting the customer experience of retail banking. The banks will
be able to compete with this, but having something like this to model their
improved experience on is good for them and good for customers in general. I
am, however, a bit cynical about this. I think if Monzo takes off they will
hit a hurdle or event along the way that can sink the company. Most fintech
companies are overly vulnerable to making the same mistakes as most banks made
many decades ago. In the end it will probably mean they will get salvaged by
being bought by a bank. Then the customer experience won't improve so fast any
more, while the back end and processes are being made robust to allow it to
remain in business long term. The culture in the company will also change to
be somewhat more, uhm, traditional and boring. The result will still be that
the customer experience has been pushed forward in general and that is a good
thing.

~~~
elcct
I was trying to find an analogy, and I think this company is doing something
like making TV guide more accessible and not having in mind that in a few
years TV will be obsolete. Probably this will be a good business for the next
decade, but it is not doing anything really disruptive.

------
hanief
But will it run FORTRAN or COBOL? ;)

~~~
nickpsecurity
And does it support decimal math or will it have floating point errors
instead? ;)

------
redwood
Anyone here using k8s in prod? If so, what's your use case? What parts of the
stack run inside k8s? Even state? What kind of org do you work for? Mission
critical?

------
trastentrasten
What you using for persistence of user data?

------
isostatic
Do you completely rely on AWS? I.e. if amazon goes bust your company also
dies? Or are you just using AWS as a provider of VMs, and could move to
rackspace or linode.

~~~
mandudebruh
Kubernetes has tutorials for various other cloud providers
[http://kubernetes.io/docs/getting-started-guides#turn-key-
cl...](http://kubernetes.io/docs/getting-started-guides#turn-key-cloud-
solutions)

~~~
elcct
some of those documents are quite outdated

------
0xmohit
Is there a video available elsewhere? Last I checked, it wasn't possible to
download those from SkillsMatter.

~~~
harshreality

        % youtube-dl 'https://skillsmatter.com/skillscasts/9146-building-a-microservices-with-kubernetes'    
        [generic] 9146-building-a-microservices-with-kubernetes: Requesting header                         
        WARNING: Falling back on generic information extractor.                                            
        [generic] 9146-building-a-microservices-with-kubernetes: Downloading webpage                       
        [generic] 9146-building-a-microservices-with-kubernetes: Extracting information                    
        [vimeo] 188042022: Downloading webpage                                                             
        [vimeo] 188042022: Extracting information                                                          
        [vimeo] 188042022: Downloading JSON metadata                                                       
        [vimeo] 188042022: Downloading m3u8 information                                                    
        [download] Destination: Meetups_Oct19_19-43-44-188042022.mp4                                       
        [download]   7.9% of 152.75MiB at 82.49KiB/s ETA 29:05

~~~
runeks
Cool! Although, seems like "youtube-dl" may not be the right name for that app
anymore.

~~~
timothyb89
youtube-dl is still plenty active, its last commit was just a few hours ago:
[https://github.com/rg3/youtube-dl/](https://github.com/rg3/youtube-dl/)

~~~
karakal
How come this project doesn't get a takedown request?

~~~
timothyb89
No idea, but I'm not going to complain. It's incredibly useful.

