
How we test at Nubank [video] - simonpure
https://youtu.be/1dW4V3stFX0
======
lovebes
Why do banking institutions or related fintech use functional languages? Is
there some more guarantee than OOP flavored languages?

~~~
Barrin92
I personally would want to use an immutable functional language wherever I can
and only not use it if I have a good reason not too. Immutability makes
reasoning about programs significantly easier, especially if they rely on
concurrency.

And for finance it particular it's a very natural fit because there's just a
lot of transformation of data and business logic.

~~~
carterehsmith
So, how does immutability matter if we deal with the database -- which is what
holds the state, and is obviously mutable.

~~~
fnordsensei
Datomic is immutable in the sense that what you had for lunch today doesn't
change what you had for lunch yesterday, where "lunch" is any arbitrary fact
stored in the database.

I.e., you can ask to look at the entire database as it was yesterday, and run
arbitrary queries against it.

You can also do speculative updates to it, in the sense of "show me the entire
database as it would be if I were to have pizza for lunch".

It models this as a strictly linear succession of assertions and retractions
of facts. Yesterday, `A` was true, today `A` is no longer true. While this new
fact is recorded, it doesn't change the fact that yesterday, `A` was true.

~~~
carterehsmith
Sounds great in theory.

What we see in reality is that append-only database is unusable without making
additional "projections" or whatever you call them, databases that are ready
to be queried/updated, with maybe specific denormalizations, indexes and so
on.

And oh, btw, those later databases are not "imutable".

~~~
fnordsensei
It’s structured so that these operations can be done pretty much
instantaneously. Schema is sort of asserted at read time, not write time.

I highly recommend the talk “Domain modeling and with Datalog”[1]. It gives an
explanation of how all this works, including indexing.

1: [https://youtu.be/oo-7mN9WXTw](https://youtu.be/oo-7mN9WXTw)

------
Scarbutt
Looks like picking Clojure and Datomic has created a great deal of technical
debt for them. They started adding Spec everywhere to specify their data
because they were having big problems scaling their wild west code base. But
now Spec is dead and there's a new Spec2.alpha version.

I don't even know how they have manage to scale datomic to that level, the
support contract we had for datomic was only really used to report bugs[0] but
they have more than 2000 datomic transactors? ouch.

[0] Yes, too much bugs and slow, but databases are hard so I guess this was
expected for a closed-source niche DB with little users.

~~~
jonahbenton
I was at the talk. From having engaged with a lot of organizations working at
scale, it was pretty clear they were near the top in terms of not being
hampered by technical debt. They have the capability to move quickly and
evolve the system in pragmatic ways to dial the knobs towards correctness or
speed or functionality or whatever. A tremendous achievement and speaks
volumes for the tooling.

There is no "Datomic scale" problem. Datomic transactors are just another
singleton service to deploy with your microservice pod. The underlying storage
can obviously be consolidated and scaled independently. I don't recall what
they're using, would guess it is pg.

~~~
Scarbutt
Multiple datomic transactors can't talk to the same database, even if they
share the same storage.

~~~
jonahbenton
Sure they can. Just give them different database names.

~~~
Scarbutt
A different name is literally a different database.

~~~
jonahbenton
A single postgres daemon- a unit of scaling- can support many datomic
transactors, each talking to a named "database" hosted by that one daemon.

You don't have to scale your postgres _daemons_ with your microservices- each
of which have their own transactor- which would be painful and out of the
ordinary for an ops team.

Scaling datomic on top of postgres is no different from scaling any other
microservice.

That all said, the architectural point that _did_ sound painful in the
presentation- and which is a common pain point in microservice architectures,
not unique to what NuBank is doing with Datomic- was having to maintain an ETL
for analytics purposes, to pull together all of the distinct microservice-
specific Datomic-hosted data sets into a single uniformly SQL-queryable data
set. The details of that implementation, and whether it made use of the new
SQL interface supported by Datomic, were not discussed. But it smelled brittle
and fragile.

~~~
Scarbutt
_A single postgres daemon- a unit of scaling- can support many datomic
transactors_

yes

 _,each talking to a named "database" hosted by that one daemon._

No. That would imply distributed writes and break the single write
serializability property of datomic. Think about it, transactors don't sync
with each other(only one for HA but that's orthogonal).

To put it simply, multiple transactors can share the same storage but only one
can write to a single database at a time.

