
Scaling Financial Reporting at Airbnb - knighthacker
https://medium.com/airbnb-engineering/tracking-the-money-scaling-financial-reporting-at-airbnb-6d742b80f040#.alfxjkhrn
======
sachinag
If this is generalizable, and it looks like it is, it would be pretty smart to
spin this out as a new company, make Airbnb the company's first customer, and
go after Oracle's financial products.

(Disclosure: I used to work at Oracle on Financials Cloud.)

------
elbasti
There seems to be an accounting equivalent of Greenspun's tenth rule:

"Any sufficiently complicated financial reporting system contains an ad-hoc,
informally-specified, bug-ridden implementation of double-entry accounting."

It's such a powerful yet simple way of thinking about money that anyone who
builds a billing or accounting system should be intimately familiar with it.

~~~
hcarvalhoalves
Related, might be interesting: "Building a powerful double-entry accounting
system" ->
[https://www.youtube.com/watch?v=aw6y4r4NAlw](https://www.youtube.com/watch?v=aw6y4r4NAlw)

------
arafa
"Going from imperative programming to functional programming has been a
powerful paradigm shift for us to think about financial processing and
accounting. We can now think of this system as a straightforward actor/handler
system rather than getting mired in complicated SQL-join logic."

Though whether SQL is a functional language (or a programming language at all,
if you're talking ANSI SQL) is a subtle question, I would at the very least
not describe it as a traditional imperative programming language. I think this
is an important distinction for the article because, contrary to what this
article suggests, I've found SQL to be quite helpful for understanding
functional and declarative programming concepts. That said, it might be a lot
easier to express the types of tasks in the article as straight-forward
functions rather than getting wrapped up in all this set-based talk in SQL.

~~~
a_liang
Hi, I'm the author. Yeah, you have a good point. Imperative programming was
just the way that we were using SQL to build that system.

~~~
bigger_cheese
Did you consider an off the shelf ERP product?

I've written something similar to the first part - extract raw data out of
various source DB's using SQL queries then push it to our organisation's ERP
product (SAP) using A2A messaging.

From my view SAP is black box but it handles the actual accounting/ financial
logic part i.e Ledgers, product tracking, inventory management etc. Our
Accountants all seem pretty comfortable using it.

~~~
a_liang
Airbnb's data models weren't initially designed to be financially reported on,
and by the time we needed better financial reporting, it was too late to
change those models. 90% of the work was about rethinking the way to think
about all of this, what financial impact should be booked, and how it could be
derived from the data. None of the ERP solutions fit our use case, and I think
it would have been very difficult to integrate.

We still use a general ledger to book the outputs of our new financial
pipeline, but I don't think we have a traditional business model (no
traditional inventory management). That's more in the finance and accounting
department though, and I can't speak much to that.

------
nadocrew
This is very interesting, but I would love to know more details. Are they
using something like Amazon Kinesis or Kafka to send events and handle missed
events? What serialization format are the messages? How do they manage keeping
the schemas of the events in sync?

------
plehoux
At my company we built accounting on top of Xero.com API. Even at our small
scale we're already busting their capacity. Some reports can't even be
generated anymore because we have too much transactions.

This post is a fascinating outlook of how things can be ran at big corp. I
always wondered how big companies did accounting. This stuff can't be
outsourced since it's so tightly coupled to your business logic.

Is there any _tech_ conferences geared toward this stuff?

Any accounting saas that can scale indefinitely? As opposed to Xero?

~~~
richdougherty
> Any accounting saas that can scale indefinitely? As opposed to Xero?

Need to wait for AWS to release a version of Airbnb's system. ;)

------
timae
This is great, thanks for the writeup! We are actually building something
quite similar. We have a rough landing page up at hireross.com. Airbnb folks,
I'd love to chat if you're up for it. Just to learn more. email is at the
bottom of the landing page.

~~~
timae
changed the landing page a bit to include a signup form instead of email.
Email is ross@hireross.com.

------
philfrasty
„...On the date of the reservation confirmation, we have a guest receivable of
$100, and a future host payable of $90...“

The one thing I hate about AirBnB.... Looking to rent a nice place for 10k 9
months in advance? Ok...we'll charge it RIGHT NOW.

Why not do it like Amazon and charge when the package is shipped (trip date
has arrived)? I get the cashflow thing but come on... (100 bookings and
counting...)

~~~
rtpg
I bet if they didn't charge now, suddenly they have to deal with "card charge
failed the day before the flight and now the reservation isn't paid for and
the person is in front of the door"-style issues.

Of course AirBnB could forward the money to the host but doing it they way
they do cuts out the problem entirely.

------
dantiberian
I've previously worked with enterprise financial systems, and most of what was
described here was standard functionality, or would be customisable to do so
without too much effort. I would have loved to have heard more about whether
they evaluated traditional ERP systems and why they chose to build their own
event sourcing system instead.

~~~
ernestbro
Same story with Superset, their homegrown BI system. Why re-invent the wheel?

~~~
sheeshkebab
If it was not for "reinventing the wheel" we'd all be coding in cobol or
Fortean against IBM supplied mainframes.

In other words - because they can and should (and can afford to and not be
hooked up to some third party vendor milking them for the rest of their
companies existence).

------
caseyf7
Did you consult your auditors through this? I would expect their answer to be
something along the lines of "Spark? Scala? We don't think we can trust this
so we'll need to do a full audit." versus "Oh, you're using SAP, ok."

------
bigzen
I'm worn out on articles dissing the performance of SQL databases without
quoting any hard numbers and then proceeding to replace the systems with no
thanks of development in the latest and great tech. I have nothing against
spark, but I find it very hard to believe that alarm code is now readable than
SQL. In fact, my experience is just the opposite.

AirBnB is using an extract, load and transform architecture. No mention of the
hardware, data through put, whether they have a message broker/queue to ease
the burden of peak volume but work.

I have a strong feeling that they could have 1.) Kept the system exactly how
it is and done some performance tuning. But that's not sexxy anymore. Things
are just supposed to scale. Which brings me to

2.) Moved transformation logic to its own server or multiple servers using a
message broker and queue to aid the transfer of data between systems. It would
have been more readable and could have been done in a mo the or less.

In summary I believe they should have put some effort in to keep SQL.
Especially for the purpose of accounting because spark does not lend itself to
readable logic.

------
spIrr
Alice, do you recognise guest receivables the day a booking is made, not the
day after check-in ("services rendered")? If that's true, not sure if this is
correct, i.e., this event doesn't seem to meet the asset recognition
criteria... My intuition would tell me Dt Accounts receivable - Cr Revenue on
the day after check-in.

~~~
spIrr
On a second look, I don't seem to understand the flow of your accounting
entries:

 _Booking date_ (I understand that on that day, the booking is confirmed, but
the guest's payment method has not yet been charged)

    
    
      Dt Receivables from guests 100
    
        Cr Future host payout 90 (Problematic: how can you pre-recognise a liability on your balance sheet? Unless this is an off-balance sheet account...)
    
        Cr Deferred income 10 (Problematic: deferred revenue arises when you receive a pre-payment from customers and have a standing obligation to them to render the services)
    

_Payment made by guest_

    
    
      Dt Cash 100
    
        Cr Receivable 100
    

_Check-in date_

    
    
      Dt Deferred revenue 10
    
      Dt ??? ??? (seems to be missing to balance the double-entry)
    
        Cr Payable 90
    

To me, the natural way to do this would be:

 _Booking date_ \- no accounting entries, no effect on books. You did not
render a service to the customer yet, nor fulfil or incur any obligations yet.

 _Payment received from guest_ \- you received a prepayment for future
services to be rendered, so now have a liability to the customer to fulfil
this obligation, i.e., deferred revenue.

    
    
      Dr Cash 100
    
        Cr Deferred revenue 10
    
        Cr Payables to host 90 (unsure about this one, as this goes into the whole gross vs. net revenue recognition discussion for marketplace-type businesses)
    

_Day after check-in_ \- recognising the revenue on one single day might work
now, but it's only a makeshift solution - what if your customer stays for a
longer period of time, or between two accounting periods, and this becomes
material? By definition, you recognise revenues proportionally, but I get that
the cost vs. benefit of doing this now might be unfavourable.

    
    
      Dr Deferred revenue 10
    
        Cr Revenue 10

------
jaysoncena
This new system that uses events looks more flexible since it is decoupled on
the application logic. I think the downside with this one is that the new
system has a lot of moving parts. Also, changes in logic/new events must be
communicated and should be supported before the main app is put into
production.

------
d--b
Is it really that big that we need to talk about "scaling" it? 4-5 hours of
processing time is a lot... I mean how many transactions would they do on a
normal day?

~~~
dangoldin
Yea - it does seem a bit high. We use Spark for our adtech data pipeline and
we're handling tens of billions of events a day in less time. It may be a
function of how much data they're pulling in from other systems or dumping the
data back into a variety of systems. Spark itself is parallelizable so in
theory can be sped up just by running more nodes.

~~~
sheeshkebab
financial processing is typically sequential - can't calculate some metric
until some other thing was calculated (or pulled data for)... not well
parallelizable in other words. or so it is with some systems I deal with.

------
ransom1538
tldr; They built an event/messaging system. When an event occurs they
broadcast that event with meta data. (EG. event: reservation_booked, meta
{..}). Before they tightly coupled reporting sql inside the business logic.

