Hacker News new | comments | show | ask | jobs | submit login

I'm worn out on articles dissing the performance of SQL databases without quoting any hard numbers and then proceeding to replace the systems with no thanks of development in the latest and great tech. I have nothing against spark, but I find it very hard to believe that alarm code is now readable than SQL. In fact, my experience is just the opposite.

AirBnB is using an extract, load and transform architecture. No mention of the hardware, data through put, whether they have a message broker/queue to ease the burden of peak volume but work.

I have a strong feeling that they could have 1.) Kept the system exactly how it is and done some performance tuning. But that's not sexxy anymore. Things are just supposed to scale. Which brings me to

2.) Moved transformation logic to its own server or multiple servers using a message broker and queue to aid the transfer of data between systems. It would have been more readable and could have been done in a mo the or less.

In summary I believe they should have put some effort in to keep SQL. Especially for the purpose of accounting because spark does not lend itself to readable logic.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact