Hacker News new | comments | ask | show | jobs | submit login

what is the data format FB stores? (xml/json/?)



Most of the data in our data warehouse is stored as relational records using Hive's RCFile format, though we are actively working on converting it to the new ORCFile format.

The Hive data model supports complex types including arrays, maps and structs (nested tables), and those are used liberally. We also have a fair amount of data stored as JSON inside of string columns in Hive tables, though much of this is actually structured and could be better modeled using Hive's complex types.

Presto currently has limited support for Hive complex types. All complex types are converted to JSON at query time and can be accessed using the JSON functions: http://prestodb.io/docs/current/functions/json.html




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: