I’ve seen workflows with data chunked out into multiple excel files because the results exceeded the row limit (4 million?) Going through all that by hand was a pain, and it took hours to go from a flat file format to spreadsheets. Parquet took this down to seconds, and provided a rich data analysis API.
Spreadsheets scale for small to medium-ish workflows and are very much a drag for anything remotely “big.”
I’ve seen workflows with data chunked out into multiple excel files because the results exceeded the row limit (4 million?) Going through all that by hand was a pain, and it took hours to go from a flat file format to spreadsheets. Parquet took this down to seconds, and provided a rich data analysis API.
Spreadsheets scale for small to medium-ish workflows and are very much a drag for anything remotely “big.”