On: Turbopuffer: Fast search on object storage

[rohitnair]

Elasticsearch and OpenSearch already support S3 backed indices. See features like https://opensearch.org/docs/latest/tuning-your-cluster/avail... The files in S3 are plain old Lucene segment files (just wrapped in OpenSearch snapshots which provide a way to track metadata around those files).

[francoismassot → rohitnair]

But you don’t have fast search on those files stored on object storage.

[rohitnair → francoismassot]

Yes, there is a cold start penalty but once the data is cached, it is equivalent to disk backed indices. There is also active work being done to improve the performance, example https://github.com/opensearch-project/OpenSearch/issues/1380...