On: 650GB of Data (Delta Lake on S3). Polars vs. DuckDB vs. Daft vs. Spark

[saxenaabhi]

Then concurrency suffers since you have to have locks when you update files.

That's also why ducklake performs better than others.

For many use cases this trade-off is worth it.