Hacker News new | past | comments | ask | show | jobs | submit login

Since I know there will be Cloudflare people reading this (hi!), I'm begging you: please wrestle control of the blob storage API standard from AWS.

AWS has zero interest in S3’s API being a universal standard for blob storage and you can tell from its design. What happens in practice is that everybody (including R2) implements some subset of the S3 API, so everyone ends up with a jagged API surface where developers can use a standard API library but then have to refer to docs of each S3-compatible vendor to see figure out whether the subset of the S3 API you need will be compatible with different vendors.

This makes it harder than it needs to be to make vendor-agnostic open source projects that are backed by blob storage, which would otherwise be an excellent lowest-common-denominator storage option.

Blob storage is the most underused cloud tech IMHO largely because of the lack of a standard blob storage API. Cloudflare is in the rare position where you have a fantastic S3 alternative that people love, and you would be doing the industry a huge service by standardizing the API.




I think the subtle API differences reflect bigger and deeper implementation differences...

For example, "Can one append to an existing blob/resume an upload?" leads to lots of questions about data immutability, cacheability of blobs, etc.

"What happens if two things are uploaded with the same name at the same time" leads into data models, mastership/eventual consistency, etc.

Basically, these 'little' differences are in fact huge differences on the inside, and fixing them probably involves a total redesign.


This is a good point, but just a standard for the standard create/read/update (replace)/delete operations combined with some baseline guarantees (like approximately-last-write-wins eventual consistency) would probably cover a whole lot of applications that currently use S3 (which doesn't support appends anyway).

Heck, HTTP already provides verbs that would cover this, it would just require a vendor to carve out a subset of HTTP that a standard-compliant server would support, plus standardize an auth/signing mechanism.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: