
Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API - tmm_
https://cloudrail.com/enterprise-cloud-storage-interface/
======
jameshart
Do all these services provide equivalent or abstractable guarantees? Amazon
S3, for example, provides 'read-after-write' consistency, meaning once you've
received a positive response to a put operation, you can expect to be
immediately able to retrieve that object. But it used to be 'eventually
consistent', meaning it was possible to receive a positive response to a put,
but then not be able to read the object immediately after.

Similar guarantees are needed around how soon after deleting something can
readers expect to get 404s.

If these guarantees differ, you might find abstracting over the stores doesn't
work the way you'd like...

~~~
gaul
You can compare these guarantees and results from testing:

[https://github.com/andrewgaul/are-we-consistent-
yet](https://github.com/andrewgaul/are-we-consistent-yet)

~~~
ejdyksen
That is pretty old, considering what S3 has changed in the past two years. In
particular, it talks about reading S3's us-standard region from the East and
West coasts, and that has evolved to where us-standard (which is really us-
east-1) is now the same as all other regions when it comes to consistency
(presumably because all requests physically go to the East coast now, but I
haven't tested that):

[http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction....](http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel)

~~~
gaul
Fair point, I will update the AWS us-standard results and include Backblaze as
well.

~~~
finnh
Per that AWS page, you might want to add an asterisk that you lose read-after-
write if you make a GET or HEAD to the keyname before your first PUT. So
checking if the object exists will punk you down to eventual consistency.

------
0xmohit
Open source alternative: Rclone -- [http://rclone.org/](http://rclone.org/)

~~~
brandur
I switched to Rclone recently for personal backups and it's unbelievable how
slick, stable, and well-behaved this program is.

As far as I can tell though, it does not expose an API beyond its command line
interface though.

~~~
saganus
Wish I knew about this a week ago..

I'm using CrashPlan to backup from my laptop to another computer on the
network. I see that rclone can backup to a local filesystem. Do you happen to
know if I can I do with rclone what I do with CrashPlan? i.e. use rclone to
backup my local filesystem to a remote server?

It looks very sleek though, I'll have to give it a try.

------
tmm_
This is our brand new unified API for enterprise cloud storage providers. It's
part of the CloudRail API Integration Solution which consists of multiple
unified APIs for different categories like social, payment, consumer cloud
storage etc. Our value props are: A single API for multiple providers & No API
changes since we keep the integrations up-to-date. All that without a hosted
middleware. So we never touch the data. Looking forward to hear your feedback.

~~~
reilly3000
Looks like a handy set of tools. Check the spelling of 'Download' on the
linked page :)

~~~
tmm_
Thanks. I'll correct that typo asap but currently I don't want to clear the
cache :)

------
gaul
S3Proxy offers similar multi-protocol access at the S3 API layer:

[https://github.com/andrewgaul/s3proxy](https://github.com/andrewgaul/s3proxy)

It has compatibility with local filesystem, Backblaze B2, Google Cloud
Storage, Microsoft Azure, and OpenStack Swift. Note that this is software and
not a service.

[S3Proxy author plug]

~~~
bedros
is this compatible with django-storages (which supports S3)

~~~
gaul
S3Proxy uses Apache jclouds underneath which has broad compatibility with
object stores including S3 clones. However no two S3 implementations are alike
so you will need to test.

------
notmyname
The Rackspace API is OpenStack Swift, a project used by many more public and
private cloud providers than just Rackspace.

I've been working on Swift for the past 6 years, and I'd be happy to help with
your OpenStack Swift API integration. See my profile for contact info.

~~~
tmm_
Sounds amazing. I'll make sure to reach out to you tomorrow.

------
eriksssss
Also check out libcloud by Apache. Pretty solid Python API wrapping various
cloud services. :)

~~~
tmm_
Definitely a good alternative until we have a Python SDK :)

------
frugalmail
No monthly fees, support open source:
[http://jclouds.apache.org/](http://jclouds.apache.org/)

Wouldn't surprise me if that's what they're using behind the SaaS

~~~
tmm_
Nope, we don't use it but it is a very cool project. We have SDKs for Android,
Objective-C, Swift, Java and Node.js so far. That's why we couldn't us it
anyways. We don't want to compete with open source projects like this. Our
goal is to offer easy integrations for all platforms and all major use cases
(not only cloud storage). Btw, to support open source, we made our solution
free for non commercial projects.

~~~
Senji
Is anyone going to do one of these in C#?

------
kierenj
It's a nice feature-set, looks well-designed, and handy - but anyone using one
of these APIs would find it super trivial to implement this themselves? I'm
not sure I see the cost-benefit making sense here? I'm keen to be convinced,
though - what's the killer use case? (I'm not aware of APIs changing or being
deprecated in any big way)

Edit: ah, the other APIs make way more sense to me. You can offer multiple
options to a customer. But this doesn't seem to have an equivalent?

~~~
tmm_
This allows you to easily integrate multiple of these providers if necessary
or easily switch. Of course you can integrate them one by one on your own but
that takes a lot of time time. With CloudRail it's one API which works even
cross platform. Our ultimate goal is to handle all your integrations and not
only cloud storage. So unified APIs for fast API integrations and API Change
Management to keep your integrations running forever.

~~~
okreallywtf
Can you provide a use-case for when you would need to be able to easily switch
(or potentially hot-swap) cloud storage? It sounds like some of the
justification for ORM's; while it would be nice to have if it took no more
time to integrate with that API its hard for me to see how it could justify
extra effort in implementation/maintenance unless you have a concrete
understanding of when you need it.

~~~
blantonl
Pricing.

------
45h34jh53k4j
Its nice but what incentive does cloudrail have to provide perpetual gratis
access to someone elses API? Unlike an app that can still work when the
original developer goes broke, an API requires that always on access.

Does my whole build break because cloudrail can't handle the traffic anymore?
At least with AWS or GCS or Azure, etc, one has a business relationship with
the API provider and so can keep the lights on.

~~~
tmm_
CloudRail doesn't touch the data. Everything flows P2P between the app and the
provider. It's like if you integrate the native AWS SDK. So even if CloudRail
goes completely down, your integrations will keep running.

------
mboehm
Happy to see you guys having success! I've done my master at the University of
Mannheim. Never thought that this would happen, but I kind of miss the city.

~~~
cloudrail
Thanks. Viele Grüße aus Mannheim!

~~~
mboehm
Viele Grüße aus Passau! :)

------
atmos
Stuff like this isn't exactly new. You get lowest common denominator
functionality.

Simon Wardley summed this stuff up pretty well about 6 years ago,
[http://blog.gardeviance.org/2010/08/islands-in-
sky.html](http://blog.gardeviance.org/2010/08/islands-in-sky.html)

------
doczoidberg
please add "storage APIs" to the headline. without it, it is misleading

~~~
tmm_
I know, but it's the limitation on characters for headlines on HN. Isn't it
clear with S3, which is storage only?

------
going_to_800
I was looking for the same thing but for email providers. I would like to let
users to subscribe to mailchimp,aweber,getresponse etc using a single
integrations instead of having to integrate each one of them.

~~~
tmm_
We have a very simple interface for email here:
[https://cloudrail.com/unified-email-api/](https://cloudrail.com/unified-
email-api/) But it just offers sending emails. What you described is actually
a potential candidate for a next interface. Would love to discuss that use
case with you. Feel free to reach out: support@cloudrail.com

------
crb
No Google Cloud Storage? It even has an S3 API you could use...

~~~
johansch
Yeah, seriously speaking. Azure, Rackspace and Backblaze before Google? lol.

------
jsiepkes
Isn't this what Hashicorp's Terraform also provides?

~~~
movedx
Not quite.

Terraform abstracts the APIs of multiple cloud providers into HCL, HashiCorp
Configuration Langauge. What Terraform is essentially doing is enabling you to
skip the part where you write code to talk to AWS' API, allowing you to spin
an EC2 instance, or talk to GCE and store something in GCS, and instead just
work in one language, via one tool. If the underlaying APIs to those providers
shift and change over time, HashiCorp updates the logic under the hood within
Terraform, and your code continues working. What it does not do is provide a
single resource type or function/method that you can use to upload objects to
both AWS and GCE - you have to write two separate resources to work with each
of the two example cloud providers.

What CloudRail is saying is: the upload() function will work whether you tell
it to push the object to S3 or GCS.

What Terraform forces you to do is change `resource "aws_s3_bucket_object"
"picture" {}` to `resource "google_storage_bucket_object" "picture" {}` when
you want to change from uploading to GCE from AWS.

Make sense? :)

------
sigden
Is this not simply a library that unifies multiple services into a consistent
abstraction?

------
ninju
Your Android example has _strange_ quotes on the .createBucket(...) call --
line 9

------
pfista
There's a typo: "This means for example the function donwload()"

~~~
tmm_
Thanks! Will fix that soon.

