
Amazon Cognito - moonlighter
https://aws.amazon.com/cognito/
======
amirmc
The interesting thing about all these services is that they're trying to
abstract away the same types of problems that all developers come across:
Identity, connectivity, sync/backup (and to some extent, deployment).

A different (and more disruptive?) approach to this would be to put more
control of these things into the hands of end-users such that they provide the
'backend' into which you (the developer) load your application. i.e something
like the app-store model but it connects to the user's 'personal cloud' (or
the desktops of old - if you prefer). Such a system needs to be FOSS at its
core but with a way for developers to get paid for providing value to end-
users (who themselves get control of their data/networks).

Of course, this won't happen overnight, but the alternative is that _everyone_
ends up using proprietary silos, with huge lock-in and innovation suffers as
the tech giants get distracted with lawsuits.

I'm working with others on the distributed systems infrastructure we need to
make this possible [1], so that we can get to a place where everyone can have
their own piece of the cloud. In fact, one of the major components of this is
Mirage, which has been discussed quite a bit on HN recently [2].

[1] [http://nymote.org/blog/2013/introducing-
nymote/](http://nymote.org/blog/2013/introducing-nymote/)

[2]
[https://news.ycombinator.com/item?id=8020125](https://news.ycombinator.com/item?id=8020125)
and
[https://news.ycombinator.com/item?id=8005130](https://news.ycombinator.com/item?id=8005130)
and
[https://news.ycombinator.com/item?id=7726748](https://news.ycombinator.com/item?id=7726748)

~~~
michaelmior
Do users really care about this? My guess is that 99% don't and don't want to
deal with these details. The project sounds cool and like something the HN
crowd might want, but my intuition is that most users won't understand the
implications.

It also adds an extra step in the data storage process within the app, which
increases friction and seems like it would be hard to get developers to adopt.
All that said, I'd love to be proven wrong and see an implementation that is
killer enough to solve these problems.

~~~
amirmc
Yes, users care about this but like most things you have to segment the market
appropriately. I meet more and more people who are wary of putting all their
trust/data/etc into only _one_ provider, be that Google, Apple or anyone else.
It's not just a privacy thing but a growing awareness of the sheer
_dependency_ on that one company. (Aside: I have to say FB is being pretty
clever by buying up apps and letting them stay 'independent' \- users don't
notice that as much and reminds me of FMCG.).

We don't need _most_ users in the beginning. Just those that are technically
savvy enough to understand the landscape and the way the world is headed. From
there we can develop more stuff and _eventually_ get it into a state ready for
the masses. For example, I can run my personal website as a unikernel [1],
which is not suitable for mom-and-pop but great for anyone currently using any
static site generators. As more libraries get released I can build on that an
eventually run more of my infrastructure this way (my prototypical examples
are mail, contacts and calendars).

In terms of the friction for developers, we simply have to make the libraries
and tooling fun and easy to use. With well-designed systems it's not
necessarily any more friction that what developers have to do now (i.e build
their own backend).

> _" All that said, I'd love to be proven wrong and see an implementation that
> is killer enough to solve these problems."_

We're working on it.

[1] [http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-
li...](http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/)

~~~
josh2600
I thought like this once.

Futurists have a tendency to imagine a world of changed human behavior and
it's compelling to do so. The reality is that the future rarely arrives as
sweeping change, but rather as metaphor and specialization.

Whereas you can imagine others adopting new patterns of behavior because you
understand the underlying reasons why such behavior is reasonable, the
metaphor through which you explain this change is not readily understood. Why,
as a User do I want this? If the answer is control and privacy, you might be
barking up the wrong tree (time and again we've shown that those are not
things consumers want or are willing to pay for).

If you want to drive dynamic change in the world, you have to change the
underlying structure of complicated systems while steadfastly avoiding changes
in user behavior. It turns out this is quite hard.

I applaud your efforts but encourage you to avoid the rabbit hole of endless
specialization and to improve the marketing metaphor/rhetoric.

~~~
quadrangle
> time and again we've shown that those are not things consumers want or are
> willing to pay for

CITATION please. I think we have never seen such things. We've not had studies
that control for all other factors and then conclude that users don't want
control and privacy. I think your claim is flat out wrong. You're taking the
fact that _most_ users won't sacrifice by using a lower-quality, obscure thing
for privacy and control, and leaping to the baseless conclusion that nobody
cares about these things. I think you're completely wrong.

~~~
josh2600
There have not been any case studies that have empirically demonstrated this
to be true. If that is your basis for evaluating my statements then I cannot
support my claim in a manner that you would find satisfactory.

I would however, anecdotally highlight the progression of social services from
places of privacy to places of publicity. One could argue that sites like "The
Well" were the original social networks but they did not grow (whether by
choice is of course a matter for debate). Over time, each successive social
startup defaulted to a more open stance and scaled an order of magnitude in
users with each step. The trend suggests that all things being equal, users
will say they want privacy but want the features publishing and syndication
provides (voting purely with their registrations and advertising value).

I agree that privacy has value, but I would argue that the anecdotal evidence
suggests that users want less private services (and yes I'm aware that
correlation is not causation, but I think this observation is apt). This is
not law or dogma but rather a thought and I'm sorry if it was communicated
poorly previously.

~~~
gurkendoktor
> the progression of social services from places of privacy to places of
> publicity

I don't think that's a clear-cut anecdote. Facebook does actually feel very
private, because it's isolated from Google and outside users. Look at the
number of female profiles where everything but the name is hidden from non-
friends. I don't remember MySpace or blog users ever being this paranoid. And
then, anecdotally, many people around me have started to live in WhatsApp/LINE
gossip groups, which is something that could easily be decentralised.

------
dperfect
Why do so many people (and companies) pretend that app backends are just
(logic-less) data stores?

I can definitely see the argument that some mobile games only need to
store/sync some basic state information on behalf of the user, but I just
don't see it as being that simple for most other apps.

I suppose this datastore-only backend model works for apps with users who are
completely isolated from each other, but any time you want to allow users to
interact with each other (or provide rules for how their resources interact
with the world outside their little "user bubble"), you need business logic
that resides somewhere other than the app; otherwise, your business logic
(which may affect more than just the authenticated user) is in an untrusted
environment. Not only that, but there are huge scalability problems associated
with using a logic-less backend storage model if you want to do _anything_
that involves more than just a single user's data.

Even for games, something as simple as providing a leaderboard is impossible
(or extremely impractical) with this model. If you have no server-side logic
to perform aggregate operations over multiple users, the only other option
you'd have is to basically grab _all_ the data and perform those operation in
your app, which is absurd.

Backend development, server management, and deployment can be hard (though
many PaaS solutions are making it easier than ever), but personally I'd
_never_ recommend building an app with a logic-less backend data store like
this. Even if you don't think you'll need it today, in most cases you're going
to need logic on the backend eventually. If you've started with this, you're
just creating a _lot_ more work for yourself when you finally do need to
implement a feature that crosses the line from isolated-users-only to users-
that-can-interact (or even just obtaining aggregate user data for business
intelligence purposes).

~~~
akrymski
Completely agree! At post.fm we're trying to design an email client which will
work offline, and synchronize data when online (IMAP isn't sufficient for our
needs) - Exchange style. We've thought long and hard about various solutions,
and have tried syncing logic-less document-stores such as this. As you say -
this doesn't really work, and there's definitely no way to handle anything
remotely complex like transactions. In our case "app data" is effectively all
your mail and more, and we can't expect each client to have a local copy of
everything, meaning partial syncing of a collection is important for us (not
something any solution out here seems to handle).

The solution we've come up with, which we'll hopefully get to open-source one
day, is to model state over time as a persistent "event log", which means
instead of syncing documents we sync events that clients perform, replaying
events instead of document-update statements. This approach is infinitely
flexible I believe, but requires more work if your clients are written in
different languages (we use JS throughout so that isn't an issue).

~~~
Chronic28
There was no need for you to mention your company's name.

------
fragsworth
With so many new AWS features rolling out, I am worried that Amazon is
spreading themselves too thin and will soon decide to stop supporting some of
their less-popular services that I happen to be using.

As far as I can tell, there is no guarantee anywhere that they'll maintain any
service for any length of time.

If that happens, it could waste weeks or months of engineering time trying to
migrate to something else, or I'll just have to shut down my own services if I
decide it's not worth migrating.

~~~
moonlighter
It's a valid concern. AWS has been around for many years now with numerous
services. Has there been a single offering which they retired or stopped
supporting? I'm not aware of one.

Amazon doesn't just create services, throws them over the wall and see if they
stick. Rather, they almost always RESPOND to customer requests. Dr. Werner
Vogels, AWS's CTO makes a point about that approach at about any keynote he
gives. Similarly, they try to also avoid feature creep by only providing
features which customers really ask for.

~~~
riffraff
I am unaware of AWS modules discontinued by amazon, but it is also true that
the oldest services are the most popular (S3, EC2) while some of the most
recent ones have a (apparently?) smaller user base (SWF, Kinesis, AppStream).

I think I remember amazon people saying they are actually dogfooding this
stuff so it should stay available for a long time anyway, rather than being a
random Google Reader.

~~~
Twirrim
There is a strong culture of dog-fooding in Amazon. The advantages to it are
quite straight forward :) If you've got a whole bunch of DBA experts already
hired, why would you hire additional people to do DBA work for every team,
when they could just use RDS? Along with that is the huge added benefit of a
direct feedback loop, with consumers and producers able to directly meet and
deal with feature requests, problems etc. etc.

Retiring any of the services that have been launched would cause not
inconsiderable pain to other teams :)

------
yourad_io
While this (and other similar) Backend-aaS look very appealing at first
glance, it seems to suffer from the same problem as most AWS offerings -
terrible platform lock-in. Show me that I _could_ move all of this to another
provider if I need to, and then we can talk seriously.

I also wonder how they "manage[s] the complexity of conflict resolution"
without manual intervention.

~~~
pbreit
Is anyone delivering the equivalent with less lock-in? I can understand both
sides of the argument but it seems that over time stuff always migrates
towards more control (ie, people migrate OFF of Parse, Heroku, AWS, etc). I'd
envision a much better set of images, Fabric/Chef recipes, to the point where
you get your AWS but with vanilla OSS on generic hosting providers.

~~~
rbdone
A couple of open source options that you can deploy to your own servers (or
your cloud provider): \- Apache UserGrid (incubator project) \- DreamFactory
\- BaaSBox \- LoopBack \- Helios \- Deployd

Any other OSS out there for this?

~~~
Ecio78
I'm not sure it could be considered the same thing, but by visiting the
DreamFactory's site I found this in the comments
[http://www.wakanda.org/](http://www.wakanda.org/)

Any feedback on it?

------
orandolabs
The similarities between Cognito and EnduroSync (Orando Labs,
[https://orandolabs.com](https://orandolabs.com) \- announced in May) are
striking. Including the latin sounding names. Except that EnduroSync is an
object store and has no data size limitations. Even the pricing is similar.

As far as Hacker New goes, we announced EnduroSync back in May, and got no
promotion. Not one up vote?

~~~
aendruk
My impression is that the interest here is not in the product itself, but in
the insight that it lends about a known player.

------
bellerocky
I don't know how Amazon comes up with names for its products, but recently
they all strike me as weird and stupid names. Do they confer with marketing
first? Zocalo, Cognito? Really? These are the kinds of names you invent when
you're looking for a cheap domain name for your startup. For Amazon, they
don't make a lot of sense. They just seem like some executive out of their
marketing depth and out of touch trying to be clever and anyone with sense
afraid to speak up during the meeting when they come up with these names.

~~~
danneu
They're unique while remaining loosely related to the products they describe.

The
[http://en.wikipedia.org/wiki/Z%C3%B3calo](http://en.wikipedia.org/wiki/Z%C3%B3calo)
is the city center (of Mexico City's historical district), a simple and
untaken metaphor for a service that "...provides users with a central location
for both the documents and files they are...".

~~~
Chronic28
Great. You are 1 out of 100,000 people that understand the metaphor.

------
sophacles
Is there anything like this in the enterprise space? Many of our customers
want authentication against their internal Active Directory or other single
sign-on solution. It would be nice if there was a company that exported an
api, and did all the work of connecting up the various types of auth, so that
we can focus on our own product rather than redo integration work that has
probably been done by many other companies already, probably even for the
customer's other services.

~~~
idea_shot
Check out Mobile Identity Connect -> [http://www.kinvey.com/blog/3856/mobile-
identity-for-the-ente...](http://www.kinvey.com/blog/3856/mobile-identity-for-
the-enterprise-kinvey-launches-mobile-identity-connect) and
[http://devcenter.kinvey.com/ios/guides/auth-link-for-
ldap](http://devcenter.kinvey.com/ios/guides/auth-link-for-ldap)

~~~
sophacles
Thanks for the pointer. This particular service is useful to know about, but a
bit more mobile focused than is appropriate for our current product. Noted for
future reference tho!

------
yalogin
Isn't there a startup that does this already? Seems way too intuitive and
natural to not have happen already.

~~~
cobrabyte
Strongloop has [http://loopback.io](http://loopback.io) which has been
interesting to work with. Looking to free up some time to investigate a little
further!

~~~
nni
there's a lot of energy/activity with the strongloop project - I am looking at
that as well... and now Cognito will demand my attention, too :)

------
SimianLogic2
seems kind of expensive for what it does.

imagine a game with 100k DAU (big, but not huge)

2 sessions per user per day

say 10 "saves" per session

100,000 * 2 * 10 = 2 million synchs / 10k * $0.15 = $30/day

for $900/month you could do FAR better just rolling your own

these numbers are all reasonable for a mobile game... so it seems to me that
the main use pattern would be for apps that don't need to synch very often.
and if an app doesn't synch very often, why even bother throwing an auth wall
at your users?

~~~
nni
interesting. Higher than their example pricing which was based on 500k
sessions/month with a sync at begin and end
([https://aws.amazon.com/cognito/pricing/](https://aws.amazon.com/cognito/pricing/)).

I find the aws pricing (and "cloud pricing" in general) fairly confusing, and
the things is that you have to be so careful that the wrong checkbox isn't
checked or something that opens the spigot to the credit card# you had to
enter to try out the service on the free tier in the first place. Note that I
saw that you can get an alert when it first charges _anything_ via billing
alerts ([http://docs.aws.amazon.com/gettingstarted/latest/awsgsg-
intr...](http://docs.aws.amazon.com/gettingstarted/latest/awsgsg-intro/gsg-
aws-billing-alert.html)), but still...

~~~
ColinCera
I think Amazon's pricing example is based on a fairly unrealistic use case
where you only sync at the end of a session. With real world applications,
most of them sync multiple times during a session, e.g., every 5 minutes,
every 30 minutes, every time a significant atom of work is done (e.g., Gmail
saving drafts). With mobile devices it's common to turn them off abruptly, or
lose connectivity abruptly. A user's not going to enjoy the experience if
_nothing_ they did in their session on their phone is available when they fire
up their laptop because the phone app only syncs at the end of a session.

------
joeframbach
For those wary of vendor lock-in, or for those wanting a self-hosted similar
service: Just yesterday I spun up my own oauth.io oauthd server. It's working
out fairly well. It's only third-party authentication, it doesn't handle guest
users or local authentication.

------
skram
Am I missing something or could this be useful for non-mobile applications as
well like HTML/JS apps if they provided an API outside of the iOS/Android
SDKs?

~~~
athrun
You're right. And considering they have just added Cognito support to their
Node.js SDK, we can suppose this is the master plan.
[http://aws.amazon.com/releasenotes/JavaScript/99205414836272...](http://aws.amazon.com/releasenotes/JavaScript/9920541483627238)

~~~
skram
Thanks for the link!

------
diafygi
A buzzword used in Cognito and Zocalo is "secure". I'm assuming that means
that it's server-side encrypted at rest.

What would be really impressive is if there was an option to client-side
encrypt the data before sending it to AWS. Of course, that would mean you'd
have to move your syncing logic to the client-side, too, but having a good
client-side encryption option would be a real differentiator.

~~~
rakoo
Wild guess: Amazon does nothing to solve sync conflicts, because it's highly
correlated with your app business. Amazon gives you all conflicting versions
and some flags telling your app "there's a conflict, you should fix it". Much
like CouchDB.

So if it works like that, you absolutely can encrypt your payload client-side,
and use Cognito as a mere transport.

~~~
napoleond
_Much like CouchDB._

Exactly. I don't know why people are so attracted to locked in solutions like
Parse or now this when Couch has already been doing it for a long time, and
there are mature third-party providers (Cloudant, Iriscouch, etc) that can
help you get off the ground just as easily without ever being locked in with
them.

------
mikecb
Sort of like google's cloud endpoints, I suppose:
[https://developers.google.com/cloud/samples/mbs/](https://developers.google.com/cloud/samples/mbs/)
and
[https://developers.google.com/appengine/docs/java/endpoints/](https://developers.google.com/appengine/docs/java/endpoints/)

~~~
wiradikusuma
I would say it's closer to
[http://googlecloudplatform.blogspot.com/2014/06/cloud-
enabli...](http://googlecloudplatform.blogspot.com/2014/06/cloud-enabling-
your-mobile-app.html)

~~~
mikecb
Yeah, I knew there was something else too. All are little code or code-free
backend services.

------
matthewarkin
So Amazon is now competing with Facebook / Parse?

~~~
hodgesmr
And new iCloud. And new Dropbox API. And Helios.

~~~
mkr-hn
Does Apple consider iCloud a competitor to other cloud services? You need an
Apple device to sign up for it, so you can't move to it if you only have
access to Android or Windows devices.

------
jflowers45
I'm definitely finding it interesting to see Amazon releasing more and more
offerings to make life as a developer easier. Will be interesting to see
whether they can leverage some of these services to make developing for the
Fire phone extra appealing.

------
glynjackson
This may sound stupid, so I apologise in advance for my ignorance....

I watched the video and the focus of this service seems to be on storing user
data in a way that can be accessed by any device. I'm an API developer, isn't
this just what every mobile app that uses an API to store data does, right? I
personally don't write a different backend datastore for every device type
that could connect to it, nor do I 'permanently' store data on the device!
user data is always synced 'for me' over an API. I don't get this, can someone
explain why we need this service? Do people develop apps different from me?
I'm not being sarcastic, I'm genuinely interested.

~~~
Paul_Dessert
Agreed. I'm confused. I don't understand how this helps users access data on
"any" device.

The data pulled from the backed is handed off to the front end. The front end
determines what's displayed and how it's displayed based off of the device.
What does data storage have to do anything.

------
nostromo
Amazon seems to be in "me too" mode lately. They released an Android phone, a
Dropbox clone, and now a Parse clone.

Instead being the vanguard of innovation, they're letting other companies
validate the need for a service before swooping in.

Who's next? Probably Stripe.

~~~
eclipxe
[https://payments.amazon.com/home](https://payments.amazon.com/home)

------
haberman
When I see products like this, my first question is: could you write an free,
open-source app with this without having to shell out your personal money to
make it happen?

For Amazon Cognito, the answer appears to be "yes, until all of your users
together store more than 10GB". ie. until your app gets popular.

So for free open-source apps, I think I'd prefer to stick with Dropbox or
Google Drive, where the billing is associated with an account the user already
has (which most users can operate entirely within the free tier). That way you
don't run into a situation where you are a victim of your own success, by
exceeding a free tier that applies to the sum of _all_ user data in your app.

------
moonlighter
There's more information about AWS Cognito beginning at the 1:03:00 mark:
[https://www.youtube.com/watch?v=Wr6WirGn-6k](https://www.youtube.com/watch?v=Wr6WirGn-6k)

------
orandolabs
In response to this announcement, Orando Labs
([https://orandolabs.com](https://orandolabs.com)) has decided to offer
EnduroSync and Identio with open source licenses and as paid AMI instances. We
believe we have a unique solution to some difficult problems (identity and
syncing), and want to see our solutions widely adopted. Read more at
[http://orandolabs.wordpress.com/2014/07/12/amazon-
cognito](http://orandolabs.wordpress.com/2014/07/12/amazon-cognito)

------
dave1619
So, can someone correct me if I'm wrong? It appears that Amazon Cognito
handles sync for local device data, but the developer doesn't have access to
any of that data. Correct?

So this service seems to be geared toward developers who don't want to store
or see user data at all. But for a developer that wants to gather user data
(for crowdsourced insights) or for a developer that wants to offer a web app
connected to their own DB, then this isn't going to work.

------
xianshou
So, Amazon Parse? Looks like the big four (Apple, Google, Amazon, FB) all have
to _ahem_...back(end) it up now.

~~~
frik
Big five (Amazon, Apple, Facebook, Google, Microsoft)

------
donniezazen
Would you guys trust Amazon data services like EC2/S3 for backing up your
system which includes all your GPG/SSH keys, photos, important documents like
tax returns, etc.?

------
mamcx
Anything like this but for relational data? I need to sync a sqlite database
and my attemps of using this kind of service show me are not really a good
fit.

~~~
orandolabs
EnduroSync ([https://orandolabs.com](https://orandolabs.com)) will get you
close. It's a light weight object store that sits on top of sqlite.

------
aashaykumar92
Damn, these guys are on a streak! 2 great products released in 2 days. This
one seems a little more innovative than Zocalo though.

------
mey
Looking into the service, the identify providers they support are Amazon
(naturally), Google and Facebook.

------
joshdance
Amazon and their ambition has no bounds. They really are the grizzly bear in
the room of online services.

------
dksidana
I would love to know if there are any open source alternatives to such
solution.

~~~
dksidana
Closet alternative, I have seen till now, is Pouchdb
[https://github.com/pouchdb/pouchdb](https://github.com/pouchdb/pouchdb)

------
Keyframe
Why would anyone outsource such a thing to another company/service?

------
meshko
Amazon: "It is so difficult to merge user behavior data across different
devices they use... I have an idea! Let's create a service which will do it
for "them"!

------
bshimmin
Much better name than "Zocalo", if nothing else.

------
tw007kiid
[https://m.youtube.com/watch?feature=youtu.be&v=UR0qsHWl9aM](https://m.youtube.com/watch?feature=youtu.be&v=UR0qsHWl9aM)

------
infocollector
Does anyone know who made this video?

------
KamiCrit
Did Amazon just change the name of the product that was topping HN yesterday?

~~~
pests
Did you even try reading the link?

------
JTon
How did this float to the top of my front page so quickly. Are we all
genuinely _that_ interested in new AWS products? or did this post receive some
"extra" help

~~~
extesy
This makes Amazon competitors to various backend-as-a-service startups (like
Parse), so yes, it's interesting.

~~~
JTon
Yeah, it does appear to a hot topic. Still, the post was <10m old and #1 with
0 comments. Seems either extremely overhyped or artificial to me.

~~~
plorkyeran
New product announcements from large tech companies make it to #1 before
getting any comments on a regular basis. Submitting a duplicate URL counts as
an upvote for the original submission, so it's pretty easy for something like
this to make it to #1 just from people racing to be the first one to submit
it.

~~~
JTon
Ah, I didn't know about the duplicate URL rule. That makes more sense. I've
been here for quite a while and I haven't noticed such an aggressive advance
before. So it triggered an alarm. Maybe I'll start seeing it more often

