
Google Cloud Firestore NoSQL database is in GA - markoa
https://cloud.google.com/blog/products/databases/announcing-cloud-firestore-general-availability-and-updates/
======
tpetry
There is something i don‘t understand about Firestore: If you use the web
client your javascript code is directly communicating with the firestore
database, you have no control about some user specific limits. How do you
prevent some rogue actor from effectively doing billions of reads or writes.
With server side logic you can implement rate limits but not with a fire store
web client. The only thing you can do is to limit your monthly budget? So one
bad guy can burn all your money for a month without you having any possibility
of limiting it?

~~~
manigandham
In these kind of systems, your app users also become database users and
effectively each visitor gets their own token to access the database.

You then add rules to limit users so they can only read/write their own data.
This works fine for many simpler scenarios and avoids the need for a backend
completely, with the potential risk for overuse.

If you need more control and protection then you should have a backend layer
to get the data server-side.

~~~
maldeh
By "In these kind of systems, your app users also become database users", do
you mean by RBAC or some other session-based control on the backend, or does
Firestore perform some intelligent user entity resolution I'm missing?

As far as I can tell, any naked frontend-only client serving content without
its own access management sounds quite open to a single bad actor spawning a
large number of sessions on the same host, and on a small number of
distributed hosts, to eat up all available resources. I'm not inclined to ever
trust a number of users behaving nicely at n>100 personally.

Given that Firestore appears to advocate "serverless" client-driven
applications, unless there's some foolproof DDoS mitigations I'm finding it a
hard sell, especially on the whole "if you need more control and protection,
do X" argument - you rarely need that until somebody straight-up exploits you,
and when you do, you're not going to be particularly sympathetic towards
Google's marketing speak.

~~~
thecatspaw
You can require login for read/write access

------
seanwilson
For a subscription based Chrome extension
([https://www.checkbot.io/](https://www.checkbot.io/)), I use Firestore,
Firebase and Cloud Functions to handle subscription activation and
login/authentication. I use Paddle for payments which fires a web hook to a
Cloud Function when a purchase is made.

Works great so far for no cost and it hasn't required a single bug fix or any
maintenance since launch. Firestore definitely has a lot of surprising caveats
though and you need to design your app and data around this to avoid trouble
later.

Does anyone have any stories to share about when they outgrew Firestore, what
they migrated to and how? I wouldn't be keen to use NoSQL if my data became
more complex.

~~~
navs
Paddle's pricing seems a bit high at 5% + $0.5. Is there a reason why you
opted to use Paddle over Stripe? Curious because I'm similarly working on a
Firebase MVP at the moment and looking to integrate a donation feature.

EDIT: nvm, you answered the Stripe vs Paddle question in another reply.

~~~
seanwilson
It does seem high but even doing tax returns is annoying enough so I've no
desire to get involved with EU VAT and tax issues selling to other countries.
This way I can set up payment and it runs itself which is worth the cost and
Firebase means I don't have to think about DevOps too. Gumroad is another
payment option to consider.

~~~
irq-1
Interesting:

> Stripe does not calculate any tax information for you.

Paddle:

> We automatically handle the calculation, collection and remittance of taxes
> in every single country on your behalf: you don’t need to do anything.

------
alanlamm
Paying Firestore user here.

1\. Disappointing that it has gone to GA without providing proper search
(after a long beta). Can anybody explain why a service run by the worlds #1
search company continues to point users to third party services if you want to
implement a basic text search in a database? I genuinely don't understand
that.

2\. Feature requests (or complaints) re backup, queries and documentation are
not new. Nor are the answers (or excuses) we see here, which revolve around
scalability and (until now) being in beta. BUT all users I have ever heard say
they use the product for speed & ease of setup & convenience - for MVPs, not
for massive googlesque data. It almost feels like the product market fit is
not quite right. So, forgive my technical ignorance here, but worst case
scenario, why not provide the features with the caveat that they are slow or
expensive or won't work above a certain size of db? Isn't half a solution
better than no solution?

The database and whole Google/Firebase app suite thing has some strong points.
But to be frank, and I'm sorry if I'm being dismissive of hard work and
technical wonders here, from the perspective of a customer and outside
observer, a number of things smell quite off.

~~~
itcmcgrath
Thank you for being a paying user.

1\. Mainly because it's both a different search problem (general DB vs
specific to web search) and hard engineering wise given our model; we
implement not only the cloud database, but embedded versions for iOS, Android,
and Web - not to mention real-time functionality and tailoring it to how our
index engine works, etc. While we have a lot of customers and use-cases that
don't need Fulltext Search, we totally agree it's important and have done
explorations on how we'd deliver something along these lines.

2\. Agreed. During the beta program we've delivered the managed export and
import service for backups, adding array contains capabilities to queries and
have got close enough to delivering Collection Group queries to mention them
as part of GA. For documentation our tech writing team as done _a lot_ of
updates, new pages, and fixes - we know there is always more to do. Cloud
Firestore is definitely used in production and at scale by our customers, and
with nearly 1 million databases being created the range of use cases and
traffic/load patterns has been vast. Our beta program involved working with a
lot of them to improve things like hardening and scalability to ensure we can
meet our 5 nines of availability SLA.

"Isn't half a solution better than no solution?" -> In a lot of cases,
absolutely not. A half solution that falls over when you tip a certain point
of scale can result in extended downtimes, since the solution often ends up
being "we need to completely rearchitect this", which isn't easy or quick when
your business is out of commission.

"from the perspective of a customer and outside observer, a number of things
smell quite off." -> Sorry to hear this, I can only hope the continued hard
work from the team will turn you around.

~~~
alanlamm
Thank you for the answer. I have to admit it doesn't quite sway me, for
reasons such as those below, but thank you.

e.g. yes I realise they are different search problems, but I'd presume that
Google is nonetheless well-equipped to handle the document db one. the only
apps I can imagine that couldn't benefit from a search box are games -
anything content-focused or ecommerce focused needs one, and the majority of
utilities benefit too (yes I can do chat without search, but it certainly
benefits from being able to search through chats) - any examples? yes I
realize having to do Backend/iOS/Android/Web is hard (as it is for everybody
else), but with on device cases at least the db is smaller. Im sure you do
have big users, I didn't mean to imply otherwise, but with my admittedly very
limited knowledge I'd still wager that a majority do not see uptime and
scalability as the most urgent improvements, but rather those we are
discussing. In our case, give us just 2 9s of uptime, and give us the above
queries and searches even if 2x as slow and expensive as you'd like them to
be, and limited to a db the size of an average relational db, and that would
beat the extra 3 9s of uptime and the super scalability any day. Not least
because, I don't mean to be rude here, just candid, but if we were ever to
reach a point where we needed that massive scale and uptime, I'm not sure I'd
be keen to trust Google with user data.

To be clear - I like several things about Firebase/Firestore, which is why we
use it and why I'm insisting on badgering you here. I just wish I could be
completely comfortable with my choice rather than wondering every day if I
shouldn't just use something else.

------
sunsetMurk
Been loving Firestore! It has been my first real experience w/ NoSQL in an MVP
to production-ready quickly. It's been SO easy to experiment with and learn.
Community has been great.

~~~
BigBalli
Gad you been using realtime database before? Curious about experiences with
transitioning (additional fetaures do sound appealing).

~~~
vira28
We were using RTDB and migrated firestore. The querying limitations in RTDB is
really painful. Transitioning was not that difficult. Basically wrote a script
in node.js to migrate the data. (our db size is <100M)

~~~
bytematic
I'm guessing this includes updating your source code for the new
library/methods

~~~
vira28
Absolutely. Need to go through a whole bunch of changes (in architecture,
query, security rules, backend like cloud function etc...)

------
nerdywordy
Anyone with inside knowledge of Firestore care to comment on the ETA of more
advanced query functionality?

We have played with Firestore quite a bit, but rely heavily on the ability to
do aggregate queries. Reading all of the documents and performing this on the
client side is nowhere near good enough. Nor is triggering functions to update
a "count" or "sum" property on a doc.

Edit: Looks like a PM answered on another thread...

"It's a point of internal discussion on scalable ways to achieve this, but
nothing we can promise. We definitely see the need for it."

~~~
asien
The documentation clearly mention those features aren’t supported as of now.

The Firebase team is aware of that need but it’s very hard to deliver such
features at scale.

Firebase is used everywhere these days , so a single change in the platform
has impacts in petabytes and massive thousands of cpus load.

~~~
nerdywordy
I get that it isn't supported. We have followed the recommendations mentioned
in the docs but found the usage of these workarounds to be sorely lacking in
performance and reliability.

------
itcmcgrath
Product manager for Cloud Firestore here. We extremely grateful for all the
amazing beta program customers and all the feedback they've given us.

~~~
kungito
Hi.

As a student developing a system for a friendly NGO I must say that having
angularfire integrate with Angular made this an instapick because I wanted to
get started asap.

The issue now is that it really feels like the docs need more examples. I'd
take 4-6 hours to add a new feature and I'd spend them reading the docs.
Sometimes I'm not sure if I don't get something or it's a missing feature.

I guess the community is still growing but it's really hard to find answers to
some simple problems.

angularfire has an inactive gitter and there isn't one for firebase
specifically. It would be really amazing if firebase devs could take some time
to answer the community on gitter or something, at least at the beginning
before there's more people knowing the answers.

EDIT: Found your link to the google group. I guess I'll ask there then :)

~~~
danielsju6
AngularFire lead here, I'd recommend you check out our GDE Jeff Delaney's
community here [https://angularfirebase.com](https://angularfirebase.com) for
AngularFire related content. He has tutorials, lessons, and maintains a very
active Slack Channel.

There's also the Firebase community Slack
[https://firebase.community/](https://firebase.community/) and firebase-talk
is our supported community channel for all things Firebase
[https://groups.google.com/forum/#!forum/firebase-
talk](https://groups.google.com/forum/#!forum/firebase-talk)

------
_hardwaregeek
Why isn't there a hosted, easy to use relational version of Firestore? I'm
using Firestore right now and it's absolutely frustrating in the lack of
relations, the lack of accumulation queries (COUNT, MAX, MIN) and in general
the limitations of storing what's essentially arbitrary JSON. How is it that
to count the number of entries you either need to build in a counter on your
own (which can cause race conditions because counters are hard) or manually go
through the entire collection and count?

Another option would be a typed JSON db, essentially you could store JSON that
corresponds to typed structs a la serde. Would't solve a lot of my problems,
but at least I'd have some built in validation.

~~~
jimbokun
Because relational databases are much harder to scale. There are many people
working on solving this problem.

~~~
_hardwaregeek
At what level? Because I know StackExchange runs off of relational databases
and they get a solid amount of traffic (with some Redis, but the foundation is
relational). Even if relational doesn't scale for Facebook or whatever, if I'm
making a piddly little chrome extension or a small website, a simple
relational database will scale just fine.

~~~
joatmon-snoo
It's important to note that StackExchange is one of very few examples of
successful vertical scaling, at least in the post-mainframe world.

If you want a relational DB as a cloud service, it sounds like you want Cloud
SQL (as far as GCP is concerned).

(Disclaimer: I work on GCP.)

~~~
wsh91
Cloud Spanner is also relational. :)

(Fellow GCPer.)

------
EZ-E
Having heavily used Firebase Realtime Database (firestore's ancestor), I think
I will approach this one very carefully. Firebase Realtime Database was a
nightmare with frequent downtime, sometimes minutes, occasionally an hour.
Also almost weekly, all clients sometimes wouldn't get notified of document
changes which was crippling for our app.

~~~
jmvoodoo
I've been using cloud firestore for 6 months now and haven't had an issue with
it. Designing around their rules system has been annoying, and the lack of
backups bit me once pretty hard (I built
[https://firesafe.app](https://firesafe.app) as a result), but other than that
it's been great.

~~~
thesandlord
I think one of the requirements for Firestore going GA was getting a
b̶a̶c̶k̶u̶p̶ managed export system:

[https://firebase.google.com/docs/firestore/manage-
data/expor...](https://firebase.google.com/docs/firestore/manage-data/export-
import)

Edit: As wsh91 says below, you are still responsible for the scheduling.

(I work for GCP)

~~~
vira28
right, but didn't see anything mentioned about backup in the link. or am i
missing anything?

~~~
wsh91
An export is one component of a backup solution. :) Scheduling is another
([https://firebase.google.com/docs/firestore/solutions/schedul...](https://firebase.google.com/docs/firestore/solutions/schedule-
export)). (Checking restores is critical, too; that's also feasible [managed
import into a new cloud project] but requires a bit more legwork.)

(I work on Cloud Firestore.)

~~~
jmvoodoo
FWIW, I'd love it if you made my app unnecessary. You can consider my site a
list of feature requests :)

------
vira28
"More features coming soon. We're working on adding some of the most requested
features to Cloud Firestore from our developer community, such as querying for
documents across collections"

Looking forward to it. If this comes in, pretty much it removes the need for
creating top level collections or am I missing anything?

------
SnacksOnAPlane
I still haven't seen a great reason to go NoSQL over, say, Postgres. I'll
think about a good application but then realize that it'll be a PITA to do
something slightly different than what I first imagined.

What are good reasons to use NoSQL over SQL?

~~~
_hardwaregeek
I'd love to use Postgres, but Firebase is really nice in that it provides
first class libraries that abstract away the requests and querying, to the
point where querying Firestore just feels like calling an async function on
your front end. Plus, you don't even have to think about deploying. There's
something really great (and risky) about having an always deployed, always
ready backend. Also _basic_ user auth is really easy.

If you're doing a long term project, then Postgres makes sense since the
deployment/setup costs are one time. But for short projects Firebase is very
nice.

~~~
tirumaraiselvan
But there are managed Postgres services in almost all cloud providers.

------
bcheung
I've been using Firestore in a React app and I love how it has simplified
everything. There's no need to have redux or even an in-memory store in the
"context". I just wrap my components with a withFSQuery helper and it
automatically updates even when the state changes "server-side".

~~~
jen729w
Would you have time to elaborate? I’m just starting to set up a React-
Firestore app but I’m a bit of a n00b.

I’m using re-base [0] to synchronise Firestore to my local state. Do I not
need to be?

Any pointers gratefully received, thanks.

[0]: [https://github.com/tylermcginnis/re-
base](https://github.com/tylermcginnis/re-base)

~~~
bcheung
I didn't use any library. Seemed easy enough to just create a few helpers. I
rely on Firebase completely for the data. Since Firebase already keeps a local
client cache as well as an offline cache, I didn't see any need to maintain
separate state in React -- outside of the HOC components. They subscribe
automatically to the data when the component mounts and unsubscribe when the
component is destroyed. Any changes, whether from within the app, from the
server, or from another instance of the app; have changes synced everywhere
for free.

The only local state I have at the app level is state that doesn't get stored
in the DB.

Here's the helper HOC I created for it:

[https://github.com/brennancheung/wowzrs/blob/master/src/app/...](https://github.com/brennancheung/wowzrs/blob/master/src/app/core/FSQuery.js)

And an example of it being used as a render prop:

[https://github.com/brennancheung/wowzrs/blob/master/src/app/...](https://github.com/brennancheung/wowzrs/blob/master/src/app/components/actions/ActionsList.js#L34)

And another as an HOC:

[https://github.com/brennancheung/wowzrs/blob/master/src/app/...](https://github.com/brennancheung/wowzrs/blob/master/src/app/components/actions/ActionsForToday.js#L33)

And here's a mutation:

[https://github.com/brennancheung/wowzrs/blob/master/src/app/...](https://github.com/brennancheung/wowzrs/blob/master/src/app/components/actions/AddAction.js#L14)

The HOC's make it so you can create just plain stateless functional components
and not have to worry about componentDidMount.

------
TimMeade
We have been using it for over a year and this is good news. Hopefully a good
product will move to a great one.

------
burtonator
A months ago I published a write-up of building our our cloud sync in Polar:

[https://getpolarized.io/2019/01/03/building-cloud-sync-on-
go...](https://getpolarized.io/2019/01/03/building-cloud-sync-on-google-
firebase.html)

Polar is basically a document management and annotation platform. You put all
your reading in it and maintain it long term along with annotations,
highlights, etc.

Firestore is really nice in that you can target multiple-platforms pretty
easily. There are SDKs for basically every platform.

It's definitely not perfect but I'm pretty happy with the decision.

------
kylehotchkiss
Will there ever be support for aggregate queries? Or queries for non-existent
values?

I have a rather large dataset that's tough to scan over and I find the
functions/transactions to build my own aggregates not entirely accurate enough
since a document has rate limits for writes.

~~~
itcmcgrath
It's a point of internal discussion on scalable ways to achieve this, but
nothing we can promise. We definitely see the need for it.

~~~
kylehotchkiss
I totally understand its a complex issue. Is there someway I could subscribe
to updates regarding this subject for the future (even if it's 2 years away I
would enjoy testing any potential developments here)

------
ptasker
The issue we had with cloud Firestore is the inability to _really_ query
documents through the Firebase console. It’s very basic and there is no third-
party tooling available yet.

To run any kind of specific query you’d need to handle that application side.
Just something to consider.

------
microcolonel
I find the limitations of their query language interesting, no way to test for
undefined properties in a collection (the client libraries just give up, and I
don't see a workaround); and no way to match on inequality rather than
equality. This exists in some other odd database engines, but it's frustrating
in general use scenarios (where, in this case, I don't exactly see how
performance is a limiting factor).

------
camdenlock
This service feels pretty shoehorned into the rest of GCP. There are little
connectors everywhere for Firestore specifically, and they don't fit in
cleanly with the rest of GCP offerings. Why is Google so bullish on this
thing? I must not be seeing the appeal.

~~~
thesandlord
(I work for GCP)

Where do you feel it is shoehorned in? Firestore is easy to access from App
Engine and Cloud Functions (and GKE/GCE as well) using the built-in service
accounts, it has managed export into BigQuery. Would love to know where you
found friction when using it with other GCP services.

> Why is Google so bullish on this thing?

While I can't speak for Google as a whole, I'm personally bullish on Firestore
for a few reasons:

1) Gives you the same "client-side only" model that Firebase popularized. This
makes creating applications much faster as you remove the whole server side
component and basically all your ops work.

2) True "serverless" pricing and ops. Because you only pay for reads/writes
and not instance time, your costs scale linearly with your app's usage. And
you don't have to worry about sharding or other operationally complicated
things to scale your database as you grow. The caveat here is you have to
structure your data and queries well, otherwise your costs will skyrocket [0].

3) Gives you the same "server side" functionality as Cloud Datastore (which
Firestore is basically the next generation of [1]). This means you can use
Firestore in place of a traditional NoSQL database like MongoDB.

4) Strong consistency. One of the biggest problem with Cloud Datastore (and
most NoSQL databases) is eventual consistency. Datastore worked around this
with "Entity Groups" which in my opinion were super clunky to work with and
very limiting. Firestory is strongly consistent so you don't have to worry
about this even at scale, which is super nice.

5) GCF Triggers. The fact that you can trigger a cloud function when something
is written to the database is very powerful, it's basically like a traditional
database trigger or stored procedure but you can do anything.

The biggest feature gap for Firestore is the querying abilities. While it is
way better than the original Firebase DB, it's nowhere close to a relational
database or something like Cloud Spanner. The team is working on it though.

The "real time" stuff is interesting but not really super relevant to the
things I like to build.

[0] [https://hackernoon.com/how-we-spent-30k-usd-in-firebase-
in-l...](https://hackernoon.com/how-we-spent-30k-usd-in-firebase-in-less-
than-72-hours-307490bd24d) [1]
[https://cloud.google.com/datastore/docs/upgrade-to-
firestore](https://cloud.google.com/datastore/docs/upgrade-to-firestore)

~~~
mbrynard
I mentioned it in a comment above, but one way in which it feels shoehorned in
is the fact that you can only have one Firestore per Google Cloud project.
Having only one Firestore per project makes setting up test or staging
environments a hassle.

~~~
thesandlord
Oh yeah, this fact is definetly my least favorite part about Firestore. Also
the fact you can't change the location once it is created.

I don't know what the roadmap is for removing these restrictions (or even if
it is on the roadmap) but I'll update this post if I learn more.

------
oyebenny
In GA? Like the State?

~~~
black-tea
If I remember correctly from last time it means "general availability" which
is some internal corporate lingo that most people won't understand.

~~~
neom
It's not internal corporate lingo, it's how business that produce products
plan the lifecycle of their products, and while it's commonly used in
software, it's a very standard term across verticals.

------
alanlamm
curious: how big is the Firestore team? what about the team for the Firebase
products as a whole?

------
manigandham
Will there be read-only replicas in other regions available in the future?

~~~
itcmcgrath
It's technically possible for us to build that, but we're focusing on the most
common requests at this time. If anyone has that please let your account reps
know (they feed us lots of great feedback for prioritization), or start a
discussion on our Google Group:
[https://groups.google.com/forum/#!forum/google-cloud-
firesto...](https://groups.google.com/forum/#!forum/google-cloud-firestore-
discuss)

~~~
manigandham
Yes, this is what we like about RDS/DynamoDB/CosmosDB. Being able to run low-
latency replicas (bonus if they accept writes) is important for distributed
apps, but the current multi-region deployment adds too much overhead.

~~~
notyourwork
Global replication simplifies a lot of things.

------
wahnfrieden
Any notable companies/products known to use Firestore in production?

~~~
burtonator
[https://getpolarized.io/](https://getpolarized.io/)

We are... We're not massive yet but we have about 250 users who've updated
about 20GB since our launch. Still really early as we haven't had a massive
launch yet.

We're still soft launching.

------
XorNot
So next up we'll finally get an API for configuring firebase right?

~~~
asciimike
The Firebase Management API
([https://firebase.google.com/docs/projects/api/reference/rest...](https://firebase.google.com/docs/projects/api/reference/rest/))
was announced at the Firebase Summit last October, and will expand to provide
more configuration over time.

------
fnayr
I want to use Firestore in Unity. When???

------
nfRfqX5n
any implications for regular old Firebase?

~~~
itcmcgrath
None, still a fully supported option for users.

------
fukshime
Be careful when storing data. Flat earth data could be deleted :) it violates
community guidelines lol

