
CouchDB 2.1.0 - tropshop
http://docs.couchdb.org/en/2.1.0/whatsnew/2.1.html
======
tarr11
I was looking really hard at using CouchDB 2.0 for an offline-first
progressive web app. The biggest asset in my opinion of CouchDB is actually
PouchDB [1] PouchDB is really great and nicely designed.

However, the showstopper that I hit was that CouchDB still does not support
permissions on a per-user basis (even though PouchDB encourages this
explicitly [2])

What this means is that if you have a typical system with users and logins,
who can only access their own data, and you require some sort of aggregated
view where you can view all users data behind some sort of permissions (admin
user, etc), you are forced to create a new db for each user in Couch, and then
run replication for those users to a separate master.

Think for example if you wanted to store images in CouchDB, and then have some
sort of global feed (like instagram)

So, systems with many users (tens of thousands -> millions) would be stuck
replicating an equal number of couch databases constantly. Each replica in 2.0
was an erlang thread, and each database is a separate file. Apparently some of
the performance issues were fixed in 2.1, but it still requires a physical
file per db and continuous replication.

Some promising solutions like envoy [3] from Cloudant, which were based on
Mango [4] have been worked on, but don't seem to have much support in the core
Couch community and feels to have been abandoned.

Another problem is that you can't really throttle or manage pouchDB <->
couchDB replication very well. Eg, if you have a large-ish database that's out
of sync, it's just going to try to download itself as soon as you sync, which
isn't very mobile friendly, or "PWA" friendly IMO.

Net result for me was that I just stuck with my Postgres DB, Rails API, and a
bunch of Redux/React code to accomplish what I needed.

Still a fan of Couch/Pouch but wouldn't recommend it if your system requires
any kind of serious per-user or role-based permissions or aggregated views.

[1] [https://pouchdb.com/](https://pouchdb.com/)

[2] [https://pouchdb.com/2015/04/05/filtered-
replication.html](https://pouchdb.com/2015/04/05/filtered-replication.html)

[3] [https://github.com/cloudant-labs/envoy](https://github.com/cloudant-
labs/envoy)

[4] [https://blog.couchdb.org/2016/08/03/feature-mango-
query/](https://blog.couchdb.org/2016/08/03/feature-mango-query/)

~~~
tropshop
> So, systems with many users (tens of thousands -> millions) would be stuck
> replicating an equal number of couch databases constantly

I also ran into this when using `continuous: true` to create persistent
replication, which suffers from this 1-to-1 resource problem. I now use a
single listener on `_db_changes` and fire one-off replication to the aggregate
db. I'm also keeping an eye on spiegel[1]

I'm curious on how you handle offline support, sync conflicts, and multiple
devices. I started with CouchDB 1.x and have been through enough growing pains
to wonder how things would have been to stick with SQL, but I think the future
is only brighter. CouchDB 2.1 looks great, and I hope the entire erlang
ecosystem continues to see growth and adoption.

[1] [https://github.com/redgeoff/spiegel](https://github.com/redgeoff/spiegel)

~~~
tarr11
I know that it is a common trope that Couch / Pouch seems to think that SQL
databases don't work for offline replication and you need their "batteries-
included" solution. However, it's not the only solution and every app in the
app store that works offline has had to deal with it. Most of those are using
SQL databases.

In truth, what Couch does is force you to handle conflict events which gives
you a convenient platform level place to put some of your automatic
conflict/merge code. A lot of merge code is UI however and Couch does not help
with that at all.

Detecting conflicts is a subtle thing, but it's not necessarily a one-size-
fits-all problem as couch would have us believe. I

It also gives you versioning for free (which is pretty easy to implement in
SQL)

However, at the end of the day, every developer must make application level
decisions on how to handle merge conflicts. Couch/Pouch does not obviate this
- it just forces you to deal with it and lets you think about your application
a little differently. Once you have to deal with merges and conflicts in your
application code, it's more a matter of designing your application to have
fewer conflicts (either through better merging, vector clocks, pessimistic
locking, etc)

I wish I could have used it but all my other data is in a battle-tested
Postgres db, being nicely backed up regularly on Heroku without fail. My
CouchDB was on a Google Cloud server, using a brand new replication framework
as backup.

I don't really mean to rag on Couch - I think it's a promising idea. But I
ended up wasting several weeks going down this path only to throw it away
because I didn't feel like it was production-ready.

~~~
tropshop
In production, I have a horribly designed schema running couchdb 1.6 and node.
It is serving a few thousand users, but has serious architectural issues.

I started in freelance php web development, then switched to Ruby/Rails in
2006 and haven't looked back. I'm now thinking mobile access, unreliable
networks, and global connectivity so it was time to re-survey the land. My MVP
was with Node.js and CouchDB 1.6, but now I'm almost ready with a much more
robust and well defined rewrite in Phoenix 1.3 and CouchDB 2.1

I agree that the 2.0 release was subpar. I actually needed to beta test
against master because of a bug in the 2.0.0 release. But even with the rough
patches, cutting Node.js and going all in on BEAM/OTP - I'm hoping will pay
off for the next decade and then some.

As far as CouchDB, I agree nothing is a magic bullet, but after better
learning the entire CouchDB API surface, I was much better off at schema
design and where to put logic in _design files vs application layer. I
actually run a conflict-free schema now, and still benefit from master-master
sync and first class offline support. I'm not saying the concepts are trivial,
but when done in the correct CouchDB way, and getting all the benefits that
come with that, I cannot imagine what it would take to manually handle sync
logic with Ruby and SQL, which is why I asked.

~~~
moneytalks
Any parts of the project open source and/or have you written any detailed
blogs on the development process?

Would love to read more about the challenges you encountered and how you
solved them.

Even moreso the move to Phoenix is enticing for a project of mine. Any tips
there?

------
nothrows
no Per Document Authorization is still my biggest issue with couch that keeps
me from using it
[https://wiki.apache.org/couchdb/PerDocumentAuthorization](https://wiki.apache.org/couchdb/PerDocumentAuthorization)

~~~
feld
The answer I got to this problem was "design middleware that handles this for
you" which has to be a joke

~~~
random023987
> The answer I got to this problem was "design middleware that handles this
> for you" which has to be a joke

It's not a joke.

The couch security model doesn't match the requirements of multi-user
untrusted clients typical of internet distributed applications. But then most
database have a similar limitation, it's only more visible in CouchDB because
you can read/write documents directly from a browser without an application
server, so the next logical step is to just let clients read/write directly to
CouchDB over the internet without an app server.

If your data is in postgres, you will need an application server handing
access control, business logic, and serialization.

If your data is in CouchDB, you need a proxy server that handles access
control, whitelisting certain URL patterns and body content based on user
entitlements.

~~~
Xophmeister
Can you not write validation functions in design documents that handle the
security (to a degree) for you? I seem to remember being able to do this in
CouchDB 1.6 and, while it seemed like a pretty crude method, it was easier
than managing an interposing proxy server.

~~~
devrandomguy
Can you fake a reasonably indistinguishable 404 response for content that
exists, but should not be publicly discoverable?

In the validation function docs [1], there is an example of HTTP errors being
thrown, I'm just not sure if there is something like a `throw({notFound:
null})` option.

[1]
[http://guide.couchdb.org/draft/validation.html](http://guide.couchdb.org/draft/validation.html)

~~~
Xophmeister
The only validation function I've written contains this:

    
    
        throw({forbidden: 'Only administrators can write changes'});
    

IIRC, it raises either a 401 or 403 HTTP error.

------
lmcardle
Why use CouchDB instead of Couchbase?

~~~
gdelfino01
Full RESTful HTTP/JSON API

~~~
stock_toaster
I presume the parent was talking about couchbase's SyncGateway offering, but
maybe not.

------
dsun179
Has anyone tried out rxdb on the client with couchdb as server? I'm affraid
mostly about the scaling issues when you sync with many clients

~~~
nwienert
Currently using this. Haven't tested heavy scaling though because our app
doesn't have huge scale needs. BUT, will say things are looking promising.
Theres are some[1][2] improvements to replication and general speed
landing/landed.

As far RxDB, in our limited tests, using only the query-sync feature speeds up
everything a ton, not to mention optimistic updates. Overall it's been pretty
great, and developer is very active.

[1]
[https://github.com/apache/couchdb/pull/495](https://github.com/apache/couchdb/pull/495)
[2]
[https://github.com/apache/couchdb/pull/470](https://github.com/apache/couchdb/pull/470)
[2]

