
Cloud Memorystore: A fully managed in-memory data store service for Redis - manigandham
https://cloudplatform.googleblog.com/2018/05/Introducing-Cloud-Memorystore-A-fully-managed-in-memory-data-store-service-for-Redis.html
======
dudus
There's a fine print that says:

> ¹Basic Tier instances experience a downtime and a full cache flush during
> scaling. Standard Tier instance experience very minimal downtime and loss of
> some unreplicated data during scaling operation. ²Applicable for GA release
> only.

Does that mean that right now there are no persistance options with this
Managed Redis cluster? I have never tried to use Redis but I've been eyeing it
for a while and the most confusing part is how it handles persistence.

Apparently there are ways to have Redis configured to be pretty durable as
outlined here:

[https://redis.io/topics/persistence](https://redis.io/topics/persistence)

But it doesn't look like Google is supporting either RDB nor AOF with this
managed service, they say RDB is coming though.

Doesn't it heavily limits the use cases for this product and relegate it to a
cooler memcache?

This is not criticism, it's just curiosity on good use cases for this product.

~~~
mrkurt
It sounds like they're scaling by adding larger instances, replicating, and
then failing over. So they'll lose any data that hasn't been synced to the new
instance when the failover occurs. I'd bet it's rare, but they're being
explicit about it.

The Basic Tier probably only uses one instance, so they just replace it in
place.

~~~
trhway
What would be the point of downtime they mention other than doing the sync?

~~~
hinkley
From the docs:

    
    
       Basic Tier: Provides a standalone Redis instance. Use this tier for applications that require a simple Redis cache.
    

If you were only reallocating replicas you'd expect no downtime. But turning
on replication isn't seamless.

------
panarky
The three big reasons I use Redis are:

1) Data structures like lists, hashmaps and sets

2) Persistence

3) Blindingly fast

I wanted to see if Google's implementation sacrifices any speed.

 _snip_

EDIT: I deleted the benchmark results from this post because they're
meaningless. I used the "intrinsic latency" tool [0] that Redis provides, but
it must be run _on the server_.

So my results only reflected the intrinsic latency of the client VM, not the
new Cloud Memorystore.

My apologies.

[0] [https://redis.io/topics/latency](https://redis.io/topics/latency)

~~~
fusl
Your benchmarks:

Google's new Cloud Memorystore, 1.0 GB:

    
    
        redis-cli -h 10.0.0.3 --intrinsic-latency 100
        677411416 total runs in 100 seconds
        avg latency: 0.1476 microseconds / 147.62 nanoseconds per run
    

Open-source Redis on a Google Cloud Platform micro instance, 0.6 GB:

    
    
        redis-cli -h localhost --intrinsic-latency 100
        353427208 total runs in 100 seconds
        avg latency: 0.2829 microseconds / 282.94 nanoseconds per run
    

Open-source Redis on an AWS EC2 micro instance, 1.0 GB:

    
    
        redis-cli -h localhost --intrinsic-latency 100
        21681751 total runs in 100 seconds
        avg latency: 4.6122 microseconds / 4612.17 nanoseconds per run
    

My benchmark:

Open-source Redis on a Hetzner Cloud VPS, CX11 (92% cheaper than Google's
Cloud Memorystore):

    
    
        redisbench-client:~# redis-cli -h 88.99.124.195 --intrinsic-latency 100
        Max latency so far: 1 microseconds.
        Max latency so far: 77 microseconds.
        Max latency so far: 113 microseconds.
        Max latency so far: 130 microseconds.
        Max latency so far: 2562 microseconds.
        Max latency so far: 2835 microseconds.
        Max latency so far: 4165 microseconds.
        Max latency so far: 5497 microseconds.
    
        757281326 total runs (avg latency: 0.1321 microseconds / 132.05 nanoseconds per run).
        Worst run took 41628x longer than the average latency.

~~~
antirez
Unfortunately intrinsic latency does not measure the latency of the Redis
instance, but the one of the whole host, that is, the kernel scheduler max
latency.

------
caleblloyd
Why is it that GCP can provide an internal IP address for Cloud Memorystore
but not for Google Cloud SQL? It would be beneficial for these teams to work
together, no internal IP in cloud SQL makes it much less appealing then AWS
RDS.

~~~
laixer
Private IP support for Cloud SQL is on the roadmap. The reality of development
is that some things are easier to do on a new product. It's not a matter of
communication.

------
neuland
> authorized networks ensure that the Redis instance is accessible only when
> connected to the authorized VPC network

That's good that this is handled smoothly as it's the way Redis wants auth to
work.

I've always seen the security story to be one of the big weaknesses of Redis.
No TLS, opt-in password with no user and only one shared password, no per-
database privileges, etc.

Does anyone know the status of TLS in Redis? I heard somewhere that Amazon has
a patch to add that directly, rather than having to use stunnel.

~~~
kozziollek
It's being worked in in pull request:
[https://github.com/antirez/redis/pull/4855](https://github.com/antirez/redis/pull/4855)

Also, antirez recently met with patch author:
[https://news.ycombinator.com/item?id=16943289](https://news.ycombinator.com/item?id=16943289)

~~~
neuland
Cool! I saw that thread last week, but didn't check back in to see that
antirez responded.

------
jchw
This is exciting, but I really hope that Terraform will add support for it in
reasonable time. These days I'd prefer to not manually manage cloud resources
if I don't have to.

~~~
tecleandor
Hashicorp doesn't seem to like Google Cloud too much. I've been moving to
Google Cloud Builder and it's Jinja and Python templates, which I happen to be
a little familiar with...

~~~
dantiberian
I’m not sure why you’d say that Hashicorp doesn’t like Google Cloud. I can
think of recent partnerships between Hashicorp and Google on Vault and
Terraform that are both substantial integrations. Google Cloud and Hashicorp
both have people that work on the GCP Terraform Provider, and they do an
excellent job keeping it up to date. By all external measures this looks like
a great relationship.

------
Elect2
What are the benefits of using this compare to self-hosted Redis on a Compute
instance?

~~~
jpatokal
It's fully managed, so you don't need to worry about OS security patches,
rotating logs and all the other administration that comes with running your
own server. Basically the same benefits as using Cloud SQL/Aurora RDS vs
maintaining your own MySQL server.

Disclaimer: I work at GCP, but not on Memorystore.

------
anonfunction
Is it managed redis or something google made themselves that has the same API?
I wonder if they will keep up with all the new stuff in redis like streams.

~~~
itcmcgrath
It's the open source version of Redis v3.2.11

~~~
anonfunction
Thanks!

------
PRJ2999
I was pretty excited when Memorystore came out, as there is very little I
enjoy less than managing redis servers. However, I've had nothing but trouble
since switching. Applications than ran smoothly on redis instances deployed on
kubernetes engine are giving me all sorts of problems now.

"NOREPLICAS Not enough good slaves to write" errors have been extremely
common, sometimes followed by problems connecting. Then operations on the
memorystore instance start pending (I mean "repairing"), taking 20+ minutes.
During that time I obviously get connection refused errors. This usually comes
with huge spikes in network in/out that are completely unexplained by the app
pointing to the service.

I thought the pitch for MemoryStore was that it was a managed service; y'know,
less time on devops and all of that. I've found the exact opposite to be the
case. Pay more to Google, spend more time on devops, get a redis service that
doesn't work.

------
edf13
Biggest concern I have about all these hosted Redis replacements is that of
latency.

I use Redis on a highly controlled internal network and also local for its
speed... This goes when moving to a hosted solution.

~~~
jkaplowitz
If you're already accessing it from within GCP, their network is excellent,
not much lower latency than a high-quality internal physical network (under an
order of magnitude) and highly flexible and configurable. In that use case it
should be comparable to any solution you can host within GCP except hosting on
the same instance as each client and using loopback access.

For accessing from outside GCP, yeah the broader internet latency would figure
into this as any other external solution.

~~~
edf13
It depends on what you mean by not much lower latency though...

At Redis speeds the network latency can quite often be the bottleneck.

~~~
jkaplowitz
Double digits of microseconds of difference as of some time last year, if I
remember correctly - it might have gotten even better since then since I know
Google wanted to narrow that gap.

(Disclaimer: While I have in the past worked for the GCP team, nothing in this
comment relates to my time at Google or to info I learned then.)

------
nodesocket
If Redis Cluster support is coming soon, how are they currently handing cross
region replication and failover? Is Redis Sentinel (painful and requires
client support) deprecated now?

~~~
ssambros
> how are they currently handing cross region replication and failover?

We do not, unfortunately. Redis service is regional and only failover from one
zone to another within a region is provided.

\--

I am an engineer on Memorystore Redis team.

------
Arqu
I was hoping to see something of a more flexible pricing like per 100 MB
provisioned based on the far fetched assumption that it is not just a managed
redis(like?) offering but a fully abstracted managed redis service where you
are just charged by GB/s used as with lots of other services.

This just boils down to the same cost as self hosting minus the self hosting
hassle. So not too bad as its just easier to use + no vendor lock in since it
is redis compatible at a similar price point.

------
meteor333
how does this compare with Redis Labs?

~~~
manigandham
Redis Labs (the official sponsor of Redis now) uses a custom proxy layer on
top of Redis, giving you the latest 4.0 version with support for modules, no
downtime and features like storing the values of keys on SSDs. They also
support cross-region multi-master using CRDTs.

If you need VPC/internal access only, or are fine with the lower level of
features, or just want to consolidate on GCP services, then Cloud Memorystore
can work, but otherwise I'd recommend Redis Labs.

~~~
gingerlime
Unfortunately redis labs isn’t available in all regions of GCloud. I was
effectively begging them to add support in Europe a couple of months ago.

~~~
sandGorgon
same here for India datacenter of AWS or GCloud

------
merb
how can a instance with 1gb cost 35 USD per month... also I would've hoped
that google won't add tiers, more like "limits" and that if I only host 1mb on
my instance that I would only pay for 1mb... besides that I could scale to XGb
(depending on my limit)...

~~~
GordonS
I totally get the benefits of a managed service, but...$35/month for 1GB is
just silly.

~~~
dsymonds
Are you sure you fully understand the benefit?

$35 is, what, 30 minutes of an engineer's time from a company's perspective?
If the management of the service would take any more than that every month
(and it easily could!), you're ahead.

~~~
GordonS
Not everyone works in the Bay making boat-loads of cash. My comment was from
the perspective of a small business in Europe.

~~~
dsymonds
Even then, I think it'd be unlikely that the time-cost wouldn't make it
worthwhile. People generally underestimate how much time gets soaked up by
managing a computer service.

~~~
Can_Not
$35 seems to be 3x more cost than AWS's smallest tier (no further comparison
of features/value beyond price), so it does seem expensive for an introductory
project, considering a single $35~ Digital Ocean VPS might all some companies
need their first year. On the other hand, the number of times I've seen an
engineer suggest spending $500-$2000 in labor/hours (when already behind on
mission critical work) as alternative to spending $5/month in some SaaS is
just staggering.

------
f311a
Can someone explain use cases of using remote redis? Doesn't network delay is
a big downside?

~~~
monocasa
In a datacenter, a network request to a server that has the information in
DRAM can be faster than pulling it locally off of an SSD. It goes back and
forth with perf gains on either side, but they're on the same order with
different tradeoffs.

~~~
ryanworl
Especially if that "local SSD" is really a virtualized block device over a
network anyway, which is how most people are deploying their cloud stuff for
convenience.

~~~
qeternity
Is this still the case? We don’t use much cloud from the big boys but I
thought even they were moving towards local storage. All of our OVH instances
are local SSD backed.

~~~
ryanworl
Both are available and AWS and GCP are pretty up-front about what you should
use depending on the use case. The virtualized devices do have advantages in
terms of being able to be moved around and not tied to the life of a single
instance.

------
peter_retief
Great concept, I am having a serious look

