
Ask HN: Stateful Cloud Functions? - r_g
Sorry if this isn&#x27;t a place for technical questions, I just thought the subject might be unique enough to start a discussion on the idea of a stateful cloud service if it isn&#x27;t a thing yet.<p>But anyways, has anyone heard of a cloud function-like service that can hold state, i.e., keep ~3g of data in RAM on standby and scale up +3g per request? I don&#x27;t think memcache is an option and typical boxes like DO or EC2 don&#x27;t seem scalable at this level. Current requirements are only for 1 user and needs to scale to a few dozen later on..<p>The context, for anyone wondering, is a microservice on Google Cloud Functions that extracts data from news articles using CoreNLP. CoreNLP needs 2-3 Gbs of models in memory at runtime. Given that the microservice needs to process 100-1k articles per request, reloading those models each time just isn&#x27;t an option.
======
PaulHoule
You need 3GB for the trained model or 3GB to process an incoming request.

A typical way to deploy this kind of service is to have one or more servers
waiting on a message queue and process requests off of them.

I'm definitely happy to talk more.

~~~
r_g
Hey, thanks for the input.

I need 3gs in ram at all times to have the models pre-loaded, otherwise the
request time goes exponential. Then, ideally, memory could scale up/down with
each request. Probably more like +2gs per request, e.g. RAM_GB=2r+3.

Its working fine right now with an 8GB box on DO for 1 user, but each user
will add ~50% more compute power to the overhead. The cost isn't really an
issue, $40/m is a microscopic fraction of the user's monthly fee, it's just
that I'd prefer an auto-scaling system because I'm not a devops specialist and
downtime for these users isn't really an option.

I don't think a message queue is needed right now with request pooling and
socket limits,though I will need it later on.

I can shoot you an email if you'd like to chat about it. I'm no CoreNLP pro
but I might be able to provide a few tips in return.

~~~
PaulHoule
Why don't you do that?

