
Lambdoku – AWS Lambda with Heroku-like Experience - kubek2k
https://github.com/kubek2k/lambdoku
======
keithwhor
Hey! Great project. We actually built stdlib [1] to capture an experience
similar to Heroku's for building microservices, and we'll be presenting at AWS
Re:invent on Tuesday [2]. Our workflows with our public API are accessible via
our open source library, would love contributors such as yourself (and anybody
else that finds it interesting!). We have a diverse team working on the
project - Xooglers, support from early Heroku team members, and even some YC
alumni lending a hand. :)

[1] [https://github.com/poly/stdlib](https://github.com/poly/stdlib)

[2]
[https://www.portal.reinvent.awsevents.com/connect/sessionDet...](https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=8092)

------
pepicon
Lambdoku = lick from ass in portuguese

~~~
fiatjaf
ahahah, I thought things wouldn't get worse than "Heroku" and "Dokku".

------
mifreewil
Related: Apex
[https://www.github.com/apex/apex](https://www.github.com/apex/apex)

------
stcredzero
I'm wondering if it would be possible to create something like Lambda, but
optimized for multiplayer servers with game loops? What if there was a
platform that let you write pure functions to create player accounts, spawn
game instances, and handle the transformation of player and game state from
tick to tick? (You'd have to choose a particular architecture for
synchronization with the client, of course. Also, before someone makes the
mistake of assuming I'm as stupid as they are -- I'm not proposing that a
function be transmitted, compiled, then evaluated at each tick over the
network! Functions would be installed into virtualized servers.)

Clojure, Javascript, and Golang would be good languages for this. The
environment possibly would let you inject and replace functions live, lending
to a Light Table like experience, but while developing a multiplayer game.

~~~
wolfgang42
Your basic description sounds like LambdaMOO (written in 1990, the name is a
coincidence).

The LambdaMOO server has a main event loop which handles user events, then
schedules tasks which execute functions ('verbs' in LambdaMOO parlance) to
modify the game state. Verbs are not _pure_ per se but can be thought of as a
transaction which executes in an atomic, consistent, and isolated manner.
Verbs are attached to objects in the game, and can be modified in-game on the
fly by players with their programmer bit set. (Programmers can only modify
verbs on objects they own.)

The LambdaMOO language is a lot like JavaScript with a Luaesque syntax. It has
prototypical inheritance, which meshes well with its concept of objects as
physical _things._ For example, to get a new instance of a duck one would
create a Generic Duck named "My Duck" ('Generic' being the conventional prefix
for an object specifically intended to be used as a prototype.)

There was a time in the late 1990s when it regularly supported 300 active
users at once (with a certan amount of lag); on modern hardware it could
probably handle significantly more than that. If you want a modern scalable
system then LambdaMOO itself is probably not for you; the server is single-
threaded (in fact the whole thing is remarkably similar to NodeJS with
async/await support) and state is saved in-memory and checkpointed
periodically (hence the Durability missing from the above description of
verbs).

However, the basic design (prototypal inheritance, in-database verbs, ACI[D]
tasks, etc) is extremely well proven, and would lend itself well to being
copied into a modern distributed system like AWS Lambda. As long as you had an
ACID datastore to back it up, you could scale the task execution as far as you
wanted, and with a bit of clever work you could probably come up with a
partitioning scheme to make the database scale as well.

In fact, I've partially implemented this already; what's stopped me has
primarily been that I'm more interested in the technical aspects and have no
idea what I would actually use it for once I built it. If you have a use case
for this sort of thing I'd love to hear about it.

~~~
stcredzero
_There was a time in the late 1990s when it regularly supported 300 active
users at once (with a certan amount of lag)_

I had a pseudo-roguelike server first implemented in Clojure, then in Go that
updated 12 frames a second. It could support 250 simultaneous users. It even
had Conway's life as an area attack. I had it posted to Show HN.

 _However, the basic design (prototypal inheritance, in-database verbs, ACI[D]
tasks, etc) is extremely well proven, and would lend itself well to being
copied into a modern distributed system like AWS Lambda._

All that's needed is pure functions, soft realtime, and good tooling and APIs.
ACID takes some doing, in terms of implementation and will also make scaling
complicated, as you mention above. Inheritance? Nice but not necessary. Just
let developers inject functions into a game loop, then tell them their load
and when their simulation tick rate starts to drop.

 _If you have a use case for this sort of thing I 'd love to hear about it._

Yes. Something like that could be packaged into MMOGAAS -- Massively
Multiplayer Online As A Service.

~~~
BoorishBears
> Something like that could be packaged as Massively Multiplayer Online As A
> Service

That already exists, see Photon Engine's multiple PaaS options
[https://www.photonengine.com](https://www.photonengine.com)

Different ones have different APIs tuned for different types of gameplay and
or Unity3D integration (by mimicking Unity's original networking programming
API).

I've implemented them for an MMO I freelanced for a while back and it was
pretty painless.

~~~
stcredzero
So Photon is operating a farm of servers that are running game loops, which
can have pure functions injected into them? (Also, the per message pricing!)

~~~
BoorishBears
No, Photon is doing "Massively Multiplayer Online As A Service" (did you even
read their page?).

And actually they are running game loops, but you can't inject code into them
unless you run their software on-premise, or use a separate system I recall
them mentioning that offered server-side scripts that got injected in their
loops.

And cost per message make perfect sense because of the networking model, which
is fully message based. Paying per message is paying for bandwidth + cpu usage
as a single combined rate.

~~~
stcredzero
_No, Photon is doing "Massively Multiplayer Online As A Service" (did you even
read their page?)._

 _And actually they are running game loops, but you can 't inject code into
them unless you run their software on-premise_

So you are seemingly contradicting yourself. 1st you said that "they are doing
that." Now you are saying they are also doing "Massively Multiplayer Online As
A Service" \-- but in a quite different way than what I specified. So yes, I
did read what you posted. Apparently you only carelessly read mine.

~~~
BoorishBears
I think you need to read my post again then. I'm not contradicting myself in
any way.

You acted like no one offers a PaaS for MMOs (and gave it a ridiculous name).

I pointed out just one of _many_ PaaS options for MMOs (or ...MMOAAAS as you
insist on calling it.).

In fact, when working for that MMO we did evaluate options the did let you use
their game loop with your cloud functions being called on triggers, but you'd
have to look that up yourself, that was a few years ago (even Photon has added
new services since I did that research).

If you think you have an approach that can enter the space, by all means go
ahead, but PaaSes for MMOs have existed for ages.

------
coenhyde
This is great. I was going to create something similar to manage lambda
functions across environments. Awesome to see you're already using the newly
released environment variables for lambdas.

~~~
kubek2k
thanks! - please give it a try and let me know in the issues if something
doesn't work as expected.

------
oldopsguy
Another nice project that helps to ease lambda workflows is
[https://github.com/garnaat/kappa](https://github.com/garnaat/kappa)

~~~
mullen
There is similar project called Zappa.

pip install zappa

~~~
Mizza
Zappa actually uses some of Kappa under the hood, but with many many more
features. Learn more here:
[https://github.com/Miserlou/Zappa](https://github.com/Miserlou/Zappa)

~~~
mullen
Is Zappa a fork of Kappa? I have not used Kappa at all but it looks like it a
whole lot.

------
josegonzalez
For a second there, I thought someone implemented a push-to-deploy experience
for Heroku (with some of the other Heroku niceties). I actually wrote a blog
post about that this week: [http://josediazgonzalez.com/2016/11/21/push-to-
deploy-lambda...](http://josediazgonzalez.com/2016/11/21/push-to-deploy-
lambda-applications/)

~~~
kubek2k
really good read. thx! :bow:

I didn't plan to do the push-to-deploy thing, mostly because that would
totally complicate things. The rationale was more that we would use the CI
that would automatically build and deploy things to DEV env that would be then
promoted around.

------
charlesetc
You could easily prefix your special environment variable with 'lambdoku_' so
you don't have to fear that people could use it..

~~~
kubek2k
:bow:

[https://github.com/kubek2k/lambdoku/issues/8](https://github.com/kubek2k/lambdoku/issues/8)

------
aghillo
Has anyone characterised server less architectures? I'm interested in testing
/ exploring the limits with some predictive modelling work and implementing
(mathematical) functions as FAAS? No doubt, it 'depends' (e.g. nature of data
and its locality or otherwise) but interested to hear if anyone is looking at
very fine-granularity server less systems?

------
ohstopitu
How is this any different from Serverless? (not trolling, I could not find any
differences in the README)

~~~
kubek2k
I don't mind being trolled :)

Serverless is way much more advanced machinery than this tool. My point was to
create a tool, not a whole ecosystem around AWS Lambda.

When it comes to serverless - my main concern (as it was with Lambda without
env variables), is with storing configuration data along the code in the repo.
Additionally the reason for creating was mostly pipelines.

------
michaelbuckbee
Very cool project, but it has me thinking what/if it would be possible to
build out more of the infrastructure management that Heroku has to make it
even easier (domain setup, addon store, etc.)

------
tjholowaychuk
Looks cool! I like the release message stuff, I wanted to add similar to
apex(1) at some point, not having a clue what the releases actually are in
Lambda makes rollbacks a little tricky.

~~~
kubek2k
thats a wonderful improvement of apex, thats for sure :)

------
fiatjaf
The guide on the README doesn't tell me how am I supposed to push and update a
function.

Also, does it help with dependencies or something like that? Does it manage
packaging files?

~~~
kubek2k
the initial idea was that CI deploys lambda, and you promote it from stage to
stage. I might consider upload sometime.

From what you write I think apex would be better suited for you

------
stcredzero
_You acted like no one offers a PaaS for MMOs (and gave it a ridiculous
name)._

I was talking something specifically like Lambda, but where pure functions
could be injected into an already running game loop. The name was
intentionally ridiculous. Also, you suggested that my proposal would suffer
from the latency of lambda requests going over the network, which shows you
didn't read my original comment.

 _with your cloud functions being called on triggers_

Yeah, I pretty much covered that this was _not_ what I was proposing, when I
wrote "(You'd have to choose a particular architecture for synchronization
with the client, of course. Also, before someone makes the mistake of assuming
I'm as stupid as they are -- I'm not proposing that a function be transmitted,
compiled, then evaluated at each tick over the network! Functions would be
installed into virtualized servers.)" As far as I knew (which might be out of
date) Photon was developing something that would be suitable for a persistent
world MMORPG, but not quite ready with it. What I'm proposing would be
suitable for an MMORPG, not just MMO game sessions.

~~~
dang
We detached this subthread from
[https://news.ycombinator.com/item?id=13062880](https://news.ycombinator.com/item?id=13062880)
and marked it off-topic.

------
brilliantcode
what I'd really like is ability to take a swagger spec and click a button to
deploy it on AWS Lambda.

This is great for internal facing REST Api. I don't know about scaling but
hypothetically serverless means it should handle public requests fine.

~~~
kubek2k
I guess you mean the API-Gates + Lambda. Not sure - I think thats something
that should be handled by some separate tool (APIGatesgger? ;) - lets hope its
not another dirty word)

~~~
brilliantcode
that's right swagger -> api gateway triggering lambda functions.

I was using Chalice but seems like Zappas is another one a HNer pointed out. I
couldn't get Chalice to work and sort of gave up on the serverless-api-gatway-
lambda idea.

