
LocalStack – A fully functional local AWS cloud stack - manojlds
https://github.com/atlassian/localstack
======
timvdalen
Huh. One of the biggest 'drawbacks' of using AWS as a production platform is
that making your development environment look like production is hard.

Having to deploy to test is cumbersome and having a cost associated with each
test can definitely introduce some sort of 'stress' and encourage people to
not test incrementally.

I wonder if this changes that. Having services like S3, Lambda and SQS
available locally sounds super interesting.

I will definitely keep an eye on this. One other thing I would like to see is
CloudFront, which can be very hard to start with due to its opaqueness. Having
to wait 20+ minutes between configuration changes is very demotivating.

~~~
korzun
> your development environment look like production is hard.

The hard part is the cost IMHO

------
scrollaway
Really impressive. Although the title sounds a bit misleading, it's only
mocking stuff from what I can see (which makes sense).

I know there's some actual open source reimplementations of bits of the AWS
products, such as S3. Might be worth integrating into it.

~~~
nickstefan12
"Minio" as a drop in local replacement for s3 has been great for local
development. Small Golang binary.

~~~
nik736
Minio is awesome, I use it in production to store many TBs of data and never
had any issues. The clustering functionality could use some additional work
(or I just havn't understood it right) and it would be nice to be able to
generate multiple keys/users.

~~~
jamroom
Out of curiosity how have you found the performance of clustered Minio? I've
been checking out Riak S2 but the simplicity of Minio is attractive. Thanks!

~~~
im_down_w_otp
I don't think Minio is clustered.

~~~
jamroom
They have a "distributed" setup: [http://docs.minio.io/docs/distributed-minio-
quickstart-guide](http://docs.minio.io/docs/distributed-minio-quickstart-
guide)

~~~
im_down_w_otp
Ah. I stand corrected. I'm leery of their consistency model in that mode, but
could just be that I haven't dug into it far enough to make sense out of it.

~~~
coredog64
> I'm leery of their consistency model in that mode

Amazon S3 is eventually consistent, so is this really any worse?

~~~
im_down_w_otp
For authorization/authentication policy updates it isn't.

------
rch
Another option along the same lines:

[https://github.com/eucalyptus/eucalyptus](https://github.com/eucalyptus/eucalyptus)

------
cygned
Interestingly, that Atlassian co-publishes this on GitHub, keeping the main
development on BitBucket.

~~~
dkersten
GitHub has become the defacto standard place for open source. I've heard it
said that if you care about contributors or users of open source, you should
publish to GitHub even if that's not where you develop it. I guess Atlassian
have realised this too...

~~~
INTPenis
I understand what you're saying and unfortunately it's true but it's also
silly because Gitlab, Bitbucket and others can use Open Auth APIs to make
contribution easier. I for example use Google Auth on Gitlab but Github didn't
have that back when I signed up. If they did I would have used it there too.

~~~
dkersten
I guess GitHub got mindshare early and then became the entrenched option. So
its less about technical merits than not, although anecdotally, while GitHub
has its flaws, in many ways I prefer its UX to that of GitLab and Bitbucket.
For example, Bitbucket's overview page isn't very useful to me at all and yet
its the default repo page.

------
mason55
Am I missing the documentation somewhere? I see directions for setup and
installation but nothing about what's supported for each mock. Obviously they
haven't rebuilt all the service from scratch to run on my local machine, so
what can I do here? If I create an S3 bucket will the API return that bucket
in my next ListBuckets call? If so, is that persistent? Etc.

------
djhworld
My concern with relying on things like these for testing is they're not
supported by AWS, so behaviours can easily diverge without you realising.

I think we've got a lot of mileage in the past out of just mocking
request/responses when testing AWS interactions. The only time we've ever used
a pseudo-service was when testing against the dynamoDB JAR they provide with
the SDK.

~~~
ascendantlogic
That is my first concern as well but testing and bug counts are not an all-or-
nothing thing. If something like this can reduce bug counts in a complicated
distributed system while reducing costs vs testing against "the real thing" it
still may very well be worth it.

~~~
fadzlan
Yup. There's nothing to stop some to have a lot of local iterations for speed,
and then later when you are satisfied, have the test run on real AWS.

Would save some money too.

------
truetuna
I've tried localstack once a few months ago to mock SQS and found it really
slow to boot, had a pretty high memory footprint, crashed very often (was
running inside their official docker container I recall), and lacked
documentation.

If you just need to mock SQS, I recommend using elasticmq. I've used it for a
few months now and haven't had any problems at all so far.

[https://github.com/adamw/elasticmq/blob/master/README.md](https://github.com/adamw/elasticmq/blob/master/README.md)
[https://hub.docker.com/r/expert360/elasticmq/](https://hub.docker.com/r/expert360/elasticmq/)

------
qeternity
This is something I hadn't really thought about before. Aside from this, how
have AWS devs done local development (have they?).

~~~
larzang
Same way as any other API really, mocking and/or real dry run calls where
supported (which is most of the API in AWS' case).

Then a staging version of the real environment for final testing.

~~~
bpicolo
Dry run calls for AWS just check IAM permissions according to docs. Am I
missing some sort of dry run feature?

------
slobotron
Looks interesting, so far we've been using elasticmq[1] (SQS) and s3rver[2]
(S3) to do e2e integration testing on local and staging. We spin them up using
azk[3].

[1] [https://github.com/adamw/elasticmq](https://github.com/adamw/elasticmq)
[2] [https://github.com/jamhall/s3rver](https://github.com/jamhall/s3rver) [3]
[http://www.azk.io/](http://www.azk.io/)

------
justinlilly
Has anyone figured out how to create lambdas? I can't seem to create one via
the CLI because I don't have the correct IAM role setup?

~~~
happyslobro
Thank you for confirming the accuracy of the AWS Lambda test model.

------
nicholassmith
This looks excellent, I used moto to do some unit testing of a Python
application and was wondering if anything existed to make unit testing other
codebases easier. There's always the risk of diversion from AWS resources but
I think it's a small risk, generally AWS seems to stay fairly consistent for
legacy when new features are introduced.

------
foreigner
AWS supports an official DynamoDBLocal. I wish they supported local
implementations of their other services as well.

------
z3t4
I applaud everyone using such deep testing!

------
code_monkey_ed
Could this be used for learning AWS instead of starting the clock on the free
year of AWS?

~~~
mcrider
If you want to learn the ins and outs of AWS i would recommend using the real
thing. You can always create a new account when you have something in
production and want to take advantage of free-tier pricing.

------
TroubleTicket
Why are we calling this "local" and not a "private cloud"?

~~~
tuwtuwtuw
Did you read the page linked to?

Or: Because it's not a "private cloud".

