
AWS CodeBuild – Build and test code with continuous scaling - Trisell
https://aws.amazon.com/codebuild/
======
St-Clock
I'm trying to find if they offer or plan to offer these features:

1\. Caching. CircleCI and Travis cache intermediate build artifacts (e.g.,
virtualenv in python) to reduce build time.

2\. Github pull request integration (red cross on pull requests if the build
fails).

3\. Chat integration. Sending a message to slack or hipchat when the build
fails.

4\. SSH into build container. Very handy for rare but difficult to locally
reproduce build bugs.

Interesting offer though. We found that we would pay less than 5$ a month for
our build needs and they would run concurrently.

~~~
donatj
The Github PR integration is the most important part for me. It's a necessity
for us to be able to use it. We use our own install of Drone on a fairly
sizable EC2 instance and not having to manage that anymore would be wonderful.

~~~
awinder
Github does expose an API for this:
[https://developer.github.com/v3/repos/statuses/](https://developer.github.com/v3/repos/statuses/)

I mean integration is one less thing to build & manage, but you can sling
together support in a pinch

------
debaserab2
How many different CI workflow tools are on AWS these days? How do I know
which one to pick? I really wish Amazon would spend some time building
comparison guides for their services. Each one feels very silo'd off from each
other and the crossover in functionality seems very high.

~~~
vacri
Cloudformation is for when you have a team of ops or a workflow where you have
a lot of repeated resource recreation; Beanstalk is for when you have no ops,
and you're happy for someone else to handle the environment; CodeDeploy is
just yuck; and don't know about this one yet.

    
    
        Cloudformation: infrastructure-as-code (but has sharp edges) (doesn't touch your app/code directly)
        Opsworks: wizard-style 'drop your app here' kind of thing (less flexibility and control)
        Beanstalk: a simpler version of OpsWorks? (never tried it)
        CodeDeploy: install an agent, it pulls code/artifacts (janky workflow)
        CodeBuild: no idea, just been released
        Just Using The Web Console: convenient, but manual process (labour-intensive, prone to manual errors)
    

On the CI thing - from my experience at one of the places I work, there are a
thousand CI systems out there, but very few CD systems. Pretty much anything
can schedule and track builds, but few things schedule and track deploys
(which gets suprisingly tricky suprisingly quickly). CD is the 'last mile'...

EDIT: missed some that I've never looked at. It is getting crazy...

    
    
        CodeCommit: looks like it might be a 'github'?
        CodePipeline: No idea. Perhaps a spruced-up version of CodeDeploy?

~~~
debaserab2
I appreciate the breakdown. It would be awesome if there was someone with the
know-how to go more in depth on each of these.

I recently inherited an app that uses OpsWorks. The deploy process is actually
really nice, but I notice that it doesn't receive a lot of updates from AWS.
Since OpsWorks came out when Chef was hot, and now Chef seems to be less
popular than Ansible and/or docker. I wonder what the future holds for me if
Chef continues to decline in popularity.

------
luhn
I'm excited about what this means for Lambda. Building Lambda packages has
always been a pain because much of the time it needs to be built on Amazon
Linux to work, meaning most (all?) CI SaaS solutions are out the window. I
assume CodeBuild will run Amazon Linux and CodePipeline integrates with
Lambda, so this should make setting up continuous deployment with Lambda much
easier.

~~~
timoth
AWS now provides a container image for Amazon Linux which might also be an
option, depending on the CI SaaS: [https://aws.amazon.com/blogs/aws/new-
amazon-linux-container-...](https://aws.amazon.com/blogs/aws/new-amazon-linux-
container-image-for-cloud-and-on-premises-workloads/)

------
saurik
When I saw this I got super excited because I thought it was going to be "per-
minute low-latency distcc hosts" which was probably set up by having a ton of
common toolchains ready to go at all times, and my big questions were "did
they bother to support MinGW (easy)" and "do they have a reasonable story for
iOS (though I can't imagine they have armv6 working right)", but while they
got my hopes up when they said one could even upload custom toolchains I
realized that didn't make much sense and a few paragraphs later I got the
disappointing news that this is just some extremely thin wrapper over ECS that
is limited to doing builds on single computers which max out at 8 vCPU
(why?!). I mean, I guess this makes CI slightly more accessible to some
people, but it isn't anything terribly exciting and is mostly going to help
people with extremely small projects: this isn't going to scale up to the
kinds of builds where you'd expect a service billing itself on scalability to
be most valuable.

...Except for this part: _CloudBuild has per-minute billing_!! This is one of
the major complaints people have about EC2 (and all the services Amazon builds
over it), and is one of the major downsides of using it over Google's Compute
Engine. If you have any kind of task that can possibly be thought of as a
"build"\--one which can be expressed as a container of software configured to
access some external asset as input and which generates a concrete output
"artifact" (and maybe even not, right? to support some silly things people do
in their builds like "check out code from npm", you likely get network access,
and your build output could always be an empty file)--this now seems like a
depressingly hilarious way to trick Amazon's infrastructure into giving you
per-minute billing for random tasks which take less than 20 minutes to run
(important limit, as they are charging a 3x overhead vs the on demand price
for an equivalent instance: for 8 vCPU / 15 GB instance, a c4.2xlarge costs
$0.419 per hour and a build.general1.large costs $.02 per minute, which would
be $1.20 per hour).

In other words: I will argue that this service really can and maybe should
just be looked at as a different pricing model for ECS, to support any "small"
task (not just building code): if it takes less than 20 minutes and doesn't
require a massive computer, CodeBuild is not only cheaper but probably easier
to use (as it already models the problem in terms of a task queue, so you
don't have to do that part either).

------
nzoschke
This is really exciting. The lack of a build service has been apparent for a
while: [https://convox.com/blog/aws-missing-build-
service/](https://convox.com/blog/aws-missing-build-service/)

I wrote up how we plan to use this in the Convox platform here:
[https://convox.com/blog/codebuild/](https://convox.com/blog/codebuild/)

Practically speaking, we're working through PCI compliance. Getting builds off
of production services and root-enabled docker daemons is a huge win.

------
AtticusTheGreat
As someone who has had to shop around and try out a bunch of continuous
integration services (Travis, CircleCI, Snap, Solano, to name a few), this
looks pretty interesting! We've stayed away from managing our own CI
infrastructure but this could be a good (and cheaper) solution. It doesn't
seem to actually be available yet, but it'll be worth a look.

------
redgc
As it sounds like it's running inside Docker itself, I'd like to know if this
supports "Docker in Docker". My requirement is not strictly DinD however I run
multiple containers during CI (Postgres, node, test containers, etc). Possible
via different approaches in CircleCI, Shippable and SemaphoreCI. I don't
actually build any containers to save.

~~~
ethangj
This is actually how Codeship's Docker infrastructure works by default, builds
up containers and executes all commands in them natively. DinD still possibly
but somewhat yet useful with that approach.

------
msie
What's frustrating is if you go to the aws reinvent page they tell you a lot
except the dates of the conference!!!
[https://reinvent.awsevents.com/](https://reinvent.awsevents.com/) I was
wondering why all these aws announcements were here.

~~~
brazzledazzle
They used to have the date but maybe they pulled it when the conference sold
out.

------
brazzledazzle
At any time someone can show up to eat your lunch but if you're developing
software for operations or developers it seems like a scary space to be in.
AWS has a huge advantage by being able to simply add a service to their
existing catalog.

~~~
TheRealWatson
I feel like none of the AWS developer offerings (the Code* services) are
compelling enough.

Kind of like a lot of Microsoft tools, they are all lacking but, hey, the have
deep integration with each other, so easier gluing.

------
bloomark
[https://aws.amazon.com/blogs/aws/aws-codebuild-fully-
managed...](https://aws.amazon.com/blogs/aws/aws-codebuild-fully-managed-
build-service/)

~~~
AtticusTheGreat
Hmm this blog post says it is available but there seems to be no way to get to
it and it is not showing up in the console.

~~~
_superposition_
the url works

[https://console.aws.amazon.com/codebuild](https://console.aws.amazon.com/codebuild)

------
Malkav
Gave it a try, seems to be kinda slow building docker images. An image that
take 2 minutes in my machine is taking around 10 minutes in CodeBuild. The
build seems to freeze a little between docker build steps :

\--- Example 1 ---

[Container] 2016/12/01 22:39:59 Step 9 : EXPOSE 3000

[Container] 2016/12/01 22:40:31 ---> Running in 1c6e3a4dbec8

[Container] 2016/12/01 22:40:45 ---> 602aa4bc97ac

\-----------------

\--- Example 2 ---

[Container] 2016/12/01 22:36:00 Step 4 : WORKDIR /src

[Container] 2016/12/01 22:36:32 ---> Running in 98800352e6c2

[Container] 2016/12/01 22:36:45 ---> b437afe2a1c5

\-----------------

Not sure if this caused by the fact its a docker inside docker implementation.

------
davecap1
This sounds a bit like BuildKite which has worked out pretty well for us! They
provide an elastic AWS CI environment that you run in your own AWS account,
and scales up/down as builds are queued up.

------
ianceicys
I really wish AWS CodeBuild supported .net\C# as a preconfigured environment.
Hopefully .net is not too far down on their priority list...right now we are
using Jenkins.

~~~
gtsteve
I'm pretty certain someone will make a Docker image for Core CLR apps soon.
There's one for Bitbucket Pipelines already.

------
STRML
Seems pretty nice. A few things I'd like to see:

1\. We should be able to configure this for a few/all branches (including PRs)
and have conditional build tasks based on branch.

2\. We need be able to access resources inside a VPC.

3\. Turnkey chat integrations would be nice, but it's not a big deal to just
curl.

4\. We need a way to execute actions on failure.

~~~
gsharma
> 4\. We need a way to execute actions on failure.

I am curious to know what kind of actions would they be other than notifying
via chat/email?

~~~
TomFrost
For one, hitting the Github API to put failed build markers against commits
and PRs. CodeBuild doesn't appear to have the same Github integration that
most other CIs do out of the box.

------
mesozoic
FAQ says this. Is there any examples on setting it up? It isn't obvious from
inside CodePipeline.

Yes. The CodePipeline Plugin for Jenkins can be used to integrate CodeBuild
into Jenkins jobs. The build jobs are sent to CodeBuild, eliminating the need
for provisioning and managing the Jenkins worker nodes.

------
btashton
Still no way to build iOS applications.

~~~
kt9
Hi you can build iOS applications using Distelli and your own build servers -
[https://www.distelli.com/docs/kb/using-your-own-build-
server](https://www.distelli.com/docs/kb/using-your-own-build-server)

disclaimer: I'm the founder at distelli

------
crb002
This is AWS FatLambda. You provide a full container image instead of just a
zip file.

------
callumjones
Does it support parallel builds?

~~~
bkendzior
You can set up parallelism manually, yes.

------
sidcool
How does this compare with Amazon CodeDeploy?

~~~
Timmy_C
It works side-by-side with CodeDeploy.

Using CodePipelines and CodeCommit you can create a workflow where a git
commit to a CodeCommit repo can get picked up by pipelines and sent to the
build service (i.e. CodeBuild or Jenkins). Then CodeBuild will push the
resulting artifact to S3. CodeDeploy (and Elastic Beanstalk, CloudFormation
and OpsWorks) can be configured to deploy the built artifacts to your
application fleet.

It's the last piece in AWS's solution for continuous deployment.

