I like the fact that they are a practical company, using Go when needed where "static dependency compilation and fast start-up are more important". I wonder if the ClojureScript's annoucement on integration of NodeJS modules  changes that? Also, Lumo  is definitely a move in the right direction for this, addressing the slow start-up times for Clojure/ClojureScript, making it suitable for shell scripts and CLI binaries.
> Having a lingua franca also helps reduce overhead when engineers want to move between layers of the stack.
The way I see it, Clojure allows you to use a single language and syntax from the super heavy backend stuffs, to the front end and now to small, fast CLI tools and scripts. A candidate for the "Business English" of the technical world as it were.
Off the top of my head:
Naughty Dog's games (Crash Bandicoot, Jak and Daxter, Uncharted) are powered by their own Lisp implementations. Google for "GOOL" and "GOAL".
Grammarly uses Common Lisp.
Oh, and if you're searching for air travel connections, your query most likely goes through a huge Common Lisp system (search for ITA Software).
They also mention static binaries which Node does not solve.
Agreed it doesn't help with static dependencies, which is why I didn't mention it ;)
Many of us who actually run Clojure in production have been bit by this, and Clojurescript has been fast enough to alleviate the issue. Not a straw man, a community meme.
Of course, that was after trying to get shippable to work.
This doesn't seem like glowing praise to me? If I'm paying 50$ per month for a container, I want it to be faster than a laptop...
Maybe I am doing it wrong but do people create and destroy a container thingy every time they run a test?
I agree with you though. I am no expert by any means I'd expect Circle CI to be faster than a Macbook at building things like Google Chrome from scratch. But if the macbook already has a head start, then coming a close second is acceptable I suppose?
To be honest, I have no idea what I am talking about and hoping to learn more. My experience with CI is mostly with hobby projects with Gitlab CI and Travis CI. I've only ever "used" Atlassian Bamboo at work if you can call me pushing code to trunk/origin "using Bamboo".
I assume the disk is pretty much always the bottleneck? I mean even with system CTL, how much memory can a rest API flask web app take right?
That said, you don't have to use Docker and for a small project with no dependencies or no system level dependencies, I think it's overkill personally, so I'd just test outside of Docker or use the new base images from Circle.
I create and destroy a container every time I:
* Run a test
* Start a command line environment for debugging (Ruby mostly, so usually "pry" with all my apps libraries loaded)
* Build anything (e.g. update the installed gems via bundler for Ruby)
* Run any scripts that are part of the app.
It means I can be sure I always run things in precisely the environment the apps will run in, including the right interpreter or compiler, right dependencies etc. - nothing "leaks" from my laptop. Nothing "leaks" to my laptop. I can trivially test with multiple different interpreter/compilers etc. without convoluted "environment management" tools that tend to be language specific. Bugs in my script are also reasonably well contained (though I won't trust Docker for isolation as sole protection against hackers, I trust it for isolation against my own stupid mistakes most of the time).
Bind mounts of the source directory ensures I can do live reloading of code etc. when suitable, so I don't need to restart those containers all the time.
Typically starting the Docker container is one of the fastest parts of the above - often my makefile will not just start a Docker container, but for simplicity run the build whenever I do anything (like run the test suite) to make sure I run it in the newest version of the container, and thanks to extensive caching that too rarely adds more than a few seconds.
In a CI setup like CircleCI, though, the "cost" in time of this when building remotely is pulling the relevant source images from their registry, and the images can be fairly big, especially if you're not specifically going to some effort to keep it down in size.
And I'm typically not sitting there waiting for them to finish. All I care about is that they eventually do and the code deploys. Just don't take forever.
I also understand that the company is going to be serving many other customers at the same time.
And, no, the first server is free. The second server is $50.
- I'm not sure if you contacted us to improve build times but I'd love to jump on a call and discuss your experience. This is definitely not the typical feedback we get so would like to dig in deeper and improve our handling of your scenario. email@example.com
- the first build server on circleci is only free up to 1500 build mins per month.
The fastest Macbook available right now has about the same CPU power, and comes with less RAM. So no, the Macbook isn't faster than what you could get, but it makes sense if rented CI infrastructure isn't faster for that price.
First provider I checked has a multiple server offerings with quadcore i7-6700 and 64Gb DDR4 RAM for 39€/month .
It's slightly better now but I've done iOS+Android mobile projects before where CI for Android has been easy to set up and CI for iOS has been a complete nightmare.
Don't CI your apps, CI your libraries.
> Don't CI your apps, CI your libraries.
That's not scalable for complicated apps that you can write automated UI tests for though.
To me, what you described is an app that was not designed to be easy to test component-by-component & will have a high overhead on maintenance. Apple’s restriction on virtualising MacOS seems unrelated to how the app was architected, so it feels unfair to expect Apple to alter their position to better support something they weren’t responsible for.
We have complex commercial apps with fully automated user interface tests (e.g. it'll test that you can enter your username + password, actually login and you'll be able to see content). You could test all the individual components as much as you want but you're still going to get bugs at the UI layer that you can automatically test for.
We do the same for Android and CI for that is so much simpler.
> Apple’s restriction on virtualising MacOS seems unrelated to how the app was architected, so it feels unfair to expect Apple to alter their position to better support something they weren’t responsible for.
You'd think they'd want to support workflows that led to better quality apps being created. Surely it's their responsibility to support developers?
Surely you can prove the correctness with integration tests of your modules, and then your acceptance tests can be simple 'did the thing not show an error when we clicked on the thing, and it went to the right page'? We have 'simulate the world' 'acceptance' tests at work and they're terribly flaky, and I look back at the layered approach to testing we used in a previous job where the go-live test was to hit the landing page & search results page & check that the status code was 200.
To be fair, the kind of tests I'm proposing won't catch layout issues. But, neither will the tests you're using.
What made your UI tests so flaky? As long as the code for them isn't much beyond "enter text into text input X, click this button, you should be able to see this text" they're not too bad to write but the initial setup can be tiring.
> To be fair, the kind of tests I'm proposing won't catch layout issues.
Screenshot comparison based tools can be amazing for this as long as the way your app looks is fairly stable.
Automated testing is always a tradeoff with effort to setup vs the time you save (I'm against e.g. TDD for everything) but for complex apps and big teams it eventually pays off.
Overall we are happy with it. I don't think CircleCI 2.0 is available yet for iOS, though, so nothing new to discuss yet.
On the topic of QA testing/automation, it really kills me that Apple servers won't sign older iOS releases which means QA teams need to protect stock devices from accidental upgrades. The AWS Device Farm iOS devices are all jailbroken.
> (iii) to install, use and run up to two (2) additional copies or instances of the Apple Software
within virtual operating system environments on each Mac Computer you own or control that is
already running the Apple Software, for purposes of: (a) software development; (b) testing during
software development; (c) using macOS Server; or (d) personal, non-commercial use.
[ed: i guess you mean running os x server is an alternative to "non commercial use" - I agree with that reading. The option for nc use seems like a nod to not completely make experimentation and creative development/research entirely illegal.]
If you're using GitHub or Bitbucket, you can't use GitLab CI as they only support their own product. So in that case, CircleCI will be the best choice.
So whether or not you want to use GitHub or GitLab is a different discussion. :)
note: There is a hack to get GitLab CI to work with other VCS hosts but that requires mirroring everything over to GitLab which isn't ideal.
disclaimer: I work for CircleCI
Since 2.0 just launched and they have solid example projects across languages, it's a good time to give it a shot. It'll be easier if you've never used the 1.0 version of the product IMO.
When you get it going it blows everything else away in value for money. We routinely have hundreds of containers on-the-go and as many concurrent builds as we need; this all runs on a large AWS spot instance at a cost of about $50 a month. Added bonus: our docker cache is always available.
Buildkite provide a CloudFormation stack, but we just opted to run the agents as containers via ECS to make the setup quicker and managing them easier.
Another similar option we tried is AWS CodeBuild which has per-minute billing and provisions the build machines for you. However, it's very bare-bones and because you always get a from-scratch instance for each build you have to distribute your docker cache which is not ideal.
We actually use a combination of on-demand and spot container instances in production to keep costs down; we have some logic to provision more on-demand instances in case of multiple spot outages.
Other that that, great job and nice write up.