Hacker News new | past | comments | ask | show | jobs | submit login
Launch HN: Dockup (YC W19) – On demand staging environments for dev teams
133 points by emilsoman on Mar 5, 2019 | hide | past | web | favorite | 51 comments
We're Emil and Yuva, co-founders of Dockup (https://getdockup.com). Dockup spins up on-demand environments so engineering teams can quickly test their code changes. When you open pull requests, Dockup creates disposable environments automatically and delivers URLs to your chatrooms; engineers don't have to wait their turn to manually deploy to a staging server for testing their code changes. Anyone in the team can then click on these URLs and preview features before merging pull requests. Because each environment has all the services in the tech stack, it lets you catch bugs which usually happen only in production.

My co-founder and I were colleagues at our previous job (for ~6 years) where we worked as consultants, mostly working on Rails, React and Elixir projects and had the luck to work closely with many engineering teams. We often saw that teams would slowly lose confidence in their code as their codebases and team sizes grew and code changes would take longer to ship. At a payments company that we worked for, things often worked fine in dev but broke in production and eventually they started having company wide meetings before teams could deploy anything to production. Wanting to help developers ship faster and with more confidence, and also to scratch the itch of writing something in Elixir, we started building a tool as a hobby project, which eventually turned into our product.

Honestly, at first I thought that if we could build it, anyone else could build it internally too and no company would pay money for this, until we actually started talking to companies which have done it. We learned that it usually takes a few months for a developer to build an internal tool that automates PR review deployments. Most engineers told us they don't like having to support this set up and keep it running after they've built it and moved on to solving other problems. We faced many problems on the way, for example - the need for pre-seeded prod like databases for testing features, being able to support architectural changes (for example, adding a message queue in the tech stack and testing it with Dockup) etc. We have now reached a place where we are able to onboard most of our customers without having to build custom features each time. We are excited to share what we have built with all of you! We are sure the HN community will have many knowledgeable engineers who have tried solving this problem and we're looking forward to hearing your thoughts.

If you want to try Dockup now, you can do it by running the Dockup agent on your servers. If you don't want to run your own servers, you can request access for the managed Dockup cluster by going to our pricing page and we'll roll out access in a couple of days. Pricing starts at $75 for small teams.

Thank you for reading!

> It spins up containers that simulate your tech stack, but it won't be an exact replica of your production infrastructure. For example, if you use a managed postgres DB in AWS in your prod environment, in a Dockup environment, you would use a postgres Docker container that's started using the official postgres image.

I don't quite understand this. Why wouldn't I want to spin up another postgres DB in AWS for testing?

I think that's what I'm a bit stuck on. Being able to spin up a test environment on demand is cool. But if I'm going through the effort to do that, why not go through the effort to do the same for a production environment? Then I can use the exact production configuration to run my tests.

Dockup does have support for this through a feature called "resource pools" which allows you to create external resources and use them in your dockup deployments. This is more involved than simply using the containers, but the support is there for those who think it's worth the effort. Thanks for asking!

This is how I approached it. With ECS on AWS I could just make staging, production, and test envs all identical by deploying a set of containers.

But Docker for development really still feels so annoying and in the way. So I've just been using a Python virtualenv and a postgres instance installed directly on my machine. I tried to make docker dev work but it just always felt in the way, between my dev tools and the environment. It's been working fine but I'm sure there's some who will hand wave and insist this is a problem.

As a manager of a remote team, I can see how this can be doubly useful to me! We already use Docker for development too but reviews still involve stopping what I'm doing to pull another developer's code to test and merge. I couldn't find any documentation for Dockup on your website. Is documentation available only if I'm signed in? I was just curious to see how it can be setup before signing up for an account.

Glad you find it useful! If you want to give it a try, you can sign up for the "Free" plan by heading to https://getdockup.com/pricing . There's no setup documentation because we don't require you to run any scripts or maintain a config file. You can simply define your tech stack using our UI and get started immediately. We do have documentation for the self-hosted plan, and you'll see it once you sign up.

Hi nagarjun. My company (https://tonic.ai) builds dev tools and we've noticed recently that companies with remote teams have been using our product for an unexpected use case. We are trying to investigate it further. Would you be willing to chat with me? If so, I'll drop you a way to get in touch and we can chat.

quick fyi, it does not compete with OP's product.

Just a heads up, on your Product page (https://tonic.ai/product) you have the same description under both "Point-and-click" and "Built-in Statistics."

Thanks. Our landing page is currently in a constant state of flux. I'll make sure that gets fixed. I'm a bit surprised but my previous reply actually generated a non insignificant amount of traffic to our site so we also opened up app.tonic.ai for anyone that wants to give the product a whirl.

I like it. Back in 2015 I built something similar[1] for my company and it helped us make the transition from time-bound releases with QA on mainline to per-PR QA and CI/CD.

[1] https://medium.com/@kevanahlquist/dockerui-at-bluestem-2890b...

This looks neat! Thanks for sharing.

How is it different from Heroku Review apps? (https://devcenter.heroku.com/articles/github-integration-rev...)

How do you feel you compare to the similar features in Gitlab’s product?

Yeah first thing I thought of was Gitlab environments / "review app" concept when reading this, we tried using Gitlab environments and ended up rolling our own on top of Kubernetes as it seems like an under-loved area of their product with a few warts.

On demand envs for teams is a incredibly liberating experience for people who have been tied down to a single or small number of environments forever, our devs have been very happy with it. There are some interesting problems on demand environments create that persistent environments don't have, like data seeding for your apps (as naive state is an env comes up blank), extracting secrets/keys/credes/certs from the env now that they are different everytime you deploy etc. We have patterns for these problems now and has really forced us to mature our process.

Hey there, product manager at GitLab for review apps here. I'd love to hear about the warts that you ran into that caused you to move to k8s. We're always working on improving the product and feedback like yours is really the most valuable thing for us.

Gitlab is great for someone already using it and is familiar with their configuration syntax.

Dockup has these:

- Seeding DBs with prod-like data is easy and works super fast so you don't have to wait around for a long time until you do a database dump/restore. We can do this in Dockup because we can maintain a pool of single-use DBs for your deployments.

- Deployment form to mix and match different versions of your apps (example: you can try how branch-foo of frontend and branch-bar of backend work together)

- Users who know how to write Dockerfiles can easily setup Dockup. Many of our users are project managers and QA and they find the UI easy to use.

This is great. Setting up proper staging environments has been on our procrastination list for a long time and as a small team, we haven't invested engineering hours for a real setup yet. If this simplifies the process so much that we barely need to think about it, I totally get the value.

That's exactly our goal - teams that don't have the time to invest in devops should be able to use Dockup and not have to think much about it. Thanks for the kind words, glad you see the value here!

I have been searching and waiting for someone to build this! Thank you so much! Currently reviewing code is a guessing game. "Does it actually do what it's supposed to do? I don't know but the code looks sound." I have only the code to reference and maybe screen shots if they're provided. I'm not going to spend the time to download the diff and deploy it myself just to see if the application is doing what it's supposed to.

Interesting proposal! I have a number of questions:

- How much configuration is still required by the end user to leverage Dockup? How would it be different from a Vagrant/Docker setup where you would define a Vagrantfile and Dockerfile?

- Why make pull request reviews with a development environment rather than a slimmed down production environment? Testing turnaround time?

- Do you guys anticipate on-premise deployments of Dockup?

Best of luck!

Thank you!

- Regarding configuration - you need Dockerfiles for your apps and also configure environment variables and ports in Dockup. We call this configuration a "blueprint", which once created will be the template for your automated deployments. Unlike Vagrant, Dockup runs your apps on the cloud and you get URLs in your slack channel when someone opens a PR.

- Our pull request reviews (we use Dockup while developing Dockup :D) actually use production environment. We simply use a postgres container as the database and turn off analytics in our Dockup environments.

- Yes we do get requests for on-premise installations and we do support that. In fact, when we started, we only supported on-premise and later made the SaaS version. If you want to just self-host the infra and use Dockup UI, you can do it right now by signing up for the self-hosted plan.

We have our own version of this -- we're fully remote, so it's an important thing to have. Basically we can deploy a full on-demand env with a single command. It's very useful.

(Really, you just launched exactly the same features what we have already internally. :) )

Good luck.

That's awesome! We would love to interview you to understand your experience, can you please drop me a mail - emil@getdockup.com?

I have this problem, we could have a lot of branches but only a certain number of environments e.g. dev, qa, stage etc.. So would this tool allow me to create environments dynamically so that I can test each branch standalone.

Yes, every branch gets its own environment.

Why are you not listed on this page? https://www.ycombinator.com/companies/?batch=w2019

Thanks for sharing the list! We should show up there soon now that we've made a public launch.

I always thought that the 'W' in YC batch numbers stood for 'winter', or isn't it? Or is 'W19' in the title a typo and should it be 'W18'?

It does. They are probably in the current batch, which would be W19.

I think that the batches are named based on when they're due to finish YC

Suppose my application is dependent on the subdomain like "customer1.wasdapp.com". Is your application able to spin up "customer1.wasdstaging.com"?

Yes, this is possible. By default Dockup would give a random subdomain to each deployment, but some of our customers wanted to use Dockup deployments for giving customer demos so we have added support for long lived deployments that use a specified subdomain.

Congrats on the launch! Any relation to Stream [0]? Your logo is very similar.

0: https://getstream.io/

Logo "designer" here. That's an uncanny, unfortunate resemblance. All those hours taking pictures of a paper boat to redraw it seem so wasteful now :'|

Thank you! We had no idea about Stream, yes the logo does look very similar haha

As it is relevant .. plugging-in my own startup, platform.sh We do this with another twist ... our per-branch ephemeral staging clusters contain a full snapshot of production data. We believe strongly in immutability so what you are running in production is precisely what you tested in QA... And because production cloning relies on copy-on-write primitive it is basically immediate (think more in the dozens of seconds rather than multiple of minutes).

It looks like I need to host my prod environment completely on your tool to make use of the staging feature. I will look more into this though, thanks for sharing. One difference is that in Dockup, we can boot up databases instantly with obfuscated prod like data and we do this because Dockup deployments are accessible by the entire team.

How does it compare to Heroku review apps?

Dockup is like Heroku review apps, but for the entire tech stack (each PR gets a dedicated environment with all the microservices in your stack). One benefit of using Dockup is that seeding DBs with prod-like data is easy and works super fast so you don't have to wait around for a long time until you do a pgdump/restore. We can do this in Dockup because we maintain a pool of single-use DBs for your deployments.

How are you different from Jenkins X?

yea this seems a lot like how I do things with Jenkins-x preview environments. Other parts of the stack are pulled in based on the latest version of the respective helm charts.

Dockup is like Jenkins X, but easy to setup for people who do not know Kubernetes or helm charts. Also we have features like single-use pre-seeded DBs and a deployment form for deploying a custom combination of branches, for example, development branches from two different repositories.

This seems like a great tool for companies that don't have time to invest in dev ops.

Might be useful for anyone who hasn't heard of Docker yet.

no offense but not really world changing tech from YC batches

I'm confused why I'd use this. My flow now is:

1. branch off of master

2. write a bunch of code that runs the risk of breaking stuff

3. deploy it to our already existing developer/staging environment

4. test it. if it is good, promote it to QA/UAT/CAT

does Dockup assume that companies don't already have the 3 main environments (dev/CAT/prod) set up?

Why would my service be worth anything in isolation? Usually when I deploy to staging, it is after I've already made e2e+unit+integration tests work locally, so I'm checking how live databases/load balancers/routers/etc. work in staging.

Step 3 in this flow is sometimes painful for big teams because developers have to wait in line if other developers are already using staging and may have to roll back databases after testing. Dockup gives the reviewer a chance to quickly test the feature without having to manually do deployments or start the app before merging the PR. Thanks for asking!

I still don't perfectly grasp what this tool does.

How can you deploy multiple staging environments?

I guess I kind of get it now that I think through it.

Say you need 10 services for staging to work. They all talk to the staging database...

Does this DockUp thing also allow you to configure to deploy a database so developers aren't fighting over data/migrations in staging?

It sounds like a containerized development environment, but it's really only as good as the configuration job that is done for it.

How do I test new load balancer rules? Gotta have all of the load balancer images + configuration files in.

Yea, I agree. Step 3 is not a trivial setup for early-stage startups. Glad to see devtools companies trying to tackle this.

At the company I worked at, we built something similar using Heroku free dynos :) So that you can tag a PR and it automatically created a staging app using free Heroku dyno, it even came with a free database. The setup works but also quite spotty.

This was a constant pain point for us as the engineering team grew. With a single staging environment, we could only have one changeset tested at a time. We built a tool like this internally and the engineers love it. When you have 20 feature going out in a week, it's not reasonable to have to wait for staging or qa to be available. We happened to call this 'review apps' but it's based on a similar principle. You can specify in the deployment tool which service you'd like to deploy as a review app, and a PR number.

This is incredibly useful with many interconnected codebases have related changes (I know, bad SoA. best practices are hard). Then I can point review-3918.SERVICENAME.service.XXX.stag's connection string to review-1111.SERVICENAME.service.XXX.stag to ensure that everything is working as expected.

We would love to interview you to understand your experience, can you please drop me a mail here: emil@getdockup.com? Btw, we had the exact same requirement too - be able to test how a related change in multiple apps would affect the whole system. Check out our deployment form that helps us do this now: https://user-images.githubusercontent.com/1707078/53836663-6...

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact