Hacker News new | past | comments | ask | show | jobs | submit login

Why should they release it?



This site was mostly created as a joke after I watched a couple of @aspyker's great presentations about Titus and realized that we have a huge amount of exciting work to do to get our own container platform on AWS to the same level.

Netflix should do what makes sense for them. I understand that doing a good job of releasing a very complex piece of software as open source is hard and takes a lot of effort.


Could be a more useful site if it had been on a domain like http://www.isitopensourceyet.org/netflix/titus or similar. (note domain does not exists. At the time of writing...)


I'll happily create a redirect once you build that for us. :)


I'm curious, why are you rolling your own?


Not rolling our own exactly: running on ECS now and busy evaluating Kubernetes. These are still quite blunt instruments though.

At some point, as a company grows, it's not just about running some containers. You want batch jobs, complex placement constraints, scheduled tasks, etc., etc.

If you haven't watched through https://youtu.be/4OLlKGT7aVQ, I can recommend it. aspyker does a great job of enumerating many of their complex use-cases.

The Titus team at Netflix is building this on top of EC2/Mesos and ECS.

The rest of us still need to do a bit of work to find/adapt/build our own.


> The rest of us still need to do a bit of work to find/adapt/build our own.

I usually recommend people just take an existing PaaS and run with it. I'm wildly biased in this regard because I work on one as my day job.

Also because rolling your own is hard. Really really hard. We have literally hundreds of engineers in dozens of teams, and that doesn't count the engineers our partners have assigned. We've been cranking for years. It's just a lot of hard engineering for stuff you don't see until it arrives, and arrives unpleasantly.

Kubernetes makes it a lot easier, especially if you need lots of flexibility, but it still leaves you to assemble a lot of parts and management is left very much as an exercise for the reader.

If you build on Kubernetes, may I recommend Kubo[0] to manage it? We built it with Google, but because it's based on BOSH you can deploy to AWS, GCP, Azure, OpenStack, vSphere and even bare metal with RackHD. BOSH by itself is a massive force multiplier for operations.

[0] https://github.com/pivotal-cf-experimental/kubo-deployment


Note: I'm the one in the video, an original engineer on Titus, and now the product/hiring manager of the Titus team at Netflix. Also, I do not actively monitor hacker news, so it is unlikely that I'll follow-on to this conversation (@aspyker on Twitter is an easier way to reach me).

Is Titus Open Source Yet? No.

Why?

Reason #1: When we created Titus, we married the back end of a container execution engine (Titan) with the front end of a scheduling system for stream processing (Mantis). We have a goal at the time to stabilize existing usage at Netflix. We didn't have a goal of removing all code from both sides that wasn't needed after the marriage. We wanted to clean this up before releasing in open source as we would have to spend more time telling people what to ignore in the codebase that is no longer used in Titus. We also didn't have time to create proper interfaces that separate Titus from other important operational systems at Netflix. We have recently been re-implementing the engine to have clean interfaces with minimal code.

Reason #2: When Titus was created it had a user API that grew during initial usage in batch applications. Given the number of clients using this API, it was hard to change it without impacting users. Compounding this, we added service application support without thinking much about making the API well designed. We have recently been working on a new API that was designed to handle both use cases. This API is currently being adjusted based on internal feedback.

Reason #3: The space of container management platforms is truly a crowded space. There are many other great options out there. When we eventually open source Titus, we hope it is clear why Titus is unique in this space specifically for "all-in" Amazon EC2 customers, for NetflixOSS cloud platform adopters (those who use Spinnaker, Eureka, etc.), and supporting the level of complexity we do (VPC IP per container, GPU's, complex fleet and capacity management, etc.). Finally, showing how at scale production battle hardened Titus is. We realize even with these differences, we'll spend a consider amount of time justifying these points to those who look at the space more generally. We would prefer to not spend any time "marketing" our differences.

Reason #4: Our main focus at Netflix is to service Netflix developers providing the easiest to use and most reliable service. Open source is something that is valuable to us with regards to potential external contributions as well hiring and retention of the best engineering talent in the world. This value, while significant, isn't as important as providing our internal Netflix value. As a part time responsibility I help open source at Netflix. Therefore, I am also aware that doing open source well (like NetflixOSS Spinnaker as a shining example) requires more investment for teams than posting the code to github. Our team is small today and we do not believe we can make Titus open source as great as we'd want it to be. That said, we're growing (did I mention I'm the hiring manager - https://jobs.netflix.com/jobs/862432) every day and can see the value of changing this equation.

I am truly honored that someone would great a whole website to ask for Titus. https://www.istitusopensourceyet.com/ made me laugh last night and decide to write this response today. We appreciate your excitement for our container management technology and Netflix open source. As we continue to evaluate this space, we'd love to find other AWS users who are also using the Netflix cloud platform and Spinnaker to consider partnership.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: