Hacker News new | past | comments | ask | show | jobs | submit login

Similar experience here.

Back in 2013 we had to evaluate existing cloud/VM platforms in order to replace the plain KVM/libvirt and support and enable the growth. oVirt was garbage (missing installation ISOs, randomly broken install process, cluster nodes not communicating etc.), OpenNebula buggy, OpenStack seemed to be quite hard to grasp, Hyper-V Windows only and VMWare expensive as hell (even now the TCO Calculator gives us 4000+ EUR/VM - this must be joke).

We run several VMs with docker and our apps, manage dedicated servers and their networks (VLANs as provider networks in OpenStack), provide IPSec VPNs to for tenants, run Kubernetes clusters on OpenStack. We also manage several dedicated servers that are not managed by OpenStack for historical reasons and hopefully will migrate them to the cloud.

If OpenStack makes our heads hurt it is due to lack of documented design patterns. After the years, documentation is good for the initial deployment and IMHO for developers (either API consumers or contributors), but no so much for network engineers or system architects.

Some design choices are pretty crucial upfront and you will pay the price to change the design. We ended up modifying database records several times and then slowly rolled the changes to the compute nodes. Recently the OpenVSwitch flow tables were populated undeterministically after some network changes and we had to inspect sources and even then did not understand, why do we experience the issues.

But never did we encounter the stability issues, that were not caused by wild actions of an administrator.

VMWare is in the thousands of dollars per node. It's not a joke, it's very useful software with no competition, worth it.

So I guess the typical node is the beefiest you could get in order to justify the license price?

I had this conversation with a colleague who offers VMWare managed Windows VMs for his clients and he told me similar thing, but on the other hand, he was shocked of the prices of our hardware (approx. $6k per server) and was seriously considering migrating to the OpenStack.

The beefiest you can. Exactly.

A 2U or 4U server is $10k to $20k. You're going to fit all you can in the box, including a minimum of 512GB of memory.

It's not just about license costs though, it's about hardware costs and capacity management. You want to have as few servers as possible for a given capacity, it's easier to manage and cheaper. You must have VmWare to abstract the hardware, a bit like AWS. You work with virtual machines and it packs them on the hosts.

Last I bought it but that was a few years ago. VmWare was $5000 per node for the full package. There was a free edition limited to about 100 GB of memory, but without cluster management and live transfer of running VM (vMotion).

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact