
VMware vs. bhyve Performance Comparison - adamnemecek
https://b3n.org/vmware-vs-bhyve-performance-comparison/
======
blibble
VMWare's EULAs prohibit publishing of benchmarks without approval, so this is
interesting to see...

[https://www.vmware.com/pdf/benchmarking_approval_process.pdf](https://www.vmware.com/pdf/benchmarking_approval_process.pdf)

~~~
Drakim
I have a hard time understanding the reasoning for this. What argument do they
have as to why people shouldn't know about the performance?

~~~
wila
The reasoning is that it is so easy to make mistakes in performance testing
and then draw the wrong conclusion.

As somebody else mentions, this is testing NFSv3 in the VMware setup against
local disk in bhyve. While still interesting, it certainly is not a 1 on 1
comparison.

edit: to expand a bit on that.

\- the VMware storage appliance gets 24GB, the bhyve config isn't mentioned,
but the host OS can access 32 GB.

\- as he does mention his test load fits in the ARC, again interesting, but
normally you would test real-world loads where your workload does not
completely fit in the cache.

~~~
Drakim
Sure, I mean, but, anybody can say that. "We'd rather not you publish your
review of our product, it's so easy for the reviewer to make a mistake and say
something negative about our product. So, you are not allowed to publish any
examination of our product."

~~~
wila
You ARE allowed to post test results, but they like to review your tests to
make sure you didn't make any obvious mistakes.

Quote from the .pdf file higher up in this thread: <quote> You may use the
Software to conduct internal performance testing and benchmarking studies, the
results of which you (and not unauthorized third parties) may publish or
publicly disseminate; provided that VMware has reviewed and approved of the
methodology, assumptions and other parameters of the study. Please contact
VMware at benchmark@vmware.com to request such review. </ quote>

For example here's a review from storagereview.com on VSAN

[http://www.storagereview.com/vmware_virtual_san_review_overv...](http://www.storagereview.com/vmware_virtual_san_review_overview_and_configuration)

~~~
rodgerd
"You may publish results which make us look good."

~~~
wila
Don't think so if your testing has obvious flaws.

~~~
Drakim
There is clearly a conflict of interest if the producer is the one who gets to
decide if your testing has flaws or not. There would be nothing but their own
integrity preventing them from simply labeling any test with poor results as
being "flawed".

------
throwaway7767
So, the benchmark shows that NFS is slower than direct disk access. Is anyone
surprised?

The bit about VMware and bHyve is irrelevant here. If you wanted to compare
the hypervisors performance, you would do it by using a local disk in VMware,
or using NFS disks from bHyve. As it is, it looks like someone just really
wanted an opportunity to advertise bHyve.

Disclaimer: I use bHyve myself, and I think it's pretty good for what it does.

~~~
geerlingguy
...but also much faster than normal/native shared folders:
[http://www.midwesternmac.com/blogs/jeff-geerling/vagrant-
vmw...](http://www.midwesternmac.com/blogs/jeff-geerling/vagrant-vmware-7-vs-
virtualbox-5-benchmarks)

NFS can be useful in many scenarios, but using native FS (or using rsync to
sync instead of mount a shared folder) is always going to win that battle.

------
therealmarv
Very interesting. Hope to see OS X xhyve to get more adoption from tools
[https://github.com/mist64/xhyve](https://github.com/mist64/xhyve)

------
KaiserPro
I would suggest that this is apples to oranges.

When you have VMware vcenter, the power is not in the inherent speed (its not
the fastest) its the simplicity of creating a cluster.

If you have a network file store, its almost always going to be slower than
local storage (assuming using the same hardware) thats not the point. Using a
network file system/ block storage allows you to have HA disks. (or Load
balancing, or much larger storage, or wider performance)

With pNFS you can have Active-Active with _n_ servers (assuming you're using
Netapp(yes they are still relevant), or GPFS)

As for vcenter, creating a HA cluster with ten hosts takes about 3 hours from
scratch (assuming you install all ten servers at once.) Thats using the
default tools, and the only thing that's scripted would be the install. (also
using the vcenter appliance, not the shitty windows thing)

KVM whilst much faster is so very much harder to set up, monitor and operate.
Yes you can use config management, but with vmware, you don't _need_ it. (
_cough_ , ignoring machine profiles)

~~~
dchest
_the power is not in the inherent speed (its not the fastest)_

You compared the speed yourself in the sentence immediately following "apples
to oranges", so I assume it's actually "apples to apples", but your point is
that VMware is better _despite_ being slower.

------
Nux
I'd like to also see Linux KVM vs ESXi vs Bhyve benchmarks, but excellent
article! Good job pissing all over their EULA! :-)

Arguably where VMWare shines is management tools and so on, not exactly
performance - this is kind of widely known, anecdotally at least and probably
the number one reason they forbid benchmarks in the EULA. They know they're
going to suck. :-)

Big corps with big money don't care that much about performance, they're more
interested in ease of use, interoperability, SLAs, support, integration with
other enterprise crap etc.

~~~
openforce
At VMWare, I was working on ESX performance for a while, and I can say that
performance is taken very seriously. Major features have been pushed back to
the drawing board when the performance was not up to the mark. The handle on
performance of ESX is tight.

Also, If I may say so, ESXi is one of the most well written pieces of system
software to date.

~~~
nullrouted
As a former VMware employee I would agree with your sentiment, excellent
hypervisor.

------
geerlingguy
Ironic that I _just_ posted VMware vs VirtualBox performance comparison
earlier this morning: [http://www.midwesternmac.com/blogs/jeff-
geerling/vagrant-vmw...](http://www.midwesternmac.com/blogs/jeff-
geerling/vagrant-vmware-7-vs-virtualbox-5-benchmarks)

~~~
nullrouted
That was a pretty good benchmark article, thanks!

------
bechampion
Very interesting.

