
Proxmox VE 6.2 - lysp
https://www.proxmox.com/en/news/press-releases/proxmox-ve-6-2
======
gbuk2013
Proxmox is awesome - VMs and containers with an API that just works. Network
file system with a UI that just works.

We once built an internal system to make network solution demos in it - a VM
image running Proxmox that would be configured with any number of other VMs
and containers all inside of that image, so you could literally take the while
demo environment with you on a USB stick when you presented it, work on it
while on a plane etc.

I also built a mini cloud on a Proxmox cluster using Ceph that would
instantiate these images from a template using a web UI - a bit of Proxmox
recursion fun.

That was a cool project to work on. :)

------
tlamponi
Community forum thread open for discussion:
[https://forum.proxmox.com/threads/proxmox-
ve-6-2-released.69...](https://forum.proxmox.com/threads/proxmox-
ve-6-2-released.69647/)

------
icefo
I love Proxmox but the absence of incremental backups is very annoying. If you
have a large vm whose contents don't change much your options are limited as
storage (edited: and bandwidth) is not infinite.

I also don't want to use aryufan patches to enable incremental backup. I feel
like they could probably partner with duplicacy to ship that feature.
Duplicacy already released a paid version specifically designed to backup
VMware hosts so it should be possible to adapt it to backup Proxmox hosts.

(Thank you all for your comments ! They gave me some ideas)

~~~
beagle3
Use a CoW/snapshotting backing file system for your backup (btrfs/zfs), or one
of the modern de-duplicating backup systems (bup/borg/restic), and the space
taken will be proportional to the changed data, while still providing random-
access to any snapshot -- Not much more you can expect in terms of space
usage.

snapshotting filesystems will be much more efficient in terms of time/effort
compared to de-duplicating later, but other than that, all of these solutions
are mostly equivalent.

~~~
icefo
I don't argue that it would work but when you do that you're still sending all
the data over some data link.

For example if I have 1.5TB of VM and only 10-20Gb changed today I'm still
sending everything to a remote storage. Snapshotting locally and sending the
zfs volume seems to be a more efficient solution

~~~
beagle3
That's not at all true for bup and borg, almost certainly not true for restic
(Haven't used it myself and can't tell for sure).

bup and borg (and likely restic) keep local hash summaries that are
synchronized with the repository if it is remote. They would still have to
_read_ the entire 1.5TB VMS if every vm file has changed (there are no
facilities on a regular file system that they can use to figure out which
parts changed), but they WILL use those local hashes to find out parts
actually changed, and only send those changes to the remote storage. The local
hashes typically occupy 0.5% of the data size (for bup, by default: 20 byte
SHA1 per 8192 bytes on average;

You can change the chunking parameters for bup/borg/restic for a tradeoff
between cache size and granularity of change detected; for VMs it might make
more sense to e.g. have 256KB chunks in which case you'll have 0.01% local
storage overhead, and likely still negligible network transfer overhead.

> I don't argue that it would work but when you do that you're still sending
> all the data over some data link.

That's only true if you include e.g. the SATA link in that list; To avoid
that, you need a filesystem that effectively tracks change ("damage") regions,
or VM storage that uses lots of underlying files so you can track it on a
regular file system (parallels on the mac used to do that so time machine
backups are efficient). no way around it.

~~~
toomuchtodo
> there are no facilities on a regular file system that they can use to figure
> out which parts changed

Doesn't ZFS do block level checksums that could be used for this, and backup
tooling could use block pointer metadata for this purpopse?

~~~
beagle3
Yes, I mentioned that you need a file system that can do that (afaik, zfs,
btrfs on Linux, and hammer2 on dragonfly) Or that you’ll need to read it all.

------
atomicnumber3
I've been using proxmox in my homelab for ~2y, and was doing everything bare
metal for the ~3y prior to that (which is when I started my homelab).

At this point, I'd never use anything else. Proxmox "just works" in the best
sense of the phrase. Never had uptime issues or anything (knock on wood). The
only bare-metal install at this point is my plex server, which has a special
level of importance in my close and extended family, so I baby it a little
bit. A lotta bit.

I use it for a lot of things - a windows VM for running the mega client (so
it's always on + isn't on a personal machine to snoop around), a postgres
server that I use for the dev/qa versions of my personal stuff, a
jenkins+wekan server, a vm just for minecraft servers, etc etc. It's so easy
to just spin up a new VM when I need a clean slate for something random.

If I at some point actually start my own business, we'd be a proxmox shop for
any non-cloud infra. And the cost-effectiveness of even a handful of 1-gen-old
Dell servers on-prem (for things that don't need super high availability) is
really hard to overstate when your budget isn't attached to VC funding. I
would not run prod infra on-prem at this point, though: DigitalOcean makes the
cloud far too affordable for that. But I'd probably have all CI, data
analysis, etc etc local for a good long while.

One gripe I have is their subscription/license nag. It's annoying and tacky. I
would gripe less if they had an option that made it feasible for me, a guy
running it in his basement with 0 profit motive, to buy a license. Right now,
it'd be (~$85 * 2 sockets * 4 servers) for me. Sorry, that's not happening, I
could actually use DigitalOcean cheaper than that (and maybe even AWS). And it
would be a huge % of my house's tech-related OpEx (compare that to ~80/yr for
domains, $5/mo for my one droplet, $100/mo for internet, etc). If I had to
name a price, I'd say $25/yr would be about the most I'd drop (as a guy with a
basement server room, of course). Contrast this with Plex which I got a
lifetime license for for like 150 bucks or something, and plex doesn't even
really have an alternative. I could live without proxmox (libvirt is really
annoying, but not _that_ annoying), but I couldn't live without plex.

The pricing seems fine for businesses, though. I'd argue it's probably good
RoI.

~~~
lysp
> One gripe I have is their subscription/license nag. It's annoying and tacky

root@pve01:~# cd /etc/cron.daily/

root@pve01:/etc/cron.daily# cat proxmox-nosub

#!/bin/sh sed -i "s|if (data.status !== 'Active')|if (false \&\& data.status
!== 'Active')|g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js

I put it in daily in case I forget to run it during an update.

------
unixhero
Great hypervisor solution! Really excited about this release.

------
polskibus
Anyone using it with Kubernetes and Ceph in production? What are the
advantages to other open source? What's worse in Proxmox than in Hyper-V or
VMSphere?

~~~
tyingq
_" What are the advantages to other open source?"_

I don't think there is another notable, proven, open source clustered manager
of both actual VMs and containers. OpenNebula might be the closest:
[https://opennebula.io/get-your-hands-
on-v-5-8-edge/](https://opennebula.io/get-your-hands-on-v-5-8-edge/)

LXD is similar to proxmox if you only need containers, and pretty slick. They
use their own distributed version of sqlite to propagate state, so it's much
simpler than proxmox.

~~~
ymse
> _I don 't think there is another notable, proven, open source clustered
> manager of both actual VMs and containers._

Googles Ganeti is strikingly similar to Proxmox, to the point where I suspect
Proxmox started as a reimplementation:

[http://www.ganeti.org/](http://www.ganeti.org/) (developed at
[https://github.com/ganeti/ganeti](https://github.com/ganeti/ganeti))

It does not come with a GUI, though there are a few third party ones. It has
API endpoints for everything, a fully featured CLI client, and is extremely
stable.

Googles marketing department does not want you to know this, but Ganeti runs
most of its internal infrastructure (not public facing stuff).

No affiliation, just a happy user.

~~~
tlamponi
Proxmox VE dev here. Honestly, heard of Ganeti the first time today.

The basis of Proxmox VE is the real time replicated clustered configuration
file system: [https://pve.proxmox.com/pve-docs/chapter-
pmxcfs.html](https://pve.proxmox.com/pve-docs/chapter-pmxcfs.html)

I do not see anything similar for ganeti, also not much similar in general
besides that it manages VMs?

~~~
ymse
Hi, thanks for the link, very interesting.

Indeed the similarity with Ganeti is only on the surface: being a scalable
infrastructure management tool, DRBD as a first-class supported disk backend,
etc.

The cluster replication and consensus model is very different: Ganeti has a
single "master" node that makes all the decisions and replicates cluster state
to other nodes.

------
apple4ever
I really like Proxmox, except for one glaring issue: dealign with storage. It
is all command line for anything else but local storage. Trying to set up
iSCSI can be frustrating. I hope 7.0 adds that feature.

I used Proxmox in production on a decently sized ecommerce site, and it worked
well otherwise.

------
unixhero
Does anybody here know how to encrypt the root partition of the Proxmox
install? Because data at rest needs to be encrypted.

~~~
grawlinson
You could install Debian with an encrypted root, then add the Proxmox
repositories afterwards.

~~~
unixhero
Ah cool. Will try that, thanks a bunch.

