This is a cool idea. I also like the website, very clean looking, congrats.
I do have some questions: can the owner of a server split it any way they want ? For example if I have a 64 GB server with 16 cores (something OVH provides IIRC) can I split it into 32 2GB/0.5CPU VM ? (assuming firecracker can handle fractions of a CPU, I don’t know).
This could make for some really cheap VM compared to even OVH’s offering.
For that matter do you only support certain types of server or can the owner choose any server they want from Hetzner or OVH ?
So it was just a weekend of coding so far, lots more to do. Right now I have everything hard coded to just split into half based on my example server of 64, 4, 400. If you look at the script:
You can see lots of hard coded stuff just to get it to work at Hetzner on rockylinux9. I think I can make the script a lot more customizable in the UI and let you do all sort of fancy splits! Firecracker does support fractional CPU allocation through CPU shares and quota settings. When you configure a Firecracker microVM, you can specify:
CPU shares (relative weight)
CPU quota (maximum usage in microseconds per period)
CPU period (measurement period in microseconds)
As long as the server has rockylinux9 I think it should work anywhere!
The wording « We help you setup Firecracker VMs » made me think it was a SaaS at first but maybe I was mistaken. Is the idea that a community should host its own instance of shellmates ?
Yes. I have maintained my ledger with beancount for 5/6 years now. I don’t automate downloading from my bank, it’s not worth the hassle especially now that every login requires 2FA with my phone.
But I did write an importer for the csv files my bank provides and with smart_importer I don’t even have to categorize the statements anymore (although there are mistakes sometimes).
I don’t gather metrics though , I use fava to have a visual view of my books.
I usually spend half an hour per month maintaining the ledger .
Red Hat's gambit has always been to hire engineers to do good work on Linux broadly, in many areas. As opposed to just dropping crates full of money at random on projects
Of course they still get tarred and feathered with the "Red Hat wants to control Linux!" brush because they...contribute the bulk of development to projects like GNOME
I've nothing against red hat and i love that an organization is doing work in the free software space. But it's a choice as an engineer if you want to devote your loss of salary to do honest work.
I do open source in my free time because my family can't sustain itself otherwise. Kudos to all the devs out there working on open software.
The UI looks nice but it doesn't look like there's CarPlay support unfortunately.
I don't use my car that often but when I do I want to be able to access my music library. Right now I'm stuck with Plex and Prism on iOS because other solutions are not good in that regard as far as I've seen in my testing.
Hardware wise I bought a TerraMaster F4-423 used last year for about 300€. It's a great device.
Software wise, I use Fedora Server, although if I had to reinstall now I would use AlmaLinux because I don't need the new stuff coming with Fedora Server on my NAS.
I write Go, tried Odin for about 3 weeks, decided it wasn't for me.
I appreciated some things, like the fact that the syntax is close to Go, the easy interop with C, the fast compilation, somewhat complete stdlib.
Other things I didn't like much: pointers have changed to '^', I'm sure there's a reasonable but I just don't like it, I'm used to '*'.
Error handling is a PITA as far as I can remember. Meta programming felt clunky.
This is all subjective based on my limited experience, but I felt that Zig worked better for me.
I've been using Kafka professionally for more than 10 years, since 0.8 where consumer groups didn't even exist yet. In my opinion this post exagerates a lot of things to promote their product.
We don't have giant clusters but we routinely do more than a million messages produced/s so it's not a completely trivial load.
Configuration complexity: there are a couple of things we had to tune over the years, mainly regarding the log cleaner once we started leveraging compacted topics, but other than that it's pretty much the default config. Is it the most optimal ? No but it's fast enough. Hardware choice in my opinion is not really an issue: we started on HDDs switching to SSDs later on, the cluster continued working just fine with the same configuration.
Scaling I'll grant can be a pain. We had to scale our clusters mainly for two reasons: 1) more services want to use Kafka therefore there are more topics and more data to serve. This is not that hard to scale: just add brokers to have more capacity. 2) is when you need more partitions for a topic; we had to do this a couple of times over the years and it's annoying because the default tooling to do data redistribution is bad. We ended up using a third party tool (today Cruise Control does this nicely).
Maintenance: yes, you need to monitor your stuff. Just like any other system you deploy on your own hardware. Thankfully monitoring Kafka is not _that_ hard, there are ready made solutions to export the JMX monitoring data. We use Prometheus (prometheus-jmx-exporter and node_exporter) almost since the beginning and it works fine. We're still using ZooKeeper but thankfully that's no longer necessary, I just have to say our zookeeper clusters have been rock solid over the years.
Development overheads: I really can't agree with that. Yes, the "main" ecosystem is Java based but it's not like librdkafka doesn't exist, and third party libraries are not all "sub par", that's just a mischaracterization. We use Go with sarama since 2014, recently switched to using franz-go: both work great. You do need to properly evaluate your options though (but that's part of your job).
With that said, if I were to start from scratch I would absolutely suggest starting with Kafka Streams, even if your team doesn't hava java experience (I mean learning Java isn't that hard), just because it makes building a data pipeline super straightforward and handle a lot of the complexities mentioned.
