I have no use for this service but i definitely like your text centered websites. Right to the point, no disturbing and useless images or graphics. Thanks, that was a breath of a fresh air.
I like the text centered layout too, but in general I have a problem with sites not having a "legal notice", giving me a quick overview who is behind the site and service. In Germany a "legal notice" is enforced by law and this is a good thing.
There is no reason for a reputable professional data center operator to stay anonymous and not disclose a business mailing address or applicable jurisdiction.
But then again, you are free to upload 8TB of your data to an anonymous website that was created this morning.
I heard the same thing about private WHOIS data a decade ago, and this year, despite all registrars offering WHOIS privacy by default and most for free.
I don't think your point is valid.
> But then again, you are free to upload 8TB of your data
Exactly, let the market do whatever it wants. And of course I don't mind you discussing your own predilections and why you wouldn't use it, but the broad blanket statement is just naive and too simplistic.
That's because domains were increasingly registered to masses of private customers, who should have a different expectation of privacy than businesses. And privacy is a problem here. You cannot enforce a privacy policy (if there only was one) or data protection law against an anonymous website, so expect all your data to go to the highest bidder. Naively, maybe, I would expect that to be unacceptable, although there seem to be people in favour of this. It's possible that you'll get enough data later in the ordering process, but it's still highly unusable (compared to e.g. rsync.net and all the other big clouds).
It's not about commercial, but "geschäftsmäßig" which is a way broader term. Any form of doing things in a repititve form can inder some circumstance fall inter it's this includes non-profit work, can contain hobby things, ...
This is correct. Also everyone using GitHub or similar services and uploading code must also have an impressum on their GitHub profiles. It applies to all kinds of web presences.
Commercial offering encompasses "serving ads", "writing promotional articles for own services or third party services".
It's rather that you don't need an Impressum if and only if the website is neither directly or indirectly linked with a commercial purpose/gain/strategy.
but plain text is better than 99% website's aesthetics, most people just have no idea how to design anything. I think plain text belongs to "outsider art", where a lot of people practicing don't even notice how visually appealing it is (when done right).
The ZFS site is more of a man page vibe to me, rather than retro/90s per se, as it's using some basic bold/underline and not just a pure ASCII doc. Maybe it's just designed for expedience, but I think it's a perfect fit for a service like this.
-_- really? Industrial, UX and UI designers are manipulators now? Is Jony Ive the head manipulator?
There are more ways to communicate than a black and white wall of text. One of them is called language of design. That's why designers use visual language to communicate with their users.
Sorry, I didn't mean to say that all design was manipulative. It's so often abused to hide manipulation though I'd rather just go without and I know I'm not alone.
> UX and UI designers are manipulators now?
of course not, but usually web pages creation is driven by marketing, so in most cases "sell" stuff has much more priority than actual UX/UI. So usually UX scarified for marketing needs (even UX and UI designers gains that).
I find this site much, much less useful than your average image-heavy marketing website.
It completely fails to explain, in simple, general terms, what the hell it is or why I should care.
The first, most prominent item on the page is some numbers that mean nothing to me.
The second item says this is beta, which I don't care about since I don't know what this is.
The third items claims to be the "purpose", which is exactly what I want to know, but the one and only piece of information is a weird command line that, again, means nothing to me.
The fourth item tries to explain what this is, but just goes into a lot of detail when I am still wondering what this is for.
That is, why this site is good. People who know what ZFS is and used it before, also get immediately the point of the site.
This site does not try explain ZFS to newbies, which makes it very concise.
I think it is not the purpose of this site to explain somebody how ZFS works, because this is simply just not done in 2 sentences. You need some hours and days to get the idea of ZFS.
Hashicorp is great but should really start adding some real-world-sysadmin language to their websites.
Every piece of tech is generally worth the time investment, but you have to trust them and start learning the tool before you can realise what it actually does.
Consul is one such case...
Consul is much more than DNS of course, but at the core it's DNS.
Unless you describe it as fancy DNS with extra features, it's description page doesn't make much sense.
>Hashicorp is great but should really start adding some real-world-sysadmin language to their websites.
Oh yes Hashicorp is great please don't get me wrong, but if you just read that landing-page it's like WTF. But if you know what Hashi is doing and read the Documentation just a little bit everything becomes clear...but that page alone just sounds like pure Marketing blabla cloud stuff.
I think engineers know the reputation of hashicorp products at this point. The marketing site is for the C-suite people, documentation is for the technical people.
FWIW, their documentation pages [1] are actually a great landing page for technical readers. The comparison to other software is pretty good for "grounding" consul into your other infra.
I've used Consul, Vault, Vagrant and Terraform so far and all of them are worth the time investment (Nomad is on the todo-list but for personal reasons I cannot play with it right now).
It'd be great if Hashicorp used less marketing bs and provided a clearer description... At first I glanced the Vault description, didn't really understand what it was supposed to do and walked away thinking "I probably don't need it". Which is bad because once I started using it I really loved it.
If you don’t understand it, maybe that was intentional on their part to not do business with you. It’s not a successful path in all cases, but if you have a niche product targeting a small audience, I can see value in what they did.
If you don't understand how you may need their service, you do not need their service. I'd welcome being proven wrong, but I would bet money there's not a single customer they're missing by their current front page.
The only way you, as a person not knowing what ZFS is, would have any use of their service is if you, after explanation, are so motivated to pay them money that you change your own data hosting to match a service you previously didn't know you needed even if stumbling over it.
1. Near-zero conversions. If you aren't already using ZFS, you aren't going to start just to use this service. And If you do use ZFS, you likely know what `zfs send` does.
2. The few conversions that do happen are almost certain to be high-maintenance.
I use ZFS on my NAS at home so presumably I am the target audience. What I failed to learn:
1. What OS is this whole thing running?
2. How many drives do I get? What’s the point of ZFS on a single drive? Is this just a way for me to temporarily hold a file system?
3. I don’t remember all the flags for the various zfs/zpool commands so the example command means little to me. This seems to be the main thing explaining what the service is for, yet I would have to go and look at the man pages to figure out what it does. Best I can tell: I can send my whole filesystem from my local NAS to my drive in the sky, but why would I want to do that?
4. Why do I get root access to the box? And if that’s the case, why do I need it?
5. What are the details of the data center they are in? Or the location? Or how did they arrive at their 99.999% reliability?
I have a pretty low attention span, but know the answer to several of those questions without even having to go back to the page (though I will, for the sake of argument). This might not be the case for everyone, but for the way I consume information, the product is described pretty well.
1. What OS is this whole thing running?.
Listed under "storage" section:
OpenZFS + CentOS 8.2 (maintence support until 2029)
OpenZFS + Ubuntu 20.04 (maintence support until 2025)
2. How many drives do I get? What’s the point of ZFS on a single drive? Is this just a way for me to temporarily hold a file system?
Listed under the "storage" section:
We rent out KVM virtual machines with dedicated 8-TB hard drives.
Listed under "pricing model" section:
Rent out multiple drives if you wish to create a RAIDZ or Mirror ZPool.
3. I don’t remember all the flags for the various zfs/zpool commands so the example command means little to me. This seems to be the main thing explaining what the service is for, yet I would have to go and look at the man pages to figure out what it does. Best I can tell: I can send my whole filesystem from my local NAS to my drive in the sky, but why would I want to do that?
`zfs send` isn't exactly an exotic command, and while I can't blame you for not knowing each and every flag, does that matter? The site clearly states that you have a raw disk passed through to a KVM machine, which is pre-configured in a storage pool, but that you can configure it however you wish.
4. Why do I get root access to the box? And if that’s the case, why do I need it?
I agree that the unmanaged aspect of the service should perhaps be more prominently highlighted, but the site clearly states that you configure the machine however you like. To do that, root access is needed.
5. What are the details of the data center they are in? Or the location? Or how did they arrive at their 99.999% reliability?
I agree that more information about the data center would be relevant. The location of the DC is listed under "set-up time" section (Sacramento).
The OS options are listed. The pricing is clearly per disk, and "Rent out multiple drives if you wish to create a RAIDZ or Mirror ZPool.". Backup. With root access you can do whatever you want with your system, including (as the page mentions) non-ZFS things.
I know exactly what this service is, but it still took me a minute to figure out what the use case is, and why I'd want it (off site backup; I struggled initially because the focus is on ZFS) - I like the text-based content, but like you I think the copy could clearer.
It works both ways, you don't waste their time if you are not going to be a customer, they don't waste your time if you don't really need their service.
I think the target market is technologically minded people who have servers at home but want a backup in case their house burns down/gets flooded/etc. You can probably also run things like nextcloud from it.
They look like they’re trying to target the raw storage price floor. In such a market support costs could eat them alive.
As a market hypothesis, there are many folks hosting their own media/storage servers for various purposes. Often these servers run zfs and they can’t. Be oushed to the cloud due to inefficient pricing for TB scale, low frequency, low concurrency, moderate latency storage solutions.
Offering raw drives in zfs configs could crack this market, if customers buy into the raw drive solution then they can work on simplifying the solution and educating their audience.
It all made perfect sense to me. I'd argue it did its job very well. If you disagree, it probably didn't make sense to you and doesn't appeal to you and is therefore also doing its job.
Recommendation: since the "rent to own model" is so built in already, also permit people to ship you THEIR drive (which they still get to own). This allows them to have a backup of existing data without the cost of upload.
This is pretty attractive to me. One thing I am wondering is, do you offer some sort of simple status panel where I can see the amount of data transfer that I have used for the month?
Also, another thing. I might want to use this service for storing copies of important data that I download from the net. But rather than first downloading it to my own computer and then sending it to your servers, does the account on your servers have a shell so that I can run for example wget directly on the server and is it able to connect to arbitrary remote IP or only to whitelisted IP addresses?
Also, your page lists IPv4 but not IPv6. Any word on IPv6?
Hi! We are currently working on a _simple_ account management portal backed by our PostgreSQL instance. This will print out a monthly table with your bandwidth usage/quota and provide a Web Interface + API to purchase more bandwidth, add drives, etc. Our design is API first, with a simple wrapper around the API as our Web UI. They both will be well-documented and functionally the same.
This Hacker News soft launch was mainly to see if we could generate enough users to purchase a second and third server and scale-out. And by the looks of it, we can!
---
As for downloading large files directly to the instance, I'm with you 100%. That's exactly one one of the use cases. (I'm looking at you: massive linux ISOs and +30 GB Xilinx toolchains.) Personally, I like to download them first to a remote machine (takes about 5-10 minutes for 30 GB) and slowly rsync them to my local dev machine with `rsync --partial --bwlimit=10M`.
Think of this service as $5/month DigitalOcean VPS, but it happens to have a hefty native storage unit passed through.
You can SSH into it and do whatever you want with it. In fact, you don't even have to run ZFS. You could run any filesystem -- EXT4, XFS, BTRFS, etc. Compute-wise, it's not going to be anything groundbreaking -- just a 16-core Ryzen 3700 with a couple of cores passed through to you. Ryzen CPUs support ECC and are still priced as consumer-CPUs which is fantastic.
The nice thing about KVM is that even if you don't explicity allocate CPU resources, if the guest machine needs to burst compute, it will "Steal time" from the hypervisor as long as there is extra headroom. This isn't the case with RAM. (For RAM, whatever you set, that is the hard maximum limit.)
Just curious if you've looked at cost for Threadripper for the extra PCIe lanes. There's definitely a premium over mainstream processors, but they're also cheaper than server SKUs.
Generally I find the threadripper not worth it. Radically less motherboard selection, radically less chassis options, half the bandwidth, less pci-e, and near zero price savings.
I looked recently and found the epyc and TR motherboards from the same manufacturer within $1.00. The Epyc 7302P cost $450 for the CPU, cheaper than any TR. Sure TR has some options with higher clock speeds, but generally that's less useful for fileservers.
Generally TR made sense ... before AMD brought out the single socket Epycs.
Also, I can enable IPv6. I think we get a block of /64 of IPv6 for every IPv4 address. Most people are worried about getting their hands on a dedicated IPv4 address (~$1/month @ most co-locations) so that's what I mentioned.
>There is a limit to how many drives we can fit in a 4U rackmount server.
Anecdote: Let me introduce you to the SuperMicro 6047R-E1R36L, a 4U chassis with 36 (!) 3.5" drive bays. How, you ask? By having bays in the front and back, of course. From experience, I can tell you this baby weighs a metric fuck-ton when loaded with drives and is an absolute abomination, but it's a wonder to behold.
That thing is quite deep, don't assume it will fit in a normal rack, and even if you manage it you may find that the ports on a zero-U PDU are blocked.
Exposing drives directly with no virtualization layer - is there a risk that a malicious user might somehow flash a trojan firmware onto the drive, and then later the drive is used for a different customer?
I'd imagine people that are paranoid about these things wouldn't be sending their unencrypted data to third parties anyway. So there's that.
I'm a little skeptical to the claim that passthrough is so much faster than virtio-scsi or something similar. I'd imagine that the disk controllers and network would be the bottleneck. And as an added bonus you prevent people from doing shenanigans on disk firmware. We're talking about plate spinners here, latency isn't really what they're known for.
1. This seems very, very cheap per-TB compared to other hosted storage. Are you making any compromises here that your users should know about? How do you explain the price difference between this and, for example, a Hetzner storage box or rsync.net?
2. Do you intend or allow these VMs to be used as web servers or any other kind of server?
rsync.net is expensive because they are more of a "cold-storage be-all and end-all" data sink. Their backend is AWS, and so the cost reflects that. I might be wrong on this, but I think they use Glacier.
Here, we are pretty much providing a datacenter hookup for your hard-drive. If that hard-drive fails, we can ship it to you for recovery, but that's on you.
The idea for this site is that most people with ZFS pools already have a mirror/RAIDZ setup at home, and would enjoy the peace of mind if they had one-extra data-sink to send their snapshots.
Therefore, true data-loss will occur if:
* Their house burns down
* AND the remote zfs.rent hard drive fails
...all at the same time.
---
Personally, I live about 20 minutes aways from my parents' house. I have two separate ZPools (one for my place, one for theirs). I sync the two every time I visit.
I wanted an extra cloud machine to sync to once a week or so -- you know, in case Northern California fires really get out-of-hand :/
---
EDIT:
I went to rsync.net just to double-check. It looks like they are running their own infrastructure now.
"Their backend is AWS, and so the cost reflects that."
A few things ...
I love your website and domain name, etc. I wish more people operated like this.
I want very much not to mention, or have anyone else discuss, (other competitors) because this is your HN frontpage day.
... so I will just quickly clarify: rsync.net (which predates AWS) has always run on our own hardware that we assemble ourselves. Also, we have no cold storage option or functions - we only provide live, online, random access storage.
... and now let's return to zfs.rent and their day in the sun :)
Thanks for acknowledging your mistake. I also have a tendency to conflate rsync.net and tarsnap as I believe I learned about them at about the same time. This is despite having 2 clients to whom I have recommended rsync.net and whose backup I check each and every day.
You might not fully appreciate how graciously rsync handled this: they could have trivially won a lawsuit against you. Had you made this kind of mistake about another company’s offering, and they discovered it, you might expect to hear from lawyers very quickly.
Avoid making comments about competitors unless you are certain it is the truth.
Perhaps you're confusing rsync.net with tarsnap? When I compared services for my personal off-site backup I came across Tarsnap and their AWS-based infra as well.
I think the better comparison would be an SX-line dedicated server from Hetzner [0] and this server isn't too far off that.
This service: $10 for 8TB, or $1.25/TB.
Hetzner SX62: 64€ for 40TB, or €1.6/TB ($1.9 USD).
> rsync.net
Rsync.net is ridiculously overpriced. Even at their cheapest tier, 1TBmonth costs $15, for which you can buy 7.9TB at Hetzner. Even if you created 3 redundant copies at Hetzner, you'd be paying only half what rsync.net would charge.
I think you're selling rsync.net a little short here. I'm not aware of another service that offers a turnkey remote full filesystem (ie not just an object store) that you can access over SSH.
Your Hetzner numbers are impressive, but I don't think there's anything available in the US for a similar cost. Most block storage I've seen is ~$10/mo/TB. I would love to be proven wrong on that.
> I'm not aware of another service that offers a turnkey remote full filesystem (ie not just an object store) that you can access over SSH.
I know it's not what you meant but keep in mind that as-described, that's every single server provider.
And I think including what you meant (e.g. ZFS set up), it's literally a one-line command to create a new zpool on a new server.
I'm not saying that rsync.net is worthless but it's certainly not worth the price it's charging for anyone who cares about the price.
> Your Hetzner numbers are impressive, but I don't think there's anything available in the US for a similar cost.
When I was in the US I came to the same conclusion, so went with colocation.
However OVH has a datacenter just over the border in Canada and its brands Kimsufi, SoYouStart and OVH [0-2] can get you down to $5/TBmonth.
And it looks like they actually have a couple of US datacenters now, which is new. A quick look shows that you can get a 16TB server for $92/month [3], which is $5.75/TBmonth.
But overall it looks like the dedicated server market in the US is still super expensive. You can always keep your bulk data in Europe.
I think you have a different idea of "turnkey" than I do. For me, having an OVH account, provisioning a VM, setting up the user account, hardening the server, managing backups, etc etc doesn't count.
They charge $5/TB for traffic over initial 1 TB free, and Hetzner charges $1/TB over initial 20TB free, so doesn't make sense to use them as a web server. BTW, Hetzner also has storage servers - 40TB for 64 euro, which is only slightly more expensive per TB.
>1. This seems very, very cheap per-TB compared to other hosted storage. Are you making any compromises here that your users should know about? How do you explain the price difference between this and, for example, a Hetzner storage box or rsync.net?
The compromises seem to be spelled out in the description in a fairly straight forward manner. There isn't flexibility beyond 8TB storage blocks, and it defaults to just one which in my opinion is fairly useless for ZFS, you lose some of the real self-healing and safety advantages (yeah you can have copies within a single drive, but that halves/thirds your space just like a mirror and it'll be slower and with less redundancy than just two half sized drives in a mirror). So basically double or more the pricing there. The data transfer limits and pricing on that is also somewhat higher, so not for high churn. I'm assuming that "1 TB included" is per month, so for offsite replication essentially there'd be a "loading fee" per drive, then bandwidth is covered going forward if churn is less than 1 TB/month. And even in the best case they don't claim you'd be able to saturate even 1 Gbps, so loading will take a while. At 100 Mbps bringing an 8 TB pool to ~3/4 would take around 6 days. There is no SLA, appears to be no autobackups or anything like that.
The pricing could use some more clarity too. Under rent-to-own for example, it's not clear what the pricing is after you have covered the drive(s). Not clear what payments they take and through what service. I wouldn't personally put keys there either, but since ZFS supports raw send/receive there is less need to trust the far side unless your metadata is sensitive as well.
So basically it's ultra bare bones, for cheap.
Which isn't a bad thing to have in the mix. I'm happy to see more space for people to experiment with very minimal focused efforts that lean more heavily on ZFS itself to try to economize vs traditional models. Personally, right now this is still a bit farther on the rough side of the line then I'd feel comfortable with (I'd want to at least have a vague idea of who they were and where exactly the data center in question is). But the basic idea is a neat one, amortizing something someone wanted to do anyway.
Fantastic service! I also have thought of offering this as a side project, but I'm glad to see someone else get to it first so I don't have to. Also loving the website layout. No muss no fuss.
One question I don't see answered: If I go the rent-to-own route, then what's the monthly price (for use of the VM and to pay the colo fees) after I've purchased the drive? $5? And is there a price cut if I want to purchase the drive outright in one payment instead of split monthly?
> For rent-to-own drives, we purchase an 8-TB drive from Amazon
Last I heard, Amazon still can't ship a hard drive properly; I would recommend a source that won't put the hard drive box in a much larger box with a garnish of air pillows, so the drive can rattle around and likely end up damaged.
I might be missing something, but the whole point of zpools is to guard against disks failure; this service only gives you one drive at a time, so you are back to square one. Is it not better to provide multiple smaller disks, so that the sum goes to 8TB ?
My thinking is the aim of this is a cheap way to store large volumes of data (eg from a ZFS NAS). Multiple disks would offer redundancy but would be more expensive. Since this is a backup service, the outcome of having that drive fail is less severe (ie you can put a new disk in and repopulate the backup) than if that was your primary storage device. For home users this might be an acceptable risk given there isn't much else out there in the same cost point ($/TB) and which also support ZFS.
Businesses and enterprise users would likely prefer an organisation with a proven track record (eg rsync.net) anyway -- even data replication aside.
It seems to me if you want multiple drives you're gonna have multiple accounts.
But this seems to be a beta phase so I guess it's more important to iron out the issues around creating a business that works, then you can have more drives in your account.
Usually yes, but i wouldn't mind a single 8TB drive just as a backup location. All my data is already on a nice redundant system at home, but having just 1 extra replica somewhere else would be really good.
Personally I'd use this as a backup target that I could stream ZFS snapshots to.
Zpools with the correct level of redundancy protects against bit rot, occasional hardrive failure and the like(as you correctly point out), but not against total machine loss.
Its probably helpful to think of it as an offsite backup more than anything else.
Do you have to end the key in the remote machine to send it encrypted backups? Can you merge in incremental sends without decrypting the backup on the remote machine?
For encrypted datasets, send data exactly as it exists on disk. This allows backups to be taken even if encryption keys are not currently loaded. The backup may then be received on an untrusted machine since that machine will not have the encryption keys to read the protected data or alter it without being detected. Upon being received, the dataset will have the same encryption keys as it did on the send side, although the keylocation property will be defaulted to prompt if not otherwise provided. For unencrypted datasets, this flag will be equivalent to -Lec Note that if you do not use this flag for sending encrypted datasets, data will be sent unencrypted and may be re-encrypted with a different encryption key on the receiving system, which will disable the ability to do a raw send to that system for incrementals."
This would seem to make it the best backup system, you can merge your incremental backups over time, so never have to reset, without requiring any backup storage software on the remote computer, and it's never decrypted
I totally agree. I actually just built some scripts to do this using rclone, but they could easily be modified to work with a remote ZFS which is actually way more ideal. It is encrypted in-flight, and at rest on the remote, and merging block-level increments never require decryption. My key never has to leave my local machine.
Tools like Duplicati and Cryptomator also allow this. I'm using Jottacloud, 7,50 EUR for 5 TB (actually 'unlimited' but the speed gets slower after 5 TB). Rclone also has support for Jottacloud. Nice thing about Jottacloud is its hosted on Norway. They got privacy laws there. I know if you encrypt in-flight without having your private key remotely this is far less important, but still. Arguably, these products are therefore competitors.
Yup. I believe ZFS supports sending incremental snapshots without having the decryption key.
However, it become a pain-in-the-ass if you have recursive snapshots where only some of the datasets are encrypted. I think... in order to accomplish this, you need to send separate batches of snapshots... one for each set of encrypted vs. decrypted.
E.g.
/zpool/tmp # not encrypted
/zpool/home/someone # not encrypted
/zpool/home/someone/thunderbird # encrypted
$ zfs snapshot -r /zpool zpool@today
# this will probably bork
$ zfs send -v -w -R -I zpool@yesterday zpool@today | ...
# but this will work
$ zfs send -v -R -I zpool/tmp@yeserday zpool/tmp@today | ...
$ zfs send -v -R -I zpool/home/someone@yeserday zpool/home/someone@today | ...
$ zfs send -v -w -R -I zpool/home/someone/thunderbird@yeserday zpool/home/someone/thunderbird@today | ...
NOTE: I haven't fully explored this. But from experience, loading the key on the remote solves a lot of problems. The most import feature of encryption for us is encryption-at-rest. I just want to pull the AC plug and ensure that the data is protected.
Does the nominal bandwidth cost of $5 for 1 TB mean it's $5 each month for a TB per month? Or is it simply "every 1TB transferred adds $5", no matter how long it took to use that much?
For the rent to own plan, the only thing you're paying after payoff is the bandwidth fee? (Which for minimal usage might be under $5 or $5 depending on the above answer) Is that right?
Yeah, I guess the word "nominal" is a little awkward in this case.
We meant it in the sense that most users... will not exceed 1 TB/month data movement, therefore the _nominal_ or _expected_ charge would be $5. Take myself for example, once my data was in the system, I maybe move 30 GB a month and that's it.
(Although we're dealing with ZFS datahoarders here... so some assumptions may need to be discarded ...)
---
> For the rent to own plan, the only thing you're paying after payoff is the bandwidth fee? (Which for minimal usage might be under $5 or $5 depending on the above answer) Is that right?
Yup. That's correct. You're just paying for a slot in the machine + bandwidth. If you need extra bandwidth, it's $5/TB.
Despite the ZFS in the service name, there doesn't seem to be any actual ties/restrictions to ZFS, as far as I can see?
It looks like you just rent a VM from them that has a dedicated hard drive attached to it. How you choose to format that drive would seem to be an implementation detail that can be left to the user. If so, it seems odd to limit your potential customers to people who use ZFS?
The marketing is key here. You guide potential customers to a solution for their problem which is perfectly normal even if the solution could be used in a different way too
This seems to expose a common misconception among ZFS people (and one I previously erroneously harbored) that an SLOG device is useful on a remote machine (that you are not connected to via nfs).
It is not. It is useful for synchronous writes only, such as those done by a local database server, or remotely via NFS. If you’re just sshing in to send data, an SLOG device is a waste.
But again, only if whatever you are running does sync writes to storage. Many things will never benefit from or even use a SLOG. A SLOG on a pool doing async writes doesn't get used.
So maybe some databases or some specific apps, but whatever you'd run in a 4GM RAM KVM won't be much.
Dumb question: if all you plan for the users to do is send ZFS snapshots (as a backup-service or whatever), isnt 12GBs of RAM per host quite excessive?
Wouldn’t you be able to implement this cheaper using only 4-8GBs?
Only thing I could think of was maybe dedup. However, dedup chews up _lots_ of ram. Gobs of it. To be honest, 12gb doesn't seem like enough to run dedup on something like a 6TB pool (just an example) anyway. I recall reading rough estimates of 4x (or even 5x depending) ram for dedup sizing guidelines at one time.
It will provide you a ZFS mirror of two 5 TB 2.5 drives.
I repaste the table from the article for convenience.
PRICE TIME SERVICE
$447 5 YEAR(S) Self Build NAS
$567 5 YEAR(S) Self Build NAS (assuming one of the drives failed)
$1500 5 YEAR(S) Amazon Drive
$1550 5 YEAR(S) Google One
$1200 5 YEAR(S) Amazon S3 Glacier Storage
$2250 5 YEAR(S) Backblaze B2 Cloud Storage
The Backblaze B2 prices are not correct. In the original article, it states:
"For the Backblaze B2 Cloud Storage I assumed average between upload/download price because upload is two times cheaper then download."
In fact, uploads on B2 are free, and download bandwidth is cheaper than any other supplier I know of at 1 cent per GB, with a free daily allowance of 1GB. Storage rates are 1/2 cent per GB per month, of $5/mo per TB, including redundancy.
The storage cost of 5TB for Backblaze B2 for 5 years would be $60/year or $300 for 5 years, the upload cost is free, and if you downloaded your entire 5TB each month, it would cost less than $600 over 5 years because you'd also have the 1GB/day free download allowance. HashBackup (I'm the author) uses the free B2 daily allowance to pack older backup data, which sometimes requires an download and upload cycle.
I couldn't find the upload/download bandwidth assumptions, but these numbers are way off IMO.
I have a Helios64 NAS that I'm running a raidz pool of 5 3tb drives on with no issues. Full kit of board and enclosure was $295.
Overall I've been very happy with it. The only negative from a ZFS perspective is it doesn't have enough ram to support deduplication but that's not a big issue for my use case and otherwise I've been very happy with it.
I’m using the latest 5.9 kernel on Debian buster. I had to compile from source and make a small patch to get the build working. Happy to share a gist. I wanted to use ads 2.0 but if you want something g mower friction use Ubuntu focal I think there are packages already in the distro
The helios64 looks so damn cool ... I'm currently running an aging HP Microsoerver which has served me well, but I think the Helios will be my next build.
Since you mention QNAP, an alternative if you're mostly just looking for a NAS then an old computer with TrueNAS (formerly FreeNAS) is a nice alternative.
You don't need more than about 4GB of RAM, however ZFS is quite good at using it for cache so the more the better.
Anyway, whatever you do, don't enable deduplication. You can't simply turn it off and it imposes sever limitations.
I highly recommend the BYO route, especially if you have part of a suitable machine already lying around. Just about any x86 based system that can take 16GB+ of memory will be a good candidate. ECC is recommended but many people, myself included, have run many stable years without ECC. Load the latest Ubuntu up and give this [1] ars how-to article a read (keeping in mind it is several years old now). Power efficiency will mostly hinge on the cpu and how many hard drives you keep spinning. To scale past the the number of SATA ports you have on a motherboard, add an LSI HBA card like the 9300-8E or 9207-8E.
I bought a mini-ITX board with 4 SATA ports and an embedded CPU. I added a PCIe card with 2 additional SATA ports. I bought a mini-ITX case with mounts for 6 HDDs. I have 5 8TB drives (RAIDZ2) and 1 SSD as the boot drive. The system runs Ubuntu Server.
I found this setup a lot cheaper than the dedicated NAS boxes you can buy and more flexible.
>ECC is recommended but many people, myself included, have run many stable years without ECC.
zfs requires ECC[0][1], do not skimp on this.
I ran many zfs pools for >8 years without issues until silent memory corruption happened to me, and now I can't access my data anymore (zfs won't let me mount a zpool that fails scrub checks).
The problem isn’t that ZFS needs ECC more. It’s that other filesystems will silently ignore the errors that ZFS points out. With other filesystems, you data just eats some corruption. With ZFS, you’re likely to encounter errors because you notice them - Because it’s extensive checksumming points out when a bit flips.
I’d much prefer to know, but if you’re used to not knowing, knowing is scary.
As the sibling comment says, ZFS does not _require_ ECC. It's your data that requires it. Whichever filesystem you use. If you care about the data, of course.
I went and picked up an old dual-CPU Mac Pro 4.1 - four disk slots with WD Red 4TB and one SSD in the 2nd ODD slot for Debian.
It is extremely silent after cleaning the fans and replacing the aged GPU with a silent one but not power efficient, that's granted. But at least it's having actual processing power and can sustain ~120MB/s write unlike its predecessor (Lenovo ix4-300d) that could barely be bothered to do 10 MB/s...
I'd guess that OP is willing to be transparent about this, given their philosophy of "Potential customers should not need to grovel to discover the pricing model." (https://radious.co/philosophy.txt)
No association with this project at all, but "discover the pricing model" is information a customer needs to know in order to make a purchasing decision.
"be transparent [about costs and profit margins]" isn't a necessary factor. It's useful information for the customer, and even more useful for the competitors of the business, but absolutely not the same thing.
> probably higher than the "real" cost of the traffic
Hopefully the project is setup in a way so it avoids the scam AWS et al does with charging for "premium" bandwidth, meaning the cost of the bandwidth should either be a static sum each month, way cheaper or free.
"Free" or "Unlimited" bandwidth always feels worse to me than charging the appropriate amount as you know there's a limit somewhere. No such thing as unlimited bandwidth.
I agree with charging the "appropriate amount", but that's not what AWS and the other hosting providers are offering though. They are offering you a price that is vastly different than what it actually costs.
Instead hosts like Hetzner and OVH uses a price closer to the true cost for bandwidth for their users. Take a look at their dedicated hosting and the pricing. I'm not sponsored by either, but I am a happy user of Hetzner.
I've been renting servers at Scaleway Dedibox (formerly Online.net) for years, with free unlimited bandwidth. I can be pretty bandwidth-intensive for a few weeks at a time (eg. rebalancing a storage cluster) and never had any trouble.
That’s my concern. I would 100% sign up at this price, but I’m doing the math in my head:
They offer 4GB VMs; Assuming they’re oversold and using a good backblaze-pod like design, they can fit ~32 drives per U. 32 * $5/mo is $160; Assuming they’re paying $50/U and the customer’s guaranteed bandwidth is included in their colo cost, that gives them $110 gross margin. Cost of drives is retail - If they have a better supplier that’s a few $ more, but not a lot. The hardware to run that is another real cost. I doubt it costs less than $2000/U to outfit the hardware; It doesn’t have to be beefy but SATA slots add up.
That gives a payoff time of ~18 months. Doable, but not amazing. At a scale of a few dozen racks, this could be a lifestyle business, but it’s sorta marginally so. Assuming a hardware lifetime of 3 years, you start breaking even halfway in. Lifetime margin is about the cost of the hardware - $2000/u. If this is a single man show, about 2 racks (= ~3000 customers) gives a reasonable “salary”, but the upfront costs are high.
I'd guess that at this stage, it's probably not impossible, but may simply not be worth the paperwork and admin of ensuring compliance. Using a forwarding service shifts some of the liability.
I suppose the forwarding service might be different because it's you sending it to yourself; the US company only sent it to your US address. i.e. the restriction might be on US companies rather than 'exports from the US'. Just speculating.
Hahaha no I have several sites running with that Let's Encrypt cert. I'm working on separating them out, but until I get my DNS Let's Encrypt process automated, I'm just extending the cert via certbot.sh :/
The bandwidth is what kills this for me. I actually have a use-case for something like this. I'm re-partitioning my ZFS array, adding some space, and need to host all of the data for a few months while I get it all setup and tested, but the bandwidth cost at $5/TB is prohibitive.
What's the best place to do this? I'm using google drive, but they throttle your upload quite significantly. I can see right now I'm getting about ~150kB/sec even though I have a 1GB upload speed.
I left a comment recommending that they allow people to send _them_ 8TB drives (which the customer then already owns), which would invalidate the initial upload cost :D
The biggest difference that I can see is that this is more like a colo, where you own (or rent) your own equipment (in this case hard drives) to be stored in their facility, where you'll get a (rather generous IMO) VPS that can access your data.
They seem to offer different use cases - I'm not sure you can say one is better than the other.
rsync.net is more expensive, but has included snapshots and better fault tolerance/data retention policies. It's also been around for nearly 20 years, they own their own datacenters, and have a proven track record with thousands of users.
This is cheaper (maybe, depending on your bandwidth usage) and gives you better "ownership" of the hardware, but because it is hardware its just as likely to fail or corrupt data as a hard drive sitting under your desk. This service is also in beta with just a handful of users.
Both companies seem to have really good policies and to be unix user-friendly.
Perhaps I'm wrong but this seems to give you full access to a VM whereas with rsync.net you only have very restricted access mostly to support storage and little else.
You probably could use something like hdparm to power off
unused drives and save power/watts/data center bill when they are not being used to send receive ZFS snapshots.
$5/TB with only a guarantee of 100mbps is kind of unacceptable no? I definitely think you would have benefited from finding a cheaper datacenter, or moving your profit margins elsewhere.
Honestly seems really interesting sans that tidbit, would be interesting to have unlimited 100mbps but gigabit at $5/TB or something like that. But $40 just to fill the drive is a bit much, and with being hard disk with only 4GB of RAM I'd think backups are what this is targeting, and not something with a lot of activity.
Looks like a hobby project so it's no big deal, just giving my opinion.
Why don't you span a ceph cluster across your servers and combine all these 8 TB drives into one big pool. Then you could sell it as cloud storage with freely configurable storage sizes.
I've been looking for an off-site cloud storage provider that takes encrypted zfs sends, so I believe I'm in the target market for this. My use case is a home storage server with a zpool of several hundred terabytes.
However, there are some product compromises that would preclude me from using this. First, I already manage drive purchases, parity, dealing with scrubs and errors, etc. for my pool. I don't want to duplicate my effort for a second backup pool. ZFS sends are, ultimately just a long binary, and I simply want them stored, without much hassle. If I'm planning a zpool expansion, the work in using this service is now doubled, having to (non-trivially) determine the best pool geometry for my own pool, and for a backup pool made up of differently sized drives.
I also don't understand why someone would want to give you their encryption passphrase. A service like this should just take encrypted sends, and send them back when requested. I don't want my plaintext anywhere other than my own hardware.
That said, your main competitor suffers from all of these problems as well.
> I also don't understand why someone would want to give you their encryption passphrase. A service like this should just take encrypted sends, and send them back when requested. I don't want my plaintext anywhere other than my own hardware.
You lose a key feature of ZFS, which is scrubbing. This feature requires multiple drives in a zpool, while you only provides one. Scrubbing generally happens once a month to prevent double or triple bit flips in the same location, however you might get bit rot with your service.
You can scrub but not repair, ZFS will tell you of corruption but not be able to repair.
You can also set copies blocks/data (copies=2) which stores two copies on one drive, allowing you to repair. Doesn't cover you for drive failures but can for bitflips. And you'd get half the storage.
what are uses of this? if I want to back up linux I can now via https://help.backblaze.com/hc/en-us/articles/217664628-How-d... . This option seems more expensive per month. In the future I could even see backblaze offering something similar to this since they are a zfs shop anyways
I entered 8TB initial load/1TB monthly transfer into BackBlaze's "calculator" for "cloud storage" (is this the same as their personal service? It's the only pricing info I could find even after three min of looking..) and it quoted me $735/year.
Also this https://radious.co/philosophy.txt
I wish such design was usable outside tech community.