Hacker News new | comments | show | ask | jobs | submit login
The 'hidden' cost of using ZFS for your home NAS (louwrentius.com)
84 points by walterbell 497 days ago | hide | past | web | 83 comments | favorite



The 'hidden' cost of traditional RAID for your home NAS:

I've learned the hard way that the 'R' in traditional RAID truly does stand only for "redundant" and not "reliable". Reliability in traditional RAID is predicated an complete, catastrophic failure of a drive such that it is either working wholly and completely or failing wholly and completely.

In a traditional RAID, for any failure mode in which a drive or its controller starts to report bad data before total failure, the bad data is propagated like a virus to the other drives. The corruption returned by a failing drive is lovingly and redundantly replicated to the other drives in the RAID.

This is the advantage of ZFS (or BTRFS). Blocks of data are checksummed and verified and corruption isolated and repaired. Yay for reliable data.


I'm about to either buy a home NAS or build my own with an atom Mini ITX. I plan to expand my array in the future. So is there any configuration that gives me the best of both worlds, ie the expandability of traditional RAID with checksums to prevent replicated errors?


BTRFS allows you to add, remove drives, and change RAID levels on a mounted filesystem, while able to use differently sized drives efficiently. That alone wins me over. Some people still say BTRFS isn't ready for production, but I've had fewer problems with it than with ZFS (YMMV). Still, I don't care much as I have multiple backups.


I decided to try BTRFS on my NAS because it didn't require rebuilding a kernel. The ability to add disks to an array and have it rebalance made it very appealing.

Unfortunately, the three-drive filesystem lasted two weeks before it became unmountable. The only thing that let me mount it was finally running btrfsck. I was left with 57 unrecoverable errors, and lots of lost data.

I would not recommend running BTRFS in RAID5 or RAID6 just yet. Stick with mirroring, if you want to use it, and rebalance to RAID5/6 later on when it's more stable.


Which kernel, and did you file a bug report?

e: To anyone not up on btrfs, its features are closely tied to the kernel version it's used with. For example, raid56 scrub and device replace, and recovery and rebuild code were not available prior to kernel 3.19.

I also believe the only way to use 5/6 modes before they were stable was to explicitly compile with them enabled. It wasn't just something you could accidentally do.


It was 4.2.5-1-ARCH, and I didn't file a bug report, no.

I didn't have much data to submit. No kernel panics, no useful error messages, nothing beyond it saying it wouldn't mount. One could read the tea leaves from the filesystem as it sat, but such data spelunking could take a while on an 8TB partition, and I wanted to get the disks back into use.

I didn't notice the corruption until after I had unmounted it, so scrubbing it wasn't an option.


Thanks, that is what I wanted to know. Not what I hoped to hear, but what are you gonna do?

I recently got myself a new home server and decided to use FreeBSD and ZFS, so the question is settled for me for the moment. I still hope Btrfs gets there in the not-too-distant future, though.


When I was looking to install BTRFS, I saw tons of warnings not to install RAID5 or RAID6 because the code is not finished yet. The other levels are fine.


The last time I checked, using the RAID5/6-like feature was not stable, yet. Have they made progress on that?


I have been saying this for a long time. And yet no one seems to care about client data corruption. Synology only recently has NAS with BRTFS, but they are on the expensive range.

As far as I can tell, the only consumer NAS maker that offer BTRFS is Netgear.


Sadly it looks like Synology's implementation of BTRFS does not implement correction, only detection at this stage: http://forum.synology.com/enu/viewtopic.php?f=260&t=104519


No, except possibly paid Solaris from Oracle (which has a newer version of ZFS), but that would probably cost a lot more than buying more disks.

What I do is have my ZFS in two "layers" (each of them 4 disks in raidz2, i.e. resilient against any two failures), and replace a whole layer at a time. So I started with 4x500GB drives for 1TB of capacity. Then I added 4x1TB drives, total capacity 3TB. Then I replaced the 500GB drives with 2TB drives, total capacity 6TB (and throwing away the 500GB disks, so "losing" 1TB). I'm shortly going to replace the 1TB drives with 4TB drives in the same way.


I personally use 2 ZFS groups of 2 disks, and backup one on the other 'manually' [0]. I get the advantage of both worlds mostly.

I use a HP microstation gen 8, it's lovely hardware; and I use nas4free on a usb stick in the internal port as software.

[0]: by manually I mean the machine that use obnam to backup use one or the other group as a destination, depending on the day.


Look at the Netgear 516. It's got BTRFS and is plug and play. It has a small plugin library, super quiet, except for the first 20 seconds it starts up. Holds 6 disks. And it has intel server processor and more reliable sever ram. Unlike say qnap and synology which have probably more features / plugins it has less but I was sold on the file system and overall it does what I need. Support updates have been fast and perfect from what I've seen too.

If you go your own route, look at this server case silverstone cs-ds380b. I think paired with a mini-itx server board with the most sata ports, maybe get some ECC ram, server rated intel processor. I wish I knew the best software route to go. I've considered Amahi in the past and of course FreeNAS. I supect even a couple commercial packages could be sweet to have.


I have had one for four years.

My Setup: OpenSUSE (BTRFS with their Snappy is awesome)

Two drives that are redundant.

CrashPlan for off site backup

I also run my IRC (Weechat) on the box and glowing-bear.org as the front end and this is the BEST thing ever. Run my printer from the server and have a personal R-Studio and Juypter Notebook server running on it. I love it.


What hardware failures have you survived through so far, and how was your setup able to cope?

If you haven't had a failed disk yet, consider yourself lucky! :-)


<----- Lucky (No Failures) I think I paid $50 for the chip and motherboard. Western Digital Green 1 TB drives.


Do you know, can you use ZFS for the checksumming on top of md for RAID?

This way you get the expandability of RAID with the checksumming and snapshots of ZFS.


According to the article the ONLY way to expand ZFS after the fact is to "replace every hard drive in the VDEV, one by one, with a higher capacity hard drive." If you have some better work-around that gives you both, you need to either explain it or link to an article that does.


My thinking was that expanding your underlying md RAID would be the same as replacing the initial disk ZFS sees with a bigger one, thus enabling easier expansion at the md level and presenting a "bigger disk" to the zfs vdev.

I haven't seen it done, it's just a theory, hence why I asked. I'm just not sure if zfs needs to see actual disks, or if it can work on top of any block device, like an md RAID.


That's not a hidden cost. That's what you'd normally expect with any traditional RAID. However like many modern solutions, ZFS does have a few workarounds:

   MULTIPLE ARRAYS PER TANK
If you have space in your storage servers housing, then you can buy multiple disks at a time as a separate array (eg 3 disk raidz1) and add them to your existing "tank" (ie storage pool).

   AUTOEXPANDING INTO LARGER DISKS
If you have autoexpand=on set on your tank then you can upgrade the capacity of each disk in a raidz, one disk at a time. The caveat here is that your array doesn't increase in capacity until every disk has been upgraded, but it does still allow you to start the upgrade process incrementally.

   ALTERNATIVE SOLUTIONS
However, if one needs the support of upgrading their array, increasing the capacity with each new disk added, then ZFS is definitely the wrong choice. Any traditional RAID array would be ill suited for this. Instead I would recommend unRAID or similar (I think Windows might have something built in, but I'm a UNIX guy so couldn't comment there).


> Other software RAID solutions like Linux MDADM lets you grow an existing RAID array with one disk at a time.

His issue isn't with ZFS, it's that most parity raid (raidz, raidz2, raid5, raid6, etc) doesn't support safely rebalancing an array to a different number of disks.

With mirrors, the things he describes aren't an issue, especially in a home server. You can start with one disk, mirror it when you're ready; then add additional vdevs of mirrored pairs extending your pool as necessary. Or upgrade two disks to grow a vdev.

http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-...


The article you link to is disingenuous (IMHO), it talks about performance during a rebuild, but then says this: "When you replace and resilver a disk in a mirror vdev, your pool is again minimally impacted – you’re doing simple reads from the remaining member of the vdev, and simple writes to the new member of the vdev." If you're only doing "simple reads" then your storage array is not serving any writes, in which case why are you concerned about performance at all?

Each "Home NAS" is going to have very different requirements (to each his own). If you are concerned about reliability and want to be protected against a dual-drive failure, you're better off with RAIDZ2 than with a bunch of striped mirrors, because with RAIDZ2 you can have any two drives fail, where as with an array of mirrors you still have this nagging chance that the second disk in your degraded VDEV will fail and you will loose the entire pool.

Chances are that your home NAS does not have a 4-hour disk replacement SLA.

Also, just blindly recommending to mirror everything may not be the best option for a lot of home users that don't want to loose half of their raw capacity to parity.


Linux mdadm supports re-striping raid5 devices in place. I've done it, it's fun. If you're raid doesn't support that, it's deficient.


The reason that ZFS has such trouble with this notion is that a lot of its internal architecture depends on "block pointers" (offsets of data into a given vdev) being immutable - this is, among other things, why CoW snapshots are so cheap. And attempting to rewrite all of these without having any gaps in consistent on-disk state is...challenging, to say the least.

Sun apparently had internally done most(?) of the work on BP rewrite (as it's referred to), but performance sucked rather badly, and the work has not been shipped by Sun/Oracle, let alone anything that came out before the F/OSS code sharing ceased. [1]

(Performance sucked, in particular, for similar reasons to why dedup performance has such a high penalty on ZFS - they end up storing a lookup table that gets hit for every block that gets written after you turn on dedup, and so if that table stops fitting in RAM, it's a bad time - let alone the issue of making sure the disk backing the DDT storage is sufficiently performant...)

[1] - http://www.listbox.com/member/archive/182180/2012/01/sort/th...


> If you're raid doesn't support that, it's deficient.

Nowadays, RAID5 is deficient.


That's another topic entirely. I'm just saying if you're going to support it, support it correctly.


Five or six drives for a home NAS? This, IMO, is madness.

RAID-1 a couple of drives, and take backups. If the worst happens, you can restore offline from your backups. If you seriously need 100% uptime at home, in the face of multiple drive failures, you are doing something badly wrong. Or, at least, doing something far beyond 'home use'...


A person having different needs than yours isn't madness.

Many of us here work from home at times, either occasionally or regularly, and we have our own projects in addition to our day job. We each make our own decisions as to what configuration gives us the most acceptable level as performance given the level of risk we're willing to accept.

If a couple of drives in a RAID1 configuration works for you, great! It doesn't fit my needs since my media PC has about 20TB of usable space (~15TB in use) and I don't want to have to hunt through backups of any form when I want to watch or listen to something.

I also have a NAS for my more important files. It's backed up to a cloud provider as well, but the amount of time it would take to download everything after a failure would be very stressful for me, so I spent a bit of money up front for my setup. It's not 100% protection, but so far I've had two drives start to fail, but I was notified in plenty of time to get new drives in and let the array rebuild.

I'm happy with my setup. Well, happy based on the amount of money I was willing to put into it, obviously.

Either way, my setup isn't madness. It's simply different than what you do.


> work from home at times

That's business use, it just happens to be at home. Home use is family-style, some documents and pictures, maybe some larger multimedia.


Not in the slightest. I'd rather have some redundancy up front than try to restore a bunch of junk from backups.

Backups are important, but not ever having to use them is even better.

Not only that, but having RAID or mirror sets gives you flexibility in ways that you might not think about up front. For instance, I just replaced some old 750GB disks with new 5TB disks. I added the new 5TBs to the the mirror set and let it bring all the data across automatically. When it was done, I dropped the 750s out and resized the raid set to use the whole 5TB (I told it it was already clean so it didn't have to sync up a bunch of unused space). Then, finally, I resized the filesystem that was mounted on that mirror set. This was all done live, with data actively being read and written to while all this volume manipulation stuff was happening.

That is why you use RAID at home (and lvm, too).


RAID is not a replacement for backups; they do different things.

RAID preserves performance and/or uptime in the face of hardware failures. It does not protect against data corruption or deletion. It will happily sync those across all your disks.

Backups protect against any data loss: hardware, software, even intentional. The key feature of a backup is that it is insulated from live data operations (including RAID sync).


I never said it was a replacement. I wholeheartedly agree that backups are indispensable, but when dealing with drive failure, I'd rather just swap out a drive and not have to deal with restoring from backups. I find backups are good for fine grained laser focused restores—trying to restore an entire drive is either slow, or error prone, especially if one deals with hard links a lot.


Alternatively,

* Plug new hard drive in

* cp data on to new hard drive

* ...profit?

IMO, much simpler, straightforward, less prone to errors and your data is perfectly safe until you do something with the old hard drive. Again, it's not 100% uptime but this is your home, not a mission-critical business.


Not much simpler. Replacing my disk as described above was literally 4 commands from bash. And zero reboots.

And less error prone? Are you joking? In a non-mirror system you have to re-partition and set everything up the way it was, you have to really hope you copy things back correctly (do you have a bunch of hard links? Does cp copy them correctly? What's that option to rsync again? Do you have enough memory to copy hard links correctly? Are xattrs going to copy correctly?). Sigh. On top of that all you can't actually use that disk while any of this is happening. Grrr, and it's going to take 11 hours to copy 5 TB of data back (at 125MB/s)! If you're perfectly happy with your computer being out of commission for 11 hours, be my guest.

Also, when the old disk is removed from the mirror set, it's a complete image copy of the data. I'm not sure why you think that isn't the case...

Seriously, making every disk in your computer a mirror set is really a phenomenal idea. I've been doing it for years now and every time I have to upgrade or replace a disk it's made it heavenly. You shouldn't pooh pooh it until you've actually tried it.


Backups are more useful to me, because RAID won't save you from a wayward rm -f.


RAID itself won't, but ZFS snapshots roll back nearly instantly from a rm -f. Can't imagine living without them for home or business these days.


People don't tend to use parity RAID primarily for uptime, they do it because it's cheaper - HDD costs for a mirror/striped-mirror implementation are always 1/N, whereas with a RAID-6 it's 1-(2/N), or 1-(1/N) for RAID-5.

In my experience, mirrored and/or striped-mirror type RAID implementations are more reliable, performant, and easier to maintain than any variation of parity RAID. Even for a home setup, it's just not worth my time.


> People don't tend to use parity RAID primarily for uptime/multiple drive failures, they do it because it's cheaper - HDD costs for a mirror/striped-mirror implementation are always 1/N, whereas with a RAID-6 it's 1-(2/N), or 1-(1/N) for RAID-5.

A few years ago I built a home NAS. Back then, maximum drive sizes were 3TB for 3.5", and 1TB for 2.5". The tiny ITX case I had had one 3.5" slot, and one 5.25" slot.

So, I could either do a RAID1 with 2x3TB 3.5" disks; or get a 6x2.5" adapter for the 5.25" bay and do a RAID5/6 with 7x1TB for a net capacity of 5/6 TB.

I'm still not sure whether it was a good idea – the hardware looks a bit adventurous: http://dl.creshal.de/IMG_0873.JPG and was loud as hell –, but it worked well for about five years without any drive failures.

(Then I replaced it with a smaller NAS with 2x2TB drives in RAID, because as it turned out, my interests went from binge-watching storage intensive TV series to reading more books, and I never ended up using more than 1TB storage anyway.)


I am currently sitting in my pjs at home next to my two year old 45 TB ZFS server. It really just what you define 'home use' and everyone defines that differently.


RAID-1 a couple of drives

I'm going to hijack this thread slightly to ask: if I want to have a couple of RAID-1 drives running an open source NAS, what's the lowest power hardware solution with acceptable performance?


> what's the lowest power hardware solution with acceptable performance?

Depends what you define as acceptable. RAID-1 (as in mdadm) is fairly low on CPU requirements, generally something like an Intel atom chipset (late cedarview chipsets have a TDP around 6.5w) should give reasonable performance, otherwise a mobile/low end i3 would give even better performance.


Would there be any advantage to running ZFS on that pair of disks in that situation? I hear it has much more demanding requirements including memory/disk ratio.


If you're just using it as "dumb" storage, then ZFS gets you block-summing, so better data integrity guarantees - but in order to rely on that you need ECC RAM, which means a server-class CPU (and more $$$ for RAM). The Avoton range of Atoms do support this (I'm potentially looking at getting an 8-core one myself in the next 6 months).

Other than that, if you're not planning to exploit any of ZFS's functionality, and are looking to deploy on as cheap hardware as possible, mdadm RAID1 (with LVM over the top) is just the ticket. If you need to expand, just add another mirrored pair, assign it a physical volume, add that to the volume group, expand your logical volume, then expand your filesystem (ext4 supports online resize and is very stable, XFS is also a solid contender). All of this is fairly boring compared to ZFS, but it works and it works well..


You don't strictly need ECC RAM to run ZFS any more than you do when running other filesystems, the issues that comes with a bad stick of RAM will just become more obvious which is not a bad thing. It's just assumed that if you run ZFS you do care about getting the same data out that you put in, so it's highly recommended. Bottom line is, ZFS without ECC isn't worse than other filesystems without ECC.


When a block of data on a mirrored disk goes bad, mdadm won't be able to fix it and often doesn't even know that the data is bad.

Since there is a mirrored copy this a correct copy of your data but since there is no metadata or checksums, mdadm has no way to tell which is correct.

ZFS will silently detect the corruption and will rewrite the data restoring the duplicate copy of the data again.

All this works great without ECC RAM, recent studies of data corruption have shown that RAM is much more reliable then previously thought. ECCisnt needed for ZFS for home use.


Mac mini, using software RAID. Only annoying thing about it is that it's fairly aggressive in powering down external drives, so sometimes you have to wait for spin-up.


The question then becomes, where do you put those backups? I personally use CrashPlan and have suffered the long upload times and the bandwidth throttling, others may not be as patient as I was.


turn off or disable the de-duping and the speed goes back to almost normal.


it's not madness if you do video or professional photography. Or if you are storing your family history of video or photography. Or frankly people leverage their home server to grab all their download torrents and newsgroup stuff.

It's not madness if you're backing up 2-4 home PCs either, storing your movie library. I know somebody that rips their blu-ray disc library to full 32gb video files. That's madness but it doesn't suprise me people do that though. Some of us are hoarders for data. Some people behave like they are the internet archive of the darknets too.


The BSD now podcast http://www.bsdnow.tv/episodes/2015_01_06-zfs_in_the_trenches talks about this article and gives some solid recommendations on how to do your setup. It helps that one of the hosts co-wrote a book on ZFS.


And in the most recent episode, #123, this specific article was addressed (starting at about 11:55):

https://youtu.be/B_OEUfOmU8w?t=11m55s


You're just reiterating what the previous comment said


Sorry, but, no.

If you've gone to the effort of getting a massive raid like that for home use, you need a backup.

If you don't have a backup, stop reading, you're an arse.

Run a full backup, check restoration.

Blast away your NAS (its ZFS, relies on free space. The fuller it gets the slower it gets (unless you have SSDs or lots of write cache)) It's probably due and upgrade, (OS/XFS)

re-arrange your array, you're changing the way you use it, so you need to do it.

Restore data back over the top (with compression...)

as for "it costs money" well thats why you chose zfs, for redundancy....


It depends what you're doing really. I run a 5 disk ZFS cluster to host media files. I'd rather not deal with failure very often, but when it does happen and I lose a whole bunch of media, I can download it again.

The second I elect to host anything even remotly important on it, it'll be backed up, but until then there's no need for me to pay to host that elsewhere.

Professionals who think they don't need backup's because they have redundancy do need a slap though.


For that use case ZFS seems overkill.


Possibly, but what's the problem? It's pretty much a "just works" file system where you can lose 1-2 disks where blocks are continuously checksummed and scrubbed to ensure consistency.

What exactly would you rather be using?


>If you've gone to the effort of getting a massive raid like that for home use, you need a backup. If you don't have a backup, stop reading, you're an arse.

Or maybe I need to have large NAS in volume BUT for transient data sets which I could not care less to lose (and thus backup), which makes you an ass for ass-summing.


You'd be using raid0 or similar with XFS...


What are the cheapest drives we can use for ZFS NAS for occasional media access?

Can we use the absolute cheapest Western Digital Green drives, with limited lifespans? Does ZFS spin down drives when not in use?

Actually, how long do the cheapest drives last when spinning 24x7?

I ask because Best Buy has 5TB Western Digital drives for $110 today: http://www.bestbuy.com/site/wd-my-book-5tb-external-usb-3-0-...

But these consumer drives never list reliability figures, unlike data center drives.


I would definitely go for Red or other RAID/NAS type drives, because these are the only ones to have Time-Limited Error Recovery (TLER). Without this, your disk will hang for indefinite amounts of time when an error occurs. That means that a single sector read error can cause the entire disk to be thrown out of your RAID array, instead of letting the RAID system recover from the error gracefully.


Greens would work, but are limited to 5400RPM so your data transfer rate will take a hit with that.

Reds are WD's "NAS" drives, with some advanced features, but the Segate ST2000DM001/ST3000DM001 have worked very well for me with multiple years of almost constant data access.

Sidenote: the upgrade from 4x2TB to 4x3TB was extremely simple, I just failed my drives over one at a time, so as long as you do everything at once you can expand everything really easily with ZFS.


The seagates worked well for me for a long time when they were directly attached to SATA ports, until I got a chassis with a SAS backplane. Their firmware had some bugs which would stall all transfers on the whole backplane (not just the one disk) for about 30 seconds when there was a lot of activity.

I switched to WD REDs after that, haven't had any issues since.


Really? That's interesting (and shitty!). Thanks for the heads up!


I ran a 6x2TB raidz2 pool for years built with Seagate greens pulled from Craigslist-bought USB external drives. FreeBSD and/or ZFS were always dribbling stuff to disk (logs, metadata updates, etc.), so the drives never spun down. These were using the native motherboard SATA ports.

ZFS is very reliable. Sometimes a scrub would find a problem and fix it. I'd then offline the device causing problems, assuming the sectors were getting "soft", and then I'd run a destructive read/write badblocks on it to flush out bad sectors, then reintroduce it to the pool. The drive would be once again be solid for a good long while.

I'm anxiously waiting for the day inline encryption makes it to FreeBSD's ZFS, as the standard GELI+ZFS method has always seemed a bit clunky to me.


In case anybody's still reading this, I have a couple of use-case questions.

I'm planning on setting up my first volume pool using 4 5TB Seagate ECs. I can't get any more than four to begin with.

I was intending on using mirrored vdevs, as I'd heard that was basically the easiest way to get started. This is entirely for personal use, and I don't need "enterprise" read/write rates (the hardware I'll be starting with certainly won't manage that), and it won't be the end of the world for (very very rare) resilvers to take all night.

Are there any advantages to not using mirroring, ie using one of the RAIDZ variants? I've already accepted the idea of having to buy new disks two at a time so I can spin up a new vdev with them.


The biggest advantage is storage space. You lose more potential storage space with mirroring since it's a 1:1, whereas as raidz1 across 4 drives would give you much more storage space but with the caveat that you can only lose 1 drive at a time before your entire pool is lost.


Hmm. 15TB definitely sounds nicer than 10.... :D :D

I fully intend to buy disks with widely spaced apart manufacturing dates, so theoretically this should be manageable.

Now I get why 8 disks is recommended... you can make a mirrored pair of 4-disk raidz1s, you get >50% storage efficiency, and you can lose any two disks before it dies.


My experience with ZFS on my home NAS: started with 6 2TB greens, 1 boot, 5 raidz1, got 5 3TB greens in another raidz1. One of each failed, replaced. Recently got 6 8TB SMR drives in a raidz2.

Never used ECC ram, too expensive. No problem with LSI raid cards, or SAS backplane. Most important advice: People aren't kidding when they say your zpool will slow down when it's almost full.

Online resizing of md arrays isn't reliable, my friend lost terabytes. ZFS is 10 years old, well tested.


I know everyone is talking about btrfs and zfs, but I would just like to give a shoutout to DFLYBSD and HAMMER2, which is also doing great work in this regard (among other things)


HAMMER2 has been in the works since 2012 and still isn't considered something you should put any data on that you care about. No offense to Matthew, and I applaud his hard work, but HAMMER2 isn't being mentioned because it isn't even part of the conversation. At this point we can't even be sure it will ever make it to a production ready state.


You are correct about Hammer 2, but dont forget about Hammer 1.


In the article, adding a vdev of a raid-1 pair of disks would "break" raidz2 redundancy of the other vdevs. Why?

Is it best practice that all vdevs in a zpool use the same RAID level?


To answer your second question, I think the answer is generally yes, because loss of any vdev in a pool means the entire pool is lost.

For example, if you initially created the pool with a single raidz2 vdev, probably the only way it makes sense to add a second raidz1 vdev is if you later changed your mind about how important your data is.


That's why I like Synology for a home NAS: you can just add/replace drives with a larger one any time you like, and the Synology box sorts it out.


ITT: people who've never done an online resize


With the wide availability of cheap backup services like backblaze, this nas building nonsense is a complete waste of time.

I have 3-4 direct attach usb and FireWire disks. One is a working disk, one backup, the rest for various things.

If one of the working disks fails, I have a local backup. If they all fail, call backblaze.

In all cases, I avoid wasting time and money setting up ersatz infrastructure that in reality is less reliable and more expensive than the simple solution.


For the tower of babel that is linux mdadm/lvm/... I agree, but ZFS really is trivial to get running, and easier to use than multiple disks - rather than worrying about what I put on each drive, and manually doing backups, I ran a couple of ZFS admin commands three years ago that told it to put all the disks in a big raidz, then stopped worrying about it. When a disk fails I replace it and run one command, no messing around restoring from backup / changing shortcuts / reinstalling.


> With the wide availability of cheap backup services like backblaze, this nas building nonsense is a complete waste of time.

My home upload speed is 3mbps.

I make weekly image backups on top of my increment file backups. The images are ~800GB and 200GB. Additionally, I have a ~1TB media collection.

Uploading the images to Backblaze weekly is clearly unfeasible. Restoring from old images in the event of a failure would be extremely time consuming.

Therefore, I have a 14 TB NAS that should last me a long, long while and also serves as a Plex media server. To lose data I really care about requires that four drives / two computers fail or my house burns down. I am willing to accept that risk.


> I make weekly image backups on top of my increment file backups. The images are ~800GB and 200GB.

Yeow. This needs a good helping of ZFS snapshots!


The images are Windows Backup vhdx's, so I'm not sure that's possible. It isn't really worth my time to investigate - it happens automatically on a schedule and over a gigabit LAN it goes fast enough and neither the performance of the computer nor the NAS are noticeably impacted.

In an ideal world, Windows would just use ZFS, but hey, you can't have everything. ;)


Your solution does not protect you from silent data corruption.


That's great if your content can fit on one physical drive - but that's not the case for everyone.

However, even that aside, I would likely still use ZFS even with your solution since I make sure of other ZFS features aside just running a software RAID (eg ZFS snapshots, checksumming, send/receive, etc).




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: