How do hard drives die? Do they need to be running continuously. Is it in use that they usually pop their clogs?
If I save some material off to a HDD and shelve it, should I feel confident that in two years I can pull that drive off the shelf and expect it to work? Assuming I haven’t stored it either too hot or too cold or too humid. Or on top of a giant magnet (speaker).
Hmm. I’m not an expert, but, that gives me the heebie jeebies. It all depends. How old is it? How many hours are on it? How much dust from your kids and cats have found its way into your drive? Are you really temperature controlled? Under 35-45% humidity? There isn’t a business in the world that stores data on a spinning disk on a shelf that has a good time.
This was sort of my pessimistic thought @randy
I feel that without access to another type of hardware, the safest bet would be to save two copies but even then…
I mean, maybe it’ll work. But, RAIDs and NAS are sooo cheap these days and accessible there’s no reason to not have 2 RAIDS around, one primary one backup.
And depending on the volume, you can likely find cheap cloud/dropbox/google drive storage. Google Drive is unlimited data* so I just don’t see spinning disks as ever a solid storage solution . Especially only 1 of them. Not raided. And likely in an enclosure that could also die.
In fact, I’ve got a couple of OWC Thunderbay 6’s for sale if you want 'em.
Interesting. Of course. Why shelf it when I could RAID it. Keep the RAID storage as a regular storage with some protection.
Ha my little garage operation was only meant to be small. Watch it grow!
For archiving, I use old server-class HDDs I replaced with newer ones in my NASes. All of them are duplicated for data safety reasons, but I had not a single failure ever.
Yeah I said that last year. Ya buy one piece of kit. Then you need a backup. Then you need a way for it not to lose power. Then you need a second piece of kit. Then you need them to communicate with each other.
It. Never. Stops.
How much data is it?
At this stage, very little. 2TB.
It was more conceptual. It has raised some interesting questions that I might return to at some point.
Always good talking to the more hardware savvy people! 
Only 2TB? Ish? Well, then, the world is your oyster!
Here are some top level things to think about. Every situation is different, so, if at some point you want to dive in post some details about what you want and what you need and we can help.
So, for storage, ya gots options. It all depends on how much data it is, how often you access it, how many people access it, how critical it is to you and your family’s livelihood, and how much you want to spend.
Storing disks on a shelf is scary for me. It may work for others, and thats great. But, my house’s temp/humidity/dust is super off the charts sometimes and with the risk of local flooding and fire then and that my data is very important to my family’s livelihood, I’m going to make a different choice in my situation.
The most basic option is some kinda direct attached storage (DAS). DAS can have the best performance of all the disks, but, are attached to a computer and doesn’t give you a ton of options if you have more a few computers that want to use that data. A 4 disk array would give you slower performance but the ability to either RAID 5 the lot, or maybe even RAID 10. There are loads of these out there, but the 2 most common are the Promise Pegasus and the OWC Thunderbay series. The Promise is hardware raid, which means there is some piece of kit in the device that manages the RAID. It is a bit higher $/TB ratio, but, many people on these forums use them and swear by them (@SamE and @johnag , I believe). Typically they have improved rebuild times for new disks. I had a Pegasus r4i in my Mac Pro in RAID 5 and it never skipped a beat. I also have the Thunderbays using SoftRAID. The RAID is managed via software, which, means you can take all the disks out of the enclosure and move them into another one in case the enclosure dies, but, the rebuild performance is typically less than dedicated hardware raid controllers. I’ve got a couple of Thunderbay 6 and they’ve served me very well over the last 18 months. Each have 6x8TB disks and I can combine the 2 enclosures with SoftRAID and then create a RAID 10 across all the disks such that 40TB usable but lotsa protection from disk failures.
Another great option is the NAS route. @finnjaeger is a fan of this approach. At its bare minimum, you could get a couple of 2 slot Synology NAS, with 8TB disks in each one, RAID 1 the disks, and then setup Snapshots and Replications to effectively clone an entire Synology NAS to the other giving you a reasonably inexpensive but pretty darn powerful introduction to the Synology ecosystem. Then hide an online backup in Google Drive or Dropbox for a couple hundred a year or a cloud service like Wasabi for $6TB/mo and off you go. The challenge with NAS devices is that pretty quickly you’re gonna want to take advantage of 10GB networking and then all of a sudden the costs jump up a little bit. But, at the higher end of prosumer 12 bay NAS devices over 10GB you can get amazing performance for your buck. I’m transitioning to the NAS approach and whilst Synology gets poopooed by the purists who prefer ZFS or TrueNAS (@andy_dill , how you likin’ yours?), the fact that in this day in age you can find Synology products in every online marketplace and every YouTuber under the sun has a Synology NAS How To video about the very thing you’re trying to figure out, it’s a highly compelling solution. Just be careful if you go the NAS route, as you’re data is now possibly easily accessible to all the inter web knuckleheads so kill the admin accounts, change a couple default ports, turn off external access, turn on firewalls, and setup a VPN into it if you need to remotely access it. There are far too many bots auto scanning IPs looking for Synology admin port 5000 and default admin accounts to ransom ware your deets.
The best designed systems, which, isn’t anything you’ll need on day 1 to day 600, have local redundancy in case of a disk/enclosure/NAS failure, a local/possible offsite copy in case of malware/cryptolocker/malicious whatevs, and a traditional offsite/cloud backup in either cloud or Google Drive/equivalent. So if you’re going to get into the wide world of home data hoarding, you’ll need to budget for a least 2 local copies and an offsite. It’s a very humbling feeling to see a bunch of spinning disks spin up, vibrate the shite outta your desk, heat up your home office, and then you’ll realize holy cow if this thing dies I’m screwed.
I try to make the clients responsible for the backup. In episodic TV, they have the infrastructure for backing up the entire season to LTO tape, which has a much longer shelf life than a hard drive.
As independent operators, we can’t spend our lives worrying about maintaining data that has about a 10% chance of ever being needed.
If and when they come back, and the data is needed, I ask them to send it over. If they have trouble, I just charge them to re-do it. If I happen to still have their data, I’m a hero.
That said, I have more than 10 Promise raids in my home office, and I only erase when I must in order to keep them under 90% full.
Early in my career, I was told, ‘If your data isn’t in two places, don’t assume you have it.’ This simple advice has saved my bacon from hardware failure and human error several times.
My TrueNas is fine. It’s been turned off for a while cos I’ve been remoting into other people’s flames for the last few months and haven’t had anything local to back up, so I’m not gonna be much help in this thread.
For what it’s worth, I set it up as raid10.
Some random thoughts:
A factor to keep in mind is that drive interfaces change over time, do you still have a machine with eSATA? Firewall 400 / 800?
“RAID is not a backup”: unless you have regular snapshots (and catch the problem before the last snapshot is overwritten), it doesn’t protect you against accidental deletion / overwrite.
RAID 5 is risky with today’s large drives: the Bit Error Rate of typical desktop SATA drives means that you are likely to get another drive error while rebuilding the array. RAID6 / RAID-Z is preferable.
If all your data is in a single location, you are at risk in case of fire / flooding / … (an increasingly frequent risk).
A solid NAS with replication to cloud storage (AWS, Wasabi, many others…) is probably the approach I would take if I had important data to preserve (luckily I don’t), and that monthly cloud storage invoice helps to force the issue of whether data is actually important enough to keep around.
Hey, Have you thought about investing in an LTO unit? I’ve found the tapes are more reliable if it’s just sitting on the shelf for years.
That’s a good long term option, often overlooked, not as sexy as NAS and other things. Slow and awkward? Maybe, but pretty battle-tested technology
Yeah but with an investment of $3-5k investment for a modern loader and you need to deal with offsite copies?
It is the cheapest per TB but for an individual its a tough bit to…umm… swallow.