Hey everyone,
I have been archiving Project files and Flame Archives to LTO for years now. I still find it the most reliable but I am always looking forward and I wondered what you are all using to archiving your Flame Media and Setups at the end of a project?
I am interested in exploring cloud options and wondered if any of you have recommendations here or is good old LTO still the best and most reliable solution?
Thanks 
Dan
Not sure what sort of work you do, but for tv spots I need to have full access to any given project for about 3 years. I break that access up into a few catagories. I have a 50tb terrablock which I archive to pretty much on a daily basis. It holds about 9 months of jobs, give or take. Before I remove a job from flame, I add those archives to an LTO8, so that I always have 2 copies of everything. When the LTO8 is full I make a copy and if need be, remove the archive from the terrablock. ( generally never delete anything until I need the space) In this way, I always have 2 copies, and using YoYotta, I have moderate quick access to those archives for later retrieval. I miss not being able to go to and load from LTO direct to flame, but such is progress . . .
We priced cloud archiving and found that it cost about the same as the hardware set-up, but that was about 2 years ago. I share the cost of the LTO reader/writer with the editorial side of the company who use it far more than I, so that helps, and we already have the infrastructure for the terrablock.
Still using LTO as it remains the best price when youāre dealing with large amount of data, and also reasonable technology stability.
I have a long established folder structure I maintain per job. That covers source files, project files (non-Flame), renders, deliverables, etc.
As for Flame, I have a running Flame archive while working, which I do backup to the cloud to provide daily offsite coverage. But it only includes the setups, key renders, etc, not the source files, etc. to keep it manageable in size. For cloud backup I use LucidLink with a Wasabi back-end. That costs about $20-60/mo depending on what is going on.
Once the project wraps, I make a new archive that covers more detail, but not all the intermediate desktop snapshots, etc. That gets saved in my folder structure with everything else.
I retain a copy of this folder structure on a RAID for a while, and also make a copy to LTO. Once I run out of space on the RAID older copies roll off, they only exist if the client comes back for anything after we wrap. Similar to Tim, I have at least two copies of it for about 6 months, then it might be single copy on LTO after.
As for LTO, I use a OWC Mercury LTO8 drive via Thunderbolt connected to an older MacbookPro. I use Hedgeās Canister software, and LTFS to just mirror folders onto LTO. Iāve previously used more comprehensive backup software (Retrospect + YoYotta), but donāt like their proprietary formats (Retrospect) and complexity (YoYotta), and I donāt really need incremental backups and whatnot, since thatās covered in the cloud, this is just archiving completed jobs. LTFS provides a bit more stability there. LTO drives can read two generations back, so at some point, I may have to forward copy any archives I really care about long-term.
As for files that are not job related, but cover my basic business operations, I have an occasional LTO snapshot of those, plus I have a Cloud version. Right now they also go to LucidLink for the everyday backups. In the past Iāve copied them to AWS buckets, but itās more hassle. At present I use GoodSync as a utility to sync folders between my RAIDs and LucidLink, but Iāve also used SyncTime (Mac) and SyncBackPro (Win) in the past. GoodSync costs a bit more, but has good schedule job schemas, runs on all three OS (Mac, Win, Linux), and can access all the various cloud providers, so I can use it for bulk downloads from Dropbox, G-Drive, ftp, etc. as a more versatile tool.
Re: LTO drive, I saw that Hedge in their resent newsletter mentioned a new drive enclosure someone else built that is 10GbE enabled, in case you want your drive further away from the desk for minimizing noise, as that can be a hassle.
I think I did the math again, not too long ago, and LTO starts becoming cost effective once youāre past around 200-300TB of archival storage. Less than that and youāre better off with naked HDD in a SATA dock (if you do single copy). If you want RAID coverage, what a lot of folks do is find a buddy and do remote sync of a large Synology RAID. Bit more on a per TB cost, but more redundancy. But has some built-in headroom limits, so you would only use it for recent jobs and then drop everything after a while. I still have files from jobs I worked on 10+ years ago, and did recently go back to one of them not too long ago (for source files, not project though). Archiving projects for the duration is a whole separate topic. Though we have a recent job that is an Avid edit that has been going for more than 10 years.
Thanks YTF - I have pretty much the exact same workflow archiving to nearline storage and the to LTO. As a freelancer I have only myself to manage (85% commercial/music video workflow) but find myself archiving on average 40-60TBs annually so am always curious to know what options are out there. LTO for me for now is the most cost efficient but I have not looked at cloud options for a while.
Hereās the current break-even point (prices from B&H)
12TB Seagate Ironwolf SATA drive naked: $199
OWC LTO8 Drive TB: $4,679
Software: $300
HP LTO8 30TB cartrdige (13TB compressed): $54
I always go with compressed capacity for LTO, since most of our big files donāt compress any further.
At 35 units, or 420TB you break even.
35x $199 for naked drives: $6,965
vs.
1x LTO Drive + Software, 35x LTO cartridges: $6,869
Prices may be slightly different if you use larger or smaller drives, there are sweet spots on $/TB, but I kept the math simple by going with same as what one LTO cartridge can hold.
420TB on LucidLink could cost you as much as $420/mo $8,400/mo in cloud storage cost and even more economic AWS Glacier Deep Archive would still be $420/mo. Cloud is good for small and flexible, not bulk. Unless you gamble on āunlimitedā which never works long-term. See Dropbox.
The big issue is with LTO is the bulk of the cost is Capex upfront, as with naked drives they roll through as you bill for jobs (or even bill for the archiving).
LTO gang here as well.
As we work on film and episodic, projects can last for months, so I periodically archive, (on a shot by shot basis) to our server and backup server, then when the job is finished that archive is then put on to LTO. Itās very rare that Iāll need to pull anything back, but on the occasion it does, as Iāve archived shot by shot, it;'s a simple matter to restore anything.
I also keep my archive segments to 1GB.
Hi Tim,
Iām curious how your experience with YoYotta has been. Iāve been using BRU PE for many years, but they went under during the pandemic, and sold out to OWC who has rebranded it to Argest backup. The usual backup and restore has been fine, but their import software for older tapes has not been working. Have you been using LTFS, or does YoYotta have a proprietary format?
There was a similar thread a couple years back, and several users (including myself) indicated ditching LTO for just writing to a solid drive.
A lot of my jobs are for broadcast promo, and basically never ever come back after 4 months top, so I prefer the ease and speed, not to mention not hearing the loud LTO grinding away for hours.
But yeah, if you need to access jobs possibly years down the line, LTO makes sense.
Iāve used YoYotta for 2 years before switching to Canister in '23. It does use LTFS, so no concerns about data security if the software were to go away unexpectedly. It also keeps catalog files that make it easier to find files for restore, which Iāve done on a few occasions.
It worked quite well in general. But itās very feature rich, and itās primary target are on-location camera file backups and transcoding, as well as media media management that can trim camera files. As such I made a few mistakes early on in organization because I didnāt completely understand how they set things up. They distinguish archives, and jobs, and have complex selection logic. Once you write it in the wrong folder on LTFS the fix isnāt as easy as on a hard drive. So that became my biggest headache.
With canister which is no-frills drag & drop to LTFS, fewer things might go wrong. I already have a master spreadsheet for all our jobs, so after I archive I just write down which Tape # the job go archived and thatās it. No need for complex catalog files. Each job has a folder based on itās ID.
So if you use LTO on very large jobs with trimming camera files, or just pulling partial files off a drive based on EDL, YoYotta is by all accounts perfect for you. If on the other side you want to just archive a folder to LTO after the work is done, donāt need incremental backups or any of the elaborate backup management, then there are cheaper/easier options.
Thanks Jan!
The info is really helpful.
It is LTFS. Itās a bit overkill for me: it provides for far more orginization than I really need so I think I only use about 10% of its capabilities, but it does the trick. I can open an LTO like an external drive and only recover what I need.
Thanks Tim, I appreciate it.
I will add that I share it with the editorial side of the company and they use more of the features, so as Jan says, there can be cheaper alternatives for our simple needs.
I was about to hem and haw with my āold man that got burned by stuck hard drive headsā spiel yet again, but realizedā¦are SSD costs low enough now that you can just archive to them? A 2.5 GB LTO 6 tape retails for $25, and a 3 TB Samsung portable SSD is on sale for $110. 4x the price, but not a ton of money in the grand scheme of things. You donāt need an expensive LTO drive and you can write and restore stuff a whole lot faster.
Our projects grow to 10s of TBs each. So we must use LTO.
My projects are not in the 10ās, but they are often over 3Tb, and I prefer double redundancy if possible. We compared LTO, AWS and drives for price. Drives were far too expensive in the long run. And for what itās worth, we have had a shitload of bad drives come in from clients lately. More in the last few months than I have seen in previous years combined. Several have been the small SSDās and we have had multiple failures with the 18TB spinners.
Just one note to keep in mind. Neither of the drive types is immune to problems in long-term storage.
The general recommendation has always been that each drive should be exercised on a semi regular schedule (maybe once a year?). On spinning drives that can eliminate mechanics that freeze up. On an SSD it prevents bit rot. SSDs should be powered on occasionally so that they can run their internal refresh cycle.
There are lots of stories, some good, some bad on these topics if you Google it. And technology is evolving, so some guidance from the 2010s may not longer be accurate.
The main point being though that while SSD have lower risk of mechanical failure, that doesnāt make them risk free or maintenance free.
Of course LTO cartridges have their own set of problems too.
I would be cautious of any data that hasnāt been forward-copied in some fashion at least every 3-5 years. Which made me always chuckle because the Magneto Optical drives HP sold in my early career where guaranteed to 100 years (not that anyone could prove that beyond theoreticals), yet even if the cartridge contained data, you were unlikely to get anything off them even 20 years down the road for many other reasons.
HPās brand back then was āHP SureStoreā, and the internal joke was that it wasnāt āHP SureRetrieveāā¦ Everyone can store, retrieving (or restoring backups) is the trickier part.
As for failures, I have at least one SSD that is no longer working or readable (incidentally a Samsung T-5, on the shelf since before Covid). Iāve recently had two spinning drives that were on my shelf for many years not mount but be recoverable with software.
Hereās one article on SSD offline durability thatās balanced: The Truth About SSD Data Retention
this has been so great information about LTOs!
I have just recently inherited the responsibility of my post houseās LTO archiving. The person who ran it before is no longer with the company.
We had been using BRU for years, but as @dcrites mentioned OWC took them over. While part of me wants to not change what isnāt broken, Iām also taking this as a chance for updating our archive process.
We back up our job folders as a group for each year and just span them across as many tapes as weād need. BRU was great because we could then search for old files and restore as needed (not a lot, but it does happen enough).
I would love to move us to LTFS for file security and ability to access if the software goes bye bye.
Cataloging is also needed so we can search our archives.
It looks like we also have YoYotta, but I have also taken a peek at Canister that @allklier mentioned he uses and it looks so simple.
Iām intrigued by the streamlined drag and drop of Canister, is it really that simple? Or is YoYotta just as good of an option since I have the license already.
The hope is that I can train multiple people on the team so the knowledge isnāt locked to one person like it used to be.
Iām about to start archiving our 2022 projects which is about 150TB 
It really is that simple. Takes just a few clicks to setup a job. Itās barebones, but sometimes that good enough.
I believe they do generate a TOC text file that you could incorporate as some sort of index. I havenāt needed that, so didnāt bother.
YoYotta spits out larger catalog files you can browse inside the app.