Hi guys:
Is there a way to archive or de-archive in flame in the background through python or any other solution? I always find myself waiting for the process to finish wondering if there’s something out there.
Tx
Ed
2 Likes
Sounds good right. There must be a way. When we had a few flare/flame assist license I could archive another project in the background whilst still working on the Flame. Not the project I was working on though.
I noted down some command line code that I thought I might use one day when I got back into python. Automating my daily archive was one of my highest requests.
flame_archive
You can now exclude renders and media cache when archiving using the flame_archive command-line tool. Use the -O switch.
-O, --omit
is a comma-separated list of at least one of the following:
sources : Do not include source caches in archive (only relevant with --linked).
renders : Do not include intermediate renders in archive.
Yes. You can archive via the command line.
Many studios have built auto archiving at night through this tool.
Also, our @Gunpowder friends have tons of experience in this area if you need some assistance.
Also, running Flame on any other Linux machine or Mac on the network means you can launch Flame with any framestore on your network and archive projects remotely.
2 Likes
Thanks @randy. That’s definitely the way to go.
Bringing back this topic, is there any way to add a bunch of archives to be restored, rather than doing it one by one. Is this something that can e done via command line?
Bring back backdraft!
Yes this is possible. It should only be one more step on the command line. The simplest if most labor intensive way to do this would be to set up your archive restore command for the first project on the command line and then put a semi-colon followed by the archive restore command for the second project. You can continue to put subsequent commands followed by semi-colons into the terminal and then when you run the commands they will execute one at a time in order.
1 Like
The easiest and fastest is to launch any other Flame on your network, choose the machine you want to restore to as the Host in the Project Launcher, and boom…you are restoring an archive in the background.
So, I can launch Flame 1, and while I’m on Flame 3, select Flame 1 to restore to?
However, I’d like the option to set up a a list of archives to restore.
I tend to archive on a shot by shot basis, rather than the whole project, (which is currently 8-9TB)
So I want to choose a bunch of these, then hit restore, just like backdraft used to do.
Oh shot by shot? Ya like doing it the hard way don’t ya?
Then the command line archiving and restoring option is your thing.
If ya did it project style then yeah…just choose the remote flame’s framestore whilst launching your local flame and off ya go.
Main reason is, I’ve been bitten by the corrupt segment before, so I now keep them small, 1G, so if one segment gets corrupted, it’s not quite so bad.
Yup. I get it. Unfortunately the shot by shot archive style puts you in a pretty difficult spot in situations like this where automation and ease of management becomes a thing. I do know that our @Gunpowder friends have setup multiple studios with automated archiving via the command line tools and shiny front end GUIs that make things really slick.
The one thingy that I’m not correctly remembering if I’ve ever seen anyone build a queuing system on the restores side. I’m kinda thinking no because there likely is a good reason to avoid having multiple processes opening the same Flame archive.
Moving forward, the flame segment corruption is unbelievably rare and if you have two local copies then it’s perhaps even a little less rare.
One word Randy, Backdraft.
Apologies for bumping this thread!
I’m looking for a way to schedule nightly archives to be made of all projects on multiple Flames simultaneously, but wondering if I need to instruct artists to close projects every night for this to happen successfully. There will also be occasions where exports etc will be running overnight.
If running archiving from any other machine on the network does the artist need to have closed the project?
I wouldn’t recommend archiving a project while its open. You might risk corrupting the project, the archive, or both, especially if someone is actively using it or you are exporting files. That said there are probably a few ways around the issue.
When you archive from the command line without some trickery the command will fail. Unless a project will repeatedly be open during the period when archiving is scheduled its probably not a huge issue to miss one day.
You could also schedule archiving to happen twice a night. Once after most people have logged off at night and again some time early in the morning when the late night work is finished.
One of the downsides to archiving every night is file size, but one of the upsides is that the archives are relatively quick since you’re only adding one day’s work at a time. Unless you’re doing longform work, lots of lengthy multi-render shots, or have tons of timelines and versions you can probably archive most projects in under half an hour if your throughput is decent.
This is just spitballing but I think you could probably set your automatic archiving to fire off every hour and even get a project or two done while people go to lunch. I wouldn’t recommend it but its really just to say there are definitely ways of sneaking your archiving time in without having to do it while the projects are open.
I was just reading through the documentation on the command again and you could also benchmark your archiving times, build in a function where the command checks to see how much needs to be added to the archive, estimates how long it would take and then only proceeds if it will happen within a certain period of time.
Also you can set the command to kill flame before it starts archiving so that you don’t need to have a person quit out of the program.
Thanks for the info @Ryland!
We are working in long form. So what I would like to do is do a nightly archive of just the project metadata. No cache, no renders. All source material will exist elsewhere on shared storage (we are soft importing) and my understanding is that any renders could be recreated by restoring from an archive, and then rerendering?
Given the above then my automated nightly archives should be very small. I would then therefore create a fresh archive each night, rather than append.
Would my project still be at risk of corruption if I was to automate an archive via command line with the project unknowingly open?
If the machine is logged in, don’t archive it. That’s how I’d recommend setting it up.
I think it really depends on whether there are any changes happening when the archive goes off. Like Randy said, best practice is to just be logged out.
Ultimately I think you’re safer not archiving at all than archiving while the project is open.
Given what you’re saying about how small you’re hoping the archives will be then it shouldn’t be that hard to find time for them to run during some downtime.
You could probably set up an archive that ran when you closed a project. That way every time you stopped working on it then it would backup. Someone will have to correct me if I’m wrong but I think there is a python hook that you could catch on closing the program or switching projects.