Flame project and HUGE ARCHIVES

Hi,

There are a few points that might help you out.

Firstly, my advice is generally to work with the intermediates that match the resolution. Working at a much higher resolution when not needed only eats at your storage. For example if you have 10-bit ProRes, there is no reason to cache it as an uncompressed or RAW source. The media will still be 10-bit but with a much bigger overhead and much larger file sizes. In certain circumstances, Flame will adjust the cache format if the intermediate choice cannot contain all the data for the source file. So if you cache a 16-bit file into the project set to 10-bit ProRes intermediates, the 16-bit data will be cached in an EXR or RAW format depending on the project settings. This is why you see “Alternative” options when you set the intermediates in the project creation screen. So in terms of cache and renders, no quality should be lost and you’re making the most of your storage space.

When it comes to archiving, the story is very different because we aim to maintain everything as uncompressed RGB data. So when you do an archive estimate, regardless of your intermediate format, it will use calculations of uncompressed RBG to estimate the archive size. This is why your source media may be 20 gig but the archive will be 200gig. Any intermediates including cache and renders will be recached and stored as RAW RGB data.

Honestly, archiving everything uncompressed may not be a necessity these days with the quality of some compressed formats however that is how Flame currently works.

This is where the options to exclude source caching and timeline renders comes in because you can store just the metadata in the archive to keep the files size small and then it is your responsibility to backup the original source media. So you may land up with a 25gig combination of source media and archive as opposed to one archive of 200gig.

Now one point of confusion that I hope to explain in a video one day is more to do with renders. Excluding renders refers to the TimelineFX renders but NOT Batch renders.

TimelineFX renders can be flushed or ignored but you can ALWAYS re-render them as they are part of the metadata of a sequence. So I feel it is safe to exclude these renders knowing I can just render new ones if I restore the archive at a later date.

On the other hand, Batch renders are very different to TimelineFX renders. The rendered Batch clips have no connection to their Batch setup or the originating source media. You cannot un-link, un-render or flush processed Batch clip. It is considered as generated media and nothing more. Sure you have the setup but when you render/process the setup, it will create a new independent clip. This makes a huge point when archiving.

When creating the archive with exclude renders enabled, TimelineFX renders will not be part of the archive as Flame “knows” that they can always be re-created. For media generated as a Batch Render, there is no way to trace it in Flame or to an external source. Therefore Flame will ALWAYS include Batch Renders in the archive regardless whether exclude cache and exclude renders are enabled. Flame will preserve any data in the archive that CANNOT be linked or recreated. There is no choice in the matter.

In summary, you can shrink your archive sizes using exclude source cache and TimelineFX renders but it will be bigger than expected if there are lots of Batch renders in the project.

As always, this is my interpretation of how I was taught about archiving and feel to chime in if you have different experiences :slight_smile:

Grant

17 Likes