Ok so as flame does absolutely NOT support relative paths when collaboratively working and most freelancer I want to hire are on MacOS we need to setup a way to have completely mirrored paths across systems.
we also usually give people access to the whole project folder with readOnly permissions for many subfolders and write perms for many other which means they dont want to pull down everything but rsther just a subset of files, they all call this something else like “selective sync”
So far solutions like dropbox, googleDrive, synology drive and many more are working in macOS just fine, you can set a random folder for example one thats on your large raid - and be happy. just need to do a symlink to a common path and bobs your uncle
But Apple is actively working on screwing their customers over, synology allready does not work anymore with selective sync in custom folders on macOS due to a API change!!!
Dropbox is about to follow this, bascially your sync target has to be on your system drive completely killing the option to sync to a large external or internal drive.
this is completely ridicolous, i dont know how I can even work with freelancers on MacOS after this change.
Only option I see is for everyone to get a NAS and let that sync with another NAS.
It mounts each file space as a volume, so you have identical paths as /Volume/filespace/project.
You can set permissions on a user level for which folders they have access to. And metadata is immediately synced, while files are only downloaded and locally cached once access (just-in-time) with the option of pinning files to prime the cache. You definitely want 2-4TB as NVMe storage for the Lucid cache though for best performance, make sure to budget for that.
Autodesk officially supports LucidLink.
Maybe not quite as cheap as some of the others you listed. But should work and is quite user friendly. I use it all the time, though not in a shared Flame setup so far.
The most recent version updated the control app which allows you to mount multiple file spaces at once.
While I had some issues with Lucid lets say 2 years ago and iver gotten reports from users that the wasabi tier is problematic - its defientely on my radar and them requireing special permissions and even to disable T2 security on a mac is definetely interesting -
My main gripe with it is how I can do a hybrid workflow, i have around 10 machines in the office that all need to connect to the same data that lucid has, and i dont want to run lucid client with caches on each machine but rather keep people hitting the NAS while external users do it via lucid .
Yes it does require permission to install - because Apple locks everything down and can’t envision other people might actually build helpful apps that serve real use cases.
That aside, I’ve used it with Wasabi and IBM storage and it has never crashed my system in 3 years on multiple systems.
But if it reduces stress, a mirrored NAS externalizes things from the system.
trying to come up with a hybrid solution using lucid as you said synced NAS.
One thing I am trying is to share the lucid folder from one mac that acts like a NAS to another mac but it makes the first mac just straight up hardcrash…
On a job right now using Lucid.
It’s a fresh install of Lucid and you do not need to disable T2 security to install it(if that’s what you’re referring too). It installed like any other networked mounted type app on Mac with the usual permissions needed to be given.
It’s working great so far I would say. Stable with zero hiccups so far. But I’m not the one managing it just a freelancer.
I don’t really remember any issues when I had used it in the past as well.
Yea I was more going for " them using all the custom permissions stuff and their own filesystem makes them probably not subjective to the thing that dropbox is having problems with" type deal
All this used to be a bigger issue when they still charged you egress, and so having multiple people sync could get expensive. But there no more egress charges, so there’s no downside for everyone connecting to the cloud directly.
They kinda do support it just not on a macOS server (I am trying to find a use for a 2019 mac pro , want to throw in 64Tb of NVME storage and use it as a nas or sort of like “cloud cache” type deal .
I have 10 workstations, all of them accessing the same files for something would mean 10 machines need to download stuff and all have local caches and whatnot that seems like a uneccessary bottleneck. (also considering pricing per user for all the local workstations and the renderfarm machines and everything)
also I have around 60TB in active storage that would be a cool $1200 a month on Lucid not counting snapshots.
id rather have a local NAS in the studio utilizing my 10Gbit infrastructure than to rely on 4Tb cache drives and lucid, i often have projects that come in with like 14Tb of rushes nowadays…
So your use case is freelancers being in the office or remoting into your system, not them using their own machines wherever they are? Or are those 10 workstations on top of the freelancers you have remotely?
LucidLink is definitely seems more designed for the distributed use case. Though that article you shared is intereting.
There’s a guy on our other Discord who does all the system design & support for a large content publisher. He was an early adopter of Lucid Link but has since then tinkered with different setups. Not sure what his current solution is. They have a decent number of editors and post folks. Maybe you guys can chat and share ideas if you want?
basically I have roughly 10 systems in the office, some of them are flames some of them are nuke and some houdini , some are just rendernodes.
Most are used by staff on-site or remoting in (parsec) , currently I have a Synology NAS serving all of those, performance is OK.
Then I have Some freelancers with their own machines at home working remotely for us by syncing files, usually using SynologyDrive, they have selective sync and its been generally great! But now we cant sync to a folder other than ~/library/cloudstorage which is a big issue as nobody has a lot of local storage on a mac. (( And permission management is utter garbage on synology)
So its a very ramp-up-down scenario, might just be 2 freelancers with remote file access but then for a week might be 10 or so, its not very constant at all. hence something like lucid where its all or nothing does not seem that useable in my data workflows.
happy to talk to whoever has great ideas of how to solve this kinda stuff, i am looking into all the options at the moment trying to find something that works a bit better especially for our clients.
I really LOVE the ease of use and sharing stuff from Resillio its SO GREAT , i just cant make it to be fast at all which is obviously problematic
Heard of Resilio but haven’t used it. One of the studios I do work with, we use SyncThing (Syncthing | Downloads) to mirror folders. It’s not integrated into the OS like Lucid, but just sits in user space and sync folders. Can saturate any high speed link. We have synced 3TB folders without any issues.
You would have to manually make sure everything is everyone uses the same path when they setup their local copy. But that’s manageable.
see thats interesting syncthing also uses bitorrent uses the same tech as resillio and i also tried this and i cant get more than 15MB/s throughput which is approx the same as resillio .
Maybe I just need to get resillio to work faster somehow…
This last job I did as a Lucid show. I ran chronos sync on a Mac mini that kept a copy of the project on the Synology. The Synology in turn uses cloud sync to keep synced version in Dropbox.
The key with Lucid for these kids of jobs are basically front ending it on a nfs/cifs server with an NVME pci card acting as its cache. Mac’s sadly don’t work, but Linux boxes work a treat.
Freelancers outside just logged in. Permissions were managed with Lucid groups and everyone mounted at the same location so it wasn’t a thing.
There are other hacks out there with own cloud and whatnot. Hit me up if you want to discuss… short version would be, sync what need to be synced from the synology to Lucid for outside access, not the other way around. Lucid should be your transport not your working space.
thats interesting, glad to hear this worked i have done some synology drive → gdrive sync shenanigans like that and i had a huge fckup / rsync because someone accidentally deleted stuff.
I quiet like this idea of sharing out whats needed only from the syno or whatever to lucid, thats a interesting approach
…you could argue that a whole project is your “atomic unit.” Or you could get more granular and setup separate sync jobs for specific artists but then it also becomes a management thing. How much of your time do you want to waste making those decisions?
Regardless I would pick a lucid pricing model that includes ingress and egress. Makes calculating your costs considerably simpler.
Not to beat a dead horse but if we had user-definable paths as tokens @Slabrie then this conversation becomes moot and @finnjaeger is once again a happy man.