Lucid vs...?

Sysadmins Remote Workflows
1

We’re trialing LucidLink right now. Any feedback from those using it in production? I do recall some grumpy posts last year when it went down and even though you had pinned files, without authentication, couldn’t work.

SuiteStudios seemed pretty robust in our tests. Same situation with having to ping an authentication server every 10 mins or so.

2

I run a very small company and use Lucid. Due to the nature of file sizes these days I was paying around 450 dollars a month. With the help of Claude I have now setup a 16TB drive on a Mac mini 10gb tunnelling though nextcloud free subscription. My artist have their own private folder on their desktops and for me, and them it functions exactly as Lucid does. I still have Lucid for a while but now mainly using Nextcloud setup completely free of charge after the initial drive purchase. Not sure if this if use but food for thought.

3

And if it helps I can ask Claude to summarise the process.

4

I’m curious! Feels like ‘the way to go’..

5

The big idea: You wanted your own private version of Dropbox/Google Drive — one you own outright, that keeps all your footage and files on your own hardware instead of paying a company to store it in the cloud, but still lets your team and clients get to it from anywhere.

The pieces, in order:

  1. The Mac mini is the server. It’s just always-on, plugged into your 16TB hard drive, quietly doing the work in the background — nobody needs to be sitting at it.
  2. Nextcloud is the actual “Dropbox” software. It’s free, open-source software that runs on that Mac mini and does everything Dropbox does: accounts, folders, permissions, a website you log into, apps for your phone/computer. The difference is you own it and you control where the data physically lives, instead of renting space from someone else.
  3. Files are organized into shared folders. Each piece of work gets its own folder with its own guest list — you decide who has full access and who has more limited access, and only those people can see that folder.
  4. Cloudflare Tunnel is how people reach it from outside. Normally, running your own server means poking a hole in your home/office network’s firewall — risky. Instead, the Mac mini opens a secure outbound connection to Cloudflare, and Cloudflare hands it a proper web address that anyone can safely reach, without ever exposing your network directly to the internet.
  5. The desktop app makes it invisible. Once installed and logged in, Nextcloud just looks like a normal folder in Finder. Drop a file in, it uploads. Someone else with access sees it appear on their machine too. That’s the whole “magic” — everything underneath is just handling the account/permissions/sync side quietly.
  6. Backups run automatically every night, just for the account records and settings — the actual files already live safely on the big drive itself.

So in one sentence: it’s your own private, self-hosted Dropbox, running on hardware you own, reachable securely from anywhere, with your own rules about who sees what.

6

Nice work.

Tell your robot to make an exhaustively detailed guide for the technically proficient.

Tell your robot to make a step by step task checklist for non technical people.

Tell your robot to bundle the solution as a python based installer with prompts for localization and personalization.

Give it to the world.

Or not.

:smiley:

7

I’m guessing a Claude skill might do it. Is that what we’re thinking? Let me look into it.

8

We’ve used it in-house for 3 years. Since it’s pay per TB, I don’t recommend using it on its own as an archiving directory. You can layer it on to a NAS though. For us, it’s most effective as a project folder.

LL’s support team has always been really responsive and proficient with us. As with most enterprise software, that support is what we’re really paying for. If you pull some threads on Claude you’ll learn you can build something similar in Swift for Mac with the same macFUSE backend: S3 or IBM object storage + a mounting layer + a caching layer + a metadata layer + a GUI with Finder integration. If you junk that up, you’ll have to hire yourself as IT support.

I remember @philm opining that LucidLink on macOS doesn’t enable symlinks, and it’d be a lot cooler if it did.

9

Thanks all, we’re happy to pay someone else to be our IT-department-cloud-storage-service.

LL working as advertised. Does seem a bit slower that SuiteStudios overall, pushing up to cloud, live streaming and jumping around folders hunting and pecking for that “one file” no one remembers. Bounces around a lot in network speeds, I caught glimpses of 2+gbps just a few times.

One drawback we want to talk to them about: your cache is capped at 10tb. Normal practice for us would be to pin our active jobs (so you’re not relying on good internet, etc), and a handful of clients can be 3-5TB per job.

10

I’m a big fan of one lucid filespace per job plus a single admin share. Paired with Logik project or some other single project root symlinking strategy and you’re gold.

11

Would like to pick you brain a little on this if we go ahead!

12

For sure. Anytime…

13

So how performant is LucidLink and does the local cache management work well or is it a PITA that seems to flush the wrong clips but keep the ones you don’t actually need? If you’re working with lossless media such as PIZ/ZIP exr sequences, how long does it take until you get the exrs locally cached?

If you are after a self hosted & maintained storage environment with similar properties to LucidLink then I am experimenting with something but am a long way off properly setting it up and pressure testing it. Happy to talk through the concept if anyone else wants to explore it. I have a lot of extra hurdles to satisfy due to Studio Content Security protocols and user management needs but if you’re working on TVCs and are happy dealing with POSIX permissions I can talk through if you PM me.

14

I should also add here this which I have stumbled across. I have zero idea if it is any good or not though:

15

It doesn’t go into the depths of setup and use of Lucid Link, but @philm shows it at work when he was demoing his amazing LogikProjekt

16

Team Fancy Underpants: assemble!

We are about 8 months into running logik projekt over lucidlink classic on I think the ibm cloud for 8 artists. It is awesome.

We use 4k-ish dwaa exrs for just about everything because we don’t have to deal with Netflix hairy eyeball qc. Lucid cache size hasn’t been an issue at all for us but we are 95% ads/short form. Not sure how it would hold up to trying to do a big VFX-heavy feature in its entirety, but it works really well for our mostly-wfh distributed team. We tend to use pinning sparingly, mostly when an artist knows they’re going to be rolling into a job later. Keeping it on during active production adds a lot of unneeded overhead. I don’t need my teammate’s neat video etc pre-renders, just her hero comp render. I do not notice the caching at all once the footage is local on my nvme

I have a 2.5Gbit/450Mbit cable connection and can sustain 80-120MB/s down and about 40MB/s up. It can take a bit (20 mins depending) to get everything down initially, but since everything is frames you can do stuff like just tell flame to render the timeline or the batch you need to revise and it’ll just pull down what it needs as it needs it, and is then subsequently fast. You don’t want a guy with 40Mbit upload doing your initial publishing or shipping, but aside from that it’s been surprisingly flexible

The rest of the company (offline, audio, mograph, cg) are on lucid as well with some caveats like the Houdini guy needs to sim locally. We ditched the on-prem render farm and send everything big to a cloud rendering service now.

17

Lucid is great but the 4x jump in price from us Wasabi-level $20/TB/mo holdouts means Lucid is dead to us moving forward. We’re Shade customers, mid transition. Shade’s dev team wrote the Rocky installer for us, and have put out way more features in the last 5 months since NAB then we ever saw with Lucid the past 3 years combined.

Shade is R2 backed which has zero egress hence the far more reasonable pricing structure.

But yeah, the whole FUSE-mount situation with local pinnable bits is solid and helps us with out geographically distributed workforce.

You still have to have a local backup, so fire up some serverpartsdeals.com 28TBs in a Mini R and localize that shit for nearline access and a rainy day.