With xsan beign free and me having more mac minis than i can use, i am still looking at a shared projectserver/framestore.
As it will be a mac based thing with thunderbolt NVME storage the mini has another 3 ports to perfectly serve files to the 3 flames we have using thunderbolt networking (~20Gbit/s) .
Its just for timeline caches and project metadata so nothing massive at all, no cached media as we use lucid/unmanaged for that this is for timelines only and thats it no flame generated media, just a temp-cache.
As all is mac based i do have free access to xSAN , so now I am contemplating which option would be the fastest for a shared cache.
I would probably also use this for a shared resolve cache because why not.
xSAN, data over thunderbolt-networking , metadata over 10Gbit/E, “shared” framestore setup with nvme raid as lun.
non-shared framestore host just has a local framestore thats shared via nfs using flames built-in tools , no mounting no nothing will all be taken care of by flame.
afp/smb filesharing in macOS instead of xSAN.
Personally i am intrigued about setting up the xsan as its probably the best performer in terms of access latency ?
I have mixed feeling about Stornext… most of them are negative and legacy as I haven’t used it in forever.
That being said. This shit intrigues me
If you’re up for managing an mdc (and honestly a failover mdc), a sep vol for metadata and a completely separate network for metadata traffic to keep that latency low, then sure, why not.
It’s a bit Rube Goldberg but but as long as you don’t need more than 3 clients and they’re all less than 2 meters from the mdc…
Who’s with me? Who wants to see @finnjaeger hot-rod these minis?
the good thing is - i dont loose anything even if I throw the whole san out of the window, so I can totally yolo a raid 0 thudnerblade as central framestore.
That’s brave in this day and age. I had considered building one at one stage myself but the lack of support, or even a community to run ideas past, made it a non-starter.
If you’ve got the time and patience it would be a fun project but I wouldn’t want to rely on it for commercial purposes. I’d be more inclined to build my own Linux hosted NAS setup a bit like what Alan has done. Or look at building a TrueNAS flash setup. Cheaper to setup and probably just as performant but with a massive community to lean on when you get into trouble.
Another alternative to XSAN is looking at an open source distributed file system such as MooseFS or LizardFS. If you were prepared to pay for it, Quobyte is really impressive.
Think it would mean installing Linux as your OS but the Mac Minis would be recoverable if it didn’t work.
hahah its just a cache so it doesnt REALLY matter, i just wonder of the brunt force performacne benefits vs running SMB/NFS
its 4 minis
it might not be worth it at all. it might be, idk , last time i have seen XSAN it was like 10 years ago, but they just have that stuff free integrated and so… why not
I am planning on putting all mac studios in my server rack and then run thunderbolt fibre to the desks with a thunderbolt dock thingy.
Don’t make it complicated. Using simple Hardware XFS raid on linux with NFS4 you can get line rate speed of whatever your network hardware supports. In fact I just did testing on this yesterday, and we consistently, and easily get line rate transfers to/from our NASes with our workstation connected at 25gb ethernet. No licenses, all open source, on used hardware from ebay. You can also use SAMBA to server an SMB share for mac & win if you prefer that type of network mount (although my speed tests were NFS).
I’d also recommend using an internal PCI card instead of a thunderbolt dock.
mac studios only have thunderbolt - anyhow linespeed is one thing but latency another.
I want it complicated hahah
But hey i just kinda installed flame in the mac mini and trying out remote project and just junping between uncached prores files feels like utte rgarbage lots or wait and freezes.
A lot of SDS solutions have native Linux and Mac clients so SMB not always an issue.
I’ve tested running Flame on a Mac connected via SMB to a SDS SSD NAS and latency was a non-issue. That’s using PixStor which is certified for Flame.
Can understand /appreciate the idea of setting up Xsan just for the challenge of it though.
I ran an Xsan for about 10 years and it worked really well. I had storeNext licenses for my linux flames.
If you want an old 4Gb/sec Xsan you can pick mine up in Santa Monica. 32 Hard drives! The flame had 4 connections to the switch, so it was getting 1.5GB/sec, which was huge back in the day. Macs with 2 ports were getting 800MB/sec.
It probably still works.
A shared framestore is the bomb.
using NFS I get 1.6GB/s to my client, thats the built in NFS in flame, cant manage to get good speeds at all via SMB somehow disabling smb signing doesnt seem to work on the mac host… lame
