Just wondering if anyone is using a SDS storage solution in their facility to host their flame framestore?
Pixstor is flame certified but I’d love to hear from someone using it in a real world experience.
I know Weka & Qumulo have been extensively in the cloud for M&E scenarios but has anyone used them locally? Especially interested how Flame would go using it as a framestore. Ceph is another SDS option.
Any SDs insights or anecdotes would be greatly appreciated.
Thanks
I’ve been very interested in Ceph for awhile, and try to stay informed on its development. I love the idea of scale-out instead of scale up. Always wondered what its performance would be in our type of workload. All the other you mentioned, are big money.
In the past we did have a large ZFS nas, which was always about 3-4x slower than HW-raid and XFS on the same exact gear. I love the feature set of ZFS though, the slowness was affecting our productivity and I couldn’t justify it anymore.
We are also looking at Ceph as an option as we have not even begun to talk about money yet. Considering how expensive a SAN is, the expense of an enterprise SDS solution may still end up being more affordable. Ceph seems utilised widely in other industries but I have only heard of one use case in M&E. Another option is LizardFS which is also open source but I have not seen it mentioned much anywhere.
I love the idea of SDS but want to hear stories of people using it successfully.
ZFS is an interesting one. I love the idea of it but anyone I know who has used it has found it really slow and doesn’t live up to the promise of it.
LizardFS is a fork of MooseFS, and from what I recall, has had lots of management/ownership issues recently. There is also https://www.beegfs.io which comes from Fraunhofer in Germany.
Quobyte has continued to look very interesting. Seems on the surface to be something sort of similar architecture wise to Qumulo, Isilon, etc but is truly a software only product, a free version of which is immediately downloadable. They claim to have native drivers for windows, linux and macOS. I believe they may even do some level of testing / qualification with ADSK linux and possibly Flame. I met their crew at NAB a few years back. I’ve just not had the hardware available to play with it yet myself - I believe you need minimum 3 or 4 nodes.
I have stumbled across Quobyte in my searches. There actually seems to be a multitude of SDS solutions out there now.
Would be great to hear some real world use cases of people actually using a SDS solution in a post production environment.
We’ve got both FreeNAS and Open-E NAS on 40gbe here - it’s fine for import/export, archive, even soft-imported frame sequences, etc but wouldn’t want to be stuck running Flame with the framestore located entirely on the NAS.
There is no way I would ever host a Flame framestore on a NAS. SAN definitely (and it works well), SDS with block level capabilities then potentially a yes as well. Was hoping to hear some real world anecdotes. PixStor has Flame certification for storage though so that definitely works.
We’ve been running our central SW off our NAS for a long time with out at all.
What protocol do you use? NFS? Am I right in thinking there is a flash tier in your storage as well, or is it all flash?
NFS…we get line rate 25gigE.
For the past ~6 months we been exclusively on spinning disk NAS, including our S+W stuff, without issue. We are now putting new S+W partition on our upgraded all-SSD server, but general storage is still on disk NAS.
What does your NAS config look like? Something you built I assume? I’ve been curious what might be possible using S+W rather than vanilla NFS via 25gbe or 100gbe.
re: ZFS… past few years have been sort of all over the place, performance has been very much a moving target. That being said, I think it could be worth exploring again in 2023 if HDD’s are still part of our equation.
Pretty standard fair. RockyLinux → Hardware Raid → XFS → NFS
On the all SSD NAS, it is connected 100gig.
On the spinning NAS 50gigE
All the workstations are on 25gigE and get line rate speeds.