# OpenZFS crash course

**URL:** <https://forum.logik.tv/t/openzfs-crash-course/2380>\
**Category:** Hardware Questions\
**Created:** [February 13, 2021, 2:24am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380 "2021-02-13T02:24:47Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 1:44am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/1 "2021-02-13T01:44:20Z")

</div>

Two enclosures - raid 10 openZFS  
Internal NVMe - L2RC  
Internal NVMe - ZIL  
More RAM

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [February 13, 2021, 2:24am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/2 "2021-02-13T02:24:47Z")

</div>

I am intrigued yet naive to OpenZFS. Are you able to elaborate? I’m a beginner in this area.

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 2:36am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/3 "2021-02-13T02:36:49Z")

</div>

Happy to:  
It would be better to explain a little about your system first.  
It will make things more straightforward since it would be a targeted conversation.  
Alan will also chip in I’m sure.  
If you want to keep your specifics private you can send me a direct message or an email.  
[phil\_man@mac.com](mailto:phil_man@mac.com)

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 2:37am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/4 "2021-02-13T02:37:38Z")

</div>

And happy Chinese new year

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [February 13, 2021, 2:41am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/5 "2021-02-13T02:41:19Z")

</div>

System A  
16core Mac Pro 160GB RAM, Big Sur  
Internal 32TB Pegasus j4i RAID5  
Highpoint SD7201-A with x4 2TB NVME RAID0 framestore

x2 Thunderbay 6 enclosures, each with x6 8TB Iron Wolf, currently SoftRAID6+

System B  
28core Mac Pro 160GB RAM, Catalina  
Highpoint SD7201-A with x4 2TB NVME RAID0 framestore

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 2:52am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/6 "2021-02-13T02:52:02Z")

</div>

Ok  
Give me a while to drag up all the system details and build a spreadsheet.  
What kind of reliable throughput (write/read) are you currently getting and how are you networking to date?

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [February 13, 2021, 3:03am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/7 "2021-02-13T03:03:39Z")

</div>

Off the top of my head…

The 32TB Pegasus is 535MB/s write and 160MB/s read  
The Thunderbay RAID 6+ is 512MB write and 1660MB/s read  
The NVME raids are about 6600MB/s read and 7600MB/sec write

10GB networking.

The 32TB Pegasus I’d rather not touch right now…thats got a bunch of stuff on it that keeps me workin.

The Thunderbays I have the data elsewhere local and offsite so those could be the experiment.

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 3:07am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/8 "2021-02-13T03:07:16Z")

</div>

Ok  
Good to know  
Gigabit networking?  
10Gb networking?

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [February 13, 2021, 3:18am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/9 "2021-02-13T03:18:37Z")

</div>

(10GB networking, post edited with this update. Thanks)

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [February 13, 2021, 3:19am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/10 "2021-02-13T03:19:27Z")

</div>

And I can donate 1 of the NVME RAIDs to science this right now, as its not booked coming up this week.

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 3:32am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/11 "2021-02-13T03:32:16Z")

</div>

Are your machines the new format Mac Pro?  
Is networking native or on PCIe cards?

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [February 13, 2021, 3:33am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/12 "2021-02-13T03:33:10Z")

</div>

Yes, 2019 Mac Pro 7,1, native networking.

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 3:34am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/13 "2021-02-13T03:34:25Z")

</div>

Copy that  
Give me a while  
To drag all the bits together  
Shouldn’t be long  
I’m just pouring a large quarantini.

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 3:43am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/14 "2021-02-13T03:43:14Z")

</div>

I’m dragging all of the details out of every Mac dot com  
Then I’ll drag all the details for the storage  
Then I’ll pull the ideas for openZFS  
Give me a little while.  
The good news is that your gear is all documented and tested.  
(Unlike M1 chips)

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [February 13, 2021, 3:45am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/15 "2021-02-13T03:45:28Z")

</div>

I’m on my 3rd gin and tonic and watching West Wing so take all the time you need.

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 6:11am UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/16 "2021-02-13T06:11:34Z")

</div>

It must be that day of the week  
Gin for the win!  
I’m just reading about the thunder bay drives  
Connected via thunderbolt 3 to the macs correct?

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [February 13, 2021, 12:57pm UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/17 "2021-02-13T12:57:52Z")

</div>

Correct.

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 1:52pm UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/18 "2021-02-13T13:52:06Z")

</div>

Ok  
I’m just waking up on this west coast  
Do you have both of thes enclosures connected to one workstation?  
Does your network switch support bonding?  
Do you have flash drives for both of your thunderbolt enclosures?

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [February 13, 2021, 1:56pm UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/19 "2021-02-13T13:56:47Z")

</div>

The 16core has them direct attached. No flash avail in those enclosures.

Bonding capability is unknown.

[https://www.ui.com/unifi-switching/unifi-switch-16-xg/](https://www.ui.com/unifi-switching/unifi-switch-16-xg/)

---

<div class="post-metadata">

**Author:** ![philm](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/philm/32/9286_2.png) [@philm](https://forum.logik.tv/u/philm)\
**Post date:** [February 13, 2021, 2:04pm UTC](https://forum.logik.tv/t/openzfs-crash-course/2380/20 "2021-02-13T14:04:11Z")

</div>

Ok let me look up the switch  
Meantime can you look up compatible flash from OWC (small quantities like 32GB or 64GB) because you can use them as write caches (ZIL for zfs)  
I checked to see if Nic teaming is back on apples menu and it is.  
I harbor an idea that you should be able to get roughly 750 Megabytes/s read and write to and from your enclosures and 20Gigabits per second to and from each Mac.  
Grab the openZFS installer for Big Sur from the openZFS website.  
You may want to sell your ram and buy a bigger set.  
Like 384 gigabytes or something.  
OpenZFS will use all available ram and once data is in Ram it moves faster than nvme .

[Next page](https://forum.logik.tv/t/openzfs-crash-course/2380.md?page=2)
