Linux file compression tool that's multicore/multithreaded?

Running RL 8. I’m trying to compress some files and the built-in compression tool in the Files app seems to be running on one core at a time. I want to see every avail core pegged at 100% for something like this. Does anyone have any recommendations? BTW - would love it if the tool has .7z as a file type option.

7z in particular is not very multi-threadable due to the nature of the compression. It builds a ‘dictionary’ of patterns of bytes that it sees multiple times in order to compress them, and it’s not a static dictionary, it changes over time based on what it sees or stops seeing. So it’s a very serialized process that can’t really be broken up into multiple simultaneous tasks. It’s efficient compression, but unfortunately slow because of it.

Good to know! Thanks.

I just use the terminal command to 7z, tar -cf - (folder/) | 7za a -si -mx=7 (folder name.7z)

the mx=# is the compression amount if you set it to 10 is the highest I found 7 to be kind of the sweet spot.

to extract 7za e (filename)

2 Likes

Taking a closer look at this, it seems like 7z has advanced its multithreaded capabilities over the years since I started using it well over a decade ago on Windows :smiley: (when I had a similar question, why is this not utilizing my CPUs very well?) So, very old info in my brain there.

I’m not sure what the ‘Compress…’ function in gnome is doing, if it’s using an old library, or just not enabling things like chunking (which can be threaded) or what… but commandline 7z is drastically faster, even with no complicated switches.

7z a MyArchive.7z SomeDirectoryOrFile

Doing a quick test on a folder full of DPXs, commandline 7z made a smaller archive than ‘Compress’ via the GUI did, and in a fraction of the time.

However 7z (p7zip) is in EPEL, not in the standard distro, so you’d need that to install.

1 Like