I know some of the differences. For example, ZFS leaks some metadata, mostly dataset and snapshots names (which are useless in my case). LUKS encrypts everything, has a better KDF and multiple key slots, and is mature at this point. ZFS uses AES-GCM which is more complex but stronger than AES XTS.
Are there other reasons to use one versus the other? What do you choose?
Openzfs encryption is still rather new and I worry there might be bugs or pitfalls breaking confidentiality, or cause pool corruption.
Any feedback on the implementation quality of ZFS native encryption?
To me, a big benefit of ZFS' rampant layering violation is that it shrinks the surface of weird interactions and edge cases to learn about. You get to know the FS, play with it, and then off you go. Most other solutions require a lot of thought and planning. Perhaps there is a better way to tune the storage setup to exactly what you want, but with ZFS you aren't seduced into getting creative with the architecture.
So I'm in favor of native ZFS encryption due to its usability.
That said, it is not as mature as I'd like, and there is at least one very big issue to know about: I recently sent an encrypted FS with zfs send/receive, and the received copy ended up being unreadable. I'll see if I can find the bug again. So that means you should test your setup precisely.
[edit] Here is the issue: https://github.com/openzfs/zfs/issues/12594
The technical detail from that thread: "This happens because when sending raw encrypted datasets the userspace accounting is present when it's not expected to be. This leads to the subsequent mount failure due a checksum error when verifying the local mac. I tried unsuccessfully to tackle this in #11300. See also: #10523, #11221, #11294.
Edit: If you have critical data lost due to this case I could help you recover them."
And there's this comment [https://github.com/openzfs/zfs/issues/12594#issuecomment-929...] which has a lot more pointers to recent problems.
There is an outstanding issue [0] that I've encountered on two separate machines running in this setup. The issue occurs when using ZFS Native Encryption + (NVMe?) SSDs. The issue appears to be snapshots getting corrupted (or maybe failing to create?) occasionally. It happens roughly 1/1000 snapshots in my case. I take a lot of snapshots so this occurs weekly.
I haven't lost any data yet but this issue is annoying because I get spooky alerts from ZFS warning me about "potential data corruption". To clear them I have to manually intervene by deleting the corrupt snapshots, restart the machine, then scrub.
What's most annoying is that this breaks ZFS send/recv until I intervene. send/recv was the whole reason I went with native encryption instead of LUKS in the first place.
Again, no data corruption so far (if you don't count the snapshots that get lost, which I don't, because I have so many of them). Just very annoying and tedious.
As others have said, I like the benefits, which are easy send/receive, etc (but mind the bugs some other commenter mentioned).
I will also give a few drawbacks / points in favor of using LUKS.
In my case, Arch usually has up-to-date kernels. These are not always compatible with the latest stable OpenZFS release. Issues go from benign: ultra-high CPU usage so my 980 PRO ssd is slower than spinning rust to the module not compiling at all.
Another drawback is that booting with systemd in initramfs doesn't support zfs native encryption. There's no support for using a TPM [0]. So you have to type in your password every time you boot up the computer. If you want hibernation to work, you need a separate swap, encrypted separately, so that's two passwords to type in.
Systemd recently got support for TPM so can automatically decrypt devices. I have this set up on a separate machine (with ext4) and it's nice.
---
[0] I know not everyone likes the idea of automatically decrypting the drive, but that's another debate. If your threat model allows for this, it can be done with LUKS but not with ZFS.
I don't think I'd use it on cold storage though, I like using it on systems I daily drive, such as workstations and laptops, where I feel sacrificing convenience is not worth it, and where I don't want to lose the option to not be able to select what's encrypted and what's not (for instance, I don't want or need for all of my VMs ZVOLs to be encrypted). With ZFS you can pick what gets encrypted, and with what key, without having to fumble around with loopback devices and other strange machinery.
In my experience it is very reliable, I didn't really feel any performance degradation after moving my system on top of encrypted datasets on any of my systems. I also happen to have to use every once in a while a system installed on an external USB-C SSD (with Arch+ZFS), and it works very well without any performance hiccup even with encryption on.
The only time I had to fumble with the dataset was when encryption still wasn't stable and I had to rsync everything to newer format datasets - something around 2018 I guess? Still, it was a pretty straightforward experience. I never had any other issue with encryption after it got moved to stable (3 years ago I think?), daily driving several (personal and work) systems.
On my cold storage, such as drives I backup my stuff onto, I still use LUKS + XFS/ext4/ZFS because it's tested and solid though - I don't really feel that moving disks I often keep offline to ZFS native encryption would help at all. I honestly worry more about them being misplaced or lost, than convenience.
I can't speak to the metadata leakage, but from my grumpy perspective the fewer layers in my IO path the better. Disks should be boring (ish).
That being said, I do use it on a laptop where losing the data wouldn't be a big deal, but having it easily compromised would be. It was just so much easier to transition to encrypted datasets; the whole thing was done with maybe 10 minutes of downtime.
Shameless plug but i wrote a nice cli script to manage luks containers.
I'm not sure if complex/stronger is the right phrase to use here. If someone wants to learn more, this is a nice summary of differences https://crypto.stackexchange.com/questions/84188/is-xts-alwa...
ZFS's encryption is also certainly not as thorough as LUKS is. Depends on your threat model if that's a problem for you.
LUKS and other FDE schemes tend not to have the ability to have fine grained encryption keys. If you have raw access to the LUKS device on which your filesystem resides, you can recover plaintext of blocks that have not been overwritten or erased via TRIM or similar.
zfs on a self encrypting drive seems like a good combination. Even if the Opal implementation is garbage (see CVEs from a couple years back) the exposure is minimal because zfs is encrypting data. My measurements of performance impact IO with and without encryption suggest there is no performance nor increased cpu consumption with self-encrypting drives.
More information at: https://blog.heckel.io/2017/01/08/zfs-encryption-openzfs-zfs...
With ZFS there are other things to consider as well, but I would recommend looking at an article like here https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-... that goes into much detail.
Like always it mostly depends on your use case/scenarios.
It works well but be aware that SWAP specifically on encrypted ZFS will lead to deadlock: https://github.com/openzfs/zfs/issues/7734
Thus I have planned to move the swap back to LUKS. If I cannot unlock everything by entering only 1 password entering due to that, I might move everything back to LUKS for that reason.
If you stream your ZFS data from one host to another then the encrypted filesystem remains encrypted at the destination.
If your encryption is done outside of ZFS then the data on the destination is in plaintext.
This might seem obvious to those experienced but to a newcomer it caught me off guard and the interaction between recursive send/receive and encryption was more complicated than I had expected.
I should add that I haven’t looked into this for a while and could be completely wrong but it’s worth researching, at least, because my spideysense is definitely tingling to the tune of something about this was odd and worth knowing about.
This post might give a better hint to the actual details: https://www.truenas.com/community/threads/cant-zfs-send-betw...
Sorry to not be more authoritative. Bottom line: if you are going to use send/receive, spend some time to build a run book for your use case sooner rather than later.
Arch has a good wiki page comparing LUKS vs ZFS native vs a few others: https://wiki.archlinux.org/title/Data-at-rest_encryption
And more discussion of differences over at /r/zfs: https://www.reddit.com/r/zfs/search?q=zfs+luks&restrict_sr=o...
I guess it is about to proceed onto zfs native encryption.