ZFS Configuration Guide for Optimal Performance
ZFS (Zettabyte File System) is a revolutionary filesystem that combines the roles of a traditional filesystem and volume manager. This guide covers everything you need to know to configure ZFS for optimal performance in your homelab.
What Makes ZFS Special?
ZFS offers unique features that set it apart:
- Data Integrity: Built-in checksumming detects and corrects silent data corruption
- Snapshots: Instant, space-efficient point-in-time copies
- RAID-Z: Software RAID with better performance than traditional hardware RAID
- Compression: Transparent data compression to save space
- Deduplication: Eliminate duplicate data blocks
- ARC Cache: Intelligent caching system for better performance
System Requirements
Minimum Requirements
- RAM: 8GB (16GB+ recommended)
- CPU: Any 64-bit processor
- Disks: At least 2 drives for redundancy
Recommended Configuration
- RAM: 1GB per TB of storage (for deduplication: 5GB per TB)
- CPU: Multi-core processor for better performance
- Disks: Enterprise-grade drives with power loss protection
Installing ZFS
Ubuntu/Debian
sudo apt update
sudo apt install zfsutils-linux
Proxmox
ZFS is included by default:
zfs version
Creating ZFS Pools
Single Disk (No Redundancy)
⚠️ Not recommended for production data
sudo zpool create tank /dev/sdb
Mirror (RAID-1)
Two disks with complete redundancy:
sudo zpool create tank mirror /dev/sdb /dev/sdc
RAID-Z1 (Similar to RAID-5)
Minimum 3 disks, survives 1 disk failure:
sudo zpool create tank raidz1 /dev/sdb /dev/sdc /dev/sdd
RAID-Z2 (Similar to RAID-6)
Minimum 4 disks, survives 2 disk failures:
sudo zpool create tank raidz2 /dev/sdb /dev/sdc /dev/sdd /dev/sde
RAID-Z3
Minimum 5 disks, survives 3 disk failures:
sudo zpool create tank raidz3 /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf
ZFS Pool Recommendations
Home/Small Office
- 3-4 drives: Use RAID-Z1
- 5-8 drives: Use RAID-Z2
- 9+ drives: Consider RAID-Z3 or multiple RAID-Z2 vdevs
Enterprise/Critical Data
- Always use RAID-Z2 or RAID-Z3
- Add hot spares
- Use enterprise drives
Optimizing ZFS Performance
ARC Cache Tuning
Limit ARC cache size (in /etc/modprobe.d/zfs.conf):
# Limit ARC to 8GB
options zfs zfs_arc_max=8589934592
# Minimum ARC size
options zfs zfs_arc_min=4294967296
Apply changes:
sudo update-initramfs -u
sudo reboot
Enable Compression
LZ4 compression is fast and saves space:
sudo zfs set compression=lz4 tank
Configure Record Size
For databases (small random I/O):
sudo zfs set recordsize=16K tank/database
For media files (large sequential I/O):
sudo zfs set recordsize=1M tank/media
Disable Access Time Updates
Improves performance:
sudo zfs set atime=off tank
Enable ZFS Auto-Trim
For SSDs:
sudo zpool set autotrim=on tank
Working with Datasets
Create Dataset
sudo zfs create tank/documents
sudo zfs create tank/photos
sudo zfs create tank/videos
Set Quotas
sudo zfs set quota=100G tank/documents
Set Reservations
sudo zfs set reservation=50G tank/database
List Datasets
zfs list
Snapshots
Create Snapshot
sudo zfs snapshot tank/documents@backup-2024-01-15
List Snapshots
zfs list -t snapshot
Rollback to Snapshot
sudo zfs rollback tank/documents@backup-2024-01-15
Delete Snapshot
sudo zfs destroy tank/documents@backup-2024-01-15
Automated Snapshots
Install Sanoid for automated snapshot management:
sudo apt install sanoid
Configure /etc/sanoid/sanoid.conf:
[tank/documents]
use_template = production
recursive = yes
[template_production]
frequently = 0
hourly = 24
daily = 7
monthly = 12
yearly = 0
autosnap = yes
autoprune = yes
Send and Receive (Replication)
Send Snapshot to File
sudo zfs send tank/documents@backup | gzip > backup.gz
Receive Snapshot from File
gunzip -c backup.gz | sudo zfs receive tank/restore
Incremental Replication
Initial full send:
sudo zfs send tank/documents@initial | ssh user@backup-server \
sudo zfs receive backup/documents
Subsequent incremental sends:
sudo zfs send -i @initial tank/documents@today | ssh user@backup-server \
sudo zfs receive backup/documents
Monitoring ZFS
Pool Status
sudo zpool status
Pool I/O Statistics
sudo zpool iostat -v 2
ARC Statistics
cat /proc/spl/kstat/zfs/arcstats
Or use arc_summary:
sudo arc_summary
Dataset Usage
zfs list -o name,used,avail,refer,mountpoint
Check for Errors
sudo zpool status -v
Maintenance
Scrub Pool
Regular scrubs verify data integrity:
sudo zpool scrub tank
Check scrub progress:
sudo zpool status
Schedule monthly scrubs in crontab:
# Run on the first Sunday of each month at 2 AM
0 2 1-7 * * [ "$(date +\%u)" = "7" ] && /sbin/zpool scrub tank
Replace Failed Drive
sudo zpool replace tank /dev/sdb /dev/sdf
Monitor rebuild:
watch -n 5 zpool status
Add Hot Spare
sudo zpool add tank spare /dev/sdg
Troubleshooting
Pool Won't Import
Force import:
sudo zpool import -f tank
Degraded Pool
Check status:
sudo zpool status -v
Clear errors (if transient):
sudo zpool clear tank
Out of Space
Check space usage:
zfs list -o name,used,avail,refer
zfs list -t snapshot
Delete old snapshots:
sudo zfs destroy tank/documents@old-snapshot
Performance Issues
Check ARC hit rate:
sudo arc_summary | grep "Hit Rate"
Check pool fragmentation:
sudo zpool list -o name,fragmentation
Best Practices
Do's
✅ Use ECC RAM for data integrity ✅ Regular scrubs (monthly) ✅ Enable compression (lz4) ✅ Take frequent snapshots ✅ Test your backups ✅ Monitor pool health ✅ Use whole disks, not partitions
Don'ts
❌ Never exceed 80% pool capacity ❌ Don't use RAID-Z with different sized drives ❌ Avoid deduplication unless you have 5GB RAM per TB ❌ Don't expand RAID-Z vdevs (not supported) ❌ Never use consumer drives without testing ❌ Don't ignore scrub errors
Advanced Features
Encryption
Create encrypted dataset:
sudo zfs create -o encryption=aes-256-gcm \
-o keyformat=passphrase tank/secure
Deduplication
⚠️ Requires significant RAM
sudo zfs set dedup=on tank/vms
NFS Shares
sudo zfs set sharenfs=on tank/share
sudo zfs set sharenfs="rw=@192.168.1.0/24" tank/share
SMB Shares
sudo zfs set sharesmb=on tank/share
Performance Benchmarks
Test sequential write:
dd if=/dev/zero of=/tank/testfile bs=1M count=10000
Test sequential read:
dd if=/tank/testfile of=/dev/null bs=1M
Use fio for comprehensive testing:
sudo apt install fio
fio --name=seqwrite --rw=write --bs=1M --size=10G \
--filename=/tank/testfile
Conclusion
ZFS provides enterprise-grade storage features for your homelab. With proper configuration and maintenance, it delivers exceptional data integrity and performance.
Key takeaways:
- Use RAID-Z2 for important data
- Enable LZ4 compression
- Schedule regular scrubs
- Take frequent snapshots
- Monitor pool health
- Never exceed 80% capacity