Asus Energy settings

Model tested: RS700-E12-RS12U

Problem: Poor NVMe pool(ZFS) performance.

Cause: CPU energy settings Energy Performance Bias

Energy Performance Bias (EPB) setting in ASUS server BIOS (often found under Intel/AMD Power Management options) dictates how the CPU balances performance and power efficiency. It provides a "hint" to the processor's microcode to prefer higher frequency (performance) or lower voltage/reduced turbo (power saving). 

Solution: Disable energy saving features in BIOS to increase NVMe performance.

Socket Configuration -> Energy Perf Bias -> Workload Configuration

CPU - Advanced PM Tuning

Latency Optimized Mode

Refer to ticket: https://ahead-it.kayako.com/agent/conversations/135060?postId=1040711

Disabling energy savings features decreases the latency and increases IOPS.