KVM 磁盘缓存模式

本笔记内容无原创，整理自如下资料：

Cache modes

KVM 目前支持的磁盘缓存模式主要有 none，writethrough，writeback，directsync，unsafe 这 5 种。后面 2 种通常很少使用到。

none

I/O from the guest is not cached on the host, but may be kept in a writeback disk cache. Use this option for guests with large I/O requirements.（guest 使用 writeback， host 不使用 page cache，相当于 guest 直接访问主机磁盘。）

这种模式下虚拟机磁盘镜像文件或者块设备会使用 O_DIRECT 语义，则 host 的 page cache 被绕过， I/O 直接在 qemu-kvm 的用户空间 buffers 和 host 的存储设备间发生。因为实际存储设备可能在写数据放到写队列后就上报写完成，虚拟机上的存储控制器被告知有回写缓存（writeback cache），所以 guest 需要下发刷盘（flush）命令来保证数据一致（落盘）。相当于直接访问 host 磁盘，性能不错。

This mode causes qemu-kvm to interact with the disk image file or block device with O_DIRECT semantics, so the host page cache is bypassed and I/O happens directly between the qemu-kvm userspace buffers and the storage device. Because the actual storage device may report a write as completed when placed in its write queue only, the guest’s virtual storage adapter is informed that there is a writeback cache, so the guest would be expected to send down flush commands as needed to manage data integrity. Equivalent to direct access to your hosts’ disk,performance wise.

writethrough

I/O from the guest is cached on the host but written through to the physical medium. （guest 使用 wirtethrough， host 使用 page cache。）

这种模式下会为每次写操作执行 fdatasync（原文是 fsync 与后面 O_DSYNC 不太一致），是一种安全的缓存模式，不用担心丢失数据，但同时也比较慢。这种模式下虚拟机磁盘镜像文件或者块设备被设置成 O_DSYNC 语义，必须等待所有写数据落盘以后才上报写完成。 host 的 page cache 工作在所谓的 writethrough 模式。guest 的存储控制器会被告知没有 writecache，所以 guest 不必下发 flush 命令来保证数据一致。存储设备的行为好像是透过缓存（writethrough cache）。

Writethrough make a fsync for each write. So it’s the more secure cache mode, you can’t loose data. It’s also the slower. This mode causes qemu-kvm to interact with the disk image file or block device with O_DSYNC semantics, where writes are reported as completed only when the data has been committed to the storage device. The host page cache is used in what can be termed a writethrough caching mode. The guest’s virtual storage adapter is informed that there is no writeback cache, so the guest would not need to send down flush commands to manage data integrity. The storage behaves as if there is a writethrough cache.

writeback

I/O from the guest is cached on the host.（guest 使用 writeback， host 使用 page cache。）

这种模式下虚拟机磁盘镜像文件或者块设备不用使用 O_DSYNC 与 O_DIRECT 语义。guest 写数据到达 host page cache 便上报写完成，由 host 的 page cache 管理机制来负责将数据落盘。另外，guest 的虚拟磁盘控制器被告知使用 wirteback 缓存，需要下发 flush 命令来保证数据一致性。类似待由 RAM 缓存的 RAID 控制器。

This mode causes qemu-kvm to interact with the disk image file or block device with neither O_DSYNC nor O_DIRECT semantics, so the host page cache is used and writes are reported to the guest as completed when placed in the host page cache, and the normal page cache management will handle commitment to the storage device. Additionally, the guest’s virtual storage adapter is informed of the writeback cache, so the guest would be expected to send down flush commands as needed to manage data integrity. Analogous to a raid controller with RAM cache.

directsync

Similar to writethrough, but I/O from the guest bypasses the host page cache.（guest 使用 writethrough， host 不使用 page cache）

这种模式下虚拟机磁盘镜像文件或块设置同时使用 O_DSYNC 和 O_DIRECT 语义，绕过 host page cache，等数据落盘才上报写完成。与 writethrough 模式一样， directsync 模式不需要下发 flush 命令，对于不支持 writeback cache 的 guest 很有用（it is helpful to guests that do not send flushes when needed.）。这种模式是最后一种缓存与直接访问的组合语义，是缓存方式的补充。

This mode causes qemu-kvm to interact with the disk image file or block device with both O_DSYNC and O_DIRECT semantics, where writes are reported as completed only when the data has been committed to the storage device, and when it is also desirable to bypass the host page cache. Like cache=writethrough, it is helpful to guests that do not send flushes when needed. It was the last cache mode added, completing the possible combinations of caching and direct access semantics.

unsafe

The host may cache all disk I/O, and sync requests from guest are ignored.

这种模式与前面讨论的 writecache 模式非常相似。该模式的关键点是由 guest 下发的 flush 命令都会被忽略。使用这种模式也就意味着接受 host 故障导致数据丢失的风险，以换取性能。这个模式可以在安装系统的时候使用，但是不应该在生成环境使用。

This mode is similar to the cache=writeback mode discussed above. The key aspect of this unsafe mode, is that all flush commands from the guests are ignored. Using this mode implies that the user has accepted the trade-off of performance over risk of data loss in the event of a host failure. Useful, for example, during guest install, but not for production workloads.

Conclusion

Mode	Host Page Cache	Disk Write Cache	Notes
none	disable	enable	balances performance and safety (better writes)
writethrough	enabled	disable	balances performance and safety (better reads)
writeback	enabled	enable	fast, can loose data on power outage depending on hardware used
directsync	disable	disable	safest but slowest (relative to the others)
unsafe	enable	enable	doesn’t flush data, fastest and unsafest