I’ve spent some time exploring and studying the use and configuration of VMware Flash Read Cache (vFRC) and its benefits.  These are my notes.
Useful Resources
On a guest virtual machine, vFRC is configured in Disk configuration area. Â The virtual machine needs to be on version 10 hardware. Â vSphere needs to be minimum version 5.5.
Benchmarks
I took a baseline benchmark of a simple Windows Server 2016 virtual machine that had a thin provisioned 20GB disk using DskSpd (formerly sqlio). Â The virtual machine disk disk is connected to an IBM DS3400 LUN with 4 x 300GB 15k RPM disks in RAID-10.
Baseline Virtual Machine
- OS: Windows Server 2016
- vCPU: 1
- vRAM: 4GB
- SCSI Controller: LSI Logic SAS
- Virtual Disk: Â 40GB thin provisioned
- Virtual machine hardware: Â vmx-10
- Virtual Flash Read Cache: 0
Some notes before running a test. Â This is geared toward SQL workloads and identifies the type of I/O for the different SQL workload.
DskSpd test
diskspd.exe -c30G -d300 -r -w0 -t8 -o8 -b8K -h -L E: estfile.dat
Results of testing.
Command Line: diskspd.exe -c30G -d300 -r -w0 -t8 -o8 -b8K -h -L E: estfile.dat Input parameters: timespan: 1 ------------- duration: 300s warm up time: 5s cool down time: 0s measuring latency random seed: 0 path: 'E: estfile.dat' think time: 0ms burst size: 0 software cache disabled hardware write cache disabled, writethrough on performing read test block size: 8192 using random I/O (alignment: 8192) number of outstanding I/O operations: 8 thread stride size: 0 threads per file: 8 using I/O Completion Ports IO priority: normal Results for timespan 1: ******************************************************************************* actual test time: 301.18s thread count: 8 proc count: 1 CPU | Usage | User | Kernel | Idle ------------------------------------------- 0| 99.23%| 7.05%| 92.18%| 0.77% ------------------------------------------- avg.| 99.23%| 7.05%| 92.18%| 0.77% Total IO thread | bytes | I/Os | MB/s | I/O per s | AvgLat | LatStdDev | file ----------------------------------------------------------------------------------------------------- 0 | 7940784128 | 969334 | 25.14 | 3218.43 | 2.471 | 22.884 | E: estfile.dat (30GB) 1 | 8152604672 | 995191 | 25.81 | 3304.28 | 2.401 | 22.211 | E: estfile.dat (30GB) 2 | 8116256768 | 990754 | 25.70 | 3289.55 | 2.408 | 22.080 | E: estfile.dat (30GB) 3 | 8180006912 | 998536 | 25.90 | 3315.38 | 2.394 | 22.936 | E: estfile.dat (30GB) 4 | 8192147456 | 1000018 | 25.94 | 3320.30 | 2.395 | 22.569 | E: estfile.dat (30GB) 5 | 8283185152 | 1011131 | 26.23 | 3357.20 | 2.375 | 21.607 | E: estfile.dat (30GB) 6 | 7820320768 | 954629 | 24.76 | 3169.60 | 2.508 | 21.745 | E: estfile.dat (30GB) 7 | 7896784896 | 963963 | 25.00 | 3200.59 | 2.479 | 21.981 | E: estfile.dat (30GB) ----------------------------------------------------------------------------------------------------- total: 64582090752 | 7883556 | 204.49 | 26175.34 | 2.428 | 22.258 Read IO thread | bytes | I/Os | MB/s | I/O per s | AvgLat | LatStdDev | file ----------------------------------------------------------------------------------------------------- 0 | 7940784128 | 969334 | 25.14 | 3218.43 | 2.471 | 22.884 | E: estfile.dat (30GB) 1 | 8152604672 | 995191 | 25.81 | 3304.28 | 2.401 | 22.211 | E: estfile.dat (30GB) 2 | 8116256768 | 990754 | 25.70 | 3289.55 | 2.408 | 22.080 | E: estfile.dat (30GB) 3 | 8180006912 | 998536 | 25.90 | 3315.38 | 2.394 | 22.936 | E: estfile.dat (30GB) 4 | 8192147456 | 1000018 | 25.94 | 3320.30 | 2.395 | 22.569 | E: estfile.dat (30GB) 5 | 8283185152 | 1011131 | 26.23 | 3357.20 | 2.375 | 21.607 | E: estfile.dat (30GB) 6 | 7820320768 | 954629 | 24.76 | 3169.60 | 2.508 | 21.745 | E: estfile.dat (30GB) 7 | 7896784896 | 963963 | 25.00 | 3200.59 | 2.479 | 21.981 | E: estfile.dat (30GB) ----------------------------------------------------------------------------------------------------- total: 64582090752 | 7883556 | 204.49 | 26175.34 | 2.428 | 22.258 Write IO thread | bytes | I/Os | MB/s | I/O per s | AvgLat | LatStdDev | file ----------------------------------------------------------------------------------------------------- 0 | 0 | 0 | 0.00 | 0.00 | 0.000 | N/A | E: estfile.dat (30GB) 1 | 0 | 0 | 0.00 | 0.00 | 0.000 | N/A | E: estfile.dat (30GB) 2 | 0 | 0 | 0.00 | 0.00 | 0.000 | N/A | E: estfile.dat (30GB) 3 | 0 | 0 | 0.00 | 0.00 | 0.000 | N/A | E: estfile.dat (30GB) 4 | 0 | 0 | 0.00 | 0.00 | 0.000 | N/A | E: estfile.dat (30GB) 5 | 0 | 0 | 0.00 | 0.00 | 0.000 | N/A | E: estfile.dat (30GB) 6 | 0 | 0 | 0.00 | 0.00 | 0.000 | N/A | E: estfile.dat (30GB) 7 | 0 | 0 | 0.00 | 0.00 | 0.000 | N/A | E: estfile.dat (30GB) ----------------------------------------------------------------------------------------------------- total: 0 | 0 | 0.00 | 0.00 | 0.000 | N/A %-ile | Read (ms) | Write (ms) | Total (ms) ---------------------------------------------- min | 0.068 | N/A | 0.068 25th | 0.261 | N/A | 0.261 50th | 0.274 | N/A | 0.274 75th | 0.305 | N/A | 0.305 90th | 0.413 | N/A | 0.413 95th | 3.097 | N/A | 3.097 99th | 57.644 | N/A | 57.644 3-nines | 198.563 | N/A | 198.563 4-nines | 995.725 | N/A | 995.725 5-nines | 1896.496 | N/A | 1896.496 6-nines | 1954.282 | N/A | 1954.282 7-nines | 1954.318 | N/A | 1954.318 8-nines | 1954.318 | N/A | 1954.318 9-nines | 1954.318 | N/A | 1954.318 max | 1954.318 | N/A | 1954.318
The important part of this shows that at 204MB/s throughput and 26k IOPs, I had average 2ms latency.
thread | bytes | I/Os | MB/s | I/O per s | AvgLat | LatStdDev ---------------------------------------------------------------------------------------- total: 64582090752 | 7883556 | 204.49 | 26175.34 | 2.428 | 22.258
Here is a view from my monitoring software, essentially validating the latency.
A good starting point for SQL workload testing would be something like:
diskspd –b8K –d30 –o4 –t8 –h –r –w25 –L –Z1G –c20G D:iotest.dat > DiskSpeedResults.txt
At this point, I just need to get the SSD installed on the host and test VMware Flash Read Cache.
To be continued…