I/O usage analyses: vmstat

Performance Diagnosis
Technical articles
vmstat
uses information from the following files:
  • /proc/meminfo
  • /proc/stat
  • /proc/*/stat
where * represents the process IDs

I/O usage and vmstat
vmstat does more than give virtual memory statistics. vmstat can show:
  • The amount of each type of disk I/O  (read/write) on the system
  • The devices servicing most of the disk I/O

Note, information on other uses of vmstat in these articles:  
   
The modes of vmstat
It is useful to think of vmstat having two modes:
  • Average mode
  • Sampling mode

The first line is average mode.

If delay parameter is specified, subsequent lines are sample mode. vmstat is most useful in sampling mode. 
delay: The amount of seconds delay before the next line is displayed
count: The amount of lines displayed
    
The best arguments for CPU usage analyses are:
vmstat [-D] [-d] [-p partition] [-n] [interval [count]]

The command-line options are:
  • -D: Displays I/O subsystem total statistics. Does not give statistics on individual disks but will give a good idea of how the I/O subsystem is being used. The statistics are the totals since system boot.
  • -d: Displays individual disk statistics at a rate of one sample per interval. The statistics are the totals since system boot.
  • -p partition: The statistics are the totals since system boot.
  • -n: cause the header to be displayed only once rather than periodically. Useful for scripting or redirecting the output for a file that will be analyzed in a spreadsheet. Without -n, the header is displayed periodically to make reading the data easier over time.
  • interval: length of time
  • count: total number of samples to take
    

vmstat Output
vmstat output has a header that takes up two lines. The first line indicate categories of output and the second line the specific fields within those categories.

The categories and their associated fields that are useful for I/O (input/output, i.e., storage usage) analyses are:
  • bi: blocks received from a block device (blocks/s) in the previous interval. A block is usually 1K (see below).
  • bo: blocks sent to a block device (blocks/s) in the previous interval. A block is usually 1K (see below).
  • wa: amount of CPU time spent waiting for I/O


Block Size
Linux block size is usually 1K. The following commands will get you the block size:
blockdev --getbsze
example:
blockdev --getbsze /dev/sda1

fdisk -l
example:
fdisk -l /dev/hda

stat -f
example:
stat -f .

Note, the block size is stored in files that can be examined without root access. Check:
  • /sys/class/block//size
  • /sys/block//queue/physical_block_size
  • /proc/partitions

For example, to see the block size for sda, try:
cat /sys/class/block/sda/size
cat /sys/block/sda/queue/physical_block_size
cat /proc/partitions

The I/O limits for a disk can be found at:
  • /sys/block//alignment_offset
  • /sys/block///alignment_offset
  • /sys/block//queue/physical_block_size
  • /sys/block//queue/logical_block_size
  • /sys/block//queue/minimum_io_size
  • /sys/block//queue/optimal_io_size

vmstat -D
vmstat -D will provide the following statistics:
  • disks: total number of disks
  • partitions: total number of disks
  • total reads: total number of reads that have been requested
  • merged reads: total number of times that reads to adjacent locations were merged to improve performance
  • read sectors: total number of sectors read from disk
  • milli reading: amount of time in ms reading from disk
  • writes: total number of writes that had been requested
  • merged writes: total number of times that writes to adjacent locations were merged to improve performance
  • written sectors: total number of writes that had been requested
  • milli writing: amount of time in ms writing to disk
  • inprogress IO: total number of I/O currently in progress (some versions in vmstat have bug with this)
  • milli spent IO: total number of milliseconds spent waiting for I/O to complete (some versions in vmstat have bug with this)


vmstat -d
vmstat -d will provide the following statistics (by disk in columns broken into 3 sections):
reads
  • total: total number of reads that have been requested
  • merged: total number of times that reads to adjacent locations were merged to improve performance
  • sectors: total number of sectors read from disk
  • ms: amount of time in milliseconds reading from disk
writes
  • total: total number of writes that had been requested
  • merged: total number of times that writes to adjacent locations were merged to improve performance
  • sectors: total number of writes that had been requested
  • ms: amount of time in milliseconds writing to disk
IO
  • cur: total number of I/O currently in progress (some versions in vmstat have bug with this)
  • sec: total number of seconds spent waiting for I/O to complete


vmstat Output That is Not I/O Related
The categories and their associated fields that are useful for CPU analyses are:
  • procs (processes)
    • r: run queue (number of processes waiting for run time)
    • b: number of processes in uninterruptable sleep (i.e., blocked for resources such as I/O, paging, et cetera)​
  • system (number of times kernel switches into kernel code)
    • in: the number of interupts per second
    • cs: the number of context switches per second
  • cpu (percentages of total cpu time)
    • us: user time (running non-kernel code)
    • sy: system time (running kernel code) us: user time (running non-kernel code)
    • id: idle time
    • wa: waiting for I/O
    • st: stolen from a virtual machine

The categories and their associated fields that are useful for Memory and swap analyses are:
  • memory (RAM usage)
    • swpd: amount of virtual memory used
    • free: amount of idle memory
    • buff: amount of memory used as buffers
    • cache: amount of memory used as cache
    • inact: amount of inactive memory (appears with -a option)
    • active: amount of active memory (appears with -a option)
  • swap (pages pulled in and out of swap)
    • si: amount of memory swapped in from disk (/s) (per second)
    • so: amount of memory swapped out from disk (/s) (per second)

Suggestions for Future Learning
This information will be in the second edition of UNIX For Application Support Staff . The ETA for the second edition is December 1, 2016.



Performance Diagnosis
Technical articles