Gauging system activity on Linux with dstat

Linux provides a number of commands that provide information about disk and system activity along with system and resource stats. One of the most useful commands is dstat. It provides an extensive number of detailed statistics on system activity.

If dstat is not yet installed on your Linux system, you can use a command like one of these given root access:

For Redhat etc, use:

# yum install dstat

For Ubuntu, Debian and related distributions, use:

# apt install dstat

Disk activity

Disk activity includes not just how full your disks are, but how busy they are as they are being read and written to by various commands that are being run. If you want to see just the disk read and write stats, use the -d option. Your output should look something like this:

$ dstat -d
-dsk/total-
 read  writ
   0     0
   0     0
 488k    0
 336k    0
   0     0
   0  8192B
   0     0
 343k 2051k
7687k    0
 246k  425k
1160k  217k
 305k    0

Notice that the output includes one column for disk reads and another for disk writes.

Default dstat options

Were you to use the dstat command with no options, the output would include a lot more details than you might expect and would look more like what you see below.

$ dstat
You did not select any stats, using -cdngy by default.
----total-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai stl| read  writ| recv  send|  in   out | int   csw
  1   1  96   1   0|  16k 1128k|  54B  674B|   0     0 | 195   295
  0   0  99   0   0|   0  4093B|  54B  354B|   0     0 | 129   208
  1   0  99   0   0|   0     0 | 500B  354B|   0     0 | 115   170
  0   0  99   0   0|   0     0 |  54B  354B|   0     0 |  65   111
  1   1  99   0   0|   0     0 |  54B  354B|   0     0 |  99   157
  0   0  99   0   0|   0     0 |  54B  354B|   0     0 |  63   113^c

Notice that this output includes stats usage, disk read and write, network activity, paging and system stats. The output from using dstat without options is the same as using dstat with the “cdngy” options (dstat -cdngy).

To get eight lines of this information in 2-second intervals, add the numeric arguments as shown in this command:

$ dstat 2 8
You did not select any stats, using -cdngy by default.
----total-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai stl| read  writ| recv  send|  in   out | int   csw
  0   0  99   0   0|   0     0 | 298B  541B|   0     0 | 123   197
  0   0  99   0   0|   0     0 |  54B  346B|   0     0 |  64   114
  0   0  99   0   0|   0     0 |  87B  515B|   0     0 |  66   117
  0   0  99   0   0|   0     0 |  54B  346B|   0     0 |  78   125
  1   0 100   0   0|   0     0 | 277B  346B|   0     0 |  62   113
  1   1  99   0   0|   0     0 |3893B  411B|   0     0 | 110   144
  5   0  85  10   0|   0   128k|  54B  346B|   0     0 | 213   152
  1   0  98   0   0|   0   180k|  54B  354B|   0     0 |  92   149

Getting help

Note that the dstat command doesn’t always come with a man page. However, you can get a list of the command’s options and what each option represents using the dstat –help command.

$ dstat --help
Usage: dstat [-afv] [options...] [delay [count]]
Versatile tool for generating system resource statistics

Dstat options:
  -c, --cpu             enable cpu stats
     -C 0,3,total          include cpu0, cpu3 and total
  -d, --disk            enable disk stats
     -D total,sda          include sda and total
  --dm, --device-mapper enable device mapper stats
     -L root,home,total    include root, home and total
  --md, --multi-device  enable multi-device driver stats
     -M total,md-0         include md-0 and total
  --part, --partition   enable disk partition stats
     -P total,sdb2         include sdb2 and total
  -g, --page            enable page stats
  -i, --int             enable interrupt stats
     -I 9,CAL              include int9 and function call interrupts
  -l, --load            enable load stats
  -m, --mem             enable memory stats
  -n, --net             enable network stats
     -N eth1,total         include eth1 and total
  -p, --proc            enable process stats
  -r, --io              enable io stats (I/O requests completed)
  -s, --swap            enable swap stats
     -S swap1,total        include swap1 and total
  -t, --time            enable time/date output
  --time-adv            enable time/date output (with milliseconds)
  -T, --epoch           enable time counter (seconds since epoch)
  --epoch-adv           enable time counter (milliseconds since epoch)
  -y, --sys             enable system stats

  --aio                 enable aio stats
  --fs, --filesystem    enable fs stats
  --ipc                 enable ipcstats
  --lock                enable lockstats
  --raw                 enable rawstats
  --socket              enable socketstats
  --tcp                 enable tcpstats
  --udp                 enable udpstats
  --unix                enable unixstats
  --vm                  enable vmstats
  --vm-adv              enable advanced vm stats

  --list                list all available plugins
  --plugin              enable external plugin by name, see --list

  -a, --all             equals -cdngy (default)
  -f, --full            automatically expand -C, -D, -I, -N and -S lists
  -v, --vmstat          equals -pmgdsc -D total

  --bits                force bits for values expressed in bytes
  --float               force float values on screen
  --integer             force integer values on screen

  --bw, --blackonwhite  change colors for white background terminal
  --color               force colors
  --nocolor             disable colors
  --noheaders           disable repetitive headers
  --noupdate            disable intermediate updates
  --nomissed            disable missed ticks warnings
  -o file, --output=file
                        write CSV output to file

Using other options

The cdngy arguments (as you can tell from the output above) represent cpu, disk, network, paging and system stats. To display used and free memory with dstat, use a command like this:

$ dstat -s
---total---
 used  free
 768k 3783M
 768k 3783M
 768k 3783M
 768k 3783M
 768k 3783M
 768k 3783M

To display top CPU usages, you could use a command like this:

$ dstat -c --top-cpu
----total-usage---- -most-expensive-
usr sys idl wai stl|  cpu process
 20   7  70   1   0|firefox       14
 20  11  55  12   0|firefox
 17   4  74   1   0|firefox      8.5
 20   5  75   2   0|firefox      8.5
 21   6  70   1   0|firefox       14
  2   1  97   0   0|pmdaproc     1.0
  4   2  95   1   0|firefox      1.0
  1   0  97   0   0|Isolated Web 1.0
  2   0  96   0   0|Isolated Web 1.0
  3   2  97   0   0|Isolated Web 1.0

With the dstat -c (CPU) option, dstat displays CPU stats. In the example below, we’re asking for two-second intervals and six reports.

$ dstat -c 2 6
----total-usage----
usr sys idl wai stl
  0   0 100   0   0
  0   0  99   0   0
  1   1  99   0   0
  0   0  99   0   0
  0   0 100   0   0
  1   0  99   0   0

Wrap-up

One key to becoming adept at evaluating system performance is to run commands like these periodically — even when you don’t see the need to question how well a system is running. If you come to know what normal performance looks like for a server, you will have a much easier time spotting problems.

The commands included in this post display some of the most useful dstat command options, but check the options list for others that will help you see the details that you most need to see. The dstat command provides a vast range of options for viewing system and resource stats.