FM / doc

/usr/share/doc/bpfcc-tools/examples/doc/

/usr/share/doc/bpfcc-tools/examples/doc

Name	Size	Mode	Actions
lib/	-	0755	rm
argdist_example.txt	23029	0644	edit dl rm
bashreadline_example.txt	882	0644	edit dl rm
bindsnoop_example.txt	4530	0644	edit dl rm
biolatency_example.txt	24021	0644	edit dl rm
biolatpcts_example.txt	3040	0644	edit dl rm
biopattern_example.txt	1406	0644	edit dl rm
biosnoop_example.txt	3549	0644	edit dl rm
biotop_example.txt	9326	0644	edit dl rm
bitesize_example.txt	5100	0644	edit dl rm
bpflist_example.txt	2179	0644	edit dl rm
btrfsdist_example.txt	9545	0644	edit dl rm
btrfsslower_example.txt	6812	0644	edit dl rm
cachestat_example.txt	4012	0644	edit dl rm
cachetop_example.txt	3918	0644	edit dl rm
capable_example.txt	6655	0644	edit dl rm
cobjnew_example.txt	3044	0644	edit dl rm
compactsnoop_example.txt	10160	0644	edit dl rm
cpudist_example.txt	16876	0644	edit dl rm
cpuunclaimed_example.txt	15568	0644	edit dl rm
criticalstat_example.txt	4926	0644	edit dl rm
cthreads_example.txt	2132	0644	edit dl rm
dbslower_example.txt	3981	0644	edit dl rm
dbstat_example.txt	6656	0644	edit dl rm
dcsnoop_example.txt	4371	0644	edit dl rm
dcstat_example.txt	3343	0644	edit dl rm
deadlock_example.txt	16643	0644	edit dl rm
dirtop_example.txt	5097	0644	edit dl rm
drsnoop_example.txt	5120	0644	edit dl rm
execsnoop_example.txt	6795	0644	edit dl rm
exitsnoop_example.txt	6373	0644	edit dl rm
ext4dist_example.txt	8991	0644	edit dl rm
ext4slower_example.txt	11335	0644	edit dl rm
filegone_example.txt	743	0644	edit dl rm
filelife_example.txt	2092	0644	edit dl rm
fileslower_example.txt	5711	0644	edit dl rm
filetop_example.txt	6968	0644	edit dl rm
funccount_example.txt	13605	0644	edit dl rm
funcinterval_example.txt	15646	0644	edit dl rm
funclatency_example.txt	21485	0644	edit dl rm
funcslower_example.txt	6786	0644	edit dl rm
gethostlatency_example.txt	1317	0644	edit dl rm
hardirqs_example.txt	37936	0644	edit dl rm
inject_example.txt	6832	0644	edit dl rm
javacalls_example.txt	4000	0644	edit dl rm
javaflow_example.txt	6017	0644	edit dl rm
javagc_example.txt	3867	0644	edit dl rm
javaobjnew_example.txt	3044	0644	edit dl rm
javastat_example.txt	3052	0644	edit dl rm
javathreads_example.txt	2132	0644	edit dl rm
killsnoop_example.txt	1339	0644	edit dl rm
klockstat_example.txt	8536	0644	edit dl rm
kvmexit_example.txt	11909	0644	edit dl rm
llcstat_example.txt	3315	0644	edit dl rm
mdflush_example.txt	1780	0644	edit dl rm
memleak_example.txt	10263	0644	edit dl rm
mountsnoop_example.txt	1481	0644	edit dl rm
mysqld_qslower_example.txt	2351	0644	edit dl rm
netqtop_example.txt	12497	0644	edit dl rm
nfsdist_example.txt	8508	0644	edit dl rm
nfsslower_example.txt	7867	0644	edit dl rm
nodegc_example.txt	3867	0644	edit dl rm
nodestat_example.txt	3052	0644	edit dl rm
offcputime_example.txt	19662	0644	edit dl rm
offwaketime_example.txt	38258	0644	edit dl rm
oomkill_example.txt	1925	0644	edit dl rm
opensnoop_example.txt	10573	0644	edit dl rm
perlcalls_example.txt	4000	0644	edit dl rm
perlflow_example.txt	6017	0644	edit dl rm
perlstat_example.txt	3052	0644	edit dl rm
phpcalls_example.txt	4000	0644	edit dl rm
phpflow_example.txt	6017	0644	edit dl rm
phpstat_example.txt	3052	0644	edit dl rm
pidpersec_example.txt	677	0644	edit dl rm
ppchcalls_example.txt	7097	0644	edit dl rm
profile_example.txt	31826	0644	edit dl rm
pythoncalls_example.txt	4000	0644	edit dl rm
pythonflow_example.txt	6017	0644	edit dl rm
pythongc_example.txt	3867	0644	edit dl rm
pythonstat_example.txt	3052	0644	edit dl rm
rdmaucma_example.txt	1983	0644	edit dl rm
readahead_example.txt	3248	0644	edit dl rm
reset-trace_example.txt	9365	0644	edit dl rm
rubycalls_example.txt	4000	0644	edit dl rm
rubyflow_example.txt	6017	0644	edit dl rm
rubygc_example.txt	3867	0644	edit dl rm
rubyobjnew_example.txt	3044	0644	edit dl rm
rubystat_example.txt	3052	0644	edit dl rm
runqlat_example.txt	32051	0644	edit dl rm
runqlen_example.txt	12136	0644	edit dl rm
runqslower_example.txt	2184	0644	edit dl rm
shmsnoop_example.txt	2798	0644	edit dl rm
slabratetop_example.txt	5347	0644	edit dl rm
sofdsnoop_example.txt	3211	0644	edit dl rm
softirqs_example.txt	11286	0644	edit dl rm
solisten_example.txt	2355	0644	edit dl rm
sslsniff_example.txt	6902	0644	edit dl rm
stackcount_example.txt	21965	0644	edit dl rm
statsnoop_example.txt	3091	0644	edit dl rm
swapin.txt	2632	0644	edit dl rm
swapin_example.txt	1421	0644	edit dl rm
syncsnoop_example.txt	387	0644	edit dl rm
syscount_example.txt	6419	0644	edit dl rm
tclcalls_example.txt	4000	0644	edit dl rm
tclflow_example.txt	6017	0644	edit dl rm
tclobjnew_example.txt	3044	0644	edit dl rm
tclstat_example.txt	3052	0644	edit dl rm
tcpaccept_example.txt	2822	0644	edit dl rm
tcpcong_example.txt	34106	0644	edit dl rm
tcpconnect_example.txt	6420	0644	edit dl rm
tcpconnlat_example.txt	2616	0644	edit dl rm
tcpdrop_example.txt	2001	0644	edit dl rm
tcplife_example.txt	6995	0644	edit dl rm
tcpretrans_example.txt	3938	0644	edit dl rm
tcprtt_example.txt	10070	0644	edit dl rm
tcpstates_example.txt	2908	0644	edit dl rm
tcpsubnet_example.txt	5503	0644	edit dl rm
tcpsynbl_example.txt	1179	0644	edit dl rm
tcptop_example.txt	5890	0644	edit dl rm
tcptracer_example.txt	2029	0644	edit dl rm
threadsnoop_example.txt	1094	0644	edit dl rm
tplist_example.txt	4507	0644	edit dl rm
trace_example.txt	22136	0644	edit dl rm
ttysnoop_example.txt	3315	0644	edit dl rm
vfscount_example.txt	2221	0644	edit dl rm
vfsstat_example.txt	1696	0644	edit dl rm
virtiostat_example.txt	2678	0644	edit dl rm
wakeuptime_example.txt	34048	0644	edit dl rm
xfsdist_example.txt	6928	0644	edit dl rm
xfsslower_example.txt	7080	0644	edit dl rm
zfsdist_example.txt	9753	0644	edit dl rm
zfsslower_example.txt	7551	0644	edit dl rm

Edit: /usr/share/doc/bpfcc-tools/examples/doc/syscount_example.txt (6419B)

Demonstrations of syscount, the Linux/eBPF version.

syscount summarizes syscall counts across the system or a specific process,
with optional latency information. It is very useful for general workload
characterization, for example:

# syscount
Tracing syscalls, printing top 10... Ctrl+C to quit.
[09:39:04]
SYSCALL             COUNT
write               10739
read                10584
wait4                1460
nanosleep            1457
select                795
rt_sigprocmask        689
clock_gettime         653
rt_sigaction          128
futex                  86
ioctl                  83
^C

These are the top 10 entries; you can get more by using the -T switch. Here,
the output indicates that the write and read syscalls were very common, followed
immediately by wait4, nanosleep, and so on. By default, syscount counts across
the entire system, but we can point it to a specific process of interest:

# syscount -p $(pidof dd)
Tracing syscalls, printing top 10... Ctrl+C to quit.
[09:40:21]
SYSCALL             COUNT
read              7878397
write             7878397
^C

Indeed, dd's workload is a bit easier to characterize. Occasionally, the count
of syscalls is not enough, and you'd also want an aggregate latency:

# syscount -L
Tracing syscalls, printing top 10... Ctrl+C to quit.
[09:41:32]
SYSCALL                   COUNT        TIME (us)
select                       16      3415860.022
nanosleep                   291        12038.707
ftruncate                     1          122.939
write                         4           63.389
stat                          1           23.431
fstat                         1            5.088
[unknown: 321]               32            4.965
timerfd_settime               1            4.830
ioctl                         3            4.802
kill                          1            4.342
^C

The select and nanosleep calls are responsible for a lot of time, but remember
these are blocking calls. This output was taken from a mostly idle system. Note
the "unknown" entry -- syscall 321 is the bpf() syscall, which is not in the
table used by this tool (borrowed from strace sources).

Another direction would be to understand which processes are making a lot of
syscalls, thus responsible for a lot of activity. This is what the -P switch
does:

# syscount -P
Tracing syscalls, printing top 10... Ctrl+C to quit.
[09:58:13]
PID    COMM               COUNT
13820  vim                  548
30216  sshd                 149
29633  bash                  72
25188  screen                70
25776  mysqld                30
31285  python                10
529    systemd-udevd          9
1      systemd                8
494    systemd-journal        5
^C

This is again from a mostly idle system over an interval of a few seconds.

Sometimes, you'd only care about failed syscalls -- these are the ones that
might be worth investigating with follow-up tools like opensnoop, execsnoop,
or trace. Use the -x switch for this; the following example also demonstrates
the -i switch, for printing at predefined intervals:

# syscount -x -i 5
Tracing failed syscalls, printing top 10... Ctrl+C to quit.
[09:44:16]
SYSCALL             COUNT
futex                  13
getxattr               10
stat                    8
open                    6
wait4                   3
access                  2
[unknown: 321]          1

[09:44:21]
SYSCALL             COUNT
futex                  12
getxattr               10
[unknown: 321]          2
wait4                   1
access                  1
pause                   1
^C

Similar to -x/--failures, sometimes you only care about certain syscall
errors like EPERM or ENONET -- these are the ones that might be worth
investigating with follow-up tools like opensnoop, execsnoop, or
trace. Use the -e/--errno switch for this; the following example also
demonstrates the -e switch, for printing ENOENT failures at predefined intervals:

# syscount -e ENOENT -i 5
Tracing syscalls, printing top 10... Ctrl+C to quit.
[13:15:57]
SYSCALL                   COUNT
stat                       4669
open                       1951
access                      561
lstat                        62
openat                       42
readlink                      8
execve                        4
newfstatat                    1

[13:16:02]
SYSCALL                   COUNT
lstat                     18506
stat                      13087
open                       2907
access                      412
openat                       19
readlink                     12
execve                        7
connect                       6
unlink                        1
rmdir                         1
^C

Sometimes, you'd only care about a single syscall rather than all syscalls.
Use the --syscall option for this; the following example also demonstrates
the --syscall option, for printing at predefined intervals:

# syscount --syscall stat -i 1
Tracing syscall 'stat'... Ctrl+C to quit.
[12:51:06]
SYSCALL                   COUNT
stat                        310

[12:51:07]
SYSCALL                   COUNT
stat                        316
^C

USAGE:
# syscount -h
usage: syscount.py [-h] [-p PID] [-t TID] [-i INTERVAL] [-d DURATION] [-T TOP]
                   [-x] [-e ERRNO] [-L] [-m] [-P] [-l] [--syscall SYSCALL]

Summarize syscall counts and latencies.

optional arguments:
  -h, --help            show this help message and exit
  -p PID, --pid PID     trace only this pid
  -t TID, --tid TID     trace only this tid
  -c PPID, --ppid PPID  trace only child of this pid
  -i INTERVAL, --interval INTERVAL
                        print summary at this interval (seconds)
  -d DURATION, --duration DURATION
                        total duration of trace, in seconds
  -T TOP, --top TOP     print only the top syscalls by count or latency
  -x, --failures        trace only failed syscalls (return < 0)
  -e ERRNO, --errno ERRNO
                        trace only syscalls that return this error (numeric or
                        EPERM, etc.)
  -L, --latency         collect syscall latency
  -m, --milliseconds    display latency in milliseconds (default:
                        microseconds)
  -P, --process         count by process and not by syscall
  -l, --list            print list of recognized syscalls and exit
  --syscall SYSCALL     trace this syscall only (use option -l to get all
                        recognized syscalls)

Demonstrations of syscount, the Linux/eBPF version. syscount summarizes syscall counts across the system or a specific process, with optional latency information. It is very useful for general workload characterization, for example: # syscount Tracing syscalls, printing top 10... Ctrl+C to quit. [09:39:04] SYSCALL COUNT write 10739 read 10584 wait4 1460 nanosleep 1457 select 795 rt_sigprocmask 689 clock_gettime 653 rt_sigaction 128 futex 86 ioctl 83 ^C These are the top 10 entries; you can get more by using the -T switch. Here, the output indicates that the write and read syscalls were very common, followed immediately by wait4, nanosleep, and so on. By default, syscount counts across the entire system, but we can point it to a specific process of interest: # syscount -p $(pidof dd) Tracing syscalls, printing top 10... Ctrl+C to quit. [09:40:21] SYSCALL COUNT read 7878397 write 7878397 ^C Indeed, dd's workload is a bit easier to characterize. Occasionally, the count of syscalls is not enough, and you'd also want an aggregate latency: # syscount -L Tracing syscalls, printing top 10... Ctrl+C to quit. [09:41:32] SYSCALL COUNT TIME (us) select 16 3415860.022 nanosleep 291 12038.707 ftruncate 1 122.939 write 4 63.389 stat 1 23.431 fstat 1 5.088 [unknown: 321] 32 4.965 timerfd_settime 1 4.830 ioctl 3 4.802 kill 1 4.342 ^C The select and nanosleep calls are responsible for a lot of time, but remember these are blocking calls. This output was taken from a mostly idle system. Note the "unknown" entry -- syscall 321 is the bpf() syscall, which is not in the table used by this tool (borrowed from strace sources). Another direction would be to understand which processes are making a lot of syscalls, thus responsible for a lot of activity. This is what the -P switch does: # syscount -P Tracing syscalls, printing top 10... Ctrl+C to quit. [09:58:13] PID COMM COUNT 13820 vim 548 30216 sshd 149 29633 bash 72 25188 screen 70 25776 mysqld 30 31285 python 10 529 systemd-udevd 9 1 systemd 8 494 systemd-journal 5 ^C This is again from a mostly idle system over an interval of a few seconds. Sometimes, you'd only care about failed syscalls -- these are the ones that might be worth investigating with follow-up tools like opensnoop, execsnoop, or trace. Use the -x switch for this; the following example also demonstrates the -i switch, for printing at predefined intervals: # syscount -x -i 5 Tracing failed syscalls, printing top 10... Ctrl+C to quit. [09:44:16] SYSCALL COUNT futex 13 getxattr 10 stat 8 open 6 wait4 3 access 2 [unknown: 321] 1 [09:44:21] SYSCALL COUNT futex 12 getxattr 10 [unknown: 321] 2 wait4 1 access 1 pause 1 ^C Similar to -x/--failures, sometimes you only care about certain syscall errors like EPERM or ENONET -- these are the ones that might be worth investigating with follow-up tools like opensnoop, execsnoop, or trace. Use the -e/--errno switch for this; the following example also demonstrates the -e switch, for printing ENOENT failures at predefined intervals: # syscount -e ENOENT -i 5 Tracing syscalls, printing top 10... Ctrl+C to quit. [13:15:57] SYSCALL COUNT stat 4669 open 1951 access 561 lstat 62 openat 42 readlink 8 execve 4 newfstatat 1 [13:16:02] SYSCALL COUNT lstat 18506 stat 13087 open 2907 access 412 openat 19 readlink 12 execve 7 connect 6 unlink 1 rmdir 1 ^C Sometimes, you'd only care about a single syscall rather than all syscalls. Use the --syscall option for this; the following example also demonstrates the --syscall option, for printing at predefined intervals: # syscount --syscall stat -i 1 Tracing syscall 'stat'... Ctrl+C to quit. [12:51:06] SYSCALL COUNT stat 310 [12:51:07] SYSCALL COUNT stat 316 ^C USAGE: # syscount -h usage: syscount.py [-h] [-p PID] [-t TID] [-i INTERVAL] [-d DURATION] [-T TOP] [-x] [-e ERRNO] [-L] [-m] [-P] [-l] [--syscall SYSCALL] Summarize syscall counts and latencies. optional arguments: -h, --help show this help message and exit -p PID, --pid PID trace only this pid -t TID, --tid TID trace only this tid -c PPID, --ppid PPID trace only child of this pid -i INTERVAL, --interval INTERVAL print summary at this interval (seconds) -d DURATION, --duration DURATION total duration of trace, in seconds -T TOP, --top TOP print only the top syscalls by count or latency -x, --failures trace only failed syscalls (return < 0) -e ERRNO, --errno ERRNO trace only syscalls that return this error (numeric or EPERM, etc.) -L, --latency collect syscall latency -m, --milliseconds display latency in milliseconds (default: microseconds) -P, --process count by process and not by syscall -l, --list print list of recognized syscalls and exit --syscall SYSCALL trace this syscall only (use option -l to get all recognized syscalls)