6 perf-sched - Tool to trace/measure scheduler properties (latencies)
11 'perf sched' {record|latency|map|replay|script|timehist}
15 There are several variants of 'perf sched':
17 'perf sched record <command>' to record the scheduling events
18 of an arbitrary workload.
20 'perf sched latency' to report the per task scheduling latencies
21 and other scheduling properties of the workload.
24 perf sched record -- sleep 1
27 -------------------------------------------------------------------------------------------------------------------------------------------
28 Task | Runtime ms | Count | Avg delay ms | Max delay ms | Max delay start | Max delay end |
29 -------------------------------------------------------------------------------------------------------------------------------------------
30 perf:(2) | 2.804 ms | 66 | avg: 0.524 ms | max: 1.069 ms | max start: 254752.314960 s | max end: 254752.316029 s
31 NetworkManager:1343 | 0.372 ms | 13 | avg: 0.008 ms | max: 0.013 ms | max start: 254751.551153 s | max end: 254751.551166 s
32 kworker/1:2-xfs:4649 | 0.012 ms | 1 | avg: 0.008 ms | max: 0.008 ms | max start: 254751.519807 s | max end: 254751.519815 s
33 kworker/3:1-xfs:388 | 0.011 ms | 1 | avg: 0.006 ms | max: 0.006 ms | max start: 254751.519809 s | max end: 254751.519815 s
34 sleep:147736 | 0.938 ms | 3 | avg: 0.006 ms | max: 0.007 ms | max start: 254751.313817 s | max end: 254751.313824 s
36 It shows Runtime(time that a task spent actually running on the CPU),
37 Count(number of times a delay was calculated) and delay(time that a
38 task was ready to run but was kept waiting).
40 Tasks with the same command name are merged and the merge count is
41 given within (), However if -p option is used, pid is mentioned.
43 'perf sched script' to see a detailed trace of the workload that
44 was recorded (aliased to 'perf script' for now).
46 'perf sched replay' to simulate the workload that was recorded
47 via perf sched record. (this is done by starting up mockup threads
48 that mimic the workload based on the events in the trace. These
49 threads can then replay the timings (CPU runtime and sleep patterns)
50 of the workload as it occurred when it was recorded - and can repeat
51 it a number of times, measuring its performance.)
53 'perf sched map' to print a textual context-switching outline of
54 workload captured via perf sched record. Columns stand for
55 individual CPUs, and the two-letter shortcuts stand for tasks that
56 are running on a CPU. A '*' denotes the CPU that had the event, and
57 a dot signals an idle CPU.
59 'perf sched timehist' provides an analysis of scheduling events.
62 perf sched record -- sleep 1
65 By default it shows the individual schedule events, including the wait
66 time (time between sched-out and next sched-in events for the task), the
67 task scheduling delay (time between runnable and actually running) and
68 run time for the task:
70 time cpu task name wait time sch delay run time
71 [tid/pid] (msec) (msec) (msec)
72 -------------- ------ -------------------- --------- --------- ---------
73 79371.874569 [0011] gcc[31949] 0.014 0.000 1.148
74 79371.874591 [0010] gcc[31951] 0.000 0.000 0.024
75 79371.874603 [0010] migration/10[59] 3.350 0.004 0.011
76 79371.874604 [0011] <idle> 1.148 0.000 0.035
77 79371.874723 [0005] <idle> 0.016 0.000 1.383
78 79371.874746 [0005] gcc[31949] 0.153 0.078 0.022
81 Times are in msec.usec.
87 Input file name. (default: perf.data unless stdin is a fifo)
91 Be more verbose. (show symbol address, etc)
95 Display verbose dump of the sched data.
99 Don't complain, do it.
101 OPTIONS for 'perf sched latency'
102 -------------------------------
110 latency stats per pid instead of per command name.
113 --sort <key[,key2...]>::
114 sort by key(s): runtime, switch, avg, max
115 by default it's sorted by "avg ,max ,switch ,runtime".
117 OPTIONS for 'perf sched map'
118 ----------------------------
121 Show only CPUs with activity. Helps visualizing on high core
125 Show just entries with activities for the given CPUs.
128 Highlight the given cpus.
131 Highlight the given pids.
134 Map output only for the given task name(s). Separate the
135 task names with a comma (without whitespace). The sched-out
136 time is printed and is represented by '*-' for the given
138 ('-' indicates other tasks while '.' is idle).
141 Given task name(s) can be partially matched (fuzzy matching).
143 OPTIONS for 'perf sched timehist'
144 ---------------------------------
154 Display call chains if present (default on).
157 Maximum number of functions to display in backtrace, default 5.
161 Only show events for the given CPU(s) (comma separated list).
165 Only show events for given process ID (comma separated list).
169 Only show events for given thread ID (comma separated list).
173 Show only a summary of scheduling by thread with min, max, and average
174 run times (in sec) and relative stddev.
178 Show all scheduling events followed by a summary by thread with min,
179 max, and average run times (in sec) and relative stddev.
181 --symfs=<directory>::
182 Look for files with symbols relative to this directory.
186 Show visual aid for sched switches by CPU: 'i' marks idle time,
187 's' are scheduler events.
195 Show migration events.
203 Show idle-related events only.
206 Only analyze samples within given time window: <start>,<stop>. Times
207 have the format seconds.microseconds. If start is not given (i.e., time
208 string is ',x.y') then analysis starts at the beginning of the file. If
209 stop time is not given (i.e, time string is 'x.y,') then analysis goes
213 Show task state when it switched out.
219 Only show events for given task priority(ies). Multiple priorities can be
220 provided as a comma-separated list with no spaces: 0,120. Ranges of
221 priorities are specified with -: 120-129. A combination of both can also be
226 Show pre-migration wait time. pre-migration wait time is the time spent
227 by a task waiting on a runqueue but not getting the chance to run there
228 and is migrated to a different runqueue where it is finally run. This
229 time between sched_wakeup and migrate_task is the pre-migration wait
232 OPTIONS for 'perf sched replay'
233 ------------------------------
237 repeat the workload n times (0: infinite). Default is 10.
241 linkperf:perf-record[1]