Root/Documentation/dynamic-debug-howto.txt

1
2Introduction
3============
4
5This document describes how to use the dynamic debug (ddebug) feature.
6
7Dynamic debug is designed to allow you to dynamically enable/disable kernel
8code to obtain additional kernel information. Currently, if
9CONFIG_DYNAMIC_DEBUG is set, then all pr_debug()/dev_debug() calls can be
10dynamically enabled per-callsite.
11
12Dynamic debug has even more useful features:
13
14 * Simple query language allows turning on and off debugging statements by
15   matching any combination of:
16
17   - source filename
18   - function name
19   - line number (including ranges of line numbers)
20   - module name
21   - format string
22
23 * Provides a debugfs control file: <debugfs>/dynamic_debug/control which can be
24   read to display the complete list of known debug statements, to help guide you
25
26Controlling dynamic debug Behaviour
27===================================
28
29The behaviour of pr_debug()/dev_debug()s are controlled via writing to a
30control file in the 'debugfs' filesystem. Thus, you must first mount the debugfs
31filesystem, in order to make use of this feature. Subsequently, we refer to the
32control file as: <debugfs>/dynamic_debug/control. For example, if you want to
33enable printing from source file 'svcsock.c', line 1603 you simply do:
34
35nullarbor:~ # echo 'file svcsock.c line 1603 +p' >
36                <debugfs>/dynamic_debug/control
37
38If you make a mistake with the syntax, the write will fail thus:
39
40nullarbor:~ # echo 'file svcsock.c wtf 1 +p' >
41                <debugfs>/dynamic_debug/control
42-bash: echo: write error: Invalid argument
43
44Viewing Dynamic Debug Behaviour
45===========================
46
47You can view the currently configured behaviour of all the debug statements
48via:
49
50nullarbor:~ # cat <debugfs>/dynamic_debug/control
51# filename:lineno [module]function flags format
52/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:323 [svcxprt_rdma]svc_rdma_cleanup - "SVCRDMA Module Removed, deregister RPC RDMA transport\012"
53/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:341 [svcxprt_rdma]svc_rdma_init - "\011max_inline : %d\012"
54/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:340 [svcxprt_rdma]svc_rdma_init - "\011sq_depth : %d\012"
55/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:338 [svcxprt_rdma]svc_rdma_init - "\011max_requests : %d\012"
56...
57
58
59You can also apply standard Unix text manipulation filters to this
60data, e.g.
61
62nullarbor:~ # grep -i rdma <debugfs>/dynamic_debug/control | wc -l
6362
64
65nullarbor:~ # grep -i tcp <debugfs>/dynamic_debug/control | wc -l
6642
67
68Note in particular that the third column shows the enabled behaviour
69flags for each debug statement callsite (see below for definitions of the
70flags). The default value, no extra behaviour enabled, is "-". So
71you can view all the debug statement callsites with any non-default flags:
72
73nullarbor:~ # awk '$3 != "-"' <debugfs>/dynamic_debug/control
74# filename:lineno [module]function flags format
75/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c:1603 [sunrpc]svc_send p "svc_process: st_sendto returned %d\012"
76
77
78Command Language Reference
79==========================
80
81At the lexical level, a command comprises a sequence of words separated
82by whitespace characters. Note that newlines are treated as word
83separators and do *not* end a command or allow multiple commands to
84be done together. So these are all equivalent:
85
86nullarbor:~ # echo -c 'file svcsock.c line 1603 +p' >
87                <debugfs>/dynamic_debug/control
88nullarbor:~ # echo -c ' file svcsock.c line 1603 +p ' >
89                <debugfs>/dynamic_debug/control
90nullarbor:~ # echo -c 'file svcsock.c\nline 1603 +p' >
91                <debugfs>/dynamic_debug/control
92nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
93                <debugfs>/dynamic_debug/control
94
95Commands are bounded by a write() system call. If you want to do
96multiple commands you need to do a separate "echo" for each, like:
97
98nullarbor:~ # echo 'file svcsock.c line 1603 +p' > /proc/dprintk ;\
99> echo 'file svcsock.c line 1563 +p' > /proc/dprintk
100
101or even like:
102
103nullarbor:~ # (
104> echo 'file svcsock.c line 1603 +p' ;\
105> echo 'file svcsock.c line 1563 +p' ;\
106> ) > /proc/dprintk
107
108At the syntactical level, a command comprises a sequence of match
109specifications, followed by a flags change specification.
110
111command ::= match-spec* flags-spec
112
113The match-spec's are used to choose a subset of the known dprintk()
114callsites to which to apply the flags-spec. Think of them as a query
115with implicit ANDs between each pair. Note that an empty list of
116match-specs is possible, but is not very useful because it will not
117match any debug statement callsites.
118
119A match specification comprises a keyword, which controls the attribute
120of the callsite to be compared, and a value to compare against. Possible
121keywords are:
122
123match-spec ::= 'func' string |
124           'file' string |
125           'module' string |
126           'format' string |
127           'line' line-range
128
129line-range ::= lineno |
130           '-'lineno |
131           lineno'-' |
132           lineno'-'lineno
133// Note: line-range cannot contain space, e.g.
134// "1-30" is valid range but "1 - 30" is not.
135
136lineno ::= unsigned-int
137
138The meanings of each keyword are:
139
140func
141    The given string is compared against the function name
142    of each callsite. Example:
143
144    func svc_tcp_accept
145
146file
147    The given string is compared against either the full
148    pathname or the basename of the source file of each
149    callsite. Examples:
150
151    file svcsock.c
152    file /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c
153
154module
155    The given string is compared against the module name
156    of each callsite. The module name is the string as
157    seen in "lsmod", i.e. without the directory or the .ko
158    suffix and with '-' changed to '_'. Examples:
159
160    module sunrpc
161    module nfsd
162
163format
164    The given string is searched for in the dynamic debug format
165    string. Note that the string does not need to match the
166    entire format, only some part. Whitespace and other
167    special characters can be escaped using C octal character
168    escape \ooo notation, e.g. the space character is \040.
169    Alternatively, the string can be enclosed in double quote
170    characters (") or single quote characters (').
171    Examples:
172
173    format svcrdma: // many of the NFS/RDMA server dprintks
174    format readahead // some dprintks in the readahead cache
175    format nfsd:\040SETATTR // one way to match a format with whitespace
176    format "nfsd: SETATTR" // a neater way to match a format with whitespace
177    format 'nfsd: SETATTR' // yet another way to match a format with whitespace
178
179line
180    The given line number or range of line numbers is compared
181    against the line number of each dprintk() callsite. A single
182    line number matches the callsite line number exactly. A
183    range of line numbers matches any callsite between the first
184    and last line number inclusive. An empty first number means
185    the first line in the file, an empty line number means the
186    last number in the file. Examples:
187
188    line 1603 // exactly line 1603
189    line 1600-1605 // the six lines from line 1600 to line 1605
190    line -1605 // the 1605 lines from line 1 to line 1605
191    line 1600- // all lines from line 1600 to the end of the file
192
193The flags specification comprises a change operation followed
194by one or more flag characters. The change operation is one
195of the characters:
196
197-
198    remove the given flags
199
200+
201    add the given flags
202
203=
204    set the flags to the given flags
205
206The flags are:
207
208p
209    Causes a printk() message to be emitted to dmesg
210
211Note the regexp ^[-+=][scp]+$ matches a flags specification.
212Note also that there is no convenient syntax to remove all
213the flags at once, you need to use "-psc".
214
215
216Debug messages during boot process
217==================================
218
219To be able to activate debug messages during the boot process,
220even before userspace and debugfs exists, use the boot parameter:
221ddebug_query="QUERY"
222
223QUERY follows the syntax described above, but must not exceed 1023
224characters. The enablement of debug messages is done as an arch_initcall.
225Thus you can enable debug messages in all code processed after this
226arch_initcall via this boot parameter.
227On an x86 system for example ACPI enablement is a subsys_initcall and
228ddebug_query="file ec.c +p"
229will show early Embedded Controller transactions during ACPI setup if
230your machine (typically a laptop) has an Embedded Controller.
231PCI (or other devices) initialization also is a hot candidate for using
232this boot parameter for debugging purposes.
233
234
235Examples
236========
237
238// enable the message at line 1603 of file svcsock.c
239nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
240                <debugfs>/dynamic_debug/control
241
242// enable all the messages in file svcsock.c
243nullarbor:~ # echo -n 'file svcsock.c +p' >
244                <debugfs>/dynamic_debug/control
245
246// enable all the messages in the NFS server module
247nullarbor:~ # echo -n 'module nfsd +p' >
248                <debugfs>/dynamic_debug/control
249
250// enable all 12 messages in the function svc_process()
251nullarbor:~ # echo -n 'func svc_process +p' >
252                <debugfs>/dynamic_debug/control
253
254// disable all 12 messages in the function svc_process()
255nullarbor:~ # echo -n 'func svc_process -p' >
256                <debugfs>/dynamic_debug/control
257
258// enable messages for NFS calls READ, READLINK, READDIR and READDIR+.
259nullarbor:~ # echo -n 'format "nfsd: READ" +p' >
260                <debugfs>/dynamic_debug/control
261

Archive Download this file



interactive