| blob | 01f849845bd926f9d6f27e5e90cc76874b327e07 |
1 .\"
2 .\" CDDL HEADER START
3 .\"
4 .\" The contents of this file are subject to the terms of the
5 .\" Common Development and Distribution License (the "License").
6 .\" You may not use this file except in compliance with the License.
7 .\"
8 .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9 .\" or https://opensource.org/licenses/CDDL-1.0.
10 .\" See the License for the specific language governing permissions
11 .\" and limitations under the License.
12 .\"
13 .\" When distributing Covered Code, include this CDDL HEADER in each
14 .\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15 .\" If applicable, add the following below this CDDL HEADER, with the
16 .\" fields enclosed by brackets "[]" replaced with your own identifying
17 .\" information: Portions Copyright [yyyy] [name of copyright owner]
18 .\"
19 .\" CDDL HEADER END
20 .\"
21 .\" Copyright (c) 2007, Sun Microsystems, Inc. All Rights Reserved.
22 .\" Copyright (c) 2012, 2018 by Delphix. All rights reserved.
23 .\" Copyright (c) 2012 Cyril Plisko. All Rights Reserved.
24 .\" Copyright (c) 2017 Datto Inc.
25 .\" Copyright (c) 2018 George Melikov. All Rights Reserved.
26 .\" Copyright 2017 Nexenta Systems, Inc.
27 .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved.
28 .\" Copyright (c) 2024, Klara Inc.
29 .\"
30 .Dd February 28, 2024
31 .Dt ZPOOL-EVENTS 8
32 .Os
33 .
34 .Sh NAME
35 .Nm zpool-events
36 .Nd list recent events generated by kernel
37 .Sh SYNOPSIS
38 .Nm zpool
39 .Cm events
40 .Op Fl vHf
41 .Op Ar pool
42 .Nm zpool
43 .Cm events
44 .Fl c
45 .
46 .Sh DESCRIPTION
47 Lists all recent events generated by the ZFS kernel modules.
48 These events are consumed by the
49 .Xr zed 8
50 and used to automate administrative tasks such as replacing a failed device
51 with a hot spare.
52 For more information about the subclasses and event payloads
53 that can be generated see
54 .Sx EVENTS
55 and the following sections.
56 .
57 .Sh OPTIONS
58 .Bl -tag -compact -width Ds
59 .It Fl c
60 Clear all previous events.
61 .It Fl f
62 Follow mode.
63 .It Fl H
64 Scripted mode.
65 Do not display headers, and separate fields by a
66 single tab instead of arbitrary space.
67 .It Fl v
68 Print the entire payload for each event.
69 .El
70 .
71 .Sh EVENTS
72 These are the different event subclasses.
73 The full event name would be
74 .Sy ereport.fs.zfs.\& Ns Em SUBCLASS ,
75 but only the last part is listed here.
76 .Pp
77 .Bl -tag -compact -width "vdev.bad_guid_sum"
78 .It Sy checksum
79 Issued when a checksum error has been detected.
80 .It Sy io
81 Issued when there is an I/O error in a vdev in the pool.
82 .It Sy data
83 Issued when there have been data errors in the pool.
84 .It Sy deadman
85 Issued when an I/O request is determined to be "hung", this can be caused
86 by lost completion events due to flaky hardware or drivers.
87 See
88 .Sy zfs_deadman_failmode
89 in
90 .Xr zfs 4
91 for additional information regarding "hung" I/O detection and configuration.
92 .It Sy delay
93 Issued when a completed I/O request exceeds the maximum allowed time
94 specified by the
95 .Sy zio_slow_io_ms
96 module parameter.
97 This can be an indicator of problems with the underlying storage device.
98 The number of delay events is ratelimited by the
99 .Sy zfs_slow_io_events_per_second
100 module parameter.
101 .It Sy dio_verify_rd
102 Issued when there was a checksum verify error after a Direct I/O read has been
103 issued.
104 .It Sy dio_verify_wr
105 Issued when there was a checksum verify error after a Direct I/O write has been
106 issued.
107 This event can only take place if the module parameter
108 .Sy zfs_vdev_direct_write_verify
109 is not set to zero.
110 See
111 .Xr zfs 4
112 for more details on the
113 .Sy zfs_vdev_direct_write_verify
114 module paramter.
115 .It Sy config
116 Issued every time a vdev change have been done to the pool.
117 .It Sy zpool
118 Issued when a pool cannot be imported.
119 .It Sy zpool.destroy
120 Issued when a pool is destroyed.
121 .It Sy zpool.export
122 Issued when a pool is exported.
123 .It Sy zpool.import
124 Issued when a pool is imported.
125 .It Sy zpool.reguid
126 Issued when a REGUID (new unique identifier for the pool have been regenerated)
127 have been detected.
128 .It Sy vdev.unknown
129 Issued when the vdev is unknown.
130 Such as trying to clear device errors on a vdev that have failed/been kicked
131 from the system/pool and is no longer available.
132 .It Sy vdev.open_failed
133 Issued when a vdev could not be opened (because it didn't exist for example).
134 .It Sy vdev.corrupt_data
135 Issued when corrupt data have been detected on a vdev.
136 .It Sy vdev.no_replicas
137 Issued when there are no more replicas to sustain the pool.
138 This would lead to the pool being
139 .Em DEGRADED .
140 .It Sy vdev.bad_guid_sum
141 Issued when a missing device in the pool have been detected.
142 .It Sy vdev.too_small
143 Issued when the system (kernel) have removed a device, and ZFS
144 notices that the device isn't there any more.
145 This is usually followed by a
146 .Sy probe_failure
147 event.
148 .It Sy vdev.bad_label
149 Issued when the label is OK but invalid.
150 .It Sy vdev.bad_ashift
151 Issued when the ashift alignment requirement has increased.
152 .It Sy vdev.remove
153 Issued when a vdev is detached from a mirror (or a spare detached from a
154 vdev where it have been used to replace a failed drive - only works if
155 the original drive have been re-added).
156 .It Sy vdev.clear
157 Issued when clearing device errors in a pool.
158 Such as running
159 .Nm zpool Cm clear
160 on a device in the pool.
161 .It Sy vdev.check
162 Issued when a check to see if a given vdev could be opened is started.
163 .It Sy vdev.spare
164 Issued when a spare have kicked in to replace a failed device.
165 .It Sy vdev.autoexpand
166 Issued when a vdev can be automatically expanded.
167 .It Sy io_failure
168 Issued when there is an I/O failure in a vdev in the pool.
169 .It Sy probe_failure
170 Issued when a probe fails on a vdev.
171 This would occur if a vdev
172 have been kicked from the system outside of ZFS (such as the kernel
173 have removed the device).
174 .It Sy log_replay
175 Issued when the intent log cannot be replayed.
176 The can occur in the case of a missing or damaged log device.
177 .It Sy resilver.start
178 Issued when a resilver is started.
179 .It Sy resilver.finish
180 Issued when the running resilver have finished.
181 .It Sy scrub.start
182 Issued when a scrub is started on a pool.
183 .It Sy scrub.finish
184 Issued when a pool has finished scrubbing.
185 .It Sy scrub.abort
186 Issued when a scrub is aborted on a pool.
187 .It Sy scrub.resume
188 Issued when a scrub is resumed on a pool.
189 .It Sy scrub.paused
190 Issued when a scrub is paused on a pool.
191 .It Sy bootfs.vdev.attach
192 .El
193 .
194 .Sh PAYLOADS
195 This is the payload (data, information) that accompanies an
196 event.
197 .Pp
198 For
199 .Xr zed 8 ,
200 these are set to uppercase and prefixed with
201 .Sy ZEVENT_ .
202 .Pp
203 .Bl -tag -compact -width "vdev_cksum_errors"
204 .It Sy pool
205 Pool name.
206 .It Sy pool_failmode
207 Failmode -
208 .Sy wait ,
209 .Sy continue ,
210 or
211 .Sy panic .
212 See the
213 .Sy failmode
214 property in
215 .Xr zpoolprops 7
216 for more information.
217 .It Sy pool_guid
218 The GUID of the pool.
219 .It Sy pool_context
220 The load state for the pool (0=none, 1=open, 2=import, 3=tryimport, 4=recover
221 5=error).
222 .It Sy vdev_guid
223 The GUID of the vdev in question (the vdev failing or operated upon with
224 .Nm zpool Cm clear ,
225 etc.).
226 .It Sy vdev_type
227 Type of vdev -
228 .Sy disk ,
229 .Sy file ,
230 .Sy mirror ,
231 etc.
232 See the
233 .Sy Virtual Devices
234 section of
235 .Xr zpoolconcepts 7
236 for more information on possible values.
237 .It Sy vdev_path
238 Full path of the vdev, including any
239 .Em -partX .
240 .It Sy vdev_devid
241 ID of vdev (if any).
242 .It Sy vdev_fru
243 Physical FRU location.
244 .It Sy vdev_state
245 State of vdev (0=uninitialized, 1=closed, 2=offline, 3=removed, 4=failed to
246 open, 5=faulted, 6=degraded, 7=healthy).
247 .It Sy vdev_ashift
248 The ashift value of the vdev.
249 .It Sy vdev_complete_ts
250 The time the last I/O request completed for the specified vdev.
251 .It Sy vdev_delta_ts
252 The time since the last I/O request completed for the specified vdev.
253 .It Sy vdev_spare_paths
254 List of spares, including full path and any
255 .Em -partX .
256 .It Sy vdev_spare_guids
257 GUID(s) of spares.
258 .It Sy vdev_read_errors
259 How many read errors that have been detected on the vdev.
260 .It Sy vdev_write_errors
261 How many write errors that have been detected on the vdev.
262 .It Sy vdev_cksum_errors
263 How many checksum errors that have been detected on the vdev.
264 .It Sy parent_guid
265 GUID of the vdev parent.
266 .It Sy parent_type
267 Type of parent.
268 See
269 .Sy vdev_type .
270 .It Sy parent_path
271 Path of the vdev parent (if any).
272 .It Sy parent_devid
273 ID of the vdev parent (if any).
274 .It Sy zio_objset
275 The object set number for a given I/O request.
276 .It Sy zio_object
277 The object number for a given I/O request.
278 .It Sy zio_level
279 The indirect level for the block.
280 Level 0 is the lowest level and includes data blocks.
281 Values > 0 indicate metadata blocks at the appropriate level.
282 .It Sy zio_blkid
283 The block ID for a given I/O request.
284 .It Sy zio_err
285 The error number for a failure when handling a given I/O request,
286 compatible with
287 .Xr errno 3
288 with the value of
289 .Sy EBADE
290 used to indicate a ZFS checksum error.
291 .It Sy zio_offset
292 The offset in bytes of where to write the I/O request for the specified vdev.
293 .It Sy zio_size
294 The size in bytes of the I/O request.
295 .It Sy zio_flags
296 The current flags describing how the I/O request should be handled.
297 See the
298 .Sy I/O FLAGS
299 section for the full list of I/O flags.
300 .It Sy zio_stage
301 The current stage of the I/O in the pipeline.
302 See the
303 .Sy I/O STAGES
304 section for a full list of all the I/O stages.
305 .It Sy zio_pipeline
306 The valid pipeline stages for the I/O.
307 See the
308 .Sy I/O STAGES
309 section for a full list of all the I/O stages.
310 .It Sy zio_delay
311 The time elapsed (in nanoseconds) waiting for the block layer to complete the
312 I/O request.
313 Unlike
314 .Sy zio_delta ,
315 this does not include any vdev queuing time and is
316 therefore solely a measure of the block layer performance.
317 .It Sy zio_timestamp
318 The time when a given I/O request was submitted.
319 .It Sy zio_delta
320 The time required to service a given I/O request.
321 .It Sy prev_state
322 The previous state of the vdev.
323 .It Sy cksum_algorithm
324 Checksum algorithm used.
325 See
326 .Xr zfsprops 7
327 for more information on the available checksum algorithms.
328 .It Sy cksum_byteswap
329 Whether or not the data is byteswapped.
330 .It Sy bad_ranges
331 .No [\& Ns Ar start , end )
332 pairs of corruption offsets.
333 Offsets are always aligned on a 64-bit boundary,
334 and can include some gaps of non-corruption.
335 (See
336 .Sy bad_ranges_min_gap )
337 .It Sy bad_ranges_min_gap
338 In order to bound the size of the
339 .Sy bad_ranges
340 array, gaps of non-corruption
341 less than or equal to
342 .Sy bad_ranges_min_gap
343 bytes have been merged with
344 adjacent corruption.
345 Always at least 8 bytes, since corruption is detected on a 64-bit word basis.
346 .It Sy bad_range_sets
347 This array has one element per range in
348 .Sy bad_ranges .
349 Each element contains
350 the count of bits in that range which were clear in the good data and set
351 in the bad data.
352 .It Sy bad_range_clears
353 This array has one element per range in
354 .Sy bad_ranges .
355 Each element contains
356 the count of bits for that range which were set in the good data and clear in
357 the bad data.
358 .It Sy bad_set_bits
359 If this field exists, it is an array of
360 .Pq Ar bad data No & ~( Ns Ar good data ) ;
361 that is, the bits set in the bad data which are cleared in the good data.
362 Each element corresponds a byte whose offset is in a range in
363 .Sy bad_ranges ,
364 and the array is ordered by offset.
365 Thus, the first element is the first byte in the first
366 .Sy bad_ranges
367 range, and the last element is the last byte in the last
368 .Sy bad_ranges
369 range.
370 .It Sy bad_cleared_bits
371 Like
372 .Sy bad_set_bits ,
373 but contains
374 .Pq Ar good data No & ~( Ns Ar bad data ) ;
375 that is, the bits set in the good data which are cleared in the bad data.
376 .El
377 .
378 .Sh I/O STAGES
379 The ZFS I/O pipeline is comprised of various stages which are defined below.
380 The individual stages are used to construct these basic I/O
381 operations: Read, Write, Free, Claim, Flush and Trim.
382 These stages may be
383 set on an event to describe the life cycle of a given I/O request.
384 .Pp
385 .TS
386 tab(:);
387 l l l .
388 Stage:Bit Mask:Operations
389 _:_:_
390 ZIO_STAGE_OPEN:0x00000001:RWFCXT
392 ZIO_STAGE_READ_BP_INIT:0x00000002:R-----
393 ZIO_STAGE_WRITE_BP_INIT:0x00000004:-W----
394 ZIO_STAGE_FREE_BP_INIT:0x00000008:--F---
395 ZIO_STAGE_ISSUE_ASYNC:0x00000010:-WF--T
396 ZIO_STAGE_WRITE_COMPRESS:0x00000020:-W----
398 ZIO_STAGE_ENCRYPT:0x00000040:-W----
399 ZIO_STAGE_CHECKSUM_GENERATE:0x00000080:-W----
401 ZIO_STAGE_NOP_WRITE:0x00000100:-W----
403 ZIO_STAGE_BRT_FREE:0x00000200:--F---
405 ZIO_STAGE_DDT_READ_START:0x00000400:R-----
406 ZIO_STAGE_DDT_READ_DONE:0x00000800:R-----
407 ZIO_STAGE_DDT_WRITE:0x00001000:-W----
408 ZIO_STAGE_DDT_FREE:0x00002000:--F---
410 ZIO_STAGE_GANG_ASSEMBLE:0x00004000:RWFC--
411 ZIO_STAGE_GANG_ISSUE:0x00008000:RWFC--
413 ZIO_STAGE_DVA_THROTTLE:0x00010000:-W----
414 ZIO_STAGE_DVA_ALLOCATE:0x00020000:-W----
415 ZIO_STAGE_DVA_FREE:0x00040000:--F---
416 ZIO_STAGE_DVA_CLAIM:0x00080000:---C--
418 ZIO_STAGE_READY:0x00100000:RWFCIT
420 ZIO_STAGE_VDEV_IO_START:0x00200000:RW--XT
421 ZIO_STAGE_VDEV_IO_DONE:0x00400000:RW--XT
422 ZIO_STAGE_VDEV_IO_ASSESS:0x00800000:RW--XT
424 ZIO_STAGE_CHECKSUM_VERIFY:0x01000000:R-----
425 ZIO_STAGE_DIO_CHECKSUM_VERIFY:0x02000000:-W----
427 ZIO_STAGE_DONE:0x04000000:RWFCXT
428 .TE
429 .
430 .Sh I/O FLAGS
431 Every I/O request in the pipeline contains a set of flags which describe its
432 function and are used to govern its behavior.
433 These flags will be set in an event as a
434 .Sy zio_flags
435 payload entry.
436 .Pp
437 .TS
438 tab(:);
439 l l .
440 Flag:Bit Mask
441 _:_
442 ZIO_FLAG_DONT_AGGREGATE:0x00000001
443 ZIO_FLAG_IO_REPAIR:0x00000002
444 ZIO_FLAG_SELF_HEAL:0x00000004
445 ZIO_FLAG_RESILVER:0x00000008
446 ZIO_FLAG_SCRUB:0x00000010
447 ZIO_FLAG_SCAN_THREAD:0x00000020
448 ZIO_FLAG_PHYSICAL:0x00000040
450 ZIO_FLAG_CANFAIL:0x00000080
451 ZIO_FLAG_SPECULATIVE:0x00000100
452 ZIO_FLAG_CONFIG_WRITER:0x00000200
453 ZIO_FLAG_DONT_RETRY:0x00000400
454 ZIO_FLAG_NODATA:0x00001000
455 ZIO_FLAG_INDUCE_DAMAGE:0x00002000
457 ZIO_FLAG_IO_ALLOCATING:0x00004000
458 ZIO_FLAG_IO_RETRY:0x00008000
459 ZIO_FLAG_PROBE:0x00010000
460 ZIO_FLAG_TRYHARD:0x00020000
461 ZIO_FLAG_OPTIONAL:0x00040000
463 ZIO_FLAG_DONT_QUEUE:0x00080000
464 ZIO_FLAG_DONT_PROPAGATE:0x00100000
465 ZIO_FLAG_IO_BYPASS:0x00200000
466 ZIO_FLAG_IO_REWRITE:0x00400000
467 ZIO_FLAG_RAW_COMPRESS:0x00800000
468 ZIO_FLAG_RAW_ENCRYPT:0x01000000
470 ZIO_FLAG_GANG_CHILD:0x02000000
471 ZIO_FLAG_DDT_CHILD:0x04000000
472 ZIO_FLAG_GODFATHER:0x08000000
473 ZIO_FLAG_NOPWRITE:0x10000000
474 ZIO_FLAG_REEXECUTED:0x20000000
475 ZIO_FLAG_DELEGATED:0x40000000
476 ZIO_FLAG_FASTWRITE:0x80000000
477 .TE
478 .
479 .Sh SEE ALSO
480 .Xr zfs 4 ,
481 .Xr zed 8 ,
482 .Xr zpool-wait 8