| blob | 234612baea8d2eeb395ee3bef1d8bfe411717801 |
1 .\"
2 .\" CDDL HEADER START
3 .\"
4 .\" The contents of this file are subject to the terms of the
5 .\" Common Development and Distribution License (the "License").
6 .\" You may not use this file except in compliance with the License.
7 .\"
8 .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9 .\" or https://opensource.org/licenses/CDDL-1.0.
10 .\" See the License for the specific language governing permissions
11 .\" and limitations under the License.
12 .\"
13 .\" When distributing Covered Code, include this CDDL HEADER in each
14 .\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15 .\" If applicable, add the following below this CDDL HEADER, with the
16 .\" fields enclosed by brackets "[]" replaced with your own identifying
17 .\" information: Portions Copyright [yyyy] [name of copyright owner]
18 .\"
19 .\" CDDL HEADER END
20 .\"
21 .\" Copyright (c) 2007, Sun Microsystems, Inc. All Rights Reserved.
22 .\" Copyright (c) 2012, 2018 by Delphix. All rights reserved.
23 .\" Copyright (c) 2012 Cyril Plisko. All Rights Reserved.
24 .\" Copyright (c) 2017 Datto Inc.
25 .\" Copyright (c) 2018 George Melikov. All Rights Reserved.
26 .\" Copyright 2017 Nexenta Systems, Inc.
27 .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved.
28 .\" Copyright (c) 2024, Klara Inc.
29 .\"
30 .Dd February 28, 2024
31 .Dt ZPOOL-EVENTS 8
32 .Os
33 .
34 .Sh NAME
35 .Nm zpool-events
36 .Nd list recent events generated by kernel
37 .Sh SYNOPSIS
38 .Nm zpool
39 .Cm events
40 .Op Fl vHf
41 .Op Ar pool
42 .Nm zpool
43 .Cm events
44 .Fl c
45 .
46 .Sh DESCRIPTION
47 Lists all recent events generated by the ZFS kernel modules.
48 These events are consumed by the
49 .Xr zed 8
50 and used to automate administrative tasks such as replacing a failed device
51 with a hot spare.
52 For more information about the subclasses and event payloads
53 that can be generated see
54 .Sx EVENTS
55 and the following sections.
56 .
57 .Sh OPTIONS
58 .Bl -tag -compact -width Ds
59 .It Fl c
60 Clear all previous events.
61 .It Fl f
62 Follow mode.
63 .It Fl H
64 Scripted mode.
65 Do not display headers, and separate fields by a
66 single tab instead of arbitrary space.
67 .It Fl v
68 Print the entire payload for each event.
69 .El
70 .
71 .Sh EVENTS
72 These are the different event subclasses.
73 The full event name would be
74 .Sy ereport.fs.zfs.\& Ns Em SUBCLASS ,
75 but only the last part is listed here.
76 .Pp
77 .Bl -tag -compact -width "vdev.bad_guid_sum"
78 .It Sy checksum
79 Issued when a checksum error has been detected.
80 .It Sy io
81 Issued when there is an I/O error in a vdev in the pool.
82 .It Sy data
83 Issued when there have been data errors in the pool.
84 .It Sy deadman
85 Issued when an I/O request is determined to be "hung", this can be caused
86 by lost completion events due to flaky hardware or drivers.
87 See
88 .Sy zfs_deadman_failmode
89 in
90 .Xr zfs 4
91 for additional information regarding "hung" I/O detection and configuration.
92 .It Sy delay
93 Issued when a completed I/O request exceeds the maximum allowed time
94 specified by the
95 .Sy zio_slow_io_ms
96 module parameter.
97 This can be an indicator of problems with the underlying storage device.
98 The number of delay events is ratelimited by the
99 .Sy zfs_slow_io_events_per_second
100 module parameter.
101 .It Sy dio_verify
102 Issued when there was a checksum verify error after a Direct I/O write has been
103 issued.
104 This event can only take place if the module parameter
105 .Sy zfs_vdev_direct_write_verify
106 is not set to zero.
107 See
108 .Xr zfs 4
109 for more details on the
110 .Sy zfs_vdev_direct_write_verify
111 module paramter.
112 .It Sy config
113 Issued every time a vdev change have been done to the pool.
114 .It Sy zpool
115 Issued when a pool cannot be imported.
116 .It Sy zpool.destroy
117 Issued when a pool is destroyed.
118 .It Sy zpool.export
119 Issued when a pool is exported.
120 .It Sy zpool.import
121 Issued when a pool is imported.
122 .It Sy zpool.reguid
123 Issued when a REGUID (new unique identifier for the pool have been regenerated)
124 have been detected.
125 .It Sy vdev.unknown
126 Issued when the vdev is unknown.
127 Such as trying to clear device errors on a vdev that have failed/been kicked
128 from the system/pool and is no longer available.
129 .It Sy vdev.open_failed
130 Issued when a vdev could not be opened (because it didn't exist for example).
131 .It Sy vdev.corrupt_data
132 Issued when corrupt data have been detected on a vdev.
133 .It Sy vdev.no_replicas
134 Issued when there are no more replicas to sustain the pool.
135 This would lead to the pool being
136 .Em DEGRADED .
137 .It Sy vdev.bad_guid_sum
138 Issued when a missing device in the pool have been detected.
139 .It Sy vdev.too_small
140 Issued when the system (kernel) have removed a device, and ZFS
141 notices that the device isn't there any more.
142 This is usually followed by a
143 .Sy probe_failure
144 event.
145 .It Sy vdev.bad_label
146 Issued when the label is OK but invalid.
147 .It Sy vdev.bad_ashift
148 Issued when the ashift alignment requirement has increased.
149 .It Sy vdev.remove
150 Issued when a vdev is detached from a mirror (or a spare detached from a
151 vdev where it have been used to replace a failed drive - only works if
152 the original drive have been re-added).
153 .It Sy vdev.clear
154 Issued when clearing device errors in a pool.
155 Such as running
156 .Nm zpool Cm clear
157 on a device in the pool.
158 .It Sy vdev.check
159 Issued when a check to see if a given vdev could be opened is started.
160 .It Sy vdev.spare
161 Issued when a spare have kicked in to replace a failed device.
162 .It Sy vdev.autoexpand
163 Issued when a vdev can be automatically expanded.
164 .It Sy io_failure
165 Issued when there is an I/O failure in a vdev in the pool.
166 .It Sy probe_failure
167 Issued when a probe fails on a vdev.
168 This would occur if a vdev
169 have been kicked from the system outside of ZFS (such as the kernel
170 have removed the device).
171 .It Sy log_replay
172 Issued when the intent log cannot be replayed.
173 The can occur in the case of a missing or damaged log device.
174 .It Sy resilver.start
175 Issued when a resilver is started.
176 .It Sy resilver.finish
177 Issued when the running resilver have finished.
178 .It Sy scrub.start
179 Issued when a scrub is started on a pool.
180 .It Sy scrub.finish
181 Issued when a pool has finished scrubbing.
182 .It Sy scrub.abort
183 Issued when a scrub is aborted on a pool.
184 .It Sy scrub.resume
185 Issued when a scrub is resumed on a pool.
186 .It Sy scrub.paused
187 Issued when a scrub is paused on a pool.
188 .It Sy bootfs.vdev.attach
189 .El
190 .
191 .Sh PAYLOADS
192 This is the payload (data, information) that accompanies an
193 event.
194 .Pp
195 For
196 .Xr zed 8 ,
197 these are set to uppercase and prefixed with
198 .Sy ZEVENT_ .
199 .Pp
200 .Bl -tag -compact -width "vdev_cksum_errors"
201 .It Sy pool
202 Pool name.
203 .It Sy pool_failmode
204 Failmode -
205 .Sy wait ,
206 .Sy continue ,
207 or
208 .Sy panic .
209 See the
210 .Sy failmode
211 property in
212 .Xr zpoolprops 7
213 for more information.
214 .It Sy pool_guid
215 The GUID of the pool.
216 .It Sy pool_context
217 The load state for the pool (0=none, 1=open, 2=import, 3=tryimport, 4=recover
218 5=error).
219 .It Sy vdev_guid
220 The GUID of the vdev in question (the vdev failing or operated upon with
221 .Nm zpool Cm clear ,
222 etc.).
223 .It Sy vdev_type
224 Type of vdev -
225 .Sy disk ,
226 .Sy file ,
227 .Sy mirror ,
228 etc.
229 See the
230 .Sy Virtual Devices
231 section of
232 .Xr zpoolconcepts 7
233 for more information on possible values.
234 .It Sy vdev_path
235 Full path of the vdev, including any
236 .Em -partX .
237 .It Sy vdev_devid
238 ID of vdev (if any).
239 .It Sy vdev_fru
240 Physical FRU location.
241 .It Sy vdev_state
242 State of vdev (0=uninitialized, 1=closed, 2=offline, 3=removed, 4=failed to
243 open, 5=faulted, 6=degraded, 7=healthy).
244 .It Sy vdev_ashift
245 The ashift value of the vdev.
246 .It Sy vdev_complete_ts
247 The time the last I/O request completed for the specified vdev.
248 .It Sy vdev_delta_ts
249 The time since the last I/O request completed for the specified vdev.
250 .It Sy vdev_spare_paths
251 List of spares, including full path and any
252 .Em -partX .
253 .It Sy vdev_spare_guids
254 GUID(s) of spares.
255 .It Sy vdev_read_errors
256 How many read errors that have been detected on the vdev.
257 .It Sy vdev_write_errors
258 How many write errors that have been detected on the vdev.
259 .It Sy vdev_cksum_errors
260 How many checksum errors that have been detected on the vdev.
261 .It Sy parent_guid
262 GUID of the vdev parent.
263 .It Sy parent_type
264 Type of parent.
265 See
266 .Sy vdev_type .
267 .It Sy parent_path
268 Path of the vdev parent (if any).
269 .It Sy parent_devid
270 ID of the vdev parent (if any).
271 .It Sy zio_objset
272 The object set number for a given I/O request.
273 .It Sy zio_object
274 The object number for a given I/O request.
275 .It Sy zio_level
276 The indirect level for the block.
277 Level 0 is the lowest level and includes data blocks.
278 Values > 0 indicate metadata blocks at the appropriate level.
279 .It Sy zio_blkid
280 The block ID for a given I/O request.
281 .It Sy zio_err
282 The error number for a failure when handling a given I/O request,
283 compatible with
284 .Xr errno 3
285 with the value of
286 .Sy EBADE
287 used to indicate a ZFS checksum error.
288 .It Sy zio_offset
289 The offset in bytes of where to write the I/O request for the specified vdev.
290 .It Sy zio_size
291 The size in bytes of the I/O request.
292 .It Sy zio_flags
293 The current flags describing how the I/O request should be handled.
294 See the
295 .Sy I/O FLAGS
296 section for the full list of I/O flags.
297 .It Sy zio_stage
298 The current stage of the I/O in the pipeline.
299 See the
300 .Sy I/O STAGES
301 section for a full list of all the I/O stages.
302 .It Sy zio_pipeline
303 The valid pipeline stages for the I/O.
304 See the
305 .Sy I/O STAGES
306 section for a full list of all the I/O stages.
307 .It Sy zio_delay
308 The time elapsed (in nanoseconds) waiting for the block layer to complete the
309 I/O request.
310 Unlike
311 .Sy zio_delta ,
312 this does not include any vdev queuing time and is
313 therefore solely a measure of the block layer performance.
314 .It Sy zio_timestamp
315 The time when a given I/O request was submitted.
316 .It Sy zio_delta
317 The time required to service a given I/O request.
318 .It Sy prev_state
319 The previous state of the vdev.
320 .It Sy cksum_algorithm
321 Checksum algorithm used.
322 See
323 .Xr zfsprops 7
324 for more information on the available checksum algorithms.
325 .It Sy cksum_byteswap
326 Whether or not the data is byteswapped.
327 .It Sy bad_ranges
328 .No [\& Ns Ar start , end )
329 pairs of corruption offsets.
330 Offsets are always aligned on a 64-bit boundary,
331 and can include some gaps of non-corruption.
332 (See
333 .Sy bad_ranges_min_gap )
334 .It Sy bad_ranges_min_gap
335 In order to bound the size of the
336 .Sy bad_ranges
337 array, gaps of non-corruption
338 less than or equal to
339 .Sy bad_ranges_min_gap
340 bytes have been merged with
341 adjacent corruption.
342 Always at least 8 bytes, since corruption is detected on a 64-bit word basis.
343 .It Sy bad_range_sets
344 This array has one element per range in
345 .Sy bad_ranges .
346 Each element contains
347 the count of bits in that range which were clear in the good data and set
348 in the bad data.
349 .It Sy bad_range_clears
350 This array has one element per range in
351 .Sy bad_ranges .
352 Each element contains
353 the count of bits for that range which were set in the good data and clear in
354 the bad data.
355 .It Sy bad_set_bits
356 If this field exists, it is an array of
357 .Pq Ar bad data No & ~( Ns Ar good data ) ;
358 that is, the bits set in the bad data which are cleared in the good data.
359 Each element corresponds a byte whose offset is in a range in
360 .Sy bad_ranges ,
361 and the array is ordered by offset.
362 Thus, the first element is the first byte in the first
363 .Sy bad_ranges
364 range, and the last element is the last byte in the last
365 .Sy bad_ranges
366 range.
367 .It Sy bad_cleared_bits
368 Like
369 .Sy bad_set_bits ,
370 but contains
371 .Pq Ar good data No & ~( Ns Ar bad data ) ;
372 that is, the bits set in the good data which are cleared in the bad data.
373 .El
374 .
375 .Sh I/O STAGES
376 The ZFS I/O pipeline is comprised of various stages which are defined below.
377 The individual stages are used to construct these basic I/O
378 operations: Read, Write, Free, Claim, Flush and Trim.
379 These stages may be
380 set on an event to describe the life cycle of a given I/O request.
381 .Pp
382 .TS
383 tab(:);
384 l l l .
385 Stage:Bit Mask:Operations
386 _:_:_
387 ZIO_STAGE_OPEN:0x00000001:RWFCXT
389 ZIO_STAGE_READ_BP_INIT:0x00000002:R-----
390 ZIO_STAGE_WRITE_BP_INIT:0x00000004:-W----
391 ZIO_STAGE_FREE_BP_INIT:0x00000008:--F---
392 ZIO_STAGE_ISSUE_ASYNC:0x00000010:-WF--T
393 ZIO_STAGE_WRITE_COMPRESS:0x00000020:-W----
395 ZIO_STAGE_ENCRYPT:0x00000040:-W----
396 ZIO_STAGE_CHECKSUM_GENERATE:0x00000080:-W----
398 ZIO_STAGE_NOP_WRITE:0x00000100:-W----
400 ZIO_STAGE_BRT_FREE:0x00000200:--F---
402 ZIO_STAGE_DDT_READ_START:0x00000400:R-----
403 ZIO_STAGE_DDT_READ_DONE:0x00000800:R-----
404 ZIO_STAGE_DDT_WRITE:0x00001000:-W----
405 ZIO_STAGE_DDT_FREE:0x00002000:--F---
407 ZIO_STAGE_GANG_ASSEMBLE:0x00004000:RWFC--
408 ZIO_STAGE_GANG_ISSUE:0x00008000:RWFC--
410 ZIO_STAGE_DVA_THROTTLE:0x00010000:-W----
411 ZIO_STAGE_DVA_ALLOCATE:0x00020000:-W----
412 ZIO_STAGE_DVA_FREE:0x00040000:--F---
413 ZIO_STAGE_DVA_CLAIM:0x00080000:---C--
415 ZIO_STAGE_READY:0x00100000:RWFCIT
417 ZIO_STAGE_VDEV_IO_START:0x00200000:RW--XT
418 ZIO_STAGE_VDEV_IO_DONE:0x00400000:RW--XT
419 ZIO_STAGE_VDEV_IO_ASSESS:0x00800000:RW--XT
421 ZIO_STAGE_CHECKSUM_VERIFY:0x01000000:R-----
422 ZIO_STAGE_DIO_CHECKSUM_VERIFY:0x02000000:-W----
424 ZIO_STAGE_DONE:0x04000000:RWFCXT
425 .TE
426 .
427 .Sh I/O FLAGS
428 Every I/O request in the pipeline contains a set of flags which describe its
429 function and are used to govern its behavior.
430 These flags will be set in an event as a
431 .Sy zio_flags
432 payload entry.
433 .Pp
434 .TS
435 tab(:);
436 l l .
437 Flag:Bit Mask
438 _:_
439 ZIO_FLAG_DONT_AGGREGATE:0x00000001
440 ZIO_FLAG_IO_REPAIR:0x00000002
441 ZIO_FLAG_SELF_HEAL:0x00000004
442 ZIO_FLAG_RESILVER:0x00000008
443 ZIO_FLAG_SCRUB:0x00000010
444 ZIO_FLAG_SCAN_THREAD:0x00000020
445 ZIO_FLAG_PHYSICAL:0x00000040
447 ZIO_FLAG_CANFAIL:0x00000080
448 ZIO_FLAG_SPECULATIVE:0x00000100
449 ZIO_FLAG_CONFIG_WRITER:0x00000200
450 ZIO_FLAG_DONT_RETRY:0x00000400
451 ZIO_FLAG_NODATA:0x00001000
452 ZIO_FLAG_INDUCE_DAMAGE:0x00002000
454 ZIO_FLAG_IO_ALLOCATING:0x00004000
455 ZIO_FLAG_IO_RETRY:0x00008000
456 ZIO_FLAG_PROBE:0x00010000
457 ZIO_FLAG_TRYHARD:0x00020000
458 ZIO_FLAG_OPTIONAL:0x00040000
460 ZIO_FLAG_DONT_QUEUE:0x00080000
461 ZIO_FLAG_DONT_PROPAGATE:0x00100000
462 ZIO_FLAG_IO_BYPASS:0x00200000
463 ZIO_FLAG_IO_REWRITE:0x00400000
464 ZIO_FLAG_RAW_COMPRESS:0x00800000
465 ZIO_FLAG_RAW_ENCRYPT:0x01000000
467 ZIO_FLAG_GANG_CHILD:0x02000000
468 ZIO_FLAG_DDT_CHILD:0x04000000
469 ZIO_FLAG_GODFATHER:0x08000000
470 ZIO_FLAG_NOPWRITE:0x10000000
471 ZIO_FLAG_REEXECUTED:0x20000000
472 ZIO_FLAG_DELEGATED:0x40000000
473 ZIO_FLAG_FASTWRITE:0x80000000
474 .TE
475 .
476 .Sh SEE ALSO
477 .Xr zfs 4 ,
478 .Xr zed 8 ,
479 .Xr zpool-wait 8