| blob | 7006d8209d2ede981e8f7e23fd9eb7489dd1b0f5 |
1 ===========================
2 Linux for S/390 and zSeries
3 ===========================
5 Common Device Support (CDS)
6 Device Driver I/O Support Routines
8 Authors:
9 - Ingo Adlung
10 - Cornelia Huck
12 Copyright, IBM Corp. 1999-2002
14 Introduction
15 ============
17 This document describes the common device support routines for Linux/390.
18 Different than other hardware architectures, ESA/390 has defined a unified
19 I/O access method. This gives relief to the device drivers as they don't
20 have to deal with different bus types, polling versus interrupt
21 processing, shared versus non-shared interrupt processing, DMA versus port
22 I/O (PIO), and other hardware features more. However, this implies that
23 either every single device driver needs to implement the hardware I/O
24 attachment functionality itself, or the operating system provides for a
25 unified method to access the hardware, providing all the functionality that
26 every single device driver would have to provide itself.
28 The document does not intend to explain the ESA/390 hardware architecture in
29 every detail.This information can be obtained from the ESA/390 Principles of
30 Operation manual (IBM Form. No. SA22-7201).
32 In order to build common device support for ESA/390 I/O interfaces, a
33 functional layer was introduced that provides generic I/O access methods to
34 the hardware.
36 The common device support layer comprises the I/O support routines defined
37 below. Some of them implement common Linux device driver interfaces, while
38 some of them are ESA/390 platform specific.
40 Note:
41 In order to write a driver for S/390, you also need to look into the interface
42 described in Documentation/s390/driver-model.rst.
44 Note for porting drivers from 2.4:
46 The major changes are:
48 * The functions use a ccw_device instead of an irq (subchannel).
49 * All drivers must define a ccw_driver (see driver-model.txt) and the associated
50 functions.
51 * request_irq() and free_irq() are no longer done by the driver.
52 * The oper_handler is (kindof) replaced by the probe() and set_online() functions
53 of the ccw_driver.
54 * The not_oper_handler is (kindof) replaced by the remove() and set_offline()
55 functions of the ccw_driver.
56 * The channel device layer is gone.
57 * The interrupt handlers must be adapted to use a ccw_device as argument.
58 Moreover, they don't return a devstat, but an irb.
59 * Before initiating an io, the options must be set via ccw_device_set_options().
60 * Instead of calling read_dev_chars()/read_conf_data(), the driver issues
61 the channel program and handles the interrupt itself.
63 ccw_device_get_ciw()
64 get commands from extended sense data.
66 ccw_device_start(), ccw_device_start_timeout(), ccw_device_start_key(), ccw_device_start_key_timeout()
67 initiate an I/O request.
69 ccw_device_resume()
70 resume channel program execution.
72 ccw_device_halt()
73 terminate the current I/O request processed on the device.
75 do_IRQ()
76 generic interrupt routine. This function is called by the interrupt entry
77 routine whenever an I/O interrupt is presented to the system. The do_IRQ()
78 routine determines the interrupt status and calls the device specific
79 interrupt handler according to the rules (flags) defined during I/O request
80 initiation with do_IO().
82 The next chapters describe the functions other than do_IRQ() in more details.
83 The do_IRQ() interface is not described, as it is called from the Linux/390
84 first level interrupt handler only and does not comprise a device driver
85 callable interface. Instead, the functional description of do_IO() also
86 describes the input to the device specific interrupt handler.
88 Note:
89 All explanations apply also to the 64 bit architecture s390x.
92 Common Device Support (CDS) for Linux/390 Device Drivers
93 ========================================================
95 General Information
96 -------------------
98 The following chapters describe the I/O related interface routines the
99 Linux/390 common device support (CDS) provides to allow for device specific
100 driver implementations on the IBM ESA/390 hardware platform. Those interfaces
101 intend to provide the functionality required by every device driver
102 implementation to allow to drive a specific hardware device on the ESA/390
103 platform. Some of the interface routines are specific to Linux/390 and some
104 of them can be found on other Linux platforms implementations too.
105 Miscellaneous function prototypes, data declarations, and macro definitions
106 can be found in the architecture specific C header file
107 linux/arch/s390/include/asm/irq.h.
109 Overview of CDS interface concepts
110 ----------------------------------
112 Different to other hardware platforms, the ESA/390 architecture doesn't define
113 interrupt lines managed by a specific interrupt controller and bus systems
114 that may or may not allow for shared interrupts, DMA processing, etc.. Instead,
115 the ESA/390 architecture has implemented a so called channel subsystem, that
116 provides a unified view of the devices physically attached to the systems.
117 Though the ESA/390 hardware platform knows about a huge variety of different
118 peripheral attachments like disk devices (aka. DASDs), tapes, communication
119 controllers, etc. they can all be accessed by a well defined access method and
120 they are presenting I/O completion a unified way : I/O interruptions. Every
121 single device is uniquely identified to the system by a so called subchannel,
122 where the ESA/390 architecture allows for 64k devices be attached.
124 Linux, however, was first built on the Intel PC architecture, with its two
125 cascaded 8259 programmable interrupt controllers (PICs), that allow for a
126 maximum of 15 different interrupt lines. All devices attached to such a system
127 share those 15 interrupt levels. Devices attached to the ISA bus system must
128 not share interrupt levels (aka. IRQs), as the ISA bus bases on edge triggered
129 interrupts. MCA, EISA, PCI and other bus systems base on level triggered
130 interrupts, and therewith allow for shared IRQs. However, if multiple devices
131 present their hardware status by the same (shared) IRQ, the operating system
132 has to call every single device driver registered on this IRQ in order to
133 determine the device driver owning the device that raised the interrupt.
135 Up to kernel 2.4, Linux/390 used to provide interfaces via the IRQ (subchannel).
136 For internal use of the common I/O layer, these are still there. However,
137 device drivers should use the new calling interface via the ccw_device only.
139 During its startup the Linux/390 system checks for peripheral devices. Each
140 of those devices is uniquely defined by a so called subchannel by the ESA/390
141 channel subsystem. While the subchannel numbers are system generated, each
142 subchannel also takes a user defined attribute, the so called device number.
143 Both subchannel number and device number cannot exceed 65535. During sysfs
144 initialisation, the information about control unit type and device types that
145 imply specific I/O commands (channel command words - CCWs) in order to operate
146 the device are gathered. Device drivers can retrieve this set of hardware
147 information during their initialization step to recognize the devices they
148 support using the information saved in the struct ccw_device given to them.
149 This methods implies that Linux/390 doesn't require to probe for free (not
150 armed) interrupt request lines (IRQs) to drive its devices with. Where
151 applicable, the device drivers can use issue the READ DEVICE CHARACTERISTICS
152 ccw to retrieve device characteristics in its online routine.
154 In order to allow for easy I/O initiation the CDS layer provides a
155 ccw_device_start() interface that takes a device specific channel program (one
156 or more CCWs) as input sets up the required architecture specific control blocks
157 and initiates an I/O request on behalf of the device driver. The
158 ccw_device_start() routine allows to specify whether it expects the CDS layer
159 to notify the device driver for every interrupt it observes, or with final status
160 only. See ccw_device_start() for more details. A device driver must never issue
161 ESA/390 I/O commands itself, but must use the Linux/390 CDS interfaces instead.
163 For long running I/O request to be canceled, the CDS layer provides the
164 ccw_device_halt() function. Some devices require to initially issue a HALT
165 SUBCHANNEL (HSCH) command without having pending I/O requests. This function is
166 also covered by ccw_device_halt().
169 get_ciw() - get command information word
171 This call enables a device driver to get information about supported commands
172 from the extended SenseID data.
174 ::
176 struct ciw *
177 ccw_device_get_ciw(struct ccw_device *cdev, __u32 cmd);
179 ==== ========================================================
180 cdev The ccw_device for which the command is to be retrieved.
181 cmd The command type to be retrieved.
182 ==== ========================================================
184 ccw_device_get_ciw() returns:
186 ===== ================================================================
187 NULL No extended data available, invalid device or command not found.
188 !NULL The command requested.
189 ===== ================================================================
191 ::
193 ccw_device_start() - Initiate I/O Request
195 The ccw_device_start() routines is the I/O request front-end processor. All
196 device driver I/O requests must be issued using this routine. A device driver
197 must not issue ESA/390 I/O commands itself. Instead the ccw_device_start()
198 routine provides all interfaces required to drive arbitrary devices.
200 This description also covers the status information passed to the device
201 driver's interrupt handler as this is related to the rules (flags) defined
202 with the associated I/O request when calling ccw_device_start().
204 ::
206 int ccw_device_start(struct ccw_device *cdev,
207 struct ccw1 *cpa,
208 unsigned long intparm,
209 __u8 lpm,
210 unsigned long flags);
211 int ccw_device_start_timeout(struct ccw_device *cdev,
212 struct ccw1 *cpa,
213 unsigned long intparm,
214 __u8 lpm,
215 unsigned long flags,
216 int expires);
217 int ccw_device_start_key(struct ccw_device *cdev,
218 struct ccw1 *cpa,
219 unsigned long intparm,
220 __u8 lpm,
221 __u8 key,
222 unsigned long flags);
223 int ccw_device_start_key_timeout(struct ccw_device *cdev,
224 struct ccw1 *cpa,
225 unsigned long intparm,
226 __u8 lpm,
227 __u8 key,
228 unsigned long flags,
229 int expires);
231 ============= =============================================================
232 cdev ccw_device the I/O is destined for
233 cpa logical start address of channel program
234 user_intparm user specific interrupt information; will be presented
235 back to the device driver's interrupt handler. Allows a
236 device driver to associate the interrupt with a
237 particular I/O request.
238 lpm defines the channel path to be used for a specific I/O
239 request. A value of 0 will make cio use the opm.
240 key the storage key to use for the I/O (useful for operating on a
241 storage with a storage key != default key)
242 flag defines the action to be performed for I/O processing
243 expires timeout value in jiffies. The common I/O layer will terminate
244 the running program after this and call the interrupt handler
245 with ERR_PTR(-ETIMEDOUT) as irb.
246 ============= =============================================================
248 Possible flag values are:
250 ========================= =============================================
251 DOIO_ALLOW_SUSPEND channel program may become suspended
252 DOIO_DENY_PREFETCH don't allow for CCW prefetch; usually
253 this implies the channel program might
254 become modified
255 DOIO_SUPPRESS_INTER don't call the handler on intermediate status
256 ========================= =============================================
258 The cpa parameter points to the first format 1 CCW of a channel program::
260 struct ccw1 {
261 __u8 cmd_code;/* command code */
262 __u8 flags; /* flags, like IDA addressing, etc. */
263 __u16 count; /* byte count */
264 __u32 cda; /* data address */
265 } __attribute__ ((packed,aligned(8)));
267 with the following CCW flags values defined:
269 =================== =========================
270 CCW_FLAG_DC data chaining
271 CCW_FLAG_CC command chaining
272 CCW_FLAG_SLI suppress incorrect length
273 CCW_FLAG_SKIP skip
274 CCW_FLAG_PCI PCI
275 CCW_FLAG_IDA indirect addressing
276 CCW_FLAG_SUSPEND suspend
277 =================== =========================
280 Via ccw_device_set_options(), the device driver may specify the following
281 options for the device:
283 ========================= ======================================
284 DOIO_EARLY_NOTIFICATION allow for early interrupt notification
285 DOIO_REPORT_ALL report all interrupt conditions
286 ========================= ======================================
289 The ccw_device_start() function returns:
291 ======== ======================================================================
292 0 successful completion or request successfully initiated
293 -EBUSY The device is currently processing a previous I/O request, or there is
294 a status pending at the device.
295 -ENODEV cdev is invalid, the device is not operational or the ccw_device is
296 not online.
297 ======== ======================================================================
299 When the I/O request completes, the CDS first level interrupt handler will
300 accumulate the status in a struct irb and then call the device interrupt handler.
301 The intparm field will contain the value the device driver has associated with a
302 particular I/O request. If a pending device status was recognized,
303 intparm will be set to 0 (zero). This may happen during I/O initiation or delayed
304 by an alert status notification. In any case this status is not related to the
305 current (last) I/O request. In case of a delayed status notification no special
306 interrupt will be presented to indicate I/O completion as the I/O request was
307 never started, even though ccw_device_start() returned with successful completion.
309 The irb may contain an error value, and the device driver should check for this
310 first:
312 ========== =================================================================
313 -ETIMEDOUT the common I/O layer terminated the request after the specified
314 timeout value
315 -EIO the common I/O layer terminated the request due to an error state
316 ========== =================================================================
318 If the concurrent sense flag in the extended status word (esw) in the irb is
319 set, the field erw.scnt in the esw describes the number of device specific
320 sense bytes available in the extended control word irb->scsw.ecw[]. No device
321 sensing by the device driver itself is required.
323 The device interrupt handler can use the following definitions to investigate
324 the primary unit check source coded in sense byte 0 :
326 ======================= ====
327 SNS0_CMD_REJECT 0x80
328 SNS0_INTERVENTION_REQ 0x40
329 SNS0_BUS_OUT_CHECK 0x20
330 SNS0_EQUIPMENT_CHECK 0x10
331 SNS0_DATA_CHECK 0x08
332 SNS0_OVERRUN 0x04
333 SNS0_INCOMPL_DOMAIN 0x01
334 ======================= ====
336 Depending on the device status, multiple of those values may be set together.
337 Please refer to the device specific documentation for details.
339 The irb->scsw.cstat field provides the (accumulated) subchannel status :
341 ========================= ============================
342 SCHN_STAT_PCI program controlled interrupt
343 SCHN_STAT_INCORR_LEN incorrect length
344 SCHN_STAT_PROG_CHECK program check
345 SCHN_STAT_PROT_CHECK protection check
346 SCHN_STAT_CHN_DATA_CHK channel data check
347 SCHN_STAT_CHN_CTRL_CHK channel control check
348 SCHN_STAT_INTF_CTRL_CHK interface control check
349 SCHN_STAT_CHAIN_CHECK chaining check
350 ========================= ============================
352 The irb->scsw.dstat field provides the (accumulated) device status :
354 ===================== =================
355 DEV_STAT_ATTENTION attention
356 DEV_STAT_STAT_MOD status modifier
357 DEV_STAT_CU_END control unit end
358 DEV_STAT_BUSY busy
359 DEV_STAT_CHN_END channel end
360 DEV_STAT_DEV_END device end
361 DEV_STAT_UNIT_CHECK unit check
362 DEV_STAT_UNIT_EXCEP unit exception
363 ===================== =================
365 Please see the ESA/390 Principles of Operation manual for details on the
366 individual flag meanings.
368 Usage Notes:
370 ccw_device_start() must be called disabled and with the ccw device lock held.
372 The device driver is allowed to issue the next ccw_device_start() call from
373 within its interrupt handler already. It is not required to schedule a
374 bottom-half, unless a non deterministically long running error recovery procedure
375 or similar needs to be scheduled. During I/O processing the Linux/390 generic
376 I/O device driver support has already obtained the IRQ lock, i.e. the handler
377 must not try to obtain it again when calling ccw_device_start() or we end in a
378 deadlock situation!
380 If a device driver relies on an I/O request to be completed prior to start the
381 next it can reduce I/O processing overhead by chaining a NoOp I/O command
382 CCW_CMD_NOOP to the end of the submitted CCW chain. This will force Channel-End
383 and Device-End status to be presented together, with a single interrupt.
384 However, this should be used with care as it implies the channel will remain
385 busy, not being able to process I/O requests for other devices on the same
386 channel. Therefore e.g. read commands should never use this technique, as the
387 result will be presented by a single interrupt anyway.
389 In order to minimize I/O overhead, a device driver should use the
390 DOIO_REPORT_ALL only if the device can report intermediate interrupt
391 information prior to device-end the device driver urgently relies on. In this
392 case all I/O interruptions are presented to the device driver until final
393 status is recognized.
395 If a device is able to recover from asynchronously presented I/O errors, it can
396 perform overlapping I/O using the DOIO_EARLY_NOTIFICATION flag. While some
397 devices always report channel-end and device-end together, with a single
398 interrupt, others present primary status (channel-end) when the channel is
399 ready for the next I/O request and secondary status (device-end) when the data
400 transmission has been completed at the device.
402 Above flag allows to exploit this feature, e.g. for communication devices that
403 can handle lost data on the network to allow for enhanced I/O processing.
405 Unless the channel subsystem at any time presents a secondary status interrupt,
406 exploiting this feature will cause only primary status interrupts to be
407 presented to the device driver while overlapping I/O is performed. When a
408 secondary status without error (alert status) is presented, this indicates
409 successful completion for all overlapping ccw_device_start() requests that have
410 been issued since the last secondary (final) status.
412 Channel programs that intend to set the suspend flag on a channel command word
413 (CCW) must start the I/O operation with the DOIO_ALLOW_SUSPEND option or the
414 suspend flag will cause a channel program check. At the time the channel program
415 becomes suspended an intermediate interrupt will be generated by the channel
416 subsystem.
418 ccw_device_resume() - Resume Channel Program Execution
420 If a device driver chooses to suspend the current channel program execution by
421 setting the CCW suspend flag on a particular CCW, the channel program execution
422 is suspended. In order to resume channel program execution the CIO layer
423 provides the ccw_device_resume() routine.
425 ::
427 int ccw_device_resume(struct ccw_device *cdev);
429 ==== ================================================
430 cdev ccw_device the resume operation is requested for
431 ==== ================================================
433 The ccw_device_resume() function returns:
435 ========= ==============================================
436 0 suspended channel program is resumed
437 -EBUSY status pending
438 -ENODEV cdev invalid or not-operational subchannel
439 -EINVAL resume function not applicable
440 -ENOTCONN there is no I/O request pending for completion
441 ========= ==============================================
443 Usage Notes:
445 Please have a look at the ccw_device_start() usage notes for more details on
446 suspended channel programs.
448 ccw_device_halt() - Halt I/O Request Processing
450 Sometimes a device driver might need a possibility to stop the processing of
451 a long-running channel program or the device might require to initially issue
452 a halt subchannel (HSCH) I/O command. For those purposes the ccw_device_halt()
453 command is provided.
455 ccw_device_halt() must be called disabled and with the ccw device lock held.
457 ::
459 int ccw_device_halt(struct ccw_device *cdev,
460 unsigned long intparm);
462 ======= =====================================================
463 cdev ccw_device the halt operation is requested for
464 intparm interruption parameter; value is only used if no I/O
465 is outstanding, otherwise the intparm associated with
466 the I/O request is returned
467 ======= =====================================================
469 The ccw_device_halt() function returns:
471 ======= ==============================================================
472 0 request successfully initiated
473 -EBUSY the device is currently busy, or status pending.
474 -ENODEV cdev invalid.
475 -EINVAL The device is not operational or the ccw device is not online.
476 ======= ==============================================================
478 Usage Notes:
480 A device driver may write a never-ending channel program by writing a channel
481 program that at its end loops back to its beginning by means of a transfer in
482 channel (TIC) command (CCW_CMD_TIC). Usually this is performed by network
483 device drivers by setting the PCI CCW flag (CCW_FLAG_PCI). Once this CCW is
484 executed a program controlled interrupt (PCI) is generated. The device driver
485 can then perform an appropriate action. Prior to interrupt of an outstanding
486 read to a network device (with or without PCI flag) a ccw_device_halt()
487 is required to end the pending operation.
489 ::
491 ccw_device_clear() - Terminage I/O Request Processing
493 In order to terminate all I/O processing at the subchannel, the clear subchannel
494 (CSCH) command is used. It can be issued via ccw_device_clear().
496 ccw_device_clear() must be called disabled and with the ccw device lock held.
498 ::
500 int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm);
502 ======= ===============================================
503 cdev ccw_device the clear operation is requested for
504 intparm interruption parameter (see ccw_device_halt())
505 ======= ===============================================
507 The ccw_device_clear() function returns:
509 ======= ==============================================================
510 0 request successfully initiated
511 -ENODEV cdev invalid
512 -EINVAL The device is not operational or the ccw device is not online.
513 ======= ==============================================================
515 Miscellaneous Support Routines
516 ------------------------------
518 This chapter describes various routines to be used in a Linux/390 device
519 driver programming environment.
521 get_ccwdev_lock()
523 Get the address of the device specific lock. This is then used in
524 spin_lock() / spin_unlock() calls.
526 ::
528 __u8 ccw_device_get_path_mask(struct ccw_device *cdev);
530 Get the mask of the path currently available for cdev.