1 Linux Kernel 2.6 series
2 SCSI mid_level - lower_level driver interface
3 =============================================
7 This document outlines the interface between the Linux SCSI mid level and
8 SCSI lower level drivers. Lower level drivers (LLDs) are variously called
9 host bus adapter (HBA) drivers and host drivers (HD). A "host" in this
10 context is a bridge between a computer IO bus (e.g. PCI or ISA) and a
11 single SCSI initiator port on a SCSI transport. An "initiator" port
12 (SCSI terminology, see SAM-3 at http://www.t10.org) sends SCSI commands
13 to "target" SCSI ports (e.g. disks). There can be many LLDs in a running
14 system, but only one per hardware type. Most LLDs can control one or more
15 SCSI HBAs. Some HBAs contain multiple hosts.
17 In some cases the SCSI transport is an external bus that already has
18 its own subsystem in Linux (e.g. USB and ieee1394). In such cases the
19 SCSI subsystem LLD is a software bridge to the other driver subsystem.
20 Examples are the usb-storage driver (found in the drivers/usb/storage
21 directory) and the ieee1394/sbp2 driver (found in the drivers/ieee1394
24 For example, the aic7xxx LLD controls Adaptec SCSI parallel interface
25 (SPI) controllers based on that company's 7xxx chip series. The aic7xxx
26 LLD can be built into the kernel or loaded as a module. There can only be
27 one aic7xxx LLD running in a Linux system but it may be controlling many
28 HBAs. These HBAs might be either on PCI daughter-boards or built into
29 the motherboard (or both). Some aic7xxx based HBAs are dual controllers
30 and thus represent two hosts. Like most modern HBAs, each aic7xxx host
31 has its own PCI device address. [The one-to-one correspondence between
32 a SCSI host and a PCI device is common but not required (e.g. with
35 The SCSI mid level isolates an LLD from other layers such as the SCSI
36 upper layer drivers and the block layer.
38 This version of the document roughly matches linux kernel version 2.6.8 .
42 There is a SCSI documentation directory within the kernel source tree,
43 typically Documentation/scsi . Most documents are in plain
44 (i.e. ASCII) text. This file is named scsi_mid_low_api.txt and can be
45 found in that directory. A more recent copy of this document may be found
46 at http://web.archive.org/web/20070107183357rn_1/sg.torque.net/scsi/.
47 Many LLDs are documented there (e.g. aic7xxx.txt). The SCSI mid-level is
48 briefly described in scsi.txt which contains a url to a document
49 describing the SCSI subsystem in the lk 2.4 series. Two upper level
50 drivers have documents in that directory: st.txt (SCSI tape driver) and
51 scsi-generic.txt (for the sg driver).
53 Some documentation (or urls) for LLDs may be found in the C source code
54 or in the same directory as the C source code. For example to find a url
55 about the USB mass storage driver see the
56 /usr/src/linux/drivers/usb/storage directory.
60 Traditionally an LLD for the SCSI subsystem has been at least two files in
61 the drivers/scsi directory. For example, a driver called "xyz" has a header
62 file "xyz.h" and a source file "xyz.c". [Actually there is no good reason
63 why this couldn't all be in one file; the header file is superfluous.] Some
64 drivers that have been ported to several operating systems have more than
65 two files. For example the aic7xxx driver has separate files for generic
66 and OS-specific code (e.g. FreeBSD and Linux). Such drivers tend to have
67 their own directory under the drivers/scsi directory.
69 When a new LLD is being added to Linux, the following files (found in the
70 drivers/scsi directory) will need some attention: Makefile and Kconfig .
71 It is probably best to study how existing LLDs are organized.
73 As the 2.5 series development kernels evolve into the 2.6 series
74 production series, changes are being introduced into this interface. An
75 example of this is driver initialization code where there are now 2 models
76 available. The older one, similar to what was found in the lk 2.4 series,
77 is based on hosts that are detected at HBA driver load time. This will be
78 referred to the "passive" initialization model. The newer model allows HBAs
79 to be hot plugged (and unplugged) during the lifetime of the LLD and will
80 be referred to as the "hotplug" initialization model. The newer model is
81 preferred as it can handle both traditional SCSI equipment that is
82 permanently connected as well as modern "SCSI" devices (e.g. USB or
83 IEEE 1394 connected digital cameras) that are hotplugged. Both
84 initialization models are discussed in the following sections.
86 An LLD interfaces to the SCSI subsystem several ways:
87 a) directly invoking functions supplied by the mid level
88 b) passing a set of function pointers to a registration function
89 supplied by the mid level. The mid level will then invoke these
90 functions at some point in the future. The LLD will supply
91 implementations of these functions.
92 c) direct access to instances of well known data structures maintained
95 Those functions in group a) are listed in a section entitled "Mid level
96 supplied functions" below.
98 Those functions in group b) are listed in a section entitled "Interface
99 functions" below. Their function pointers are placed in the members of
100 "struct scsi_host_template", an instance of which is passed to
101 scsi_host_alloc() ** . Those interface functions that the LLD does not
102 wish to supply should have NULL placed in the corresponding member of
103 struct scsi_host_template. Defining an instance of struct
104 scsi_host_template at file scope will cause NULL to be placed in function
105 pointer members not explicitly initialized.
107 Those usages in group c) should be handled with care, especially in a
108 "hotplug" environment. LLDs should be aware of the lifetime of instances
109 that are shared with the mid level and other layers.
111 All functions defined within an LLD and all data defined at file scope
112 should be static. For example the slave_alloc() function in an LLD
113 called "xxx" could be defined as
114 "static int xxx_slave_alloc(struct scsi_device * sdev) { /* code */ }"
116 ** the scsi_host_alloc() function is a replacement for the rather vaguely
117 named scsi_register() function in most situations. The scsi_register()
118 and scsi_unregister() functions remain to support legacy LLDs that use
119 the passive initialization model.
122 Hotplug initialization model
123 ============================
124 In this model an LLD controls when SCSI hosts are introduced and removed
125 from the SCSI subsystem. Hosts can be introduced as early as driver
126 initialization and removed as late as driver shutdown. Typically a driver
127 will respond to a sysfs probe() callback that indicates an HBA has been
128 detected. After confirming that the new device is one that the LLD wants
129 to control, the LLD will initialize the HBA and then register a new host
130 with the SCSI mid level.
132 During LLD initialization the driver should register itself with the
133 appropriate IO bus on which it expects to find HBA(s) (e.g. the PCI bus).
134 This can probably be done via sysfs. Any driver parameters (especially
135 those that are writable after the driver is loaded) could also be
136 registered with sysfs at this point. The SCSI mid level first becomes
137 aware of an LLD when that LLD registers its first HBA.
139 At some later time, the LLD becomes aware of an HBA and what follows
140 is a typical sequence of calls between the LLD and the mid level.
141 This example shows the mid level scanning the newly introduced HBA for 3
142 scsi devices of which only the first 2 respond:
144 HBA PROBE: assume 2 SCSI devices found in scan
146 ===-------------------=========--------------------===------
147 scsi_host_alloc() -->
148 scsi_add_host() ---->
149 scsi_scan_host() -------+
152 slave_configure() --> scsi_change_queue_depth()
159 ------------------------------------------------------------
161 If the LLD wants to adjust the default queue settings, it can invoke
162 scsi_change_queue_depth() in its slave_configure() routine.
164 *** For scsi devices that the mid level tries to scan but do not
165 respond, a slave_alloc(), slave_destroy() pair is called.
167 When an HBA is being removed it could be as part of an orderly shutdown
168 associated with the LLD module being unloaded (e.g. with the "rmmod"
169 command) or in response to a "hot unplug" indicated by sysfs()'s
170 remove() callback being invoked. In either case, the sequence is the
173 HBA REMOVE: assume 2 SCSI devices attached
175 ===----------------------=========-----------------===------
176 scsi_remove_host() ---------+
181 ------------------------------------------------------------
183 It may be useful for a LLD to keep track of struct Scsi_Host instances
184 (a pointer is returned by scsi_host_alloc()). Such instances are "owned"
185 by the mid-level. struct Scsi_Host instances are freed from
186 scsi_host_put() when the reference count hits zero.
188 Hot unplugging an HBA that controls a disk which is processing SCSI
189 commands on a mounted file system is an interesting situation. Reference
190 counting logic is being introduced into the mid level to cope with many
191 of the issues involved. See the section on reference counting below.
194 The hotplug concept may be extended to SCSI devices. Currently, when an
195 HBA is added, the scsi_scan_host() function causes a scan for SCSI devices
196 attached to the HBA's SCSI transport. On newer SCSI transports the HBA
197 may become aware of a new SCSI device _after_ the scan has completed.
198 An LLD can use this sequence to make the mid level aware of a SCSI device:
202 ===-------------------=========--------------------===------
203 scsi_add_device() ------+
206 slave_configure() [--> scsi_change_queue_depth()]
207 ------------------------------------------------------------
209 In a similar fashion, an LLD may become aware that a SCSI device has been
210 removed (unplugged) or the connection to it has been interrupted. Some
211 existing SCSI transports (e.g. SPI) may not become aware that a SCSI
212 device has been removed until a subsequent SCSI command fails which will
213 probably cause that device to be set offline by the mid level. An LLD that
214 detects the removal of a SCSI device can instigate its removal from
215 upper layers with this sequence:
217 SCSI DEVICE hot unplug
219 ===----------------------=========-----------------===------
220 scsi_remove_device() -------+
223 ------------------------------------------------------------
225 It may be useful for an LLD to keep track of struct scsi_device instances
226 (a pointer is passed as the parameter to slave_alloc() and
227 slave_configure() callbacks). Such instances are "owned" by the mid-level.
228 struct scsi_device instances are freed after slave_destroy().
231 Passive initialization model
232 ============================
233 These older LLDs include a file called "scsi_module.c" [yes the ".c" is a
234 little surprising] in their source code. For that file to work an
235 instance of struct scsi_host_template with the name "driver_template"
236 needs to be defined. Here is a typical code sequence used in this model:
237 static struct scsi_host_template driver_template = {
240 #include "scsi_module.c"
242 The scsi_module.c file contains two functions:
243 - init_this_scsi_driver() which is executed when the LLD is
244 initialized (i.e. boot time or module load time)
245 - exit_this_scsi_driver() which is executed when the LLD is shut
246 down (i.e. module unload time)
247 Note: since these functions are tagged with __init and __exit qualifiers
248 an LLD should not call them explicitly (since the kernel does that).
250 Here is an example of an initialization sequence when two hosts are
251 detected (so detect() returns 2) and the SCSI bus scan on each host
252 finds 1 SCSI device (and a second device does not respond).
255 ===----------------------=========-----------------===------
256 init_this_scsi_driver() ----+
258 detect() -----------------+
264 slave_configure() --> scsi_change_queue_depth()
272 ------------------------------------------------------------
274 The mid level invokes scsi_change_queue_depth() with "cmd_per_lun" for that
275 host as the queue length. These settings can be overridden by a
276 slave_configure() supplied by the LLD.
278 *** For scsi devices that the mid level tries to scan but do not
279 respond, a slave_alloc(), slave_destroy() pair is called.
281 Here is an LLD shutdown sequence:
284 ===----------------------=========-----------------===------
285 exit_this_scsi_driver() ----+
288 release() --> scsi_unregister()
291 release() --> scsi_unregister()
292 ------------------------------------------------------------
294 An LLD need not define slave_destroy() (i.e. it is optional).
296 The shortcoming of the "passive initialization model" is that host
297 registration and de-registration are (typically) tied to LLD initialization
298 and shutdown. Once the LLD is initialized then a new host that appears
299 (e.g. via hotplugging) cannot easily be added without a redundant
300 driver shutdown and re-initialization. It may be possible to write an LLD
301 that uses both initialization models.
306 The Scsi_Host structure has had reference counting infrastructure added.
307 This effectively spreads the ownership of struct Scsi_Host instances
308 across the various SCSI layers which use them. Previously such instances
309 were exclusively owned by the mid level. LLDs would not usually need to
310 directly manipulate these reference counts but there may be some cases
313 There are 3 reference counting functions of interest associated with
315 - scsi_host_alloc(): returns a pointer to new instance of struct
316 Scsi_Host which has its reference count ^^ set to 1
317 - scsi_host_get(): adds 1 to the reference count of the given instance
318 - scsi_host_put(): decrements 1 from the reference count of the given
319 instance. If the reference count reaches 0 then the given instance
322 The Scsi_device structure has had reference counting infrastructure added.
323 This effectively spreads the ownership of struct Scsi_device instances
324 across the various SCSI layers which use them. Previously such instances
325 were exclusively owned by the mid level. See the access functions declared
326 towards the end of include/scsi/scsi_device.h . If an LLD wants to keep
327 a copy of a pointer to a Scsi_device instance it should use scsi_device_get()
328 to bump its reference count. When it is finished with the pointer it can
329 use scsi_device_put() to decrement its reference count (and potentially
332 ^^ struct Scsi_Host actually has 2 reference counts which are manipulated
333 in parallel by these functions.
338 First, Linus Torvalds's thoughts on C coding style can be found in the
339 Documentation/process/coding-style.rst file.
341 Next, there is a movement to "outlaw" typedefs introducing synonyms for
342 struct tags. Both can be still found in the SCSI subsystem, but
343 the typedefs have been moved to a single file, scsi_typedefs.h to
344 make their future removal easier, for example:
345 "typedef struct scsi_cmnd Scsi_Cmnd;"
347 Also, most C99 enhancements are encouraged to the extent they are supported
348 by the relevant gcc compilers. So C99 style structure and array
349 initializers are encouraged where appropriate. Don't go too far,
350 VLAs are not properly supported yet. An exception to this is the use of
351 "//" style comments; /*...*/ comments are still preferred in Linux.
353 Well written, tested and documented code, need not be re-formatted to
354 comply with the above conventions. For example, the aic7xxx driver
355 comes to Linux from FreeBSD and Adaptec's own labs. No doubt FreeBSD
356 and Adaptec have their own coding conventions.
359 Mid level supplied functions
360 ============================
361 These functions are supplied by the SCSI mid level for use by LLDs.
362 The names (i.e. entry points) of these functions are exported
363 so an LLD that is a module can access them. The kernel will
364 arrange for the SCSI mid level to be loaded and initialized before any LLD
365 is initialized. The functions below are listed alphabetically and their
366 names all start with "scsi_".
369 scsi_add_device - creates new scsi device (lu) instance
370 scsi_add_host - perform sysfs registration and set up transport class
371 scsi_change_queue_depth - change the queue depth on a SCSI device
372 scsi_bios_ptable - return copy of block device's partition table
373 scsi_block_requests - prevent further commands being queued to given host
374 scsi_host_alloc - return a new scsi_host instance whose refcount==1
375 scsi_host_get - increments Scsi_Host instance's refcount
376 scsi_host_put - decrements Scsi_Host instance's refcount (free if 0)
377 scsi_partsize - parse partition table into cylinders, heads + sectors
378 scsi_register - create and register a scsi host adapter instance.
379 scsi_remove_device - detach and remove a SCSI device
380 scsi_remove_host - detach and remove all SCSI devices owned by host
381 scsi_report_bus_reset - report scsi _bus_ reset observed
382 scsi_scan_host - scan SCSI bus
383 scsi_track_queue_full - track successive QUEUE_FULL events
384 scsi_unblock_requests - allow further commands to be queued to given host
385 scsi_unregister - [calls scsi_host_put()]
391 * scsi_add_device - creates new scsi device (lu) instance
392 * @shost: pointer to scsi host instance
393 * @channel: channel number (rarely other than 0)
394 * @id: target id number
395 * @lun: logical unit number
397 * Returns pointer to new struct scsi_device instance or
398 * ERR_PTR(-ENODEV) (or some other bent pointer) if something is
399 * wrong (e.g. no lu responds at given address)
403 * Notes: This call is usually performed internally during a scsi
404 * bus scan when an HBA is added (i.e. scsi_scan_host()). So it
405 * should only be called if the HBA becomes aware of a new scsi
406 * device (lu) after scsi_scan_host() has completed. If successful
407 * this call can lead to slave_alloc() and slave_configure() callbacks
410 * Defined in: drivers/scsi/scsi_scan.c
412 struct scsi_device * scsi_add_device(struct Scsi_Host *shost,
413 unsigned int channel,
414 unsigned int id, unsigned int lun)
418 * scsi_add_host - perform sysfs registration and set up transport class
419 * @shost: pointer to scsi host instance
420 * @dev: pointer to struct device of type scsi class
422 * Returns 0 on success, negative errno of failure (e.g. -ENOMEM)
426 * Notes: Only required in "hotplug initialization model" after a
427 * successful call to scsi_host_alloc(). This function does not
428 * scan the bus; this can be done by calling scsi_scan_host() or
429 * in some other transport-specific way. The LLD must set up
430 * the transport template before calling this function and may only
431 * access the transport class data after this function has been called.
433 * Defined in: drivers/scsi/hosts.c
435 int scsi_add_host(struct Scsi_Host *shost, struct device * dev)
439 * scsi_change_queue_depth - allow LLD to change queue depth on a SCSI device
440 * @sdev: pointer to SCSI device to change queue depth on
441 * @tags Number of tags allowed if tagged queuing enabled,
442 * or number of commands the LLD can queue up
443 * in non-tagged mode (as per cmd_per_lun).
449 * Notes: Can be invoked any time on a SCSI device controlled by this
450 * LLD. [Specifically during and after slave_configure() and prior to
451 * slave_destroy().] Can safely be invoked from interrupt code.
453 * Defined in: drivers/scsi/scsi.c [see source code for more notes]
456 int scsi_change_queue_depth(struct scsi_device *sdev, int tags)
460 * scsi_bios_ptable - return copy of block device's partition table
461 * @dev: pointer to block device
463 * Returns pointer to partition table, or NULL for failure
467 * Notes: Caller owns memory returned (free with kfree() )
469 * Defined in: drivers/scsi/scsicam.c
471 unsigned char *scsi_bios_ptable(struct block_device *dev)
475 * scsi_block_requests - prevent further commands being queued to given host
477 * @shost: pointer to host to block commands on
483 * Notes: There is no timer nor any other means by which the requests
484 * get unblocked other than the LLD calling scsi_unblock_requests().
486 * Defined in: drivers/scsi/scsi_lib.c
488 void scsi_block_requests(struct Scsi_Host * shost)
492 * scsi_host_alloc - create a scsi host adapter instance and perform basic
494 * @sht: pointer to scsi host template
495 * @privsize: extra bytes to allocate in hostdata array (which is the
496 * last member of the returned Scsi_Host instance)
498 * Returns pointer to new Scsi_Host instance or NULL on failure
502 * Notes: When this call returns to the LLD, the SCSI bus scan on
503 * this host has _not_ yet been done.
504 * The hostdata array (by default zero length) is a per host scratch
505 * area for the LLD's exclusive use.
506 * Both associated refcounting objects have their refcount set to 1.
507 * Full registration (in sysfs) and a bus scan are performed later when
508 * scsi_add_host() and scsi_scan_host() are called.
510 * Defined in: drivers/scsi/hosts.c .
512 struct Scsi_Host * scsi_host_alloc(struct scsi_host_template * sht,
517 * scsi_host_get - increment Scsi_Host instance refcount
518 * @shost: pointer to struct Scsi_Host instance
522 * Might block: currently may block but may be changed to not block
524 * Notes: Actually increments the counts in two sub-objects
526 * Defined in: drivers/scsi/hosts.c
528 void scsi_host_get(struct Scsi_Host *shost)
532 * scsi_host_put - decrement Scsi_Host instance refcount, free if 0
533 * @shost: pointer to struct Scsi_Host instance
537 * Might block: currently may block but may be changed to not block
539 * Notes: Actually decrements the counts in two sub-objects. If the
540 * latter refcount reaches 0, the Scsi_Host instance is freed.
541 * The LLD need not worry exactly when the Scsi_Host instance is
542 * freed, it just shouldn't access the instance after it has balanced
543 * out its refcount usage.
545 * Defined in: drivers/scsi/hosts.c
547 void scsi_host_put(struct Scsi_Host *shost)
551 * scsi_partsize - parse partition table into cylinders, heads + sectors
552 * @buf: pointer to partition table
553 * @capacity: size of (total) disk in 512 byte sectors
554 * @cyls: outputs number of cylinders calculated via this pointer
555 * @hds: outputs number of heads calculated via this pointer
556 * @secs: outputs number of sectors calculated via this pointer
558 * Returns 0 on success, -1 on failure
562 * Notes: Caller owns memory returned (free with kfree() )
564 * Defined in: drivers/scsi/scsicam.c
566 int scsi_partsize(unsigned char *buf, unsigned long capacity,
567 unsigned int *cyls, unsigned int *hds, unsigned int *secs)
571 * scsi_register - create and register a scsi host adapter instance.
572 * @sht: pointer to scsi host template
573 * @privsize: extra bytes to allocate in hostdata array (which is the
574 * last member of the returned Scsi_Host instance)
576 * Returns pointer to new Scsi_Host instance or NULL on failure
580 * Notes: When this call returns to the LLD, the SCSI bus scan on
581 * this host has _not_ yet been done.
582 * The hostdata array (by default zero length) is a per host scratch
585 * Defined in: drivers/scsi/hosts.c .
587 struct Scsi_Host * scsi_register(struct scsi_host_template * sht,
592 * scsi_remove_device - detach and remove a SCSI device
593 * @sdev: a pointer to a scsi device instance
595 * Returns value: 0 on success, -EINVAL if device not attached
599 * Notes: If an LLD becomes aware that a scsi device (lu) has
600 * been removed but its host is still present then it can request
601 * the removal of that scsi device. If successful this call will
602 * lead to the slave_destroy() callback being invoked. sdev is an
603 * invalid pointer after this call.
605 * Defined in: drivers/scsi/scsi_sysfs.c .
607 int scsi_remove_device(struct scsi_device *sdev)
611 * scsi_remove_host - detach and remove all SCSI devices owned by host
612 * @shost: a pointer to a scsi host instance
614 * Returns value: 0 on success, 1 on failure (e.g. LLD busy ??)
618 * Notes: Should only be invoked if the "hotplug initialization
619 * model" is being used. It should be called _prior_ to
622 * Defined in: drivers/scsi/hosts.c .
624 int scsi_remove_host(struct Scsi_Host *shost)
628 * scsi_report_bus_reset - report scsi _bus_ reset observed
629 * @shost: a pointer to a scsi host involved
630 * @channel: channel (within) host on which scsi bus reset occurred
636 * Notes: This only needs to be called if the reset is one which
637 * originates from an unknown location. Resets originated by the
638 * mid level itself don't need to call this, but there should be
639 * no harm. The main purpose of this is to make sure that a
640 * CHECK_CONDITION is properly treated.
642 * Defined in: drivers/scsi/scsi_error.c .
644 void scsi_report_bus_reset(struct Scsi_Host * shost, int channel)
648 * scsi_scan_host - scan SCSI bus
649 * @shost: a pointer to a scsi host instance
653 * Notes: Should be called after scsi_add_host()
655 * Defined in: drivers/scsi/scsi_scan.c
657 void scsi_scan_host(struct Scsi_Host *shost)
661 * scsi_track_queue_full - track successive QUEUE_FULL events on given
662 * device to determine if and when there is a need
663 * to adjust the queue depth on the device.
664 * @sdev: pointer to SCSI device instance
665 * @depth: Current number of outstanding SCSI commands on this device,
666 * not counting the one returned as QUEUE_FULL.
668 * Returns 0 - no change needed
669 * >0 - adjust queue depth to this new depth
670 * -1 - drop back to untagged operation using host->cmd_per_lun
671 * as the untagged command depth
675 * Notes: LLDs may call this at any time and we will do "The Right
676 * Thing"; interrupt context safe.
678 * Defined in: drivers/scsi/scsi.c .
680 int scsi_track_queue_full(struct scsi_device *sdev, int depth)
684 * scsi_unblock_requests - allow further commands to be queued to given host
686 * @shost: pointer to host to unblock commands on
692 * Defined in: drivers/scsi/scsi_lib.c .
694 void scsi_unblock_requests(struct Scsi_Host * shost)
698 * scsi_unregister - unregister and free memory used by host instance
699 * @shp: pointer to scsi host instance to unregister.
705 * Notes: Should not be invoked if the "hotplug initialization
706 * model" is being used. Called internally by exit_this_scsi_driver()
707 * in the "passive initialization model". Hence a LLD has no need to
708 * call this function directly.
710 * Defined in: drivers/scsi/hosts.c .
712 void scsi_unregister(struct Scsi_Host * shp)
719 Interface functions are supplied (defined) by LLDs and their function
720 pointers are placed in an instance of struct scsi_host_template which
721 is passed to scsi_host_alloc() [or scsi_register() / init_this_scsi_driver()].
722 Some are mandatory. Interface functions should be declared static. The
723 accepted convention is that driver "xyz" will declare its slave_configure()
725 static int xyz_slave_configure(struct scsi_device * sdev);
726 and so forth for all interface functions listed below.
728 A pointer to this function should be placed in the 'slave_configure' member
729 of a "struct scsi_host_template" instance. A pointer to such an instance
730 should be passed to the mid level's scsi_host_alloc() [or scsi_register() /
731 init_this_scsi_driver()].
733 The interface functions are also described in the include/scsi/scsi_host.h
734 file immediately above their definition point in "struct scsi_host_template".
735 In some cases more detail is given in scsi_host.h than below.
737 The interface functions are listed below in alphabetical order.
740 bios_param - fetch head, sector, cylinder info for a disk
741 detect - detects HBAs this driver wants to control
742 eh_timed_out - notify the host that a command timer expired
743 eh_abort_handler - abort given command
744 eh_bus_reset_handler - issue SCSI bus reset
745 eh_device_reset_handler - issue SCSI device reset
746 eh_host_reset_handler - reset host (host bus adapter)
747 info - supply information about given host
748 ioctl - driver can respond to ioctls
749 proc_info - supports /proc/scsi/{driver_name}/{host_no}
750 queuecommand - queue scsi command, invoke 'done' on completion
751 release - release all resources associated with given host
752 slave_alloc - prior to any commands being sent to a new device
753 slave_configure - driver fine tuning for given device after attach
754 slave_destroy - given device is about to be shut down
760 * bios_param - fetch head, sector, cylinder info for a disk
761 * @sdev: pointer to scsi device context (defined in
762 * include/scsi/scsi_device.h)
763 * @bdev: pointer to block device context (defined in fs.h)
764 * @capacity: device size (in 512 byte sectors)
765 * @params: three element array to place output:
766 * params[0] number of heads (max 255)
767 * params[1] number of sectors (max 63)
768 * params[2] number of cylinders
770 * Return value is ignored
774 * Calling context: process (sd)
776 * Notes: an arbitrary geometry (based on READ CAPACITY) is used
777 * if this function is not provided. The params array is
778 * pre-initialized with made up values just in case this function
779 * doesn't output anything.
781 * Optionally defined in: LLD
783 int bios_param(struct scsi_device * sdev, struct block_device *bdev,
784 sector_t capacity, int params[3])
788 * detect - detects HBAs this driver wants to control
789 * @shtp: host template for this driver.
791 * Returns number of hosts this driver wants to control. 0 means no
792 * suitable hosts found.
796 * Calling context: process [invoked from init_this_scsi_driver()]
798 * Notes: First function called from the SCSI mid level on this
799 * driver. Upper level drivers (e.g. sd) may not (yet) be present.
800 * For each host found, this method should call scsi_register()
803 * Defined in: LLD (required if "passive initialization mode" is used,
804 * not invoked in "hotplug initialization mode")
806 int detect(struct scsi_host_template * shtp)
810 * eh_timed_out - The timer for the command has just fired
811 * @scp: identifies command timing out
815 * EH_HANDLED: I fixed the error, please complete the command
816 * EH_RESET_TIMER: I need more time, reset the timer and
817 * begin counting again
818 * EH_NOT_HANDLED Begin normal error recovery
823 * Calling context: interrupt
825 * Notes: This is to give the LLD an opportunity to do local recovery.
826 * This recovery is limited to determining if the outstanding command
827 * will ever complete. You may not abort and restart the command from
830 * Optionally defined in: LLD
832 int eh_timed_out(struct scsi_cmnd * scp)
836 * eh_abort_handler - abort command associated with scp
837 * @scp: identifies command to be aborted
839 * Returns SUCCESS if command aborted else FAILED
843 * Calling context: kernel thread
845 * Notes: If 'no_async_abort' is defined this callback
846 * will be invoked from scsi_eh thread. No other commands
847 * will then be queued on current host during eh.
848 * Otherwise it will be called whenever scsi_times_out()
849 * is called due to a command timeout.
851 * Optionally defined in: LLD
853 int eh_abort_handler(struct scsi_cmnd * scp)
857 * eh_bus_reset_handler - issue SCSI bus reset
858 * @scp: SCSI bus that contains this device should be reset
860 * Returns SUCCESS if command aborted else FAILED
864 * Calling context: kernel thread
866 * Notes: Invoked from scsi_eh thread. No other commands will be
867 * queued on current host during eh.
869 * Optionally defined in: LLD
871 int eh_bus_reset_handler(struct scsi_cmnd * scp)
875 * eh_device_reset_handler - issue SCSI device reset
876 * @scp: identifies SCSI device to be reset
878 * Returns SUCCESS if command aborted else FAILED
882 * Calling context: kernel thread
884 * Notes: Invoked from scsi_eh thread. No other commands will be
885 * queued on current host during eh.
887 * Optionally defined in: LLD
889 int eh_device_reset_handler(struct scsi_cmnd * scp)
893 * eh_host_reset_handler - reset host (host bus adapter)
894 * @scp: SCSI host that contains this device should be reset
896 * Returns SUCCESS if command aborted else FAILED
900 * Calling context: kernel thread
902 * Notes: Invoked from scsi_eh thread. No other commands will be
903 * queued on current host during eh.
904 * With the default eh_strategy in place, if none of the _abort_,
905 * _device_reset_, _bus_reset_ or this eh handler function are
906 * defined (or they all return FAILED) then the device in question
907 * will be set offline whenever eh is invoked.
909 * Optionally defined in: LLD
911 int eh_host_reset_handler(struct scsi_cmnd * scp)
915 * info - supply information about given host: driver name plus data
916 * to distinguish given host
917 * @shp: host to supply information about
919 * Return ASCII null terminated string. [This driver is assumed to
920 * manage the memory pointed to and maintain it, typically for the
921 * lifetime of this host.]
925 * Calling context: process
927 * Notes: Often supplies PCI or ISA information such as IO addresses
928 * and interrupt numbers. If not supplied struct Scsi_Host::name used
929 * instead. It is assumed the returned information fits on one line
930 * (i.e. does not included embedded newlines).
931 * The SCSI_IOCTL_PROBE_HOST ioctl yields the string returned by this
932 * function (or struct Scsi_Host::name if this function is not
934 * In a similar manner, init_this_scsi_driver() outputs to the console
935 * each host's "info" (or name) for the driver it is registering.
936 * Also if proc_info() is not supplied, the output of this function
939 * Optionally defined in: LLD
941 const char * info(struct Scsi_Host * shp)
945 * ioctl - driver can respond to ioctls
946 * @sdp: device that ioctl was issued for
948 * @arg: pointer to read or write data from. Since it points to
949 * user space, should use appropriate kernel functions
950 * (e.g. copy_from_user() ). In the Unix style this argument
951 * can also be viewed as an unsigned long.
953 * Returns negative "errno" value when there is a problem. 0 or a
954 * positive value indicates success and is returned to the user space.
958 * Calling context: process
960 * Notes: The SCSI subsystem uses a "trickle down" ioctl model.
961 * The user issues an ioctl() against an upper level driver
962 * (e.g. /dev/sdc) and if the upper level driver doesn't recognize
963 * the 'cmd' then it is passed to the SCSI mid level. If the SCSI
964 * mid level does not recognize it, then the LLD that controls
965 * the device receives the ioctl. According to recent Unix standards
966 * unsupported ioctl() 'cmd' numbers should return -ENOTTY.
968 * Optionally defined in: LLD
970 int ioctl(struct scsi_device *sdp, int cmd, void *arg)
974 * proc_info - supports /proc/scsi/{driver_name}/{host_no}
975 * @buffer: anchor point to output to (0==writeto1_read0) or fetch from
976 * (1==writeto1_read0).
977 * @start: where "interesting" data is written to. Ignored when
979 * @offset: offset within buffer 0==writeto1_read0 is actually
980 * interested in. Ignored when 1==writeto1_read0 .
981 * @length: maximum (or actual) extent of buffer
982 * @host_no: host number of interest (struct Scsi_Host::host_no)
983 * @writeto1_read0: 1 -> data coming from user space towards driver
984 * (e.g. "echo some_string > /proc/scsi/xyz/2")
985 * 0 -> user what data from this driver
986 * (e.g. "cat /proc/scsi/xyz/2")
988 * Returns length when 1==writeto1_read0. Otherwise number of chars
989 * output to buffer past offset.
993 * Calling context: process
995 * Notes: Driven from scsi_proc.c which interfaces to proc_fs. proc_fs
996 * support can now be configured out of the scsi subsystem.
998 * Optionally defined in: LLD
1000 int proc_info(char * buffer, char ** start, off_t offset,
1001 int length, int host_no, int writeto1_read0)
1005 * queuecommand - queue scsi command, invoke scp->scsi_done on completion
1006 * @shost: pointer to the scsi host object
1007 * @scp: pointer to scsi command object
1009 * Returns 0 on success.
1011 * If there's a failure, return either:
1013 * SCSI_MLQUEUE_DEVICE_BUSY if the device queue is full, or
1014 * SCSI_MLQUEUE_HOST_BUSY if the entire host queue is full
1016 * On both of these returns, the mid-layer will requeue the I/O
1018 * - if the return is SCSI_MLQUEUE_DEVICE_BUSY, only that particular
1019 * device will be paused, and it will be unpaused when a command to
1020 * the device returns (or after a brief delay if there are no more
1021 * outstanding commands to it). Commands to other devices continue
1022 * to be processed normally.
1024 * - if the return is SCSI_MLQUEUE_HOST_BUSY, all I/O to the host
1025 * is paused and will be unpaused when any command returns from
1026 * the host (or after a brief delay if there are no outstanding
1027 * commands to the host).
1029 * For compatibility with earlier versions of queuecommand, any
1030 * other return value is treated the same as
1031 * SCSI_MLQUEUE_HOST_BUSY.
1033 * Other types of errors that are detected immediately may be
1034 * flagged by setting scp->result to an appropriate value,
1035 * invoking the scp->scsi_done callback, and then returning 0
1036 * from this function. If the command is not performed
1037 * immediately (and the LLD is starting (or will start) the given
1038 * command) then this function should place 0 in scp->result and
1041 * Command ownership. If the driver returns zero, it owns the
1042 * command and must take responsibility for ensuring the
1043 * scp->scsi_done callback is executed. Note: the driver may
1044 * call scp->scsi_done before returning zero, but after it has
1045 * called scp->scsi_done, it may not return any value other than
1046 * zero. If the driver makes a non-zero return, it must not
1047 * execute the command's scsi_done callback at any time.
1049 * Locks: up to and including 2.6.36, struct Scsi_Host::host_lock
1050 * held on entry (with "irqsave") and is expected to be
1051 * held on return. From 2.6.37 onwards, queuecommand is
1052 * called without any locks held.
1054 * Calling context: in interrupt (soft irq) or process context
1056 * Notes: This function should be relatively fast. Normally it
1057 * will not wait for IO to complete. Hence the scp->scsi_done
1058 * callback is invoked (often directly from an interrupt service
1059 * routine) some time after this function has returned. In some
1060 * cases (e.g. pseudo adapter drivers that manufacture the
1061 * response to a SCSI INQUIRY) the scp->scsi_done callback may be
1062 * invoked before this function returns. If the scp->scsi_done
1063 * callback is not invoked within a certain period the SCSI mid
1064 * level will commence error processing. If a status of CHECK
1065 * CONDITION is placed in "result" when the scp->scsi_done
1066 * callback is invoked, then the LLD driver should perform
1067 * autosense and fill in the struct scsi_cmnd::sense_buffer
1068 * array. The scsi_cmnd::sense_buffer array is zeroed prior to
1069 * the mid level queuing a command to an LLD.
1073 int queuecommand(struct Scsi_Host *shost, struct scsi_cmnd * scp)
1077 * release - release all resources associated with given host
1078 * @shp: host to be released.
1080 * Return value ignored (could soon be a function returning void).
1084 * Calling context: process
1086 * Notes: Invoked from scsi_module.c's exit_this_scsi_driver().
1087 * LLD's implementation of this function should call
1088 * scsi_unregister(shp) prior to returning.
1089 * Only needed for old-style host templates.
1091 * Defined in: LLD (required in "passive initialization model",
1092 * should not be defined in hotplug model)
1094 int release(struct Scsi_Host * shp)
1098 * slave_alloc - prior to any commands being sent to a new device
1099 * (i.e. just prior to scan) this call is made
1100 * @sdp: pointer to new device (about to be scanned)
1102 * Returns 0 if ok. Any other return is assumed to be an error and
1103 * the device is ignored.
1107 * Calling context: process
1109 * Notes: Allows the driver to allocate any resources for a device
1110 * prior to its initial scan. The corresponding scsi device may not
1111 * exist but the mid level is just about to scan for it (i.e. send
1112 * and INQUIRY command plus ...). If a device is found then
1113 * slave_configure() will be called while if a device is not found
1114 * slave_destroy() is called.
1115 * For more details see the include/scsi/scsi_host.h file.
1117 * Optionally defined in: LLD
1119 int slave_alloc(struct scsi_device *sdp)
1123 * slave_configure - driver fine tuning for given device just after it
1124 * has been first scanned (i.e. it responded to an
1126 * @sdp: device that has just been attached
1128 * Returns 0 if ok. Any other return is assumed to be an error and
1129 * the device is taken offline. [offline devices will _not_ have
1130 * slave_destroy() called on them so clean up resources.]
1134 * Calling context: process
1136 * Notes: Allows the driver to inspect the response to the initial
1137 * INQUIRY done by the scanning code and take appropriate action.
1138 * For more details see the include/scsi/scsi_host.h file.
1140 * Optionally defined in: LLD
1142 int slave_configure(struct scsi_device *sdp)
1146 * slave_destroy - given device is about to be shut down. All
1147 * activity has ceased on this device.
1148 * @sdp: device that is about to be shut down
1154 * Calling context: process
1156 * Notes: Mid level structures for given device are still in place
1157 * but are about to be torn down. Any per device resources allocated
1158 * by this driver for given device should be freed now. No further
1159 * commands will be sent for this sdp instance. [However the device
1160 * could be re-attached in the future in which case a new instance
1161 * of struct scsi_device would be supplied by future slave_alloc()
1162 * and slave_configure() calls.]
1164 * Optionally defined in: LLD
1166 void slave_destroy(struct scsi_device *sdp)
1172 struct scsi_host_template
1173 -------------------------
1174 There is one "struct scsi_host_template" instance per LLD ***. It is
1175 typically initialized as a file scope static in a driver's header file. That
1176 way members that are not explicitly initialized will be set to 0 or NULL.
1178 name - name of driver (may contain spaces, please limit to
1179 less than 80 characters)
1180 proc_name - name used in "/proc/scsi/<proc_name>/<host_no>" and
1181 by sysfs in one of its "drivers" directories. Hence
1182 "proc_name" should only contain characters acceptable
1183 to a Unix file name.
1184 (*queuecommand)() - primary callback that the mid level uses to inject
1185 SCSI commands into an LLD.
1186 The structure is defined and commented in include/scsi/scsi_host.h
1188 *** In extreme situations a single driver may have several instances
1189 if it controls several different classes of hardware (e.g. an LLD
1190 that handles both ISA and PCI cards and has a separate instance of
1191 struct scsi_host_template for each class).
1195 There is one struct Scsi_Host instance per host (HBA) that an LLD
1196 controls. The struct Scsi_Host structure has many members in common
1197 with "struct scsi_host_template". When a new struct Scsi_Host instance
1198 is created (in scsi_host_alloc() in hosts.c) those common members are
1199 initialized from the driver's struct scsi_host_template instance. Members
1201 host_no - system wide unique number that is used for identifying
1202 this host. Issued in ascending order from 0.
1203 can_queue - must be greater than 0; do not send more than can_queue
1204 commands to the adapter.
1205 this_id - scsi id of host (scsi initiator) or -1 if not known
1206 sg_tablesize - maximum scatter gather elements allowed by host.
1207 0 implies scatter gather not supported by host
1208 max_sectors - maximum number of sectors (usually 512 bytes) allowed
1209 in a single SCSI command. The default value of 0 leads
1210 to a setting of SCSI_DEFAULT_MAX_SECTORS (defined in
1211 scsi_host.h) which is currently set to 1024. So for a
1212 disk the maximum transfer size is 512 KB when max_sectors
1213 is not defined. Note that this size may not be sufficient
1214 for disk firmware uploads.
1215 cmd_per_lun - maximum number of commands that can be queued on devices
1216 controlled by the host. Overridden by LLD calls to
1217 scsi_change_queue_depth().
1218 unchecked_isa_dma - 1=>only use bottom 16 MB of ram (ISA DMA addressing
1219 restriction), 0=>can use full 32 bit (or better) DMA
1221 use_clustering - 1=>SCSI commands in mid level's queue can be merged,
1222 0=>disallow SCSI command merging
1223 no_async_abort - 1=>Asynchronous aborts are not supported
1224 0=>Timed-out commands will be aborted asynchronously
1225 hostt - pointer to driver's struct scsi_host_template from which
1226 this struct Scsi_Host instance was spawned
1227 hostt->proc_name - name of LLD. This is the driver name that sysfs uses
1228 transportt - pointer to driver's struct scsi_transport_template instance
1229 (if any). FC and SPI transports currently supported.
1230 sh_list - a double linked list of pointers to all struct Scsi_Host
1231 instances (currently ordered by ascending host_no)
1232 my_devices - a double linked list of pointers to struct scsi_device
1233 instances that belong to this host.
1234 hostdata[0] - area reserved for LLD at end of struct Scsi_Host. Size
1235 is set by the second argument (named 'xtr_bytes') to
1236 scsi_host_alloc() or scsi_register().
1237 vendor_id - a unique value that identifies the vendor supplying
1238 the LLD for the Scsi_Host. Used most often in validating
1239 vendor-specific message requests. Value consists of an
1240 identifier type and a vendor-specific value.
1241 See scsi_netlink.h for a description of valid formats.
1243 The scsi_host structure is defined in include/scsi/scsi_host.h
1247 Generally, there is one instance of this structure for each SCSI logical unit
1248 on a host. Scsi devices connected to a host are uniquely identified by a
1249 channel number, target id and logical unit number (lun).
1250 The structure is defined in include/scsi/scsi_device.h
1254 Instances of this structure convey SCSI commands to the LLD and responses
1255 back to the mid level. The SCSI mid level will ensure that no more SCSI
1256 commands become queued against the LLD than are indicated by
1257 scsi_change_queue_depth() (or struct Scsi_Host::cmd_per_lun). There will
1258 be at least one instance of struct scsi_cmnd available for each SCSI device.
1259 Members of interest:
1260 cmnd - array containing SCSI command
1261 cmnd_len - length (in bytes) of SCSI command
1262 sc_data_direction - direction of data transfer in data phase. See
1263 "enum dma_data_direction" in include/linux/dma-mapping.h
1264 request_bufflen - number of data bytes to transfer (0 if no data phase)
1265 use_sg - ==0 -> no scatter gather list, hence transfer data
1266 to/from request_buffer
1267 - >0 -> scatter gather list (actually an array) in
1268 request_buffer with use_sg elements
1269 request_buffer - either contains data buffer or scatter gather list
1270 depending on the setting of use_sg. Scatter gather
1271 elements are defined by 'struct scatterlist' found
1272 in include/linux/scatterlist.h .
1273 done - function pointer that should be invoked by LLD when the
1274 SCSI command is completed (successfully or otherwise).
1275 Should only be called by an LLD if the LLD has accepted
1276 the command (i.e. queuecommand() returned or will return
1277 0). The LLD may invoke 'done' prior to queuecommand()
1279 result - should be set by LLD prior to calling 'done'. A value
1280 of 0 implies a successfully completed command (and all
1281 data (if any) has been transferred to or from the SCSI
1282 target device). 'result' is a 32 bit unsigned integer that
1283 can be viewed as 4 related bytes. The SCSI status value is
1284 in the LSB. See include/scsi/scsi.h status_byte(),
1285 msg_byte(), host_byte() and driver_byte() macros and
1287 sense_buffer - an array (maximum size: SCSI_SENSE_BUFFERSIZE bytes) that
1288 should be written when the SCSI status (LSB of 'result')
1289 is set to CHECK_CONDITION (2). When CHECK_CONDITION is
1290 set, if the top nibble of sense_buffer[0] has the value 7
1291 then the mid level will assume the sense_buffer array
1292 contains a valid SCSI sense buffer; otherwise the mid
1293 level will issue a REQUEST_SENSE SCSI command to
1294 retrieve the sense buffer. The latter strategy is error
1295 prone in the presence of command queuing so the LLD should
1296 always "auto-sense".
1297 device - pointer to scsi_device object that this command is
1299 resid - an LLD should set this signed integer to the requested
1300 transfer length (i.e. 'request_bufflen') less the number
1301 of bytes that are actually transferred. 'resid' is
1302 preset to 0 so an LLD can ignore it if it cannot detect
1303 underruns (overruns should be rare). If possible an LLD
1304 should set 'resid' prior to invoking 'done'. The most
1305 interesting case is data transfers from a SCSI target
1306 device (e.g. READs) that underrun.
1307 underflow - LLD should place (DID_ERROR << 16) in 'result' if
1308 actual number of bytes transferred is less than this
1309 figure. Not many LLDs implement this check and some that
1310 do just output an error message to the log rather than
1311 report a DID_ERROR. Better for an LLD to implement
1314 It is recommended that a LLD set 'resid' on data transfers from a SCSI
1315 target device (e.g. READs). It is especially important that 'resid' is set
1316 when such data transfers have sense keys of MEDIUM ERROR and HARDWARE ERROR
1317 (and possibly RECOVERED ERROR). In these cases if a LLD is in doubt how much
1318 data has been received then the safest approach is to indicate no bytes have
1319 been received. For example: to indicate that no valid data has been received
1320 a LLD might use these helpers:
1321 scsi_set_resid(SCpnt, scsi_bufflen(SCpnt));
1322 where 'SCpnt' is a pointer to a scsi_cmnd object. To indicate only three 512
1323 bytes blocks has been received 'resid' could be set like this:
1324 scsi_set_resid(SCpnt, scsi_bufflen(SCpnt) - (3 * 512));
1326 The scsi_cmnd structure is defined in include/scsi/scsi_cmnd.h
1331 Each struct Scsi_Host instance has a spin_lock called struct
1332 Scsi_Host::default_lock which is initialized in scsi_host_alloc() [found in
1333 hosts.c]. Within the same function the struct Scsi_Host::host_lock pointer
1334 is initialized to point at default_lock. Thereafter lock and unlock
1335 operations performed by the mid level use the struct Scsi_Host::host_lock
1336 pointer. Previously drivers could override the host_lock pointer but
1337 this is not allowed anymore.
1342 Autosense (or auto-sense) is defined in the SAM-2 document as "the
1343 automatic return of sense data to the application client coincident
1344 with the completion of a SCSI command" when a status of CHECK CONDITION
1345 occurs. LLDs should perform autosense. This should be done when the LLD
1346 detects a CHECK CONDITION status by either:
1347 a) instructing the SCSI protocol (e.g. SCSI Parallel Interface (SPI))
1348 to perform an extra data in phase on such responses
1349 b) or, the LLD issuing a REQUEST SENSE command itself
1351 Either way, when a status of CHECK CONDITION is detected, the mid level
1352 decides whether the LLD has performed autosense by checking struct
1353 scsi_cmnd::sense_buffer[0] . If this byte has an upper nibble of 7 (or 0xf)
1354 then autosense is assumed to have taken place. If it has another value (and
1355 this byte is initialized to 0 before each command) then the mid level will
1356 issue a REQUEST SENSE command.
1358 In the presence of queued commands the "nexus" that maintains sense
1359 buffer data from the command that failed until a following REQUEST SENSE
1360 may get out of synchronization. This is why it is best for the LLD
1361 to perform autosense.
1364 Changes since lk 2.4 series
1365 ===========================
1366 io_request_lock has been replaced by several finer grained locks. The lock
1367 relevant to LLDs is struct Scsi_Host::host_lock and there is
1370 The older error handling mechanism has been removed. This means the
1371 LLD interface functions abort() and reset() have been removed.
1372 The struct scsi_host_template::use_new_eh_code flag has been removed.
1374 In the 2.4 series the SCSI subsystem configuration descriptions were
1375 aggregated with the configuration descriptions from all other Linux
1376 subsystems in the Documentation/Configure.help file. In the 2.6 series,
1377 the SCSI subsystem now has its own (much smaller) drivers/scsi/Kconfig
1378 file that contains both configuration and help information.
1380 struct SHT has been renamed to struct scsi_host_template.
1382 Addition of the "hotplug initialization model" and many extra functions
1388 The following people have contributed to this document:
1389 Mike Anderson <andmike at us dot ibm dot com>
1390 James Bottomley <James dot Bottomley at hansenpartnership dot com>
1391 Patrick Mansfield <patmans at us dot ibm dot com>
1392 Christoph Hellwig <hch at infradead dot org>
1393 Doug Ledford <dledford at redhat dot com>
1394 Andries Brouwer <Andries dot Brouwer at cwi dot nl>
1395 Randy Dunlap <rdunlap at xenotime dot net>
1396 Alan Stern <stern at rowland dot harvard dot edu>
1400 dgilbert at interlog dot com