1Linux for S/390 and zSeries
3Common Device Support (CDS)
4Device Driver I/O Support Routines
6Authors : Ingo Adlung
7      Cornelia Huck
9Copyright, IBM Corp. 1999-2002
13This document describes the common device support routines for Linux/390.
14Different than other hardware architectures, ESA/390 has defined a unified
15I/O access method. This gives relief to the device drivers as they don't
16have to deal with different bus types, polling versus interrupt
17processing, shared versus non-shared interrupt processing, DMA versus port
18I/O (PIO), and other hardware features more. However, this implies that
19either every single device driver needs to implement the hardware I/O
20attachment functionality itself, or the operating system provides for a
21unified method to access the hardware, providing all the functionality that
22every single device driver would have to provide itself.
24The document does not intend to explain the ESA/390 hardware architecture in
25every detail.This information can be obtained from the ESA/390 Principles of
26Operation manual (IBM Form. No. SA22-7201).
28In order to build common device support for ESA/390 I/O interfaces, a
29functional layer was introduced that provides generic I/O access methods to
30the hardware.
32The common device support layer comprises the I/O support routines defined
33below. Some of them implement common Linux device driver interfaces, while
34some of them are ESA/390 platform specific.
37In order to write a driver for S/390, you also need to look into the interface
38described in Documentation/s390/driver-model.txt.
40Note for porting drivers from 2.4:
41The major changes are:
42* The functions use a ccw_device instead of an irq (subchannel).
43* All drivers must define a ccw_driver (see driver-model.txt) and the associated
44  functions.
45* request_irq() and free_irq() are no longer done by the driver.
46* The oper_handler is (kindof) replaced by the probe() and set_online() functions
47  of the ccw_driver.
48* The not_oper_handler is (kindof) replaced by the remove() and set_offline()
49  functions of the ccw_driver.
50* The channel device layer is gone.
51* The interrupt handlers must be adapted to use a ccw_device as argument.
52  Moreover, they don't return a devstat, but an irb.
53* Before initiating an io, the options must be set via ccw_device_set_options().
54* Instead of calling read_dev_chars()/read_conf_data(), the driver issues
55  the channel program and handles the interrupt itself.
58   get commands from extended sense data.
64   initiate an I/O request.
67   resume channel program execution.
70   terminate the current I/O request processed on the device.
73   generic interrupt routine. This function is called by the interrupt entry
74   routine whenever an I/O interrupt is presented to the system. The do_IRQ()
75   routine determines the interrupt status and calls the device specific
76   interrupt handler according to the rules (flags) defined during I/O request
77   initiation with do_IO().
79The next chapters describe the functions other than do_IRQ() in more details.
80The do_IRQ() interface is not described, as it is called from the Linux/390
81first level interrupt handler only and does not comprise a device driver
82callable interface. Instead, the functional description of do_IO() also
83describes the input to the device specific interrupt handler.
85Note: All explanations apply also to the 64 bit architecture s390x.
88Common Device Support (CDS) for Linux/390 Device Drivers
90General Information
92The following chapters describe the I/O related interface routines the
93Linux/390 common device support (CDS) provides to allow for device specific
94driver implementations on the IBM ESA/390 hardware platform. Those interfaces
95intend to provide the functionality required by every device driver
96implementation to allow to drive a specific hardware device on the ESA/390
97platform. Some of the interface routines are specific to Linux/390 and some
98of them can be found on other Linux platforms implementations too.
99Miscellaneous function prototypes, data declarations, and macro definitions
100can be found in the architecture specific C header file
103Overview of CDS interface concepts
105Different to other hardware platforms, the ESA/390 architecture doesn't define
106interrupt lines managed by a specific interrupt controller and bus systems
107that may or may not allow for shared interrupts, DMA processing, etc.. Instead,
108the ESA/390 architecture has implemented a so called channel subsystem, that
109provides a unified view of the devices physically attached to the systems.
110Though the ESA/390 hardware platform knows about a huge variety of different
111peripheral attachments like disk devices (aka. DASDs), tapes, communication
112controllers, etc. they can all be accessed by a well defined access method and
113they are presenting I/O completion a unified way : I/O interruptions. Every
114single device is uniquely identified to the system by a so called subchannel,
115where the ESA/390 architecture allows for 64k devices be attached.
117Linux, however, was first built on the Intel PC architecture, with its two
118cascaded 8259 programmable interrupt controllers (PICs), that allow for a
119maximum of 15 different interrupt lines. All devices attached to such a system
120share those 15 interrupt levels. Devices attached to the ISA bus system must
121not share interrupt levels (aka. IRQs), as the ISA bus bases on edge triggered
122interrupts. MCA, EISA, PCI and other bus systems base on level triggered
123interrupts, and therewith allow for shared IRQs. However, if multiple devices
124present their hardware status by the same (shared) IRQ, the operating system
125has to call every single device driver registered on this IRQ in order to
126determine the device driver owning the device that raised the interrupt.
128Up to kernel 2.4, Linux/390 used to provide interfaces via the IRQ (subchannel).
129For internal use of the common I/O layer, these are still there. However,
130device drivers should use the new calling interface via the ccw_device only.
132During its startup the Linux/390 system checks for peripheral devices. Each
133of those devices is uniquely defined by a so called subchannel by the ESA/390
134channel subsystem. While the subchannel numbers are system generated, each
135subchannel also takes a user defined attribute, the so called device number.
136Both subchannel number and device number cannot exceed 65535. During sysfs
137initialisation, the information about control unit type and device types that
138imply specific I/O commands (channel command words - CCWs) in order to operate
139the device are gathered. Device drivers can retrieve this set of hardware
140information during their initialization step to recognize the devices they
141support using the information saved in the struct ccw_device given to them.
142This methods implies that Linux/390 doesn't require to probe for free (not
143armed) interrupt request lines (IRQs) to drive its devices with. Where
144applicable, the device drivers can use issue the READ DEVICE CHARACTERISTICS
145ccw to retrieve device characteristics in its online routine.
147In order to allow for easy I/O initiation the CDS layer provides a
148ccw_device_start() interface that takes a device specific channel program (one
149or more CCWs) as input sets up the required architecture specific control blocks
150and initiates an I/O request on behalf of the device driver. The
151ccw_device_start() routine allows to specify whether it expects the CDS layer
152to notify the device driver for every interrupt it observes, or with final status
153only. See ccw_device_start() for more details. A device driver must never issue
154ESA/390 I/O commands itself, but must use the Linux/390 CDS interfaces instead.
156For long running I/O request to be canceled, the CDS layer provides the
157ccw_device_halt() function. Some devices require to initially issue a HALT
158SUBCHANNEL (HSCH) command without having pending I/O requests. This function is
159also covered by ccw_device_halt().
162get_ciw() - get command information word
164This call enables a device driver to get information about supported commands
165from the extended SenseID data.
167struct ciw *
168ccw_device_get_ciw(struct ccw_device *cdev, __u32 cmd);
170cdev - The ccw_device for which the command is to be retrieved.
171cmd - The command type to be retrieved.
173ccw_device_get_ciw() returns:
174NULL - No extended data available, invalid device or command not found.
175!NULL - The command requested.
178ccw_device_start() - Initiate I/O Request
180The ccw_device_start() routines is the I/O request front-end processor. All
181device driver I/O requests must be issued using this routine. A device driver
182must not issue ESA/390 I/O commands itself. Instead the ccw_device_start()
183routine provides all interfaces required to drive arbitrary devices.
185This description also covers the status information passed to the device
186driver's interrupt handler as this is related to the rules (flags) defined
187with the associated I/O request when calling ccw_device_start().
189int ccw_device_start(struct ccw_device *cdev,
190             struct ccw1 *cpa,
191             unsigned long intparm,
192             __u8 lpm,
193             unsigned long flags);
194int ccw_device_start_timeout(struct ccw_device *cdev,
195                 struct ccw1 *cpa,
196                 unsigned long intparm,
197                 __u8 lpm,
198                 unsigned long flags,
199                 int expires);
200int ccw_device_start_key(struct ccw_device *cdev,
201             struct ccw1 *cpa,
202             unsigned long intparm,
203             __u8 lpm,
204             __u8 key,
205             unsigned long flags);
206int ccw_device_start_key_timeout(struct ccw_device *cdev,
207                 struct ccw1 *cpa,
208                 unsigned long intparm,
209                 __u8 lpm,
210                 __u8 key,
211                 unsigned long flags,
212                 int expires);
214cdev : ccw_device the I/O is destined for
215cpa : logical start address of channel program
216user_intparm : user specific interrupt information; will be presented
217           back to the device driver's interrupt handler. Allows a
218               device driver to associate the interrupt with a
219               particular I/O request.
220lpm : defines the channel path to be used for a specific I/O
221               request. A value of 0 will make cio use the opm.
222key : the storage key to use for the I/O (useful for operating on a
223           storage with a storage key != default key)
224flag : defines the action to be performed for I/O processing
225expires : timeout value in jiffies. The common I/O layer will terminate
226           the running program after this and call the interrupt handler
227           with ERR_PTR(-ETIMEDOUT) as irb.
229Possible flag values are :
231DOIO_ALLOW_SUSPEND - channel program may become suspended
232DOIO_DENY_PREFETCH - don't allow for CCW prefetch; usually
233                           this implies the channel program might
234                           become modified
235DOIO_SUPPRESS_INTER - don't call the handler on intermediate status
237The cpa parameter points to the first format 1 CCW of a channel program :
239struct ccw1 {
240      __u8 cmd_code;/* command code */
241      __u8 flags; /* flags, like IDA addressing, etc. */
242      __u16 count; /* byte count */
243      __u32 cda; /* data address */
244} __attribute__ ((packed,aligned(8)));
246with the following CCW flags values defined :
248CCW_FLAG_DC - data chaining
249CCW_FLAG_CC - command chaining
250CCW_FLAG_SLI - suppress incorrect length
251CCW_FLAG_SKIP - skip
253CCW_FLAG_IDA - indirect addressing
254CCW_FLAG_SUSPEND - suspend
257Via ccw_device_set_options(), the device driver may specify the following
258options for the device:
260DOIO_EARLY_NOTIFICATION - allow for early interrupt notification
261DOIO_REPORT_ALL - report all interrupt conditions
264The ccw_device_start() function returns :
266      0 - successful completion or request successfully initiated
267-EBUSY - The device is currently processing a previous I/O request, or there is
268          a status pending at the device.
269-ENODEV - cdev is invalid, the device is not operational or the ccw_device is
270          not online.
272When the I/O request completes, the CDS first level interrupt handler will
273accumulate the status in a struct irb and then call the device interrupt handler.
274The intparm field will contain the value the device driver has associated with a
275particular I/O request. If a pending device status was recognized,
276intparm will be set to 0 (zero). This may happen during I/O initiation or delayed
277by an alert status notification. In any case this status is not related to the
278current (last) I/O request. In case of a delayed status notification no special
279interrupt will be presented to indicate I/O completion as the I/O request was
280never started, even though ccw_device_start() returned with successful completion.
282The irb may contain an error value, and the device driver should check for this
285-ETIMEDOUT: the common I/O layer terminated the request after the specified
286            timeout value
287-EIO: the common I/O layer terminated the request due to an error state
289If the concurrent sense flag in the extended status word (esw) in the irb is
290set, the field erw.scnt in the esw describes the number of device specific
291sense bytes available in the extended control word irb->scsw.ecw[]. No device
292sensing by the device driver itself is required.
294The device interrupt handler can use the following definitions to investigate
295the primary unit check source coded in sense byte 0 :
302SNS0_OVERRUN 0x04
305Depending on the device status, multiple of those values may be set together.
306Please refer to the device specific documentation for details.
308The irb->scsw.cstat field provides the (accumulated) subchannel status :
310SCHN_STAT_PCI - program controlled interrupt
311SCHN_STAT_INCORR_LEN - incorrect length
312SCHN_STAT_PROG_CHECK - program check
313SCHN_STAT_PROT_CHECK - protection check
314SCHN_STAT_CHN_DATA_CHK - channel data check
315SCHN_STAT_CHN_CTRL_CHK - channel control check
316SCHN_STAT_INTF_CTRL_CHK - interface control check
317SCHN_STAT_CHAIN_CHECK - chaining check
319The irb->scsw.dstat field provides the (accumulated) device status :
321DEV_STAT_ATTENTION - attention
322DEV_STAT_STAT_MOD - status modifier
323DEV_STAT_CU_END - control unit end
324DEV_STAT_BUSY - busy
325DEV_STAT_CHN_END - channel end
326DEV_STAT_DEV_END - device end
327DEV_STAT_UNIT_CHECK - unit check
328DEV_STAT_UNIT_EXCEP - unit exception
330Please see the ESA/390 Principles of Operation manual for details on the
331individual flag meanings.
333Usage Notes :
335ccw_device_start() must be called disabled and with the ccw device lock held.
337The device driver is allowed to issue the next ccw_device_start() call from
338within its interrupt handler already. It is not required to schedule a
339bottom-half, unless a non deterministically long running error recovery procedure
340or similar needs to be scheduled. During I/O processing the Linux/390 generic
341I/O device driver support has already obtained the IRQ lock, i.e. the handler
342must not try to obtain it again when calling ccw_device_start() or we end in a
343deadlock situation!
345If a device driver relies on an I/O request to be completed prior to start the
346next it can reduce I/O processing overhead by chaining a NoOp I/O command
347CCW_CMD_NOOP to the end of the submitted CCW chain. This will force Channel-End
348and Device-End status to be presented together, with a single interrupt.
349However, this should be used with care as it implies the channel will remain
350busy, not being able to process I/O requests for other devices on the same
351channel. Therefore e.g. read commands should never use this technique, as the
352result will be presented by a single interrupt anyway.
354In order to minimize I/O overhead, a device driver should use the
355DOIO_REPORT_ALL only if the device can report intermediate interrupt
356information prior to device-end the device driver urgently relies on. In this
357case all I/O interruptions are presented to the device driver until final
358status is recognized.
360If a device is able to recover from asynchronously presented I/O errors, it can
361perform overlapping I/O using the DOIO_EARLY_NOTIFICATION flag. While some
362devices always report channel-end and device-end together, with a single
363interrupt, others present primary status (channel-end) when the channel is
364ready for the next I/O request and secondary status (device-end) when the data
365transmission has been completed at the device.
367Above flag allows to exploit this feature, e.g. for communication devices that
368can handle lost data on the network to allow for enhanced I/O processing.
370Unless the channel subsystem at any time presents a secondary status interrupt,
371exploiting this feature will cause only primary status interrupts to be
372presented to the device driver while overlapping I/O is performed. When a
373secondary status without error (alert status) is presented, this indicates
374successful completion for all overlapping ccw_device_start() requests that have
375been issued since the last secondary (final) status.
377Channel programs that intend to set the suspend flag on a channel command word
378(CCW) must start the I/O operation with the DOIO_ALLOW_SUSPEND option or the
379suspend flag will cause a channel program check. At the time the channel program
380becomes suspended an intermediate interrupt will be generated by the channel
383ccw_device_resume() - Resume Channel Program Execution
385If a device driver chooses to suspend the current channel program execution by
386setting the CCW suspend flag on a particular CCW, the channel program execution
387is suspended. In order to resume channel program execution the CIO layer
388provides the ccw_device_resume() routine.
390int ccw_device_resume(struct ccw_device *cdev);
392cdev - ccw_device the resume operation is requested for
394The ccw_device_resume() function returns:
396        0 - suspended channel program is resumed
397-EBUSY - status pending
398-ENODEV - cdev invalid or not-operational subchannel
399-EINVAL - resume function not applicable
400-ENOTCONN - there is no I/O request pending for completion
402Usage Notes:
403Please have a look at the ccw_device_start() usage notes for more details on
404suspended channel programs.
406ccw_device_halt() - Halt I/O Request Processing
408Sometimes a device driver might need a possibility to stop the processing of
409a long-running channel program or the device might require to initially issue
410a halt subchannel (HSCH) I/O command. For those purposes the ccw_device_halt()
411command is provided.
413ccw_device_halt() must be called disabled and with the ccw device lock held.
415int ccw_device_halt(struct ccw_device *cdev,
416                    unsigned long intparm);
418cdev : ccw_device the halt operation is requested for
419intparm : interruption parameter; value is only used if no I/O
420          is outstanding, otherwise the intparm associated with
421          the I/O request is returned
423The ccw_device_halt() function returns :
425      0 - request successfully initiated
426-EBUSY - the device is currently busy, or status pending.
427-ENODEV - cdev invalid.
428-EINVAL - The device is not operational or the ccw device is not online.
430Usage Notes :
432A device driver may write a never-ending channel program by writing a channel
433program that at its end loops back to its beginning by means of a transfer in
434channel (TIC) command (CCW_CMD_TIC). Usually this is performed by network
435device drivers by setting the PCI CCW flag (CCW_FLAG_PCI). Once this CCW is
436executed a program controlled interrupt (PCI) is generated. The device driver
437can then perform an appropriate action. Prior to interrupt of an outstanding
438read to a network device (with or without PCI flag) a ccw_device_halt()
439is required to end the pending operation.
441ccw_device_clear() - Terminage I/O Request Processing
443In order to terminate all I/O processing at the subchannel, the clear subchannel
444(CSCH) command is used. It can be issued via ccw_device_clear().
446ccw_device_clear() must be called disabled and with the ccw device lock held.
448int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm);
450cdev: ccw_device the clear operation is requested for
451intparm: interruption parameter (see ccw_device_halt())
453The ccw_device_clear() function returns:
455      0 - request successfully initiated
456-ENODEV - cdev invalid
457-EINVAL - The device is not operational or the ccw device is not online.
459Miscellaneous Support Routines
461This chapter describes various routines to be used in a Linux/390 device
462driver programming environment.
466Get the address of the device specific lock. This is then used in
467spin_lock() / spin_unlock() calls.
470__u8 ccw_device_get_path_mask(struct ccw_device *cdev);
472Get the mask of the path currently available for cdev.

Archive Download this file