Root/Documentation/PCI/pci.txt

1
2            How To Write Linux PCI Drivers
3
4        by Martin Mares <mj@ucw.cz> on 07-Feb-2000
5    updated by Grant Grundler <grundler@parisc-linux.org> on 23-Dec-2006
6
7~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8The world of PCI is vast and full of (mostly unpleasant) surprises.
9Since each CPU architecture implements different chip-sets and PCI devices
10have different requirements (erm, "features"), the result is the PCI support
11in the Linux kernel is not as trivial as one would wish. This short paper
12tries to introduce all potential driver authors to Linux APIs for
13PCI device drivers.
14
15A more complete resource is the third edition of "Linux Device Drivers"
16by Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman.
17LDD3 is available for free (under Creative Commons License) from:
18
19    http://lwn.net/Kernel/LDD3/
20
21However, keep in mind that all documents are subject to "bit rot".
22Refer to the source code if things are not working as described here.
23
24Please send questions/comments/patches about Linux PCI API to the
25"Linux PCI" <linux-pci@atrey.karlin.mff.cuni.cz> mailing list.
26
27
28
290. Structure of PCI drivers
30~~~~~~~~~~~~~~~~~~~~~~~~~~~
31PCI drivers "discover" PCI devices in a system via pci_register_driver().
32Actually, it's the other way around. When the PCI generic code discovers
33a new device, the driver with a matching "description" will be notified.
34Details on this below.
35
36pci_register_driver() leaves most of the probing for devices to
37the PCI layer and supports online insertion/removal of devices [thus
38supporting hot-pluggable PCI, CardBus, and Express-Card in a single driver].
39pci_register_driver() call requires passing in a table of function
40pointers and thus dictates the high level structure of a driver.
41
42Once the driver knows about a PCI device and takes ownership, the
43driver generally needs to perform the following initialization:
44
45    Enable the device
46    Request MMIO/IOP resources
47    Set the DMA mask size (for both coherent and streaming DMA)
48    Allocate and initialize shared control data (pci_allocate_coherent())
49    Access device configuration space (if needed)
50    Register IRQ handler (request_irq())
51    Initialize non-PCI (i.e. LAN/SCSI/etc parts of the chip)
52    Enable DMA/processing engines
53
54When done using the device, and perhaps the module needs to be unloaded,
55the driver needs to take the follow steps:
56    Disable the device from generating IRQs
57    Release the IRQ (free_irq())
58    Stop all DMA activity
59    Release DMA buffers (both streaming and coherent)
60    Unregister from other subsystems (e.g. scsi or netdev)
61    Release MMIO/IOP resources
62    Disable the device
63
64Most of these topics are covered in the following sections.
65For the rest look at LDD3 or <linux/pci.h> .
66
67If the PCI subsystem is not configured (CONFIG_PCI is not set), most of
68the PCI functions described below are defined as inline functions either
69completely empty or just returning an appropriate error codes to avoid
70lots of ifdefs in the drivers.
71
72
73
741. pci_register_driver() call
75~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
76
77PCI device drivers call pci_register_driver() during their
78initialization with a pointer to a structure describing the driver
79(struct pci_driver):
80
81    field name Description
82    ---------- ------------------------------------------------------
83    id_table Pointer to table of device ID's the driver is
84            interested in. Most drivers should export this
85            table using MODULE_DEVICE_TABLE(pci,...).
86
87    probe This probing function gets called (during execution
88            of pci_register_driver() for already existing
89            devices or later if a new device gets inserted) for
90            all PCI devices which match the ID table and are not
91            "owned" by the other drivers yet. This function gets
92            passed a "struct pci_dev *" for each device whose
93            entry in the ID table matches the device. The probe
94            function returns zero when the driver chooses to
95            take "ownership" of the device or an error code
96            (negative number) otherwise.
97            The probe function always gets called from process
98            context, so it can sleep.
99
100    remove The remove() function gets called whenever a device
101            being handled by this driver is removed (either during
102            deregistration of the driver or when it's manually
103            pulled out of a hot-pluggable slot).
104            The remove function always gets called from process
105            context, so it can sleep.
106
107    suspend Put device into low power state.
108    suspend_late Put device into low power state.
109
110    resume_early Wake device from low power state.
111    resume Wake device from low power state.
112
113        (Please see Documentation/power/pci.txt for descriptions
114        of PCI Power Management and the related functions.)
115
116    shutdown Hook into reboot_notifier_list (kernel/sys.c).
117            Intended to stop any idling DMA operations.
118            Useful for enabling wake-on-lan (NIC) or changing
119            the power state of a device before reboot.
120            e.g. drivers/net/e100.c.
121
122    err_handler See Documentation/PCI/pci-error-recovery.txt
123
124
125The ID table is an array of struct pci_device_id entries ending with an
126all-zero entry; use of the macro DEFINE_PCI_DEVICE_TABLE is the preferred
127method of declaring the table. Each entry consists of:
128
129    vendor,device Vendor and device ID to match (or PCI_ANY_ID)
130
131    subvendor, Subsystem vendor and device ID to match (or PCI_ANY_ID)
132    subdevice,
133
134    class Device class, subclass, and "interface" to match.
135            See Appendix D of the PCI Local Bus Spec or
136            include/linux/pci_ids.h for a full list of classes.
137            Most drivers do not need to specify class/class_mask
138            as vendor/device is normally sufficient.
139
140    class_mask limit which sub-fields of the class field are compared.
141            See drivers/scsi/sym53c8xx_2/ for example of usage.
142
143    driver_data Data private to the driver.
144            Most drivers don't need to use driver_data field.
145            Best practice is to use driver_data as an index
146            into a static list of equivalent device types,
147            instead of using it as a pointer.
148
149
150Most drivers only need PCI_DEVICE() or PCI_DEVICE_CLASS() to set up
151a pci_device_id table.
152
153New PCI IDs may be added to a device driver pci_ids table at runtime
154as shown below:
155
156echo "vendor device subvendor subdevice class class_mask driver_data" > \
157/sys/bus/pci/drivers/{driver}/new_id
158
159All fields are passed in as hexadecimal values (no leading 0x).
160The vendor and device fields are mandatory, the others are optional. Users
161need pass only as many optional fields as necessary:
162    o subvendor and subdevice fields default to PCI_ANY_ID (FFFFFFFF)
163    o class and classmask fields default to 0
164    o driver_data defaults to 0UL.
165
166Note that driver_data must match the value used by any of the pci_device_id
167entries defined in the driver. This makes the driver_data field mandatory
168if all the pci_device_id entries have a non-zero driver_data value.
169
170Once added, the driver probe routine will be invoked for any unclaimed
171PCI devices listed in its (newly updated) pci_ids list.
172
173When the driver exits, it just calls pci_unregister_driver() and the PCI layer
174automatically calls the remove hook for all devices handled by the driver.
175
176
1771.1 "Attributes" for driver functions/data
178
179Please mark the initialization and cleanup functions where appropriate
180(the corresponding macros are defined in <linux/init.h>):
181
182    __init Initialization code. Thrown away after the driver
183            initializes.
184    __exit Exit code. Ignored for non-modular drivers.
185
186
187    __devinit Device initialization code.
188            Identical to __init if the kernel is not compiled
189            with CONFIG_HOTPLUG, normal function otherwise.
190    __devexit The same for __exit.
191
192Tips on when/where to use the above attributes:
193    o The module_init()/module_exit() functions (and all
194      initialization functions called _only_ from these)
195      should be marked __init/__exit.
196
197    o Do not mark the struct pci_driver.
198
199    o The ID table array should be marked __devinitconst; this is done
200      automatically if the table is declared with DEFINE_PCI_DEVICE_TABLE().
201
202    o The probe() and remove() functions should be marked __devinit
203      and __devexit respectively. All initialization functions
204      exclusively called by the probe() routine, can be marked __devinit.
205      Ditto for remove() and __devexit.
206
207    o If mydriver_remove() is marked with __devexit(), then all address
208      references to mydriver_remove must use __devexit_p(mydriver_remove)
209      (in the struct pci_driver declaration for example).
210      __devexit_p() will generate the function name _or_ NULL if the
211      function will be discarded. For an example, see drivers/net/tg3.c.
212
213    o Do NOT mark a function if you are not sure which mark to use.
214      Better to not mark the function than mark the function wrong.
215
216
217
2182. How to find PCI devices manually
219~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
220
221PCI drivers should have a really good reason for not using the
222pci_register_driver() interface to search for PCI devices.
223The main reason PCI devices are controlled by multiple drivers
224is because one PCI device implements several different HW services.
225E.g. combined serial/parallel port/floppy controller.
226
227A manual search may be performed using the following constructs:
228
229Searching by vendor and device ID:
230
231    struct pci_dev *dev = NULL;
232    while (dev = pci_get_device(VENDOR_ID, DEVICE_ID, dev))
233        configure_device(dev);
234
235Searching by class ID (iterate in a similar way):
236
237    pci_get_class(CLASS_ID, dev)
238
239Searching by both vendor/device and subsystem vendor/device ID:
240
241    pci_get_subsys(VENDOR_ID,DEVICE_ID, SUBSYS_VENDOR_ID, SUBSYS_DEVICE_ID, dev).
242
243You can use the constant PCI_ANY_ID as a wildcard replacement for
244VENDOR_ID or DEVICE_ID. This allows searching for any device from a
245specific vendor, for example.
246
247These functions are hotplug-safe. They increment the reference count on
248the pci_dev that they return. You must eventually (possibly at module unload)
249decrement the reference count on these devices by calling pci_dev_put().
250
251
252
2533. Device Initialization Steps
254~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
255
256As noted in the introduction, most PCI drivers need the following steps
257for device initialization:
258
259    Enable the device
260    Request MMIO/IOP resources
261    Set the DMA mask size (for both coherent and streaming DMA)
262    Allocate and initialize shared control data (pci_allocate_coherent())
263    Access device configuration space (if needed)
264    Register IRQ handler (request_irq())
265    Initialize non-PCI (i.e. LAN/SCSI/etc parts of the chip)
266    Enable DMA/processing engines.
267
268The driver can access PCI config space registers at any time.
269(Well, almost. When running BIST, config space can go away...but
270that will just result in a PCI Bus Master Abort and config reads
271will return garbage).
272
273
2743.1 Enable the PCI device
275~~~~~~~~~~~~~~~~~~~~~~~~~
276Before touching any device registers, the driver needs to enable
277the PCI device by calling pci_enable_device(). This will:
278    o wake up the device if it was in suspended state,
279    o allocate I/O and memory regions of the device (if BIOS did not),
280    o allocate an IRQ (if BIOS did not).
281
282NOTE: pci_enable_device() can fail! Check the return value.
283
284[ OS BUG: we don't check resource allocations before enabling those
285  resources. The sequence would make more sense if we called
286  pci_request_resources() before calling pci_enable_device().
287  Currently, the device drivers can't detect the bug when when two
288  devices have been allocated the same range. This is not a common
289  problem and unlikely to get fixed soon.
290
291  This has been discussed before but not changed as of 2.6.19:
292    http://lkml.org/lkml/2006/3/2/194
293]
294
295pci_set_master() will enable DMA by setting the bus master bit
296in the PCI_COMMAND register. It also fixes the latency timer value if
297it's set to something bogus by the BIOS. pci_clear_master() will
298disable DMA by clearing the bus master bit.
299
300If the PCI device can use the PCI Memory-Write-Invalidate transaction,
301call pci_set_mwi(). This enables the PCI_COMMAND bit for Mem-Wr-Inval
302and also ensures that the cache line size register is set correctly.
303Check the return value of pci_set_mwi() as not all architectures
304or chip-sets may support Memory-Write-Invalidate. Alternatively,
305if Mem-Wr-Inval would be nice to have but is not required, call
306pci_try_set_mwi() to have the system do its best effort at enabling
307Mem-Wr-Inval.
308
309
3103.2 Request MMIO/IOP resources
311~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
312Memory (MMIO), and I/O port addresses should NOT be read directly
313from the PCI device config space. Use the values in the pci_dev structure
314as the PCI "bus address" might have been remapped to a "host physical"
315address by the arch/chip-set specific kernel support.
316
317See Documentation/IO-mapping.txt for how to access device registers
318or device memory.
319
320The device driver needs to call pci_request_region() to verify
321no other device is already using the same address resource.
322Conversely, drivers should call pci_release_region() AFTER
323calling pci_disable_device().
324The idea is to prevent two devices colliding on the same address range.
325
326[ See OS BUG comment above. Currently (2.6.19), The driver can only
327  determine MMIO and IO Port resource availability _after_ calling
328  pci_enable_device(). ]
329
330Generic flavors of pci_request_region() are request_mem_region()
331(for MMIO ranges) and request_region() (for IO Port ranges).
332Use these for address resources that are not described by "normal" PCI
333BARs.
334
335Also see pci_request_selected_regions() below.
336
337
3383.3 Set the DMA mask size
339~~~~~~~~~~~~~~~~~~~~~~~~~
340[ If anything below doesn't make sense, please refer to
341  Documentation/DMA-API.txt. This section is just a reminder that
342  drivers need to indicate DMA capabilities of the device and is not
343  an authoritative source for DMA interfaces. ]
344
345While all drivers should explicitly indicate the DMA capability
346(e.g. 32 or 64 bit) of the PCI bus master, devices with more than
34732-bit bus master capability for streaming data need the driver
348to "register" this capability by calling pci_set_dma_mask() with
349appropriate parameters. In general this allows more efficient DMA
350on systems where System RAM exists above 4G _physical_ address.
351
352Drivers for all PCI-X and PCIe compliant devices must call
353pci_set_dma_mask() as they are 64-bit DMA devices.
354
355Similarly, drivers must also "register" this capability if the device
356can directly address "consistent memory" in System RAM above 4G physical
357address by calling pci_set_consistent_dma_mask().
358Again, this includes drivers for all PCI-X and PCIe compliant devices.
359Many 64-bit "PCI" devices (before PCI-X) and some PCI-X devices are
36064-bit DMA capable for payload ("streaming") data but not control
361("consistent") data.
362
363
3643.4 Setup shared control data
365~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
366Once the DMA masks are set, the driver can allocate "consistent" (a.k.a. shared)
367memory. See Documentation/DMA-API.txt for a full description of
368the DMA APIs. This section is just a reminder that it needs to be done
369before enabling DMA on the device.
370
371
3723.5 Initialize device registers
373~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
374Some drivers will need specific "capability" fields programmed
375or other "vendor specific" register initialized or reset.
376E.g. clearing pending interrupts.
377
378
3793.6 Register IRQ handler
380~~~~~~~~~~~~~~~~~~~~~~~~
381While calling request_irq() is the last step described here,
382this is often just another intermediate step to initialize a device.
383This step can often be deferred until the device is opened for use.
384
385All interrupt handlers for IRQ lines should be registered with IRQF_SHARED
386and use the devid to map IRQs to devices (remember that all PCI IRQ lines
387can be shared).
388
389request_irq() will associate an interrupt handler and device handle
390with an interrupt number. Historically interrupt numbers represent
391IRQ lines which run from the PCI device to the Interrupt controller.
392With MSI and MSI-X (more below) the interrupt number is a CPU "vector".
393
394request_irq() also enables the interrupt. Make sure the device is
395quiesced and does not have any interrupts pending before registering
396the interrupt handler.
397
398MSI and MSI-X are PCI capabilities. Both are "Message Signaled Interrupts"
399which deliver interrupts to the CPU via a DMA write to a Local APIC.
400The fundamental difference between MSI and MSI-X is how multiple
401"vectors" get allocated. MSI requires contiguous blocks of vectors
402while MSI-X can allocate several individual ones.
403
404MSI capability can be enabled by calling pci_enable_msi() or
405pci_enable_msix() before calling request_irq(). This causes
406the PCI support to program CPU vector data into the PCI device
407capability registers.
408
409If your PCI device supports both, try to enable MSI-X first.
410Only one can be enabled at a time. Many architectures, chip-sets,
411or BIOSes do NOT support MSI or MSI-X and the call to pci_enable_msi/msix
412will fail. This is important to note since many drivers have
413two (or more) interrupt handlers: one for MSI/MSI-X and another for IRQs.
414They choose which handler to register with request_irq() based on the
415return value from pci_enable_msi/msix().
416
417There are (at least) two really good reasons for using MSI:
4181) MSI is an exclusive interrupt vector by definition.
419   This means the interrupt handler doesn't have to verify
420   its device caused the interrupt.
421
4222) MSI avoids DMA/IRQ race conditions. DMA to host memory is guaranteed
423   to be visible to the host CPU(s) when the MSI is delivered. This
424   is important for both data coherency and avoiding stale control data.
425   This guarantee allows the driver to omit MMIO reads to flush
426   the DMA stream.
427
428See drivers/infiniband/hw/mthca/ or drivers/net/tg3.c for examples
429of MSI/MSI-X usage.
430
431
432
4334. PCI device shutdown
434~~~~~~~~~~~~~~~~~~~~~~~
435
436When a PCI device driver is being unloaded, most of the following
437steps need to be performed:
438
439    Disable the device from generating IRQs
440    Release the IRQ (free_irq())
441    Stop all DMA activity
442    Release DMA buffers (both streaming and consistent)
443    Unregister from other subsystems (e.g. scsi or netdev)
444    Disable device from responding to MMIO/IO Port addresses
445    Release MMIO/IO Port resource(s)
446
447
4484.1 Stop IRQs on the device
449~~~~~~~~~~~~~~~~~~~~~~~~~~~
450How to do this is chip/device specific. If it's not done, it opens
451the possibility of a "screaming interrupt" if (and only if)
452the IRQ is shared with another device.
453
454When the shared IRQ handler is "unhooked", the remaining devices
455using the same IRQ line will still need the IRQ enabled. Thus if the
456"unhooked" device asserts IRQ line, the system will respond assuming
457it was one of the remaining devices asserted the IRQ line. Since none
458of the other devices will handle the IRQ, the system will "hang" until
459it decides the IRQ isn't going to get handled and masks the IRQ (100,000
460iterations later). Once the shared IRQ is masked, the remaining devices
461will stop functioning properly. Not a nice situation.
462
463This is another reason to use MSI or MSI-X if it's available.
464MSI and MSI-X are defined to be exclusive interrupts and thus
465are not susceptible to the "screaming interrupt" problem.
466
467
4684.2 Release the IRQ
469~~~~~~~~~~~~~~~~~~~
470Once the device is quiesced (no more IRQs), one can call free_irq().
471This function will return control once any pending IRQs are handled,
472"unhook" the drivers IRQ handler from that IRQ, and finally release
473the IRQ if no one else is using it.
474
475
4764.3 Stop all DMA activity
477~~~~~~~~~~~~~~~~~~~~~~~~~
478It's extremely important to stop all DMA operations BEFORE attempting
479to deallocate DMA control data. Failure to do so can result in memory
480corruption, hangs, and on some chip-sets a hard crash.
481
482Stopping DMA after stopping the IRQs can avoid races where the
483IRQ handler might restart DMA engines.
484
485While this step sounds obvious and trivial, several "mature" drivers
486didn't get this step right in the past.
487
488
4894.4 Release DMA buffers
490~~~~~~~~~~~~~~~~~~~~~~~
491Once DMA is stopped, clean up streaming DMA first.
492I.e. unmap data buffers and return buffers to "upstream"
493owners if there is one.
494
495Then clean up "consistent" buffers which contain the control data.
496
497See Documentation/DMA-API.txt for details on unmapping interfaces.
498
499
5004.5 Unregister from other subsystems
501~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
502Most low level PCI device drivers support some other subsystem
503like USB, ALSA, SCSI, NetDev, Infiniband, etc. Make sure your
504driver isn't losing resources from that other subsystem.
505If this happens, typically the symptom is an Oops (panic) when
506the subsystem attempts to call into a driver that has been unloaded.
507
508
5094.6 Disable Device from responding to MMIO/IO Port addresses
510~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
511io_unmap() MMIO or IO Port resources and then call pci_disable_device().
512This is the symmetric opposite of pci_enable_device().
513Do not access device registers after calling pci_disable_device().
514
515
5164.7 Release MMIO/IO Port Resource(s)
517~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
518Call pci_release_region() to mark the MMIO or IO Port range as available.
519Failure to do so usually results in the inability to reload the driver.
520
521
522
5235. How to access PCI config space
524~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
525
526You can use pci_(read|write)_config_(byte|word|dword) to access the config
527space of a device represented by struct pci_dev *. All these functions return 0
528when successful or an error code (PCIBIOS_...) which can be translated to a text
529string by pcibios_strerror. Most drivers expect that accesses to valid PCI
530devices don't fail.
531
532If you don't have a struct pci_dev available, you can call
533pci_bus_(read|write)_config_(byte|word|dword) to access a given device
534and function on that bus.
535
536If you access fields in the standard portion of the config header, please
537use symbolic names of locations and bits declared in <linux/pci.h>.
538
539If you need to access Extended PCI Capability registers, just call
540pci_find_capability() for the particular capability and it will find the
541corresponding register block for you.
542
543
544
5456. Other interesting functions
546~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
547
548pci_find_slot() Find pci_dev corresponding to given bus and
549                slot numbers.
550pci_set_power_state() Set PCI Power Management state (0=D0 ... 3=D3)
551pci_find_capability() Find specified capability in device's capability
552                list.
553pci_resource_start() Returns bus start address for a given PCI region
554pci_resource_end() Returns bus end address for a given PCI region
555pci_resource_len() Returns the byte length of a PCI region
556pci_set_drvdata() Set private driver data pointer for a pci_dev
557pci_get_drvdata() Return private driver data pointer for a pci_dev
558pci_set_mwi() Enable Memory-Write-Invalidate transactions.
559pci_clear_mwi() Disable Memory-Write-Invalidate transactions.
560
561
562
5637. Miscellaneous hints
564~~~~~~~~~~~~~~~~~~~~~~
565
566When displaying PCI device names to the user (for example when a driver wants
567to tell the user what card has it found), please use pci_name(pci_dev).
568
569Always refer to the PCI devices by a pointer to the pci_dev structure.
570All PCI layer functions use this identification and it's the only
571reasonable one. Don't use bus/slot/function numbers except for very
572special purposes -- on systems with multiple primary buses their semantics
573can be pretty complex.
574
575Don't try to turn on Fast Back to Back writes in your driver. All devices
576on the bus need to be capable of doing it, so this is something which needs
577to be handled by platform and generic code, not individual drivers.
578
579
580
5818. Vendor and device identifications
582~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
583
584One is not not required to add new device ids to include/linux/pci_ids.h.
585Please add PCI_VENDOR_ID_xxx for vendors and a hex constant for device ids.
586
587PCI_VENDOR_ID_xxx constants are re-used. The device ids are arbitrary
588hex numbers (vendor controlled) and normally used only in a single
589location, the pci_device_id table.
590
591Please DO submit new vendor/device ids to pciids.sourceforge.net project.
592
593
594
5959. Obsolete functions
596~~~~~~~~~~~~~~~~~~~~~
597
598There are several functions which you might come across when trying to
599port an old driver to the new PCI interface. They are no longer present
600in the kernel as they aren't compatible with hotplug or PCI domains or
601having sane locking.
602
603pci_find_device() Superseded by pci_get_device()
604pci_find_subsys() Superseded by pci_get_subsys()
605pci_find_slot() Superseded by pci_get_slot()
606
607
608The alternative is the traditional PCI device driver that walks PCI
609device lists. This is still possible but discouraged.
610
611
612
61310. MMIO Space and "Write Posting"
614~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
615
616Converting a driver from using I/O Port space to using MMIO space
617often requires some additional changes. Specifically, "write posting"
618needs to be handled. Many drivers (e.g. tg3, acenic, sym53c8xx_2)
619already do this. I/O Port space guarantees write transactions reach the PCI
620device before the CPU can continue. Writes to MMIO space allow the CPU
621to continue before the transaction reaches the PCI device. HW weenies
622call this "Write Posting" because the write completion is "posted" to
623the CPU before the transaction has reached its destination.
624
625Thus, timing sensitive code should add readl() where the CPU is
626expected to wait before doing other work. The classic "bit banging"
627sequence works fine for I/O Port space:
628
629       for (i = 8; --i; val >>= 1) {
630               outb(val & 1, ioport_reg); /* write bit */
631               udelay(10);
632       }
633
634The same sequence for MMIO space should be:
635
636       for (i = 8; --i; val >>= 1) {
637               writeb(val & 1, mmio_reg); /* write bit */
638               readb(safe_mmio_reg); /* flush posted write */
639               udelay(10);
640       }
641
642It is important that "safe_mmio_reg" not have any side effects that
643interferes with the correct operation of the device.
644
645Another case to watch out for is when resetting a PCI device. Use PCI
646Configuration space reads to flush the writel(). This will gracefully
647handle the PCI master abort on all platforms if the PCI device is
648expected to not respond to a readl(). Most x86 platforms will allow
649MMIO reads to master abort (a.k.a. "Soft Fail") and return garbage
650(e.g. ~0). But many RISC platforms will crash (a.k.a."Hard Fail").
651
652

Archive Download this file



interactive