[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: PCI I/O Space Consumption Limitation




The issue of using I/O mapped BAR is not as clear cut as the spec.
might suggest.   Certainly the advice of avoiding I/O BAR is bad
advice.   Please see my sermon on the subject below.

>>Devendra K Tripathi <tripathi@Synopsys.COM>
>> It may be pointed here that you have to have a Memory BAR for every
>> I/O BAR. The point is that PCI does not encourage to use I/O space
>> at all. 
>
>Indeed, revision 2.1 of the spec positively recommends *against* it.
>See the "Implementation Note: Device Address Space" on page 26:
>
>	At 11:08 AM 8/22/96 -0400, Bob Goudreau wrote:
>       It is highly recommended, [sic -- yes, there's an extraneous
>	comma there] that a device request (via Base Address
>	Register(s)) that its internal registers be mapped into Memory
>	Space and not I/O Space.  In PC systems, I/O Space is limited
>	and highly fragmented and will become more difficult to
>	allocate in the future, however the use of I/O Space is
>	allowed.  Requesting Memory Space instead of I/O Space allows
>	a device to be used in a system that does not support I/O
>	Space....

The memory and I/O decoding design of PCI boards is really highly
dependent on the intended function of the the device and on the
targeted "host system" market for the device.

For "function" of the device, the two key classes are "boot" devices
and "non-boot" devices.  These can be broken down as devices with
option ROM's and devices without option ROM's.  

The "host system" market can be broken down into "boot-time memory
constrained systems" and "boot-time memory non-constrained" systems.
The Intel x86 "IBM PC-legacy" architecture falls into the boot-time
memory constrained category.   Most RISC-based systems, Sparc, PowerPC,
Alpha, whatever fall into the non-constrained category.

The x86 IBM PC-legacy architecture machines (i.e. biggest chunk of the
current market) are constrained because this architecture uses the
x86 CPU's REAL mode for the execution of option ROM code.   Accesses
to memory in REAL mode are confined to 1MB or less.  Memory above 1MB
is not accessible from within the option ROM code.   The option ROM code
is NOT allowed under any circumstances to alter the REAL mode state 
of the machine in order to provide itself with a mechanism to access
high memory.   The only exception to this is INT 15, func 87 of the
PC-legacy (EISA created) support function in the BIOS.  Many system
BIOS do NOT correctly support this function during expansion ROM (option ROM)
INIT time making this function unuseable for option ROM writers.
The NET result is that an option rom on a PC-legacy machine cannot 
access memory above 1MB.   The PCI device's memory mappped BAR will
invariably be assigned ABOVE 1MB by a PC-legacy system BIOS because
the memory below 1MB is needed for the sole purpose of mapping in 
expansion ROM's (which MUST be mapped below 1MB because they contain
REAL mode code which can only correctly run from a relative memory
location below 1MB).  

This means that systems with option ROM's cannot access a memory mapped
region above 1MB and (in general) all PCI device memory mapped regions
will be above 1MB.   This means that a PCI device with an option ROM
in a boot-time memory access constrained system MUST supply I/O space
in order for the option ROM to accesses the device during its INIT
phase and during system Boot-time.  System boot-time is defined as the
period of time when access to PCI devices is provided solely through
the use of the system BIOS and though the use of the expansion ROM code.
ALL PC-legacy machines (including ALL x86 based machines on the market
today) have this constraint.   If your device is to used for booting
the system then you MUST supply an I/O space BAR for status/control/data
path to your PCI device.   This is NOT a requirement of the PCI specification
but rather an IMPLICIT REQUIREMENT of the PC-legacy expansion ROM definition
and the fact that system BIOS will NOT assign memory mapped BAR's below 1MB.

The PCI Specification itself is quite clear on the fact that a PCI device
MUST provide a memory mapped interface BAR for its control/status/data
functions because all systems support memory mapping and some systems support
only memory mapping.   What is less clear is that designers of bootable
devices MUST also supply a I/O interface IF they wish their device to work
correctly on the PC-x86 architecture of today.   If they do not have an I/O
interface then their expansion ROM will have no way of transfering data or
doing control/status functions since the expansion ROM will NOT be able
to access the high memory (above 1MB) memory mapped device functions of the
device.

Further, as described by others on this thread already, the devices I/O 
function must be broken up into 256 byte chunks in order allow the system
BIOS to place all such functions into the aliases of the 0-FFh range of
a 4K chunk.    This later restriction is based on the fact that some 16-bit
ISA devices were specifically designed to avoid conflicts with 10-bit I/O
devices and yet have more I/O function registers.  The 16-bit devices did
this by using one 10-bit I/O address range, and then using every alias
of the 10-bit address.   The devices themselves were 16-bit devices and
decoded address bits 10-15 so that they could destinguish the high aliases
of their 10-bit address.   The other 10-bit device would not decode these
aliases since they were aliases of th 10-bit address which did not belong
to them.   (Very few devices had this 16-bit alias quality.  The PCI spec
writers did us a diservice by requiring the 256 byte limit to maintain
compatiblity with these dying devices...)   Anyway, the 0-FFh region of
a 4K chunk is reserved to the system, and the high aliases of this region
are off-limits to the old alias using 16-bit ISA devices.  So the aliases
of the 0-FFh region are used to stick the I/O address BAR requests of PCI
devices.   Since each of the alias regions is only 256 bytes large, that
is where the 256 byte restriction in the 2.1 specification came from.
(How the hell the 2.1 writers changed the 2.0 **suggestion** for limiting
I/O requests to 256 bytes into the 2.1 **requirement** that no allocation
request be larger than 256 bytes is beyond me since there are probably a 
total of 15 of these cards out there and who cares about them anyway...)

So, if you are designing for the PC-legacy market and your device is a
bootable device, (ie. SCSI adapter, FDDI adapter, Fibre Channel adpater,
CD-ROM adapter, Network Interface Card (NIC), etc....   then you had
better put in BOTH a memory interface (a good fast one), and an I/O 
interface (at least a crappy simple one) otherwise your expansion ROM
writer will NOT be able to do her (or his) job correctly without putting
in some evil, illegal hack into the expansion ROM.  The I/O functions if
they require more than 256 bytes need to be seperated into seperate I/O
BARS of no more than 256 bytes each.    If the expansion ROM writer
puts in code to go into protected mode in order to allow access to the
PCI device in high memory then the PCI device expansion ROM will in 
definitely have compatibility problems in a large number of software
configurations.

If your PCI device will be supported on MS-DOS by a legal MS-DOS device
driver and you do not want to have to mess around with the MAJOR hassles
of using the DMPI interface from within a MS-DOS device driver; then you as
well should put in an I/O interface (even if your device is NOT a bootable
device).   The only legal way for a MS-DOS driver to get to a memory
mapped PCI device above 1MB is to use a DMPI (DOS Protected Mode Interface)
service to map a legal memory desciptor to this high memory location. 
This is usually very slow from within an MS-DOS device driver and depends
on the presense of a DPMI server [Older MS-DOS don't have them], and is
generally
a major hassle.   Most software people still write all of the diagnostics
and "setup" programs for their devices as MS-DOS programs.   If your device
has one then you should be worried about MS-DOS, even if the target market is
Windows95, or WindowsNT, or SCO UNIX, or Lynix, or Novell Netware, or 
OS/2 or AIX, or whatever.....

Intel/IBM x86 PC-architecture PCI devices just really have to have BOTH a 
memory and an I/O interface.   For ASCI devices the cost is of I/O is
negligible. 
For FPGA type designs the cost is an issue, but then so is the device
actually working in the market you intend.

So, the recommendation against using I/O space is misleading at best.
What you really need to do is design you device with BOTH a memory and
an I/O space, since the vast majority of PCI devices DO care about
PC-legacy boot-time and MS-DOS time operation via expansion ROMs or via
MS-DOS drivers.   These legacy requirements may someday go away, but not
any time soon.

-David O'Shea
daveo@corollary.com
ql[