Re: PCI BAR and Expansion ROM

["David O'Shea" <daveo@corollary.com>]

Windson,

You just keep reading that PCI spec and finding all of the features which
are supposed to help you out, but really don't......   Of course the 
reasons for not doing things the way that it seems clear in the SPEC
that you should be able to do them is not written down anywhere.....

My comments are folded in below:
-David O'Shea 

At 10:30 AM 8/13/96 PDT, w_wong@emulex.com wrote:
>
>Hello, everyone,
>
>First of all, thank you very much for the enthusiastic inputs from
>everybody especially David O'Shea from Corollary.
>it's really helpful. I have some more questions, though.
>
>ASSUMPTION:
>(1) Architecture: Intel PC and PCI boot device.
>(2) O.S:  Window NT, Window95 or SCO and others.
>(3) BAR:  The memory map address only; NO ALTERNATIVE
>I/O ACCESS TO THE BAR.
>
>PCI BAR QUESTION:
>Can I force the TYPE bits of the BAR to be under 1M and
>implement INT13 in real mode to access the BAR? Any
>unforseen problems with above assumptions?

-daveo
This is a technically legal solution.  But don't do it.  Many system
BIOS (on x86 PC's) simply refuse to configure devices which request
memory mapping below 1MB using the "under 1M" bit in the BAR.   The
reason is that they have to satisfy such a request using the C8000-E0000h
region.  This same region used for mapping in Expansion ROM's, including
PCI Expansion ROM's after shadowing.   This region is so tight on space
that BIOS writers put in arbitrary, but "reasonable", restrictions on
what they will do with this space.   For some BIOS's, this includes
refusing to configure devices which request memory below 1M using the
1M BAR bit.   (Other BIOS's, as talked about in the Expansion ROM thread
that has been going on, also refuse to let expansion ROM's ask for
too much space.) 

Quite simply, although keeping your devices interface to a single convient
memory interface is a laudible goal, its not a pratical goal in the x86
PC environment for a BOOT device.   Since the BOOT ROM is going to run
in x86 REAL mode, it is stuck with the 1984 era 8086 code base and machine
architecture of the IBM PC.   That means terribly little room for 
< 1MB device memory, terribly little room for expansion ROM's, and no good 
practical way to take a trip into protected mode during POST or during
real mode OS realtime (i.e. DOS runtime).  

You really just need to put in an I/O interface.   [Now if the chip is
already done, and you have some means of setting the < 1MB bit in the
BAR through firmware, then by all means go for it.   At least it will work
in some machines.]

THREAD CHANGE:
-daveo
A note on the expasion ROM issue:   Although expansion ROM's can "shrink",
they do so after the INIT routine has run.   The INIT code still needs to
be shadowed into the lower memory region for this initial run on x86
systems because the code base to be run is in x86 REAL mode coding style,
as per the IBM PC AT Tech ref. of 1986....  The only region that is below
1MB and which can therefore support the REAL mode addressing in the expansion
ROM is in the lower memory BIOS shadow region at C8000-E0000h.  There is 
not much space in this region.   This means that if a given expansion ROM
were the very first to run, then it would have a maximum size of E0000h-C80000h
= 64K=32K=96K.    That is the maximum if the Expansion ROM is run FIRST!
If other ROM's have gone before you, then they are already mapped, and
the available space is probably less.  Most likely much less.  This means
that as a practical matter the system BIOS writer has two "easy" bad choices.
Choice one is to put no restrictions on the size, and potentially have the
Expansion ROM not shrink, and then have no space left for anyone (else
he has to "unconfigure the device", which is a real hassle).  Or two, put
on some sensible restriction.   Based on the notes that have come in
earlier mail, this restriction tends between 32K and 64K.   [Remember, total
space for all ROM is 96K].

Since the system BIOS does not know what the INIT code will do, the BIOS
cannot allow the INIT code to run and then upon seeing its final size
decide that it should undo the configuration of the device.  For instance,
a BOOT ROM's INIT code will chain itself into the INT 13 and INT 19 real
mode vector chains on an x86 PC architecture machine.   Once this is done,
there is no way for the system BIOS to "undo it".   So once the system BIOS
allows the INIT code to run it must keep it in there, even if the code
does not "shrink" itself.   This means that the BIOS essentially has to
protect itself early and not wait for the final size of the possibly shrunk
ROM.   That is the primary reason that most system BIOS put in a size 
restriction on the Expansion ROM code.   There is no solution to this 
problem.   The PCI BIOS Expansion ROM header was defined stupidly without
consideration for this problem and hence there is no good solution.  
If the header contained both the total size and the "after INIT" size,
then the BIOS could no ahead of letting the INIT code run what the final
size would be.   Life would have then been much better.   But that it not
how it is.....   So that is why you have size restrictions....
END THREAD CHANGE:

>
>PCI EXPANSION ROM QUESTIONS:
>Since Gilbert Yeung from Lanworks Technologies Inc brought up
>some interesting issues about PCI Expansion ROM, I have
>some questions also.
>
>(1) During INIT before the O.S is booted, can the Intialization routine
>switch to Protected mode to access Expansion Rom Register
>instead of calling INT15 function 87?

No.  The system BIOS is allowed to assume that it may have set up its own
memory selectors in protected mode and then left that mode.  If you 
go into protected mode and reload the selectors, then they will not be in
the state the system BIOS left them in.  This can cause problems for the
system BIOS.   You are not allowed to do this.  You will break crater
some system BIOS by doing this.   The reaons for an INT 15, fn 87 call
is so that the BIOS can return the descriptors to that state that it
wants them if it needs to.  (Its just a shame that the BIOS writers never
check out there bloody code to make sure the the protected mode functions
really work correctly.....)

>(2) If there is an address in the Expansion ROM Base Register after
>the O.S. is booted, is it safe for the Device Driver to claim it (in
>Protected Mode) by setting bit 0 of the ROM BAR? This is under the
>assumption that BAR and Expansion ROM DO NOT share the same
>address decoder for the Boot Device.

No.  The area that was used for Expansion ROM decode may not be accessible
via the Host to PCI bus bridge programming at run time.   This is a detail
of the system BIOS implementation.   For instance, the BIOS may map in
a region of the memory space for PCI for dealing with expansion ROM that
it finds convenient during PCI processing during POST.   This area can
be different than the area it uses to actually assign device memory space
to the other BAR's.   When the Expansion ROM processing is done, then it
can map that memory space back into "memory", instead of to PCI memory 
space.  This is a re-configuration of the Host-to-PCI bridge.   This case
I gave is just an example...    But basically if the expansion ROM enable
bit is OFF, then you can't do anything with the Expansion ROM BAR value,
even if the BIOS failed to clear it to 0 as it is supposed to.

>(3) According to the book " PCI Hardware and Software Architecture
>& Design", author by Edward Solari & George Willse, PCI System
>BIOS will zero out the ROM Register after it copied the ROM image to
>RAM area (c8000h to e0000h).  However, it did not happened to one
>of my PC system.  Is it up to the individual System BIOS vendor to
>zero out after the ROM image was copied to the RAM area? In
>other words, some will do and some will not do?

Yes.  Some BIOS will clear the register and others do not.  Some BIOS
find it sufficient to just clear the Expansion ROM enable bit.   The
BIOS that leave the register set (except for the enable bit) really should
clear it, some many don't.


>Thank you very much.
>
>Windson Wong
>(714) 513-8119
>Emulex Corp Network Systems.
>3535 Harbor Bl.,
>Costa Mesa, Ca92626
>
>
#
¤”