Re: 32MB BAR Limitation with NT?

["David O'Shea" <daveo@corollary.com>]

At 09:20 AM 1/23/97 -0600, Brian Carlson wrote:
>Hello,
>
>Last week Ken Crocker identified an apparent limitation 
>of a maximum of 32MB of BAR memory space allocation under
>Windows NT.  We independently verified that when over a 
>32MB region was requested in the BAR, that NT (3.51 and 4.0) displayed:
>
>"Insufficient system resources exist to complete the 
>requested service."
>
>I would like to allocate a 256MB region for my application,
>but this prevents me from doing it. 
>
>Does anyone know what is causing this limitation, is it
>documented anywhere, and is there any way around it?  
>
>Could it be caused by the PCI BIOS, NT kernal services,
>or the PCI bridge?

The restriction is created by the NT kernel services.  Specifically,
the three services that device drivers use to reserve I/O and device
memory locations in the NT resource map.  NT may feel that devices
that want more than 32MB are being too hoggy.   Another possibility
is that these routines do not actually restrict the allocations to 32MB,
but instead believe that they are detecting a resource conflict.  You
should be able to dump the BAR values of all installed devices using
a DOS program (or DEBUG.COM if you labor at it) to verify if there are
not any resource conflicts being created by your BIOS.  

Most likely however, the BIOS is doing its job correctly, and NT has
an "artificial" limit on BAR allocations.   I use "artificial" here
because one person's definition of a restrictive improper limit that
fowls things up is often in conflict with another person's definition
of putting reasonable resource allocation limits on individual device
drivers.

For instance, all device drivers in the system that map device memory 
into the system must be mapped into the "system" address map in NT.
The total system address map is only 2GB.  The other 2GB belong to each
individual program.   Of the 2GB for system address space, some of it
is actually used for mapping real live "normal" DRAM for main memory use
in the NT kernel.  This would leave somewhat less for all of the device
mappings.    Also, NT only allocates just so many page directory entries
and page tables with page table entries for all of system use.    They
may not allocate enough to necessarily map all of the 4GB address space.
This would mean that the page table entries needed to map physical device
memory into system virtual address space may also be an "allocated" resource
within the NT system.   32MB of space would use up XX entries
in the page maps.   The writers of NT may feel that 32MB is all that
they can afford to give to each device.   This could be either because
of the limits on total address space, or because of limits on the
availability of page table entries.

Personally, I think 32MB is a touch restrictive.   However, I would not
call 128MB restrictive, if they had set the limit at that.   

As far as 256MB of space goes.   That would allow only 8 devices to be
installed if you used the ENTIRE system 2GB virtual address space up.  With
none left for actually mapping memory, etc (which is impossible).
Realistically, it probably allows for only 5 or 6 devices.   That is just
to few to design an operating system to if you were writing Windows NT
kernel code.

My personal space would be about 64MB.  That would allow for 8 * 2 *2 = 32
devices if you used up all the space, so it would allow for about 25
devices in real life.

Whomever designed the restrictions in NT probably felt that even 25 devices
was not allowing for enough devices to be guaranteed access to mapping their
device into the system.    That's why they choose 32MB, which would allow for
about 64 - 16 = 48 to 50 devices to be guaranteed a chance at a memory
allocation.

My figures are just ballpark, except for the 2GB limitation on the NT system 
virtual space, which is a definite constant.   

Anyway, I don't know of any way to get the resource allocator to allow larger
allocations.   It is either compiled into the codebase, or it uses some
double-secret NT registry item for holding the limit.   If the later, and
you knew the key, then you could of course change it.   

Only the designer at Microsoft would know how it was implemented, and
whether and where you could change an registry key value to modify the behavior
of the system.

But, as I showed in the calculations above, the OS designers were not all
that "weird" thinking in choossing 32MB as a restriction if in fact it is
a restriction.   256MB for each device is way to much.    There is just not
that much memory space around for people to design devices with huge
memory interfaces because it is either easier, or faster.   Just
implement a paging register in your device for mapping a 32MB space to
successive function blocks.

It would seem that if your implement a BusMaster device, that you don't
need anywhere near 256MB of device address space.   You just need a
small region for sending and receiving "mail boxes" of information
telling the BusMaster what to do, and to where or from where the data
should be transfered.

Large interfaces only seem usefull for "slave" devices.   Slave devices only 
redeeming qualities are that they are easier to implement than BusMasters, and
simpler to implement.   Otherwise, they have no redeeming qualities over a 
BusMaster device.   The only exception that I can think of is when the data
transfer sizes of the device are so small, that just reading the device
is just as fast as sending and receiving control "mail box" information.
But when that condition is true, the device would not typically need a 256MB
address space.

So, in short, what do you need 256MB of space for?    What is the 
legitimazation of that type of design?   Elsewise, I tend to agree that
32MB is not all that restrictive.  (Again, I think I would have picked 64MB,
since I can think of uses for that much space for video devices....)

Regards,

David O'Shea
daveo@corollary.com
Corollary Inc.
Principle Software Engineer

,Œ|