Re: proposal to fix ordering problem in PCI 2.1

["McDonald, Ed" <mcd@msgate.ColumbiaSC.NCR.COM>]

I agree with HP and others that PCI 2.1 has ordering
problems.   In addition, I think systems designers desiring
robust systems using PCI are being challenged to solve
some of the PCI weaknesses regarding split transaction
support that David James alludes to.

I'd suggest that the HP proposal is the first step to
solving this particular ordering problem.  However,
I think the basic forward progress issue is also
unresolved and left up to the system designer to address.
The problem with this is that "off the shelf" PCI arbiters,
PCI chipset, and PCI P2P bridge vendors can inadvertently
make it very difficult to build a robust system--especially
for larger servers that typically desire to have "wider"
and/or "deeper"  PCI tree topologies.

A couple years ago we seriously explored doing the same
thing as the HP proposal coupled with a reservation
strategy to ensure forward progress both upstream
and downstream from PCI.  Using these "master" or
"owner" ID's coupled with a reservation strategy we
could ensure forward progress throughout the system
up to the interface to our system's connectors.
The ordering problems HP and others refer to
were inherently solved by this solution due to the
"tagging" of the two portions of the split transaction.
However, because we couldn't control adapters with
P2P bridges with on board arbiters, etc.  there was/is
still some exposure to forward progress issues.

I suggest that the HP proposal be further extended/considered
in light of forward progress issues.  Addressing forward
progress can be non-trivial effort depending on
the extent of coverage (e.g. if trying to address a wide
range of latency and throughput characteristics for
the numerous "PCI-tree" topologies ("width and depth"
can be quite varied in large PCI systems)).  I suspect,
however, that perhaps there are some simplified
constraints/suggestions for P2P bridge and system
vendors that could make it significantly easier to
use "off the shelf" chips if these chips had some
minimal support to address forward progress (for
example, suggestions on how to use the "master"
ID's and delayed transactions effectively as P2P
bridges are crossed).

I would suggest considering extending the HP
proposal to allow multiple pending transactions
from the same agent (i.e. a transaction ID in
addition to an "master/owner/agent" ID) and
how to propagate this information across P2P
bridges be considered.  (This can allow much
more efficient hardware for reservation algorithms.)

In summary, I think David Gustavson hit the nail on
the head about trying to use PCI for some things
that it wasn't initially designed for.  There are other
features that more robust split transaction systems
and/or protocols consider that are not in PCI (such
as protocol checks, timeout protection, etc.).
This was obviously a trade-off, and was likely done
to keep it simple and that's okay.   Unfortunately,
I think the forward progress weakness has been in the
spec from day one.   If the PCI-SIG doesn't want to
address the forward progress issue, I still see the
HP proposal making it considerably easier for system
designers to  address forward progress in larger
systems (in addition to solving the transaction
ordering problems).

Ed McDonald
NCR   (Opinions above are mine, and not necessarily NCR's)


���