Tuesday, June 26, 2007

How HyperTransport Handles Configuration Accesses

Configuration Cycles Are Memory Mapped

To generate a configuration space read or write, a HyperTransport bridge simply sends a RdSized or non-posted WrSized request using a reserved address range in the 40-bit HyperTransport memory map. This 32MB range, recognized by all devices,

How The 32MB Configuration Area Is Used

The 32MB HyperTransport memory map address space reserved for configuration cycles is used to access the 256 byte configuration space of each function in each device on each bus. How the address range is interpreted and how a particular device can recognize configuration cycles it should claim vs. those it must forward

Upper 16 Address Bits Indicate Type 0 And Type 1 Cycle

As in PCI configuration cycles, HyperTransport requires two variants of configuration read/write cycles, type 1 and type 0. The type 0 configuration is generated by a bridge when the cycle has reached the target bus (chain) where the device being accessed resides; the type 1 cycle is in transit to the target bus and should be forwarded by bridges or tunnels in the target path. The bridge to the destination bus will convert it to type 0.

Because HyperTransport configuration cycles are distinguished from other read/write requests only by the fact they target the 32MB reserved configuration address range, the first problem is how to distinguish type 1 from type 0 cycles. The 32MB configuration address range is further divided into two parts: request packets carrying addresses in the upper 16MB of the range are type 1 cycles; requests with addresses in the lower 16MB are type 0 cycles.

HyperTransport Type 1 Configuration Cycle

If a SizedRD or SizedWt request carries an address with the upper 16 bits set = FDFFh, then the cycle is a type 1 configuration request. Only bridges are allowed to accept these requests, and only if the bus number field in the address (bits 23:16) falls into the range defined by the bridges Secondary-Subordinate bus number registers. The bridge then passes the request downstream.

HyperTransport Type 0 Configuration Cycle

If a SizedRD or SizedWt request carries an address with the upper 16 bits set = FDF8h, then the cycle is a type 0 configuration request. This will be claimed by the device that also has a match when the device number field (bits 15:11) in the address matches one of its UnitIDs. It then uses the function number and Dword fields to target the particular internal function and configuration space offset.

No IDSEL Signal Needed In HyperTransport

Finally, there is no IDSEL signal to accompany a type 0 configuration cycle in the HyperTransport protocol. The need for this signal has been eliminated because a Base UnitID field has been included in the HyperTransport advanced capability register block so that a device is programmed to "know" its UnitID number(s). This allows the device to decode its own configuration cycles rather than depending on the upstream bridge to do it with IDSEL.

Events In HT Configuration Example
  1. Low level software executing on the CPU requires access of the configuration space in Device 2 on Bus (chain) number 1.

  2. The Host Bridge checks its secondary bus number register, recognizes the target bus is not its secondary bus, and sends a request packet for type 1 configuration cycle onto bus 0 (using the upper half of configuration address range).

  3. The HT-to-HT bridge on Bus 0 checks the bus number field in the request and compares it with its own secondary, and subordinate bus numbers. Because the target bus is below it, the HT-to-HT bridge forwards the configuration cycle onto bus 1; at the same time it converts it to a type 0 because the target bus has been reached. Conversion to type 0 simply means shifting the configuration address into the lower half of the configuration address range. Note that the bus number field is stripped off when the cycle is converted to type 0.

  4. Device 1 claims the cycle because it is a type 0 configuration cycle AND it carries a device number which matches one of its assigned UnitIDs.

  5. Device 1 then uses function number and dword offset fields in the request packet to target the specific internal function and offset location in its configuration space

Initializing Bus Numbers And Unit IDs

One of the first steps in HyperTransport configuration is the initial assignment of bus numbers and UnitIDs for each device and chain in the topology. Using a depth-first search algorithm, enumeration software assigns IDs to each device it discovers; if it finds any HyperTransport bridges, it also assigns the primary, secondary, and subordinate bus numbers so that later configuration cycles may find their way to target buses other than bus 0.

Case 1: A Single Chain With One Host Bridge

In a single chain with only one host bridge, enumeration is fairly simple:

  1. Following a reset assertion on a chain, the Base UnitID field in the Slave Command register of each in HyperTransport device is cleared to "0".

  2. In addition, reset forces the primary, secondary, and subordinate bus number registers in each HyperTransport bridge and the secondary bus number register in host bridges to "0" as well.

  3. The transmitter and receiver interfaces on each link perform the low-level negotiation to determine starting bus width. They also perform the required link initialization sequence. Once synchronization is complete, the Initialization Complete bit in each active Link Control register is set.

  4. After link synchronization, each active transmitter issues buffer release (NOP) packets to the corresponding receiver to indicate its own input flow control buffer capacities. Once this is done, each transmitter issues NOPs until configuration starts.

  5. The host bridge initializes its UnitID counter so it can start assigning UnitIDs to slave devices it discovers (it reserves UnitID 0 for itself).

  6. If the host bridge's Link Control register Initialization Complete and End Of Chain bits indicate that another device is attached to its secondary bus, the host bridge sends a series of configuration cycles to the first device in the chain. These type 0 configuration cycles target Bus 0, Device 0 (UnitID 0), Function 0. Because all devices default to UnitID 0, the first device will claim the cycles. Read cycles will target configuration space locations containing Vendor ID, Device ID, Class Code, Header Type, etc.

  7. At some point, the host bridge assigns new UnitID(s) to the device by reading the Unit Count field in the Slave Command Register and then programming (writing) the Base UnitID field with the next available UnitID (1). For devices which request more than one Unit ID, this Base UnitID is the first in a sequential set. Note that the act of writing the Command register causes the Base UnitID field to be updated and the Master Host bit to be set (indicating the device link which points towards the host bridge). Thereafter, the device uses its new UnitID when claiming configuration cycles, etc. Only a reset or rewriting the Slave Command register causes the Base UnitID field to change.

  8. Once all functions in the first device are configured, the host repeats the process to access the next device in the chain. It again uses the configuration cycle attributes of Bus 0, Device 0 (UnitID 0), Function 0. Now, the device which is already assigned as UnitID 1, forwards the transaction downstream because the UnitID in the request (0) does not match. The second device is then programmed as the first one was, but the UnitID(s) assigned to it start where the previous device left off (i.e., UnitID 2).

  9. After programming each device, the host bridge checks the End-Of-Chain (EOC) bit set in the device's downstream Link Control Register. If this bit is set = 1, the enumeration process for the chain is complete.

Case 2: A HyperTransport Bridge Is Discovered

If the enumeration process on a chain encounters a HyperTransport-To-HyperTransport bridge or a bridge from HyperTransport to a compatible protocol (PCI, AGP, PCI-X), then some additional initialization is needed. A bridge is detected when a read of the Header Type field in the configuration header indicates that the device uses the type 1 header format. Software must program the device in accordance with the type 1 header format which includes:

  1. Programming the secondary and subordinate bus number registers with the next available bus number (1). This will allow this bridge to forward and/or convert subsequent configuration cycles targeting the new bus(ses) below the bridge.

  2. Setting up the Base Address Registers and other fields in the configuration header in accordance with the protocol being used on the secondary bus (HyperTransport, PCI-X, PCI, etc.).

It is permissible for a HyperTransport bridge to have more than one secondary bus and/or a tunnel interface for its primary bus.

A Note About Bus Numbering In HyperTransport

Bus numbering in HyperTransport systems makes no distinction between HyperTransport, PCI, AGP, or PCI-X buses. As bridges to other protocols are discovered during enumeration, bus numbers are assigned without regard to the particular protocol.

No comments: