The Packet-Based Protocol in HT Technology

The Packet-Based Protocol

HyperTransport employs a packet-based protocol in which all information —address, commands, and data — travel in packets which are multiples of four bytes each. Packets are used in link management (e.g. flow control and error reporting) and as building blocks in constructing more complex transactions such as read and write data transfers.

It should be noted that, while packet descriptions in this chapter are in terms of bytes, the link's bidirectional interface width (2, 4, 8, 16, or 32 bits) ultimately determines the amount of packet information sent during each bit time on HyperTransport links. There are two bit times per clock period.

Before looking at packet function and use, the following sections describe the mechanics of packet delivery over 2,4,8,16, and 32 bit scalable link interfaces.

8 Bit Interfaces

For 8-bit interfaces, one byte of packet information may be sent in each bit time. For example, a 4-byte request packet would be sent by the transmitter during four adjacent bit times, least significant byte first as shown. Total time to complete a four-byte packet is two clock periods.

Interfaces Wider Than 8 Bits

For 16 or 32 bit interfaces, packet delivery is accelerated by sending multiple bytes of packet information in parallel with each other.

The Two Packet Types: Control And Data

Packets moving across links fall into two groups: control packets and data packets. Control packet types are further divided into three additional classes: Information, Request, and Response.

Control Packet Purpose

The three classes of control packets serve the following purposes on a HyperTransport link:

Information packets

Information packets are always 4 bytes each. They are used for nearest neighbor communication between the transmitter-receiver pairs on each link; communication between these nodes is necessary for dynamic flow control updates and other miscellaneous functions. Information packets are not buffered internally or subject to flow control; when sent by a transmitter they must be accepted by the receiver.

Request packets

Requests are 4 bytes in length if there is no address field, or 8 bytes if the packet does include an address field. They may be either posted or non-posted, and the basic job of a request is to define a pending data or message transaction, or to help bridges manage posted write transactions (through the use of Flush and Fence commands). These packets originate at a source device and are accepted by a target device.

Devices in the path between the source and target forward requests along, subject to HyperTransport rules for ordering.

Response packets

Responses are always 4 bytes each. They are returned by the target after it has serviced a non-posted request. Devices in the path between the response sender and the original requester forward responses along, subject to HyperTransport rules for ordering.

When associated with a non-posted write or flush request, the target done response packet acts as a confirmation (returned to the source device) that the operation has completed. In the event of a problem delivering non-posted write data or completing the flush, the response packet will contain an error flag and a bit indicating whether the target done response is being returned by the intended target OR by another device acting on its behalf (e.g. end-of-chain device).

For read transactions, which are always split in HyperTransport, the read response packet precedes the returning data and identifies the specific read request being serviced. In the event of an error when fetching the data, the read response will contain an error flag and a bit indicating whether the problem occurred at the intended target or at an end-of-chain device acting on its behalf. If there is an error, all data is driven back as FFh by either the target or the end-of-chain device.

Data Packets

While there is only one type of data packet, consisting of 1-16 Dwords, the payload of valid information within a data packet ranges from 0-64 valid bytes--depending on the attributes of the request that caused it. The appropriate time to send a data packet also depends on the request/response associated with it:

1. For write requests, the data packet is sent immediately after the request. Because there is no routing information in a data packet, the request is used to deliver the data to the intended target.

2. For read requests, the data packet immediately follows the read response. Key fields in the response are filled in with requester transaction stream information provided in the read request (e.g. UnitID and Source Tag). The response is then used to route the read data packet back to the original requester.

3. Atomic read-modify-write requests are a hybrid. A data packet is sent with the request (as in a write transaction) and another data packet is returned following the read response (as in a read transaction).

4. Finally, some requests don't have data packets at all (e.g. Flush and Fence).

The Need To Interleave Control And Data Packets

An important feature of HyperTransport packet management is that a transmitter may interleave control packets with data packets associated with earlier requests. Interleaving control packets with data helps mitigate "stalls" in sending new control packets on the multiplexed CAD bus when large data transfers are in progress. An example of such as stall is as follows:

1. A transmitter starts a Sized Dword Write of 64 bytes (16 dwords) on a 2-bit HyperTransport link interface.

2. After the write transaction commences, the transmitter realizes it needs to send a read request or NOP information packet over the bus.

3. Without the ability to interleave control packets, the transmitter would have to send the entire data payload first (64 bytes x 4 bit times/byte = 256 bit times). This represents a worst-case latency of 128 clocks to start sending the new control packet.

To avoid such situations, HyperTransport allows a transmitter to insert new control packets into a data payload on four byte boundaries, as long as the control packets do not have any immediate data of their own. For example, read requests and NOP flow control packets are candidates for interleaving; write requests would not be candidates for interleaving because they are accompanied by immediate data.

The CTL Signal Indicates Packet Type

A transmitter uses the CTL signal on a HyperTransport link interface to indicate the presence of control vs. data packets it is sending concurrently on the CAD bus. When CTL is asserted (high), a control packet is in transit on the CAD bus; when CTL is deasserted (low), a data packet is being sent. During idle periods, CTL is asserted and control information NOP packets are sent.

When interleaving control packets and data packets on a link, the transmitter is required to observe the following rules as it asserts and deasserts the CTL signal:

1. CTL is always asserted and deasserted on four byte boundaries.

2. The only time CTL is deasserted is when a data packet associated with an earlier control packet (e.g., request or response packet) is being sent.

3. CTL is asserted during all bit times of a control packet; for control packets which are either 4 or 8 bytes, the packet must be sent in its entirety without deasserting CTL (there is no interleaving within control packets). This also means that flow control must assure that transmitters never start sending a control packet if the receiver lacks sufficient buffer space to accept all bytes at full speed. Changes in flow control buffer availability are reported by means of NOP packets.

4. CTL is deasserted through all bit times of data packets.

5. Re-assertion of CTL within data packets is permitted on four byte boundaries if the transmitter decides to interleave a new control packet, providing it does not have immediate data of its own. After the control packet is sent, CTL is again deasserted and the current data packet transfer resumes.

6. Only one data packet may be in progress at a time, although it may be paused for the interleaving of control packet(s).

7. Ordering of control packets is not affected by the fact that data packets may be paused to interleave them.

8. The bit time immediately following the end of a data packet is always the start of a control packet, and CTL must be asserted.

• For each packet variant, the virtual channel (VChan) is indicated in the second column: posted, non-posted, or response. Note: information packets do not travel in any of the virtual channels and are not subject to flow control.

• The first byte in each control packet type contains a 6-bit Command (CMD) Code. By sending this information at the beginning of a control packet, the receiver is informed immediately of the type of packet being transferred, the number of bytes to expect, and the format of the bit fields contained within.

• In some Command Codes, a number of bits are variables (indicated by ".xxx") which are used to select transaction options: dword vs. byte transfer count, isochronous flag, coherency requirement, etc.;