Tuesday, June 26, 2007

More about CPU Packets

Rejecting Packets

Once system initialization is complete, packet rejection should be a rare event. The only non-error rejection after initialization would be the arrival of a broadcast message which is intended to travel to the end of the chain and then be dropped by the end-of-chain device. Prior to the completion of system initialization, other devices may have not completed link initialization (Link Initialization bit is clear). If they have not been programmed to hold incoming packets pending completion of initialization (Drop on Uninitialized Link bit is set), they will behave as an end-of-chain device and reject packets temporarily.

Actions taken when a packet is rejected by an end-of-chain device, or by an interior device which has not completed initialization and is behaving temporarily as one, depends on the type of packet.

Rules For Rejection

  1. Broadcast requests which have completed the trip to the end of a chain are silently dropped. These are always posted, so no response is expected or sent.

  2. Non-posted downstream directed requests (UnitID = 0) cause the return of a Target Done response (for non-posted writes or Flush) or a Data Response (for reads or Atomic RMW). The response for rejected non-posted downstream requests will have the Error and NXA error bits set, and the bridge bit clear. The UnitID field will be set to either 0 or that of the end-of-chain device. Read responses are accompanied by all requested data (driven to value of FFh)

  3. Non-Posted upstream directed requests (non-zero UnitID) cause the return of a Target Done response (for non-posted writes or Flush) or a Data Response (for reads or Atomic RMW). The response for rejected non-posted upstream requests will have the Error and NXA error bits set, and the bridge bit set. The UnitID field must be set to that of the original requester (without its UnitID and the bridge bit set in the response, an interior node won't accept the response). Read responses are accompanied by all requested data (driven to value of FFh)

  4. Rejected response and posted request packets are dropped.


Host Bridge Behavior

Host bridges always reside at the ends of HyperTransport chains. They never forward packets, but will have occasion to accept and reject packets. There is additional complexity caused by the responsibilities host bridges may have in supporting double-hosted chains (this is optional) and peer-to-peer transfers (required). The following sections describe host bridge behavior in accepting and rejecting packets they receive.

Directed Request With UnitID = 0

Directed requests with a UnitID = 0 detected by the HyperTransport interface of a host bridge are inbound from the host at the other end of a double-hosted chain.

Accepted

If the host bridge recipient of the request has implemented internal memory or I/O space and owns the address range targeted in the request, it will respond to the request as an interior node would: accepting posted requests and returning Target Done or Read responses/data for non-posted requests.

The host bridge will similarly accept Type 0 configuration cycles carrying the proper fields if two conditions are met:

  1. The host bridge supports double-hosted chains

  2. The Host/Secondary Interface Command Register Host Hide bit is clear

Rejected

If the host bridge does not support double-hosted chains or doesn't own the address range contained in the request, it will be rejected in a similiar way to any other end-of-chain device. Posted requests are dropped (they may be reported as end-of-chain errors); non-posted requests cause the return of a Target Done or Read response with Error and NXA error bits set; for read requests, all requested data is also returned with the response (driven to value of FFh).

Response UnitID And Bridge Fields

A host bridge receiving a non-posted, directed request with UnitID = 0 would also set the response Unit ID field = 0 unless it is programmed to act as the slave bridge in a double-hosted chain. If this is the case, the Host/Secondary Interface Command Register Act As Slave bit would be set = 1, and the bridge would set the UnitID in the response to the value programmed into its Host/Secondary Interface Device Number Register.

Broadcast Request

Broadcast requests detected by a host bridge could only be coming from another host bridge on the other end of a double-hosted chain.

Always Accepted

If a broadcast message is seen by a host bridge, it has already been seen throughout the chain. The host bridge accepts it, and silently drops it. The specification indicates that a host bridge could optionally implement an internal space addressable with a broadcast message; if it does this, the message would be accepted and routed to the internal target.

Directed Request With Non-Zero UnitID

Directed requests with a non-zero UnitID are sourced by interior nodes and may be destined either for internal space within the host bridge (e.g. main memory) or for another device downstream (a peer-to-peer request). The host bridge evaluates the command type and address contained in the request and routes it accordingly. It is also possible that the request packet does not map to any internal space or device on the chain; in that case it will be rejected.

Accepted Requests
Internal Target

If the host bridge recipient of the request has implemented internal memory or I/O space and owns the address range targeted in the request, it will respond to the request as an interior node would: accepting posted requests and returning Target Done or Read responses/data for non-posted requests. This would be the typical behavior during DMA transfers from HyperTransport peripherals to and from main memory via the host bridge.

Peer-to-Peer Target

It is also possible that a host bridge examines the request packet address and determines that the address range is actually downstream in the HyperTransport topology. The target could either be on the same chain as the requester or on another chain if the host supports more than one. Hypertransport doesn't support direct peer-to-peer transfers and requires bridges to handle them as two separate requests (followed by two separate responses if non-posted). The sequence of events in accepting a peer-to-peer transactions include: (assume a non-posted peer-to-peer request)

  1. An interior node issues a non-posted request. The UnitID is that of the requester; the Source tag field and Sequence ID fields are assigned by the requester from its pool of available tags.

  2. The host bridge examines the request and determines it is peer-to-peer. It reissues the request downstream on the appropriate chain. When it does this, it changes the UnitID to 0 (its own) and changes the Source Tag and Sequence ID fields to values from its own pool of available tags. All other fields are passed through unchanged.

  3. The host bridge must track outstanding transactions so that when the response is returned, the original UnitID and Source Tag values can be restored when the response is reissued downstream to the original requester.

  4. Upon return of a response from the target (bridge bit is clear), the bridge sends the response downstream on the chain containing the requester with the bridge bit set = 1, and UnitID and Source Tag restored. The bridge may then retire the transaction from its outstanding request queue.

  5. The response (and data, if any) is claimed by the original requester based on the UnitID field and the fact the bridge bit = 1. It uses the Source Tag to determine the specific transaction which has been serviced.

Compatibility Chain Requests

Another peer-to-peer possibility is that an inbound directed request from an interior node (non-zero UnitID) targets a legacy device on the compatibility chain. If the host bridge supports a compatibility chain, it may reissue requests that don't target any other address space to that chain. Again, it replaces the UnitID, Source Tag (for non-posted requests), and Sequence ID field (if non-zero) with its own values. It also sets the Compat bit in the request it sends onto the compatibility chain. This informs all devices which see it that it should be forwarded downstream to the subtractive decoder (compatibility bridge).

If the original request was non-posted, the host bridge will again track the outstanding request so it can restore the original UnitID and Source Tag information in the response packet it sends to the original requester.

Rejected Requests

If the host bridge does not recognize the address carried by an inbound directed request from an interior node and doesn't support a compatibility chain, the packet will be rejected in a similiar way to any other end-of-chain error. Posted requests are dropped (they may be reported as end-of-chain errors); non-posted requests cause the return of a Target Done or Read response with Error and NXA error bits set; for read requests, all requested data is returned (driven to value of FFh).

Responses Received By The Host Bridge

When a response is received by a host bridge, either the bridge bit is set or it is not.

Response With Bridge Bit = 1

A response with the bridge bit set always originates at a host bridge. If one is received by a host bridge, it means that another host bridge in a double-hosted chain attempted to respond to an interior node and the node failed to claim it. In this case, it continued to the end of the chain where it will be handled as an end-of-chain error by the other host bridge. The response is dropped, and the receiving host bridge may log it as an end-of-chain error.

Response With Bridge Bit = 0

Responses arriving at a host bridge with the bridge bit cleared belong to the host bridge. The Target Done or Read response/data is being returned in response to a non-posted request issued by this host. Devices are required to track all of their outstanding requests (those requiring responses), so the bridge simply uses the Source Tag field in the response to determine which transaction is being serviced. In the event a response returns with the bridge bit clear, but the response Source Tag doesn't match any outstanding transactions, the node may log the error and report it.

No comments: