Visible to Intel only — GUID: ldh1665161368209
Ixiasoft
Visible to Intel only — GUID: ldh1665161368209
Ixiasoft
3.6.6. Horizontal Bandwidth Considerations
The horizontal network of switches that comprise the hard memory NoC connect with 512-bit wide links. Within each hard memory NoC, there are two 512-bit links carrying request transactions (AW, W, AR) left-to-right, and an additional two 512-bit links carrying request transactions right-to-left. Similarly, there are two 512-bit links carrying response transactions (R, B) left-to-right, and two 512-bit links carrying response transactions right-to-left.
You can calculate the maximum bandwidth of each bus by multiplying the NoC operating frequency with the width of the bus. For example, if the hard memory NoC is operating at 1.4 GHz, then each 64-byte bus contributes 89.6 GB/s. Table 4. Hard Memory NoC Horizontal Bandwidth summarizes the total bandwidth in each direction for each transaction type. Note that the Link Fmax and Max Bandwidth are lower on speed-grade 3 devices.
Transaction Type | Direction | Link Count | Link Width (bit) | -1, -2 Speed Grade Link Fmax (GHz) | -1, -2 Speed Grade Max Bandwidth (GB/s) | -3 Speed Grade Link Fmax (GHz) | -3 Speed Grade Max Bandwidth (GB/s) |
---|---|---|---|---|---|---|---|
Request (AW, W, AR) | Left-to-Right | 2 | 512 | 1.4 | 179.2 | 1.0 | 128 |
Right-to-Left | 2 | 512 | 1.4 | 179.2 | 1.0 | 128 | |
Response (R, B) |
Left-to-Right | 2 | 512 | 1.4 | 179.2 | 1.0 | 128 |
Right-to-Left | 2 | 512 | 1.4 | 179.2 | 1.0 | 128 |
Figure 13. Horizontal Link Allocation for Top-Edge NoC shows the links that connect to each NoC target. For each hard memory NoC along the top edge and bottom edge of the device, there are 24 NoC target interface bridges for connecting to HBM2e or external memory controllers. Again, for request transactions, there are two links in each direction. Each of these links connects to alternating NoC targets. LR0 and LR1 are the two links that carry traffic left-to-right. RL0 and RL1 are the two links that carry traffic right-to-left. Some of the connections are Local, which indicates that the initiator and target connect to the same switch. Traffic between such an initiator and target need not transfer horizontally.
When placing NoC initiators and NoC targets, placing some initiators to the left of their targets and some initiators to the right of their targets can reduce congestion on the hard memory NoC. Connections where the initiator is to the left of the target do not compete for bandwidth with connections where the initiator is to the right of the target because there are separate links for carrying traffic right-to-left versus left-to-right. Also, because separate links service consecutive targets (for example RL0 versus RL1 ), traffic to one target does not compete for bandwidth with traffic to an adjacent target.
For example, two initiators on the right side of the die with high bandwidth requirements for targets on the left side of the die can require communication over the same links if the RL0 link services both targets.
If the RL0 link services one of the targets, while the RL1 link services the other target, the two connections do not compete for bandwidth. Alternatively, if one of the initiators moves to the left of its target, the read traffic then uses one of the left-to-right links (for example LR0), and does not compete for bandwidth with other traffic using the right-to-left links (such as RL0).
Figure 15. Example NoC Exceeding Horizontal Bandwidth Limits shows an example of the top-edge hard memory NoC with targets in the UIB segment sending read data to initiators to the left side.
In Figure 15. Example NoC Exceeding Horizontal Bandwidth Limits if each of these connections carries 20 GB/s of data, the maximum demand on the horizontal link is 120 GB/s. This maximum demand is greater than the 89.6 GB/s capacity of the link. Note that this example only shows targets that the LR0 horizontal link services. The LR1 link services other targets and does not compete for bandwidth with these initiator-target connections.
You can alleviate such bandwidth overload by placing some high bandwidth initiators to the left of their targets, and other high bandwidth initiators to the right of their targets.
Transactions between initiators and targets that connect directly to the same switch route locally, and need not transfer horizontally along these links. Therefore, the horizontal bandwidth that these local connections use does not count against the horizontal bandwidth limits.
In Figure 16. Example NoC Within Horizontal Bandwidth Limits, the locations of initiators from the previous example change. Some initiators are to the left of their targets (using the LR0 horizontal link), some initiators are to the right of their targets (using the RL0 horizontal link), and some initiators are directly across from their targets (represented by a vertical arrow using local connections instead of the horizontal links).