Focuses on PCIe topology, device layers, transaction flow, and the significance of each layer.


1. PCIe Topology Overview

PCIe uses a point-to-point switched architecture, unlike legacy PCI buses:

  • Root Complex (RC): Connects CPU/memory to PCIe fabric
  • Switches / Bridges: Connect multiple endpoints efficiently
  • Endpoints (EP): Actual devices like GPUs, NICs, storage controllers

ASCII Topology Example

          +------------+
          | Root Complex|
          +------+-----+
                 |
         +-------+-------+
         | PCIe Switch   |
         +-------+-------+
         |       |       |
      Endpoint1 Endpoint2 Endpoint3

✅ This topology allows scalable lanes (x1, x4, x8, x16) and high-speed communication.


2. PCIe Device Layers

Each PCIe device implements three logical layers:

LayerFunction
Transaction LayerGenerates TLPs, handles transaction ordering, addresses, and headers
Data Link LayerEnsures reliable delivery, flow control, DLLPs, and replay
Physical LayerConverts digital packets to electrical signals, manages encoding, equalization, and link training

ASCII Layer Diagram

+--------------------+
| Transaction Layer  |
+--------------------+
| Data Link Layer    |
+--------------------+
| Physical Layer     |
+--------------------+

3. PCIe Transaction Flow

  1. CPU / Root Complex initiates a request (read/write)
  2. Transaction Layer forms a TLP with headers and routing info
  3. Data Link Layer appends DLLPs for reliability (ACK/NAK)
  4. Physical Layer encodes and sends the TLP across lanes
  5. Endpoint Device receives, checks integrity, and processes request
  6. Completion TLP is sent back using the same flow

ASCII Transaction Flow Example

[CPU/RC] -> TLP -> [TX DLLP + PHY] -> [PCIe Link] -> [Endpoint RX PHY + DLL]
Endpoint processes -> Completion TLP -> Back to RC

4. Significance of Each Layer

LayerKey Responsibilities
TransactionAddressing, routing, TLP formation, QoS, ordering
Data LinkFlow control, ACK/NAK, replay, sequence numbers, DLLP integrity
PhysicalEncoding/decoding, scrambling, link training, equalization, lane alignment

✅ Each layer works together to ensure high-speed, reliable, and ordered delivery of data.


5. TLP Routing & Address Spaces

5.1 TLP Routing Methods

  • Address Routing: Uses destination address to select endpoint
  • ID Routing: Uses requester ID for routing inside switches
  • Implicit Routing: Uses TLP headers and VC/Traffic Class hints

ASCII Routing Example

TLP Dest: 0x8000 -> Switch -> Endpoint2 (Address Routing)
TLP ReqID: 0x01 -> Switch -> Endpoint1 (ID Routing)

5.2 Address Spaces

  • Memory Space: Device memory and registers
  • I/O Space: Legacy I/O devices
  • Configuration Space: BARs, capability registers, device configuration
  • Extended Memory / MMIO: Modern devices support larger address ranges

6. Summary – Topology & Transaction Flow

Responsibilities:

  • Root Complex coordinates PCIe transactions
  • Switches enable efficient multi-endpoint communication
  • Endpoints process TLPs with correct ordering and routing
  • Layers interact: Transaction -> Data Link -> Physical
  • TLP routing can use address, ID, or implicit methods
  • Supports multiple address spaces (memory, I/O, config)

Scroll to Top