Focuses on PCIe topology, device layers, transaction flow, and the significance of each layer.
1. PCIe Topology Overview
PCIe uses a point-to-point switched architecture, unlike legacy PCI buses:
- Root Complex (RC): Connects CPU/memory to PCIe fabric
- Switches / Bridges: Connect multiple endpoints efficiently
- Endpoints (EP): Actual devices like GPUs, NICs, storage controllers
ASCII Topology Example
+------------+
| Root Complex|
+------+-----+
|
+-------+-------+
| PCIe Switch |
+-------+-------+
| | |
Endpoint1 Endpoint2 Endpoint3
✅ This topology allows scalable lanes (x1, x4, x8, x16) and high-speed communication.
2. PCIe Device Layers
Each PCIe device implements three logical layers:
| Layer | Function |
|---|---|
| Transaction Layer | Generates TLPs, handles transaction ordering, addresses, and headers |
| Data Link Layer | Ensures reliable delivery, flow control, DLLPs, and replay |
| Physical Layer | Converts digital packets to electrical signals, manages encoding, equalization, and link training |
ASCII Layer Diagram
+--------------------+
| Transaction Layer |
+--------------------+
| Data Link Layer |
+--------------------+
| Physical Layer |
+--------------------+
3. PCIe Transaction Flow
- CPU / Root Complex initiates a request (read/write)
- Transaction Layer forms a TLP with headers and routing info
- Data Link Layer appends DLLPs for reliability (ACK/NAK)
- Physical Layer encodes and sends the TLP across lanes
- Endpoint Device receives, checks integrity, and processes request
- Completion TLP is sent back using the same flow
ASCII Transaction Flow Example
[CPU/RC] -> TLP -> [TX DLLP + PHY] -> [PCIe Link] -> [Endpoint RX PHY + DLL]
Endpoint processes -> Completion TLP -> Back to RC
4. Significance of Each Layer
| Layer | Key Responsibilities |
|---|---|
| Transaction | Addressing, routing, TLP formation, QoS, ordering |
| Data Link | Flow control, ACK/NAK, replay, sequence numbers, DLLP integrity |
| Physical | Encoding/decoding, scrambling, link training, equalization, lane alignment |
✅ Each layer works together to ensure high-speed, reliable, and ordered delivery of data.
5. TLP Routing & Address Spaces
5.1 TLP Routing Methods
- Address Routing: Uses destination address to select endpoint
- ID Routing: Uses requester ID for routing inside switches
- Implicit Routing: Uses TLP headers and VC/Traffic Class hints
ASCII Routing Example
TLP Dest: 0x8000 -> Switch -> Endpoint2 (Address Routing)
TLP ReqID: 0x01 -> Switch -> Endpoint1 (ID Routing)
5.2 Address Spaces
- Memory Space: Device memory and registers
- I/O Space: Legacy I/O devices
- Configuration Space: BARs, capability registers, device configuration
- Extended Memory / MMIO: Modern devices support larger address ranges
6. Summary – Topology & Transaction Flow
Responsibilities:
- Root Complex coordinates PCIe transactions
- Switches enable efficient multi-endpoint communication
- Endpoints process TLPs with correct ordering and routing
- Layers interact: Transaction -> Data Link -> Physical
- TLP routing can use address, ID, or implicit methods
- Supports multiple address spaces (memory, I/O, config)

