Peripheral Component Interconnect Express (PCIe) is the modern standard for connecting high-speed devices to a computer system. It replaced parallel buses like PCI and PCI-X with a high-speed serial, point-to-point interface, offering:
- High bandwidth (up to 64 GT/s in Gen6)
- Scalable lanes (x1, x4, x8, x16, x32)
- Low latency
- Hot-plug support
- Advanced error handling and power management
Real-world applications include:
- GPUs (NVIDIA, AMD)
- NVMe SSDs (Samsung, WD)
- Network cards (10/25/100 GbE)
- AI accelerators, FPGA cards
- WiFi/5G modules
2. PCI vs PCI-X vs PCIe (Evolution)
| Feature | PCI (1992) | PCI-X (1998) | PCIe (2003–Now) |
|---|---|---|---|
| Type | Parallel Bus | Parallel Bus | Serial, Point-to-Point |
| Width | 32/64 bit | 64 bit | 1, 4, 8, 16, 32 lanes |
| Max Speed | 133 MB/s | 1 GB/s | 64 GB/s+ (Gen6 x16) |
| Sharing | Shared bus | Shared bus | Switched, dedicated |
| Signal Integrity | Poor at high freq | Better | Excellent |
| Hot Plug | Limited | Limited | Supported |
| Error Handling | Basic | Limited | Advanced (ACK/NAK, Retry) |
| Power Management | Minimal | Minimal | Advanced (ASPM, L-states) |
Key Takeaways:
- PCIe is serial and point-to-point, eliminating the shared bus bottleneck.
- Each device gets a dedicated link, scalable in width.
- PCIe adds features like flow control, QoS, interrupts, error reporting, and power states.
3. On-Chip vs Peripheral Protocols
On-Chip Protocols
Used inside SoCs for connecting modules like CPU, memory controller, or accelerators.
- Examples: AXI, AHB, OCP, TileLink
- Pros: Very fast, simple
- Cons: Limited to a single chip, no standard between devices
- Limitation: Not scalable for high-speed peripherals
Peripheral Protocols
Used to connect external devices to CPU/system.
- Examples: PCI, PCI-X, PCIe, USB, SATA
- Pros: Standardized, scalable, hot-plug capable
- PCIe: Industry standard for high-speed peripherals
ASCII Illustration: On-Chip vs Peripheral
[CPU]---AXI/AHB---[GPU Core]
|
+--PCIe-->[External GPU/SSD/FPGA]
Conclusion: On-chip protocols are great for internal SoC communication, but PCIe is required for external devices.
4. PCIe Topology
Unlike PCI/PCI-X (shared bus), PCIe uses a switched, point-to-point topology:
CPU / Root Complex
|
---------------------
| | |
x16 GPU x4 SSD x1 WiFi
|
Switch
/ | \
EP1 EP2 EP3
(NIC) (FPGA) (USB)
Notes:
- Root Complex (RC) connects CPU to PCIe devices.
- Switches extend connectivity to multiple endpoints.
- Each link is dedicated, preventing bandwidth sharing.
5. PCIe Layers Overview (Brief Intro)
PCIe uses a layered architecture:
| Layer | Purpose |
|---|---|
| Transaction Layer | Forms TLPs (Transaction Layer Packets), routing, addresses |
| Data Link Layer | Ensures reliable delivery using DLLPs (ACK/NAK), flow control |
| Physical Layer | Serial transmission, encoding, equalization, training |
Each layer has specific roles, which we’ll explore in detail in later parts.
6. PCIe Protocol Features (High-Level)
- Packet-based communication (TLP/DLLP)
- Switched, point-to-point links
- Lane scalability (x1–x16+)
- Quality of Service (QoS) support
- Flow control with credit mechanism
- Hot-plug & power management
- Error detection and recovery
- Supports multiple generations (Gen1–Gen6)
7. PCIe Generations (Speed Table)
| Gen | Year | Per-Lane Speed | Encoding | Max x16 Bandwidth |
|---|---|---|---|---|
| Gen1 | 2003 | 2.5 GT/s | 8b/10b | 4 GB/s |
| Gen2 | 2007 | 5 GT/s | 8b/10b | 8 GB/s |
| Gen3 | 2010 | 8 GT/s | 128b/130b | 16 GB/s |
| Gen4 | 2017 | 16 GT/s | 128b/130b | 32 GB/s |
| Gen5 | 2019 | 32 GT/s | 128b/130b | 64 GB/s |
| Gen6 | 2023 | 64 GT/s | 1b/1b Flit | 128 GB/s |
