hesela.devRoCEv2
RoCEv2 carries InfiniBand transport packets over UDP/IP on Ethernet, enabling RDMA communication across IP-routed networks.
Encapsulation: RoCEv1 uses Ethernet framing without IP routing; RoCEv2 adds IP and UDP headers. Routability does not establish compatible endpoints or a latency bound. [1]
Reliability: NVMe/RDMA supports reliable queue-pair modes. UDP encapsulation does not change that binding into an unreliable storage protocol. [2]
Network policy: Cumulus Linux 5.12 documents ECN-only lossy mode and ECN-plus-PFC lossless mode. This is an implementation example, not a universal configuration rule or evidence that both modes perform equally. [3]
Failure containment: Azure's 2023 report describes a PFC watchdog that drops packets on a persistently paused queue to stop pause propagation. This deployment-specific policy is not a universal losslessness or recovery guarantee. [4]
Relations
- related
- RDMA storage networking, NVMe/RDMA, iWARP
Citations
- NVIDIA MLNX_OFED 23.10-3.2.2.0 LTS: RoCE modes
- NVM Express NVMe over RDMA Transport 1.3, section 2
- NVIDIA Cumulus Linux 5.12: RoCE lossy and lossless modes
- Bai et al., Empowering Azure Storage with RDMA, NSDI 2023, section 3.1
Sources & caveats
- Specification and versioned vendor-documentation review; no hardware benchmark, congestion experiment or interoperability test.
- Published deployment report used for failure-containment context; no network experiment reproduced.