hesela.devNVMe completion queue
On the NVMe PCIe path, a completion queue is a memory ring where a controller posts command results for host software.
A completion identifies its command by submission-queue ID and command ID. Completion order need not follow submission order. SQHD is a separate slot-consumption report, not a list of completed commands [1].
The phase bit distinguishes the expected traversal of the ring, not successful status. Linux v6.12 checks that bit, applies a DMA read barrier before handling the rest of the entry, and changes its expected phase when the completion head wraps [2].
A full completion queue prevents further status posting there. The controller may pause associated submission queues until slots are returned, but this condition must not stop unrelated submission queues [1].
Posting a completion entry is separate from notifying the host. An interrupt can prompt processing of several entries; polling can inspect the queue without waiting for that interrupt. Coalescing changes notification frequency, not completion identity [3].
Relations
- related
- NVMe, NVMe submission queue, NVMe doorbell, Queue depth, Interrupt coalescing, I/O completion polling, NVMe Phase Tag, NVMe Command Identifier
Citations
- NVM Express Base 2.0a, sections 2.1-2.2, 3.1.1, 3.3.1.2 and 3.3.3.2; copyright 2007-2021 NVM Express, Inc., all rights reserved
- Linux v6.12: NVMe PCIe queue submission, completion handling and doorbell-buffer checks
- NVM Express PCIe Transport 1.0d, sections 3.2, 3.4.1 and 3.5; copyright 2008-2023 NVM Express, Inc., all rights reserved
Sources & caveats
- The memory-ring description is PCIe-scoped; NVMe over Fabrics uses transport-specific messaging.
- Historical specification and pinned implementation references describe mechanisms, not current feature coverage or measured performance.