Retry amplification
Retry amplification is the additional request-processing load created by repeated attempts to complete logical operations.
Delayed replies can induce more attempts and further delay, creating a sustaining feedback loop. This is distinct from NAND write amplification. [1]
Repeated storage-server membership queries can saturate the metadata service that supplies them. This is a service recovery mechanism, not evidence that every retry causes an outage. [2]
A chain of retrying callers can multiply total attempts. Count the initial attempt separately from retries and name the layer where attempts are measured. A per-operation limit differs from an aggregate retry budget. [3]
Relations
Citations
- Bronson et al. Metastable Failures in Distributed Systems. HotOS 2021; DOI 10.1145/3458336.3465286, sections 1-3.
- AWS. DynamoDB service disruption and related impacts in US-East: 20 September 2015 incident.
- Mike Ulrich. Google SRE, chapter 22: Addressing Cascading Failures, retry guidance.
Sources & caveats
- The incident evidence is AWS's account of the 20 September 2015 DynamoDB disruption. [2]