Dataset · vendor telemetry · two NetApp field studies

Low-end NetApp interconnect events exceed disk events, 4,338 to 3,230

In Jiang et al., FAST 2008, Table 1, low-end NetApp systems recorded 4,338 physical-interconnect failure events and 3,230 disk failure events from January 2004 through August 2007 (22,031 systems and 264,983 FC disks ever installed). For Figure 4(b), which excludes subsystems that used Disk H, that class has a stated disk AFR of 0.9% and a storage-subsystem AFR of 4.6%. In Maneas et al., FAST 2020, Table 2, SCSI error is 32.78% of SSD replacements (ARR 0.055%) on almost 1.4 million NetApp SSDs; that signal is a hardware error the SSD reports (the example is drive DRAM ECC), while aborted commands are 13.56% (ARR 0.023%) and include a host or connection case where write data never reaches the device.

File /data/datasets/what-fails-comparison.csv · JSON metadata · All datasets · afr · nearline-hdd · disk-shelf · enterprise-ssd

Observatory card: nearline-hdd

Method

The CSV transcribes stated figures. It does not digitize a chart and it does not refit a rate. Jiang et al. report NetApp AutoSupport logs from about 39,000 commercially deployed storage systems over 44 months, January 2004 through August 2007, with about 1,800,000 disks in about 155,000 shelf enclosures. Table 1 prints the class counts and the four failure-event counts. Those event rows are the jiang-table1 cut. The disk column is disks ever installed during the 44 months, including replacements, not a one-day census.

Figure 4(b) excludes subsystems that used the problematic Disk H family. The AFR cells are the figures written in that section, including Finding (2): near-line disk AFR 1.9%, near-line subsystem AFR 3.4%, low-end disk AFR 0.9%, low-end subsystem AFR 4.6%. Nearby sentences also say “about” for the 1.9%, 3.4%, and 4.6% figures. The low-end example says disks are about 20% of that subsystem AFR. Those four rows are jiang-figure4b. They are not Table 1 events, and this file does not divide 0.9 by 4.6.

The share rows are the cross-class prose ranges: disk failures 20–55% of storage-subsystem failures, physical interconnects including shelf enclosures 27–68%, protocol failures 5–10%, and performance failures 4–8%. They are not percentages of the Table 1 event counts. Annual rate is AFR only on the Figure 4(b) rows. ARR is a different definition, failures divided by device-years, and appears only on the Maneas rows.

Maneas et al. use ten NetApp Active IQ snapshots (previously AutoSupport), January and June 2017, January, May, August, and December 2018, and February, March, April, and May 2019: almost 1.4 million enterprise SSDs, three manufacturers, 18 models, over 30 months. Table 2 prints nine replacement reasons. The reason was missing for 40% of replacements, and the percent of replacements is normalized for that gap. SCSI error is a hardware error the SSD reports; the paper’s example is drive DRAM ECC. Aborted commands are the host-or-connection case. There is no line-rate or throughput column in this file.

Limits

• Population and date, HDD side: NetApp customer AutoSupport, four system classes, January 2004 through August 2007. Not an industry sample. The counts are the paper’s figures from that vendor telemetry, not a recount of the logs.
• Population and date, SSD side: a sample of NetApp Active IQ, almost 1.4 million SSDs, ten snapshots spanning January 2017 through May 2019 (the paper says 30 months). Not a daily failure log, and not the HDD population.
• Not measured here: mid-range and high-end point disk AFR, and mid-range and high-end subsystem AFR. The prose says disk AFR for low-end, mid-range, and high-end together is under 0.9%, and separately states the low-end point as 0.9%. This file does not pick a point for the other two classes. Figure 4(a), which still includes Disk H, is not transcribed.
• Not measured here: failures that never reach the RAID layer. The paper’s example is an interconnect failure recovered by SCSI retries or tolerated by multipathing. Higher-layer redundancy above the storage subsystem is outside the 2008 study.
• Not measured here: raw SSD replacement counts, a per-reason confidence interval, and the missing reason on 40% of SSD replacements. The bundles do not contain copies of customers’ data. Lost writes stay ambiguous between drive firmware and another layer in the stack.
• Table 1 event shares are not the 20–55% or 27–68% ranges. Those ranges are the paper’s failure-composition prose. Do not treat the summed ARR cells as a published fleet ARR. Neither paper links a public dump.

Counted here

From the Table 1 rows, not from the paper’s rounded “about” lines: 39,115 systems, 156,990 shelf enclosures, 1,819,423 disks ever installed. The paper says about 39,000 systems, about 155,000 shelf enclosures, and about 1,800,000 disks. This file does not force those sentences to equal the sums.

• near-line: 17,892 failure events across the four Table 1 types.
• low-end: 9,824 failure events across the four Table 1 types.
• mid-range: 21,296 failure events across the four Table 1 types.
• high-end: 17,364 failure events across the four Table 1 types.

Low-end physical-interconnect events (4,338) exceed low-end disk events (3,230). Near-line disk events are 10,105 against 4,888 physical-interconnect events, so the low-end ordering is not the ordering in every class.

The nine Table 2 share cells sum to 100. Predictive failures, threshold exceeded, and recommended failures sum to 34.44% of replacements. The paper calls that preventative group about one third and does not print 34.44. The nine ARR cells sum to 0.166. Table 2 does not print that sum, so this file does not call 0.166% the fleet annual replacement rate.

Comparison

Window, fleet, and population are columns in the CSV. Jiang rows use NetApp AutoSupport, 2004-01/2007-08. Maneas rows use NetApp Active IQ, 2017-01/2019-05. The population cell states which cut the numbers belong to.

Columns

row_kind (string)

Which stated cut the row transcribes: jiang-table1, jiang-figure4b, jiang-share, or maneas-table2.

study (string)

Citation id of the paper that states the row.

fleet (string)

Named fleet: NetApp AutoSupport or NetApp Active IQ.

window (string)

Study window as year-month bounds. Jiang is January 2004 through August 2007 (44 months). Maneas is the 30 months covered by 10 snapshots from January 2017 through May 2019.

system_class (string)

Jiang storage-system class (near-line, low-end, mid-range, high-end, or all four classes) or ssd sample.

population (string)

Who was counted, which cut (Table 1, Figure 4(b) excluding Disk H, cross-class prose, or Table 2), and what the count is not.

fault (string)

Named failure or replacement reason, using the paper's categories.

systems_count (integer)

Systems (count) from Jiang Table 1. Empty when the row is not a Table 1 class row. Not a Figure 4(b) population.

shelves_count (integer)

Shelf enclosures (count) from Jiang Table 1. Empty when the row is not a Table 1 class row.

disks_ever_installed_count (integer)

Disks ever installed (count) during the 44 months, including replacements, from Jiang Table 1. Empty when the row is not a Table 1 class row. Not the Maneas SSD population.

failure_events (integer)

Failure events (count) for that fault in Jiang Table 1. Empty when the source did not state an event count for the row.

share_low_percent (number)

Share, low (%). Jiang cross-class prose uses a range. The low-end Figure 4(b) disk row uses 20 for the paper's about-20% of subsystem AFR. Maneas Table 2 percent of replacements is a point, so low equals high. Empty when not stated.

share_high_percent (number)

Share, high (%). Equal to share_low_percent when the source stated a point rather than a range. Empty when not stated.

annual_rate_percent (number)

Annual rate (%). AFR for Jiang Figure 4(b) prose, or ARR for Maneas Table 2. Empty when the source did not state an annual rate for the row. Not a line rate and not a throughput.

annual_rate_kind (string)

AFR or ARR when annual_rate_percent is set. Empty otherwise. AFR and ARR are different definitions and are not converted into each other.

signal (string)

The log or telemetry signal the paper says catches this fault.

misses (string)

What that signal does not prove, from a sentence in the same paper.

citation_id (string)

Id into the citations list. Open that URL for the sentence.

License

Small derived table of figures stated in Jiang et al., FAST 2008, and Maneas et al., FAST 2020. Not a Creative Commons license. The opened FAST '14 Consent Form for Refereed Papers says USENIX lets authors retain copyright and asks only for a right to publish. That grant is non-exclusive, worldwide, perpetual, and irrevocable, and also non-sublicensable and non-transferable. The form contains no CC grant. This CSV is not the NetApp AutoSupport database and not the Active IQ bundles. Neither paper links a public dump or a data DOI for those logs. Canonical files are the USENIX PDFs.

FAST ’14 refereed-paper consent form · Canonical FAST 2008 PDF (AutoSupport logs are not in the file) · Canonical FAST 2020 PDF (Active IQ bundles are not in the file)

Sources

1. Are Disks the Dominant Contributor for Storage Failures? A Comprehensive Study of Storage Subsystem Failure Characteristics (accessed 2026-10-08)
2. A Study of SSD Reliability in Large Scale Enterprise Storage Deployments (accessed 2026-10-08)
3. FAST '14 Consent Form for Refereed Papers (accessed 2026-10-08)