hesela.devNumPy NPZ archive
NPZ is NumPy's ZIP-based container for storing multiple arrays as NPY members.
NpzFile loads array members lazily when a caller accesses them. Successful construction or access to one member is not evidence that another member was read and checked. [2] [3]
ZIP member checks and array semantics answer different questions. A member containing a changed NPY header can receive a valid new CRC when repackaged; the CRC does not identify the intended shape, units or coordinate mapping. [1] [5]
A preservation package needs both an expected inventory and checked bytes. BagIt 1.0 formalizes this distinction, but NPZ is not thereby a BagIt bag. Checksums alone do not authenticate the source. [4]
Relations
- related
- NumPy NPY format, ZIP archive, Archive member, Checksum
Limitations
- These notes do not certify hostile archives, duplicate names, pickle safety or resource limits.
- A trusted expected member inventory and application schema are separate from a successful selected read.
Citations
- NumPy format documentation: NPY and NPZ layout
- NumPy NpzFile API
- NumPy 2.5.4 pinned NpzFile implementation
- RFC 8493: BagIt completeness, validity and active-attack limits
- CPython 3.12 zipfile API
Sources & caveats
- Format/API documentation and NumPy commit a3d0ef1df045b858e68115001fdabc930bb3b68c.
- Controlled NumPy 2.5.4 / CPython 3.12.3 observations use synthetic numeric arrays and BytesIO, not physical media, mmap or a corruption-rate sample.
- Reproduction: https://hesela.dev/research/npy-interpretation/reproduction.zip