Object storage
Object storage organizes data as independently addressed objects with associated metadata, accessed through object identifiers or keys rather than a block-device interface.
An object API's guarantees depend on the provider and operation. In S3, an object key identifies an object within a bucket, with a version ID identifying a retained version when versioning is enabled. Distinguish the object's bytes, its ETag validator, explicitly named checksums, and any version ID when designing integrity checks or concurrent updates.
The namespace model also matters: S3 general-purpose buckets use a flat key space, whereas directory buckets organize keys hierarchically. A folder-like display is not a portable storage-interface guarantee. [8]
Relations
- related
- File storage, Block storage, Cold archive tier, Air-gapped backup, Reed–Solomon codes (storage), Entity tag, Multipart upload, Conditional write, Object versioning, Object version ID, Key-value SSD, Object key, Object key prefix, Media type
Advantages
- Objects package data with descriptive metadata and can be addressed by bucket and key, with optional version-specific retrieval in S3. [2]
- S3 supports multipart transfers and conditional publication, allowing large-object transfer workflows to combine independent part uploads with a destination-object precondition. [3, 4]
Limitations
- Object-storage interfaces do not imply identical consistency guarantees across providers; S3's strong consistency and single-key atomic updates still do not provide a multi-key transaction. [2, 7]
- An S3 ETag is not a universal full-object checksum: multipart construction and encryption affect its meaning, and metadata-only changes are not reflected in it. [5]
- S3 versioning preserves versions for recovery but does not reject a stale application update; preventing lost updates requires an appropriate conditional-write protocol or other coordination. [4, 6, 7]
Citations
- Amazon S3: Objects, keys, version IDs, and data consistency
- Amazon S3: Uploading and copying objects using multipart upload
- Amazon S3: How to prevent object overwrites with conditional writes
- Amazon S3 API Reference: Object, ETag semantics
- Amazon S3: Retaining multiple versions of objects with S3 Versioning
- Object ETags: checksums and conditional writes are different
- Amazon S3: Directory buckets and hierarchical namespaces