character-encoding · shipping · also Text encoding

Character encoding

A character encoding specifies how text is represented using coded values and their storage or interchange representation.

Layers: Unicode distinguishes code points, encoding forms that map scalar values to code units, and encoding schemes that serialize code units as bytes. UTF-8, UTF-16 and UTF-32 are different representations of Unicode text. A code point, code unit and byte are not interchangeable counting units. [1]

Counting: UTF-8 stores U+0041 in one byte and U+20AC in three. A user-perceived character can contain several code points, so neither byte count nor code-point count is a universal display-character count. Specify the unit when sizing a text field or comparing names. [2] [3]

Recovery: Record the encoding and decoding-error policy with text-bearing metadata. Replacing undecodable input can discard distinctions in the original bytes; a readable display string is not automatically a reversible filename identifier. Keep the original representation when byte identity is required. [4]

Relations

related
ASCII, UTF-32, UTF-8, Unicode normalization, Byte, Byte order, Filename collision, Unicode scalar value, Grapheme cluster, ISO-8859-1 (Latin-1), Windows-1252, Comma-separated values (CSV)

Limitations

  • An encoding label does not establish Unicode normalization, case-insensitive equality or a filesystem's filename comparison policy.

Citations

  1. Unicode 17.0.0: chapter 3, sections 3.9 and 3.10, encoding forms and schemes (accessed 2026-11-17)
  2. RFC 3629: section 3, UTF-8 definition (accessed 2026-11-17)
  3. Unicode Standard Annex 29 revision 47: grapheme cluster boundaries (accessed 2026-11-17)
  4. Hesela: filename identity and backup restores (accessed 2026-11-17)
Sources & caveats
  • Source-reviewed terminology with constructed text examples, not storage-device measurements.

JSON: /data/terms/character-encoding.json

Published by Hesela team · Updated

Report a correction · Editorial standards · Reuse guidance

Cite this page
Hesela team. Character encoding. Hesela. 2026-11-21. https://hesela.dev/terms/character-encoding/