Unicode normalization
Unicode normalization transforms Unicode text into a specified form so that canonically or compatibility-equivalent strings can be compared consistently.
Forms: NFD applies canonical decomposition; NFC additionally applies canonical composition. NFKD and NFKC use compatibility decomposition, with composition for NFKC. Compatibility normalization can remove distinctions that matter to an application: U+2460 becomes U+0031 under NFKC but not NFC. [1]
Boundaries: Normalizing fragments independently does not guarantee that their concatenation is normalized. U+0065 and U+0301 are each NFC, but their concatenation becomes U+00E9 when normalized together. Recheck the assembled text or use a normalization-aware boundary algorithm. [2]
Storage: Normalization is not case folding, visual-confusable detection or a universal filename-equality rule. It may change bytes and collapse distinct original names under an application's mapping. Keep original names and validate the actual destination comparison policy when restoring an archive. [3]
Relations
Limitations
- The selected form and Unicode version must be explicit; normalization does not preserve the original code-point sequence or encoded bytes in every case.
Citations
- Unicode Standard Annex 15 revision 57: sections 1.1 and 1.2, equivalence and normalization forms
- Unicode Standard Annex 15 revision 57: section 1.4, concatenation of normalized strings
- Hesela: filename identity and backup restores
Sources & caveats
- Pinned to Unicode 17.0.0 / UAX 15 revision 57. Exact small examples use the runtime's Unicode database; the executable probe reports Node, ICU and Unicode versions separately.