Unicode Normalizer

What this tool is used for

• Making two strings that look identical actually compare equal.
• Normalising user input before storing it so lookups match reliably.
• Diagnosing why a filename does not match between two systems.
• Choosing between composed and decomposed forms for a database column.
• Checking what a compatibility normalisation does to a string before applying it.

Frequently Asked Questions

What is the difference between NFC and NFD?

How an accented letter is stored. NFC composes it into one code point (é = U+00E9); NFD decomposes it into a base letter plus a combining mark (e + U+0301). Both display identically, both are correct, and they are not equal as strings.

Which form should I store?

NFC, almost always — it is what the W3C recommends for the web, what most systems produce, and it is shorter. Normalise on input, once, at the boundary, so everything downstream can compare strings with `==` and be right.

What do the K forms do differently?

NFKC and NFKD apply *compatibility* mappings, which are lossy: fi becomes fi, ① becomes 1, ℌ becomes H, and a full-width Ａ becomes A. Useful for search indexing and username checks, destructive for anything you intend to display back.

Why does my filename not match on macOS?

Because HFS+ stored filenames in a decomposed form close to NFD while nearly everything else uses NFC. A file called "café" created on a Mac can fail to match the same name typed on Linux — the classic cause of a mysterious "file not found" in a build.

Why does the character count change?

Because decomposing adds code points: café is 4 in NFC and 5 in NFD, since the accent becomes its own character. What a reader sees is unchanged, which is why a length check should count grapheme clusters rather than code units.

Which normalisation form should I use?

NFC for storage and interchange — it is what the web platform assumes and keeps text compact. NFKC only when you deliberately want compatibility folding, since it rewrites characters such as ligatures and fullwidth forms.

Common errors and gotchas

• Using a compatibility form on data you must preserve exactly, since it changes characters rather than reordering them.
• Normalising a password before hashing on one system and not the other, which breaks every login.
• Assuming normalisation is idempotent across forms, so applying two in sequence is safe.
• Normalising a filename on a platform that stores a different form, which then mismatches.
• Treating normalisation as sanitisation, which it is not.