TL;DR / Summary: Text that looks identical on screen can hash differently when one copy uses CRLF and the other uses LF. Normalize newlines to a single convention before hashing configs, license files, or fixture payloads. Otherwise checksums disagree across Windows checkouts and Linux build agents for content that is logically the same.
The invisible byte that breaks equality
A line ending is either 0x0A or 0x0D0A. Editors hide that difference. Hash functions do not. Teams paste the same JSON into a ticket on Windows and into a container on Linux, then conclude the hashing tool is flaky when SHA-256 diverges.
This is not limited to source code. CSV exports, PEM files opened in Notepad, and README snippets copied from Slack all pick up local newline habits. If the hash is used as a cache key or a release attestation, newline drift becomes a product bug rather than a curiosity. Treat the digest as a contract over bytes, not over what the screen rendered.
Release managers should print byte length beside the digest in every promotion checklist. A length that jumps by the number of lines almost always means CRLF appeared. That one metric turns a vague “hash mismatch” into a one-minute diagnosis for people who do not live in hex dumps.
Choose LF unless a format demands otherwise
Most web and cloud tooling expects LF. Normalize to LF for JSON, YAML, TOML, and generic UTF-8 text unless a partner specification explicitly requires CRLF. For CSV destined to legacy Windows importers, you may keep CRLF—but then hash after that choice is applied, not before.
Document the rule next to the hash step: “UTF-8, LF newlines, no BOM.” Ambiguity here recreates the same incident every quarter. Include “exactly one trailing newline” or “no trailing newline” in the same note, because end-of-file blank lines are another human-invisible delta.
Git attributes are not normalize-on-hash
Working-tree conversion via .gitattributes helps humans, but blobs already stored, files uploaded through browsers, and payloads from APIs never see that conversion. The hash function in your service must normalize what it receives, not assume Git already did the job.
Likewise, “autocrlf was on my laptop” is not an architecture. Services should not depend on contributor settings for digest stability. Clipboard paste into vendor portals can reintroduce CRLF even when your repo is clean; download what the vendor stored and hash that artifact, not your local editor buffer.
A concrete mismatch walkthrough
A release script hashed policy.json before upload. Mac developers produced LF files. A Windows support engineer saved a hotfix in an editor that wrote CRLF. Support’s file failed the gateway’s digest check even though a visual diff tool showed no textual change. The gateway was correct; the bytes differed.
The repair was a shared normalize step: read bytes, decode UTF-8, replace CRLF and lone CR with LF, re-encode, then hash. After that, both editors produced the same digest. A line ending converter made the one-off repair easy; the permanent fix lived in the upload path so the next hotfix could not regress.
Minify, BOM, and binary-safe paths
Minifying JSON removes insignificant whitespace, which also removes newline variance—but only for JSON. Plain text licenses, CSV, and XML comments still need an explicit newline policy. Do not assume “we minify everything” covers non-JSON artifacts hanging off the same pipeline. If you both minify and hash, define order: parse, emit canonical JSON, then hash.
A UTF-8 BOM at the start changes the digest without changing visible glyphs in many editors. Keep “no BOM” beside the LF rule. Do not run newline replacement on zip files, PNG assets, or protobuf payloads. Gate normalization on content type. Keep two code paths—textNormalizeThenHash and rawHash—and make call sites choose explicitly.
Testing across operating systems
In CI, feed the same logical string once with LF and once with CRLF into the normalize-then-hash helper and expect one digest. Add a mixed-ending case. If the helper only replaces \r\n and leaves lone \r, old Mac-style files will still flake.
Store golden digests for normalized fixtures, not for raw editor output. Golden files checked into Git should already be LF so reviewers are not fighting noise. When a portal only accepts paste, normalize immediately before submit and save those normalized bytes as the release attachment CI will hash.
When a mismatch points past newlines
Teams that only compare digests without lengths also miss UTF-16 uploads saved from older Windows editors. If length is roughly double what you expect for the character count, you are not in a newline problem anymore—you are in an encoding problem—and normalize-on-LF will not save you until you decode with the right codec first.
After newlines match, remaining digest gaps often come from tab-versus-space indentation, Unicode normalization forms, or a silent reordering of object keys. Expand the debug view to a hex dump of the first differing offset. The first wrong byte usually names the next fix.
Keep a small internal page that runs normalize, shows length before and after, and prints the digest via your usual hash tool. Support staff can paste a suspect file and see whether newline cleanup alone resolves the alert before engineering opens a cryptography ticket. Publish the normalize rules next to that page so the ritual survives shift changes.
If legal text must not be mutated, refuse the upload when endings differ instead of auto-normalizing, and show both lengths. Auto-fix is fine for configs you own; warning-only is safer for contracts you merely attest. Choose the mode per artifact class in the same rules document.
Finally, teach support to ask “did you download the stored object and hash that?” before escalating to cryptography. Most tickets die at that question once the portal’s rewrite habits are known.