TL;DR / Summary: A UTF-8 byte order mark at the start of a config file is invisible in many editors yet fatal to strict JSON and YAML parsers. Strip the BOM before parse, fix the exporter that added it, and add a CI check so Notepad-saved secrets stop breaking deploys.
What the BOM actually is
UTF-8 does not need a byte order mark, but some Windows tools still prepend the bytes EF BB BF. On screen the file looks identical. At parse time the first key becomes \ufeffapiVersion or the JSON tokenizer rejects the document as invalid unexpectedly near line 1.
YAML libraries vary: some ignore the BOM, others treat it as content. Relying on “it worked on my machine” hides the hazard until a stricter runner appears.
Symptoms that point to a BOM
Classic clues include parsers complaining about unexpected character at position 0, Kubernetes rejecting a manifest that validates in a web linter after a re-paste, and diff tools showing no visible change while hashes differ by three bytes.
Hex-dump the first eight bytes. If you see ef bb bf 7b before a JSON object brace, you found the culprit. Removing those three bytes restores a clean document.
Strip locally, then fix the source
Emergency fix: open the file in a tool that can rewrite UTF-8 without signature, or strip the prefix in a small script before parse. For a one-off paste into a ticket, run the text through a BOM-aware cleaner on Masters Tool, then re-copy the result.
Permanent fix: stop the exporter from writing a BOM. In .NET, prefer encodings that omit the preamble. In editors, set “UTF-8” not “UTF-8 with BOM” as the default for repo file types.
CI should refuse signed UTF-8
Add a cheap check that fails when any *.json, *.yml, or *.yaml in config paths starts with EF BB BF. Failing early is kinder than debugging a production ConfigMap that “looks fine.”
JSON vs YAML vs dotenv edges
JSON is unforgiving: a BOM before { is not whitespace in older parsers. YAML 1.1/1.2 libraries differ on whether a BOM is allowed. Dotenv loaders may include the BOM in the first key name, producing lookups that miss DATABASE_URL because the real key is \ufeffDATABASE_URL.
Treat each format explicitly in your strip helper. A shared “trim weird whitespace” function that only removes ASCII spaces will not catch a BOM.
Editors and copy-paste pipelines
Notepad historically wrote BOM-marked UTF-8. Spreadsheet exports and some email clients introduce the mark when “helpful.” Any pipeline that round-trips config through Office tools needs a sanitize step before the file re-enters git.
When reviewing a PR that only “touches whitespace,” inspect the binary prefix. Three invisible bytes are a real change.
Library defaults worth pinning
Document which HTTP clients or SDK serializers emit BOM-marked bodies. If a partner API sends BOM-prefixed JSON, strip on ingest at the edge rather than sprinkling try/except around every parse call.
Unit-test your strip function with fixtures that include BOM and fixtures that do not. Both must parse to identical structures.
Operational playbook for the next outage.
When a config suddenly fails after an innocent edit, hex-dump first, ask about editor settings second, blame the schema third. That order saves hours. Keep a known-good canonical copy in the runbook and a one-liner to compare prefixes.
After recovery, file a ticket to encode the CI guard. Outages that only teach one person are scheduled to recur.
Git attributes and checkout on Windows.
Working trees on Windows are more likely to reintroduce BOMs when someone saves with the wrong encoding. Add a note to CONTRIBUTING that config files must be UTF-8 without signature, and consider a pre-commit hook that rejects BOM prefixes. Hooks catch the problem before the PR, which is cheaper than a failed deploy.
If a colleague must use an editor that defaults to BOM, give them a project-level settings snippet that overrides the default for this repository only.
HTTP APIs that echo BOM-prefixed bodies.
Partners who export “JSON” from Excel-related pipelines sometimes POST bodies that start with a BOM. Strip at your API edge and log a metric when you do, so you can show the partner evidence. Silently accepting forever trains them never to fix the exporter.
Return a clear 400 with a message naming the BOM when the client is under your control; be more tolerant at the edge when a critical vendor cannot change quickly—but still meter the tolerance.
Container images and baked-in config.
Config copied into Docker images during build can freeze a BOM forever if the build context file was saved incorrectly. Rebuilds will keep failing until the source file in git is fixed—editing the running container is a temporary mirage. Add the BOM check to the image build job as well as to the app repo so baked configs cannot bypass developer hooks.
When debugging, od -An -tx1 or a PowerShell format-hex on the first bytes inside the container confirms whether the image layer still carries the mark.
Clipboard paths from chat apps.
Paste from some chat and ticket systems prepends invisible characters beyond BOM, including zero-width spaces. Your stripper should target the UTF-8 BOM specifically and optionally a small allowlist of other known junk, without aggressively deleting legitimate leading whitespace in YAML block scalars. Test fixtures should include a BOM case and a zero-width-space case as separate expectations.
Document for support staff: “re-type the first key” is a valid emergency workaround when paste is contaminated.
Treat the BOM as noise, then parse with confidence
Invisible prefixes are still bytes. Strip UTF-8 BOMs before JSON or YAML parse, ban them in CI, and correct exporters at the source. A quick sanitize pass—whether in your pipeline or via online text utilities—turns mysterious position-zero errors into a boring, solved class of incident.