{"id":50,"date":"2026-10-07T17:11:00","date_gmt":"2026-10-07T17:11:00","guid":{"rendered":"https:\/\/www.masters-tool.com\/blog\/text-encoding-basics-before-you-convert\/"},"modified":"2026-10-08T03:04:42","modified_gmt":"2026-10-08T03:04:42","slug":"text-encoding-basics-before-you-convert","status":"publish","type":"post","link":"https:\/\/www.masters-tool.com\/blog\/text-encoding-basics-before-you-convert\/","title":{"rendered":"Text Encoding Basics Before You Convert"},"content":{"rendered":"

TL;DR:<\/strong> Text encoding is the agreement about which bytes mean which characters. UTF-8, legacy Chinese encodings such as GBK, and HTML entities solve different problems. Convert only when you know both the encoding you have and the encoding the destination expects.<\/p>\n

Bytes are not the letters you see<\/h2>\n

A text file is a sequence of bytes plus an assumption about how to read them. UTF-8 is the assumption most of the web uses now. Older Chinese pages and some desktop software still use GBK or GB2312. If you open GBK bytes as if they were UTF-8, you do not get a mildly misspelled sentence. You get replacement characters, question marks, or a string of Latin letters that never formed a word. The file did not \u201cget corrupted\u201d in a vague sense. It was interpreted with the wrong map.<\/p>\n

Masters Tool\u2019s reference tables, including ASCII and GBK listings, are maps you consult. They do not change your file. The encoding conversion tools, such as HTML entity encoding, change a string from one written form to another. Use a table when you are asking \u201cwhat is this code.\u201d Use a converter when you are asking \u201crewrite this string so the next system accepts it.\u201d<\/p>\n

ASCII, entities, and the characters beyond them<\/h2>\n

ASCII covers a small set: English letters, digits, and basic punctuation. The moment a sentence includes an accent, a curly quote, a degree sign, or any Chinese character, you are outside ASCII. HTML entities exist so a character that would break a markup file can be written with plain ASCII anyway, like a named or numeric reference. That is a convenience for HTML. It is not a general encoding for storing documents. If you entity-encode a paragraph and paste it into a place that expected raw UTF-8, the reader will see the entity codes instead of the letters.<\/p>\n

Curly quotes and non-breaking spaces are the everyday version of this pain. A word processor inserts them. A strict code tool or a CSV parser chokes, or it splits a field in a place you did not intend. Converting those characters to plain quotes and ordinary spaces is a reasonable cleaning step. Converting an entire multilingual paragraph to ASCII by dropping \u201cunsupported\u201d letters is not cleaning. It deletes the language.<\/p>\n

When a conversion is actually called for<\/h2>\n

Convert when a destination has told you its requirement. A stylesheet or an HTML attribute might need a URL-encoded or entity-encoded fragment. A legacy importer might demand GBK. A programming literal might need escape sequences. In each case you should be able to name the source form and the target form. \u201cMake this work\u201d is not a target. If you do not know the target, look at a file that destination already accepts, and match that, or read the error. Guessing UTF-8 when the importer wanted a legacy code page will fail in a way that looks like random punctuation.<\/p>\n

Do not convert twice. Entity-encoding a string that is already entity-encoded turns an ampersand into a second layer of codes, and the page will display the codes rather than the character. Base64, which belongs to binary-as-text rather than to character sets, has the same double-wrap failure: encoding an already encoded blob produces a longer string that decodes to the previous string, not to the original file. If you are unsure, decode once in a scratch note and see whether the result is readable text or a sensible file header.<\/p>\n

Separators and sample lines<\/h2>\n