Copied text can look correct and still behave differently when it reaches a CMS, a search field or a prompt. Formatting markers, unusual whitespace and line breaks are separate problems, and each needs a different check.
I build TextScope, a free browser toolkit for checking and cleaning existing text. This is the checklist I use when moving text between tools. The checks are useful with any editor; the examples do not depend on TextScope.
1. Decide whether you want plain text or Markdown
**Release notes** is valid Markdown. In a plain text title field, the asterisks are visible characters. Removing them makes sense there. In a Markdown editor, keeping them may be the correct choice.
Start by identifying the destination. Keep a copy of the original before removing formatting, especially if it contains code blocks or links. Flattening a document can destroy useful structure.
2. Inspect suspicious whitespace before deleting it
Two strings may look the same while containing different characters. A non-breaking space, U+00A0, is different from the ordinary U+0020 space. A zero-width space, U+200B, may affect matching while remaining invisible.
This JavaScript snippet prints the code points in a short sample:
const sample = 'A\u00A0B';
console.table(Array.from(sample, character => ({
character,
codePoint: 'U+' + character.codePointAt(0).toString(16).toUpperCase().padStart(4, '0')
})));
This is a diagnostic, not a universal cleaner. Zero-width joiners can be necessary in emoji sequences and some writing systems. Do not remove every invisible character from every language. Invisible characters also do not prove that a passage was written by AI or contains a watermark.
3. Treat line wrapping separately from paragraph breaks
Text copied from a PDF may contain a newline at the end of every visual line. Text copied from a document may contain real paragraphs. Joining every newline can merge headings, list items and separate thoughts.
Review a short sample first. Join wrapped lines only when you can identify the intended paragraphs. Preserve a code sample's line breaks unless you know that reformatting is safe.
4. Choose the count that matches the limit
A word limit, a character limit and a token limit are different. JavaScript string length counts UTF-16 code units. It is not a count of visible characters. Some emoji use several code points; combining marks can make several code points appear as one character.
For a publishing form, use its own counter as the final authority. For a model prompt, a rough token estimate can help size a draft, but the model's tokenizer is the right source for exact counts. Include instructions and other context when checking the whole prompt.
5. Use readability scores as a clue
Flesch scores can help compare English drafts. They do not measure factual accuracy, usefulness or whether the reader understands a technical term. A short sentence full of unfamiliar vocabulary can still confuse a reader.
After checking a score, read the text aloud and ask whether each sentence has a clear subject and action. Keep technical terms when the reader needs them, and explain them in context.
6. Review the final output in its destination
Paste the cleaned text into a preview or draft. Check headings, links, punctuation and paragraph boundaries. A cleanup step is successful when the destination behaves as intended, not merely when a counter changes.
If you want a browser interface for these checks, TextScope combines word counts, readability, phrase frequency, Markdown and hidden-character cleanup, title checks and estimated LLM token counts. It does not require an account, and text processing runs locally in the browser. It checks and transforms existing text; it does not generate text or detect AI authorship.
Top comments (0)