A list of keywords exported from three different tools, a set of URLs pulled from two crawls, an email list merged from two sign-up forms — the same problem shows up constantly: a plain text list with repeats in it, and the repeats need to go before the list is useful. The tool above does exactly that, entirely in the browser.

The options, and why each exists

• Case-insensitive (default on). Apple and apple are treated as the same line — the common case for keyword lists and tags, where casing is usually accidental, not meaningful. Turn it off when case genuinely matters, like deduplicating code identifiers or IDs.
• Trim whitespace (default on). A trailing space or a stray tab makes two visually identical lines compare as different — trimming catches the duplicate anyway. Turn it off only when leading or trailing whitespace is meaningful in your data.
• Remove empty lines (default on). Blank lines from a copy-paste or an export are noise, not data, in almost every real use case.
• Sort A→Z (default off). Off by default to preserve the first-seen order of your original list — turn it on when you want an alphabetized result instead.

What counts as a duplicate

Only an exact line match (after the options above are applied) counts. “running shoes” and “running shoe” are different lines — this tool removes copies, not near-duplicates or plurals. For comparing whether two entire pages or paragraphs are near-duplicates of each other, rather than deduplicating a list, see the Duplicate Content Checker instead — a different problem with a different tool.

Common uses

• Cleaning a keyword list merged from several research tools before planning content around it.
• Deduplicating a URL list pulled from a crawl or an export before running it through a batch checker.
• Merging two email or contact lists without ending up with the same address twice.
• Cleaning up a CSV column pasted as plain text before re-importing it.

Frequently asked

Does this tool sort my list?

Not unless you turn Sort A→Z on. By default the original order is preserved, with duplicates simply dropped in place.

Can it handle a very large list?

Since everything runs in your browser rather than being uploaded, the practical limit is your browser’s own memory — comfortable for tens of thousands of lines, the way almost every real list in practice is sized.

Does it work on CSV files, not just plain lists?

It compares whole lines, so a CSV pasted as text is deduplicated row by row — useful when entire rows repeat, not when only one column does.