Duplicate Line Remover — Paste your list. Remove duplicates instantly.

A duplicate line remover reads a block of text line by line, identifies lines that appear more than once, and removes all but the first occurrence of each. The result is a list where every line is unique. Options like case-insensitive matching and whitespace trimming control how “duplicate” is defined — for example, whether “Apple” and “apple” count as the same line, or whether a line with a trailing space is treated as a copy of the same line without one.

Yes, by default. When “Sort A–Z” is off, the tool preserves the order of first occurrences from the input. The first time a line appears, it is kept; every subsequent identical line is removed. The relative order of the surviving lines matches their order in the original input. If you need alphabetical output, enable the “Sort A–Z” option before processing.

With case-insensitive matching (the default), lines are compared after converting them to lowercase. “Apple”, “apple”, and “APPLE” are all treated as the same line, and only the first one encountered is kept. With case-sensitive matching, those three are treated as three distinct lines and all three are kept. Use case-insensitive matching for most natural language lists. Use case-sensitive matching for code, file paths, passwords, or any data where capitalisation carries meaning.

When you copy data from spreadsheets, word processors, or web pages, lines often arrive with invisible leading or trailing spaces. Without trimming, “apple” and “apple ” (with a trailing space) are treated as different lines and both survive deduplication — even though they look identical on screen. With whitespace trimming on, both are normalised to “apple” before comparison, and only one is kept. Trimming is on by default for this reason.

There is no hard limit. The deduplication algorithm uses a JavaScript Set for lookup, which operates in constant time regardless of list size. In practice, lists of 100,000 lines process in under a second in a modern browser. If you are working with extremely large datasets (millions of lines), a command-line tool or database query will be faster — but for the list sizes most people paste into a browser, this tool handles it without issue.