Duplicate Line Counter
Count how many times each line appears — sorted by frequency.
Lines: 0Unique Lines: 0Duplicate Lines: 0
Finding and counting duplicate lines
This tool groups every line and shows a count — handy for auditing log files, word lists, or CSV data where repeated entries indicate duplicates. Lines are sorted highest-to-lowest frequency so the most common entries surface immediately. To remove duplicates instead of counting them, use Remove Duplicate Lines. For word-level frequency analysis, try the Word Frequency Counter.
Built and maintained by Meet Shah · Last updated
What this tool is used for
- Finding which log message repeats most often.
- Counting how many times each value appears in a pasted list.
- Identifying duplicates before deduplicating so you know what you are losing.
- Spotting an unexpectedly frequent entry in an export.
- Producing a frequency table from a column of values.
Frequently Asked Questions
- Is the comparison case-sensitive?
- By default yes, so 'Apple' and 'apple' count as different lines. Whether that is right depends entirely on your data — email addresses should usually fold case, while identifiers and code should not.
- Does whitespace affect matching?
- Significantly, and invisibly. Two lines differing only by a trailing space are different strings, so they count separately while looking identical. Trimming before comparison is almost always what you want.
- How does this differ from removing duplicates?
- This REPORTS counts, keeping every line and telling you how many times each appears. Removal discards repeats. Counting first is the safer order — you find out what you would have deleted before deleting it.
- How does it compare to uniq?
- Unix uniq only collapses ADJACENT duplicates, which is why it is almost always used as `sort | uniq -c`. This counts across the whole input regardless of position, so no pre-sorting is needed.
- What is it useful for?
- Finding repeated entries in log files, spotting duplicate rows in exported data, checking for repeated keys in config, and quantifying how much of a list is redundant before deduplicating it.
- Are trailing carriage returns a problem?
- Yes, and a common one. A file mixing CRLF and LF endings produces lines that look identical and differ by one invisible character, so genuine duplicates are reported as unique. Normalise line endings first.
- Does the order of first appearance matter?
- It is preserved in the report, which makes the output diffable against the input. Sorting by count is more useful for finding the worst offender; sorting by position is more useful for editing the file.
- How does this scale to a very large file?
- Counting is linear and memory holds one entry per DISTINCT line, so a large file of mostly-repeated lines is cheap. A large file of mostly-unique lines is where memory becomes the constraint.
Common errors and gotchas
- Not deciding on case sensitivity, which splits or merges counts unexpectedly.
- Overlooking trailing whitespace, which makes identical-looking lines distinct.
- Counting blank lines as an entry, which usually dominates the result.
- Assuming line order is preserved in the output, when it is sorted by frequency.
- Treating a high count as an anomaly when the data is legitimately repetitive.
Related Text Tools tools
Case Converter
Convert between Sentence, Title, camelCase, snake_case, and more.
Word Counter
Count words, characters, and analyze keyword density.
Text to Binary
Convert text into its 8-bit binary representation.
Binary to Text
Decode 8-bit binary back into readable text.
Text Reverser
Reverse text by characters, words, or lines.
Remove Duplicate Lines
Delete repeated lines and keep your list unique.
Remove Empty Lines
Strip blank and whitespace-only lines from text.
Sort Lines Alphabetically
Sort lines A–Z or Z–A, case-sensitive or not.