Thursday, November 10, 2016

duplicate lines in a file - command line


Lets say you have a file with some duplicate lines.
How do you figure out them using Linux command line utilities?


 For example, file contents as this:
~$ cat test.txt
line 1
line 2
line 3
line 4
line 5
line 5
line 6
line 7


Now use the command to get the count of duplicate lines.
so, "line 5" is repeated twice.

~$ sort test.txt  | uniq -c
      1 line 1
      1 line 2
      1 line 3
      1 line 4
      2 line 5
      1 line 6
      1 line 7

To get that specific line alone use -cd
~$ sort test.txt  | uniq -cd
      2 line 5

No comments:

Post a Comment