A common task when editing files is to remove duplicate lines. While this can be done manually, it
is often easier to leverage Vim's command-line mode to leverage the uniq command.
Suppose we have the following file in a buffer and want to remove duplicates:
The command to execute is:
:%!uniq
Let's break this down to review what is happening: To start, this follows the pattern:
:[range]!{cmd} {args}
where
:%
specifies the range as the entire file, although this can be replaced with a range specifying a subset of lines, or a visual selection. Next,
!{cmd}
indicates that an external command is to be called with the specified range and arguments. In this case the external command to be called is:
uniq
with (initially) no arguments. By default uniq takes input from stdin, so there is no need to
specify the input source.
Executing this command produces the following output:
which is not exactly what we were looking for. It turns out that by default uniq only detects adjacent repeating lines, so apple still appears twice in the output.
If we take a look at the help page for uniq we see that the -u option can be used to print only
unique lines, which is what we were looking for. Lets undo to get back to the original buffer,
then try again using the -u argument:
That is more like it. We should note that uniq also offers the -d option, which performs the
opposite function and returns only the duplicate lines. Starting from the original buffer we can
execute this command to see the result: