Vim Regular Expressions

Regular expressions appear to be rapidly gaining in popularity among VIM users as they discover the sheer programming power that regular expressions can provide. Historically, regular expressions have been associated with the UNIX platform and scripting languages like Perl (Practical Extraction and Report Language).

The syntax in VIM is slightly different then in Perl, but is pretty close.  This makes Perl regular expression examples relevant to VIM users.

Softpanorama RegEx page contain basic information about regular expressions.  Among the differences between Perl and Vim  we can note (different meta characters are in yellow

#	Metacharater in Perl	#	Metacharacter in VIM
.	any character except new line
\s	whitespace character	\S	non-whitespace character
\d	digit	\D	non-digit
\x	hex digit	\X	non-hex digit
\o	octal digit	\O	non-octal digit
\h	head of word character (a,b,c...z,A,B,C...Z and _)	\H	non-head of word character
\p	printable character	\P	like \p, but excluding digits
\w	word character	\W	non-word character
\a	alphabetic character	\A	non-alphabetic character
\l	lowercase character	\L	non-lowercase character
\u	uppercase character	\U	non-uppercase character

Many special characters need to be escaped. For example:

\+  matches 1 or more of the preceding characters...

\{n,m} matches from n to m of the preceding characters...

\= is used instead of \? (matches 0 or 1 more of the preceding characters)

Quantifier	Description
*	matches 0 or more of the preceding characters, ranges or metacharacters .* matches everything including empty line
\+	matches 1 or more of the preceding characters...
\=	matches 0 or 1 more of the preceding characters...
\{n,m}	matches from n to m of the preceding characters...
\{n}	matches exactly n times of the preceding characters...
\{,m}	matches at most m (from 0 to m) of the preceding characters...
\{n,}	matches at least n of of the preceding characters...

Alternatives (OR) need to be escaped

Using "\|" you can combine several expressions into one which matches any of its components. The first one matched will be used.

\(Date:\|Subject:\|From:\)\(\s.*\)

will parse various mail headings and their contents into \1 and \2, respectively. The thing to remember about VIM alternation that it is not greedy. It won't search for the longest possible match, it will use the first that matched. That means that the order of the items in the alternation is important!

Tip 3:  Quick mapping to put \(\) in your pattern string

cmap ;\ \(\)<Left><Left>

Non-greed modifiers are different and more obscure then in Perl.  Perl allows you to convert any quantifier into a non-greedy version by adding an extra ? after it. So *? is a non-greedy version of a special character *

Quantifier	Description
\{-}	matches 0 or more of the preceding atom, as few as possible
\{-n,m}	matches 1 or more of the preceding characters...
\{-n,}	matches at lease or more of the preceding characters...
\{-,m}	matches 1 or more of the preceding characters...

Replacement rules are different

You can group parts of the pattern expression enclosing them with "\(" and "\)" and refer to them inside the replacement pattern by their special number \1, \2 ... \9. Typical example is swapping first two words of the line:

s:\(\w\+\)\(\s\+\)\(\w\+\):\3\2\1:

where \1 holds the first word, \2 - any number of spaces or tabs in between and \3 - the second word. How to decide what number holds what pair of \(\) ? - count opening "\(" from the left.

Replacement part of the S&R has its own special characters which we are going to use to fix grammar:

#	Meaning	#	Meaning
&	the whole matched pattern	\L	the following characters are made lowercase
\0	the whole matched pattern	\U	the following characters are made uppercase
\1	the matched pattern in the first pair of \(\)	\E	end of \U and \L
\2	the matched pattern in the second pair of \(\)	\e	end of \U and \L
...	...	\r	split line in two at this point
\9	the matched pattern in the ninth pair of \(\)	\l	next character made lowercase
~	the previous substitute string	\u	next character made uppercase

Now the full S&R to correct non-capital words at the beginning of the sentences looks like

s:\([.!?]\)\s\+\([a-z]\):\1  \u\2:g

We have corrected our grammar and as an extra job we replaced variable number of spaces between punctuation and the first letter of the next sentence with exactly two spaces.

• Perl supports a more options that can be appended to the regexp, or even embedded in it.
• You can also embed variable names in a Perl regular expression. Perl replaces the name with its value; this is called "variable interpolation".

The most common task is to make replacements in a text following some certain rules using VIM search and replace command (S&R) :s(substitute). For example here is how globally replace all occurrences of vi with VIM.

%s/1999/2003/g

This is a very common idiom in vi/vim.  Like in Perl you can also use several modifiers

• c  -- Confirm each substitution
• g  -- Replace all occurrences in the line (without g - only first).
• i  -- Ignore case for the pattern.
• I -- Don't ignore case for the pattern.

Using Anchors

Adapted from volontir Vim regular expression tutorial

Suppose you want to replace all occurrences of vi with VIM. This can be easily done with

s/vi/VIM/g

If you've tried this example then you, no doubt, noticed that VIM replaced all occurrences of vi even if it's a part of the word (e.g. navigator). If we want to be more specific and replace only whole words vi then we need to correct our pattern. We may rewrite it by putting spaces around vi:

s: vi : VIM :g

But it will still miss vi followed by the punctuation or at the end of the line/file. The right way is to put special word boundary symbols "\<" and "\>" around vi.

s:\<vi\>:VIM:g

The beginning and the end of the line have their own special anchors - "^" and "$", respectively. So, for all vi only at the start of the line:

s:^vi\>:VIM:

To match the lines where vi is the only word:

s:^vi$:VIM:

Now suppose you want to replace not only all vi but also Vi and VI. There are several ways to do this:

• probably the simplest way is to put "i" - ignore case in a pattern %s:vi:VIM:gi
• define a class of characters. This is a sequence of characters enclosed by square brackets "[" and "]". It matches any character from this set. So :%s:[Vv]i:VIM: will match vi and Vi. More on character ranges in the following section.

Examples

Adapted from volontir Vim regular expression tutorial

Some examples of :g usage:

:g/^$/ d

- delete all empty lines in a file

:g/^$/,/./-j

- reduce multiple blank lines to a single blank

:10,20g/^/ mo 10

- reverse the order of the lines starting from the line 10 up to the line 20.

Here is a modified example from Walter Zintz vi tutorial:

:'a,'b g/^Error/ . w >> errors.txt

- in the text block marked by 'a and 'b find all the lines starting with Error and copy (append) them to "errors.txt" file. Note:  . (current line address) in front of the w is very important, omitting it will cause :write to write the whole file to "errors.txt" for every Error line found.

You can give multiple commands after :global using "|" as a separator. If you want to use "|' in an argument, precede it with "\'. Another example from Zintz tutorial:

:g/^Error:/ copy $ | s /Error/copy of the error/

- will copy all Error line to the end of the file and then make a substitution in the copied line. Without giving the line address :s will operate on the current line, which is the newly copied line.

:g/^Error:/ s /Error/copy of the error/ | copy $

- here the order is reversed: first modify the string then copy to the end.

NEWS CONTENTS

• 210210 : volontir Vim regular expression tutorial ( volontir Vim regular expression tutorial, )
• 210210 : Vim Regular Expressions ( Vim Regular Expressions, )
• 210210 : A Tao of Regular Expressions ( A Tao of Regular Expressions, )

Old News ;-)

volontir Vim regular expression tutorial

Some examples of :g usage:

:g/^$/ d

- delete all empty lines in a file

:g/^$/,/./-j

- reduce multiple blank lines to a single blank

:10,20g/^/ mo 10

- reverse the order of the lines starting from the line 10 up to the line 20.

Here is a modified example from Walter Zintz vi tutorial:

:'a,'b g/^Error/ . w >> errors.txt

- in the text block marked by 'a and 'b find all the lines starting with Error and copy (append) them to "errors.txt" file. Note: . (current line address) in front of the w is very important, omitting it will cause :write to write the whole file to "errors.txt" for every Error line found.

You can give multiple commands after :global using "|" as a separator. If you want to use "|' in an argument, precede it with "\'. Another example from Zintz tutorial:

:g/^Error:/ copy $ | s /Error/copy of the error/

- will copy all Error line to the end of the file and then make a substitution in the copied line. Without giving the line address :s will operate on the current line, which is the newly copied line.

:g/^Error:/ s /Error/copy of the error/ | copy $

- here the order is reversed: first modify the string then copy to the end.

Vim Regular Expressions

I started this tutorial for one simple reason - I like regular expressions. Nothing compares to the satisfaction from a well-crafted regexp which does exactly what you wanted it to do :-).

And yes, I have a life too. I hope it's passable as a foreword. Feel free to send me your comments, corrections and suggestions concerning this tutorial.

A Tao of Regular Expressions

A regular expression is a formula for matching strings that follow some pattern. Many people are afraid to use them because they can look confusing and complicated. Unfortunately, nothing in this write up can change that. However, I have found that with a bit of practice, it's pretty easy to write these complicated expressions. Plus, once you get the hang of them, you can reduce hours of laborious and error-prone text editing down to minutes or seconds. Regular expressions are supported by many text editors, class libraries such as Rogue Wave's Tools.h++, scripting tools such as awk, grep, sed, and increasingly in interactive development environments such as Microsoft's Visual C++.

Regular expressions usage is explained by examples in the sections that follow. Most examples are presented as vi substitution commands or as grep file search commands, but they are representative examples and the concepts can be applied in the use of tools such as sed, awk, perl and other programs that support regular expressions. Have a look at Regular Expressions In Various Tools for examples of regular expression usage in other tools. A short explanation of vi's substitution command and syntax is provided at the end of this document.

Recommended Links

Books

• "Learning the vi Editor" by Linda Lamb and Arnold Robbins.
• "vi Improved - VIM" by Steve Oualline (online book)
• See more

***** volontir Vim regular expression tutorial by Oleg Raisky. A very nice tutorial with some useful examples

VimRegEx.vim - Regular Expression Developer for Vim vim online

created by Dave Silvia

script type

utility

description

Start with one of the commands

:Vimrex
or
:VimRegEx

The first opens within the current (g)vim session, the second starts a new (g)vim

'z?' or the menu 'Usage' button opens a usage buffer.

VimRegEx.vim is a plugin that aids in development/construction/analysis of Vim regular expressions. It is not a search utility, although it certainly does that. It is a development/learning environment that allows writing Vim regular expressions, element-by-element analysis of these expressions, and execution of the expressions against source text with the results illustrated both verbally in text and graphically in different highlighting for different types of regular expression atoms.

It allows you to see, in verbal/graphical analytical terms, just what is being searched for and what will result with a given Vim regular expression.

There are comparable tools on the net for other environments (notably .NET, Perl, and Python), but nothing exists for Vim ... until now!

The plugin is written specifically for Vim, so you can build your regular expressions as you would in Vim. In other tools, you'd have to make allowances for backslash escaping specific elements (like '(', ')', '+', '?', etc.) to use what you had developed in those tools. You'd also have to make substitutions, like '\<' for '\b' for word boundary ('\b' has its own meaning in Vim), and '\(...\)\@=' for '(?=...)'. This renders these other tools all but useless for Vim.

VimRegEx has its own builtin documentation, a rich set of variables for tuning its performance/appearance to your liking in your vimrc, an easy menu driven interface for gvim as well as easy to use keyboard mappings for the same commands in vim. There's also file generation capability to save your windows in text or html.

If you've been struggling with a regular expression and can't seem to get it to do what you want, maybe what you think you're doing isn't what Vim sees. See what Vim sees in VimRegEx. In the VimRegEx enviroment, what you see is what you get!

http://www.vim.org/scripts/script.php?script_id=1056

place both in your .vim|vimfiles plugin dir

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: July 28, 2019