Syntax of Regular Expressions
Regular expressions have a syntax in which a few characters are
special constructs and the rest are ordinary. An ordinary
character is a simple regular expression which matches that same
character and nothing else. The special characters are $,
^, ., *, +, ?, [, ] and
\. Any other character appearing in a regular expression is
ordinary, unless a \ precedes it. (When you use regular
expressions in a Lisp program, each \ must be doubled, see the
example near the end of this section.)
For example, f is not a special character, so it is ordinary, and
therefore f is a regular expression that matches the string
f and no other string. (It does not match the string
ff.) Likewise, o is a regular expression that matches
only o. (When case distinctions are being ignored, these regexps
also match F and O, but we consider this a generalization
of "the same string," rather than an exception.)
Any two regular expressions a and b can be concatenated. The
result is a regular expression which matches a string if a matches
some amount of the beginning of that string and b matches the rest of
the string.
As a simple example, we can concatenate the regular expressions f
and o to get the regular expression fo, which matches only
the string fo. Still trivial. To do something nontrivial, you
need to use one of the special characters. Here is a list of them.
- . (Period)
- is a special character that matches any single character except a newline.
Using concatenation, we can make regular expressions like
a.b, which
matches any three-character string that begins with a and ends with
b.
- *
- is not a construct by itself; it is a postfix operator that means to
match the preceding regular expression repetitively as many times as
possible. Thus,
o* matches any number of os (including no
os).
* always applies to the smallest possible preceding
expression. Thus, fo* has a repeating o, not a repeating
fo. It matches f, fo, foo, and so on.
The matcher processes a * construct by matching, immediately,
as many repetitions as can be found. Then it continues with the rest
of the pattern. If that fails, backtracking occurs, discarding some
of the matches of the *-modified construct in case that makes
it possible to match the rest of the pattern. For example, in matching
ca*ar against the string caaar, the a* first
tries to match all three as; but the rest of the pattern is
ar and there is only r left to match, so this try fails.
The next alternative is for a* to match only two as.
With this choice, the rest of the regexp matches successfully.
- +
- is a postfix operator, similar to
* except that it must match
the preceding expression at least once. So, for example, ca+r
matches the strings car and caaaar but not the string
cr, whereas ca*r matches all three strings.
- ?
- is a postfix operator, similar to
* except that it can match the
preceding expression either once or not at all. For example,
ca?r matches car or cr; nothing else.
- *?, +?, ??
- are non-greedy variants of the operators above. The normal operators
*, +, ? are greedy in that they match as
much as they can, as long as the overall regexp can still match. With
a following ?, they are non-greedy: they will match as little
as possible.
Thus, both ab* and ab*? can match the string a
and the string abbbb; but if you try to match them both against
the text abbb, ab* will match it all (the longest valid
match), while ab*? will match just a (the shortest
valid match).
- \{n\}
- is a postfix operator that specifies repetition n times--that
is, the preceding regular expression must match exactly n times
in a row. For example,
x\{4\} matches the string xxxx
and nothing else.
- \{n,m\}
- is a postfix operator that specifies repetition between n and
m times--that is, the preceding regular expression must match
at least n times, but no more than m times. If m is
omitted, then there is no upper limit, but the preceding regular
expression must match at least n times.
\{0,1\} is
equivalent to ?.
\{0,\} is equivalent to
*.
\{1,\} is equivalent to +.
- [ ... ]
- is a character set, which begins with
[ and is terminated
by ]. In the simplest case, the characters between the two
brackets are what this set can match.
Thus, [ad] matches either one a or one d, and
[ad]* matches any string composed of just as and ds
(including the empty string), from which it follows that c[ad]*r
matches cr, car, cdr, caddaar, etc.
You can also include character ranges in a character set, by writing the
starting and ending characters with a - between them. Thus,
[a-z] matches any lower-case ASCII letter. Ranges may be
intermixed freely with individual characters, as in [a-z$%.],
which matches any lower-case ASCII letter or $, % or
period.
Note that the usual regexp special characters are not special inside a
character set. A completely different set of special characters exists
inside character sets: ], - and ^.
To include a ] in a character set, you must make it the first
character. For example, []a] matches ] or a. To
include a -, write - as the first or last character of the
set, or put it after a range. Thus, []-] matches both ]
and -.
To include ^ in a set, put it anywhere but at the beginning of
the set. (At the beginning, it complements the set--see below.)
When you use a range in case-insensitive search, you should write both
ends of the range in upper case, or both in lower case, or both should
be non-letters. The behavior of a mixed-case range such as A-z
is somewhat ill-defined, and it may change in future Emacs versions.
- [^ ... ]
[^ begins a complemented character set, which matches any
character except the ones specified. Thus, [^a-z0-9A-Z] matches
all characters except ASCII letters and digits.
^ is not special in a character set unless it is the first
character. The character following the ^ is treated as if it
were first (in other words, - and ] are not special there).
A complemented character set can match a newline, unless newline is
mentioned as one of the characters not to match. This is in contrast to
the handling of regexps in programs such as grep.
- ^
- is a special character that matches the empty string, but only at the
beginning of a line in the text being matched. Otherwise it fails to
match anything. Thus,
^foo matches a foo that occurs at
the beginning of a line.
- $
- is similar to
^ but matches only at the end of a line. Thus,
x+$ matches a string of one x or more at the end of a line.
- \
- has two functions: it quotes the special characters (including
\), and it introduces additional special constructs.
Because \ quotes special characters, \$ is a regular
expression that matches only $, and \[ is a regular
expression that matches only [, and so on.
Note: for historical compatibility, special characters are treated as
ordinary ones if they are in contexts where their special meanings make no
sense. For example, *foo treats * as ordinary since there is
no preceding expression on which the * can act. It is poor practice
to depend on this behavior; it is better to quote the special character anyway,
regardless of where it appears.
For the most part, \ followed by any character matches only that
character. However, there are several exceptions: two-character
sequences starting with \ that have special meanings. The second
character in the sequence is always an ordinary character when used on
its own. Here is a table of \ constructs.
- \|
- specifies an alternative. Two regular expressions a and b
with
\| in between form an expression that matches some text if
either a matches it or b matches it. It works by trying to
match a, and if that fails, by trying to match b.
Thus, foo\|bar matches either foo or bar
but no other string.
\| applies to the largest possible surrounding expressions. Only a
surrounding \( ... \) grouping can limit the grouping power of
\|.
Full backtracking capability exists to handle multiple uses of \|.
- \( ... \)
- is a grouping construct that serves three purposes:
- To enclose a set of
\| alternatives for other operations.
Thus, \(foo\|bar\)x matches either foox or barx.
- To enclose a complicated expression for the postfix operators
*,
+ and ? to operate on. Thus, ba\(na\)* matches
bananana, etc., with any (zero or more) number of na
strings.
- To record a matched substring for future reference.
This last application is not a consequence of the idea of a
parenthetical grouping; it is a separate feature that is assigned as a
second meaning to the same \( ... \) construct. In practice
there is usually no conflict between the two meanings; when there is
a conflict, you can use a "shy" group.
- \(?: ... \)
- specifies a "shy" group that does not record the matched substring;
you can't refer back to it with
\d. This is useful
in mechanically combining regular expressions, so that you
can add groups for syntactic purposes without interfering with
the numbering of the groups that were written by the user.
- \d
- matches the same text that matched the dth occurrence of a
\( ... \) construct.
After the end of a \( ... \) construct, the matcher remembers
the beginning and end of the text matched by that construct. Then,
later on in the regular expression, you can use \ followed by the
digit d to mean "match the same text matched the dth time
by the \( ... \) construct."
The strings matching the first nine \( ... \) constructs
appearing in a regular expression are assigned numbers 1 through 9 in
the order that the open-parentheses appear in the regular expression.
So you can use \1 through \9 to refer to the text matched
by the corresponding \( ... \) constructs.
For example, \(.*\)\1 matches any newline-free string that is
composed of two identical halves. The \(.*\) matches the first
half, which may be anything, but the \1 that follows must match
the same exact text.
If a particular \( ... \) construct matches more than once
(which can easily happen if it is followed by *), only the last
match is recorded.
- \`
- matches the empty string, but only at the beginning
of the buffer or string being matched against.
- \'
- matches the empty string, but only at the end of
the buffer or string being matched against.
- \=
- matches the empty string, but only at point.
- \b
- matches the empty string, but only at the beginning or
end of a word. Thus,
\bfoo\b matches any occurrence of
foo as a separate word. \bballs?\b matches
ball or balls as a separate word.
\b matches at the beginning or end of the buffer
regardless of what text appears next to it.
- \B
- matches the empty string, but not at the beginning or
end of a word.
- \<
- matches the empty string, but only at the beginning of a word.
\< matches at the beginning of the buffer only if a
word-constituent character follows.
- \>
- matches the empty string, but only at the end of a word.
\>
matches at the end of the buffer only if the contents end with a
word-constituent character.
- \w
- matches any word-constituent character. The syntax table
determines which characters these are. See Syntax.
- \W
- matches any character that is not a word-constituent.
- \sc
- matches any character whose syntax is c. Here c is a
character that designates a particular syntax class: thus,
w
for word constituent, - or for whitespace, .
for ordinary punctuation, etc. See Syntax.
- \Sc
- matches any character whose syntax is not c.
- \cc
- matches any character that belongs to the category c. For
example,
\cc matches Chinese characters, \cg matches
Greek characters, etc. For the description of the known categories,
type M-x describe-categories <RET>.
- \Cc
- matches any character that does not belong to category
c.
The constructs that pertain to words and syntax are controlled by the
setting of the syntax table (see Syntax).
Here is a complicated regexp, stored in sentence-end and used
by Emacs to recognize the end of a sentence together with any
whitespace that follows. We show its Lisp syntax to distinguish the
spaces from the tab characters. In Lisp syntax, the string constant
begins and ends with a double-quote. \" stands for a
double-quote as part of the regexp, \\ for a backslash as part
of the regexp, \t for a tab, and \n for a newline.
"[.?!][]\"')]*\\($\\| $\\|\t\\| \\)[ \t\n]*"
This contains four parts in succession: a character set matching
period, ?, or !; a character set matching
close-brackets, quotes, or parentheses, repeated zero or more times; a
set of alternatives within backslash-parentheses that matches either
end-of-line, a space at the end of a line, a tab, or two spaces; and a
character set matching whitespace characters, repeated any number of
times.
To enter the same regexp interactively, you would type <TAB> to
enter a tab, and C-j to enter a newline. (When typed
interactively, C-j should be preceded by a C-q, to prevent
Emacs from running the command bound to a newline.) You would also type
single backslashes as themselves, instead of doubling them for Lisp
syntax.
|
