Module std.regex
Regular expressions are a commonly used method of pattern matching on strings, with regex being a catchy word for a pattern in this domain specific language. Typical problems usually solved by regular expressions include validation of user input and the ubiquitous find & replace in text processing utilities.
Category | Functions |
---|---|
Matching | bmatch
match
matchAll
matchFirst
|
Building | ctRegex
escaper
regex
|
Replace | replace
replaceAll
replaceAllInto
replaceFirst
replaceFirstInto
|
Split | split
splitter
|
Objects | Captures
Regex
RegexException
RegexMatch
Splitter
StaticRegex
|
Synopsis
import std .regex;
import std .stdio;
void main()
{
// Print out all possible dd/mm/yy(yy) dates found in user input.
auto r = regex(r"\b[0-9][0-9]?/[0-9][0-9]?/[0-9][0-9](?:[0-9][0-9])?\b");
foreach (line; stdin .byLine)
{
// matchAll() returns a range that can be iterated
// to get all subsequent matches.
foreach (c; matchAll(line, r))
writeln(c .hit);
}
}
...
// Create a static regex at compile-time, which contains fast native code.
auto ctr = ctRegex!(`^.*/([^/]+)/?$`);
// It works just like a normal regex:
auto c2 = matchFirst("foo/bar", ctr); // First match found here, if any
assert(!c2 .empty); // Be sure to check if there is a match before examining contents!
assert(c2[1] == "bar"); // Captures is a range of submatches: 0 = full match.
...
// multi-pattern regex
auto multi = regex([`\d+,\d+`,`(a-z]+):(\d+)`]);
auto m = "abc:43 12,34" .matchAll(multi);
assert(m .front .whichPattern == 2);
assert(m .front[1] == "abc");
assert(m .front[2] == "43");
m .popFront();
assert(m .front .whichPattern == 1);
assert(m .front[1] == "12");
...
// The result of the `matchAll/matchFirst` is directly testable with if/assert/while.
// e.g. test if a string consists of letters:
assert(matchFirst("Letter", `^\p{L}+$`));
Syntax and general information
The general usage guideline is to keep regex complexity on the side of simplicity, as its capabilities reside in purely character-level manipulation. As such it's ill-suited for tasks involving higher level invariants like matching an integer number bounded in an [a,b] interval. Checks of this sort of are better addressed by additional post-processing. The basic syntax shouldn't surprise experienced users of regular expressions.
For an introduction to std
see a
short tour of the module API
and its abilities.
There are other web resources on regular expressions to help newcomers, and a good reference with tutorial can easily be found.
This library uses a remarkably common ECMAScript syntax flavor with the following extensions:
- Named subexpressions, with Python syntax.
- Unicode properties such as Scripts, Blocks and common binary properties e.g Alphabetic, White_Space, Hex_Digit etc.
- Arbitrary length and complexity lookbehind, including lookahead in lookbehind and vise-versa.
Pattern syntax
std.regex operates on codepoint level,
'character' in this table denotes a single Unicode codepoint.
Pattern element | Semantics |
Atoms | Match single characters |
any character except [{|*+?()^$ | Matches the character itself. |
. | In single line mode matches any character. Otherwise it matches any character except '\n' and '\r'. |
[class] | Matches a single character that belongs to this character class. |
[^class] | Matches a single character that does not belong to this character class. |
\cC | Matches the control character corresponding to letter C |
\xXX | Matches a character with hexadecimal value of XX. |
\uXXXX | Matches a character with hexadecimal value of XXXX. |
\U00YYYYYY | Matches a character with hexadecimal value of YYYYYY. |
\f | Matches a formfeed character. |
\n | Matches a linefeed character. |
\r | Matches a carriage return character. |
\t | Matches a tab character. |
\v | Matches a vertical tab character. |
\d | Matches any Unicode digit. |
\D | Matches any character except Unicode digits. |
\w | Matches any word character (note: this includes numbers). |
\W | Matches any non-word character. |
\s | Matches whitespace, same as \p{White_Space}. |
\S | Matches any character except those recognized as \s . |
\\\\ | Matches \ character. |
\c where c is one of [|*+?() | Matches the character c itself. |
\p{PropertyName} | Matches a character that belongs to the Unicode PropertyName set. Single letter abbreviations can be used without surrounding {,}. |
\P{PropertyName} | Matches a character that does not belong to the Unicode PropertyName set. Single letter abbreviations can be used without surrounding {,}. |
\p{InBasicLatin} | Matches any character that is part of the BasicLatin Unicode block. |
\P{InBasicLatin} | Matches any character except ones in the BasicLatin Unicode block. |
\p{Cyrillic} | Matches any character that is part of Cyrillic script. |
\P{Cyrillic} | Matches any character except ones in Cyrillic script. |
Quantifiers | Specify repetition of other elements |
* | Matches previous character/subexpression 0 or more times. Greedy version - tries as many times as possible. |
*? | Matches previous character/subexpression 0 or more times. Lazy version - stops as early as possible. |
+ | Matches previous character/subexpression 1 or more times. Greedy version - tries as many times as possible. |
+? | Matches previous character/subexpression 1 or more times. Lazy version - stops as early as possible. |
{n} | Matches previous character/subexpression exactly n times. |
{n,} | }, Matches previous character/subexpression n times or more. Greedy version - tries as many times as possible. |
{n,}? | }?, Matches previous character/subexpression n times or more. Lazy version - stops as early as possible. |
{n,m} | m}, Matches previous character/subexpression n to m times. Greedy version - tries as many times as possible, but no more than m times. |
{n,m}? | m}?, Matches previous character/subexpression n to m times. Lazy version - stops as early as possible, but no less then n times. |
Other | Subexpressions & alternations |
(regex) | Matches subexpression regex, saving matched portion of text for later retrieval. |
(?#comment) | An inline comment that is ignored while matching. |
(?:regex) | Matches subexpression regex, not saving matched portion of text. Useful to speed up matching. |
A|B | Matches subexpression A, or failing that, matches B. |
(?P<name>regex) | Matches named subexpression regex labeling it with name 'name'. When referring to a matched portion of text, names work like aliases in addition to direct numbers. |
Assertions | Match position rather than character |
^ | Matches at the begining of input or line (in multiline mode). |
$ | Matches at the end of input or line (in multiline mode). |
\b | Matches at word boundary. |
\B | Matches when not at word boundary. |
(?=regex) | Zero-width lookahead assertion. Matches at a point where the subexpression regex could be matched starting from the current position. |
(?!regex) | Zero-width negative lookahead assertion. Matches at a point where the subexpression regex could not be matched starting from the current position. |
(?<=regex) | Zero-width lookbehind assertion. Matches at a point where the subexpression regex could be matched ending at the current position (matching goes backwards). |
(?<!regex) | Zero-width negative lookbehind assertion. Matches at a point where the subexpression regex could not be matched ending at the current position (matching goes backwards). |
Character classes
Pattern element | Semantics |
Any atom | Has the same meaning as outside of a character class, except for ] which must be written as \\] |
a-z | Includes characters a, b, c, ..., z. |
[a||b], [a--b], [a~~b], [a&&b] | [a--b], [a~~b], [a&&b], Where a, b are arbitrary classes, means union, set difference, symmetric set difference, and intersection respectively. Any sequence of character class elements implicitly forms a union. |
Regex flags
Flag | Semantics |
g | Global regex, repeat over the whole input. |
i | Case insensitive matching. |
m | Multi-line mode, match ^, $ on start and end line separators as well as start and end of input. |
s | Single-line mode, makes . match '\n' and '\r' as well. |
x | Free-form syntax, ignores whitespace in pattern, useful for formatting complex regular expressions. |
Unicode support
This library provides full Level 1 support* according to UTS 18. Specifically:
- 1.1 Hex notation via any of \uxxxx, \U00YYYYYY, \xZZ.
- 1.2 Unicode properties.
- 1.3 Character classes with set operations.
- 1.4 Word boundaries use the full set of "word" characters.
- 1.5 Using simple casefolding to match case insensitively across the full range of codepoints.
- 1.6 Respecting line breaks as any of \u000A | \u000B | \u000C | \u000D | \u0085 | \u2028 | \u2029 | \u000D\u000A.
- 1.7 Operating on codepoint level.
Replace format string
A set of functions in this module that do the substitution rely
on a simple format to guide the process. In particular the table below
applies to the format
argument of
replaceFirst
and replaceAll
.
The format string can reference parts of match using the following notation.
Format specifier | Replaced by |
$& | the whole match. |
$` | part of input preceding the match. |
$' | part of input following the match. |
$$ | '$' character. |
\c , where c is any character | where c is any character, the character c itself. |
\\\\ | '\\' character. |
$1 .. $99 | submatch number 1 to 99 respectively. |
Slicing and zero memory allocations orientation
All matches returned by pattern matching functionality in this library
are slices of the original input. The notable exception is the replace
family of functions that generate a new string from the input.
In cases where producing the replacement is the ultimate goal
replaceFirstInto
and replaceAllInto
could come in handy
as functions that avoid allocations even for replacement.
Functions
Name | Description |
---|---|
bmatch(input, re)
|
Start matching of input to regex pattern re ,
using traditional backtracking matching scheme.
|
escaper(r)
|
A range that lazily produces a string output escaped to be used inside of a regular expression. |
match(input, re)
|
Start matching input to regex pattern re ,
using Thompson NFA matching scheme.
|
matchAll(input, re)
|
Initiate a search for all non-overlapping matches to the pattern re
in the given input . The result is a lazy range of matches generated
as they are encountered in the input going left to right.
|
matchFirst(input, re)
|
Find the first (leftmost) slice of the input that
matches the pattern re . This function picks the most suitable
regular expression engine depending on the pattern properties.
|
regex(patterns, flags)
|
Compile regular expression pattern for the later execution. |
replace(input, re, format)
|
Old API for replacement, operation depends on flags of pattern re .
With "g" flag it performs the equivalent of replaceAll otherwise it
works the same as replaceFirst .
|
replaceAll(input, re, format)
|
Construct a new string from input by replacing all of the
fragments that match a pattern re with a string generated
from the match according to the format specifier.
|
replaceAll(input, re)
|
This is a general replacement tool that construct a new string by replacing
matches of pattern re in the input . Unlike the other overload
there is no format string instead captures are passed to
to a user-defined functor fun that returns a new string
to use as replacement.
|
replaceAllInto(sink, input, re, format)
|
A variation on replaceAll that instead of allocating a new string
on each call outputs the result piece-wise to the sink . In particular
this enables efficient construction of a final output incrementally.
|
replaceFirst(input, re, format)
|
Construct a new string from input by replacing the first match with
a string generated from it according to the format specifier.
|
replaceFirst(input, re)
|
This is a general replacement tool that construct a new string by replacing
matches of pattern re in the input . Unlike the other overload
there is no format string instead captures are passed to
to a user-defined functor fun that returns a new string
to use as replacement.
|
replaceFirstInto(sink, input, re, format)
|
A variation on replaceFirst that instead of allocating a new string
on each call outputs the result piece-wise to the sink . In particular
this enables efficient construction of a final output incrementally.
|
split(input, rx)
|
An eager version of splitter that creates an array with splitted slices of input .
|
splitter(r, pat)
|
Splits a string r using a regular expression pat as a separator.
|
Structs
Name | Description |
---|---|
Captures
|
Captures object contains submatches captured during a call
to match or iteration over RegexMatch range.
|
RegexMatch
|
A regex engine state, as returned by match family of functions.
|
Splitter
|
Splits a string r using a regular expression pat as a separator.
|
Manifest constants
Name | Type | Description |
---|---|---|
ctRegex
|
Compile regular expression using CTFE and generate optimized native machine code for matching it. |
Aliases
Name | Type | Description |
---|---|---|
Regex
|
std
|
Regex object holds regular expression pattern in compiled form.
|
RegexException
|
std
|
Exception object thrown in case of errors during regex compilation. |
StaticRegex
|
Regex
|
A StaticRegex is Regex object that contains D code specially
generated at compile-time to speed up matching.
|
Authors
Dmitry Olshansky,
API and utility constructs are modeled after the original std
by Walter Bright and Andrei Alexandrescu.