Regular expressions for metadata detection

When creating or editing metadata definitions for Metadata Detection, you can use a regular expression for the Value. Use of a regular expression enables you to perform pattern matching for different permutations of metadata. A regular expression is more powerful than searching for exact string literals.

Use of regular expressions is recommended for advanced users only. No validation of the regular expression occurs. Test each regular expression before deploying it widely in watch list List that contains one of more voiceprints of people who are of particular interest to an enterprise.s.

Definitions

Term	Meaning
literal	Any character used in a search or matching expression. For example, to find ind in windows the ind is a literal string; each character plays a part in the expression – it is literally the string you are searching for.
metacharacter	One or more special characters that have a unique meaning and are NOT used as literals. For example, in the search expression, the character ^ (circumflex or caret) is a metacharacter.
escape sequence	A way of indicating that you want to use a metacharacter as a literal. In a regular expression an escape sequence involves placing the metacharacter \ (backslash) in front of the metacharacter to be used as a literal. For example, to find ^ind in w^indow, use the search string \^ind. To find \\file in the string c:\\file, use the search string \\\\file, that is, each \ you are searching for (a literal) is preceded by an escape sequence \.
target string	Term used to describe the string being searched for by the expression. In other words, the string in which you want to find a match or search pattern.
search expression	Term used to describe the expression being used to search the target string, that is, the pattern you use to find what you want.

The following characters are not supported:

<> (Greater than, Less than symbols)
& (Ampersand)
" (Quotation marks)
! (Exclamation symbol)

Pattern Matching Examples: Basic

The examples below refer to the following two target strings:

STRING1: Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)

STRING2: Mozilla/4.75 [en](X11;U;Linux2.2.16-22 i586)

Search for		Result	Description
m	STRING1	match	Finds the m in compatible.
m	STRING2	no match	There is no lower case m in this string. Searches are case sensitive unless you take special action.
a/4	STRING1	match	Found in Mozilla/4.0 - any combination of characters can be used for the match.
a/4	STRING2	match	Found in same place as in STRING1.
in	STRING1	match	Found in Windows.
in	STRING2	match	Found in Linux.
le	STRING1	match	Found in compatible.
le	STRING2	no match	There is an l and an e in this string but they are not adjacent (or contiguous).

Pattern Matching Examples: Brackets, Ranges and Negation

Bracket expressions introduce metacharacters, in this case the square brackets that allow you to define a list of items to test for, rather than the single characters in the simple matching example. These lists can be grouped into what are known as Character Classes typically comprising well known groups such as all numbers, and other logically grouped characters.

Metacharacter	Meaning
[ ] (Square Brackets)	Match anything inside the square brackets for one character position once and only once. For example, [12] means match the target to either 1 or 2 while [0123456789] means match to any character in the range 0 to 9.
- (Dash)	When used inside square brackets acts as the 'range separator' and allows you to define a range. In the example above of [0123456789] you could rewrite it as [0-9]. You can define more than one range inside a list. For example, [0-9A-C] means check for 0 to 9 and A to C (but not a to c). Note: To test for a Dash inside brackets (as a literal), the Dash must come first or last. For example, [-0-9] will test for - (the Dash) and 0 to 9.
^ (Circumflex or caret)	When used inside square brackets negates the expression (there is an alternate use for the circumflex/caret outside square brackets later). For example, [^Ff] means anything except upper or lower case F and [^a-z] means everything except lower case a to z. Spaces, or in this case the lack of them, between ranges are very important.

Metacharacter

Meaning

[ ]

(Square Brackets)

Match anything inside the square brackets for one character position once and only once. For example, [12] means match the target to either 1 or 2 while [0123456789] means match to any character in the range 0 to 9.

(Dash)

When used inside square brackets acts as the 'range separator' and allows you to define a range. In the example above of [0123456789] you could rewrite it as [0-9]. You can define more than one range inside a list. For example, [0-9A-C] means check for 0 to 9 and A to C (but not a to c). Note: To test for a Dash inside brackets (as a literal), the Dash must come first or last. For example, [-0-9] will test for - (the Dash) and 0 to 9.

(Circumflex or caret)

When used inside square brackets negates the expression (there is an alternate use for the circumflex/caret outside square brackets later). For example, [^Ff] means anything except upper or lower case F and [^a-z] means everything except lower case a to z.

Spaces, or in this case the lack of them, between ranges are very important.

The examples below refer to the following two target strings:

STRING1: Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)

STRING2: Mozilla/4.75 [en](X11;U;Linux2.2.16-22 i586)

Search for	In	Result	Description
in[du]	STRING1	match	Finds ind in Windows.
in[du]	STRING2	match	Finds inu in Linux.
x[0-9A-Z]	STRING1	no match	The tests are case-sensitive. To find the xt in DigExt use [0-9a-z] or [0-9A-Zt]. You can also use this format for testing upper and lower case. For example, [Ff] will check for lower and upper case F.
x[0-9A-Z]	STRING2	match	Finds x2 in Linux2.
[^A-M]in	STRING1	match	Finds Win in Windows.
[^A-M]in	STRING2	no match	The range A to M is excluded in the search, so Linux is not found, although linux (if it were present) would be found.

Pattern Matching Examples: Positioning (Anchors)

You can control where in your target strings the matches are valid. The following is a list of metacharacters that affect the position of the search:

Metacharacter	Meaning
^ (Circumflex or caret)	When used outside square brackets, this symbol means look only at the beginning of the target string. For example, ^Win will not find Windows in STRING1 but ^Moz will find Mozilla.
$ (Dollar)	The dollar symbol means look for the preceding text at the end of any line. For example, 'fox$' will match ‘silver fox’.
. (Period)	A “.” matches any character - though it does require that there is a character to match.

Metacharacter

Meaning

(Circumflex or caret)

When used outside square brackets, this symbol means look only at the beginning of the target string. For example, ^Win will not find Windows in STRING1 but ^Moz will find Mozilla.

(Dollar)

The dollar symbol means look for the preceding text at the end of any line. For example, 'fox$' will match ‘silver fox’.

(Period)

A “.” matches any character - though it does require that there is a character to match.

The examples refer to the following two target strings:

STRING1: Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)

STRING2: Mozilla/4.75 [en](X11;U;Linux2.2.16-22 i586).

Search for	In	Result	Description
$[a-z])	STRING1	match	Finds t) in DigiExt).
$[a-z])	STRING2	no match	There is a numeric value at the end of this string but you would need [0-9a-z]) to find it.
.in	STRING1	match	Finds Win in Windows.
.in	STRING2	match	Finds Lin in Linux.

Pattern Matching Examples: Iteration Metacharacters

The following is a set of iteration metacharacters (also known as quantifiers) that can control the number of times a character or string is found in searches.

Metacharacter	Meaning
? (Question mark)	Matches the preceding character 0 or 1 times only. For example, colou?r will find both color and colour.
* (Asterisk or star)	Matches the preceding character 0 or more times. For example, tr* will find tree and tread and trough.
+ (Plus)	Matches the previous character 1 or more times. For example, tre+ will find tree and tread but not trough.
{n} (Curly brackets)	Matches the preceding character n times exactly. For example, to find a local phone number, [0-9]{3}-[0-9]{4} would find any number of the form 123-4567. The - (Dash) in the above example, because it is outside the square brackets, is a literal character.

The examples below refer to the following two target strings:

STRING1: Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)

STRING2: Mozilla/4.75 [en](X11;U;Linux2.2.16-22 i586)

Search for	in..	Result	Description
\(.*l	STRING1	match	Finds l in (compatible. The opening \ is an escape character used to indicate that it precedes is a literal and not a metacharacter.
\(.*l	STRING2	no match	Mozilla contains lls but not preceded by an open parenthesis (no match) and Linux has an upper case L (no match).
W*in	STRING1	match	Finds the Win in Windows.
W*in	STRING2	match	Finds in in Linux preceded by W zero times.
[xX][0-9a-z]{2}	STRING1	no match	Finds x in DigExt but only one t.
[xX][0-9a-z]{2}	STRING2	match	Finds X and 11 in X11.

Pattern Matching Examples: Parentheses and Pipes

The following additional metacharacters can provide added power to searches:

Metacharacter	Meaning
() Parentheses	The ( (open parenthesis) and ) (close parenthesis) may be used to group (or bind) parts of search expressions together, as shown in the examples.
\| (Vertical bar or pipe)	Technically called alternation, this character means find the left hand OR right hand values. For example, gr(a\|e)y will find gray or grey.

Metacharacter

Meaning

()

Parentheses

The ( (open parenthesis) and ) (close parenthesis) may be used to group (or bind) parts of search expressions together, as shown in the examples.

(Vertical bar or pipe)

Technically called alternation, this character means find the left hand OR right hand values. For example, gr(a|e)y will find gray or grey.

The examples below refer to the following two target strings:

STRING1: Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)

STRING2: Mozilla/4.75 [en](X11;U;Linux2.2.16-22 i586)

Search for	in	Result	Description
^([L-Z]in)	STRING1	no match	^ is an anchor indicating first position. Win does not start the string, so no match.
^([L-Z]in)	STRING2	no match	^ is an anchor indicating first position. Linux does not start the string, so no match.
((4\.[0-3])\|(2\.[0-3]))	STRING1	match	Finds the 4.0 in Mozilla/4.0.
((4\.[0-3])\|(2\.[0-3]))	STRING2	match	Finds the 2.2 in Linux2.2.16-22.
(W\|L)in	STRING1	match	Finds Win in Windows.
(W\|L)in	STRING2	match	Finds Lin in Linux.