Groupings and backreferences

This chapter will show how to reuse portion matched by capture groups via backreferences within regexp definition and replacement section. You'll also learn some special grouping syntax for cases where plain capture groups isn't enough.

Backreferences

First up, how to refer to capture group portions directly in regexp definition and replacement section. You have already seen how to refer to text captured by groups with match and matchAll methods. You've also seen how to pass captured portions to a function in replace method. More directly, you can use backreference \N (within the regexp definition) and $N (replacement section), where N is the capture group you want. What's more, you can also apply quantifiers to backreferences when used in regexp definition. All the various forms is listed below:

  • in replacement section, use $1, $2, etc to refer to the corresponding capture group
  • in replacement section, use $& to refer to entire matched portion
    • $` gives string before the matched portion
    • $' gives string after the matched portion
  • within regexp definition, use \1, \2, etc to refer to the corresponding capture group
// remove square brackets that surround digit characters
> '[52] apples [and] [31] mangoes'.replace(/\[(\d+)\]/g, '$1')
< "52 apples [and] 31 mangoes"

// replace __ with _ and delete _ if it is alone
> '_foo_ __123__ _baz_'.replace(/(_)?_/g, '$1')
< "foo _123_ baz"

// swap words that are separated by a comma
> 'good,bad 42,24'.replace(/(\w+),(\w+)/g, '$2,$1')
< "bad,good 24,42"

Here's some examples for using backreferences available by default without needing capture groups.

// add something around the entire matched portion
> '52 apples and 31 mangoes'.replace(/\d+/g, '($&)')
< "(52) apples and (31) mangoes"
> 'Hello world'.replace(/.*/, 'Hi. $&. Have a nice day')
< "Hi. Hello world. Have a nice day"

// duplicate first field and add it as last field
> 'fork,42,nice,3.14'.replace(/,.+/, '$&,$`')
< "fork,42,nice,3.14,fork"

And here's some examples for using backreferences within regexp definition.

// elements that have at least one consecutive repeated word character
> let words = ['moon', 'mono', 'excellent', 'poll']
> words.filter(w => /(\w)\1/.test(w))
< ["moon", "excellent", "poll"]

// remove any number of consecutive duplicate words separated by space
// note the use of quantifier on backreference
// use \W+ instead of space to cover cases like 'a;a<-;a'
> 'aa a a a 42 f_1 f_1 f_13.14'.replace(/\b(\w+)( \1)+\b/g, '$1')
< "aa a 42 f_1 f_13.14"

Backreference oddities

Since $ is special in replacement section, there's an issue to place it literally if followed by numbers. Usually, escaping is used for such purposes, but here you need to use $$.

// no capture group used, so '$1' is inserted literally
> 'cat'.replace(/a/, '{$1}')
< "c{$1}t"
// capture group used, '\$1' is same as '$1' here
> 'cat'.replace(/(a)/, '{\$1}')
< "c{a}t"

// use '$$' to avoid backreference and insert '$' literally
> 'cat'.replace(/(a)/, '{$$1}')
< "c{$1}t"

Another issue is how to avoid ambiguity when you have normal digits immediately following a backreference? It'll depend on how many backreferences are present in the pattern and whether you need to avoid ambiguity in regexp definition or replacement section. For example, if there are less than 10 groups, then something like $12 will refer to first capture group and 2 as a character. If there are no capture groups, then something like $5 will get inserted literally as $ and 5.

// $15 here will backreference 1st group and use 5 as a character
> '[52] apples and [31] mangoes'.replace(/\[(\d+)\]/g, '($15)')
< "(525) apples and (315) mangoes"

// $3 will be inserted literally since there is only one capture group
> '[52] apples and [31] mangoes'.replace(/\[(\d+)\]/g, '$3')
< "$3 apples and $3 mangoes"

// $1 will be inserted literally since there are no capture groups
> '[52] apples and [31] mangoes'.replace(/\[\d+\]/g, '$1')
< "$1 apples and $1 mangoes"

On the other hand, if you have more than 9 but less than 100 groups, then there would be an issue if you want to refer to single digit group followed by literal digit characters. The workaround is to prefix 0 such that number of digits after $ equals the number of digits required for the highest capture group. So, if you have more than 9 but less than 100 groups, $05 will refer to 5th capture group and any digit after that will be treated literally.

// for illustration purposes, 12 capture groups have been defined here
// if you want to reference 2-digit group, there's no issue
> 'abcdefghijklmn'.replace(/(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)/, '$11')
< "kmn"

// what if you wanted to reference 1st group followed by '1' as a character?
// using \x31 wouldn't work as it still results in $11
> 'abcdefghijklmn'.replace(/(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)/, '$1\x31')
< "kmn"
// prefix a '0' so that '$01' becomes the reference and '1' becomes character
> 'abcdefghijklmn'.replace(/(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)/, '$011')
< "a1mn"

The workaround is simpler in regexp definition. The 0 prefix trick doesn't work, but using ASCII code with \xhh works. For example, \1\x31 will refer to first capture group followed by 1 as a character.

> 'abcdefghijklmna1d'.replace(/(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.).*\1\x31/, 'X')
< "Xd"

Non-capturing groups

Grouping has many uses like applying quantifier on a regexp portion, creating terse regexp by factoring common portions and so on. It also affects the output of split method as seen in split with capture groups section. Unlike similar methods in other languages, match method with g flag isn't affected by capture groups and returns entire matched portions.

// split method without capture group
> 'Sample123string42with777numbers'.split(/\d+/)
< ["Sample", "string", "with", "numbers"]
// split method with capture group
> 'Sample123string42with777numbers'.split(/(\d+)/)
< ["Sample", "123", "string", "42", "with", "777", "numbers"]

// match method example with 'g' flag and capture groups
> 'effort flee facade oddball rat tool'.match(/\b\w*(\w)\1\w*\b/g)
< ["effort", "flee", "oddball", "tool"]
// here's another example
> 'hi 123123123 bye 456123456'.match(/(123)+/g)
< ["123123123", "123"]

When backreferencing is not required, you can use a non-capturing group to avoid behavior change of split method. It also helps to avoid keeping a track of capture group numbers when that particular group is not needed for backreferencing. The syntax is (?:pat) to define a non-capturing group, where pat is an abbreviation for a portion of regular expression pattern. More such special groups starting with (? syntax will be discussed later on.

// here, grouping is needed to take out common portion and apply quantifier
// but using capture group will not give expected output
> '123hand42handy777handful500'.split(/hand(y|ful)?/)
< ["123", undefined, "42", "y", "777", "ful", "500"]

// non-capturing group to the rescue
> '123hand42handy777handful500'.split(/hand(?:y|ful)?/)
< ["123", "42", "777", "500"]

// with normal grouping, need to keep track of all the groups
> '1,2,3,4,5,6,7'.replace(/^(([^,]+,){3})([^,]+)/, '$1($3)')
< "1,2,3,(4),5,6,7"
// using non-capturing groups, only relevant groups have to be tracked
> '1,2,3,4,5,6,7'.replace(/^((?:[^,]+,){3})([^,]+)/, '$1($2)')
< "1,2,3,(4),5,6,7"

Referring to text matched by a capture group with a quantifier will give only the last match, not entire match. Use a capture group around the grouping and quantifier together to get the entire matching portion. In such cases, the inner grouping is an ideal candidate to use non-capturing group.

// '$1' here contains only the fourth field
> 'so:cat:rest:in:put:to'.replace(/^([^:]+:){4}/, '($1)')
< "(in:)put:to"

// '$1' will now contain the entire matching portion
> 'so:cat:rest:in:put:to'.replace(/^((?:[^:]+:){4})/, '($1)')
< "(so:cat:rest:in:)put:to"

Named capture groups

Regexp can get cryptic and difficult to maintain, even for seasoned programmers. There are a few constructs to help add clarity. One such is naming the capture groups and using that name for backreferencing instead of plain numbers. In addition, the named capture group portions can be extracted as key-value pairs using the groups property. The syntax is:

  • (?<name>pat) for naming the capture groups
  • \k<name> for backreferencing in regexp definition
  • $<name> for backreferencing in replacement section
> let row = 'today,2008-03-24,food,2008-03-24,nice,2018-10-25,5632'

// same as: /(\d{4}-\d{2}-\d{2}).*\1/
> row.match(/(?<date>\d{4}-\d{2}-\d{2}).*\k<date>/)[0]
< "2008-03-24,food,2008-03-24"

// giving names to first and second captured words
// same as: replace(/(\w+),(\w+)/g, '$2,$1')
> 'good,bad 42,24'.replace(/(?<fw>\w+),(?<sw>\w+)/g, '$<sw>,$<fw>')
< "bad,good 24,42"

Here's an example with groups property.

> let m = '2018-10-25,car'.match(/(?<date>[^,]+),(?<product>[^,]+)/)

> m.groups
< {date: "2018-10-25", product: "car"}
> m.groups.date
< "2018-10-25"
> m.groups.product
< "car"

Cheatsheet and Summary

NoteDescription
Backreferencegives matched portion of Nth capture group
use $1, $2, $3, etc in replacement section
$& gives entire matched portion
$` gives string before the matched portion
$' gives string after the matched portion
use \1, \2, \3, etc within regexp definition
$$insert $ literally in replacement section
$0Nsame as $N, allows to separate backreference and other digits
\N\xhhallows to separate backreference and digits in regexp definition
(?:pat)non-capturing group
(?<name>pat)named capture group
use \k<name> for backreferencing in regexp definition
use $<name> for backreferencing in replacement section
named captures are also accessible via groups property

This chapter covered many more features related to grouping — backreferencing to get matched portion of capture groups and naming the groups to add clarity. When capture groups results in unwanted behavior change (ex: split method), you can use non-capturing groups instead. You'll see more such special groups in Lookarounds chapter.

Exercises

a) Replace the space character that occurs after a word ending with a or r with a newline character.

> let ip = 'area not a _a2_ roar took 22'

> console.log()     // add your solution here
  area
  not a
  _a2_ roar
  took 22

b) Add [] around words starting with s and containing e and t in any order.

> let ip = 'sequoia subtle exhibit asset sets tests site'

// add your solution here
< "sequoia [subtle] exhibit asset [sets] tests [site]"

c) Replace all whole words with X that start and end with the same word character. Single character word should get replaced with X too, as it satisfies the stated condition.

> let ip = 'oreo not a _a2_ roar took 22'

// add your solution here
< "X not X X X took X"

d) Convert the given markdown headers to corresponding anchor tag. Consider the input to start with one or more # characters followed by space and word characters. The name attribute is constructed by converting the header to lowercase and replacing spaces with hyphens. Can you do it without using a capture group?

> let header1 = '# Regular Expressions'
> let header2 = '## Named capture groups'

> function hyphenify(m) {
      // add your solution here
  }

> header1.replace()     // add your solution here
< "# <a name='regular-expressions'></a>Regular Expressions"
> header2.replace()     // add your solution here
< "## <a name='named-capture-groups'></a>Named capture groups"

e) Convert the given markdown anchors to corresponding hyperlinks.

> let anchor1 = "# <a name='regular-expressions'></a>Regular Expressions"
> let anchor2 = "## <a name='subexpression-calls'></a>Subexpression calls"

> const hyperlink =         // add your solution here

> anchor1.replace()         // add your solution here
< "[Regular Expressions](#regular-expressions)"
> anchor2.replace()         // add your solution here
< "[Subexpression calls](#subexpression-calls)"

f) Check if given input strings have words with at least two consecutive repeated alphabets irrespective of case. For example, words like stillnesS and Committee should return true but words like root or readable or rotational should return false. Consider word to be as defined in regular expression parlance.

> let s1 = 'readable COMMItTEe'
> let s2 = 'rotational sti1lness _foot_'
> let s3 = 'needed repeated'
> let s4 = 'offsh00t'

> const pat1 =      // add your solution here

> pat1.test(s1)
true
> pat1.test(s2)
false
> pat1.test(s3)
false
> pat1.test(s4)
true

g) For the given input string, replace all occurrences of digit sequences with only the unique non-repeating sequence. For example, 232323 should be changed to 23 and 897897 should be changed to 897. If there no repeats (for example 1234) or if the repeats end prematurely (for example 12121), it should not be changed.

> let ip = '1234 2323 453545354535 9339 11 60260260'

// add your solution here
< "1234 23 4535 9339 1 60260260"

h) Replace sequences made up of words separated by : or . by the first word of the sequence. Such sequences will end when : or . is not followed by a word character.

> let ip = 'wow:Good:2_two:five: hi-2 bye kite.777.water.'

// add your solution here
< "wow hi-2 bye kite"

i) Replace sequences made up of words separated by : or . by the last word of the sequence. Such sequences will end when : or . is not followed by a word character.

> let ip = 'wow:Good:2_two:five: hi-2 bye kite.777.water.'

// add your solution here
< "five hi-2 bye water"

j) Split the given input string on one or more repeated sequence of cat.

> let ip = 'firecatlioncatcatcatbearcatcatparrot'

// add your solution here
< ["fire", "lion", "bear", "parrot"]

k) For the given input string, find all occurrences of digit sequences with at least one repeating sequence. For example, 232323 and 897897. If the repeats end prematurely, for example 12121, it should not be matched.

> let ip = '1234 2323 453545354535 9339 11 60260260'

> const pat2 =      // add your solution here

// entire sequences in the output
// add your solution here
< ["2323", "453545354535", "11"]

// only the unique sequence in the output
// add your solution here
< ["23", "4535", "1"]

l) Convert the comma separated strings to corresponding key-value pair mapping as shown below. The keys are name, maths and phy for the three fields in the input strings.

> let row1 = 'rohan,75,89'
> let row2 = 'rose,88,92'

> const pat3 =      // add your solution here

// add your solution here for row1
< {name: "rohan", maths: "75", phy: "89"}

// add your solution here for row2
< {name: "rose", maths: "88", phy: "92"}

m) Surround all whole words with (). Additionally, if the whole word is imp or ant, delete them. Can you do it with single substitution?

> let ip = 'tiger imp goat eagle ant important'

// add your solution here
< "(tiger) () (goat) (eagle) () (important)"