Pattern类（java JDK源码记录）
Posted zhangyishu
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Pattern类（java JDK源码记录）相关的知识，希望对你有一定的参考价值。
1 /* 2 * Copyright (c) 1999, 2018, Oracle and/or its affiliates. All rights reserved. 3 * ORACLE PROPRIETARY/CONFIDENTIAL. Use is subject to license terms. 4 * 5 * 6 * 7 * 8 * 9 * 10 * 11 * 12 * 13 * 14 * 15 * 16 * 17 * 18 * 19 * 20 * 21 * 22 * 23 * 24 */ 25 26 package java.util.regex; 27 28 import java.text.Normalizer; 29 import java.text.Normalizer.Form; 30 import java.util.Locale; 31 import java.util.Iterator; 32 import java.util.Map; 33 import java.util.ArrayList; 34 import java.util.HashMap; 35 import java.util.LinkedHashSet; 36 import java.util.List; 37 import java.util.Set; 38 import java.util.Arrays; 39 import java.util.NoSuchElementException; 40 import java.util.Spliterator; 41 import java.util.Spliterators; 42 import java.util.function.Predicate; 43 import java.util.stream.Stream; 44 import java.util.stream.StreamSupport; 45 46 47 /** 48 * A compiled representation of a regular expression. 49 * 50 * <p> A regular expression, specified as a string, must first be compiled into 51 * an instance of this class. The resulting pattern can then be used to create 52 * a @link Matcher object that can match arbitrary @linkplain 53 * java.lang.CharSequence character sequences against the regular 54 * expression. All of the state involved in performing a match resides in the 55 * matcher, so many matchers can share the same pattern. 56 * 57 * <p> A typical invocation sequence is thus 58 * 59 * <blockquote><pre> 60 * Pattern p = Pattern.@link #compile compile("a*b"); 61 * Matcher m = p.@link #matcher matcher("aaaaab"); 62 * boolean b = m.@link Matcher#matches matches();</pre></blockquote> 63 * 64 * <p> A @link #matches matches method is defined by this class as a 65 * convenience for when a regular expression is used just once. This method 66 * compiles an expression and matches an input sequence against it in a single 67 * invocation. The statement 68 * 69 * <blockquote><pre> 70 * boolean b = Pattern.matches("a*b", "aaaaab");</pre></blockquote> 71 * 72 * is equivalent to the three statements above, though for repeated matches it 73 * is less efficient since it does not allow the compiled pattern to be reused. 74 * 75 * <p> Instances of this class are immutable and are safe for use by multiple 76 * concurrent threads. Instances of the @link Matcher class are not safe for 77 * such use. 78 * 79 * 80 * <h3><a id="sum">Summary of regular-expression constructs</a></h3> 81 * 82 * <table class="borderless"> 83 * <caption style="display:none">Regular expression constructs, and what they match</caption> 84 * <thead style="text-align:left"> 85 * <tr> 86 * <th id="construct">Construct</th> 87 * <th id="matches">Matches</th> 88 * </tr> 89 * </thead> 90 * <tbody style="text-align:left"> 91 * 92 * <tr><th colspan="2" style="padding-top:20px" id="characters">Characters</th></tr> 93 * 94 * <tr><th style="vertical-align:top; font-weight: normal" id="x"><i>x</i></th> 95 * <td headers="matches characters x">The character <i>x</i></td></tr> 96 * <tr><th style="vertical-align:top; font-weight: normal" id="backslash">@code \\</th> 97 * <td headers="matches characters backslash">The backslash character</td></tr> 98 * <tr><th style="vertical-align:top; font-weight: normal" id="octal_n">@code \0<i>n</i></th> 99 * <td headers="matches characters octal_n">The character with octal value @code 0<i>n</i> 100 * (0 @code <= <i>n</i> @code <= 7)</td></tr> 101 * <tr><th style="vertical-align:top; font-weight: normal" id="octal_nn">@code \0<i>nn</i></th> 102 * <td headers="matches characters octal_nn">The character with octal value @code 0<i>nn</i> 103 * (0 @code <= <i>n</i> @code <= 7)</td></tr> 104 * <tr><th style="vertical-align:top; font-weight: normal" id="octal_nnn">@code \0<i>mnn</i></th> 105 * <td headers="matches characters octal_nnn">The character with octal value @code 0<i>mnn</i> 106 * (0 @code <= <i>m</i> @code <= 3, 107 * 0 @code <= <i>n</i> @code <= 7)</td></tr> 108 * <tr><th style="vertical-align:top; font-weight: normal" id="hex_hh">@code \x<i>hh</i></th> 109 * <td headers="matches characters hex_hh">The character with hexadecimal value @code 0x<i>hh</i></td></tr> 110 * <tr><th style="vertical-align:top; font-weight: normal" id="hex_hhhh"><code>\u</code><i>hhhh</i></th> 111 * <td headers="matches characters hex_hhhh">The character with hexadecimal value @code 0x<i>hhhh</i></td></tr> 112 * <tr><th style="vertical-align:top; font-weight: normal" id="hex_h_h"><code>\x</code><i>h...h</i></th> 113 * <td headers="matches characters hex_h_h">The character with hexadecimal value @code 0x<i>h...h</i> 114 * (@link java.lang.Character#MIN_CODE_POINT Character.MIN_CODE_POINT 115 * <= @code 0x<i>h...h</i> <= 116 * @link java.lang.Character#MAX_CODE_POINT Character.MAX_CODE_POINT)</td></tr> 117 * <tr><th style="vertical-align:top; font-weight: normal" id="unicode_name"><code>\N</code><i>name</i><code></code></th> 118 * <td headers="matches characters unicode_name">The character with Unicode character name <i>‘name‘</i></td></tr> 119 * <tr><th style="vertical-align:top; font-weight:normal" id="tab">@code \t</th> 120 * <td headers="matches characters tab">The tab character (<code>‘\u0009‘</code>)</td></tr> 121 * <tr><th style="vertical-align:top; font-weight:normal" id="newline">@code \n</th> 122 * <td headers="matches characters newline">The newline (line feed) character (<code>‘\u000A‘</code>)</td></tr> 123 * <tr><th style="vertical-align:top; font-weight:normal" id="return">@code \r</th> 124 * <td headers="matches characters return">The carriage-return character (<code>‘\u000D‘</code>)</td></tr> 125 * <tr><th style="vertical-align:top; font-weight:normal" id="form_feed">@code \f</th> 126 * <td headers="matches characters form_feed">The form-feed character (<code>‘\u000C‘</code>)</td></tr> 127 * <tr><th style="vertical-align:top; font-weight:normal" id="bell">@code \a</th> 128 * <td headers="matches characters bell">The alert (bell) character (<code>‘\u0007‘</code>)</td></tr> 129 * <tr><th style="vertical-align:top; font-weight:normal" id="escape">@code \e</th> 130 * <td headers="matches characters escape">The escape character (<code>‘\u001B‘</code>)</td></tr> 131 * <tr><th style="vertical-align:top; font-weight:normal" id="ctrl_x">@code \c<i>x</i></th> 132 * <td headers="matches characters ctrl_x">The control character corresponding to <i>x</i></td></tr> 133 * 134 * <tr><th colspan="2" style="padding-top:20px" id="classes">Character classes</th></tr> 135 * 136 * <tr><th style="vertical-align:top; font-weight:normal" id="simple">@code [abc]</th> 137 * <td headers="matches classes simple">@code a, @code b, or @code c (simple class)</td></tr> 138 * <tr><th style="vertical-align:top; font-weight:normal" id="negation">@code [^abc]</th> 139 * <td headers="matches classes negation">Any character except @code a, @code b, or @code c (negation)</td></tr> 140 * <tr><th style="vertical-align:top; font-weight:normal" id="range">@code [a-zA-Z]</th> 141 * <td headers="matches classes range">@code a through @code z 142 * or @code A through @code Z, inclusive (range)</td></tr> 143 * <tr><th style="vertical-align:top; font-weight:normal" id="union">@code [a-d[m-p]]</th> 144 * <td headers="matches classes union">@code a through @code d, 145 * or @code m through @code p: @code [a-dm-p] (union)</td></tr> 146 * <tr><th style="vertical-align:top; font-weight:normal" id="intersection">@code [a-z&&[def]]</th> 147 * <td headers="matches classes intersection">@code d, @code e, or @code f (intersection)</tr> 148 * <tr><th style="vertical-align:top; font-weight:normal" id="subtraction1">@code [a-z&&[^bc]]</th> 149 * <td headers="matches classes subtraction1">@code a through @code z, 150 * except for @code b and @code c: @code [ad-z] (subtraction)</td></tr> 151 * <tr><th style="vertical-align:top; font-weight:normal" id="subtraction2">@code [a-z&&[^m-p]]</th> 152 * <td headers="matches classes subtraction2">@code a through @code z, 153 * and not @code m through @code p: @code [a-lq-z](subtraction)</td></tr> 154 * 155 * <tr><th colspan="2" style="padding-top:20px" id="predef">Predefined character classes</th></tr> 156 * 157 * <tr><th style="vertical-align:top; font-weight:normal" id="any">@code .</th> 158 * <td headers="matches predef any">Any character (may or may not match <a href="#lt">line terminators</a>)</td></tr> 159 * <tr><th style="vertical-align:top; font-weight:normal" id="digit">@code \d</th> 160 * <td headers="matches predef digit">A digit: @code [0-9]</td></tr> 161 * <tr><th style="vertical-align:top; font-weight:normal" id="non_digit">@code \D</th> 162 * <td headers="matches predef non_digit">A non-digit: @code [^0-9]</td></tr> 163 * <tr><th style="vertical-align:top; font-weight:normal" id="horiz_white">@code \h</th> 164 * <td headers="matches predef horiz_white">A horizontal whitespace character: 165 * <code>[ \t\xA0\u1680\u180e\u2000-\u200a\u202f\u205f\u3000]</code></td></tr> 166 * <tr><th style="vertical-align:top; font-weight:normal" id="non_horiz_white">@code \H</th> 167 * <td headers="matches predef non_horiz_white">A non-horizontal whitespace character: @code [^\h]</td></tr> 168 * <tr><th style="vertical-align:top; font-weight:normal" id="white">@code \s</th> 169 * <td headers="matches predef white">A whitespace character: @code [ \t\n\x0B\f\r]</td></tr> 170 * <tr><th style="vertical-align:top; font-weight:normal" id="non_white">@code \S</th> 171 * <td headers="matches predef non_white">A non-whitespace character: @code [^\s]</td></tr> 172 * <tr><th style="vertical-align:top; font-weight:normal" id="vert_white">@code \v</th> 173 * <td headers="matches predef vert_white">A vertical whitespace character: <code>[\n\x0B\f\r\x85\u2028\u2029]</code> 174 * </td></tr> 175 * <tr><th style="vertical-align:top; font-weight:normal" id="non_vert_white">@code \V</th> 176 * <td headers="matches predef non_vert_white">A non-vertical whitespace character: @code [^\v]</td></tr> 177 * <tr><th style="vertical-align:top; font-weight:normal" id="word">@code \w</th> 178 * <td headers="matches predef word">A word character: @code [a-zA-Z_0-9]</td></tr> 179 * <tr><th style="vertical-align:top; font-weight:normal" id="non_word">@code \W</th> 180 * <td headers="matches predef non_word">A non-word character: @code [^\w]</td></tr> 181 * 182 * <tr><th colspan="2" style="padding-top:20px" id="posix"><b>POSIX character classes (US-ASCII only)</b></th></tr> 183 * 184 * <tr><th style="vertical-align:top; font-weight:normal" id="Lower">@code \pLower</th> 185 * <td headers="matches posix Lower">A lower-case alphabetic character: @code [a-z]</td></tr> 186 * <tr><th style="vertical-align:top; font-weight:normal" id="Upper">@code \pUpper</th> 187 * <td headers="matches posix Upper">An upper-case alphabetic character:@code [A-Z]</td></tr> 188 * <tr><th style="vertical-align:top; font-weight:normal" id="ASCII">@code \pASCII</th> 189 * <td headers="matches posix ASCII">All ASCII:@code [\x00-\x7F]</td></tr> 190 * <tr><th style="vertical-align:top; font-weight:normal" id="Alpha">@code \pAlpha</th> 191 * <td headers="matches posix Alpha">An alphabetic character:@code [\pLower\pUpper]</td></tr> 192 * <tr><th style="vertical-align:top; font-weight:normal" id="Digit">@code \pDigit</th> 193 * <td headers="matches posix Digit">A decimal digit: @code [0-9]</td></tr> 194 * <tr><th style="vertical-align:top; font-weight:normal" id="Alnum">@code \pAlnum</th> 195 * <td headers="matches posix Alnum">An alphanumeric character:@code [\pAlpha\pDigit]</td></tr> 196 * <tr><th style="vertical-align:top; font-weight:normal" id="Punct">@code \pPunct</th> 197 * <td headers="matches posix Punct">Punctuation: One of @code !"#$%&‘()*+,-./:;<=>?@[\]^_`|~</td></tr> 198 * <!-- @code [\!"#\$%&‘\(\)\*\+,\-\./:;\<=\>\?@\[\\\]\^_`\\|\~] 199 * @code [\X21-\X2F\X31-\X40\X5B-\X60\X7B-\X7E] --> 200 * <tr><th style="vertical-align:top; font-weight:normal" id="Graph">@code \pGraph</th> 201 * <td headers="matches posix Graph">A visible character: @code [\pAlnum\pPunct]</td></tr> 202 * <tr><th style="vertical-align:top; font-weight:normal" id="Print">@code \pPrint</th> 203 * <td headers="matches posix Print">A printable character: @code [\pGraph\x20]</td></tr> 204 * <tr><th style="vertical-align:top; font-weight:normal" id="Blank">@code \pBlank</th> 205 * <td headers="matches posix Blank">A space or a tab: @code [ \t]</td></tr> 206 * <tr><th style="vertical-align:top; font-weight:normal" id="Cntrl">@code \pCntrl</th> 207 * <td headers="matches posix Cntrl">A control character: @code [\x00-\x1F\x7F]</td></tr> 208 * <tr><th style="vertical-align:top; font-weight:normal" id="XDigit">@code \pXDigit</th> 209 * <td headers="matches posix XDigit">A hexadecimal digit: @code [0-9a-fA-F]</td></tr> 210 * <tr><th style="vertical-align:top; font-weight:normal" id="Space">@code \pSpace</th> 211 * <td headers="matches posix Space">A whitespace character: @code [ \t\n\x0B\f\r]</td></tr> 212 * 213 * <tr><th colspan="2" style="padding-top:20px" id="java">java.lang.Character classes (simple <a href="#jcc">java character type</a>)</th></tr> 214 * 215 * <tr><th style="vertical-align:top; font-weight:normal" id="javaLowerCase">@code \pjavaLowerCase</th> 216 * <td headers="matches java javaLowerCase">Equivalent to java.lang.Character.isLowerCase()</td></tr> 217 * <tr><th style="vertical-align:top; font-weight:normal" id="javaUpperCase">@code \pjavaUpperCase</th> 218 * <td headers="matches java javaUpperCase">Equivalent to java.lang.Character.isUpperCase()</td></tr> 219 * <tr><th style="vertical-align:top; font-weight:normal" id="javaWhitespace">@code \pjavaWhitespace</th> 220 * <td headers="matches java javaWhitespace">Equivalent to java.lang.Character.isWhitespace()</td></tr> 221 * <tr><th style="vertical-align:top; font-weight:normal" id="javaMirrored">@code \pjavaMirrored</th> 222 * <td headers="matches java javaMirrored">Equivalent to java.lang.Character.isMirrored()</td></tr> 223 * 224 * <tr><th colspan="2" style="padding-top:20px" id="unicode">Classes for Unicode scripts, blocks, categories and binary properties</th></tr> 225 * 226 * <tr><th style="vertical-align:top; font-weight:normal" id="IsLatin">@code \pIsLatin</th> 227 * <td headers="matches unicode IsLatin">A Latin script character (<a href="#usc">script</a>)</td></tr> 228 * <tr><th style="vertical-align:top; font-weight:normal" id="InGreek">@code \pInGreek</th> 229 * <td headers="matches unicode InGreek">A character in the Greek block (<a href="#ubc">block</a>)</td></tr> 230 * <tr><th style="vertical-align:top; font-weight:normal" id="Lu">@code \pLu</th> 231 * <td headers="matches unicode Lu">An uppercase letter (<a href="#ucc">category</a>)</td></tr> 232 * <tr><th style="vertical-align:top; font-weight:normal" id="IsAlphabetic">@code \pIsAlphabetic</th> 233 * <td headers="matches unicode IsAlphabetic">An alphabetic character (<a href="#ubpc">binary property</a>)</td></tr> 234 * <tr><th style="vertical-align:top; font-weight:normal" id="Sc">@code \pSc</th> 235 * <td headers="matches unicode Sc">A currency symbol</td></tr> 236 * <tr><th style="vertical-align:top; font-weight:normal" id="not_InGreek">@code \PInGreek</th> 237 * <td headers="matches unicode not_InGreek">Any character except one in the Greek block (negation)</td></tr> 238 * <tr><th style="vertical-align:top; font-weight:normal" id="not_uppercase">@code [\pL&&[^\pLu]]</th> 239 * <td headers="matches unicode not_uppercase">Any letter except an uppercase letter (subtraction)</td></tr> 240 * 241 * <tr><th colspan="2" style="padding-top:20px" id="bounds">Boundary matchers</th></tr> 242 * 243 * <tr><th style="vertical-align:top; font-weight:normal" id="begin_line">@code ^</th> 244 * <td headers="matches bounds begin_line">The beginning of a line</td></tr> 245 * <tr><th style="vertical-align:top; font-weight:normal" id="end_line">@code $</th> 246 * <td headers="matches bounds end_line">The end of a line</td></tr> 247 * <tr><th style="vertical-align:top; font-weight:normal" id="word_boundary">@code \b</th> 248 * <td headers="matches bounds word_boundary">A word boundary</td></tr> 249 * <tr><th style="vertical-align:top; font-weight:normal" id="grapheme_cluster_boundary">@code \bg</th> 250 * <td headers="matches bounds grapheme_cluster_boundary">A Unicode extended grapheme cluster boundary</td></tr> 251 * <tr><th style="vertical-align:top; font-weight:normal" id="non_word_boundary">@code \B</th> 252 * <td headers="matches bounds non_word_boundary">A non-word boundary</td></tr> 253 * <tr><th style="vertical-align:top; font-weight:normal" id="begin_input">@code \A</th> 254 * <td headers="matches bounds begin_input">The beginning of the input</td></tr> 255 * <tr><th style="vertical-align:top; font-weight:normal" id="end_prev_match">@code \G</th> 256 * <td headers="matches bounds end_prev_match">The end of the previous match</td></tr> 257 * <tr><th style="vertical-align:top; font-weight:normal" id="end_input_except_term">@code \Z</th> 258 * <td headers="matches bounds end_input_except_term">The end of the input but for the final 259 * <a href="#lt">terminator</a>, if any</td></tr> 260 * <tr><th style="vertical-align:top; font-weight:normal" id="end_input">@code \z</th> 261 * <td headers="matches bounds end_input">The end of the input</td></tr> 262 * 263 * <tr><th colspan="2" style="padding-top:20px" id="linebreak">Linebreak matcher</th></tr> 264 * 265 * <tr><th style="vertical-align:top; font-weight:normal" id="any_unicode_linebreak">@code \R</th> 266 * <td headers="matches linebreak any_unicode_linebreak">Any Unicode linebreak sequence, is equivalent to 267 * <code>\u000D\u000A|[\u000A\u000B\u000C\u000D\u0085\u2028\u2029] 268 * </code></td></tr> 269 * 270 * <tr><th colspan="2" style="padding-top:20px" id="grapheme">Unicode Extended Grapheme matcher</th></tr> 271 * 272 * <tr><th style="vertical-align:top; font-weight:normal" id="grapheme_any">@code \X</th> 273 * <td headers="matches grapheme grapheme_any">Any Unicode extended grapheme cluster</td></tr> 274 * 275 * <tr><th colspan="2" style="padding-top:20px" id="greedy">Greedy quantifiers</th></tr> 276 * 277 * <tr><th style="vertical-align:top; font-weight:normal" id="greedy_once_or_not"><i>X</i>@code ?</th> 278 * <td headers="matches greedy greedy_once_or_not"><i>X</i>, once or not at all</td></tr> 279 * <tr><th style="vertical-align:top; font-weight:normal" id="greedy_zero_or_more"><i>X</i>@code *</th> 280 * <td headers="matches greedy greedy_zero_or_more"><i>X</i>, zero or more times</td></tr> 281 * <tr><th style="vertical-align:top; font-weight:normal" id="greedy_one_or_more"><i>X</i>@code +</th> 282 * <td headers="matches greedy greedy_one_or_more"><i>X</i>, one or more times</td></tr> 283 * <tr><th style="vertical-align:top; font-weight:normal" id="greedy_exactly"><i>X</i><code></code><i>n</i><code></code></th> 284 * <td headers="matches greedy greedy_exactly"><i>X</i>, exactly <i>n</i> times</td></tr> 285 * <tr><th style="vertical-align:top; font-weight:normal" id="greedy_at_least"><i>X</i><code></code><i>n</i>@code ,</th> 286 * <td headers="matches greedy greedy_at_least"><i>X</i>, at least <i>n</i> times</td></tr> 287 * <tr><th style="vertical-align:top; font-weight:normal" id="greedy_at_least_up_to"><i>X</i><code></code><i>n</i>@code ,<i>m</i><code></code></th> 288 * <td headers="matches greedy greedy_at_least_up_to"><i>X</i>, at least <i>n</i> but not more than <i>m</i> times</td></tr> 289 * 290 * <tr><th colspan="2" style="padding-top:20px" id="reluc">Reluctant quantifiers</th></tr> 291 * 292 * <tr><th style="vertical-align:top; font-weight:normal" id="reluc_once_or_not"><i>X</i>@code ??</th> 293 * <td headers="matches reluc reluc_once_or_not"><i>X</i>, once or not at all</td></tr> 294 * <tr><th style="vertical-align:top; font-weight:normal" id="reluc_zero_or_more"><i>X</i>@code *?</th> 295 * <td headers="matches reluc reluc_zero_or_more"><i>X</i>, zero or more times</td></tr> 296 * <tr><th style="vertical-align:top; font-weight:normal" id="reluc_one_or_more"><i>X</i>@code +?</th> 297 * <td headers="matches reluc reluc_one_or_more"><i>X</i>, one or more times</td></tr> 298 * <tr><th style="vertical-align:top; font-weight:normal" id="reluc_exactly"><i>X</i><code></code><i>n</i><code>?</code></th> 299 * <td headers="matches reluc reluc_exactly"><i>X</i>, exactly <i>n</i> times</td></tr> 300 * <tr><th style="vertical-align:top; font-weight:normal" id="reluc_at_least"><i>X</i><code></code><i>n</i><code>,?</code></th> 301 * <td headers="matches reluc reluc_at_least"><i>X</i>, at least <i>n</i> times</td></tr> 302 * <tr><th style="vertical-align:top; font-weight:normal" id="reluc_at_least_up_to"><i>X</i><code></code><i>n</i>@code ,<i>m</i><code>?</code></th> 303 * <td headers="matches reluc reluc_at_least_up_to"><i>X</i>, at least <i>n</i> but not more than <i>m</i> times</td></tr> 304 * 305 * <tr><th colspan="2" style="padding-top:20px" id="poss">Possessive quantifiers</th></tr> 306 * 307 * <tr><th style="vertical-align:top; font-weight:normal" id="poss_once_or_not"><i>X</i>@code ?+</th> 308 * <td headers="matches poss poss_once_or_not"><i>X</i>, once or not at all</td></tr> 309 * <tr><th style="vertical-align:top; font-weight:normal" id="poss_zero_or_more"><i>X</i>@code *+</th> 310 * <td headers="matches poss poss_zero_or_more"><i>X</i>, zero or more times</td></tr> 311 * <tr><th style="vertical-align:top; font-weight:normal" id="poss_one_or_more"><i>X</i>@code ++</th> 312 * <td headers="matches poss poss_one_or_more"><i>X</i>, one or more times</td></tr> 313 * <tr><th style="vertical-align:top; font-weight:normal" id="poss_exactly"><i>X</i><code></code><i>n</i><code>+</code></th> 314 * <td headers="matches poss poss_exactly"><i>X</i>, exactly <i>n</i> times</td></tr> 315 * <tr><th style="vertical-align:top; font-weight:normal" id="poss_at_least"><i>X</i><code></code><i>n</i><code>,+</code></th> 316 * <td headers="matches poss poss_at_least"><i>X</i>, at least <i>n</i> times</td></tr> 317 * <tr><th style="vertical-align:top; font-weight:normal" id="poss_at_least_up_to"><i>X</i><code></code><i>n</i>@code ,<i>m</i><code>+</code></th> 318 * <td headers="matches poss poss_at_least_up_to"><i>X</i>, at least <i>n</i> but not more than <i>m</i> times</td></tr> 319 * 320 * <tr><th colspan="2" style="padding-top:20px" id="logical">Logical operators</th></tr> 321 * 322 * <tr><th style="vertical-align:top; font-weight:normal" id="concat"><i>XY</i></th> 323 * <td headers="matches logical concat"><i>X</i> followed by <i>Y</i></td></tr> 324 * <tr><th style="vertical-align:top; font-weight:normal" id="alternate"><i>X</i>@code |<i>Y</i></th> 325 * <td headers="matches logical alternate">Either <i>X</i> or <i>Y</i></td></tr> 326 * <tr><th style="vertical-align:top; font-weight:normal" id="group">@code (<i>X</i>@code )</th> 327 * <td headers="matches logical group">X, as a <a href="#cg">capturing group</a></td></tr> 328 * 329 * <tr><th colspan="2" style="padding-top:20px" id="backref">Back references</th></tr> 330 * 331 * <tr><th style="vertical-align:top; font-weight:normal" id="back_nth">@code \<i>n</i></th> 332 * <td headers="matches backref back_nth">Whatever the <i>n</i><sup>th</sup> 333 * <a href="#cg">capturing group</a> matched</td></tr> 334 * <tr><th style="vertical-align:top; font-weight:normal" id="back_named">@code \<i>k</i><<i>name</i>></th> 335 * <td headers="matches backref back_named">Whatever the 336 * <a href="#groupname">named-capturing group</a> "name" matched</td></tr> 337 * 338 * <tr><th colspan="2" style="padding-top:20px" id="quote">Quotation</th></tr> 339 * 340 * <tr><th style="vertical-align:top; font-weight:normal" id="quote_follow">@code \</th> 341 * <td headers="matches quote quote_follow">Nothing, but quotes the following character</td></tr> 342 * <tr><th style="vertical-align:top; font-weight:normal" id="quote_begin">@code \Q</th> 343 * <td headers="matches quote quote_begin">Nothing, but quotes all characters until @code \E</td></tr> 344 * <tr><th style="vertical-align:top; font-weight:normal" id="quote_end">@code \E</th> 345 * <td headers="matches quote quote_end">Nothing, but ends quoting started by @code \Q</td></tr> 346 * <!-- Metachars: !$()*+.<>?[\]^| --> 347 * 348 * <tr><th colspan="2" style="padding-top:20px" id="special">Special constructs (named-capturing and non-capturing)</th></tr> 349 * 350 * <tr><th style="vertical-align:top; font-weight:normal" id="named_group"><code>(?<<a href="#groupname">name</a>></code><i>X</i>@code )</th> 351 * <td headers="matches special named_group"><i>X</i>, as a named-capturing group</td></tr> 352 * <tr><th style="vertical-align:top; font-weight:normal" id="non_capture_group">@code (?:<i>X</i>@code )</th> 353 * <td headers="matches special non_capture_group"><i>X</i>, as a non-capturing group</td></tr> 354 * <tr><th style="vertical-align:top; font-weight:normal" id="flags"><code>(?idmsuxU-idmsuxU) </code></th> 355 * <td headers="matches special flags">Nothing, but turns match flags <a href="#CASE_INSENSITIVE">i</a> 356 * <a href="#UNIX_LINES">d</a> <a href="#MULTILINE">m</a> <a href="#DOTALL">s</a> 357 * <a href="#UNICODE_CASE">u</a> <a href="#COMMENTS">x</a> <a href="#UNICODE_CHARACTER_CLASS">U</a> 358 * on - off</td></tr> 359 * <tr><th style="vertical-align:top; font-weight:normal" id="non_capture_group_flags"><code>(?idmsux-idmsux:</code><i>X</i>@code ) </th> 360 * <td headers="matches special non_capture_group_flags"><i>X</i>, as a <a href="#cg">non-capturing group</a> with the 361 * given flags <a href="#CASE_INSENSITIVE">i</a> <a href="#UNIX_LINES">d</a> 362 * <a href="#MULTILINE">m</a> <a href="#DOTALL">s</a> <a href="#UNICODE_CASE">u</a > 363 * <a href="#COMMENTS">x</a> on - off</td></tr> 364 * <tr><th style="vertical-align:top; font-weight:normal" id="pos_lookahead">@code (?=<i>X</i>@code )</th> 365 * <td headers="matches special pos_lookahead"><i>X</i>, via zero-width positive lookahead</td></tr> 366 * <tr><th style="vertical-align:top; font-weight:normal" id="neg_lookahead">@code (?!<i>X</i>@code )</th> 367 * <td headers="matches special neg_lookahead"><i>X</i>, via zero-width negative lookahead</td></tr> 368 * <tr><th style="vertical-align:top; font-weight:normal" id="pos_lookbehind">@code (?<=<i>X</i>@code )</th> 369 * <td headers="matches special pos_lookbehind"><i>X</i>, via zero-width positive lookbehind</td></tr> 370 * <tr><th style="vertical-align:top; font-weight:normal" id="neg_lookbehind">@code (?<!<i>X</i>@code )</th> 371 * <td headers="matches special neg_lookbehind"><i>X</i>, via zero-width negative lookbehind</td></tr> 372 * <tr><th style="vertical-align:top; font-weight:normal" id="indep_non_capture_group">@code (?><i>X</i>@code )</th> 373 * <td headers="matches special indep_non_capture_group"><i>X</i>, as an independent, non-capturing group</td></tr> 374 * 375 * </tbody> 376 * </table> 377 * 378 * <hr> 379 * 380 * 381 * <h3><a id="bs">Backslashes, escapes, and quoting</a></h3> 382 * 383 * <p> The backslash character (@code ‘\‘) serves to introduce escaped 384 * constructs, as defined in the table above, as well as to quote characters 385 * that otherwise would be interpreted as unescaped constructs. Thus the 386 * expression @code \\ matches a single backslash and <code>\</code> matches a 387 * left brace. 388 * 389 * <p> It is an error to use a backslash prior to any alphabetic character that 390 * does not denote an escaped construct; these are reserved for future 391 * extensions to the regular-expression language. A backslash may be used 392 * prior to a non-alphabetic character regardless of whether that character is 393 * part of an unescaped construct. 394 * 395 * <p> Backslashes within string literals in Java source code are interpreted 396 * as required by 397 * <cite>The Java™ Language Specification</cite> 398 * as either Unicode escapes (section 3.3) or other character escapes (section 3.10.6) 399 * It is therefore necessary to double backslashes in string 400 * literals that represent regular expressions to protect them from 401 * interpretation by the Java bytecode compiler. The string literal 402 * <code>"\b"</code>, for example, matches a single backspace character when 403 * interpreted as a regular expression, while @code "\\b" matches a 404 * word boundary. The string literal @code "\(hello\)" is illegal 405 * and leads to a compile-time error; in order to match the string 406 * @code (hello) the string literal @code "\\(hello\\)" 407 * must be used. 408 * 409 * <h3><a id="cc">Character Classes</a></h3> 410 * 411 * <p> Character classes may appear within other character classes, and 412 * may be composed by the union operator (implicit) and the intersection 413 * operator (@code &&). 414 * The union operator denotes a class that contains every character that is 415 * in at least one of its operand classes. The intersection operator 416 * denotes a class that contains every character that is in both of its 417 * operand classes. 418 * 419 * <p> The precedence of character-class operators is as follows, from 420 * highest to lowest: 421 * 422 * <table class="striped" style="margin-left: 2em;"> 423 * <caption style="display:none">Precedence of character class operators.</caption> 424 * <thead> 425 * <tr><th scope="col">Precedence<th scope="col">Name<th scope="col">Example 426 * </thead> 427 * <tbody> 428 * <tr><th scope="row">1</th> 429 * <td>Literal escape </td> 430 * <td>@code \x</td></tr> 431 * <tr><th scope="row">2</th> 432 * <td>Grouping</td> 433 * <td>@code [...]</td></tr> 434 * <tr><th scope="row">3</th> 435 * <td>Range</td> 436 * <td>@code a-z</td></tr> 437 * <tr><th scope="row">4</th> 438 * <td>Union</td> 439 * <td>@code [a-e][i-u]</td></tr> 440 * <tr><th scope="row">5</th> 441 * <td>Intersection</td> 442 * <td>@code [a-z&&[aeiou]]</td></tr> 443 * </tbody> 444 * </table> 445 * 446 * <p> Note that a different set of metacharacters are in effect inside 447 * a character class than outside a character class. For instance, the 448 * regular expression @code . loses its special meaning inside a 449 * character class, while the expression @code - becomes a range 450 * forming metacharacter. 451 * 452 * <h3><a id="lt">Line terminators</a></h3> 453 * 454 * <p> A <i>line terminator</i> is a one- or two-character sequence that marks 455 * the end of a line of the input character sequence. The following are 456 * recognized as line terminators: 457 * 458 * <ul> 459 * 460 * <li> A newline (line feed) character (@code ‘\n‘), 461 * 462 * <li> A carriage-return character followed immediately by a newline 463 * character (@code "\r\n"), 464 * 465 * <li> A standalone carriage-return character (@code ‘\r‘), 466 * 467 * <li> A next-line character (<code>‘\u0085‘</code>), 468 * 469 * <li> A line-separator character (<code>‘\u2028‘</code>), or 470 * 471 * <li> A paragraph-separator character (<code>‘\u2029‘</code>). 472 * 473 * </ul> 474 * <p>If @link #UNIX_LINES mode is activated, then the only line terminators 475 * recognized are newline characters. 476 * 477 * <p> The regular expression @code . matches any character except a line 478 * terminator unless the @link #DOTALL flag is specified. 479 * 480 * <p> By default, the regular expressions @code ^ and @code $ ignore 481 * line terminators and only match at the beginning and the end, respectively, 482 * of the entire input sequence. If @link #MULTILINE mode is activated then 483 * @code ^ matches at the beginning of input and after any line terminator 484 * except at the end of input. When in @link #MULTILINE mode @code $ 485 * matches just before a line terminator or the end of the input sequence. 486 * 487 * <h3><a id="cg">Groups and capturing</a></h3> 488 * 489 * <h4><a id="gnumber">Group number</a></h4> 490 * <p> Capturing groups are numbered by counting their opening parentheses from 491 * left to right. In the expression @code ((A)(B(C))), for example, there 492 * are four such groups: </p> 493 * 494 * <ol style="margin-left:2em;"> 495 * <li> @code ((A)(B(C))) 496 * <li> @code (A) 497 * <li> @code (B(C)) 498 * <li> @code (C) 499 * </ol> 500 * 501 * <p> Group zero always stands for the entire expression. 502 * 503 * <p> Capturing groups are so named because, during a match, each subsequence 504 * of the input sequence that matches such a group is saved. The captured 505 * subsequence may be used later in the expression, via a back reference, and 506 * may also be retrieved from the matcher once the match operation is complete. 507 * 508 * <h4><a id="groupname">Group name</a></h4> 509 * <p>A capturing group can also be assigned a "name", a @code named-capturing group, 510 * and then be back-referenced later by the "name". Group names are composed of 511 * the following characters. The first character must be a @code letter. 512 * 513 * <ul> 514 * <li> The uppercase letters @code ‘A‘ through @code ‘Z‘ 515 * (<code>‘\u0041‘</code> through <code>‘\u005a‘</code>), 516 * <li> The lowercase letters @code ‘a‘ through @code ‘z‘ 517 * (<code>‘\u0061‘</code> through <code>‘\u007a‘</code>), 518 * <li> The digits @code ‘0‘ through @code ‘9‘ 519 * (<code>‘\u0030‘</code> through <code>‘\u0039‘</code>), 520 * </ul> 521 * 522 * <p> A @code named-capturing group is still numbered as described in 523 * <a href="#gnumber">Group number</a>. 524 * 525 * <p> The captured input associated with a group is always the subsequence 526 * that the group most recently matched. If a group is evaluated a second time 527 * because of quantification then its previously-captured value, if any, will 528 * be retained if the second evaluation fails. Matching the string 529 * @code "aba" against the expression @code (a(b)?)+, for example, leaves 530 * group two set to @code "b". All captured input is discarded at the 531 * beginning of each match. 532 * 533 * <p> Groups beginning with @code (? are either pure, <i>non-capturing</i> groups 534 * that do not capture text and do not count towards the group total, or 535 * <i>named-capturing</i> group. 536 * 537 * <h3> Unicode support </h3> 538 * 539 * <p> This class is in conformance with Level 1 of <a 540 * href="http://www.unicode.org/reports/tr18/"><i>Unicode Technical 541 * Standard #18: Unicode Regular Expression</i></a>, plus RL2.1 542 * Canonical Equivalents. 543 * <p> 544 * <b>Unicode escape sequences</b> such as <code>\u2014</code> in Java source code 545 * are processed as described in section 3.3 of 546 * <cite>The Java™ Language Specification</cite>. 547 * Such escape sequences are also implemented directly by the regular-expression 548 * parser so that Unicode escapes can be used in expressions that are read from 549 * files or from the keyboard. Thus the strings <code>"\u2014"</code> and 550 * @code "\\u2014", while not equal, compile into the same pattern, which 551 * matches the character with hexadecimal value @code 0x2014. 552 * <p> 553 * A Unicode character can also be represented by using its <b>Hex notation</b> 554 * (hexadecimal code point value) directly as described in construct 555 * <code>\x...</code>, for example a supplementary character U+2011F can be 556 * specified as <code>\x2011F</code>, instead of two consecutive Unicode escape 557 * sequences of the surrogate pair <code>\uD840</code><code>\uDD1F</code>. 558 * <p> 559 * <b>Unicode character names</b> are supported by the named character construct 560 * <code>\N</code>...<code></code>, for example, <code>\NWHITE SMILING FACE</code> 561 * specifies character <code>\u263A</code>. The character names supported 562 * by this class are the valid Unicode character names matched by 563 * @link java.lang.Character#codePointOf(String) Character.codePointOf(name). 564 * <p> 565 * <a href="http://www.unicode.org/reports/tr18/#Default_Grapheme_Clusters"> 566 * <b>Unicode extended grapheme clusters</b></a> are supported by the grapheme 567 * cluster matcher @code \X and the corresponding boundary matcher @code \bg. 568 * <p> 569 * Unicode scripts, blocks, categories and binary properties are written with 570 * the @code \p and @code \P constructs as in Perl. 571 * <code>\p</code><i>prop</i><code></code> matches if 572 * the input has the property <i>prop</i>, while <code>\P</code><i>prop</i><code></code> 573 * does not match if the input has that property. 574 * <p> 575 * Scripts, blocks, categories and binary properties can be used both inside 576 * and outside of a character class. 577 * 578 * <p> 579 * <b><a id="usc">Scripts</a></b> are specified either with the prefix @code Is, as in 580 * @code IsHiragana, or by using the @code script keyword (or its short 581 * form @code sc) as in @code script=Hiragana or @code sc=Hiragana. 582 * <p> 583 * The script names supported by @code Pattern are the valid script names 584 * accepted and defined by 585 * @link java.lang.Character.UnicodeScript#forName(String) UnicodeScript.forName. 586 * 587 * <p> 588 * <b><a id="ubc">Blocks</a></b> are specified with the prefix @code In, as in 589 * @code InMongolian, or by using the keyword @code block (or its short 590 * form @code blk) as in @code block=Mongolian or @code blk=Mongolian. 591 * <p> 592 * The block names supported by @code Pattern are the valid block names 593 * accepted and defined by 594 * @link java.lang.Character.UnicodeBlock#forName(String) UnicodeBlock.forName. 595 * <p> 596 * 597 * <b><a id="ucc">Categories</a></b> may be specified with the optional prefix @code Is: 598 * Both @code \pL and @code \pIsL denote the category of Unicode 599 * letters. Same as scripts and blocks, categories can also be specified 600 * by using the keyword @code general_category (or its short form 601 * @code gc) as in @code general_category=Lu or @code gc=Lu. 602 * <p> 603 * The supported categories are those of 604 * <a href="http://www.unicode.org/unicode/standard/standard.html"> 605 * <i>The Unicode Standard</i></a> in the version specified by the 606 * @link java.lang.Character Character class. The category names are those 607 * defined in the Standard, both normative and informative. 608 * <p> 609 * 610 * <b><a id="ubpc">Binary properties</a></b> are specified with the prefix @code Is, as in 611 * @code IsAlphabetic. The supported binary properties by @code Pattern 612 * are 613 * <ul> 614 * <li> Alphabetic 615 * <li> Ideographic 616 * <li> Letter 617 * <li> Lowercase 618 * <li> Uppercase 619 * <li> Titlecase 620 * <li> Punctuation 621 * <Li> Control 622 * <li> White_Space 623 * <li> Digit 624 * <li> Hex_Digit 625 * <li> Join_Control 626 * <li> Noncharacter_Code_Point 627 * <li> Assigned 628 * </ul> 629 * <p> 630 * The following <b>Predefined Character classes</b> and <b>POSIX character classes</b> 631 * are in conformance with the recommendation of <i>Annex C: Compatibility Properties</i> 632 * of <a href="http://www.unicode.org/reports/tr18/"><i>Unicode Regular Expression 633 * </i></a>, when @link #UNICODE_CHARACTER_CLASS flag is specified. 634 * 635 * <table class="striped"> 636 * <caption style="display:none">predefined and posix character classes in Unicode mode</caption> 637 * <thead> 638 * <tr> 639 * <th scope="col" id="predef_classes">Classes</th> 640 * <th scope="col" id="predef_matches">Matches</th> 641 * </tr> 642 * </thead> 643 * <tbody> 644 * <tr><th scope="row">@code \pLower</th> 645 * <td>A lowercase character:@code \pIsLowercase</td></tr> 646 * <tr><th scope="row">@code \pUpper</th> 647 * <td>An uppercase character:@code \pIsUppercase</td></tr> 648 * <tr><th scope="row">@code \pASCII</th> 649 * <td>All ASCII:@code [\x00-\x7F]</td></tr> 650 * <tr><th scope="row">@code \pAlpha</th> 651 * <td>An alphabetic character:@code \pIsAlphabetic</td></tr> 652 * <tr><th scope="row">@code \pDigit</th> 653 * <td>A decimal digit character:@code \pIsDigit</td></tr> 654 * <tr><th scope="row">@code \pAlnum</th> 655 * <td>An alphanumeric character:@code [\pIsAlphabetic\pIsDigit]</td></tr> 656 * <tr><th scope="row">@code \pPunct</th> 657 * <td>A punctuation character:@code \pIsPunctuation</td></tr> 658 * <tr><th scope="row">@code \pGraph</th> 659 * <td>A visible character: @code [^\pIsWhite_Space\pgc=Cc\pgc=Cs\pgc=Cn]</td></tr> 660 * <tr><th scope="row">@code \pPrint</th> 661 * <td>A printable character: @code [\pGraph\pBlank&&[^\pCntrl]]</td></tr> 662 * <tr><th scope="row">@code \pBlank</th> 663 * <td>A space or a tab: @code [\pIsWhite_Space&&[^\pgc=Zl\pgc=Zp\x0a\x0b\x0c\x0d\x85]]</td></tr> 664 * <tr><th scope="row">@code \pCntrl</th> 665 * <td>A control character: @code \pgc=Cc</td></tr> 666 * <tr><th scope="row">@code \pXDigit</th> 667 * <td>A hexadecimal digit: @code [\pgc=Nd\pIsHex_Digit]</td></tr> 668 * <tr><th scope="row">@code \pSpace</th> 669 * <td>A whitespace character:@code \pIsWhite_Space</td></tr> 670 * <tr><th scope="row">@code \d</th> 671 * <td>A digit: @code \pIsDigit</td></tr> 672 * <tr><th scope="row">@code \D</th> 673 * <td>A non-digit: @code [^\d]</td></tr> 674 * <tr><th scope="row">@code \s</th> 675 * <td>A whitespace character: @code \pIsWhite_Space</td></tr> 676 * <tr><th scope="row">@code \S</th> 677 * <td>A non-whitespace character: @code [^\s]</td></tr> 678 * <tr><th scope="row">@code \w</th> 679 * <td>A word character: @code [\pAlpha\pgc=Mn\pgc=Me\pgc=Mc\pDigit\pgc=Pc\pIsJoin_Control]</td></tr> 680 * <tr><th scope="row">@code \W</th> 681 * <td>A non-word character: @code [^\w]</td></tr> 682 * </tbody> 683 * </table> 684 * <p> 685 * <a id="jcc"> 686 * Categories that behave like the java.lang.Character 687 * boolean is<i>methodname</i> methods (except for the deprecated ones) are 688 * available through the same <code>\p</code><i>prop</i><code></code> syntax where 689 * the specified property has the name <code>java<i>methodname</i></code></a>. 690 * 691 * <h3> Comparison to Perl 5 </h3> 692 * 693 * <p>The @code Pattern engine performs traditional NFA-based matching 694 * with ordered alternation as occurs in Perl 5. 695 * 696 * <p> Perl constructs not supported by this class: </p> 697 * 698 * <ul> 699 * <li><p> The backreference constructs, <code>\g</code><i>n</i><code></code> for 700 * the <i>n</i><sup>th</sup><a href="#cg">capturing group</a> and 701 * <code>\g</code><i>name</i><code></code> for 702 * <a href="#groupname">named-capturing group</a>. 703 * </p></li> 704 * 705 * <li><p> The conditional constructs 706 * @code (?(<i>condition</i>@code )<i>X</i>@code ) and 707 * @code (?(<i>condition</i>@code )<i>X</i>@code |<i>Y</i>@code ), 708 * </p></li> 709 * 710 * <li><p> The embedded code constructs <code>(?</code><i>code</i><code>)</code> 711 * and <code>(??</code><i>code</i><code>)</code>,</p></li> 712 * 713 * <li><p> The embedded comment syntax @code (?#comment), and </p></li> 714 * 715 * <li><p> The preprocessing operations @code \l <code>\u</code>, 716 * @code \L, and @code \U. </p></li> 717 * 718 * </ul> 719 * 720 * <p> Constructs supported by this class but not by Perl: </p> 721 * 722 * <ul> 723 * 724 * <li><p> Character-class union and intersection as described 725 * <a href="#cc">above</a>.</p></li> 726 * 727 * </ul> 728 * 729 * <p> Notable differences from Perl: </p> 730 * 731 * <ul> 732 * 733 * <li><p> In Perl, @code \1 through @code \9 are always interpreted 734 * as back references; a backslash-escaped number greater than @code 9 is 735 * treated as a back reference if at least that many subexpressions exist, 736 * otherwise it is interpreted, if possible, as an octal escape. In this 737 * class octal escapes must always begin with a zero. In this class, 738 * @code \1 through @code \9 are always interpreted as back 739 * references, and a larger number is accepted as a back reference if at 740 * least that many subexpressions exist at that point in the regular 741 * expression, otherwise the parser will drop digits until the number is 742 * smaller or equal to the existing number of groups or it is one digit. 743 * </p></li> 744 * 745 * <li><p> Perl uses the @code g flag to request a match that resumes 746 * where the last match left off. This functionality is provided implicitly 747 * by the @link Matcher class: Repeated invocations of the @link 748 * Matcher#find find method will resume where the last match left off, 749 * unless the matcher is reset. </p></li> 750 * 751 * <li><p> In Perl, embedded flags at the top level of an expression affect 752 * the whole expression. In this class, embedded flags always take effect 753 * at the point at which they appear, whether they are at the top level or 754 * within a group; in the latter case, flags are restored at the end of the 755 * group just as in Perl. </p></li> 756 * 757 * </ul> 758 * 759 * 760 * <p> For a more precise description of the behavior of regular expression 761 * constructs, please see <a href="http://www.oreilly.com/catalog/regex3/"> 762 * <i>Mastering Regular Expressions, 3nd Edition</i>, Jeffrey E. F. Friedl, 763 * O‘Reilly and Associates, 2006.</a> 764 * </p> 765 * 766 * @see java.lang.String#split(String, int) 767 * @see java.lang.String#split(String) 768 * 769 * @author Mike McCloskey 770 * @author Mark Reinhold 771 * @author JSR-51 Expert Group 772 * @since 1.4 773 * @spec JSR-51 774 */ 775 776 public final class Pattern 777 implements java.io.Serializable 778 779 780 /** 781 * Regular expression modifier values. Instead of being passed as 782 * arguments, they can also be passed as inline modifiers. 783 * For example, the following statements have the same effect. 784 * <pre> 785 * Pattern p1 = Pattern.compile("abc", Pattern.CASE_INSENSITIVE|Pattern.MULTILINE); 786 * Pattern p2 = Pattern.compile("(?im)abc", 0); 787 * </pre> 788 */ 789 790 /** 791 * Enables Unix lines mode. 792 * 793 * <p> In this mode, only the @code ‘\n‘ line terminator is recognized 794 * in the behavior of @code ., @code ^, and @code $. 795 * 796 * <p> Unix lines mode can also be enabled via the embedded flag 797 * expression @code (?d). 798 */ 799 public static final int UNIX_LINES = 0x01; 800 801 /** 802 * Enables case-insensitive matching. 803 * 804 * <p> By default, case-insensitive matching assumes that only characters 805 * in the US-ASCII charset are being matched. Unicode-aware 806 * case-insensitive matching can be enabled by specifying the @link 807 * #UNICODE_CASE flag in conjunction with this flag. 808 * 809 * <p> Case-insensitive matching can also be enabled via the embedded flag 810 * expression @code (?i). 811 * 812 * <p> Specifying this flag may impose a slight performance penalty. </p> 813 */ 814 public static final int CASE_INSENSITIVE = 0x02; 815 816 /** 817 * Permits whitespace and comments in pattern. 818 * 819 * <p> In this mode, whitespace is ignored, and embedded comments starting 820 * with @code # are ignored until the end of a line. 821 * 822 * <p> Comments mode can also be enabled via the embedded flag 823 * expression @code (?x). 824 */ 825 public static final int COMMENTS = 0x04; 826 827 /** 828 * Enables multiline mode. 829 * 830 * <p> In multiline mode the expressions @code ^ and @code $ match 831 * just after or just before, respectively, a line terminator or the end of 832 * the input sequence. By default these expressions only match at the 833 * beginning and the end of the entire input sequence. 834 * 835 * <p> Multiline mode can also be enabled via the embedded flag 836 * expression @code (?m). </p> 837 */ 838 public static final int MULTILINE = 0x08; 839 840 /** 841 * Enables literal parsing of the pattern. 842 * 843 * <p> When this flag is specified then the input string that specifies 844 * the pattern is treated as a sequence of literal characters. 845 * Metacharacters or escape sequences in the input sequence will be 846 * given no special meaning. 847 * 848 * <p>The flags CASE_INSENSITIVE and UNICODE_CASE retain their impact on 849 * matching when used in conjunction with this flag. The other flags 850 * become superfluous. 851 * 852 * <p> There is no embedded flag character for enabling literal parsing. 853 * @since 1.5 854 */ 855 public static final int LITERAL = 0x10; 856 857 /** 858 * Enables dotall mode. 859 * 860 * <p> In dotall mode, the expression @code . matches any character, 861 * including a line terminator. By default this expression does not match 862 * line terminators. 863 * 864 * <p> Dotall mode can also be enabled via the embedded flag 865 * expression @code (?s). (The @code s is a mnemonic for 866 * "single-line" mode, which is what this is called in Perl.) </p> 867 */ 868 public static final int DOTALL = 0x20; 869 870 /** 871 * Enables Unicode-aware case folding. 872 * 873 * <p> When this flag is specified then case-insensitive matching, when 874 * enabled by the @link #CASE_INSENSITIVE flag, is done in a manner 875 * consistent with the Unicode Standard. By default, case-insensitive 876 * matching assumes that only characters in the US-ASCII charset are being 877 * matched. 878 * 879 * <p> Unicode-aware case folding can also be enabled via the embedded flag 880 * expression @code (?u). 881 * 882 * <p> Specifying this flag may impose a performance penalty. </p> 883 */ 884 public static final int UNICODE_CASE = 0x40; 885 886 /** 887 * Enables canonical equivalence. 888 * 889 * <p> When this flag is specified then two characters will be considered 890 * to match if, and only if, their full canonical decompositions match. 891 * The expression <code>"a\u030A"</code>, for example, will match the 892 * string <code>"\u00E5"</code> when this flag is specified. By default, 893 * matching does not take canonical equivalence into account. 894 * 895 * <p> There is no embedded flag character for enabling canonical 896 * equivalence. 897 * 898 * <p> Specifying this flag may impose a performance penalty. </p> 899 */ 900 public static final int CANON_EQ = 0x80; 901 902 /** 903 * Enables the Unicode version of <i>Predefined character classes</i> and 904 * <i>POSIX character classes</i>. 905 * 906 * <p> When this flag is specified then the (US-ASCII only) 907 * <i>Predefined character classes</i> and <i>POSIX character classes</i> 908 * are in conformance with 909 * <a href="http://www.unicode.org/reports/tr18/"><i>Unicode Technical 910 * Standard #18: Unicode Regular Expression</i></a> 911 * <i>Annex C: Compatibility Properties</i>. 912 * <p> 913 * The UNICODE_CHARACTER_CLASS mode can also be enabled via the embedded 914 * flag expression @code (?U). 915 * <p> 916 * The flag implies UNICODE_CASE, that is, it enables Unicode-aware case 917 * folding. 918 * <p> 919 * Specifying this flag may impose a performance penalty. </p> 920 * @since 1.7 921 */ 922 public static final int UNICODE_CHARACTER_CLASS = 0x100; 923 924 /** 925 * Contains all possible flags for compile(regex, flags). 926 */ 927 private static final int ALL_FLAGS = CASE_INSENSITIVE | MULTILINE | 928 DOTALL | UNICODE_CASE | CANON_EQ | UNIX_LINES | LITERAL | 929 UNICODE_CHARACTER_CLASS | COMMENTS; 930 931 /* Pattern has only two serialized components: The pattern string 932 * and the flags, which are all that is needed to recompile the pattern 933 * when it is deserialized. 934 */ 935 936 /** use serialVersionUID from Merlin b59 for interoperability */ 937 private static final long serialVersionUID = 5073258162644648461L; 938 939 /** 940 * The original regular-expression pattern string. 941 * 942 * @serial 943 */ 944 private String pattern; 945 946 /** 947 * The original pattern flags. 948 * 949 * @serial 950 */ 951 private int flags; 952 953 /** 954 * The temporary pattern flags used during compiling. The flags might be turn 955 * on and off by embedded flag. 956 */ 957 private transient int flags0; 958 959 /** 960 * Boolean indicating this Pattern is compiled; this is necessary in order 961 * to lazily compile deserialized Patterns. 962 */ 963 private transient volatile boolean compiled; 964 965 /** 966 * The normalized pattern string. 967 */ 968 private transient String normalizedPattern; 969 970 /** 971 * The starting point of state machine for the find operation. This allows 972 * a match to start anywhere in the input. 973 */ 974 transient Node root; 975 976 /** 977 * The root of object tree for a match operation. The pattern is matched 978 * at the beginning. This may include a find that uses BnM or a First 979 * node. 980 */ 981 transient Node matchRoot; 982 983 /** 984 * Temporary storage used by parsing pattern slice. 985 */ 986 transient int[] buffer; 987 988 /** 989 * A temporary storage used for predicate for double return. 990 */ 991 transient CharPredicate predicate; 992 993 /** 994 * Map the "name" of the "named capturing group" to its group id 995 * node. 996 */ 997 transient volatile Map<String, Integer> namedGroups; 998 999 /** 1000 * Temporary storage used while parsing group references. 1001 */ 1002 transient GroupHead[] groupNodes; 1003 1004 /** 1005 * Temporary storage used to store the top level closure nodes. 1006 */ 1007 transient List<Node> topClosureNodes; 1008 1009 /** 1010 * The number of top greedy closure nodes in this Pattern. Used by 1011 * matchers to allocate storage needed for a IntHashSet to keep the 1012 * beginning pos @code i of all failed match. 1013 */ 1014 transient int localTCNCount; 1015 1016 /* 1017 * Turn off the stop-exponential-backtracking optimization if there 1018 * is a group ref in the pattern. 1019 */ 1020 transient boolean hasGroupRef; 1021 1022 /** 1023 * Temporary null terminated code point array used by pattern compiling. 1024 */ 1025 private transient int[] temp; 1026 1027 /** 1028 * The number of capturing groups in this Pattern. Used by matchers to 1029 * allocate storage needed to perform a match. 1030 */ 1031 transient int capturingGroupCount; 1032 1033 /** 1034 * The local variable count used by parsing tree. Used by matchers to 1035 * allocate storage needed to perform a match. 1036 */ 1037 transient int localCount; 1038 1039 /** 1040 * Index into the pattern string that keeps track of how much has been 1041 * parsed. 1042 */ 1043 private transient int cursor; 1044 1045 /** 1046 * Holds the length of the pattern string. 1047 */ 1048 private transient int patternLength; 1049 1050 /** 1051 * If the Start node might possibly match supplementary characters. 1052 * It is set to true during compiling if 1053 * (1) There is supplementary char in pattern, or 1054 * (2) There is complement node of a "family" CharProperty 1055 */ 1056 private transient boolean hasSupplementary; 1057 1058 /** 1059 * Compiles the given regular expression into a pattern. 1060 * 1061 * @param regex 1062 * The expression to be compiled 1063 * @return the given regular expression compiled into a pattern 1064 * @throws PatternSyntaxException 1065 * If the expression‘s syntax is invalid 1066 */ 1067 public static Pattern compile(String regex) 1068 return new Pattern(regex, 0); 1069 1070 1071 /** 1072 * Compiles the given regular expression into a pattern with the given 1073 * flags. 1074 * 1075 * @param regex 1076 * The expression to be compiled 1077 * 1078 * @param flags 1079 * Match flags, a bit mask that may include 1080 * @link #CASE_INSENSITIVE, @link #MULTILINE, @link #DOTALL, 1081 * @link #UNICODE_CASE, @link #CANON_EQ, @link #UNIX_LINES, 1082 * @link #LITERAL, @link #UNICODE_CHARACTER_CLASS 1083 * and @link #COMMENTS 1084 * 1085 * @return the given regular expression compiled into a pattern with the given flags 1086 * @throws IllegalArgumentException 1087 * If bit values other than those corresponding to the defined 1088 * match flags are set in @code flags 1089 * 1090 * @throws PatternSyntaxException 1091 * If the expression‘s syntax is invalid 1092 */ 1093 public static Pattern compile(String regex, int flags) 1094 return new Pattern(regex, flags); 1095 1096 1097 /** 1098 * Returns the regular expression from which this pattern was compiled. 1099 * 1100 * @return The source of this pattern 1101 */ 1102 public String pattern() 1103 return pattern; 1104 1105 1106 /** 1107 * <p>Returns the string representation of this pattern. This 1108 * is the regular expression from which this pattern was 1109 * compiled.</p> 1110 * 1111 * @return The string representation of this pattern 1112 * @since 1.5 1113 */ 1114 public String toString() 1115 return pattern; 1116 1117 1118 /** 1119 * Creates a matcher that will match the given input against this pattern. 1120 * 1121 * @param input 1122 * The character sequence to be matched 1123 * 1124 * @return A new matcher for this pattern 1125 */ 1126 public Matcher matcher(CharSequence input) 1127 if (!compiled) 1128 synchronized(this) 1129 if (!compiled) 1130 compile(); 1131 1132 1133 Matcher m = new Matcher(this, input); 1134 return m; 1135 1136 1137 /** 1138 * Returns this pattern‘s match flags. 1139 * 1140 * @return The match flags specified when this pattern was compiled 1141 */ 1142 public int flags() 1143 return flags0; 1144 1145 1146 /** 1147 * Compiles the given regular expression and attempts to match the given 1148 * input against it. 1149 * 1150 * <p> An invocation of this convenience method of the form 1151 * 1152 * <blockquote><pre> 1153 * Pattern.matches(regex, input);</pre></blockquote> 1154 * 1155 * behaves in exactly the same way as the expression 1156 * 1157 * <blockquote><pre> 1158 * Pattern.compile(regex).matcher(input).matches()</pre></blockquote> 1159 * 1160 * <p> If a pattern is to be used multiple times, compiling it once and reusing 1161 * it will be more efficient than invoking this method each time. </p> 1162 * 1163 * @param regex 1164 * The expression to be compiled 1165 * 1166 * @param input 1167 * The character sequence to be matched 1168 * @return whether or not the regular expression matches on the input 1169 * @throws PatternSyntaxException 1170 * If the expression‘s syntax is invalid 1171 */ 1172 public static boolean matches(String regex, CharSequence input) 1173 Pattern p = Pattern.compile(regex); 1174 Matcher m = p.matcher(input); 1175 return m.matches(); 1176 1177 1178 /** 1179 * Splits the given input sequence around matches of this pattern. 1180 * 1181 * <p> The array returned by this method contains each substring of the 1182 * input sequence that is terminated by another subsequence that matches 1183 * this pattern or is terminated by the end of the input sequence. The 1184 * substrings in the array are in the order in which they occur in the 1185 * input. If this pattern does not match any subsequence of the input then 1186 * the resulting array has just one element, namely the input sequence in 1187 * string form. 1188 * 1189 * <p> When there is a positive-width match at the beginning of the input 1190 * sequence then an empty leading substring is included at the beginning 1191 * of the resulting array. A zero-width match at the beginning however 1192 * never produces such empty leading substring. 1193 * 1194 * <p> The @code limit parameter controls the number of times the 1195 * pattern is applied and therefore affects the length of the resulting 1196 * array. 1197 * <ul> 1198 * <li><p> 1199 * If the <i>limit</i> is positive then the pattern will be applied 1200 * at most <i>limit</i> - 1 times, the array‘s length will be 1201 * no greater than <i>limit</i>, and the array‘s last entry will contain 1202 * all input beyond the last matched delimiter.</p></li> 1203 * 1204 * <li><p> 1205 * If the <i>limit</i> is zero then the pattern will be applied as 1206 * many times as possible, the array can have any length, and trailing 1207 * empty strings will be discarded.</p></li> 1208 * 1209 * <li><p> 1210 * If the <i>limit</i> is negative then the pattern will be applied 1211 * as many times as possible and the array can have any length.</p></li> 1212 * </ul> 1213 * 1214 * <p> The input @code "boo:and:foo", for example, yields the following 1215 * results with these parameters: 1216 * 1217 * <table class="plain" style="margin-left:2em;"> 1218 * <caption style="display:none">Split example showing regex, limit, and result</caption> 1219 * <thead> 1220 * <tr> 1221 * <th scope="col">Regex</th> 1222 * <th scope="col">Limit</th> 1223 * <th scope="col">Result</th> 1224 * </tr> 1225 * </thead> 1226 * <tbody> 1227 * <tr><th scope="row" rowspan="3" style="font-weight:normal">:</th> 1228 * <th scope="row" style="font-weight:normal; text-align:right; padding-right:1em">2</th> 1229 * <td>@code "boo", "and:foo" </td></tr> 1230 * <tr><!-- : --> 1231 * <th scope="row" style="font-weight:normal; text-align:right; padding-right:1em">5</th> 1232 * <td>@code "boo", "and", "foo" </td></tr> 1233 * <tr><!-- : --> 1234 * <th scope="row" style="font-weight:normal; text-align:right; padding-right:1em">-2</th> 1235 * <td>@code "boo", "and", "foo" </td></tr> 1236 * <tr><th scope="row" rowspan="3" style="font-weight:normal">o</th> 1237 * <th scope="row" style="font-weight:normal; text-align:right; padding-right:1em">5</th> 1238 * <td>@code "b", "", ":and:f", "", "" </td></tr> 1239 * <tr><!-- o --> 1240 * <th scope="row" style="font-weight:normal; text-align:right; padding-right:1em">-2</th> 1241 * <td>@code "b", "", ":and:f", "", "" </td></tr> 1242 * <tr><!-- o --> 1243 * <th scope="row" style="font-weight:normal; text-align:right; padding-right:1em">0</th> 1244 * <td>@code "b", "", ":and:f" </td></tr> 1245 * </tbody> 1246 * </table> 1247 * 1248 * @param input 1249 * The character sequence to be split 1250 * 1251 * @param limit 1252 * The result threshold, as described above 1253 * 1254 * @return The array of strings computed by splitting the input 1255 * around matches of this pattern 1256 */ 1257 public String[] split(CharSequence input, int limit) 1258 int index = 0; 1259 boolean matchLimited = limit > 0; 1260 ArrayList<String> matchList = new ArrayList<>(); 1261 Matcher m = matcher(input); 1262 1263 // Add segments before each match found 1264 while(m.find()) 1265 if (!matchLimited || matchList.size() < limit - 1) 1266 if (index == 0 && index == m.start() && m.start() == m.end()) 1267 // no empty leading substring included for zero-width match 1268 // at the beginning of the input char sequence. 1269 continue; 1270 1271 String match = input.subSequence(index, m.start()).toString(); 1272 matchList.add(match); 1273 index = m.end(); 1274 else if (matchList.size() == limit - 1) // last one 1275 String match = input.subSequence(index, 1276 input.length()).toString(); 1277 matchList.add(match); 1278 index = m.end(); 1279 1280 1281 1282 // If no match was found, return this 1283 if (index == 0) 1284 return new String[] input.toString(); 1285 1286 // Add remaining segment 1287 if (!matchLimited || matchList.size() < limit) 1288 matchList.add(input.subSequence(index, input.length()).toString()); 1289 1290 // Construct result 1291 int resultSize = matchList.size(); 1292 if (limit == 0) 1293 while (resultSize > 0 && matchList.get(resultSize-1).isEmpty()) 1294 resultSize--; 1295 String[] result = new String[resultSize]; 1296 return matchList.subList(0, resultSize).toArray(result); 1297 1298 1299 /** 1300 * Splits the given input sequence around matches of this pattern. 1301 * 1302 * <p> This method works as if by invoking the two-argument @link 1303 * #split(java.lang.CharSequence, int) split method with the given input 1304 * sequence and a limit argument of zero. Trailing empty strings are 1305 * therefore not included in the resulting array. </p> 1306 * 1307 * <p> The input @code "boo:and:foo", for example, yields the following 1308 * results with these expressions: 1309 * 1310 * <table class="plain" style="margin-left:2em"> 1311 * <caption style="display:none">Split examples showing regex and result</caption> 1312 * <thead> 1313 * <tr> 1314 * <th scope="col">Regex</th> 1315 * <th scope="col">Result</th> 1316 * </tr> 1317 * </thead> 1318 * <tbody> 1319 * <tr><th scope="row" style="text-weight:normal">:</th> 1320 * <td>@code "boo", "and", "foo" </td></tr> 1321 * <tr><th scope="row" style="text-weight:normal">o</th> 1322 * <td>@code "b", "", ":and:f" </td></tr> 1323 * </tbody> 1324 * </table> 1325 * 1326 * 1327 * @param input 1328 * The character sequence to be split 1329 * 1330 * @return The array of strings computed by splitting the input 1331 * around matches of this pattern 1332 */ 1333 public String[] split(CharSequence input) 1334 return split(input, 0); 1335 1336 1337 /** 1338 * Returns a literal pattern @code String for the specified 1339 * @code String. 1340 * 1341 * <p>This method produces a @code String that can be used to 1342 * create a @code Pattern that would match the string 1343 * @code s as if it were a literal pattern.</p> Metacharacters 1344 * or escape sequences in the input sequence will be given no special 1345 * meaning. 1346 * 1347 * @param s The string to be literalized 1348 * @return A literal string replacement 1349 * @since 1.5 1350 */ 1351 public static String quote(String s) 1352 int slashEIndex = s.indexOf("\\E"); 1353 if (slashEIndex == -1) 1354 return "\\Q" + s + "\\E"; 1355 1356 int lenHint = s.length(); 1357 lenHint = (lenHint < Integer.MAX_VALUE - 8 - lenHint) ? 1358 (lenHint << 1) : (Integer.MAX_VALUE - 8); 1359 1360 StringBuilder sb = new StringBuilder(lenHint); 1361 sb.append("\\Q"); 1362 int current = 0; 1363 do 1364 sb.append(s, current, slashEIndex) 1365 .append("\\E\\\\E\\Q"); 1366 current = slashEIndex + 2; 1367 while ((slashEIndex = s.indexOf("\\E", current)) != -1); 1368 1369 return sb.append(s, current, s.length()) 1370 .append("\\E") 1371 .toString(); 1372 1373 1374 /** 1375 * Recompile the Pattern instance from a stream. The original pattern 1376 * string is read in and the object tree is recompiled from it. 1377 */ 1378 private void readObject(java.io.ObjectInputStream s) 1379 throws java.io.IOException, ClassNotFoundException 1380 1381 // Read in all fields 1382 s.defaultReadObject(); 1383 1384 // reset the flags 1385 flags0 = flags; 1386 1387 // Initialize counts 1388 capturingGroupCount = 1; 1389 localCount = 0; 1390 localTCNCount = 0; 1391 1392 // if length > 0, the Pattern is lazily compiled 1393 if (pattern.isEmpty()) 1394 root = new Start(lastAccept); 1395 matchRoot = lastAccept; 1396 compiled = true; 1397 1398 1399 1400 /** 1401 * This private constructor is used to create all Patterns. The pattern 1402 * string and match flags are all that is needed to completely describe 1403 * a Pattern. An empty pattern string results in an object tree with 1404 * only a Start node and a LastNode node. 1405 */ 1406 private Pattern(String p, int f) 1407 if ((f & ~ALL_FLAGS) != 0) 1408 throw new IllegalArgumentException("Unknown flag 0x" 1409 + Integer.toHexString(f)); 1410 1411 pattern = p; 1412 flags = f; 1413 1414 // to use UNICODE_CASE if UNICODE_CHARACTER_CLASS present 1415 if ((flags & UNICODE_CHARACTER_CLASS) != 0) 1416 flags |= UNICODE_CASE; 1417 1418 // ‘flags‘ for compiling 1419 flags0 = flags; 1420 1421 // Reset group index count 1422 capturingGroupCount = 1; 1423 localCount = 0; 1424 localTCNCount = 0; 1425 1426 if (!pattern.isEmpty()) 1427 compile(); 1428 else 1429 root = new Start(lastAccept); 1430 matchRoot = lastAccept; 1431 1432 1433 1434 /** 1435 * The pattern is converted to normalized form (@link 1436 * java.text.Normalizer.Form#NFC NFC, canonical decomposition, 1437 * followed by canonical composition for the character class 1438 * part, and @link java.text.Normalizer.Form#NFD NFD, 1439 * canonical decomposition for the rest), and then a pure 1440 * group is constructed to match canonical equivalences of the 1441 * characters. 1442 */ 1443 private static String normalize(String pattern) 1444 int plen = pattern.length(); 1445 StringBuilder pbuf = new StringBuilder(plen); 1446 char last = 0; 1447 int lastStart = 0; 1448 char cc = 0; 1449 for (int i = 0; i < plen;) 1450 char c = pattern.charAt(i); 1451 if (cc == 0 && // top level 1452 c == ‘\\‘ && i + 1 < plen && pattern.charAt(i + 1) == ‘\\‘) 1453 i += 2; last = 0; 1454 continue; 1455 1456 if (c == ‘[‘ && last != ‘\\‘) 1457 if (cc == 0) 1458 if (lastStart < i) 1459 normalizeSlice(pattern, lastStart, i, pbuf); 1460 lastStart = i; 1461 1462 cc++; 1463 else if (c == ‘]‘ && last != ‘\\‘) 1464 cc--; 1465 if (cc == 0) 1466 normalizeClazz(pattern, lastStart, i + 1, pbuf); 1467 lastStart = i + 1; 1468 1469 1470 last = c; 1471 i++; 1472 1473 assert (cc == 0); 1474 if (lastStart < plen) 1475 normalizeSlice(pattern, lastStart, plen, pbuf); 1476 return pbuf.toString(); 1477 1478 1479 private static void normalizeSlice(String src, int off, int limit, 1480 StringBuilder dst) 1481 1482 int len = src.length(); 1483 int off0 = off; 1484 while (off < limit && ASCII.isAscii(src.charAt(off))) 1485 off++; 1486 1487 if (off == limit) 1488 dst.append(src, off0, limit); 1489 return; 1490 1491 off--; 1492 if (off < off0) 1493 off = off0; 1494 else 1495 dst.append(src, off0, off); 1496 while (off < limit) 1497 int ch0 = src.codePointAt(off); 1498 if (".$|()[]^?*+\\".indexOf(ch0) != -1) 1499 dst.append((char)ch0); 1500 off++; 1501 continue; 1502 1503 int j = off + Character.charCount(ch0); 1504 int ch1; 1505 while (j < limit) 1506 ch1 = src.codePointAt(j); 1507 if (Grapheme.isBoundary(ch0, ch1)) 1508 break; 1509 ch0 = ch1; 1510 j += Character.charCount(ch1); 1511 1512 String seq = src.substring(off, j); 1513 String nfd = Normalizer.normalize(seq, Normalizer.Form.NFD); 1514 off = j; 1515 if (nfd.length() > 1) 1516 ch0 = nfd.codePointAt(0); 1517 ch1 = nfd.codePointAt(Character.charCount(ch0)); 1518 if (Character.getType(ch1) == Character.NON_SPACING_MARK) 1519 Set<String> altns = new LinkedHashSet<>(); 1520 altns.add(seq); 1521 produceEquivalentAlternation(nfd, altns); 1522 dst.append("(?:"); 1523 altns.forEach( s -> dst.append(s).append(‘|‘)); 1524 dst.delete(dst.length() - 1, dst.length()); 1525 dst.append(")"); 1526 continue; 1527 1528 1529 String nfc = Normalizer.normalize(seq, Normalizer.Form.NFC); 1530 if (!seq.equals(nfc) && !nfd.equals(nfc)) 1531 dst.append("(?:" + seq + "|" + nfd + "|" + nfc + ")"); 1532 else if (!seq.equals(nfd)) 1533 dst.append("(?:" + seq + "|" + nfd + ")"); 1534 else 1535 dst.append(seq); 1536 1537 1538 1539 private static void normalizeClazz(String src, int off, int limit, 1540 StringBuilder dst) 1541 1542 dst.append(Normalizer.normalize(src.substring(off, limit), Form.NFC)); 1543 1544 1545 /** 1546 * Given a specific sequence composed of a regular character and 1547 * combining marks that follow it, produce the alternation that will 1548 * match all canonical equivalences of that sequence. 1549 */ 1550 private static void produceEquivalentAlternation(String src, 1551 Set<String> dst) 1552 1553 int len = countChars(src, 0, 1); 1554 if (src.length() == len) 1555 dst.add(src); // source has one character. 1556 return; 1557 1558 String base = src.substring(0,len); 1559 String combiningMarks = src.substring(len); 1560 String[] perms = producePermutations(combiningMarks); 1561 // Add combined permutations 1562 for(int x = 0; x < perms.length; x++) 1563 String next = base + perms[x]; 1564 dst.add(next); 1565 next = composeOneStep(next); 1566 if (next != null) 1567 produceEquivalentAlternation(next, dst); 1568 1569 1570 1571 1572 /** 1573 * Returns an array of strings that have all the possible 1574 * permutations of the characters in the input string. 1575 * This is used to get a list of all possible orderings 1576 * of a set of combining marks. Note that some of the permutations 1577 * are invalid because of combining class collisions, and these 1578 * possibilities must be removed because they are not canonically 1579 * equivalent. 1580 */ 1581 private static String[] producePermutations(String input) 1582 if (input.length() == countChars(input, 0, 1)) 1583 return new String[] input; 1584 1585 if (input.length() == countChars(input, 0, 2)) 1586 int c0 = Character.codePointAt(input, 0); 1587 int c1 = Character.codePointAt(input, Character.charCount(c0)); 1588 if (getClass(c1) == getClass(c0)) 1589 return new String[] input; 1590 1591 String[] result = new String[2]; 1592 result[0] = input; 1593 StringBuilder sb = new StringBuilder(2); 1594 sb.appendCodePoint(c1); 1595 sb.appendCodePoint(c0); 1596 result[1] = sb.toString(); 1597 return result; 1598 1599 1600 int length = 1; 1601 int nCodePoints = countCodePoints(input); 1602 for(int x=1; x<nCodePoints; x++) 1603 length = length * (x+1); 1604 1605 String[] temp = new String[length]; 1606 1607 int combClass[] = new int[nCodePoints]; 1608 for(int x=0, i=0; x<nCodePoints; x++) 1609 int c = Character.codePointAt(input, i); 1610 combClass[x] = getClass(c); 1611 i += Character.charCount(c); 1612 1613 1614 // For each char, take it out and add the permutations 1615 // of the remaining chars 1616 int index = 0; 1617 int len; 1618 // offset maintains the index in code units. 1619 loop: for(int x=0, offset=0; x<nCodePoints; x++, offset+=len) 1620 len = countChars(input, offset, 1); 1621 for(int y=x-1; y>=0; y--) 1622 if (combClass[y] == combClass[x]) 1623 continue loop; 1624 1625 1626 StringBuilder sb = new StringBuilder(input); 1627 String otherChars = sb.delete(offset, offset+len).toString(); 1628 String[] subResult = producePermutations(otherChars); 1629 1630 String prefix = input.substring(offset, offset+len); 1631 for (String sre : subResult) 1632 temp[index++] = prefix + sre; 1633 1634 String[] result = new String[index]; 1635 System.arraycopy(temp, 0, result, 0, index); 1636 return result; 1637 1638 1639 private static int getClass(int c) 1640 return sun.text.Normalizer.getCombiningClass(c); 1641 1642 1643 /** 1644 * Attempts to compose input by combining the first character 1645 * with the first combining mark following it. Returns a String 1646 * that is the composition of the leading character with its first 1647 * combining mark followed by the remaining combining marks. Returns 1648 * null if the first two characters cannot be further composed. 1649 */ 1650 private static String composeOneStep(String input) 1651 int len = countChars(input, 0, 2); 1652 String firstTwoCharacters = input.substring(0, len); 1653 String result = Normalizer.normalize(firstTwoCharacters, Normalizer.Form.NFC); 1654 if (result.equals(firstTwoCharacters)) 1655 return null; 1656 else 1657 String remainder = input.substring(len); 1658 return result + remainder; 1659 1660 1661 1662 /** 1663 * Preprocess any \Q...\E sequences in `temp‘, meta-quoting them. 1664 * See the description of `quotemeta‘ in perlfunc(1). 1665 */ 1666 private void RemoveQEQuoting() 1667 final int pLen = patternLength; 1668 int i = 0; 1669 while (i < pLen-1) 1670 if (temp[i] != ‘\\‘) 1671 i += 1; 1672 else if (temp[i + 1] != ‘Q‘) 1673 i += 2; 1674 else 1675 break; 1676 1677 if (i >= pLen - 1) // No \Q sequence found 1678 return; 1679 int j = i; 1680 i += 2; 1681 int[] newtemp = new int[j + 3*(pLen-i) + 2]; 1682 System.arraycopy(temp, 0, newtemp, 0, j); 1683 1684 boolean inQuote = true; 1685 boolean beginQuote = true; 1686 while (i < pLen) 1687 int c = temp[i++]; 1688 if (!ASCII.isAscii(c) || ASCII.isAlpha(c)) 1689 newtemp[j++] = c; 1690 else if (ASCII.isDigit(c)) 1691 if (beginQuote) 1692 /* 1693 * A unicode escape \[0xu] could be before this quote, 1694 * and we don‘t want this numeric char to processed as 1695 * part of the escape. 1696 */ 1697 newtemp[j++] = ‘\\‘; 1698 newtemp[j++] = ‘x‘; 1699 newtemp[j++] = ‘3‘; 1700 1701 newtemp[j++] = c; 1702 else if (c != ‘\\‘) 1703 if (inQuote) newtemp[j++] = ‘\\‘; 1704 newtemp[j++] = c; 1705 else if (inQuote) 1706 if (temp[i] == ‘E‘) 1707 i++; 1708 inQuote = false; 1709 else 1710 newtemp[j++] = ‘\\‘; 1711 newtemp[j++] = ‘\\‘; 1712 1713 else 1714 if (temp[i] == ‘Q‘) 1715 i++; 1716 inQuote = true; 1717 beginQuote = true; 1718 continue; 1719 else 1720 newtemp[j++] = c; 1721 if (i != pLen) 1722 newtemp[j++] = temp[i++]; 1723 1724 1725 1726 beginQuote = false; 1727 1728 1729 patternLength = j; 1730 temp = Arrays.copyOf(newtemp, j + 2); // double zero termination 1731 1732 1733 /** 1734 * Copies regular expression to an int array and invokes the parsing 1735 * of the expression which will create the object tree. 1736 */ 1737 private void compile() 1738 // Handle canonical equivalences 1739 if (has(CANON_EQ) && !has(LITERAL)) 1740 normalizedPattern = normalize(pattern); 1741 else 1742 normalizedPattern = pattern; 1743 1744 patternLength = normalizedPattern.length(); 1745 1746 // Copy pattern to int array for convenience 1747 // Use double zero to terminate pattern 1748 temp = new int[patternLength + 2]; 1749 1750 hasSupplementary = false; 1751 int c, count = 0; 1752 // Convert all chars into code points 1753 for (int x = 0; x < patternLength; x += Character.charCount(c)) 1754 c = normalizedPattern.codePointAt(x); 1755 if (isSupplementary(c)) 1756 hasSupplementary = true; 1757 1758 temp[count++] = c; 1759 1760 1761 patternLength = count; // patternLength now in code points 1762 1763 if (! has(LITERAL)) 1764 RemoveQEQuoting(); 1765 1766 // Allocate all temporary objects here. 1767 buffer = new int[32]; 1768 groupNodes = new GroupHead[10]; 1769 namedGroups = null; 1770 topClosureNodes = new ArrayList<>(10); 1771 1772 if (has(LITERAL)) 1773 // Literal pattern handling 1774 matchRoot = newSlice(temp, patternLength, hasSupplementary); 1775 matchRoot.next = lastAccept; 1776 else 1777 // Start recursive descent parsing 1778 matchRoot = expr(lastAccept); 1779 // Check extra pattern characters 1780 if (patternLength != cursor) 1781 if (peek() == ‘)‘) 1782 throw error("Unmatched closing ‘)‘"); 1783 else 1784 throw error("Unexpected internal error"); 1785 1786 1787 1788 1789 // Peephole optimization 1790 if (matchRoot instanceof Slice) 1791 root = BnM.optimize(matchRoot); 1792 if (root == matchRoot) 1793 root = hasSupplementary ? new StartS(matchRoot) : new Start(matchRoot); 1794 1795 else if (matchRoot instanceof Begin || matchRoot instanceof First) 1796 root = matchRoot; 1797 else 1798 root = hasSupplementary ? new StartS(matchRoot) : new Start(matchRoot); 1799 1800 1801 // Optimize the greedy Loop to prevent exponential backtracking, IF there 1802 // is no group ref in this pattern. With a non-negative localTCNCount value, 1803 // the greedy type Loop, Curly will skip the backtracking for any starting 1804 // position "i" that failed in the past. 1805 if (!hasGroupRef) 1806 for (Node node : topClosureNodes) 1807 if (node instanceof Loop) 1808 // non-deterministic-greedy-group 1809 ((Loop)node).posIndex = localTCNCount++; 1810 1811 1812 1813 1814 // Release temporary storage 1815 temp = null; 1816 buffer = null; 1817 groupNodes = null; 1818 patternLength = 0; 1819 compiled = true; 1820 topClosureNodes = null; 1821 1822 1823 Map<String, Integer> namedGroups() 1824 Map<String, Integer> groups = namedGroups; 1825 if (groups == null) 1826 namedGroups = groups = new HashMap<>(2); 1827 1828 return groups; 1829 1830 1831 /** 1832 * Used to accumulate information about a subtree of the object graph 1833 * so that optimizations can be applied to the subtree. 1834 */ 1835 static final class TreeInfo 1836 int minLength; 1837 int maxLength; 1838 boolean maxValid; 1839 boolean deterministic; 1840 1841 TreeInfo() 1842 reset(); 1843 1844 void reset() 1845 minLength = 0; 1846 maxLength = 0; 1847 maxValid = true; 1848 deterministic = true; 1849 1850 1851 1852 /* 1853 * The following private methods are mainly used to improve the 1854 * readability of the code. In order to let the Java compiler easily 1855 * inline them, we should not put many assertions or error checks in them. 1856 */ 1857 1858 /** 1859 * Indicates whether a particular flag is set or not. 1860 */ 1861 private boolean has(int f) 1862 return (flags0 & f) != 0; 1863 1864 1865 /** 1866 * Match next character, signal error if failed. 1867 */ 1868 private void accept(int ch, String s) 1869 int testChar = temp[cursor++]; 1870 if (has(COMMENTS)) 1871 testChar = parsePastWhitespace(testChar); 1872 if (ch != testChar) 1873 throw error(s); 1874 1875 1876 1877 /** 1878 * Mark the end of pattern with a specific character. 1879 */ 1880 private void mark(int c) 1881 temp[patternLength] = c; 1882 1883 1884 /** 1885 * Peek the next character, and do not advance the cursor. 1886 */ 1887 private int peek() 1888 int ch = temp[cursor]; 1889 if (has(COMMENTS)) 1890 ch = peekPastWhitespace(ch); 1891 return ch; 1892 1893 1894 /** 1895 * Read the next character, and advance the cursor by one. 1896 */ 1897 private int read() 1898 int ch = temp[cursor++]; 1899 if (has(COMMENTS)) 1900 ch = parsePastWhitespace(ch); 1901 return ch; 1902 1903 1904 /** 1905 * Read the next character, and advance the cursor by one, 1906 * ignoring the COMMENTS setting 1907 */ 1908 private int readEscaped() 1909 int ch = temp[cursor++]; 1910 return ch; 1911 1912 1913 /** 1914 * Advance the cursor by one, and peek the next character. 1915 */ 1916 private int next() 1917 int ch = temp[++cursor]; 1918 if (has(COMMENTS)) 1919 ch = peekPastWhitespace(ch); 1920 return ch; 1921 1922 1923 /** 1924 * Advance the cursor by one, and peek the next character, 1925 * ignoring the COMMENTS setting 1926 */ 1927 private int nextEscaped() 1928 int ch = temp[++cursor]; 1929 return ch; 1930 1931 1932 /** 1933 * If in xmode peek past whitespace and comments. 1934 */ 1935 private int peekPastWhitespace(int ch) 1936 while (ASCII.isSpace(ch) || ch == ‘#‘) 1937 while (ASCII.isSpace(ch)) 1938 ch = temp[++cursor]; 1939 if (ch == ‘#‘) 1940 ch = peekPastLine(); 1941 1942 1943 return ch; 1944 1945 1946 /** 1947 * If in xmode parse past whitespace and comments. 1948 */ 1949 private int parsePastWhitespace(int ch) 1950 while (ASCII.isSpace(ch) || ch == ‘#‘) 1951 while (ASCII.isSpace(ch)) 1952 ch = temp[cursor++]; 1953 if (ch == ‘#‘) 1954 ch = parsePastLine(); 1955 1956 return ch; 1957 1958 1959 /** 1960 * xmode parse past comment to end of line. 1961 */ 1962 private int parsePastLine() 1963 int ch = temp[cursor++]; 1964 while (ch != 0 && !isLineSeparator(ch)) 1965 ch = temp[cursor++]; 1966 return ch; 1967 1968 1969 /** 1970 * xmode peek past comment to end of line. 1971 */ 1972 private int peekPastLine() 1973 int ch = temp[++cursor]; 1974 while (ch != 0 && !isLineSeparator(ch)) 1975 ch = temp[++cursor]; 1976 return ch; 1977 1978 1979 /** 1980 * Determines if character is a line separator in the current mode 1981 */ 1982 private boolean isLineSeparator(int ch) 1983 if (has(UNIX_LINES)) 1984 return ch == ‘\n‘; 1985 else 1986 return (ch == ‘\n‘ || 1987 ch == ‘\r‘ || 1988 (ch|1) == ‘\u2029‘ || 1989 ch == ‘\u0085‘); 1990 1991 1992 1993 /** 1994 * Read the character after the next one, and advance the cursor by two. 1995 */ 1996 private int skip() 1997 int i = cursor; 1998 int ch = temp[i+1]; 1999 cursor = i + 2; 2000 return ch; 2001 2002 2003 /** 2004 * Unread one next character, and retreat cursor by one. 2005 */ 2006 private void unread() 2007 cursor--; 2008 2009 2010 /** 2011 * Internal method used for handling all syntax errors. The pattern is 2012 * displayed with a pointer to aid in locating the syntax error. 2013 */ 2014 private PatternSyntaxException error(String s) 2015 return new PatternSyntaxException(s, normalizedPattern, cursor - 1); 2016 2017 2018 /** 2019 * Determines if there is any supplementary character or unpaired 2020 * surrogate in the specified range. 2021 */ 2022 private boolean findSupplementary(int start, int end) 2023 for (int i = start; i < end; i++) 2024 if (isSupplementary(temp[i])) 2025 return true; 2026 2027 return false; 2028 2029 2030 /** 2031 * Determines if the specified code point is a supplementary 2032 * character or unpaired surrogate. 2033 */ 2034 private static final boolean isSupplementary(int ch) 2035 return ch >= Character.MIN_SUPPLEMENTARY_CODE_POINT || 2036 Character.isSurrogate((char)ch); 2037 2038 2039 /** 2040 * The following methods handle the main parsing. They are sorted 2041 * according to their precedence order, the lowest one first. 2042 */ 2043 2044 /** 2045 * The expression is parsed with branch nodes added for alternations. 2046 * This may be called recursively to parse sub expressions that may 2047 * contain alternations. 2048 */ 2049 private Node expr(Node end) 2050 Node prev = null; 2051 Node firstTail = null; 2052 Branch branch = null; 2053 Node branchConn = null; 2054 2055 for (;;) 2056 Node node = sequence(end); 2057 Node nodeTail = root; //double return 2058 if (prev == null) 2059 prev = node; 2060 firstTail = nodeTail; 2061 else 2062 // Branch 2063 if (branchConn == null) 2064 branchConn = new BranchConn(); 2065 branchConn.next = end; 2066 2067 if (node == end) 2068 // if the node returned from sequence() is "end" 2069 // we have an empty expr, set a null atom into 2070 // the branch to indicate to go "next" directly. 2071 node = null; 2072 else 2073 // the "tail.next" of each atom goes to branchConn 2074 nodeTail.next = branchConn; 2075 2076 if (prev == branch) 2077 branch.add(node); 2078 else 2079 if (prev == end) 2080 prev = null; 2081 else 2082 // replace the "end" with "branchConn" at its tail.next 2083 // when put the "prev" into the branch as the first atom. 2084 firstTail.next = branchConn; 2085 2086 prev = branch = new Branch(prev, node, branchConn); 2087 2088 2089 if (peek() != ‘|‘) 2090 return prev; 2091 2092 next(); 2093 2094 2095 2096 @SuppressWarnings("fallthrough") 2097 /** 2098 * Parsing of sequences between alternations. 2099 */ 2100 private Node sequence(Node end) 2101 Node head = null; 2102 Node tail = null; 2103 Node node = null; 2104 LOOP: 2105 for (;;) 2106 int ch = peek(); 2107 switch (ch) 2108 case ‘(‘: 2109 // Because group handles its own closure, 2110 // we need to treat it differently 2111 node = group0(); 2112 // Check for comment or flag group 2113 if (node == null) 2114 continue; 2115 if (head == null) 2116 head = node; 2117 else 2118 tail.next = node; 2119 // Double return: Tail was returned in root 2120 tail = root; 2121 continue; 2122 case ‘[‘: 2123 if (has(CANON_EQ) && !has(LITERAL)) 2124 node = new NFCCharProperty(clazz(true)); 2125 else 2126 node = newCharProperty(clazz(true)); 2127 break; 2128 case ‘\\‘: 2129 ch = nextEscaped(); 2130 if (ch == ‘p‘ || ch == ‘P‘) 2131 boolean oneLetter = true; 2132 boolean comp = (ch == ‘P‘); 2133 ch = next(); // Consume if present 2134 if (ch != ‘‘) 2135 unread(); 2136 else 2137 oneLetter = false; 2138 2139 // node = newCharProperty(family(oneLetter, comp)); 2140 if (has(CANON_EQ) && !has(LITERAL)) 2141 node = new NFCCharProperty(family(oneLetter, comp)); 2142 else 2143 node = newCharProperty(family(oneLetter, comp)); 2144 else 2145 unread(); 2146 node = atom(); 2147 2148 break; 2149 case ‘^‘: 2150 next(); 2151 if (has(MULTILINE)) 2152 if (has(UNIX_LINES)) 2153 node = new UnixCaret(); 2154 else 2155 node = new Caret(); 2156 else 2157 node = new Begin(); 2158 2159 break; 2160 case ‘$‘: 2161 next(); 2162 if (has(UNIX_LINES)) 2163 node = new UnixDollar(has(MULTILINE)); 2164 else 2165 node = new Dollar(has(MULTILINE)); 2166 break; 2167 case ‘.‘: 2168 next(); 2169 if (has(DOTALL)) 2170 node = new CharProperty(ALL()); 2171 else 2172 if (has(UNIX_LINES)) 2173 node = new CharProperty(UNIXDOT()); 2174 else 2175 node = new CharProperty(DOT()); 2176 2177 2178 break; 2179 case ‘|‘: 2180 case ‘)‘: 2181 break LOOP; 2182 case ‘]‘: // Now interpreting dangling ] and as literals 2183 case ‘‘: 2184 node = atom(); 2185 break; 2186 case ‘?‘: 2187 case ‘*‘: 2188 case ‘+‘: 2189 next(); 2190 throw error("Dangling meta character ‘" + ((char)ch) + "‘"); 2191 case 0: 2192 if (cursor >= patternLength) 2193 break LOOP; 2194 2195 // Fall through 2196 default: 2197 node = atom(); 2198 break; 2199 2200 2201 node = closure(node); 2202 /* save the top dot-greedy nodes (.*, .+) as well 2203 if (node instanceof GreedyCharProperty && 2204 ((GreedyCharProperty)node).cp instanceof Dot) 2205 topClosureNodes.add(node); 2206 2207 */ 2208 if (head == null) 2209 head = tail = node; 2210 else 2211 tail.next = node; 2212 tail = node; 2213 2214 2215 if (head == null) 2216 return end; 2217 2218 tail.next = end; 2219 root = tail; //double return 2220 return head; 2221 2222 2223 @SuppressWarnings("fallthrough") 2224 /** 2225 * Parse and add a new Single or Slice. 2226 */ 2227 private Node atom() 2228 int first = 0; 2229 int prev = -1; 2230 boolean hasSupplementary = false; 2231 int ch = peek(); 2232 for (;;) 2233 switch (ch) 2234 case ‘*‘: 2235 case ‘+‘: 2236 case ‘?‘: 2237 case ‘‘: 2238 if (first > 1) 2239 cursor = prev; // Unwind one character 2240 first--; 2241 2242 break; 2243 case ‘$‘: 2244 case ‘.‘: 2245 case ‘^‘: 2246 case ‘(‘: 2247 case ‘[‘: 2248 case ‘|‘: 2249 case ‘)‘: 2250 break; 2251 case ‘\\‘: 2252 ch = nextEscaped(); 2253 if (ch == ‘p‘ || ch == ‘P‘) // Property 2254 if (first > 0) // Slice is waiting; handle it first 2255 unread(); 2256 break; 2257 else // No slice; just return the family node 2258 boolean comp = (ch == ‘P‘); 2259 boolean oneLetter = true; 2260 ch = next(); // Consume if present 2261 if (ch != ‘‘) 2262 unread(); 2263 else 2264 oneLetter = false; 2265 if (has(CANON_EQ) && !has(LITERAL)) 2266 return new NFCCharProperty(family(oneLetter, comp)); 2267 else 2268 return newCharProperty(family(oneLetter, comp)); 2269 2270 2271 unread(); 2272 prev = cursor; 2273 ch = escape(false, first == 0, false); 2274 if (ch >= 0) 2275 append(ch, first); 2276 first++; 2277 if (isSupplementary(ch)) 2278 hasSupplementary = true; 2279 2280 ch = peek(); 2281 continue; 2282 else if (first == 0) 2283 return root; 2284 2285 // Unwind meta escape sequence 2286 cursor = prev; 2287 break; 2288 case 0: 2289 if (cursor >= patternLength) 2290 break; 2291 2292 // Fall through 2293 default: 2294 prev = cursor; 2295 append(ch, first); 2296 first++; 2297 if (isSupplementary(ch)) 2298 hasSupplementary = true; 2299 2300 ch = next(); 2301 continue; 2302 2303 break; 2304 2305 if (first == 1) 2306 return newCharProperty(single(buffer[0])); 2307 else 2308 return newSlice(buffer, first, hasSupplementary); 2309 2310 2311 2312 private void append(int ch, int len) 2313 if (len >= buffer.length) 2314 int[] tmp = new int[len+len]; 2315 System.arraycopy(buffer, 0, tmp, 0, len); 2316 buffer = tmp; 2317 2318 buffer[len] = ch; 2319 2320 2321 /** 2322 * Parses a backref greedily, taking as many numbers as it 2323 * can. The first digit is always treated as a backref, but 2324 * multi digit numbers are only treated as a backref if at 2325 * least that many backrefs exist at this point in the regex. 2326 */ 2327 private Node ref(int refNum) 2328 boolean done = false; 2329 while(!done) 2330 int ch = peek(); 2331 switch(ch) 2332 case ‘0‘: 2333 case ‘1‘: 2334 case ‘2‘: 2335 case ‘3‘: 2336 case ‘4‘: 2337 case ‘5‘: 2338 case ‘6‘: 2339 case ‘7‘: 2340 case ‘8‘: 2341 case ‘9‘: 2342 int newRefNum = (refNum * 10) + (ch - ‘0‘); 2343 // Add another number if it doesn‘t make a group 2344 // that doesn‘t exist 2345 if (capturingGroupCount - 1 < newRefNum) 2346 done = true; 2347 break; 2348 2349 refNum = newRefNum; 2350 read(); 2351 break; 2352 default: 2353 done = true; 2354 break; 2355 2356 2357 hasGroupRef = true; 2358 if (has(CASE_INSENSITIVE)) 2359 return new CIBackRef(refNum, has(UNICODE_CASE)); 2360 else 2361 return new BackRef(refNum); 2362 2363 2364 /** 2365 * Parses an escape sequence to determine the actual value that needs 2366 * to be matched. 2367 * If -1 is returned and create was true a new object was added to the tree 2368 * to handle the escape sequence. 2369 * If the returned value is greater than zero, it is the value that 2370 * matches the escape sequence. 2371 */ 2372 private int escape(boolean inclass, boolean create, boolean isrange) 2373 int ch = skip(); 2374 switch (ch) 2375 case ‘0‘: 2376 return o(); 2377 case ‘1‘: 2378 case ‘2‘: 2379 case ‘3‘: 2380 case ‘4‘: 2381 case ‘5‘: 2382 case ‘6‘: 2383 case ‘7‘: 2384 case ‘8‘: 2385 case ‘9‘: 2386 if (inclass) break; 2387 if (create) 2388 root = ref((ch - ‘0‘)); 2389 2390 return -1; 2391 case ‘A‘: 2392 if (inclass) break; 2393 if (create) root = new Begin(); 2394 return -1; 2395 case ‘B‘: 2396 if (inclass) break; 2397 if (create) root = new Bound(Bound.NONE, has(UNICODE_CHARACTER_CLASS)); 2398 return -1; 2399 case ‘C‘: 2400 break; 2401 case ‘D‘: 2402 if (create) 2403 predicate = has(UNICODE_CHARACTER_CLASS) ? 2404 CharPredicates.DIGIT() : CharPredicates.ASCII_DIGIT(); 2405 predicate = predicate.negate(); 2406 if (!inclass) 2407 root = newCharProperty(predicate); 2408 2409 return -1; 2410 case ‘E‘: 2411 case ‘F‘: 2412 break; 2413 case ‘G‘: 2414 if (inclass) break; 2415 if (create) root = new LastMatch(); 2416 return -1; 2417 case ‘H‘: 2418 if (create) 2419 predicate = HorizWS().negate(); 2420 if (!inclass) 2421 root = newCharProperty(predicate); 2422 2423 return -1; 2424 case ‘I‘: 2425 case ‘J‘: 2426 case ‘K‘: 2427 case ‘L‘: 2428 case ‘M‘: 2429 break; 2430 case ‘N‘: 2431 return N(); 2432 case ‘O‘: 2433 case ‘P‘: 2434 case ‘Q‘: 2435 break; 2436 case ‘R‘: 2437 if (inclass) break; 2438 if (create) root = new LineEnding(); 2439 return -1; 2440 case ‘S‘: 2441 if (create) 2442 predicate = has(UNICODE_CHARACTER_CLASS) ? 2443 CharPredicates.WHITE_SPACE() : CharPredicates.ASCII_SPACE(); 2444 predicate = predicate.negate(); 2445 if (!inclass) 2446 root = newCharProperty(predicate); 2447 2448 return -1; 2449 case ‘T‘: 2450 case ‘U‘: 2451 break; 2452 case ‘V‘: 2453 if (create) 2454 predicate = VertWS().negate(); 2455 if (!inclass) 2456 root = newCharProperty(predicate); 2457 2458 return -1; 2459 case ‘W‘: 2460 if (create) 2461 predicate = has(UNICODE_CHARACTER_CLASS) ? 2462 CharPredicates.WORD() : CharPredicates.ASCII_WORD(); 2463 predicate = predicate.negate(); 2464 if (!inclass) 2465 root = newCharProperty(predicate); 2466 2467 return -1; 2468 case ‘X‘: 2469 if (inclass) break; 2470 if (create) 2471 root = new XGrapheme(); 2472 2473 return -1; 2474 case ‘Y‘: 2475 break; 2476 case ‘Z‘: 2477 if (inclass) break; 2478 if (create) 2479 if (has(UNIX_LINES)) 2480 root = new UnixDollar(false); 2481 else 2482 root = new Dollar(false); 2483 2484 return -1; 2485 case ‘a‘: 2486 return ‘\007‘; 2487 case ‘b‘: 2488 if (inclass) break; 2489 if (create) 2490 if (peek() == ‘‘) 2491 if (skip() == ‘g‘) 2492 if (read() == ‘‘) 2493 root = new GraphemeBound(); 2494 return -1; 2495 2496 break; // error missing trailing 2497 2498 unread(); unread(); 2499 2500 root = new Bound(Bound.BOTH, has(UNICODE_CHARACTER_CLASS)); 2501 2502 return -1; 2503 case ‘c‘: 2504 return c(); 2505 case ‘d‘: 2506 if (create) 2507 predicate = has(UNICODE_CHARACTER_CLASS) ? 2508 CharPredicates.DIGIT() : CharPredicates.ASCII_DIGIT(); 2509 if (!inclass) 2510 root = newCharProperty(predicate); 2511 2512 return -1; 2513 case ‘e‘: 2514 return ‘\033‘; 2515 case ‘f‘: 2516 return ‘\f‘; 2517 case ‘g‘: 2518 break; 2519 case ‘h‘: 2520 if (create) 2521 predicate = HorizWS(); 2522 if (!inclass) 2523 root = newCharProperty(predicate); 2524 2525 return -1; 2526 case ‘i‘: 2527 case ‘j‘: 2528 break; 2529 case ‘k‘: 2530 if (inclass) 2531 break; 2532 if (read() != ‘<‘) 2533 throw error("\\k is not followed by ‘<‘ for named capturing group"); 2534 String name = groupname(read()); 2535 if (!namedGroups().containsKey(name)) 2536 throw error("named capturing group <" + name + "> does not exist"); 2537 if (create) 2538 hasGroupRef = true; 2539 if (has(CASE_INSENSITIVE)) 2540 root = new CIBackRef(namedGroups().get(name), has(UNICODE_CASE)); 2541 else 2542 root = new BackRef(namedGroups().get(name)); 2543 2544 return -1; 2545 case ‘l‘: 2546 case ‘m‘: 2547 break; 2548 case ‘n‘: 2549 return ‘\n‘; 2550 case ‘o‘: 2551 case ‘p‘: 2552 case ‘q‘: 2553 break; 2554 case ‘r‘: 2555 return ‘\r‘; 2556 case ‘s‘: 2557 if (create) 2558 predicate = has(UNICODE_CHARACTER_CLASS) ? 2559 CharPredicates.WHITE_SPACE() : CharPredicates.ASCII_SPACE(); 2560 if (!inclass) 2561 root = newCharProperty(predicate); 2562 2563 return -1; 2564 case ‘t‘: 2565 return ‘\t‘; 2566 case ‘u‘: 2567 return u(); 2568 case ‘v‘: 2569 // ‘\v‘ was implemented as VT/0x0B in releases < 1.8 (though 2570 // undocumented). In JDK8 ‘\v‘ is specified as a predefined 2571 // character class for all vertical whitespace characters. 2572 // So [-1, root=VertWS node] pair is returned (instead of a 2573 // single 0x0B). This breaks the range if ‘\v‘ is used as 2574 // the start or end value, such as [\v-...] or [...-\v], in 2575 // which a single definite value (0x0B) is expected. For 2576 // compatibility concern ‘\013‘/0x0B is returned if isrange. 2577 if (isrange) 2578 return ‘\013‘; 2579 if (create) 2580 predicate = VertWS(); 2581 if (!inclass) 2582 root = newCharProperty(predicate); 2583 2584 return -1; 2585 case ‘w‘: 2586 if (create) 2587 predicate = has(UNICODE_CHARACTER_CLASS) ? 2588 CharPredicates.WORD() : CharPredicates.ASCII_WORD(); 2589 if (!inclass) 2590 root = newCharProperty(predicate); 2591 2592 return -1; 2593 case ‘x‘: 2594 return x(); 2595 case ‘y‘: 2596 break; 2597 case ‘z‘: 2598 if (inclass) break; 2599 if (create) root = new End(); 2600 return -1; 2601 default: 2602 return ch; 2603 2604 throw error("Illegal/unsupported escape sequence"); 2605 2606 2607 /** 2608 * Parse a character class, and return the node that matches it. 2609 * 2610 * Consumes a ] on the way out if consume is true. Usually consume 2611 * is true except for the case of [abc&&def] where def is a separate 2612 * right hand node with "understood" brackets. 2613 */ 2614 private CharPredicate clazz(boolean consume) 2615 CharPredicate prev = null; 2616 CharPredicate curr = null; 2617 BitClass bits = new BitClass(); 2618 BmpCharPredicate bitsP = ch -> ch < 256 && bits.bits[ch]; 2619 2620 boolean isNeg = false; 2621 boolean hasBits = false; 2622 int ch = next(); 2623 2624 // Negates if first char in a class, otherwise literal 2625 if (ch == ‘^‘ && temp[cursor-1] == ‘[‘) 2626 ch = next(); 2627 isNeg = true; 2628 2629 for (;;) 2630 switch (ch) 2631 case ‘[‘: 2632 curr = clazz(true); 2633 if (prev == null) 2634 prev = curr; 2635 else 2636 prev = prev.union(curr); 2637 ch = peek(); 2638 continue; 2639 case ‘&‘: 2640 ch = next(); 2641 if (ch == ‘&‘) 2642 ch = next(); 2643 CharPredicate right = null; 2644 while (ch != ‘]‘ && ch != ‘&‘) 2645 if (ch == ‘[‘) 2646 if (right == null) 2647 right = clazz(true); 2648 else 2649 right = right.union(clazz(true)); 2650 else // abc&&def 2651 unread(); 2652 right = clazz(false); 2653 2654 ch = peek(); 2655 2656 if (hasBits) 2657 // bits used, union has high precedence 2658 if (prev == null) 2659 prev = curr = bitsP; 2660 else 2661 prev = prev.union(bitsP); 2662 2663 hasBits = false; 2664 2665 if (right != null) 2666 curr = right; 2667 if (prev == null) 2668 if (right == null) 2669 throw error("Bad class syntax"); 2670 else 2671 prev = right; 2672 else 2673 prev = prev.and(curr); 2674 2675 else 2676 // treat as a literal & 2677 unread(); 2678 break; 2679 2680 continue; 2681 case 0: 2682 if (cursor >= patternLength) 2683 throw error("Unclosed character class"); 2684 break; 2685 case ‘]‘: 2686 if (prev != null || hasBits) 2687 if (consume) 2688 next(); 2689 if (prev == null) 2690 prev = bitsP; 2691 else if (hasBits) 2692 prev = prev.union(bitsP); 2693 if (isNeg) 2694 return prev.negate(); 2695 return prev; 2696 2697 break; 2698 default: 2699 break; 2700 2701 curr = range(bits); 2702 if (curr == null) // the bits used 2703 hasBits = true; 2704 else 2705 if (prev == null) 2706 prev = curr; 2707 else if (prev != curr) 2708 prev = prev.union(curr); 2709 2710 ch = peek(); 2711 2712 2713 2714 private CharPredicate bitsOrSingle(BitClass bits, int ch) 2715 /* Bits can only handle codepoints in [u+0000-u+00ff] range. 2716 Use "single" node instead of bits when dealing with unicode 2717 case folding for codepoints listed below. 2718 (1)Uppercase out of range: u+00ff, u+00b5 2719 toUpperCase(u+00ff) -> u+0178 2720 toUpperCase(u+00b5) -> u+039c 2721 (2)LatinSmallLetterLongS u+17f 2722 toUpperCase(u+017f) -> u+0053 2723 (3)LatinSmallLetterDotlessI u+131 2724 toUpperCase(u+0131) -> u+0049 2725 (4)LatinCapitalLetterIWithDotAbove u+0130 2726 toLowerCase(u+0130) -> u+0069 2727 (5)KelvinSign u+212a 2728 toLowerCase(u+212a) ==> u+006B 2729 (6)AngstromSign u+212b 2730 toLowerCase(u+212b) ==> u+00e5 2731 */ 2732 if (ch < 256 && 2733 !(has(CASE_INSENSITIVE) && has(UNICODE_CASE) && 2734 (ch == 0xff || ch == 0xb5 || 2735 ch == 0x49 || ch == 0x69 || //I and i 2736 ch == 0x53 || ch == 0x73 || //S and s 2737 ch == 0x4b || ch == 0x6b || //K and k 2738 ch == 0xc5 || ch == 0xe5))) //A+ring 2739 bits.add(ch, flags0); 2740 return null; 2741 2742 return single(ch); 2743 2744 2745 /** 2746 * Returns a suitably optimized, single character predicate 2747 */ 2748 private CharPredicate single(final int ch) 2749 if (has(CASE_INSENSITIVE)) 2750 int lower, upper; 2751 if (has(UNICODE_CASE)) 2752 upper = Character.toUpperCase(ch); 2753 lower = Character.toLowerCase(upper); 2754 // Unicode case insensitive matches 2755 if (upper != lower) 2756 return SingleU(lower); 2757 else if (ASCII.isAscii(ch)) 2758 lower = ASCII.toLower(ch); 2759 upper = ASCII.toUpper(ch); 2760 // Case insensitive matches a given BMP character 2761 if (lower != upper) 2762 return SingleI(lower, upper); 2763 2764 2765 if (isSupplementary(ch)) 2766 return SingleS(ch); 2767 return Single(ch); // Match a given BMP character 2768 2769 2770 /** 2771 * Parse a single character or a character range in a character class 2772 * and return its representative node. 2773 */ 2774 private CharPredicate range(BitClass bits) 2775 int ch = peek(); 2776 if (ch == ‘\\‘) 2777 ch = nextEscaped(); 2778 if (ch == ‘p‘ || ch == ‘P‘) // A property 2779 boolean comp = (ch == ‘P‘); 2780 boolean oneLetter = true; 2781 // Consume if present 2782 ch = next(); 2783 if (ch != ‘‘) 2784 unread(); 2785 else 2786 oneLetter = false; 2787 return family(oneLetter, comp); 2788 else // ordinary escape 2789 boolean isrange = temp[cursor+1] == ‘-‘; 2790 unread(); 2791 ch = escape(true, true, isrange); 2792 if (ch == -1) 2793 return predicate; 2794 2795 else 2796 next(); 2797 2798 if (ch >= 0) 2799 if (peek() == ‘-‘) 2800 int endRange = temp[cursor+1]; 2801 if (endRange == ‘[‘) 2802 return bitsOrSingle(bits, ch); 2803 2804 if (endRange != ‘]‘) 2805 next(); 2806 int m = peek(); 2807 if (m == ‘\\‘) 2808 m = escape(true, false, true); 2809 else 2810 next(); 2811 2812 if (m < ch) 2813 throw error("Illegal character range"); 2814 2815 if (has(CASE_INSENSITIVE)) 2816 if (has(UNICODE_CASE)) 2817 return CIRangeU(ch, m); 2818 return CIRange(ch, m); 2819 else 2820 return Range(ch, m); 2821 2822 2823 2824 return bitsOrSingle(bits, ch); 2825 2826 throw error("Unexpected character ‘"+((char)ch)+"‘"); 2827 2828 2829 /** 2830 * Parses a Unicode character family and returns its representative node. 2831 */ 2832 private CharPredicate family(boolean singleLetter, boolean isComplement) 2833 next(); 2834 String name; 2835 CharPredicate p = null; 2836 2837 if (singleLetter) 2838 int c = temp[cursor]; 2839 if (!Character.isSupplementaryCodePoint(c)) 2840 name = String.valueOf((char)c); 2841 else 2842 name = new String(temp, cursor, 1); 2843 2844 read(); 2845 else 2846 int i = cursor; 2847 mark(‘‘); 2848 while(read() != ‘‘) 2849 2850 mark(‘\000‘); 2851 int j = cursor; 2852 if (j > patternLength) 2853 throw error("Unclosed character family"); 2854 if (i + 1 >= j) 2855 throw error("Empty character family"); 2856 name = new String(temp, i, j-i-1); 2857 2858 2859 int i = name.indexOf(‘=‘); 2860 if (i != -1) 2861 // property construct \pname=value 2862 String value = name.substring(i + 1); 2863 name = name.substring(0, i).toLowerCase(Locale.ENGLISH); 2864 switch (name) 2865 case "sc": 2866 case "script": 2867 p = CharPredicates.forUnicodeScript(value); 2868 break; 2869 case "blk": 2870 case "block": 2871 p = CharPredicates.forUnicodeBlock(value); 2872 break; 2873 case "gc": 2874 case "general_category": 2875 p = CharPredicates.forProperty(value); 2876 break; 2877 default: 2878 break; 2879 2880 if (p == null) 2881 throw error("Unknown Unicode property name=<" + name + ">, " 2882 + "value=<" + value + ">"); 2883 2884 else 2885 if (name.startsWith("In")) 2886 // \pInBlockName 2887 p = CharPredicates.forUnicodeBlock(name.substring(2)); 2888 else if (name.startsWith("Is")) 2889 // \pIsGeneralCategory and \pIsScriptName 2890 name = name.substring(2); 2891 p = CharPredicates.forUnicodeProperty(name); 2892 if (p == null) 2893 p = CharPredicates.forProperty(name); 2894 if (p == null) 2895 p = CharPredicates.forUnicodeScript(name); 2896 else 2897 if (has(UNICODE_CHARACTER_CLASS)) 2898 p = CharPredicates.forPOSIXName(name); 2899 2900 if (p == null) 2901 p = CharPredicates.forProperty(name); 2902 2903 if (p == null) 2904 throw error("Unknown character property name In/Is" + name + ""); 2905 2906 if (isComplement) 2907 // it might be too expensive to detect if a complement of 2908 // CharProperty can match "certain" supplementary. So just 2909 // go with StartS. 2910 hasSupplementary = true; 2911 p = p.negate(); 2912 2913 return p; 2914 2915 2916 private CharProperty newCharProperty(CharPredicate p) 2917 if (p == null) 2918 return null; 2919 if (p instanceof BmpCharPredicate) 2920 return new BmpCharProperty((BmpCharPredicate)p); 2921 else 2922 return new CharProperty(p); 2923 2924 2925 /** 2926 * Parses and returns the name of a "named capturing group", the trailing 2927 * ">" is consumed after parsing. 2928 */ 2929 private String groupname(int ch) 2930 StringBuilder sb = new StringBuilder(); 2931 if (!ASCII.isAlpha(ch)) 2932 throw error("capturing group name does not start with a Latin letter"); 2933 do 2934 sb.append((char) ch); 2935 while (ASCII.isAlnum(ch=read())); 2936 if (ch != ‘>‘) 2937 throw error("named capturing group is missing trailing ‘>‘"); 2938 return sb.toString(); 2939 2940 2941 /** 2942 * Parses a group and returns the head node of a set of nodes that process 2943 * the group. Sometimes a double return system is used where the tail is 2944 * returned in root. 2945 */ 2946 private Node group0() 2947 boolean capturingGroup = false; 2948 Node head = null; 2949 Node tail = null; 2950 int save = flags0; 2951 int saveTCNCount = topClosureNodes.size(); 2952 root = null; 2953 int ch = next(); 2954 if (ch == ‘?‘) 2955 ch = skip(); 2956 switch (ch) 2957 case ‘:‘: // (?:xxx) pure group 2958 head = createGroup(true); 2959 tail = root; 2960 head.next = expr(tail); 2961 break; 2962 case ‘=‘: // (?=xxx) and (?!xxx) lookahead 2963 case ‘!‘: 2964 head = createGroup(true); 2965 tail = root; 2966 head.next = expr(tail); 2967 if (ch == ‘=‘) 2968 head = tail = new Pos(head); 2969 else 2970 head = tail = new Neg(head); 2971 2972 break; 2973 case ‘>‘: // (?>xxx) independent group 2974 head = createGroup(true); 2975 tail = root; 2976 head.next = expr(tail); 2977 head = tail = new Ques(head, Qtype.INDEPENDENT); 2978 break; 2979 case ‘<‘: // (?<xxx) look behind 2980 ch = read(); 2981 if (ch != ‘=‘ && ch != ‘!‘) 2982 // named captured group 2983 String name = groupname(ch); 2984 if (namedGroups().containsKey(name)) 2985 throw error("Named capturing group <" + name 2986 + "> is already defined"); 2987 capturingGroup = true; 2988 head = createGroup(false); 2989 tail = root; 2990 namedGroups().put(name, capturingGroupCount-1); 2991 head.next = expr(tail); 2992 break; 2993 2994 int start = cursor; 2995 head = createGroup(true); 2996 tail = root; 2997 head.next = expr(tail); 2998 tail.next = lookbehindEnd; 2999 TreeInfo info = new TreeInfo(); 3000 head.study(info); 3001 if (info.maxValid == false) 3002 throw error("Look-behind group does not have " 3003 + "an obvious maximum length"); 3004 3005 boolean hasSupplementary = findSupplementary(start, patternLength); 3006 if (ch == ‘=‘) 3007 head = tail = (hasSupplementary ? 3008 new BehindS(head, info.maxLength, 3009 info.minLength) : 3010 new Behind(head, info.maxLength, 3011 info.minLength)); 3012 else // if (ch == ‘!‘) 3013 head = tail = (hasSupplementary ? 3014 new NotBehindS(head, info.maxLength, 3015 info.minLength) : 3016 new NotBehind(head, info.maxLength, 3017 info.minLength)); 3018 3019 // clear all top-closure-nodes inside lookbehind 3020 if (saveTCNCount < topClosureNodes.size()) 3021 topClosureNodes.subList(saveTCNCount, topClosureNodes.size()).clear(); 3022 break; 3023 case ‘$‘: 3024 case ‘@‘: 3025 throw error("Unknown group type"); 3026 default: // (?xxx:) inlined match flags 3027 unread(); 3028 addFlag(); 3029 ch = read(); 3030 if (ch == ‘)‘) 3031 return null; // Inline modifier only 3032 3033 if (ch != ‘:‘) 3034 throw error("Unknown inline modifier"); 3035 3036 head = createGroup(true); 3037 tail = root; 3038 head.next = expr(tail); 3039 break; 3040 3041 else // (xxx) a regular group 3042 capturingGroup = true; 3043 head = createGroup(false); 3044 tail = root; 3045 head.next = expr(tail); 3046 3047 3048 accept(‘)‘, "Unclosed group"); 3049 flags0 = save; 3050 3051 // Check for quantifiers 3052 Node node = closure(head); 3053 if (node == head) // No closure 3054 root = tail; 3055 return node; // Dual return 3056 3057 if (head == tail) // Zero length assertion 3058 root = node; 3059 return node; // Dual return 3060 3061 3062 // have group closure, clear all inner closure nodes from the 3063 // top list (no backtracking stopper optimization for inner 3064 if (saveTCNCount < topClosureNodes.size()) 3065 topClosureNodes.subList(saveTCNCount, topClosureNodes.size()).clear(); 3066 3067 if (node instanceof Ques) 3068 Ques ques = (Ques) node; 3069 if (ques.type == Qtype.POSSESSIVE) 3070 root = node; 3071 return node; 3072 3073 tail.next = new BranchConn(); 3074 tail = tail.next; 3075 if (ques.type == Qtype.GREEDY) 3076 head = new Branch(head, null, tail); 3077 else // Reluctant quantifier 3078 head = new Branch(null, head, tail); 3079 3080 root = tail; 3081 return head; 3082 else if (node instanceof Curly) 3083 Curly curly = (Curly) node; 3084 if (curly.type == Qtype.POSSESSIVE) 3085 root = node; 3086 return node; 3087 3088 // Discover if the group is deterministic 3089 TreeInfo info = new TreeInfo(); 3090 if (head.study(info)) // Deterministic 3091 GroupTail temp = (GroupTail) tail; 3092 head = root = new GroupCurly(head.next, curly.cmin, 3093 curly.cmax, curly.type, 3094 ((GroupTail)tail).localIndex, 3095 ((GroupTail)tail).groupIndex, 3096 capturingGroup); 3097 return head; 3098 else // Non-deterministic 3099 int temp = ((GroupHead) head).localIndex; 3100 Loop loop; 3101 if (curly.type == Qtype.GREEDY) 3102 loop = new Loop(this.localCount, temp); 3103 // add the max_reps greedy to the top-closure-node list 3104 if (curly.cmax == MAX_REPS) 3105 topClosureNodes.add(loop); 3106 else // Reluctant Curly 3107 loop = new LazyLoop(this.localCount, temp); 3108 3109 Prolog prolog = new Prolog(loop); 3110 this.localCount += 1; 3111 loop.cmin = curly.cmin; 3112 loop.cmax = curly.cmax; 3113 loop.body = head; 3114 tail.next = loop; 3115 root = loop; 3116 return prolog; // Dual return 3117 3118 3119 throw error("Internal logic error"); 3120 3121 3122 /** 3123 * Create group head and tail nodes using double return. If the group is 3124 * created with anonymous true then it is a pure group and should not 3125 * affect group counting. 3126 */ 3127 private Node createGroup(boolean anonymous) 3128 int localIndex = localCount++; 3129 int groupIndex = 0; 3130 if (!anonymous) 3131 groupIndex = capturingGroupCount++; 3132 GroupHead head = new GroupHead(localIndex); 3133 root = new GroupTail(localIndex, groupIndex); 3134 3135 // for debug/print only, head.match does NOT need the "tail" info 3136 head.tail = (GroupTail)root; 3137 3138 if (!anonymous && groupIndex < 10) 3139 groupNodes[groupIndex] = head; 3140 return head; 3141 3142 3143 @SuppressWarnings("fallthrough") 3144 /** 3145 * Parses inlined match flags and set them appropriately. 3146 */ 3147 private void addFlag() 3148 int ch = peek(); 3149 for (;;) 3150 switch (ch) 3151 case ‘i‘: 3152 flags0 |= CASE_INSENSITIVE; 3153 break; 3154 case ‘m‘: 3155 flags0 |= MULTILINE; 3156 break; 3157 case ‘s‘: 3158 flags0 |= DOTALL; 3159 break; 3160 case ‘d‘: 3161 flags0 |= UNIX_LINES; 3162 break; 3163 case ‘u‘: 3164 flags0 |= UNICODE_CASE; 3165 break; 3166 case ‘c‘: 3167 flags0 |= CANON_EQ; 3168 break; 3169 case ‘x‘: 3170 flags0 |= COMMENTS; 3171 break; 3172 case ‘U‘: 3173 flags0 |= (UNICODE_CHARACTER_CLASS | UNICODE_CASE); 3174 break; 3175 case ‘-‘: // subFlag then fall through 3176 ch = next(); 3177 subFlag(); 3178 default: 3179 return; 3180 3181 ch = next(); 3182 3183 3184 3185 @SuppressWarnings("fallthrough") 3186 /** 3187 * Parses the second part of inlined match flags and turns off 3188 * flags appropriately. 3189 */ 3190 private void subFlag() 3191 int ch = peek(); 3192 for (;;) 3193 switch (ch) 3194 case ‘i‘: 3195 flags0 &= ~CASE_INSENSITIVE; 3196 break; 3197 case ‘m‘: 3198 flags0 &= ~MULTILINE; 3199 break; 3200 case ‘s‘: 3201 flags0 &= ~DOTALL; 3202 break; 3203 case ‘d‘: 3204 flags0 &= ~UNIX_LINES; 3205 break; 3206 case ‘u‘: 3207 flags0 &= ~UNICODE_CASE; 3208 break; 3209 case ‘c‘: 3210 flags0 &= ~CANON_EQ; 3211 break; 3212 case ‘x‘: 3213 flags0 &= ~COMMENTS; 3214 break; 3215 case ‘U‘: 3216 flags0 &= ~(UNICODE_CHARACTER_CLASS | UNICODE_CASE); 3217 break; 3218 default: 3219 return; 3220 3221 ch = next(); 3222 3223 3224 3225 static final int MAX_REPS = 0x7FFFFFFF; 3226 3227 static enum Qtype 3228 GREEDY, LAZY, POSSESSIVE, INDEPENDENT 3229 3230 3231 private Node curly(Node prev, int cmin) 3232 int ch = next(); 3233 if (ch == ‘?‘) 3234 next(); 3235 return new Curly(prev, cmin, MAX_REPS, Qtype.LAZY); 3236 else if (ch == ‘+‘) 3237 next(); 3238 return new Curly(prev, cmin, MAX_REPS, Qtype.POSSESSIVE); 3239 3240 if (prev instanceof BmpCharProperty) 3241 return new BmpCharPropertyGreedy((BmpCharProperty)prev, cmin); 3242 else if (prev instanceof CharProperty) 3243 return new CharPropertyGreedy((CharProperty)prev, cmin); 3244 3245 return new Curly(prev, cmin, MAX_REPS, Qtype.GREEDY); 3246 3247 3248 /** 3249 * Processes repetition. If the next character peeked is a quantifier 3250 * then new nodes must be appended to handle the repetition. 3251 * Prev could be a single or a group, so it could be a chain of nodes. 3252 */ 3253 private Node closure(Node prev) 3254 Node atom; 3255 int ch = peek(); 3256 switch (ch) 3257 case ‘?‘: 3258 ch = next(); 3259 if (ch == ‘?‘) 3260 next(); 3261 return new Ques(prev, Qtype.LAZY); 3262 else if (ch == ‘+‘) 3263 next(); 3264 return new Ques(prev, Qtype.POSSESSIVE); 3265 3266 return new Ques(prev, Qtype.GREEDY); 3267 case ‘*‘: 3268 return curly(prev, 0); 3269 case ‘+‘: 3270 return curly(prev, 1); 3271 case ‘‘: 3272 ch = temp[cursor+1]; 3273 if (ASCII.isDigit(ch)) 3274 skip(); 3275 int cmin = 0; 3276 do 3277 cmin = cmin * 10 + (ch - ‘0‘); 3278 while (ASCII.isDigit(ch = read())); 3279 int cmax = cmin; 3280 if (ch == ‘,‘) 3281 ch = read(); 3282 cmax = MAX_REPS; 3283 if (ch != ‘‘) 3284 cmax = 0; 3285 while (ASCII.isDigit(ch)) 3286 cmax = cmax * 10 + (ch - ‘0‘); 3287 ch = read(); 3288 3289 3290 3291 if (ch != ‘‘) 3292 throw error("Unclosed counted closure"); 3293 if (((cmin) | (cmax) | (cmax - cmin)) < 0) 3294 throw error("Illegal repetition range"); 3295 Curly curly; 3296 ch = peek(); 3297 if (ch == ‘?‘) 3298 next(); 3299 curly = new Curly(prev, cmin, cmax, Qtype.LAZY); 3300 else if (ch == ‘+‘) 3301 next(); 3302 curly = new Curly(prev, cmin, cmax, Qtype.POSSESSIVE); 3303 else 3304 curly = new Curly(prev, cmin, cmax, Qtype.GREEDY); 3305 3306 return curly; 3307 else 3308 throw error("Illegal repetition"); 3309 3310 default: 3311 return prev; 3312 3313 3314 3315 /** 3316 * Utility method for parsing control escape sequences. 3317 */ 3318 private int c() 3319 if (cursor < patternLength) 3320 return read() ^ 64; 3321 3322 throw error("Illegal control escape sequence"); 3323 3324 3325 /** 3326 * Utility method for parsing octal escape sequences. 3327 */ 3328 private int o() 3329 int n = read(); 3330 if (((n-‘0‘)|(‘7‘-n)) >= 0) 3331 int m = read(); 3332 if (((m-‘0‘)|(‘7‘-m)) >= 0) 3333 int o = read(); 3334 if ((((o-‘0‘)|(‘7‘-o)) >= 0) && (((n-‘0‘)|(‘3‘-n)) >= 0)) 3335 return (n - ‘0‘) * 64 + (m - ‘0‘) * 8 + (o - ‘0‘); 3336 3337 unread(); 3338 return (n - ‘0‘) * 8 + (m - ‘0‘); 3339 3340 unread(); 3341 return (n - ‘0‘); 3342 3343 throw error("Illegal octal escape sequence"); 3344 3345 3346 /** 3347 * Utility method for parsing hexadecimal escape sequences. 3348 */ 3349 private int x() 3350 int n = read(); 3351 if (ASCII.isHexDigit(n)) 3352 int m = read(); 3353 if (ASCII.isHexDigit(m)) 3354 return ASCII.toDigit(n) * 16 + ASCII.toDigit(m); 3355 3356 else if (n == ‘‘ && ASCII.isHexDigit(peek())) 3357 int ch = 0; 3358 while (ASCII.isHexDigit(n = read())) 3359 ch = (ch << 4) + ASCII.toDigit(n); 3360 if (ch > Character.MAX_CODE_POINT) 3361 throw error("Hexadecimal codepoint is too big"); 3362 3363 if (n != ‘‘) 3364 throw error("Unclosed hexadecimal escape sequence"); 3365 return ch; 3366 3367 throw error("Illegal hexadecimal escape sequence"); 3368 3369 3370 /** 3371 * Utility method for parsing unicode escape sequences. 3372 */ 3373 private int cursor() 3374 return cursor; 3375 3376 3377 private void setcursor(int pos) 3378 cursor = pos; 3379 3380 3381 private int uxxxx() 3382 int n = 0; 3383 for (int i = 0; i < 4; i++) 3384 int ch = read(); 3385 if (!ASCII.isHexDigit(ch)) 3386 throw error("Illegal Unicode escape sequence"); 3387 3388 n = n * 16 + ASCII.toDigit(ch); 3389 3390 return n; 3391 3392 3393 private int u() 3394 int n = uxxxx(); 3395 if (Character.isHighSurrogate((char)n)) 3396 int cur = cursor(); 3397 if (read() == ‘\\‘ && read() == ‘u‘) 3398 int n2 = uxxxx(); 3399 if (Character.isLowSurrogate((char)n2)) 3400 return Character.toCodePoint((char)n, (char)n2); 3401 3402 setcursor(cur); 3403 3404 return n; 3405 3406 3407 private int N() 3408 if (read() == ‘‘) 3409 int i = cursor; 3410 while (cursor < patternLength && read() != ‘‘) 3411 if (cursor > patternLength) 3412 throw error("Unclosed character name escape sequence"); 3413 String name = new String(temp, i, cursor - i - 1); 3414 try 3415 return Character.codePointOf(name); 3416 catch (IllegalArgumentException x) 3417 throw error("Unknown character name [" + name + "]"); 3418 3419 3420 throw error("Illegal character name escape sequence"); 3421 3422 3423 // 3424 // Utility methods for code point support 3425 // 3426 private static final int countChars(CharSequence seq, int index, 3427 int lengthInCodePoints) 3428 // optimization 3429 if (lengthInCodePoints == 1 && !Character.isHighSurrogate(seq.charAt(index))) 3430 assert (index >= 0 && index < seq.length()); 3431 return 1; 3432 3433 int length = seq.length(); 3434 int x = index; 3435 if (lengthInCodePoints >= 0) 3436 assert (index >= 0 && index < length); 3437 for (int i = 0; x < length && i < lengthInCodePoints; i++) 3438 if (Character.isHighSurrogate(seq.charAt(x++))) 3439 if (x < length && Character.isLowSurrogate(seq.charAt(x))) 3440 x++; 3441 3442 3443 3444 return x - index; 3445 3446 3447 assert (index >= 0 && index <= length); 3448 if (index == 0) 3449 return 0; 3450 3451 int len = -lengthInCodePoints; 3452 for (int i = 0; x > 0 && i < len; i++) 3453 if (Character.isLowSurrogate(seq.charAt(--x))) 3454 if (x > 0 && Character.isHighSurrogate(seq.charAt(x-1))) 3455 x--; 3456 3457 3458 3459 return index - x; 3460 3461 3462 private static final int countCodePoints(CharSequence seq) 3463 int length = seq.length(); 3464 int n = 0; 3465 for (int i = 0; i < length; ) 3466 n++; 3467 if (Character.isHighSurrogate(seq.charAt(i++))) 3468 if (i < length && Character.isLowSurrogate(seq.charAt(i))) 3469 i++; 3470 3471 3472 3473 return n; 3474 3475 3476 /** 3477 * Creates a bit vector for matching Latin-1 values. A normal BitClass 3478 * never matches values above Latin-1, and a complemented BitClass always 3479 * matches values above Latin-1. 3480 */ 3481 static final class BitClass extends BmpCharProperty 3482 final boolean[] bits; 3483 BitClass() 3484 this(new boolean[256]); 3485 3486 private BitClass(boolean[] bits) 3487 super( ch -> ch < 256 && bits[ch]); 3488 this.bits = bits; 3489 3490 BitClass add(int c, int flags) 3491 assert c >= 0 && c <= 255; 3492 if ((flags & CASE_INSENSITIVE) != 0) 3493 if (ASCII.isAscii(c)) 3494 bits[ASCII.toUpper(c)] = true; 3495 bits[ASCII.toLower(c)] = true; 3496 else if ((flags & UNICODE_CASE) != 0) 3497 bits[Character.toLowerCase(c)] = true; 3498 bits[Character.toUpperCase(c)] = true; 3499 3500 3501 bits[c] = true; 3502 return this; 3503 3504 3505 3506 /** 3507 * Utility method for creating a string slice matcher. 3508 */ 3509 private Node newSlice(int[] buf, int count, boolean hasSupplementary) 3510 int[] tmp = new int[count]; 3511 if (has(CASE_INSENSITIVE)) 3512 if (has(UNICODE_CASE)) 3513 for (int i = 0; i < count; i++) 3514 tmp[i] = Character.toLowerCase( 3515 Character.toUpperCase(buf[i])); 3516 3517 return hasSupplementary? new SliceUS(tmp) : new SliceU(tmp); 3518 3519 for (int i = 0; i < count; i++) 3520 tmp[i] = ASCII.toLower(buf[i]); 3521 3522 return hasSupplementary? new SliceIS(tmp) : new SliceI(tmp); 3523 3524 for (int i = 0; i < count; i++) 3525 tmp[i] = buf[i]; 3526 3527 return hasSupplementary ? new SliceS(tmp) : new Slice(tmp); 3528 3529 3530 /** 3531 * The following classes are the building components of the object 3532 * tree that represents a compiled regular expression. The object tree 3533 * is made of individual elements that handle constructs in the Pattern. 3534 * Each type of object knows how to match its equivalent construct with 3535 * the match() method. 3536 */ 3537 3538 /** 3539 * Base class for all node classes. Subclasses should override the match() 3540 * method as appropriate. This class is an accepting node, so its match() 3541 * always returns true. 3542 */ 3543 static class Node extends Object 3544 Node next; 3545 Node() 3546 next = Pattern.accept; 3547 3548 /** 3549 * This method implements the classic accept node. 3550 */ 3551 boolean match(Matcher matcher, int i, CharSequence seq) 3552 matcher.last = i; 3553 matcher.groups[0] = matcher.first; 3554 matcher.groups[1] = matcher.last; 3555 return true; 3556 3557 /** 3558 * This method is good for all zero length assertions. 3559 */ 3560 boolean study(TreeInfo info) 3561 if (next != null) 3562 return next.study(info); 3563 else 3564 return info.deterministic; 3565 3566 3567 3568 3569 static class LastNode extends Node 3570 /** 3571 * This method implements the classic accept node with 3572 * the addition of a check to see if the match occurred 3573 * using all of the input. 3574 */ 3575 boolean match(Matcher matcher, int i, CharSequence seq) 3576 if (matcher.acceptMode == Matcher.ENDANCHOR && i != matcher.to) 3577 return false; 3578 matcher.last = i; 3579 matcher.groups[0] = matcher.first; 3580 matcher.groups[1] = matcher.last; 3581 return true; 3582 3583 3584 3585 /** 3586 * Used for REs that can start anywhere within the input string. 3587 * This basically tries to match repeatedly at each spot in the 3588 * input string, moving forward after each try. An anchored search 3589 * or a BnM will bypass this node completely. 3590 */ 3591 static class Start extends Node 3592 int minLength; 3593 Start(Node node) 3594 this.next = node; 3595 TreeInfo info = new TreeInfo(); 3596 next.study(info); 3597 minLength = info.minLength; 3598 3599 boolean match(Matcher matcher, int i, CharSequence seq) 3600 if (i > matcher.to - minLength) 3601 matcher.hitEnd = true; 3602 return false; 3603 3604 int guard = matcher.to - minLength; 3605 for (; i <= guard; i++) 3606 if (next.match(matcher, i, seq)) 3607 matcher.first = i; 3608 matcher.groups[0] = matcher.first; 3609 matcher.groups[1] = matcher.last; 3610 return true; 3611 3612 3613 matcher.hitEnd = true; 3614 return false; 3615 3616 boolean study(TreeInfo info) 3617 next.study(info); 3618 info.maxValid = false; 3619 info.deterministic = false; 3620 return false; 3621 3622 3623 3624 /* 3625 * StartS supports supplementary characters, including unpaired surrogates. 3626 */ 3627 static final class StartS extends Start 3628 StartS(Node node) 3629 super(node); 3630 3631 boolean match(Matcher matcher, int i, CharSequence seq) 3632 if (i > matcher.to - minLength) 3633 matcher.hitEnd = true; 3634 return false; 3635 3636 int guard = matcher.to - minLength; 3637 while (i <= guard) 3638 //if ((ret = next.match(matcher, i, seq)) || i == guard) 3639 if (next.match(matcher, i, seq)) 3640 matcher.first = i; 3641 matcher.groups[0] = matcher.first; 3642 matcher.groups[1] = matcher.last; 3643 return true; 3644 3645 if (i == guard) 3646 break; 3647 // Optimization to move to the next character. This is 3648 // faster than countChars(seq, i, 1). 3649 if (Character.isHighSurrogate(seq.charAt(i++))) 3650 if (i < seq.length() && 3651 Character.isLowSurrogate(seq.charAt(i))) 3652 i++; 3653 3654 3655 3656 matcher.hitEnd = true; 3657 return false; 3658 3659 3660 3661 /** 3662 * Node to anchor at the beginning of input. This object implements the 3663 * match for a \A sequence, and the caret anchor will use this if not in 3664 * multiline mode. 3665 */ 3666 static final class Begin extends Node 3667 boolean match(Matcher matcher, int i, CharSequence seq) 3668 int fromIndex = (matcher.anchoringBounds) ? 3669 matcher.from : 0; 3670 if (i == fromIndex && next.match(matcher, i, seq)) 3671 matcher.first = i; 3672 matcher.groups[0] = i; 3673 matcher.groups[1] = matcher.last; 3674 return true; 3675 else 3676 return false; 3677 3678 3679 3680 3681 /** 3682 * Node to anchor at the end of input. This is the absolute end, so this 3683 * should not match at the last newline before the end as $ will. 3684 */ 3685 static final class End extends Node 3686 boolean match(Matcher matcher, int i, CharSequence seq) 3687 int endIndex = (matcher.anchoringBounds) ? 3688 matcher.to : matcher.getTextLength(); 3689 if (i == endIndex) 3690 matcher.hitEnd = true; 3691 return next.match(matcher, i, seq); 3692 3693 return false; 3694 3695 3696 3697 /** 3698 * Node to anchor at the beginning of a line. This is essentially the 3699 * object to match for the multiline ^. 3700 */ 3701 static final class Caret extends Node 3702 boolean match(Matcher matcher, int i, CharSequence seq) 3703 int startIndex = matcher.from; 3704 int endIndex = matcher.to; 3705 if (!matcher.anchoringBounds) 3706 startIndex = 0; 3707 endIndex = matcher.getTextLength(); 3708 3709 // Perl does not match ^ at end of input even after newline 3710 if (i == endIndex) 3711 matcher.hitEnd = true; 3712 return false; 3713 3714 if (i > startIndex) 3715 char ch = seq.charAt(i-1); 3716 if (ch != ‘\n‘ && ch != ‘\r‘ 3717 && (ch|1) != ‘\u2029‘ 3718 && ch != ‘\u0085‘ ) 3719 return false; 3720 3721 // Should treat /r/n as one newline 3722 if (ch == ‘\r‘ && seq.charAt(i) == ‘\n‘) 3723 return false; 3724 3725 return next.match(matcher, i, seq); 3726 3727 3728 3729 /** 3730 * Node to anchor at the beginning of a line when in unixdot mode. 3731 */ 3732 static final class UnixCaret extends Node 3733 boolean match(Matcher matcher, int i, CharSequence seq) 3734 int startIndex = matcher.from; 3735 int endIndex = matcher.to; 3736 if (!matcher.anchoringBounds) 3737 startIndex = 0; 3738 endIndex = matcher.getTextLength(); 3739 3740 // Perl does not match ^ at end of input even after newline 3741 if (i == endIndex) 3742 matcher.hitEnd = true; 3743 return false; 3744 3745 if (i > startIndex) 3746 char ch = seq.charAt(i-1); 3747 if (ch != ‘\n‘) 3748 return false; 3749 3750 3751 return next.match(matcher, i, seq); 3752 3753 3754 3755 /** 3756 * Node to match the location where the last match ended. 3757 * This is used for the \G construct. 3758 */ 3759 static final class LastMatch extends Node 3760 boolean match(Matcher matcher, int i, CharSequence seq) 3761 if (i != matcher.oldLast) 3762 return false; 3763 return next.match(matcher, i, seq); 3764 3765 3766 3767 /** 3768 * Node to anchor at the end of a line or the end of input based on the 3769 * multiline mode. 3770 * 3771 * When not in multiline mode, the $ can only match at the very end 3772 * of the input, unless the input ends in a line terminator in which 3773 * it matches right before the last line terminator. 3774 * 3775 * Note that \r\n is considered an atomic line terminator. 3776 * 3777 * Like ^ the $ operator matches at a position, it does not match the 3778 * line terminators themselves. 3779 */ 3780 static final class Dollar extends Node 3781 boolean multiline; 3782 Dollar(boolean mul) 3783 multiline = mul; 3784 3785 boolean match(Matcher matcher, int i, CharSequence seq) 3786 int endIndex = (matcher.anchoringBounds) ? 3787 matcher.to : matcher.getTextLength(); 3788 if (!multiline) 3789 if (i < endIndex - 2) 3790 return false; 3791 if (i == endIndex - 2) 3792 char ch = seq.charAt(i); 3793 if (ch != ‘\r‘) 3794 return false; 3795 ch = seq.charAt(i + 1); 3796 if (ch != ‘\n‘) 3797 return false; 3798 3799 3800 // Matches before any line terminator; also matches at the 3801 // end of input 3802 // Before line terminator: 3803 // If multiline, we match here no matter what 3804 // If not multiline, fall through so that the end 3805 // is marked as hit; this must be a /r/n or a /n 3806 // at the very end so the end was hit; more input 3807 // could make this not match here 3808 if (i < endIndex) 3809 char ch = seq.charAt(i); 3810 if (ch == ‘\n‘) 3811 // No match between \r\n 3812 if (i > 0 && seq.charAt(i-1) == ‘\r‘) 3813 return false; 3814 if (multiline) 3815 return next.match(matcher, i, seq); 3816 else if (ch == ‘\r‘ || ch == ‘\u0085‘ || 3817 (ch|1) == ‘\u2029‘) 3818 if (multiline) 3819 return next.match(matcher, i, seq); 3820 else // No line terminator, no match 3821 return false; 3822 3823 3824 // Matched at current end so hit end 3825 matcher.hitEnd = true; 3826 // If a $ matches because of end of input, then more input 3827 // could cause it to fail! 3828 matcher.requireEnd = true; 3829 return next.match(matcher, i, seq); 3830 3831 boolean study(TreeInfo info) 3832 next.study(info); 3833 return info.deterministic; 3834 3835 3836 3837 /** 3838 * Node to anchor at the end of a line or the end of input based on the 3839 * multiline mode when in unix lines mode. 3840 */ 3841 static final class UnixDollar extends Node 3842 boolean multiline; 3843 UnixDollar(boolean mul) 3844 multiline = mul; 3845 3846 boolean match(Matcher matcher, int i, CharSequence seq) 3847 int endIndex = (matcher.anchoringBounds) ? 3848 matcher.to : matcher.getTextLength(); 3849 if (i < endIndex) 3850 char ch = seq.charAt(i); 3851 if (ch == ‘\n‘) 3852 // If not multiline, then only possible to 3853 // match at very end or one before end 3854 if (multiline == false && i != endIndex - 1) 3855 return false; 3856 // If multiline return next.match without setting 3857 // matcher.hitEnd 3858 if (multiline) 3859 return next.match(matcher, i, seq); 3860 else 3861 return false; 3862 3863 3864 // Matching because at the end or 1 before the end; 3865 // more input could change this so set hitEnd 3866 matcher.hitEnd = true; 3867 // If a $ matches because of end of input, then more input 3868 // could cause it to fail! 3869 matcher.requireEnd = true; 3870 return next.match(matcher, i, seq); 3871 3872 boolean study(TreeInfo info) 3873 next.study(info); 3874 return info.deterministic; 3875 3876 3877 3878 /** 3879 * Node class that matches a Unicode line ending ‘\R‘ 3880 */ 3881 static final class LineEnding extends Node 3882 boolean match(Matcher matcher, int i, CharSequence seq) 3883 // (u+000Du+000A|[u+000Au+000Bu+000Cu+000Du+0085u+2028u+2029]) 3884 if (i < matcher.to) 3885 int ch = seq.charAt(i); 3886 if (ch == 0x0A || ch == 0x0B || ch == 0x0C || 3887 ch == 0x85 || ch == 0x2028 || ch == 0x2029) 3888 return next.match(matcher, i + 1, seq); 3889 if (ch == 0x0D) 3890 i++; 3891 if (i < matcher.to) 3892 if (seq.charAt(i) == 0x0A && 3893 next.match(matcher, i + 1, seq)) 3894 return true; 3895 3896 else 3897 matcher.hitEnd = true; 3898 3899 return next.match(matcher, i, seq); 3900 3901 else 3902 matcher.hitEnd = true; 3903 3904 return false; 3905 3906 boolean study(TreeInfo info) 3907 info.minLength++; 3908 info.maxLength += 2; 3909 return next.study(info); 3910 3911 3912 3913 /** 3914 * Abstract node class to match one character satisfying some 3915 * boolean property. 3916 */ 3917 static class CharProperty extends Node 3918 CharPredicate predicate; 3919 3920 CharProperty (CharPredicate predicate) 3921 this.predicate = predicate; 3922 3923 boolean match(Matcher matcher, int i, CharSequence seq) 3924 if (i < matcher.to) 3925 int ch = Character.codePointAt(seq, i); 3926 return predicate.is(ch) && 3927 next.match(matcher, i + Character.charCount(ch), seq); 3928 else 3929 matcher.hitEnd = true; 3930 return false; 3931 3932 3933 boolean study(TreeInfo info) 3934 info.minLength++; 3935 info.maxLength++; 3936 return next.study(info); 3937 3938 3939 3940 /** 3941 * Optimized version of CharProperty that works only for 3942 * properties never satisfied by Supplementary characters. 3943 */ 3944 private static class BmpCharProperty extends CharProperty 3945 BmpCharProperty (BmpCharPredicate predicate) 3946 super(predicate); 3947 3948 boolean match(Matcher matcher, int i, CharSequence seq) 3949 if (i < matcher.to) 3950 return predicate.is(seq.charAt(i)) && 3951 next.match(matcher, i + 1, seq); 3952 else 3953 matcher.hitEnd = true; 3954 return false; 3955 3956 3957 3958 3959 private static class NFCCharProperty extends Node 3960 CharPredicate predicate; 3961 NFCCharProperty (CharPredicate predicate) 3962 this.predicate = predicate; 3963 3964 3965 boolean match(Matcher matcher, int i, CharSequence seq) 3966 if (i < matcher.to) 3967 int ch0 = Character.codePointAt(seq, i); 3968 int n = Character.charCount(ch0); 3969 int j = i + n; 3970 while (j < matcher.to) 3971 int ch1 = Character.codePointAt(seq, j); 3972 if (Grapheme.isBoundary(ch0, ch1)) 3973 break; 3974 ch0 = ch1; 3975 j += Character.charCount(ch1); 3976 3977 if (i + n == j) // single, assume nfc cp 3978 if (predicate.is(ch0)) 3979 return next.match(matcher, j, seq); 3980 else 3981 while (i + n < j) 3982 String nfc = Normalizer.normalize( 3983 seq.toString().substring(i, j), Normalizer.Form.NFC); 3984 if (nfc.codePointCount(0, nfc.length()) == 1) 3985 if (predicate.is(nfc.codePointAt(0)) && 3986 next.match(matcher, j, seq)) 3987 return true; 3988 3989 3990 3991 ch0 = Character.codePointBefore(seq, j); 3992 j -= Character.charCount(ch0); 3993 3994 3995 if (j < matcher.to) 3996 return false; 3997 3998 matcher.hitEnd = true; 3999 return false; 4000 4001 4002 boolean study(TreeInfo info) 4003 info.minLength++; 4004 info.deterministic = false; 4005 return next.study(info); 4006 4007 4008 4009 /** 4010 * Node class that matches an unicode extended grapheme cluster 4011 */ 4012 static class XGrapheme extends Node 4013 boolean match(Matcher matcher, int i, CharSequence seq) 4014 if (i < matcher.to) 4015 int ch0 = Character.codePointAt(seq, i); 4016 i += Character.charCount(ch0); 4017 while (i < matcher.to) 4018 int ch1 = Character.codePointAt(seq, i); 4019 if (Grapheme.isBoundary(ch0, ch1)) 4020 break; 4021 ch0 = ch1; 4022 i += Character.charCount(ch1); 4023 4024 return next.match(matcher, i, seq); 4025 4026 matcher.hitEnd = true; 4027 return false; 4028 4029 4030 boolean study(TreeInfo info) 4031 info.minLength++; 4032 info.deterministic = false; 4033 return next.study(info); 4034 4035 4036 4037 /** 4038 * Node class that handles grapheme boundaries 4039 */ 4040 static class GraphemeBound extends Node 4041 boolean match(Matcher matcher, int i, CharSequence seq) 4042 int startIndex = matcher.from; 4043 int endIndex = matcher.to; 4044 if (matcher.transparentBounds) 4045 startIndex = 0; 4046 endIndex = matcher.getTextLength(); 4047 4048 if (i == startIndex) 4049 return next.match(matcher, i, seq); 4050 4051 if (i < endIndex) 4052 if (Character.isSurrogatePair(seq.charAt(i-1), seq.charAt(i)) || 4053 !Grapheme.isBoundary(Character.codePointBefore(seq, i), 4054 Character.codePointAt(seq, i))) 4055 return false; 4056 4057 else 4058 matcher.hitEnd = true; 4059 matcher.requireEnd = true; 4060 4061 return next.match(matcher, i, seq); 4062 4063 4064 4065 /** 4066 * Base class for all Slice nodes 4067 */ 4068 static class SliceNode extends Node 4069 int[] buffer; 4070 SliceNode(int[] buf) 4071 buffer = buf; 4072 4073 boolean study(TreeInfo info) 4074 info.minLength += buffer.length; 4075 info.maxLength += buffer.length; 4076 return next.study(info); 4077 4078 4079 4080 /** 4081 * Node class for a case sensitive/BMP-only sequence of literal 4082 * characters. 4083 */ 4084 static class Slice extends SliceNode 4085 Slice(int[] buf) 4086 super(buf); 4087 4088 boolean match(Matcher matcher, int i, CharSequence seq) 4089 int[] buf = buffer; 4090 int len = buf.length; 4091 for (int j=0; j<len; j++) 4092 if ((i+j) >= matcher.to) 4093 matcher.hitEnd = true; 4094 return false; 4095 4096 if (buf[j] != seq.charAt(i+j)) 4097 return false; 4098 4099 return next.match(matcher, i+len, seq); 4100 4101 4102 4103 /** 4104 * Node class for a case_insensitive/BMP-only sequence of literal 4105 * characters. 4106 */ 4107 static class SliceI extends SliceNode 4108 SliceI(int[] buf) 4109 super(buf); 4110 4111 boolean match(Matcher matcher, int i, CharSequence seq) 4112 int[] buf = buffer; 4113 int len = buf.length; 4114 for (int j=0; j<len; j++) 4115 if ((i+j) >= matcher.to) 4116 matcher.hitEnd = true; 4117 return false; 4118 4119 int c = seq.charAt(i+j); 4120 if (buf[j] != c && 4121 buf[j] != ASCII.toLower(c)) 4122 return false; 4123 4124 return next.match(matcher, i+len, seq); 4125 4126 4127 4128 /** 4129 * Node class for a unicode_case_insensitive/BMP-only sequence of 4130 * literal characters. Uses unicode case folding. 4131 */ 4132 static final class SliceU extends SliceNode 4133 SliceU(int[] buf) 4134 super(buf); 4135 4136 boolean match(Matcher matcher, int i, CharSequence seq) 4137 int[] buf = buffer; 4138 int len = buf.length; 4139 for (int j=0; j<len; j++) 4140 if ((i+j) >= matcher.to) 4141 matcher.hitEnd = true; 4142 return false; 4143 4144 int c = seq.charAt(i+j); 4145 if (buf[j] != c && 4146 buf[j] != Character.toLowerCase(Character.toUpperCase(c))) 4147 return false; 4148 4149 return next.match(matcher, i+len, seq); 4150 4151 4152 4153 /** 4154 * Node class for a case sensitive sequence of literal characters 4155 * including supplementary characters. 4156 */ 4157 static final class SliceS extends Slice 4158 SliceS(int[] buf) 4159 super(buf); 4160 4161 boolean match(Matcher matcher, int i, CharSequence seq) 4162 int[] buf = buffer; 4163 int x = i; 4164 for (int j = 0; j < buf.length; j++) 4165 if (x >= matcher.to) 4166 matcher.hitEnd = true; 4167 return false; 4168 4169 int c = Character.codePointAt(seq, x); 4170 if (buf[j] != c) 4171 return false; 4172 x += Character.charCount(c); 4173 if (x > matcher.to) 4174 matcher.hitEnd = true; 4175 return false; 4176 4177 4178 return next.match(matcher, x, seq); 4179 4180 4181 4182 /** 4183 * Node class for a case insensitive sequence of literal characters 4184 * including supplementary characters. 4185 */ 4186 static class SliceIS extends SliceNode 4187 SliceIS(int[] buf) 4188 super(buf); 4189 4190 int toLower(int c) 4191 return ASCII.toLower(c); 4192 4193 boolean match(Matcher matcher, int i, CharSequence seq) 4194 int[] buf = buffer; 4195 int x = i; 4196 for (int j = 0; j < buf.length; j++) 4197 if (x >= matcher.to) 4198 matcher.hitEnd = true; 4199 return false; 4200 4201 int c = Character.codePointAt(seq, x); 4202 if (buf[j] != c && buf[j] != toLower(c)) 4203 return false; 4204 x += Character.charCount(c); 4205 if (x > matcher.to) 4206 matcher.hitEnd = true; 4207 return false; 4208 4209 4210 return next.match(matcher, x, seq); 4211 4212 4213 4214 /** 4215 * Node class for a case insensitive sequence of literal characters. 4216 * Uses unicode case folding. 4217 */ 4218 static final class SliceUS extends SliceIS 4219 SliceUS(int[] buf) 4220 super(buf); 4221 4222 int toLower(int c) 4223 return Character.toLowerCase(Character.toUpperCase(c)); 4224 4225 4226 4227 /** 4228 * The 0 or 1 quantifier. This one class implements all three types. 4229 */ 4230 static final class Ques extends Node 4231 Node atom; 4232 Qtype type; 4233 Ques(Node node, Qtype type) 4234 this.atom = node; 4235 this.type = type; 4236 4237 boolean match(Matcher matcher, int i, CharSequence seq) 4238 switch (type) 4239 case GREEDY: 4240 return (atom.match(matcher, i, seq) && next.match(matcher, matcher.last, seq)) 4241 || next.match(matcher, i, seq); 4242 case LAZY: 4243 return next.match(matcher, i, seq) 4244 || (atom.match(matcher, i, seq) && next.match(matcher, matcher.last, seq)); 4245 case POSSESSIVE: 4246 if (atom.match(matcher, i, seq)) i = matcher.last; 4247 return next.match(matcher, i, seq); 4248 default: 4249 return atom.match(matcher, i, seq) && next.match(matcher, matcher.last, seq); 4250 4251 4252 boolean study(TreeInfo info) 4253 if (type != Qtype.INDEPENDENT) 4254 int minL = info.minLength; 4255 atom.study(info); 4256 info.minLength = minL; 4257 info.deterministic = false; 4258 return next.study(info); 4259 else 4260 atom.study(info); 4261 return next.study(info); 4262 4263 4264 4265 4266 /** 4267 * Handles the greedy style repetition with the minimum either be 4268 * 0 or 1 and the maximum be MAX_REPS, for * and + quantifier. 4269 */ 4270 static class CharPropertyGreedy extends Node 4271 final CharPredicate predicate; 4272 final int cmin; 4273 4274 CharPropertyGreedy(CharProperty cp, int cmin) 4275 this.predicate = cp.predicate; 4276 this.cmin = cmin; 4277 4278 boolean match(Matcher matcher, int i, CharSequence seq) 4279 int n = 0; 4280 int to = matcher.to; 4281 // greedy, all the way down 4282 while (i < to) 4283 int ch = Character.codePointAt(seq, i); 4284 if (!predicate.is(ch)) 4285 break; 4286 i += Character.charCount(ch); 4287 n++; 4288 4289 if (i >= to) 4290 matcher.hitEnd = true; 4291 4292 while (n >= cmin) 4293 if (next.match(matcher, i, seq)) 4294 return true; 4295 if (n == cmin) 4296 return false; 4297 // backing off if match fails 4298 int ch = Character.codePointBefore(seq, i); 4299 i -= Character.charCount(ch); 4300 n--; 4301 4302 return false; 4303 4304 4305 boolean study(TreeInfo info) 4306 info.minLength += cmin; 4307 if (info.maxValid) 4308 info.maxLength += MAX_REPS; 4309 4310 info.deterministic = false; 4311 return next.study(info); 4312 4313 4314 4315 static final class BmpCharPropertyGreedy extends CharPropertyGreedy 4316 4317 BmpCharPropertyGreedy(BmpCharProperty bcp, int cmin) 4318 super(bcp, cmin); 4319 4320 4321 boolean match(Matcher matcher, int i, CharSequence seq) 4322 int n = 0; 4323 int to = matcher.to; 4324 while (i < to && predicate.is(seq.charAt(i))) 4325 i++; n++; 4326 4327 if (i >= to) 4328 matcher.hitEnd = true; 4329 4330 while (n >= cmin) 4331 if (next.match(matcher, i, seq)) 4332 return true; 4333 i--; n--; // backing off if match fails 4334 4335 return false; 4336 4337 4338 4339 /** 4340 * Handles the curly-brace style repetition with a specified minimum and 4341 * maximum occurrences. The * quantifier is handled as a special case. 4342 * This class handles the three types. 4343 */ 4344 static final class Curly extends Node 4345 Node atom; 4346 Qtype type; 4347 int cmin; 4348 int cmax; 4349 4350 Curly(Node node, int cmin, int cmax, Qtype type) 4351 this.atom = node; 4352 this.type = type; 4353 this.cmin = cmin; 4354 this.cmax = cmax; 4355 4356 boolean match(Matcher matcher, int i, CharSequence seq) 4357 int j; 4358 for (j = 0; j < cmin; j++) 4359 if (atom.match(matcher, i, seq)) 4360 i = matcher.last; 4361 continue; 4362 4363 return false; 4364 4365 if (type == Qtype.GREEDY) 4366 return match0(matcher, i, j, seq); 4367 else if (type == Qtype.LAZY) 4368 return match1(matcher, i, j, seq); 4369 else 4370 return match2(matcher, i, j, seq); 4371 4372 // Greedy match. 4373 // i is the index to start matching at 4374 // j is the number of atoms that have matched 4375 boolean match0(Matcher matcher, int i, int j, CharSequence seq) 4376 if (j >= cmax) 4377 // We have matched the maximum... continue with the rest of 4378 // the regular expression 4379 return next.match(matcher, i, seq); 4380 4381 int backLimit = j; 4382 while (atom.match(matcher, i, seq)) 4383 // k is the length of this match 4384 int k = matcher.last - i; 4385 if (k == 0) // Zero length match 4386 break; 4387 // Move up index and number matched 4388 i = matcher.last; 4389 j++; 4390 // We are greedy so match as many as we can 4391 while (j < cmax) 4392 if (!atom.match(matcher, i, seq)) 4393 break; 4394 if (i + k != matcher.last) 4395 if (match0(matcher, matcher.last, j+1, seq)) 4396 return true; 4397 break; 4398 4399 i += k; 4400 j++; 4401 4402 // Handle backing off if match fails 4403 while (j >= backLimit) 4404 if (next.match(matcher, i, seq)) 4405 return true; 4406 i -= k; 4407 j--; 4408 4409 return false; 4410 4411 return next.match(matcher, i, seq); 4412 4413 // Reluctant match. At this point, the minimum has been satisfied. 4414 // i is the index to start matching at 4415 // j is the number of atoms that have matched 4416 boolean match1(Matcher matcher, int i, int j, CharSequence seq) 4417 for (;;) 4418 // Try finishing match without consuming any more 4419 if (next.match(matcher, i, seq)) 4420 return true; 4421 // At the maximum, no match found 4422 if (j >= cmax) 4423 return false; 4424 // Okay, must try one more atom 4425 if (!atom.match(matcher, i, seq)) 4426 return false; 4427 // If we haven‘t moved forward then must break out 4428 if (i == matcher.last) 4429 return false; 4430 // Move up index and number matched 4431 i = matcher.last; 4432 j++; 4433 4434 4435 boolean match2(Matcher matcher, int i, int j, CharSequence seq) 4436 for (; j < cmax; j++) 4437 if (!atom.match(matcher, i, seq)) 4438 break; 4439 if (i == matcher.last) 4440 break; 4441 i = matcher.last; 4442 4443 return next.match(matcher, i, seq); 4444 4445 boolean study(TreeInfo info) 4446 // Save original info 4447 int minL = info.minLength; 4448 int maxL = info.maxLength; 4449 boolean maxV = info.maxValid; 4450 boolean detm = info.deterministic; 4451 info.reset(); 4452 4453 atom.study(info); 4454 4455 int temp = info.minLength * cmin + minL; 4456 if (temp < minL) 4457 temp = 0xFFFFFFF; // arbitrary large number 4458 4459 info.minLength = temp; 4460 4461 if (maxV & info.maxValid) 4462 temp = info.maxLength * cmax + maxL; 4463 info.maxLength = temp; 4464 if (temp < maxL) 4465 info.maxValid = false; 4466 4467 else 4468 info.maxValid = false; 4469 4470 4471 if (info.deterministic && cmin == cmax) 4472 info.deterministic = detm; 4473 else 4474 info.deterministic = false; 4475 return next.study(info); 4476 4477 4478 4479 /** 4480 * Handles the curly-brace style repetition with a specified minimum and 4481 * maximum occurrences in deterministic cases. This is an iterative 4482 * optimization over the Prolog and Loop system which would handle this 4483 * in a recursive way. The * quantifier is handled as a special case. 4484 * If capture is true then this class saves group settings and ensures 4485 * that groups are unset when backing off of a group match. 4486 */ 4487 static final class GroupCurly extends Node 4488 Node atom; 4489 Qtype type; 4490 int cmin; 4491 int cmax; 4492 int localIndex; 4493 int groupIndex; 4494 boolean capture; 4495 4496 GroupCurly(Node node, int cmin, int cmax, Qtype type, int local, 4497 int group, boolean capture) 4498 this.atom = node; 4499 this.type = type; 4500 this.cmin = cmin; 4501 this.cmax = cmax; 4502 this.localIndex = local; 4503 this.groupIndex = group; 4504 this.capture = capture; 4505 4506 boolean match(Matcher matcher, int i, CharSequence seq) 4507 int[] groups = matcher.groups; 4508 int[] locals = matcher.locals; 4509 int save0 = locals[localIndex]; 4510 int save1 = 0; 4511 int save2 = 0; 4512 4513 if (capture) 4514 save1 = groups[groupIndex]; 4515 save2 = groups[groupIndex+1]; 4516 4517 4518 // Notify GroupTail there is no need to setup group info 4519 // because it will be set here 4520 locals[localIndex] = -1; 4521 4522 boolean ret = true; 4523 for (int j = 0; j < cmin; j++) 4524 if (atom.match(matcher, i, seq)) 4525 if (capture) 4526 groups[groupIndex] = i; 4527 groups[groupIndex+1] = matcher.last; 4528 4529 i = matcher.last; 4530 else 4531 ret = false; 4532 break; 4533 4534 4535 if (ret) 4536 if (type == Qtype.GREEDY) 4537 ret = match0(matcher, i, cmin, seq); 4538 else if (type == Qtype.LAZY) 4539 ret = match1(matcher, i, cmin, seq); 4540 else 4541 ret = match2(matcher, i, cmin, seq); 4542 4543 4544 if (!ret) 4545 locals[localIndex] = save0; 4546 if (capture) 4547 groups[groupIndex] = save1; 4548 groups[groupIndex+1] = save2; 4549 4550 4551 return ret; 4552 4553 // Aggressive group match 4554 boolean match0(Matcher matcher, int i, int j, CharSequence seq) 4555 // don‘t back off passing the starting "j" 4556 int min = j; 4557 int[] groups = matcher.groups; 4558 int save0 = 0; 4559 int save1 = 0; 4560 if (capture) 4561 save0 = groups[groupIndex]; 4562 save1 = groups[groupIndex+1]; 4563 4564 for (;;) 4565 if (j >= cmax) 4566 break; 4567 if (!atom.match(matcher, i, seq)) 4568 break; 4569 int k = matcher.last - i; 4570 if (k <= 0) 4571 if (capture) 4572 groups[groupIndex] = i; 4573 groups[groupIndex+1] = i + k; 4574 4575 i = i + k; 4576 break; 4577 4578 for (;;) 4579 if (capture) 4580 groups[groupIndex] = i; 4581 groups[groupIndex+1] = i + k; 4582 4583 i = i + k; 4584 if (++j >= cmax) 4585 break; 4586 if (!atom.match(matcher, i, seq)) 4587 break; 4588 if (i + k != matcher.last) 4589 if (match0(matcher, i, j, seq)) 4590 return true; 4591 break; 4592 4593 4594 while (j > min) 4595 if (next.match(matcher, i, seq)) 4596 if (capture) 4597 groups[groupIndex+1] = i; 4598 groups[groupIndex] = i - k; 4599 4600 return true; 4601 4602 // backing off 4603 i = i - k; 4604 if (capture) 4605 groups[groupIndex+1] = i; 4606 groups[groupIndex] = i - k; 4607 4608 j--; 4609 4610 4611 break; 4612 4613 if (capture) 4614 groups[groupIndex] = save0; 4615 groups[groupIndex+1] = save1; 4616 4617 return next.match(matcher, i, seq); 4618 4619 // Reluctant matching 4620 boolean match1(Matcher matcher, int i, int j, CharSequence seq) 4621 for (;;) 4622 if (next.match(matcher, i, seq)) 4623 return true; 4624 if (j >= cmax) 4625 return false; 4626 if (!atom.match(matcher, i, seq)) 4627 return false; 4628 if (i == matcher.last) 4629 return false; 4630 if (capture) 4631 matcher.groups[groupIndex] = i; 4632 matcher.groups[groupIndex+1] = matcher.last; 4633 4634 i = matcher.last; 4635 j++; 4636 4637 4638 // Possessive matching 4639 boolean match2(Matcher matcher, int i, int j, CharSequence seq) 4640 for (; j < cmax; j++) 4641 if (!atom.match(matcher, i, seq)) 4642 break; 4643 4644 if (capture) 4645 matcher.groups[groupIndex] = i; 4646 matcher.groups[groupIndex+1] = matcher.last; 4647 4648 if (i == matcher.last) 4649 break; 4650 4651 i = matcher.last; 4652 4653 return next.match(matcher, i, seq); 4654 4655 boolean study(TreeInfo info) 4656 // Save original info 4657 int minL = info.minLength; 4658 int maxL = info.maxLength; 4659 boolean maxV = info.maxValid; 4660 boolean detm = info.deterministic; 4661 info.reset(); 4662 4663 atom.study(info); 4664 4665 int temp = info.minLength * cmin + minL; 4666 if (temp < minL) 4667 temp = 0xFFFFFFF; // Arbitrary large number 4668 4669 info.minLength = temp; 4670 4671 if (maxV & info.maxValid) 4672 temp = info.maxLength * cmax + maxL; 4673 info.maxLength = temp; 4674 if (temp < maxL) 4675 info.maxValid = false; 4676 4677 else 4678 info.maxValid = false; 4679 4680 4681 if (info.deterministic && cmin == cmax) 4682 info.deterministic = detm; 4683 else 4684 info.deterministic = false; 4685 4686 return next.study(info); 4687 4688 4689 4690 /** 4691 * A Guard node at the end of each atom node in a Branch. It 4692 * serves the purpose of chaining the "match" operation to 4693 * "next" but not the "study", so we can collect the TreeInfo 4694 * of each atom node without including the TreeInfo of the 4695 * "next". 4696 */ 4697 static final class BranchConn extends Node 4698 BranchConn() ; 4699 boolean match(Matcher matcher, int i, CharSequence seq) 4700 return next.match(matcher, i, seq); 4701 4702 boolean study(TreeInfo info) 4703 return info.deterministic; 4704 4705 4706 4707 /** 4708 * Handles the branching of alternations. Note this is also used for 4709 * the ? quantifier to branch between the case where it matches once 4710 * and where it does not occur. 4711 */ 4712 static final class Branch extends Node 4713 Node[] atoms = new Node[2]; 4714 int size = 2; 4715 Node conn; 4716 Branch(Node first, Node second, Node branchConn) 4717 conn = branchConn; 4718 atoms[0] = first; 4719 atoms[1] = second; 4720 4721 4722 void add(Node node) 4723 if (size >= atoms.length) 4724 Node[] tmp = new Node[atoms.length*2]; 4725 System.arraycopy(atoms, 0, tmp, 0, atoms.length); 4726 atoms = tmp; 4727 4728 atoms[size++] = node; 4729 4730 4731 boolean match(Matcher matcher, int i, CharSequence seq) 4732 for (int n = 0; n < size; n++) 4733 if (atoms[n] == null) 4734 if (conn.next.match(matcher, i, seq)) 4735 return true; 4736 else if (atoms[n].match(matcher, i, seq)) 4737 return true; 4738 4739 4740 return false; 4741 4742 4743 boolean study(TreeInfo info) 4744 int minL = info.minLength; 4745 int maxL = info.maxLength; 4746 boolean maxV = info.maxValid; 4747 4748 int minL2 = Integer.MAX_VALUE; //arbitrary large enough num 4749 int maxL2 = -1; 4750 for (int n = 0; n < size; n++) 4751 info.reset(); 4752 if (atoms[n] != null) 4753 atoms[n].study(info); 4754 minL2 = Math.min(minL2, info.minLength); 4755 maxL2 = Math.max(maxL2, info.maxLength); 4756 maxV = (maxV & info.maxValid); 4757 4758 4759 minL += minL2; 4760 maxL += maxL2; 4761 4762 info.reset(); 4763 conn.next.study(info); 4764 4765 info.minLength += minL; 4766 info.maxLength += maxL; 4767 info.maxValid &= maxV; 4768 info.deterministic = false; 4769 return false; 4770 4771 4772 4773 /** 4774 * The GroupHead saves the location where the group begins in the locals 4775 * and restores them when the match is done. 4776 * 4777 * The matchRef is used when a reference to this group is accessed later 4778 * in the expression. The locals will have a negative value in them to 4779 * indicate that we do not want to unset the group if the reference 4780 * doesn‘t match. 4781 */ 4782 static final class GroupHead extends Node 4783 int localIndex; 4784 GroupTail tail; // for debug/print only, match does not need to know 4785 GroupHead(int localCount) 4786 localIndex = localCount; 4787 4788 boolean match(Matcher matcher, int i, CharSequence seq) 4789 int save = matcher.locals[localIndex]; 4790 matcher.locals[localIndex] = i; 4791 boolean ret = next.match(matcher, i, seq); 4792 matcher.locals[localIndex] = save; 4793 return ret; 4794 4795 boolean matchRef(Matcher matcher, int i, CharSequence seq) 4796 int save = matcher.locals[localIndex]; 4797 matcher.locals[localIndex] = ~i; // HACK 4798 boolean ret = next.match(matcher, i, seq); 4799 matcher.locals[localIndex] = save; 4800 return ret; 4801 4802 4803 4804 /** 4805 * Recursive reference to a group in the regular expression. It calls 4806 * matchRef because if the reference fails to match we would not unset 4807 * the group. 4808 */ 4809 static final class GroupRef extends Node 4810 GroupHead head; 4811 GroupRef(GroupHead head) 4812 this.head = head; 4813 4814 boolean match(Matcher matcher, int i, CharSequence seq) 4815 return head.matchRef(matcher, i, seq) 4816 && next.match(matcher, matcher.last, seq); 4817 4818 boolean study(TreeInfo info) 4819 info.maxValid = false; 4820 info.deterministic = false; 4821 return next.study(info); 4822 4823 4824 4825 /** 4826 * The GroupTail handles the setting of group beginning and ending 4827 * locations when groups are successfully matched. It must also be able to 4828 * unset groups that have to be backed off of. 4829 * 4830 * The GroupTail node is also used when a previous group is referenced, 4831 * and in that case no group information needs to be set. 4832 */ 4833 static final class GroupTail extends Node 4834 int localIndex; 4835 int groupIndex; 4836 GroupTail(int localCount, int groupCount) 4837 localIndex = localCount; 4838 groupIndex = groupCount + groupCount; 4839 4840 boolean match(Matcher matcher, int i, CharSequence seq) 4841 int tmp = matcher.locals[localIndex]; 4842 if (tmp >= 0) // This is the normal group case. 4843 // Save the group so we can unset it if it 4844 // backs off of a match. 4845 int groupStart = matcher.groups[groupIndex]; 4846 int groupEnd = matcher.groups[groupIndex+1]; 4847 4848 matcher.groups[groupIndex] = tmp; 4849 matcher.groups[groupIndex+1] = i; 4850 if (next.match(matcher, i, seq)) 4851 return true; 4852 4853 matcher.groups[groupIndex] = groupStart; 4854 matcher.groups[groupIndex+1] = groupEnd; 4855 return false; 4856 else 4857 // This is a group reference case. We don‘t need to save any 4858 // group info because it isn‘t really a group. 4859 matcher.last = i; 4860 return true; 4861 4862 4863 4864 4865 /** 4866 * This sets up a loop to handle a recursive quantifier structure. 4867 */ 4868 static final class Prolog extends Node 4869 Loop loop; 4870 Prolog(Loop loop) 4871 this.loop = loop; 4872 4873 boolean match(Matcher matcher, int i, CharSequence seq) 4874 return loop.matchInit(matcher, i, seq); 4875 4876 boolean study(TreeInfo info) 4877 return loop.study(info); 4878 4879 4880 4881 /** 4882 * Handles the repetition count for a greedy Curly. The matchInit 4883 * is called from the Prolog to save the index of where the group 4884 * beginning is stored. A zero length group check occurs in the 4885 * normal match but is skipped in the matchInit. 4886 */ 4887 static class Loop extends Node 4888 Node body; 4889 int countIndex; // local count index in matcher locals 4890 int beginIndex; // group beginning index 4891 int cmin, cmax; 4892 int posIndex; 4893 Loop(int countIndex, int beginIndex) 4894 this.countIndex = countIndex; 4895 this.beginIndex = beginIndex; 4896 this.posIndex = -1; 4897 4898 boolean match(Matcher matcher, int i, CharSequence seq) 4899 // Avoid infinite loop in zero-length case. 4900 if (i > matcher.locals[beginIndex]) 4901 int count = matcher.locals[countIndex]; 4902 4903 // This block is for before we reach the minimum 4904 // iterations required for the loop to match 4905 if (count < cmin) 4906 matcher.locals[countIndex] = count + 1; 4907 boolean b = body.match(matcher, i, seq); 4908 // If match failed we must backtrack, so 4909 // the loop count should NOT be incremented 4910 if (!b) 4911 matcher.locals[countIndex] = count; 4912 // Return success or failure since we are under 4913 // minimum 4914 return b; 4915 4916 // This block is for after we have the minimum 4917 // iterations required for the loop to match 4918 if (count < cmax) 4919 // Let‘s check if we have already tried and failed 4920 // at this starting position "i" in the past. 4921 // If yes, then just return false wihtout trying 4922 // again, to stop the exponential backtracking. 4923 if (posIndex != -1 && 4924 matcher.localsPos[posIndex].contains(i)) 4925 return next.match(matcher, i, seq); 4926 4927 matcher.locals[countIndex] = count + 1; 4928 boolean b = body.match(matcher, i, seq); 4929 // If match failed we must backtrack, so 4930 // the loop count should NOT be incremented 4931 if (b) 4932 return true; 4933 matcher.locals[countIndex] = count; 4934 // save the failed position 4935 if (posIndex != -1) 4936 matcher.localsPos[posIndex].add(i); 4937 4938 4939 4940 return next.match(matcher, i, seq); 4941 4942 boolean matchInit(Matcher matcher, int i, CharSequence seq) 4943 int save = matcher.locals[countIndex]; 4944 boolean ret = false; 4945 if (posIndex != -1 && matcher.localsPos[posIndex] == null) 4946 matcher.localsPos[posIndex] = new IntHashSet(); 4947 4948 if (0 < cmin) 4949 matcher.locals[countIndex] = 1; 4950 ret = body.match(matcher, i, seq); 4951 else if (0 < cmax) 4952 matcher.locals[countIndex] = 1; 4953 ret = body.match(matcher, i, seq); 4954 if (ret == false) 4955 ret = next.match(matcher, i, seq); 4956 else 4957 ret = next.match(matcher, i, seq); 4958 4959 matcher.locals[countIndex] = save; 4960 return ret; 4961 4962 boolean study(TreeInfo info) 4963 info.maxValid = false; 4964 info.deterministic = false; 4965 return false; 4966 4967 4968 4969 /** 4970 * Handles the repetition count for a reluctant Curly. The matchInit 4971 * is called from the Prolog to save the index of where the group 4972 * beginning is stored. A zero length group check occurs in the 4973 * normal match but is skipped in the matchInit. 4974 */ 4975 static final class LazyLoop extends Loop 4976 LazyLoop(int countIndex, int beginIndex) 4977 super(countIndex, beginIndex); 4978 4979 boolean match(Matcher matcher, int i, CharSequence seq) 4980 // Check for zero length group 4981 if (i > matcher.locals[beginIndex]) 4982 int count = matcher.locals[countIndex]; 4983 if (count < cmin) 4984 matcher.locals[countIndex] = count + 1; 4985 boolean result = body.match(matcher, i, seq); 4986 // If match failed we must backtrack, so 4987 // the loop count should NOT be incremented 4988 if (!result) 4989 matcher.locals[countIndex] = count; 4990 return result; 4991 4992 if (next.match(matcher, i, seq)) 4993 return true; 4994 if (count < cmax) 4995 matcher.locals[countIndex] = count + 1; 4996 boolean result = body.match(matcher, i, seq); 4997 // If match failed we must backtrack, so 4998 // the loop count should NOT be incremented 4999 if (!result) 5000 matcher.locals[countIndex] = count; 5001 return result; 5002 5003 return false; 5004 5005 return next.match(matcher, i, seq); 5006 5007 boolean matchInit(Matcher matcher, int i, CharSequence seq) 5008 int save = matcher.locals[countIndex]; 5009 boolean ret = false; 5010 if (0 < cmin) 5011 matcher.locals[countIndex] = 1; 5012 ret = body.match(matcher, i, seq); 5013 else if (next.match(matcher, i, seq)) 5014 ret = true; 5015 else if (0 < cmax) 5016 matcher.locals[countIndex] = 1; 5017 ret = body.match(matcher, i, seq); 5018 5019 matcher.locals[countIndex] = save; 5020 return ret; 5021 5022 boolean study(TreeInfo info) 5023 info.maxValid = false; 5024 info.deterministic = false; 5025 return false; 5026 5027 5028 5029 /** 5030 * Refers to a group in the regular expression. Attempts to match 5031 * whatever the group referred to last matched. 5032 */ 5033 static class BackRef extends Node 5034 int groupIndex; 5035 BackRef(int groupCount) 5036 super(); 5037 groupIndex = groupCount + groupCount; 5038 5039 boolean match(Matcher matcher, int i, CharSequence seq) 5040 int j = matcher.groups[groupIndex]; 5041 int k = matcher.groups[groupIndex+1]; 5042 5043 int groupSize = k - j; 5044 // If the referenced group didn‘t match, neither can this 5045 if (j < 0) 5046 return false; 5047 5048 // If there isn‘t enough input left no match 5049 if (i + groupSize > matcher.to) 5050 matcher.hitEnd = true; 5051 return false; 5052 5053 // Check each new char to make sure it matches what the group 5054 // referenced matched last time around 5055 for (int index=0; index<groupSize; index++) 5056 if (seq.charAt(i+index) != seq.charAt(j+index)) 5057 return false; 5058 5059 return next.match(matcher, i+groupSize, seq); 5060 5061 boolean study(TreeInfo info) 5062 info.maxValid = false; 5063 return next.study(info); 5064 5065 5066 5067 static class CIBackRef extends Node 5068 int groupIndex; 5069 boolean doUnicodeCase; 5070 CIBackRef(int groupCount, boolean doUnicodeCase) 5071 super(); 5072 groupIndex = groupCount + groupCount; 5073 this.doUnicodeCase = doUnicodeCase; 5074 5075 boolean match(Matcher matcher, int i, CharSequence seq) 5076 int j = matcher.groups[groupIndex]; 5077 int k = matcher.groups[groupIndex+1]; 5078 5079 int groupSize = k - j; 5080 5081 // If the referenced group didn‘t match, neither can this 5082 if (j < 0) 5083 return false; 5084 5085 // If there isn‘t enough input left no match 5086 if (i + groupSize > matcher.to) 5087 matcher.hitEnd = true; 5088 return false; 5089 5090 5091 // Check each new char to make sure it matches what the group 5092 // referenced matched last time around 5093 int x = i; 5094 for (int index=0; index<groupSize; index++) 5095 int c1 = Character.codePointAt(seq, x); 5096 int c2 = Character.codePointAt(seq, j); 5097 if (c1 != c2) 5098 if (doUnicodeCase) 5099 int cc1 = Character.toUpperCase(c1); 5100 int cc2 = Character.toUpperCase(c2); 5101 if (cc1 != cc2 && 5102 Character.toLowerCase(cc1) != 5103 Character.toLowerCase(cc2)) 5104 return false; 5105 else 5106 if (ASCII.toLower(c1) != ASCII.toLower(c2)) 5107 return false; 5108 5109 5110 x += Character.charCount(c1); 5111 j += Character.charCount(c2); 5112 5113 5114 return next.match(matcher, i+groupSize, seq); 5115 5116 boolean study(TreeInfo info) 5117 info.maxValid = false; 5118 return next.study(info); 5119 5120 5121 5122 /** 5123 * Searches until the next instance of its atom. This is useful for 5124 * finding the atom efficiently without passing an instance of it 5125 * (greedy problem) and without a lot of wasted search time (reluctant 5126 * problem). 5127 */ 5128 static final class First extends Node 5129 Node atom; 5130 First(Node node) 5131 this.atom = BnM.optimize(node); 5132 5133 boolean match(Matcher matcher, int i, CharSequence seq) 5134 if (atom instanceof BnM) 5135 return atom.match(matcher, i, seq) 5136 && next.match(matcher, matcher.last, seq); 5137 5138 for (;;) 5139 if (i > matcher.to) 5140 matcher.hitEnd = true; 5141 return false; 5142 5143 if (atom.match(matcher, i, seq)) 5144 return next.match(matcher, matcher.last, seq); 5145 5146 i += countChars(seq, i, 1); 5147 matcher.first++; 5148 5149 5150 boolean study(TreeInfo info) 5151 atom.study(info); 5152 info.maxValid = false; 5153 info.deterministic = false; 5154 return next.study(info); 5155 5156 5157 5158 static final class Conditional extends Node 5159 Node cond, yes, not; 5160 Conditional(Node cond, Node yes, Node not) 5161 this.cond = cond; 5162 this.yes = yes; 5163 this.not = not; 5164 5165 boolean match(Matcher matcher, int i, CharSequence seq) 5166 if (cond.match(matcher, i, seq)) 5167 return yes.match(matcher, i, seq); 5168 else 5169 return not.match(matcher, i, seq); 5170 5171 5172 boolean study(TreeInfo info) 5173 int minL = info.minLength; 5174 int maxL = info.maxLength; 5175 boolean maxV = info.maxValid; 5176 info.reset(); 5177 yes.study(info); 5178 5179 int minL2 = info.minLength; 5180 int maxL2 = info.maxLength; 5181 boolean maxV2 = info.maxValid; 5182 info.reset(); 5183 not.study(info); 5184 5185 info.minLength = minL + Math.min(minL2, info.minLength); 5186 info.maxLength = maxL + Math.max(maxL2, info.maxLength); 5187 info.maxValid = (maxV & maxV2 & info.maxValid); 5188 info.deterministic = false; 5189 return next.study(info); 5190 5191 5192 5193 /** 5194 * Zero width positive lookahead. 5195 */ 5196 static final class Pos extends Node 5197 Node cond; 5198 Pos(Node cond) 5199 this.cond = cond; 5200 5201 boolean match(Matcher matcher, int i, CharSequence seq) 5202 int savedTo = matcher.to; 5203 boolean conditionMatched = false; 5204 5205 // Relax transparent region boundaries for lookahead 5206 if (matcher.transparentBounds) 5207 matcher.to = matcher.getTextLength(); 5208 try 5209 conditionMatched = cond.match(matcher, i, seq); 5210 finally 5211 // Reinstate region boundaries 5212 matcher.to = savedTo; 5213 5214 return conditionMatched && next.match(matcher, i, seq); 5215 5216 5217 5218 /** 5219 * Zero width negative lookahead. 5220 */ 5221 static final class Neg extends Node 5222 Node cond; 5223 Neg(Node cond) 5224 this.cond = cond; 5225 5226 boolean match(Matcher matcher, int i, CharSequence seq) 5227 int savedTo = matcher.to; 5228 boolean conditionMatched = false; 5229 5230 // Relax transparent region boundaries for lookahead 5231 if (matcher.transparentBounds) 5232 matcher.to = matcher.getTextLength(); 5233 try 5234 if (i < matcher.to) 5235 conditionMatched = !cond.match(matcher, i, seq); 5236 else 5237 // If a negative lookahead succeeds then more input 5238 // could cause it to fail! 5239 matcher.requireEnd = true; 5240 conditionMatched = !cond.match(matcher, i, seq); 5241 5242 finally 5243 // Reinstate region boundaries 5244 matcher.to = savedTo; 5245 5246 return conditionMatched && next.match(matcher, i, seq); 5247 5248 5249 5250 /** 5251 * For use with lookbehinds; matches the position where the lookbehind 5252 * was encountered. 5253 */ 5254 static Node lookbehindEnd = new Node() 5255 boolean match(Matcher matcher, int i, CharSequence seq) 5256 return i == matcher.lookbehindTo; 5257 5258 ; 5259 5260 /** 5261 * Zero width positive lookbehind. 5262 */ 5263 static class Behind extends Node 5264 Node cond; 5265 int rmax, rmin; 5266 Behind(Node cond, int rmax, int rmin) 5267 this.cond = cond; 5268 this.rmax = rmax; 5269 this.rmin = rmin; 5270 5271 5272 boolean match(Matcher matcher, int i, CharSequence seq) 5273 int savedFrom = matcher.from; 5274 boolean conditionMatched = false; 5275 int startIndex = (!matcher.transparentBounds) ? 5276 matcher.from : 0; 5277 int from = Math.max(i - rmax, startIndex); 5278 // Set end boundary 5279 int savedLBT = matcher.lookbehindTo; 5280 matcher.lookbehindTo = i; 5281 // Relax transparent region boundaries for lookbehind 5282 if (matcher.transparentBounds) 5283 matcher.from = 0; 5284 for (int j = i - rmin; !conditionMatched && j >= from; j--) 5285 conditionMatched = cond.match(matcher, j, seq); 5286 5287 matcher.from = savedFrom; 5288 matcher.lookbehindTo = savedLBT; 5289 return conditionMatched && next.match(matcher, i, seq); 5290 5291 5292 5293 /** 5294 * Zero width positive lookbehind, including supplementary 5295 * characters or unpaired surrogates. 5296 */ 5297 static final class BehindS extends Behind 5298 BehindS(Node cond, int rmax, int rmin) 5299 super(cond, rmax, rmin); 5300 5301 boolean match(Matcher matcher, int i, CharSequence seq) 5302 int rmaxChars = countChars(seq, i, -rmax); 5303 int rminChars = countChars(seq, i, -rmin); 5304 int savedFrom = matcher.from; 5305 int startIndex = (!matcher.transparentBounds) ? 5306 matcher.from : 0; 5307 boolean conditionMatched = false; 5308 int from = Math.max(i - rmaxChars, startIndex); 5309 // Set end boundary 5310 int savedLBT = matcher.lookbehindTo; 5311 matcher.lookbehindTo = i; 5312 // Relax transparent region boundaries for lookbehind 5313 if (matcher.transparentBounds) 5314 matcher.from = 0; 5315 5316 for (int j = i - rminChars; 5317 !conditionMatched && j >= from; 5318 j -= j>from ? countChars(seq, j, -1) : 1) 5319 conditionMatched = cond.match(matcher, j, seq); 5320 5321 matcher.from = savedFrom; 5322 matcher.lookbehindTo = savedLBT; 5323 return conditionMatched && next.match(matcher, i, seq); 5324 5325 5326 5327 /** 5328 * Zero width negative lookbehind. 5329 */ 5330 static class NotBehind extends Node 5331 Node cond; 5332 int rmax, rmin; 5333 NotBehind(Node cond, int rmax, int rmin) 5334 this.cond = cond; 5335 this.rmax = rmax; 5336 this.rmin = rmin; 5337 5338 5339 boolean match(Matcher matcher, int i, CharSequence seq) 5340 int savedLBT = matcher.lookbehindTo; 5341 int savedFrom = matcher.from; 5342 boolean conditionMatched = false; 5343 int startIndex = (!matcher.transparentBounds) ? 5344 matcher.from : 0; 5345 int from = Math.max(i - rmax, startIndex); 5346 matcher.lookbehindTo = i; 5347 // Relax transparent region boundaries for lookbehind 5348 if (matcher.transparentBounds) 5349 matcher.from = 0; 5350 for (int j = i - rmin; !conditionMatched && j >= from; j--) 5351 conditionMatched = cond.match(matcher, j, seq); 5352 5353 // Reinstate region boundaries 5354 matcher.from = savedFrom; 5355 matcher.lookbehindTo = savedLBT; 5356 return !conditionMatched && next.match(matcher, i, seq); 5357 5358 5359 5360 /** 5361 * Zero width negative lookbehind, including supplementary 5362 * characters or unpaired surrogates. 5363 */ 5364 static final class NotBehindS extends NotBehind 5365 NotBehindS(Node cond, int rmax, int rmin) 5366 super(cond, rmax, rmin); 5367 5368 boolean match(Matcher matcher, int i, CharSequence seq) 5369 int rmaxChars = countChars(seq, i, -rmax); 5370 int rminChars = countChars(seq, i, -rmin); 5371 int savedFrom = matcher.from; 5372 int savedLBT = matcher.lookbehindTo; 5373 boolean conditionMatched = false; 5374 int startIndex = (!matcher.transparentBounds) ? 5375 matcher.from : 0; 5376 int from = Math.max(i - rmaxChars, startIndex); 5377 matcher.lookbehindTo = i; 5378 // Relax transparent region boundaries for lookbehind 5379 if (matcher.transparentBounds) 5380 matcher.from = 0; 5381 for (int j = i - rminChars; 5382 !conditionMatched && j >= from; 5383 j -= j>from ? countChars(seq, j, -1) : 1) 5384 conditionMatched = cond.match(matcher, j, seq); 5385 5386 //Reinstate region boundaries 5387 matcher.from = savedFrom; 5388 matcher.lookbehindTo = savedLBT; 5389 return !conditionMatched && next.match(matcher, i, seq); 5390 5391 5392 5393 /** 5394 * Handles word boundaries. Includes a field to allow this one class to 5395 * deal with the different types of word boundaries we can match. The word 5396 * characters include underscores, letters, and digits. Non spacing marks 5397 * can are also part of a word if they have a base character, otherwise 5398 * they are ignored for purposes of finding word boundaries. 5399 */ 5400 static final class Bound extends Node 5401 static int LEFT = 0x1; 5402 static int RIGHT= 0x2; 5403 static int BOTH = 0x3; 5404 static int NONE = 0x4; 5405 int type; 5406 boolean useUWORD; 5407 Bound(int n, boolean useUWORD) 5408 type = n; 5409 this.useUWORD = useUWORD; 5410 5411 5412 boolean isWord(int ch) 5413 return useUWORD ? CharPredicates.WORD().is(ch) 5414 : (ch == ‘_‘ || Character.isLetterOrDigit(ch)); 5415 5416 5417 int check(Matcher matcher, int i, CharSequence seq) 5418 int ch; 5419 boolean left = false; 5420 int startIndex = matcher.from; 5421 int endIndex = matcher.to; 5422 if (matcher.transparentBounds) 5423 startIndex = 0; 5424 endIndex = matcher.getTextLength(); 5425 5426 if (i > startIndex) 5427 ch = Character.codePointBefore(seq, i); 5428 left = (isWord(ch) || 5429 ((Character.getType(ch) == Character.NON_SPACING_MARK) 5430 && hasBaseCharacter(matcher, i-1, seq))); 5431 5432 boolean right = false; 5433 if (i < endIndex) 5434 ch = Character.codePointAt(seq, i); 5435 right = (isWord(ch) || 5436 ((Character.getType(ch) == Character.NON_SPACING_MARK) 5437 && hasBaseCharacter(matcher, i, seq))); 5438 else 5439 // Tried to access char past the end 5440 matcher.hitEnd = true; 5441 // The addition of another char could wreck a boundary 5442 matcher.requireEnd = true; 5443 5444 return ((left ^ right) ? (right ? LEFT : RIGHT) : NONE); 5445 5446 boolean match(Matcher matcher, int i, CharSequence seq) 5447 return (check(matcher, i, seq) & type) > 0 5448 && next.match(matcher, i, seq); 5449 5450 5451 5452 /** 5453 * Non spacing marks only count as word characters in bounds calculations 5454 * if they have a base character. 5455 */ 5456 private static boolean hasBaseCharacter(Matcher matcher, int i, 5457 CharSequence seq) 5458 5459 int start = (!matcher.transparentBounds) ? 5460 matcher.from : 0; 5461 for (int x=i; x >= start; x--) 5462 int ch = Character.codePointAt(seq, x); 5463 if (Character.isLetterOrDigit(ch)) 5464 return true; 5465 if (Character.getType(ch) == Character.NON_SPACING_MARK) 5466 continue; 5467 return false; 5468 5469 return false; 5470 5471 5472 /** 5473 * Attempts to match a slice in the input using the Boyer-Moore string 5474 * matching algorithm. The algorithm is based on the idea that the 5475 * pattern can be shifted farther ahead in the search text if it is 5476 * matched right to left. 5477 * <p> 5478 * The pattern is compared to the input one character at a time, from 5479 * the rightmost character in the pattern to the left. If the characters 5480 * all match the pattern has been found. If a character does not match, 5481 * the pattern is shifted right a distance that is the maximum of two 5482 * functions, the bad character shift and the good suffix shift. This 5483 * shift moves the attempted match position through the input more 5484 * quickly than a naive one position at a time check. 5485 * <p> 5486 * The bad character shift is based on the character from the text that 5487 * did not match. If the character does not appear in the pattern, the 5488 * pattern can be shifted completely beyond the bad character. If the 5489 * character does occur in the pattern, the pattern can be shifted to 5490 * line the pattern up with the next occurrence of that character. 5491 * <p> 5492 * The good suffix shift is based on the idea that some subset on the right 5493 * side of the pattern has matched. When a bad character is found, the 5494 * pattern can be shifted right by the pattern length if the subset does 5495 * not occur again in pattern, or by the amount of distance to the 5496 * next occurrence of the subset in the pattern. 5497 * 5498 * Boyer-Moore search methods adapted from code by Amy Yu. 5499 */ 5500 static class BnM extends Node 5501 int[] buffer; 5502 int[] lastOcc; 5503 int[] optoSft; 5504 5505 /** 5506 * Pre calculates arrays needed to generate the bad character 5507 * shift and the good suffix shift. Only the last seven bits 5508 * are used to see if chars match; This keeps the tables small 5509 * and covers the heavily used ASCII range, but occasionally 5510 * results in an aliased match for the bad character shift. 5511 */ 5512 static Node optimize(Node node) 5513 if (!(node instanceof Slice)) 5514 return node; 5515 5516 5517 int[] src = ((Slice) node).buffer; 5518 int patternLength = src.length; 5519 // The BM algorithm requires a bit of overhead; 5520 // If the pattern is short don‘t use it, since 5521 // a shift larger than the pattern length cannot 5522 // be used anyway. 5523 if (patternLength < 4) 5524 return node; 5525 5526 int i, j, k; 5527 int[] lastOcc = new int[128]; 5528 int[] optoSft = new int[patternLength]; 5529 // Precalculate part of the bad character shift 5530 // It is a table for where in the pattern each 5531 // lower 7-bit value occurs 5532 for (i = 0; i < patternLength; i++) 5533 lastOcc[src[i]&0x7F] = i + 1; 5534 5535 // Precalculate the good suffix shift 5536 // i is the shift amount being considered 5537 NEXT: for (i = patternLength; i > 0; i--) 5538 // j is the beginning index of suffix being considered 5539 for (j = patternLength - 1; j >= i; j--) 5540 // Testing for good suffix 5541 if (src[j] == src[j-i]) 5542 // src[j..len] is a good suffix 5543 optoSft[j-1] = i; 5544 else 5545 // No match. The array has already been 5546 // filled up with correct values before. 5547 continue NEXT; 5548 5549 5550 // This fills up the remaining of optoSft 5551 // any suffix can not have larger shift amount 5552 // then its sub-suffix. Why??? 5553 while (j > 0) 5554 optoSft[--j] = i; 5555 5556 5557 // Set the guard value because of unicode compression 5558 optoSft[patternLength-1] = 1; 5559 if (node instanceof SliceS) 5560 return new BnMS(src, lastOcc, optoSft, node.next); 5561 return new BnM(src, lastOcc, optoSft, node.next); 5562 5563 BnM(int[] src, int[] lastOcc, int[] optoSft, Node next) 5564 this.buffer = src; 5565 this.lastOcc = lastOcc; 5566 this.optoSft = optoSft; 5567 this.next = next; 5568 5569 boolean match(Matcher matcher, int i, CharSequence seq) 5570 int[] src = buffer; 5571 int patternLength = src.length; 5572 int last = matcher.to - patternLength; 5573 5574 // Loop over all possible match positions in text 5575 NEXT: while (i <= last) 5576 // Loop over pattern from right to left 5577 for (int j = patternLength - 1; j >= 0; j--) 5578 int ch = seq.charAt(i+j); 5579 if (ch != src[j]) 5580 // Shift search to the right by the maximum of the 5581 // bad character shift and the good suffix shift 5582 i += Math.max(j + 1 - lastOcc[ch&0x7F], optoSft[j]); 5583 continue NEXT; 5584 5585 5586 // Entire pattern matched starting at i 5587 matcher.first = i; 5588 boolean ret = next.match(matcher, i + patternLength, seq); 5589 if (ret) 5590 matcher.first = i; 5591 matcher.groups[0] = matcher.first; 5592 matcher.groups[1] = matcher.last; 5593 return true; 5594 5595 i++; 5596 5597 // BnM is only used as the leading node in the unanchored case, 5598 // and it replaced its Start() which always searches to the end 5599 // if it doesn‘t find what it‘s looking for, so hitEnd is true. 5600 matcher.hitEnd = true; 5601 return false; 5602 5603 boolean study(TreeInfo info) 5604 info.minLength += buffer.length; 5605 info.maxValid = false; 5606 return next.study(info); 5607 5608 5609 5610 /** 5611 * Supplementary support version of BnM(). Unpaired surrogates are 5612 * also handled by this class. 5613 */ 5614 static final class BnMS extends BnM 5615 int lengthInChars; 5616 5617 BnMS(int[] src, int[] lastOcc, int[] optoSft, Node next) 5618 super(src, lastOcc, optoSft, next); 5619 for (int cp : buffer) 5620 lengthInChars += Character.charCount(cp); 5621 5622 5623 boolean match(Matcher matcher, int i, CharSequence seq) 5624 int[] src = buffer; 5625 int patternLength = src.length; 5626 int last = matcher.to - lengthInChars; 5627 5628 // Loop over all possible match positions in text 5629 NEXT: while (i <= last) 5630 // Loop over pattern from right to left 5631 int ch; 5632 for (int j = countChars(seq, i, patternLength), x = patternLength - 1; 5633 j > 0; j -= Character.charCount(ch), x--) 5634 ch = Character.codePointBefore(seq, i+j); 5635 if (ch != src[x]) 5636 // Shift search to the right by the maximum of the 5637 // bad character shift and the good suffix shift 5638 int n = Math.max(x + 1 - lastOcc[ch&0x7F], optoSft[x]); 5639 i += countChars(seq, i, n); 5640 continue NEXT; 5641 5642 5643 // Entire pattern matched starting at i 5644 matcher.first = i; 5645 boolean ret = next.match(matcher, i + lengthInChars, seq); 5646 if (ret) 5647 matcher.first = i; 5648 matcher.groups[0] = matcher.first; 5649 matcher.groups[1] = matcher.last; 5650 return true; 5651 5652 i += countChars(seq, i, 1); 5653 5654 matcher.hitEnd = true; 5655 return false; 5656 5657 5658 5659 @FunctionalInterface 5660 static interface CharPredicate 5661 boolean is(int ch); 5662 5663 default CharPredicate and(CharPredicate p) 5664 return ch -> is(ch) && p.is(ch); 5665 5666 default CharPredicate union(CharPredicate p) 5667 return ch -> is(ch) || p.is(ch); 5668 5669 default CharPredicate union(CharPredicate p1, 5670 CharPredicate p2 ) 5671 return ch -> is(ch) || p1.is(ch) || p2.is(ch); 5672 5673 default CharPredicate negate() 5674 return ch -> !is(ch); 5675 5676 5677 5678 static interface BmpCharPredicate extends CharPredicate 5679 5680 default CharPredicate and(CharPredicate p) 5681 if(p instanceof BmpCharPredicate) 5682 return (BmpCharPredicate)(ch -> is(ch) && p.is(ch)); 5683 return ch -> is(ch) && p.is(ch); 5684 5685 default CharPredicate union(CharPredicate p) 5686 if (p instanceof BmpCharPredicate) 5687 return (BmpCharPredicate)(ch -> is(ch) || p.is(ch)); 5688 return ch -> is(ch) || p.is(ch); 5689 5690 static CharPredicate union(CharPredicate... predicates) 5691 CharPredicate cp = ch -> 5692 for (CharPredicate p : predicates) 5693 if (!p.is(ch)) 5694 return false; 5695 5696 return true; 5697 ; 5698 for (CharPredicate p : predicates) 5699 if (! (p instanceof BmpCharPredicate)) 5700 return cp; 5701 5702 return (BmpCharPredicate)cp; 5703 5704 5705 5706 /** 5707 * matches a Perl vertical whitespace 5708 */ 5709 static BmpCharPredicate VertWS() 5710 return cp -> (cp >= 0x0A && cp <= 0x0D) || 5711 cp == 0x85 || cp == 0x2028 || cp == 0x2029; 5712 5713 5714 /** 5715 * matches a Perl horizontal whitespace 5716 */ 5717 static BmpCharPredicate HorizWS() 5718 return cp -> 5719 cp == 0x09 || cp == 0x20 || cp == 0xa0 || cp == 0x1680 || 5720 cp == 0x180e || cp >= 0x2000 && cp <= 0x200a || cp == 0x202f || 5721 cp == 0x205f || cp == 0x3000; 5722 5723 5724 /** 5725 * for the Unicode category ALL and the dot metacharacter when 5726 * in dotall mode. 5727 */ 5728 static CharPredicate ALL() 5729 return ch -> true; 5730 5731 5732 /** 5733 * for the dot metacharacter when dotall is not enabled. 5734 */ 5735 static CharPredicate DOT() 5736 return ch -> 5737 (ch != ‘\n‘ && ch != ‘\r‘ 5738 && (ch|1) != ‘\u2029‘ 5739 && ch != ‘\u0085‘); 5740 5741 5742 /** 5743 * the dot metacharacter when dotall is not enabled but UNIX_LINES is enabled. 5744 */ 5745 static CharPredicate UNIXDOT() 5746 return ch -> ch != ‘\n‘; 5747 5748 5749 /** 5750 * Indicate that matches a Supplementary Unicode character 5751 */ 5752 static CharPredicate SingleS(int c) 5753 return ch -> ch == c; 5754 5755 5756 /** 5757 * A bmp/optimized predicate of single 5758 */ 5759 static BmpCharPredicate Single(int c) 5760 return ch -> ch == c; 5761 5762 5763 /** 5764 * Case insensitive matches a given BMP character 5765 */ 5766 static BmpCharPredicate SingleI(int lower, int upper) 5767 return ch -> ch == lower || ch == upper; 5768 5769 5770 /** 5771 * Unicode case insensitive matches a given Unicode character 5772 */ 5773 static CharPredicate SingleU(int lower) 5774 return ch -> lower == ch || 5775 lower == Character.toLowerCase(Character.toUpperCase(ch)); 5776 5777 5778 private static boolean inRange(int lower, int ch, int upper) 5779 return lower <= ch && ch <= upper; 5780 5781 5782 /** 5783 * Charactrs within a explicit value range 5784 */ 5785 static CharPredicate Range(int lower, int upper) 5786 if (upper < Character.MIN_HIGH_SURROGATE || 5787 lower > Character.MAX_HIGH_SURROGATE && 5788 upper < Character.MIN_SUPPLEMENTARY_CODE_POINT) 5789 return (BmpCharPredicate)(ch -> inRange(lower, ch, upper)); 5790 return ch -> inRange(lower, ch, upper); 5791 5792 5793 /** 5794 * Charactrs within a explicit value range in a case insensitive manner. 5795 */ 5796 static CharPredicate CIRange(int lower, int upper) 5797 return ch -> inRange(lower, ch, upper) || 5798 ASCII.isAscii(ch) && 5799 (inRange(lower, ASCII.toUpper(ch), upper) || 5800 inRange(lower, ASCII.toLower(ch), upper)); 5801 5802 5803 static CharPredicate CIRangeU(int lower, int upper) 5804 return ch -> 5805 if (inRange(lower, ch, upper)) 5806 return true; 5807 int up = Character.toUpperCase(ch); 5808 return inRange(lower, up, upper) || 5809 inRange(lower, Character.toLowerCase(up), upper); 5810 ; 5811 5812 5813 /** 5814 * This must be the very first initializer. 5815 */ 5816 static final Node accept = new Node(); 5817 5818 static final Node lastAccept = new LastNode(); 5819 5820 /** 5821 * Creates a predicate that tests if this pattern is found in a given input 5822 * string. 5823 * 5824 * @apiNote 5825 * This method creates a predicate that behaves as if it creates a matcher 5826 * from the input sequence and then calls @code find, for example a 5827 * predicate of the form: 5828 * <pre>@code 5829 * s -> matcher(s).find(); 5830 * </pre> 5831 * 5832 * @return The predicate which can be used for finding a match on a 5833 * subsequence of a string 5834 * @since 1.8 5835 * @see Matcher#find 5836 */ 5837 public Predicate<String> asPredicate() 5838 return s -> matcher(s).find(); 5839 5840 5841 /** 5842 * Creates a predicate that tests if this pattern matches a given input string. 5843 * 5844 * @apiNote 5845 * This method creates a predicate that behaves as if it creates a matcher 5846 * from the input sequence and then calls @code matches, for example a 5847 * predicate of the form: 5848 * <pre>@code 5849 * s -> matcher(s).matches(); 5850 * </pre> 5851 * 5852 * @return The predicate which can be used for matching an input string 5853 * against this pattern. 5854 * @since 11 5855 * @see Matcher#matches 5856 */ 5857 public Predicate<String> asMatchPredicate() 5858 return s -> matcher(s).matches(); 5859 5860 5861 /** 5862 * Creates a stream from the given input sequence around matches of this 5863 * pattern. 5864 * 5865 * <p> The stream returned by this method contains each substring of the 5866 * input sequence that is terminated by another subsequence that matches 5867 * this pattern or is terminated by the end of the input sequence. The 5868 * substrings in the stream are in the order in which they occur in the 5869 * input. Trailing empty strings will be discarded and not encountered in 5870 * the stream. 5871 * 5872 * <p> If this pattern does not match any subsequence of the input then 5873 * the resulting stream has just one element, namely the input sequence in 5874 * string form. 5875 * 5876 * <p> When there is a positive-width match at the beginning of the input 5877 * sequence then an empty leading substring is included at the beginning 5878 * of the stream. A zero-width match at the beginning however never produces 5879 * such empty leading substring. 5880 * 5881 * <p> If the input sequence is mutable, it must remain constant during the 5882 * execution of the terminal stream operation. Otherwise, the result of the 5883 * terminal stream operation is undefined. 5884 * 5885 * @param input 5886 * The character sequence to be split 5887 * 5888 * @return The stream of strings computed by splitting the input 5889 * around matches of this pattern 5890 * @see #split(CharSequence) 5891 * @since 1.8 5892 */ 5893 public Stream<String> splitAsStream(final CharSequence input) 5894 class MatcherIterator implements Iterator<String> 5895 private Matcher matcher; 5896 // The start position of the next sub-sequence of input 5897 // when current == input.length there are no more elements 5898 private int current; 5899 // null if the next element, if any, needs to obtained 5900 private String nextElement; 5901 // > 0 if there are N next empty elements 5902 private int emptyElementCount; 5903 5904 public String next() 5905 if (!hasNext()) 5906 throw new NoSuchElementException(); 5907 5908 if (emptyElementCount == 0) 5909 String n = nextElement; 5910 nextElement = null; 5911 return n; 5912 else 5913 emptyElementCount--; 5914 return ""; 5915 5916 5917 5918 public boolean hasNext() 5919 if (matcher == null) 5920 matcher = matcher(input); 5921 // If the input is an empty string then the result can only be a 5922 // stream of the input. Induce that by setting the empty 5923 // element count to 1 5924 emptyElementCount = input.length() == 0 ? 1 : 0; 5925 5926 if (nextElement != null || emptyElementCount > 0) 5927 return true; 5928 5929 if (current == input.length()) 5930 return false; 5931 5932 // Consume the next matching element 5933 // Count sequence of matching empty elements 5934 while (matcher.find()) 5935 nextElement = input.subSequence(current, matcher.start()).toString(); 5936 current = matcher.end(); 5937 if (!nextElement.isEmpty()) 5938 return true; 5939 else if (current > 0) // no empty leading substring for zero-width 5940 // match at the beginning of the input 5941 emptyElementCount++; 5942 5943 5944 5945 // Consume last matching element 5946 nextElement = input.subSequence(current, input.length()).toString(); 5947 current = input.length(); 5948 if (!nextElement.isEmpty()) 5949 return true; 5950 else 5951 // Ignore a terminal sequence of matching empty elements 5952 emptyElementCount = 0; 5953 nextElement = null; 5954 return false; 5955 5956 5957 5958 return StreamSupport.stream(Spliterators.spliteratorUnknownSize( 5959 new MatcherIterator(), Spliterator.ORDERED | Spliterator.NONNULL), false); 5960 5961
以上是关于Pattern类（java JDK源码记录）的主要内容，如果未能解决你的问题，请参考以下文章