| Message ID | 1530204275-31537-1-git-send-email-adhemerval.zanella@linaro.org |
|---|---|
| State | Accepted |
| Commit | eb04c21373e2a2885f3d52ff192b0499afe3c672 |
| Headers | show |
| Series | posix: Sync gnulib regex implementation | expand |
Adhemerval Zanella wrote: > This patch syncs regex code with gnulib (commit 16aa5a2). Thanks for doing all that work. I merged some of the changes embodied in that patch back to Gnulib; see <https://lists.gnu.org/r/bug-gnulib/2018-06/msg00095.html>. Here are some comments on the changes that I had questions about: > * regcomp: add a __libc_lock_init init for dfa lock. Why is this needed? The lock has already been initialized by the call to lock_init about 15 lines earlier. Please see attached patch, which reverts this change. > * regexec: instead of use intprops.h and its macros, this patch > the check_add_overflow_idx function to check for addition > overflow. The function need to be defined on the file because > the file will be build multiple times wit 'Idx' type being > redefined (I think intprops.h addition should be done in a > separate patch). When used with Gnulib, that implementation won't work with ICC 17.0.4 20170411, which defines __GNUC__ to be 5 but does not support __builtin_add_overflow. Also, the __GNUC__ < 5 code doesn't look right for multiple reasons: (1) it doesn't set *r when returning false, contrary to the comment describing the code (presumably the comment needs to be fixed?), (2) it incorrectly uses SSIZE_MAX instead of INT_MAX when _REGEX_LARGE_OFFSETS is not defined, and (3) it has an unnecessary test for a > 0 (you need to test only whether a < 0). Although these issues could be fixed individually, why waste time? Let's just use intprops.h and avoid the hassle of hacking on the bugs of a partial replacement. Attached please see an intprops.h addition that is done as a separate patch. The second attached patch goes back to using INT_ADD_WRAPV instead of check_add_overflow. > * regex.h: Remove _Restrict_ and _Restrict_arr_ definition based > on __STDC_VERSION__ because its usage leads to failures on > posix/check-installed-headers-c{xx}. That implementation assumes glibc, so won't work in Gnulib. I attempted to work around the check-installed-headers issue in the attached patch; if this doesn't work please let me know what the issue is. > * regex_internal.h: Define lock_fini to empty macro because setting > to 0 lead to build issues (error: statement with no effect > [-Werror=unused-value]). Making it empty is problematic, since lock_fini is supposed to be usable wherever a function call could appear. Instead, let's define it to ((void) 0). That should fix the -Wunused-value issue in a better way. See attached patch. > 2. posix/PCRE.tests: the test '(a)|\1' uses a backreference along > with group creation and I am not sure if it is the correct > behavior to accept it with regcomp (REG_EXTENDED). The GNU grep > accepts it with ERE option though. POSIX allows recomp and grep to accept this pattern as an extension. I dunno what it means, though, and no doubt regcomp and grep should report an error instead of accepting it silently. That would be a different bugfix, though. One other thing: let's use https: intead of http: in URLs, as per Gnulib style. We should be using these everywhere in Glibc, of course, but one step at a time. See attached patches for what the above comments boil down to. They are intended to be applied after the patch you posted. The patches could all be squashed. From 961eb26b900a859d2a69e6c427dac91c8b1d1eb8 Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Thu, 28 Jun 2018 12:39:30 -0700 Subject: [PATCH 1/2] posix: add intprops.h to prepare for regex sync * include/intprops.h: New file, taken from Gnulib except with reworded header comments. This will be used in a followup regex patch. --- ChangeLog | 7 + include/intprops.h | 457 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 464 insertions(+) create mode 100644 include/intprops.h diff --git a/ChangeLog b/ChangeLog index 1f35f29d77..27793d1456 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,10 @@ +2018-06-28 Paul Eggert <eggert@cs.ucla.edu> + + posix: add intprops.h to prepare for regex sync + * include/intprops.h: New file, taken from Gnulib except + with reworded header comments. This will be used in a + followup regex patch. + 2018-06-28 Szabolcs Nagy <szabolcs.nagy@arm.com> * manual/llio.texi: Remove spurious space. diff --git a/include/intprops.h b/include/intprops.h new file mode 100644 index 0000000000..a2241a2fc4 --- /dev/null +++ b/include/intprops.h @@ -0,0 +1,457 @@ +/* Properties of integer types. + Copyright (C) 2001-2018 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +/* Written by Paul Eggert. */ + +/* This filed is shared between glibc internals and Gnulib, and uses + Gnulib's _GL prefix for private macros. */ + +#ifndef _GL_INTPROPS_H +#define _GL_INTPROPS_H + +#include <limits.h> + +/* Return a value with the common real type of E and V and the value of V. */ +#define _GL_INT_CONVERT(e, v) (0 * (e) + (v)) + +/* Act like _GL_INT_CONVERT (E, -V) but work around a bug in IRIX 6.5 cc; see + <https://lists.gnu.org/r/bug-gnulib/2011-05/msg00406.html>. */ +#define _GL_INT_NEGATE_CONVERT(e, v) (0 * (e) - (v)) + +/* The extra casts in the following macros work around compiler bugs, + e.g., in Cray C 5.0.3.0. */ + +/* True if the arithmetic type T is an integer type. bool counts as + an integer. */ +#define TYPE_IS_INTEGER(t) ((t) 1.5 == 1) + +/* True if the real type T is signed. */ +#define TYPE_SIGNED(t) (! ((t) 0 < (t) -1)) + +/* Return 1 if the real expression E, after promotion, has a + signed or floating type. */ +#define EXPR_SIGNED(e) (_GL_INT_NEGATE_CONVERT (e, 1) < 0) + + +/* Minimum and maximum values for integer types and expressions. */ + +/* The width in bits of the integer type or expression T. + Padding bits are not supported; this is checked at compile-time below. */ +#define TYPE_WIDTH(t) (sizeof (t) * CHAR_BIT) + +/* The maximum and minimum values for the integer type T. */ +#define TYPE_MINIMUM(t) ((t) ~ TYPE_MAXIMUM (t)) +#define TYPE_MAXIMUM(t) \ + ((t) (! TYPE_SIGNED (t) \ + ? (t) -1 \ + : ((((t) 1 << (TYPE_WIDTH (t) - 2)) - 1) * 2 + 1))) + +/* The maximum and minimum values for the type of the expression E, + after integer promotion. E should not have side effects. */ +#define _GL_INT_MINIMUM(e) \ + (EXPR_SIGNED (e) \ + ? ~ _GL_SIGNED_INT_MAXIMUM (e) \ + : _GL_INT_CONVERT (e, 0)) +#define _GL_INT_MAXIMUM(e) \ + (EXPR_SIGNED (e) \ + ? _GL_SIGNED_INT_MAXIMUM (e) \ + : _GL_INT_NEGATE_CONVERT (e, 1)) +#define _GL_SIGNED_INT_MAXIMUM(e) \ + (((_GL_INT_CONVERT (e, 1) << (TYPE_WIDTH ((e) + 0) - 2)) - 1) * 2 + 1) + +/* Work around OpenVMS incompatibility with C99. */ +#if !defined LLONG_MAX && defined __INT64_MAX +# define LLONG_MAX __INT64_MAX +# define LLONG_MIN __INT64_MIN +#endif + +/* This include file assumes that signed types are two's complement without + padding bits; the above macros have undefined behavior otherwise. + If this is a problem for you, please let us know how to fix it for your host. + This assumption is tested by the intprops-tests module. */ + +/* Does the __typeof__ keyword work? This could be done by + 'configure', but for now it's easier to do it by hand. */ +#if (2 <= __GNUC__ \ + || (1210 <= __IBMC__ && defined __IBM__TYPEOF__) \ + || (0x5110 <= __SUNPRO_C && !__STDC__)) +# define _GL_HAVE___TYPEOF__ 1 +#else +# define _GL_HAVE___TYPEOF__ 0 +#endif + +/* Return 1 if the integer type or expression T might be signed. Return 0 + if it is definitely unsigned. This macro does not evaluate its argument, + and expands to an integer constant expression. */ +#if _GL_HAVE___TYPEOF__ +# define _GL_SIGNED_TYPE_OR_EXPR(t) TYPE_SIGNED (__typeof__ (t)) +#else +# define _GL_SIGNED_TYPE_OR_EXPR(t) 1 +#endif + +/* Bound on length of the string representing an unsigned integer + value representable in B bits. log10 (2.0) < 146/485. The + smallest value of B where this bound is not tight is 2621. */ +#define INT_BITS_STRLEN_BOUND(b) (((b) * 146 + 484) / 485) + +/* Bound on length of the string representing an integer type or expression T. + Subtract 1 for the sign bit if T is signed, and then add 1 more for + a minus sign if needed. + + Because _GL_SIGNED_TYPE_OR_EXPR sometimes returns 0 when its argument is + signed, this macro may overestimate the true bound by one byte when + applied to unsigned types of size 2, 4, 16, ... bytes. */ +#define INT_STRLEN_BOUND(t) \ + (INT_BITS_STRLEN_BOUND (TYPE_WIDTH (t) - _GL_SIGNED_TYPE_OR_EXPR (t)) \ + + _GL_SIGNED_TYPE_OR_EXPR (t)) + +/* Bound on buffer size needed to represent an integer type or expression T, + including the terminating null. */ +#define INT_BUFSIZE_BOUND(t) (INT_STRLEN_BOUND (t) + 1) + + +/* Range overflow checks. + + The INT_<op>_RANGE_OVERFLOW macros return 1 if the corresponding C + operators might not yield numerically correct answers due to + arithmetic overflow. They do not rely on undefined or + implementation-defined behavior. Their implementations are simple + and straightforward, but they are a bit harder to use than the + INT_<op>_OVERFLOW macros described below. + + Example usage: + + long int i = ...; + long int j = ...; + if (INT_MULTIPLY_RANGE_OVERFLOW (i, j, LONG_MIN, LONG_MAX)) + printf ("multiply would overflow"); + else + printf ("product is %ld", i * j); + + Restrictions on *_RANGE_OVERFLOW macros: + + These macros do not check for all possible numerical problems or + undefined or unspecified behavior: they do not check for division + by zero, for bad shift counts, or for shifting negative numbers. + + These macros may evaluate their arguments zero or multiple times, + so the arguments should not have side effects. The arithmetic + arguments (including the MIN and MAX arguments) must be of the same + integer type after the usual arithmetic conversions, and the type + must have minimum value MIN and maximum MAX. Unsigned types should + use a zero MIN of the proper type. + + These macros are tuned for constant MIN and MAX. For commutative + operations such as A + B, they are also tuned for constant B. */ + +/* Return 1 if A + B would overflow in [MIN,MAX] arithmetic. + See above for restrictions. */ +#define INT_ADD_RANGE_OVERFLOW(a, b, min, max) \ + ((b) < 0 \ + ? (a) < (min) - (b) \ + : (max) - (b) < (a)) + +/* Return 1 if A - B would overflow in [MIN,MAX] arithmetic. + See above for restrictions. */ +#define INT_SUBTRACT_RANGE_OVERFLOW(a, b, min, max) \ + ((b) < 0 \ + ? (max) + (b) < (a) \ + : (a) < (min) + (b)) + +/* Return 1 if - A would overflow in [MIN,MAX] arithmetic. + See above for restrictions. */ +#define INT_NEGATE_RANGE_OVERFLOW(a, min, max) \ + ((min) < 0 \ + ? (a) < - (max) \ + : 0 < (a)) + +/* Return 1 if A * B would overflow in [MIN,MAX] arithmetic. + See above for restrictions. Avoid && and || as they tickle + bugs in Sun C 5.11 2010/08/13 and other compilers; see + <https://lists.gnu.org/r/bug-gnulib/2011-05/msg00401.html>. */ +#define INT_MULTIPLY_RANGE_OVERFLOW(a, b, min, max) \ + ((b) < 0 \ + ? ((a) < 0 \ + ? (a) < (max) / (b) \ + : (b) == -1 \ + ? 0 \ + : (min) / (b) < (a)) \ + : (b) == 0 \ + ? 0 \ + : ((a) < 0 \ + ? (a) < (min) / (b) \ + : (max) / (b) < (a))) + +/* Return 1 if A / B would overflow in [MIN,MAX] arithmetic. + See above for restrictions. Do not check for division by zero. */ +#define INT_DIVIDE_RANGE_OVERFLOW(a, b, min, max) \ + ((min) < 0 && (b) == -1 && (a) < - (max)) + +/* Return 1 if A % B would overflow in [MIN,MAX] arithmetic. + See above for restrictions. Do not check for division by zero. + Mathematically, % should never overflow, but on x86-like hosts + INT_MIN % -1 traps, and the C standard permits this, so treat this + as an overflow too. */ +#define INT_REMAINDER_RANGE_OVERFLOW(a, b, min, max) \ + INT_DIVIDE_RANGE_OVERFLOW (a, b, min, max) + +/* Return 1 if A << B would overflow in [MIN,MAX] arithmetic. + See above for restrictions. Here, MIN and MAX are for A only, and B need + not be of the same type as the other arguments. The C standard says that + behavior is undefined for shifts unless 0 <= B < wordwidth, and that when + A is negative then A << B has undefined behavior and A >> B has + implementation-defined behavior, but do not check these other + restrictions. */ +#define INT_LEFT_SHIFT_RANGE_OVERFLOW(a, b, min, max) \ + ((a) < 0 \ + ? (a) < (min) >> (b) \ + : (max) >> (b) < (a)) + +/* True if __builtin_add_overflow (A, B, P) works when P is non-null. */ +#if 5 <= __GNUC__ && !defined __ICC +# define _GL_HAS_BUILTIN_OVERFLOW 1 +#else +# define _GL_HAS_BUILTIN_OVERFLOW 0 +#endif + +/* True if __builtin_add_overflow_p (A, B, C) works. */ +#define _GL_HAS_BUILTIN_OVERFLOW_P (7 <= __GNUC__) + +/* The _GL*_OVERFLOW macros have the same restrictions as the + *_RANGE_OVERFLOW macros, except that they do not assume that operands + (e.g., A and B) have the same type as MIN and MAX. Instead, they assume + that the result (e.g., A + B) has that type. */ +#if _GL_HAS_BUILTIN_OVERFLOW_P +# define _GL_ADD_OVERFLOW(a, b, min, max) \ + __builtin_add_overflow_p (a, b, (__typeof__ ((a) + (b))) 0) +# define _GL_SUBTRACT_OVERFLOW(a, b, min, max) \ + __builtin_sub_overflow_p (a, b, (__typeof__ ((a) - (b))) 0) +# define _GL_MULTIPLY_OVERFLOW(a, b, min, max) \ + __builtin_mul_overflow_p (a, b, (__typeof__ ((a) * (b))) 0) +#else +# define _GL_ADD_OVERFLOW(a, b, min, max) \ + ((min) < 0 ? INT_ADD_RANGE_OVERFLOW (a, b, min, max) \ + : (a) < 0 ? (b) <= (a) + (b) \ + : (b) < 0 ? (a) <= (a) + (b) \ + : (a) + (b) < (b)) +# define _GL_SUBTRACT_OVERFLOW(a, b, min, max) \ + ((min) < 0 ? INT_SUBTRACT_RANGE_OVERFLOW (a, b, min, max) \ + : (a) < 0 ? 1 \ + : (b) < 0 ? (a) - (b) <= (a) \ + : (a) < (b)) +# define _GL_MULTIPLY_OVERFLOW(a, b, min, max) \ + (((min) == 0 && (((a) < 0 && 0 < (b)) || ((b) < 0 && 0 < (a)))) \ + || INT_MULTIPLY_RANGE_OVERFLOW (a, b, min, max)) +#endif +#define _GL_DIVIDE_OVERFLOW(a, b, min, max) \ + ((min) < 0 ? (b) == _GL_INT_NEGATE_CONVERT (min, 1) && (a) < - (max) \ + : (a) < 0 ? (b) <= (a) + (b) - 1 \ + : (b) < 0 && (a) + (b) <= (a)) +#define _GL_REMAINDER_OVERFLOW(a, b, min, max) \ + ((min) < 0 ? (b) == _GL_INT_NEGATE_CONVERT (min, 1) && (a) < - (max) \ + : (a) < 0 ? (a) % (b) != ((max) - (b) + 1) % (b) \ + : (b) < 0 && ! _GL_UNSIGNED_NEG_MULTIPLE (a, b, max)) + +/* Return a nonzero value if A is a mathematical multiple of B, where + A is unsigned, B is negative, and MAX is the maximum value of A's + type. A's type must be the same as (A % B)'s type. Normally (A % + -B == 0) suffices, but things get tricky if -B would overflow. */ +#define _GL_UNSIGNED_NEG_MULTIPLE(a, b, max) \ + (((b) < -_GL_SIGNED_INT_MAXIMUM (b) \ + ? (_GL_SIGNED_INT_MAXIMUM (b) == (max) \ + ? (a) \ + : (a) % (_GL_INT_CONVERT (a, _GL_SIGNED_INT_MAXIMUM (b)) + 1)) \ + : (a) % - (b)) \ + == 0) + +/* Check for integer overflow, and report low order bits of answer. + + The INT_<op>_OVERFLOW macros return 1 if the corresponding C operators + might not yield numerically correct answers due to arithmetic overflow. + The INT_<op>_WRAPV macros also store the low-order bits of the answer. + These macros work correctly on all known practical hosts, and do not rely + on undefined behavior due to signed arithmetic overflow. + + Example usage, assuming A and B are long int: + + if (INT_MULTIPLY_OVERFLOW (a, b)) + printf ("result would overflow\n"); + else + printf ("result is %ld (no overflow)\n", a * b); + + Example usage with WRAPV flavor: + + long int result; + bool overflow = INT_MULTIPLY_WRAPV (a, b, &result); + printf ("result is %ld (%s)\n", result, + overflow ? "after overflow" : "no overflow"); + + Restrictions on these macros: + + These macros do not check for all possible numerical problems or + undefined or unspecified behavior: they do not check for division + by zero, for bad shift counts, or for shifting negative numbers. + + These macros may evaluate their arguments zero or multiple times, so the + arguments should not have side effects. + + The WRAPV macros are not constant expressions. They support only + +, binary -, and *. The result type must be signed. + + These macros are tuned for their last argument being a constant. + + Return 1 if the integer expressions A * B, A - B, -A, A * B, A / B, + A % B, and A << B would overflow, respectively. */ + +#define INT_ADD_OVERFLOW(a, b) \ + _GL_BINARY_OP_OVERFLOW (a, b, _GL_ADD_OVERFLOW) +#define INT_SUBTRACT_OVERFLOW(a, b) \ + _GL_BINARY_OP_OVERFLOW (a, b, _GL_SUBTRACT_OVERFLOW) +#if _GL_HAS_BUILTIN_OVERFLOW_P +# define INT_NEGATE_OVERFLOW(a) INT_SUBTRACT_OVERFLOW (0, a) +#else +# define INT_NEGATE_OVERFLOW(a) \ + INT_NEGATE_RANGE_OVERFLOW (a, _GL_INT_MINIMUM (a), _GL_INT_MAXIMUM (a)) +#endif +#define INT_MULTIPLY_OVERFLOW(a, b) \ + _GL_BINARY_OP_OVERFLOW (a, b, _GL_MULTIPLY_OVERFLOW) +#define INT_DIVIDE_OVERFLOW(a, b) \ + _GL_BINARY_OP_OVERFLOW (a, b, _GL_DIVIDE_OVERFLOW) +#define INT_REMAINDER_OVERFLOW(a, b) \ + _GL_BINARY_OP_OVERFLOW (a, b, _GL_REMAINDER_OVERFLOW) +#define INT_LEFT_SHIFT_OVERFLOW(a, b) \ + INT_LEFT_SHIFT_RANGE_OVERFLOW (a, b, \ + _GL_INT_MINIMUM (a), _GL_INT_MAXIMUM (a)) + +/* Return 1 if the expression A <op> B would overflow, + where OP_RESULT_OVERFLOW (A, B, MIN, MAX) does the actual test, + assuming MIN and MAX are the minimum and maximum for the result type. + Arguments should be free of side effects. */ +#define _GL_BINARY_OP_OVERFLOW(a, b, op_result_overflow) \ + op_result_overflow (a, b, \ + _GL_INT_MINIMUM (0 * (b) + (a)), \ + _GL_INT_MAXIMUM (0 * (b) + (a))) + +/* Store the low-order bits of A + B, A - B, A * B, respectively, into *R. + Return 1 if the result overflows. See above for restrictions. */ +#define INT_ADD_WRAPV(a, b, r) \ + _GL_INT_OP_WRAPV (a, b, r, +, __builtin_add_overflow, INT_ADD_OVERFLOW) +#define INT_SUBTRACT_WRAPV(a, b, r) \ + _GL_INT_OP_WRAPV (a, b, r, -, __builtin_sub_overflow, INT_SUBTRACT_OVERFLOW) +#define INT_MULTIPLY_WRAPV(a, b, r) \ + _GL_INT_OP_WRAPV (a, b, r, *, __builtin_mul_overflow, INT_MULTIPLY_OVERFLOW) + +/* Nonzero if this compiler has GCC bug 68193 or Clang bug 25390. See: + https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68193 + https://llvm.org/bugs/show_bug.cgi?id=25390 + For now, assume all versions of GCC-like compilers generate bogus + warnings for _Generic. This matters only for older compilers that + lack __builtin_add_overflow. */ +#if __GNUC__ +# define _GL__GENERIC_BOGUS 1 +#else +# define _GL__GENERIC_BOGUS 0 +#endif + +/* Store the low-order bits of A <op> B into *R, where OP specifies + the operation. BUILTIN is the builtin operation, and OVERFLOW the + overflow predicate. Return 1 if the result overflows. See above + for restrictions. */ +#if _GL_HAS_BUILTIN_OVERFLOW +# define _GL_INT_OP_WRAPV(a, b, r, op, builtin, overflow) builtin (a, b, r) +#elif 201112 <= __STDC_VERSION__ && !_GL__GENERIC_BOGUS +# define _GL_INT_OP_WRAPV(a, b, r, op, builtin, overflow) \ + (_Generic \ + (*(r), \ + signed char: \ + _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned int, \ + signed char, SCHAR_MIN, SCHAR_MAX), \ + short int: \ + _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned int, \ + short int, SHRT_MIN, SHRT_MAX), \ + int: \ + _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned int, \ + int, INT_MIN, INT_MAX), \ + long int: \ + _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned long int, \ + long int, LONG_MIN, LONG_MAX), \ + long long int: \ + _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned long long int, \ + long long int, LLONG_MIN, LLONG_MAX))) +#else +# define _GL_INT_OP_WRAPV(a, b, r, op, builtin, overflow) \ + (sizeof *(r) == sizeof (signed char) \ + ? _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned int, \ + signed char, SCHAR_MIN, SCHAR_MAX) \ + : sizeof *(r) == sizeof (short int) \ + ? _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned int, \ + short int, SHRT_MIN, SHRT_MAX) \ + : sizeof *(r) == sizeof (int) \ + ? _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned int, \ + int, INT_MIN, INT_MAX) \ + : _GL_INT_OP_WRAPV_LONGISH(a, b, r, op, overflow)) +# ifdef LLONG_MAX +# define _GL_INT_OP_WRAPV_LONGISH(a, b, r, op, overflow) \ + (sizeof *(r) == sizeof (long int) \ + ? _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned long int, \ + long int, LONG_MIN, LONG_MAX) \ + : _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned long long int, \ + long long int, LLONG_MIN, LLONG_MAX)) +# else +# define _GL_INT_OP_WRAPV_LONGISH(a, b, r, op, overflow) \ + _GL_INT_OP_CALC (a, b, r, op, overflow, unsigned long int, \ + long int, LONG_MIN, LONG_MAX) +# endif +#endif + +/* Store the low-order bits of A <op> B into *R, where the operation + is given by OP. Use the unsigned type UT for calculation to avoid + overflow problems. *R's type is T, with extrema TMIN and TMAX. + T must be a signed integer type. Return 1 if the result overflows. */ +#define _GL_INT_OP_CALC(a, b, r, op, overflow, ut, t, tmin, tmax) \ + (sizeof ((a) op (b)) < sizeof (t) \ + ? _GL_INT_OP_CALC1 ((t) (a), (t) (b), r, op, overflow, ut, t, tmin, tmax) \ + : _GL_INT_OP_CALC1 (a, b, r, op, overflow, ut, t, tmin, tmax)) +#define _GL_INT_OP_CALC1(a, b, r, op, overflow, ut, t, tmin, tmax) \ + ((overflow (a, b) \ + || (EXPR_SIGNED ((a) op (b)) && ((a) op (b)) < (tmin)) \ + || (tmax) < ((a) op (b))) \ + ? (*(r) = _GL_INT_OP_WRAPV_VIA_UNSIGNED (a, b, op, ut, t), 1) \ + : (*(r) = _GL_INT_OP_WRAPV_VIA_UNSIGNED (a, b, op, ut, t), 0)) + +/* Return the low-order bits of A <op> B, where the operation is given + by OP. Use the unsigned type UT for calculation to avoid undefined + behavior on signed integer overflow, and convert the result to type T. + UT is at least as wide as T and is no narrower than unsigned int, + T is two's complement, and there is no padding or trap representations. + Assume that converting UT to T yields the low-order bits, as is + done in all known two's-complement C compilers. E.g., see: + https://gcc.gnu.org/onlinedocs/gcc/Integers-implementation.html + + According to the C standard, converting UT to T yields an + implementation-defined result or signal for values outside T's + range. However, code that works around this theoretical problem + runs afoul of a compiler bug in Oracle Studio 12.3 x86. See: + https://lists.gnu.org/r/bug-gnulib/2017-04/msg00049.html + As the compiler bug is real, don't try to work around the + theoretical problem. */ + +#define _GL_INT_OP_WRAPV_VIA_UNSIGNED(a, b, op, ut, t) \ + ((t) ((ut) (a) op (ut) (b))) + +#endif /* _GL_INTPROPS_H */ -- 2.17.1 From 86022a73a20f701ec54eb9baa5d963d3c665051d Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Thu, 28 Jun 2018 12:49:07 -0700 Subject: [PATCH 2/2] posix: finish merge with Gnulib regex * posix/regex_internal.h: Include intprops.h. * posix/regcomp.c (re_compile_internal): Omit unnecessary call to __libc_lock_init, as the previous lock_init has already initialized the lock. * posix/regex.h (_Restrict_, _Restrict_arr_): Port to Gnulib case, where __restrict and __restrict_arr might not work. Do this in a better way than Gnulib did until today. * posix/regexec.c (check_add_overflow_idx): Remove, as it was not portable to ICC, and had some problems when __GNUC__ < 5. Use replaced by intprops.h's INT_ADD_WRAPV. --- ChangeLog | 13 +++++++++++++ posix/regcomp.c | 4 +--- posix/regex.c | 2 +- posix/regex.h | 27 ++++++++++++++++++++------- posix/regex_internal.c | 2 +- posix/regex_internal.h | 6 ++++-- posix/regexec.c | 21 ++------------------- 7 files changed, 42 insertions(+), 33 deletions(-) diff --git a/ChangeLog b/ChangeLog index 27793d1456..38935253b0 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,18 @@ 2018-06-28 Paul Eggert <eggert@cs.ucla.edu> + posix: finish merge with Gnulib regex + * posix/regex_internal.h: Include intprops.h. + * posix/regcomp.c (re_compile_internal): Omit unnecessary call to + __libc_lock_init, as the previous lock_init has already + initialized the lock. + * posix/regex.h (_Restrict_, _Restrict_arr_): + Port to Gnulib case, where __restrict and __restrict_arr + might not work. Do this in a better way than Gnulib + did until today. + * posix/regexec.c (check_add_overflow_idx): + Remove, as it was not portable to ICC, and had some problems + when __GNUC__ < 5. Use replaced by intprops.h's INT_ADD_WRAPV. + posix: add intprops.h to prepare for regex sync * include/intprops.h: New file, taken from Gnulib except with reworded header comments. This will be used in a diff --git a/posix/regcomp.c b/posix/regcomp.c index 8431268b1c..6aaa54f66d 100644 --- a/posix/regcomp.c +++ b/posix/regcomp.c @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ #ifdef _LIBC # include <locale/weight.h> @@ -784,8 +784,6 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, strncpy (dfa->re_str, pattern, length + 1); #endif - __libc_lock_init (dfa->lock); - err = re_string_construct (®exp, pattern, length, preg->translate, (syntax & RE_ICASE) != 0, dfa); if (BE (err != REG_NOERROR, 0)) diff --git a/posix/regex.c b/posix/regex.c index cfadc6b096..d6591e8670 100644 --- a/posix/regex.c +++ b/posix/regex.c @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ #ifndef _LIBC # include <config.h> diff --git a/posix/regex.h b/posix/regex.h index eef262981c..32933bc6d5 100644 --- a/posix/regex.h +++ b/posix/regex.h @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ #ifndef _REGEX_H #define _REGEX_H 1 @@ -607,19 +607,32 @@ extern int re_exec (const char *); # endif #endif -/* GCC 2.95 and later have "__restrict"; C99 compilers have +/* For plain 'restrict', use glibc's __restrict if defined. + Otherwise, GCC 2.95 and later have "__restrict"; C99 compilers have "restrict", and "configure" may have defined "restrict". Other compilers use __restrict, __restrict__, and _Restrict, and 'configure' might #define 'restrict' to those words, so pick a different name. */ #ifndef _Restrict_ -# define _Restrict_ __restrict +# if defined __restrict || 2 < __GNUC__ + (95 <= __GNUC_MINOR__) +# define _Restrict_ __restrict +# elif 199901L <= __STDC_VERSION__ || defined restrict +# define _Restrict_ restrict +# else +# define _Restrict_ +# endif #endif -/* gcc 3.1 and up support the [restrict] syntax. Don't trust - sys/cdefs.h's definition of __restrict_arr, though, as it - mishandles gcc -ansi -pedantic. */ +/* For [restrict], use glibc's __restrict_arr if available. + Otherwise, GCC 3.1 (not in C++ mode) and C99 support [restrict]. */ #ifndef _Restrict_arr_ -# define _Restrict_arr_ __restrict_arr +# ifdef __restrict_arr +# define _Restrict_arr_ __restrict_arr +# elif ((199901L <= __STDC_VERSION__ || 3 < __GNUC__ + (1 <= __GNUC_MINOR__)) \ + && !defined __GNUG__) +# define _Restrict_arr_ _Restrict_ +# else +# define _Restrict_arr_ +# endif #endif /* POSIX compatibility. */ diff --git a/posix/regex_internal.c b/posix/regex_internal.c index 2a0c01556e..7f0083b918 100644 --- a/posix/regex_internal.c +++ b/posix/regex_internal.c @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ static void re_string_construct_common (const char *str, Idx len, re_string_t *pstr, diff --git a/posix/regex_internal.h b/posix/regex_internal.h index e621b3eead..b27afbda2a 100644 --- a/posix/regex_internal.h +++ b/posix/regex_internal.h @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ #ifndef _REGEX_INTERNAL_H #define _REGEX_INTERNAL_H 1 @@ -33,11 +33,13 @@ #include <stdbool.h> #include <stdint.h> +#include "intprops.h" + #ifdef _LIBC # include <libc-lock.h> # define lock_define(name) __libc_lock_define (, name) # define lock_init(lock) (__libc_lock_init (lock), 0) -# define lock_fini(lock) +# define lock_fini(lock) ((void) 0) # define lock_lock(lock) __libc_lock_lock (lock) # define lock_unlock(lock) __libc_lock_unlock (lock) #elif defined GNULIB_LOCK && !defined USE_UNLOCKED_IO diff --git a/posix/regexec.c b/posix/regexec.c index 573fc877e1..63aef97535 100644 --- a/posix/regexec.c +++ b/posix/regexec.c @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ static reg_errcode_t match_ctx_init (re_match_context_t *cache, int eflags, Idx n); @@ -317,23 +317,6 @@ re_search_2 (struct re_pattern_buffer *bufp, const char *string1, Idx length1, weak_alias (__re_search_2, re_search_2) #endif -/* Set *R = A + B. Return true if the answer is mathematically incorrect due - to overflow; in this case, *R is the low order bits of the correct - answer. */ -static inline bool -check_add_overflow_idx (Idx a, Idx b, Idx *r) -{ -#if __GNUC__ >= 5 - return __builtin_add_overflow (a, b, r); -#else - if ((a > 0 && b > SSIZE_MAX - a) - || (a < 0 && b < SSIZE_MIN -a) - return false; - *r = a + b; - return true; -#endif -} - static regoff_t re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, Idx length1, const char *string2, Idx length2, Idx start, @@ -346,7 +329,7 @@ re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, char *s = NULL; if (BE ((length1 < 0 || length2 < 0 || stop < 0 - || check_add_overflow_idx (length1, length2, &len)), + || INT_ADD_WRAPV (length1, length2, &len)), 0)) return -2; -- 2.17.1
On 28/06/2018 17:21, Paul Eggert wrote: > Adhemerval Zanella wrote: >> This patch syncs regex code with gnulib (commit 16aa5a2). > > Thanks for doing all that work. I merged some of the changes embodied in that patch back to Gnulib; see <https://lists.gnu.org/r/bug-gnulib/2018-06/msg00095.html>. Here are some comments on the changes that I had questions about: > >> * regcomp: add a __libc_lock_init init for dfa lock. > > Why is this needed? The lock has already been initialized by the call to lock_init about 15 lines earlier. Please see attached patch, which reverts this change. I think we can remove it indeed, I did not change because on extenal project syncs I try to avoid deviate from original code and add extra changes in subsequent patches. > >> * regexec: instead of use intprops.h and its macros, this patch >> the check_add_overflow_idx function to check for addition >> overflow. The function need to be defined on the file because >> the file will be build multiple times wit 'Idx' type being >> redefined (I think intprops.h addition should be done in a >> separate patch). > > When used with Gnulib, that implementation won't work with ICC 17.0.4 20170411, which defines __GNUC__ to be 5 but does not support __builtin_add_overflow. Also, the __GNUC__ < 5 code doesn't look right for multiple reasons: (1) it doesn't set *r when returning false, contrary to the comment describing the code (presumably the comment needs to be fixed?), (2) it incorrectly uses SSIZE_MAX instead of INT_MAX when _REGEX_LARGE_OFFSETS is not defined, and (3) it has an unnecessary test for a > 0 (you need to test only whether a < 0). > > Although these issues could be fixed individually, why waste time? Let's just use intprops.h and avoid the hassle of hacking on the bugs of a partial replacement. I really want to make this change independent of intprops.h addition since previous attempts we tried to add it, some maintainers raised concerns about its inclusion. I think it would be better if we propose its addition independently and without any conditionality. To reiterate, although its macro machinery is somewhat complex, I do think its addition could make easier add checks for overflow, instead of splitting the logic over different places. For this change I have fixed the points you raised (as you pointed out the semantic in indeed to wrap over values). > > Attached please see an intprops.h addition that is done as a separate patch. The second attached patch goes back to using INT_ADD_WRAPV instead of check_add_overflow. > >> * regex.h: Remove _Restrict_ and _Restrict_arr_ definition based >> on __STDC_VERSION__ because its usage leads to failures on >> posix/check-installed-headers-c{xx}. > > That implementation assumes glibc, so won't work in Gnulib. I attempted to work around the check-installed-headers issue in the attached patch; if this doesn't work please let me know what the issue is. Thanks for checking on it, I sync with gnulib and changes seems to not trigger any check-installed-headers issues. > >> * regex_internal.h: Define lock_fini to empty macro because setting >> to 0 lead to build issues (error: statement with no effect >> [-Werror=unused-value]). > > Making it empty is problematic, since lock_fini is supposed to be usable wherever a function call could appear. Instead, let's define it to ((void) 0). That should fix the -Wunused-value issue in a better way. See attached patch. Ack, I changed on glibc side as well. > >> 2. posix/PCRE.tests: the test '(a)|\1' uses a backreference along >> with group creation and I am not sure if it is the correct >> behavior to accept it with regcomp (REG_EXTENDED). The GNU grep >> accepts it with ERE option though. > > POSIX allows recomp and grep to accept this pattern as an extension. I dunno what it means, though, and no doubt regcomp and grep should report an error instead of accepting it silently. That would be a different bugfix, though. Just to be clear, current glibc regex do accept this pattern with REG_EXTENDED. And the gnulib code does report an error instead of accepting it. So it is not clear to me, from glibc side, if we should continue to accept it or if we should provide old behaviour. > > One other thing: let's use https: intead of http: in URLs, as per Gnulib style. We should be using these everywhere in Glibc, of course, but one step at a time. Ack. > > See attached patches for what the above comments boil down to. They are intended to be applied after the patch you posted. The patches could all be squashed. Since you already pushed the changes on gnulib, I synced again. I am just not sure how we should handle the 'posix/PCRE.tests' change. -- This patch syncs regex code with gnulib (commit 16aa5a2). The main differences are: * regcomp: add a __libc_lock_init init for dfa lock. * regexec: instead of use intprops.h and its macros, this patch the check_add_overflow_idx function to check for addition overflow. The function need to be defined on the file because the file will be build multiple times wit 'Idx' type being redefined (I think intprops.h addition should be done in a separate patch). * regex.h: Remove _Restrict_ and _Restrict_arr_ definition based on __STDC_VERSION__ because its usage leads to failures on posix/check-installed-headers-c{xx}. * regex_internal.h: Define lock_fini to empty macro because setting to 0 lead to build issues (error: statement with no effect [-Werror=unused-value]). Also the prototypes of 're_string_realloc_buffers', 'build_wcs_buffer', 'build_wcs_upper_buffer', 'build_upper_buffer', 're_string_translate_buffer', 're_string_context_at' are kept in its implementation counterpart because of internal tests building. The same internal testing requires the 'bitset_*' macros to be 'static inline' instead of just 'static' plus attribute used in gnulib code. Only two changes in GLIBC regex testing are required: 1. posix/bug-regex28.c: as previously discussed [1] the change of expected results on the pattern should be safe. 2. posix/PCRE.tests: the test '(a)|\1' uses a backreference along with group creation and I am not sure if it is the correct behavior to accept it with regcomp (REG_EXTENDED). The GNU grep accepts it with ERE option though. This sync contains some patches from thread 'Regex: Make libc regex more usable outside GLIBC.' [2] which have been pushed upstream in gnulib. This patches also fixes some regex issues (BZ #23233, BZ #21163, BZ #18986, BZ #13762) and I did not add testcases for both #23233 and #13762 because I couldn't think a simple way to trigger the expected failure path to trigger them. Checked on x86_64-linux-gnu and i686-linux-gnu. [BZ #23233] [BZ #21163] [BZ #18986] [BZ #13762] * posix/Makefile (tests): Add bug-regex37 and bug-regex38. * posix/PCRE.tests: Remove invalid test. * posix/bug-regex28.c: Fix expected values for used syntax. * posix/bug-regex37.c: New file. * posix/bug-regex38.c: Likewise. * posix/regcomp.c: Sync with gnulib. * posix/regex.c: Likewise. * posix/regex.h: Likewise. * posix/regex_internal.c: Likewise. * posix/regex_internal.h: Likewise. * posix/regexec.c: Likewise. [1] https://sourceware.org/ml/libc-alpha/2017-12/msg00807.html [2] https://sourceware.org/ml/libc-alpha/2017-12/msg00237.html --- ChangeLog | 18 + posix/Makefile | 3 +- posix/PCRE.tests | 13 - posix/bug-regex28.c | 46 +-- posix/bug-regex37.c | 32 ++ posix/bug-regex38.c | 32 ++ posix/regcomp.c | 601 ++++++++++++++++++------------- posix/regex.c | 21 +- posix/regex.h | 335 ++++++++++------- posix/regex_internal.c | 295 ++++++++------- posix/regex_internal.h | 426 ++++++++++++++-------- posix/regexec.c | 952 ++++++++++++++++++++++++++----------------------- 12 files changed, 1612 insertions(+), 1162 deletions(-) create mode 100644 posix/bug-regex37.c create mode 100644 posix/bug-regex38.c diff --git a/posix/Makefile b/posix/Makefile index 989e42e..00c6284 100644 --- a/posix/Makefile +++ b/posix/Makefile @@ -95,7 +95,8 @@ tests := test-errno tstgetopt testfnm runtests runptests \ tst-posix_spawn-fd tst-posix_spawn-setsid \ tst-posix_fadvise tst-posix_fadvise64 \ tst-sysconf-empty-chroot tst-glob_symlinks tst-fexecve \ - tst-glob-tilde test-ssize-max tst-spawn4 + tst-glob-tilde test-ssize-max tst-spawn4 bug-regex37 \ + bug-regex38 tests-internal := bug-regex5 bug-regex20 bug-regex33 \ tst-rfc3484 tst-rfc3484-2 tst-rfc3484-3 \ tst-glob_lstat_compat tst-spawn4-compat diff --git a/posix/PCRE.tests b/posix/PCRE.tests index 0fb9cad..da63b86 100644 --- a/posix/PCRE.tests +++ b/posix/PCRE.tests @@ -1774,19 +1774,6 @@ No match 0: abcabc 1: abc -/(a)|\1/ - a - 0: a - 1: a - *** Failers - 0: a - 1: a - ab - 0: a - 1: a - x -No match - /abc/i ABC 0: ABC diff --git a/posix/bug-regex28.c b/posix/bug-regex28.c index 5353edf..ba263b2 100644 --- a/posix/bug-regex28.c +++ b/posix/bug-regex28.c @@ -21,18 +21,22 @@ #include <stdio.h> #include <string.h> +#include <support/test-driver.h> +#include <support/check.h> + struct tests { const char *regex; const char *string; reg_syntax_t syntax; int retval; -} tests[] = { +}; +static const struct tests tests[] = { #define EGREP RE_SYNTAX_EGREP #define EGREP_NL (RE_SYNTAX_EGREP | RE_DOT_NEWLINE) & ~RE_HAT_LISTS_NOT_NEWLINE - { "a.b", "a\nb", EGREP, -1 }, + { "a.b", "a\nb", EGREP, 0 }, { "a.b", "a\nb", EGREP_NL, 0 }, - { "a[^x]b", "a\nb", EGREP, -1 }, + { "a[^x]b", "a\nb", EGREP, 0 }, { "a[^x]b", "a\nb", EGREP_NL, 0 }, /* While \S and \W are internally handled as [^[:space:]] and [^[:alnum:]_], RE_HAT_LISTS_NOT_NEWLINE did not make any difference, so ensure @@ -42,33 +46,33 @@ struct tests { "a\\Wb", "a\nb", EGREP, 0 }, { "a\\Wb", "a\nb", EGREP_NL, 0 } }; +static const size_t tests_size = sizeof (tests) / sizeof (tests[0]); -int -main (void) +static int +do_test (void) { struct re_pattern_buffer r; - size_t i; - int ret = 0; - for (i = 0; i < sizeof (tests) / sizeof (tests[i]); ++i) + for (size_t i = 0; i < tests_size; i++) { re_set_syntax (tests[i].syntax); memset (&r, 0, sizeof (r)); - if (re_compile_pattern (tests[i].regex, strlen (tests[i].regex), &r)) - { - printf ("re_compile_pattern %zd failed\n", i); - ret = 1; - continue; - } + const char *re = re_compile_pattern (tests[i].regex, + strlen (tests[i].regex), &r); + TEST_VERIFY (re == NULL); + if (re != NULL) + continue; + size_t len = strlen (tests[i].string); int rv = re_search (&r, tests[i].string, len, 0, len, NULL); - if (rv != tests[i].retval) - { - printf ("re_search %zd unexpected value %d != %d\n", - i, rv, tests[i].retval); - ret = 1; - } + TEST_VERIFY (rv == tests[i].retval); + if (test_verbose > 0) + printf ("info: i=%zu rv=%d expected=%d\n", i, rv, tests[i].retval); + regfree (&r); } - return ret; + + return 0; } + +#include <support/test-driver.c> diff --git a/posix/bug-regex37.c b/posix/bug-regex37.c new file mode 100644 index 0000000..87a0916 --- /dev/null +++ b/posix/bug-regex37.c @@ -0,0 +1,32 @@ +/* Test regcomp return for invalid expression (BZ #21163). + Copyright (C) 2018 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <regex.h> + +#include <support/check.h> + +static int +do_test (void) +{ + char const pattern[] = "()*)|\\1)*"; + regex_t r; + TEST_VERIFY_EXIT (regcomp (&r, pattern, REG_EXTENDED) == REG_ESUBREG); + return 0; +} + +#include <support/test-driver.c> diff --git a/posix/bug-regex38.c b/posix/bug-regex38.c new file mode 100644 index 0000000..cb0eb7d --- /dev/null +++ b/posix/bug-regex38.c @@ -0,0 +1,32 @@ +/* Diagnose invalid back-reference in the ERE '()|\1' (BZ #18986). + Copyright (C) 2018 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <regex.h> + +#include <support/check.h> + +static int +do_test (void) +{ + char const pattern[] = "0|()0|\\1|0"; + regex_t r; + TEST_VERIFY_EXIT (regcomp (&r, pattern, REG_EXTENDED) == REG_ESUBREG); + return 0; +} + +#include <support/test-driver.c> diff --git a/posix/regcomp.c b/posix/regcomp.c index f5c09fe..6aaa54f 100644 --- a/posix/regcomp.c +++ b/posix/regcomp.c @@ -15,9 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <stdint.h> + <https://www.gnu.org/licenses/>. */ #ifdef _LIBC # include <locale/weight.h> @@ -51,14 +49,14 @@ static bin_tree_t *lower_subexp (reg_errcode_t *err, regex_t *preg, static reg_errcode_t calc_first (void *extra, bin_tree_t *node); static reg_errcode_t calc_next (void *extra, bin_tree_t *node); static reg_errcode_t link_nfa_nodes (void *extra, bin_tree_t *node); -static int duplicate_node (re_dfa_t *dfa, int org_idx, unsigned int constraint); -static int search_duplicated_node (const re_dfa_t *dfa, int org_node, +static Idx duplicate_node (re_dfa_t *dfa, Idx org_idx, unsigned int constraint); +static Idx search_duplicated_node (const re_dfa_t *dfa, Idx org_node, unsigned int constraint); static reg_errcode_t calc_eclosure (re_dfa_t *dfa); static reg_errcode_t calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, - int node, int root); + Idx node, bool root); static reg_errcode_t calc_inveclosure (re_dfa_t *dfa); -static int fetch_number (re_string_t *input, re_token_t *token, +static Idx fetch_number (re_string_t *input, re_token_t *token, reg_syntax_t syntax); static int peek_token (re_token_t *token, re_string_t *input, reg_syntax_t syntax); @@ -66,16 +64,16 @@ static bin_tree_t *parse (re_string_t *regexp, regex_t *preg, reg_syntax_t syntax, reg_errcode_t *err); static bin_tree_t *parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_sub_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_dup_op (bin_tree_t *dup_elem, re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, reg_syntax_t syntax, reg_errcode_t *err); @@ -87,34 +85,34 @@ static reg_errcode_t parse_bracket_element (bracket_elem_t *elem, re_token_t *token, int token_len, re_dfa_t *dfa, reg_syntax_t syntax, - int accept_hyphen); + bool accept_hyphen); static reg_errcode_t parse_bracket_symbol (bracket_elem_t *elem, re_string_t *regexp, re_token_t *token); #ifdef RE_ENABLE_I18N static reg_errcode_t build_equiv_class (bitset_t sbcset, re_charset_t *mbcset, - int *equiv_class_alloc, + Idx *equiv_class_alloc, const unsigned char *name); static reg_errcode_t build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, re_charset_t *mbcset, - int *char_class_alloc, - const unsigned char *class_name, + Idx *char_class_alloc, + const char *class_name, reg_syntax_t syntax); #else /* not RE_ENABLE_I18N */ static reg_errcode_t build_equiv_class (bitset_t sbcset, const unsigned char *name); static reg_errcode_t build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, - const unsigned char *class_name, + const char *class_name, reg_syntax_t syntax); #endif /* not RE_ENABLE_I18N */ static bin_tree_t *build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, - const unsigned char *class_name, - const unsigned char *extra, - int non_match, reg_errcode_t *err); + const char *class_name, + const char *extra, + bool non_match, reg_errcode_t *err); static bin_tree_t *create_tree (re_dfa_t *dfa, bin_tree_t *left, bin_tree_t *right, re_token_type_t type); @@ -131,7 +129,7 @@ static reg_errcode_t mark_opt_subexp (void *extra, bin_tree_t *node); POSIX doesn't require that we do anything for REG_NOERROR, but why not be nice? */ -const char __re_error_msgid[] attribute_hidden = +static const char __re_error_msgid[] = { #define REG_NOERROR_IDX 0 gettext_noop ("Success") /* REG_NOERROR */ @@ -155,9 +153,9 @@ const char __re_error_msgid[] attribute_hidden = gettext_noop ("Invalid back reference") /* REG_ESUBREG */ "\0" #define REG_EBRACK_IDX (REG_ESUBREG_IDX + sizeof "Invalid back reference") - gettext_noop ("Unmatched [ or [^") /* REG_EBRACK */ + gettext_noop ("Unmatched [, [^, [:, [., or [=") /* REG_EBRACK */ "\0" -#define REG_EPAREN_IDX (REG_EBRACK_IDX + sizeof "Unmatched [ or [^") +#define REG_EPAREN_IDX (REG_EBRACK_IDX + sizeof "Unmatched [, [^, [:, [., or [=") gettext_noop ("Unmatched ( or \\(") /* REG_EPAREN */ "\0" #define REG_EBRACE_IDX (REG_EPAREN_IDX + sizeof "Unmatched ( or \\(") @@ -185,7 +183,7 @@ const char __re_error_msgid[] attribute_hidden = gettext_noop ("Unmatched ) or \\)") /* REG_ERPAREN */ }; -const size_t __re_error_msgid_idx[] attribute_hidden = +static const size_t __re_error_msgid_idx[] = { REG_NOERROR_IDX, REG_NOMATCH_IDX, @@ -269,7 +267,7 @@ weak_alias (__re_set_syntax, re_set_syntax) int re_compile_fastmap (struct re_pattern_buffer *bufp) { - re_dfa_t *dfa = (re_dfa_t *) bufp->buffer; + re_dfa_t *dfa = bufp->buffer; char *fastmap = bufp->fastmap; memset (fastmap, '\0', sizeof (char) * SBC_MAX); @@ -303,12 +301,12 @@ static void re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, char *fastmap) { - re_dfa_t *dfa = (re_dfa_t *) bufp->buffer; - int node_cnt; - int icase = (dfa->mb_cur_max == 1 && (bufp->syntax & RE_ICASE)); + re_dfa_t *dfa = bufp->buffer; + Idx node_cnt; + bool icase = (dfa->mb_cur_max == 1 && (bufp->syntax & RE_ICASE)); for (node_cnt = 0; node_cnt < init_state->nodes.nelem; ++node_cnt) { - int node = init_state->nodes.elems[node_cnt]; + Idx node = init_state->nodes.elems[node_cnt]; re_token_type_t type = dfa->nodes[node].type; if (type == CHARACTER) @@ -317,7 +315,8 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, #ifdef RE_ENABLE_I18N if ((bufp->syntax & RE_ICASE) && dfa->mb_cur_max > 1) { - unsigned char *buf = alloca (dfa->mb_cur_max), *p; + unsigned char buf[MB_LEN_MAX]; + unsigned char *p; wchar_t wc; mbstate_t state; @@ -332,7 +331,7 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, &state) == p - buf && (__wcrtomb ((char *) buf, __towlower (wc), &state) != (size_t) -1)) - re_set_fastmap (fastmap, 0, buf[0]); + re_set_fastmap (fastmap, false, buf[0]); } #endif } @@ -352,7 +351,7 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, else if (type == COMPLEX_BRACKET) { re_charset_t *cset = dfa->nodes[node].opr.mbcset; - int i; + Idx i; # ifdef _LIBC /* See if we have to try all bytes which start multiple collation @@ -465,7 +464,7 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, the return codes and their meanings.) */ int -regcomp (regex_t *__restrict preg, const char *__restrict pattern, int cflags) +regcomp (regex_t *_Restrict_ preg, const char *_Restrict_ pattern, int cflags) { reg_errcode_t ret; reg_syntax_t syntax = ((cflags & REG_EXTENDED) ? RE_SYNTAX_POSIX_EXTENDED @@ -525,7 +524,7 @@ weak_alias (__regcomp, regcomp) from either regcomp or regexec. We don't use PREG here. */ size_t -regerror (int errcode, const regex_t *__restrict preg, char *__restrict errbuf, +regerror (int errcode, const regex_t *_Restrict_ preg, char *_Restrict_ errbuf, size_t errbuf_size) { const char *msg; @@ -546,17 +545,13 @@ regerror (int errcode, const regex_t *__restrict preg, char *__restrict errbuf, if (BE (errbuf_size != 0, 1)) { + size_t cpy_size = msg_size; if (BE (msg_size > errbuf_size, 0)) { -#if defined HAVE_MEMPCPY || defined _LIBC - *((char *) __mempcpy (errbuf, msg, errbuf_size - 1)) = '\0'; -#else - memcpy (errbuf, msg, errbuf_size - 1); - errbuf[errbuf_size - 1] = 0; -#endif + cpy_size = errbuf_size - 1; + errbuf[cpy_size] = '\0'; } - else - memcpy (errbuf, msg, msg_size); + memcpy (errbuf, msg, cpy_size); } return msg_size; @@ -574,7 +569,23 @@ weak_alias (__regerror, regerror) static const bitset_t utf8_sb_map = { /* Set the first 128 bits. */ +# if defined __GNUC__ && !defined __STRICT_ANSI__ [0 ... 0x80 / BITSET_WORD_BITS - 1] = BITSET_WORD_MAX +# else +# if 4 * BITSET_WORD_BITS < ASCII_CHARS +# error "bitset_word_t is narrower than 32 bits" +# elif 3 * BITSET_WORD_BITS < ASCII_CHARS + BITSET_WORD_MAX, BITSET_WORD_MAX, BITSET_WORD_MAX, +# elif 2 * BITSET_WORD_BITS < ASCII_CHARS + BITSET_WORD_MAX, BITSET_WORD_MAX, +# elif 1 * BITSET_WORD_BITS < ASCII_CHARS + BITSET_WORD_MAX, +# endif + (BITSET_WORD_MAX + >> (SBC_MAX % BITSET_WORD_BITS == 0 + ? 0 + : BITSET_WORD_BITS - SBC_MAX % BITSET_WORD_BITS)) +# endif }; #endif @@ -582,7 +593,7 @@ static const bitset_t utf8_sb_map = static void free_dfa_content (re_dfa_t *dfa) { - int i, j; + Idx i, j; if (dfa->nodes) for (i = 0; i < dfa->nodes_len; ++i) @@ -632,9 +643,12 @@ free_dfa_content (re_dfa_t *dfa) void regfree (regex_t *preg) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; if (BE (dfa != NULL, 1)) - free_dfa_content (dfa); + { + lock_fini (dfa->lock); + free_dfa_content (dfa); + } preg->buffer = NULL; preg->allocated = 0; @@ -687,7 +701,7 @@ re_comp (const char *s) if (re_comp_buf.fastmap == NULL) { - re_comp_buf.fastmap = (char *) malloc (SBC_MAX); + re_comp_buf.fastmap = re_malloc (char, SBC_MAX); if (re_comp_buf.fastmap == NULL) return (char *) gettext (__re_error_msgid + __re_error_msgid_idx[(int) REG_ESPACE]); @@ -704,7 +718,7 @@ re_comp (const char *s) if (!ret) return NULL; - /* Yes, we're discarding `const' here if !HAVE_LIBINTL. */ + /* Yes, we're discarding 'const' here if !HAVE_LIBINTL. */ return (char *) gettext (__re_error_msgid + __re_error_msgid_idx[(int) ret]); } @@ -739,7 +753,7 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, preg->regs_allocated = REGS_UNALLOCATED; /* Initialize the dfa. */ - dfa = (re_dfa_t *) preg->buffer; + dfa = preg->buffer; if (BE (preg->allocated < sizeof (re_dfa_t), 0)) { /* If zero allocated, but buffer is non-null, try to realloc @@ -750,11 +764,13 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, if (dfa == NULL) return REG_ESPACE; preg->allocated = sizeof (re_dfa_t); - preg->buffer = (unsigned char *) dfa; + preg->buffer = dfa; } preg->used = sizeof (re_dfa_t); err = init_dfa (dfa, length); + if (BE (err == REG_NOERROR && lock_init (dfa->lock) != 0, 0)) + err = REG_ESPACE; if (BE (err != REG_NOERROR, 0)) { free_dfa_content (dfa); @@ -768,15 +784,14 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, strncpy (dfa->re_str, pattern, length + 1); #endif - __libc_lock_init (dfa->lock); - err = re_string_construct (®exp, pattern, length, preg->translate, - syntax & RE_ICASE, dfa); + (syntax & RE_ICASE) != 0, dfa); if (BE (err != REG_NOERROR, 0)) { re_compile_internal_free_return: free_workarea_compile (preg); re_string_destruct (®exp); + lock_fini (dfa->lock); free_dfa_content (dfa); preg->buffer = NULL; preg->allocated = 0; @@ -809,6 +824,7 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, if (BE (err != REG_NOERROR, 0)) { + lock_fini (dfa->lock); free_dfa_content (dfa); preg->buffer = NULL; preg->allocated = 0; @@ -823,18 +839,32 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, static reg_errcode_t init_dfa (re_dfa_t *dfa, size_t pat_len) { - unsigned int table_size; + __re_size_t table_size; #ifndef _LIBC - char *codeset_name; + const char *codeset_name; +#endif +#ifdef RE_ENABLE_I18N + size_t max_i18n_object_size = MAX (sizeof (wchar_t), sizeof (wctype_t)); +#else + size_t max_i18n_object_size = 0; #endif + size_t max_object_size = + MAX (sizeof (struct re_state_table_entry), + MAX (sizeof (re_token_t), + MAX (sizeof (re_node_set), + MAX (sizeof (regmatch_t), + max_i18n_object_size)))); memset (dfa, '\0', sizeof (re_dfa_t)); /* Force allocation of str_tree_storage the first time. */ dfa->str_tree_storage_idx = BIN_TREE_STORAGE_SIZE; - /* Avoid overflows. */ - if (pat_len == SIZE_MAX) + /* Avoid overflows. The extra "/ 2" is for the table_size doubling + calculation below, and for similar doubling calculations + elsewhere. And it's <= rather than <, because some of the + doubling calculations add 1 afterwards. */ + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) / 2 <= pat_len, 0)) return REG_ESPACE; dfa->nodes_alloc = pat_len + 1; @@ -856,22 +886,11 @@ init_dfa (re_dfa_t *dfa, size_t pat_len) dfa->map_notascii = (_NL_CURRENT_WORD (LC_CTYPE, _NL_CTYPE_MAP_TO_NONASCII) != 0); #else -# ifdef HAVE_LANGINFO_CODESET codeset_name = nl_langinfo (CODESET); -# else - codeset_name = getenv ("LC_ALL"); - if (codeset_name == NULL || codeset_name[0] == '\0') - codeset_name = getenv ("LC_CTYPE"); - if (codeset_name == NULL || codeset_name[0] == '\0') - codeset_name = getenv ("LANG"); - if (codeset_name == NULL) - codeset_name = ""; - else if (strchr (codeset_name, '.') != NULL) - codeset_name = strchr (codeset_name, '.') + 1; -# endif - - if (strcasecmp (codeset_name, "UTF-8") == 0 - || strcasecmp (codeset_name, "UTF8") == 0) + if ((codeset_name[0] == 'U' || codeset_name[0] == 'u') + && (codeset_name[1] == 'T' || codeset_name[1] == 't') + && (codeset_name[2] == 'F' || codeset_name[2] == 'f') + && strcmp (codeset_name + 3 + (codeset_name[3] == '-'), "8") == 0) dfa->is_utf8 = 1; /* We check exhaustively in the loop below if this charset is a @@ -920,9 +939,10 @@ init_dfa (re_dfa_t *dfa, size_t pat_len) static void init_word_char (re_dfa_t *dfa) { - dfa->word_ops_used = 1; int i = 0; + int j; int ch = 0; + dfa->word_ops_used = 1; if (BE (dfa->map_notascii == 0, 1)) { /* Avoid uint32_t and uint64_t as some non-GCC platforms lack @@ -959,7 +979,7 @@ init_word_char (re_dfa_t *dfa) general_case: for (; i < BITSET_WORDS; ++i) - for (int j = 0; j < BITSET_WORD_BITS; ++j, ++ch) + for (j = 0; j < BITSET_WORD_BITS; ++j, ++ch) if (isalnum (ch) || ch == '_') dfa->word_char[i] |= (bitset_word_t) 1 << j; } @@ -969,7 +989,7 @@ init_word_char (re_dfa_t *dfa) static void free_workarea_compile (regex_t *preg) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_storage_t *storage, *next; for (storage = dfa->str_tree_storage; storage; storage = next) { @@ -988,7 +1008,7 @@ free_workarea_compile (regex_t *preg) static reg_errcode_t create_initial_state (re_dfa_t *dfa) { - int first, i; + Idx first, i; reg_errcode_t err; re_node_set init_nodes; @@ -1007,10 +1027,10 @@ create_initial_state (re_dfa_t *dfa) if (dfa->nbackref > 0) for (i = 0; i < init_nodes.nelem; ++i) { - int node_idx = init_nodes.elems[i]; + Idx node_idx = init_nodes.elems[i]; re_token_type_t type = dfa->nodes[node_idx].type; - int clexp_idx; + Idx clexp_idx; if (type != OP_BACK_REF) continue; for (clexp_idx = 0; clexp_idx < init_nodes.nelem; ++clexp_idx) @@ -1026,14 +1046,13 @@ create_initial_state (re_dfa_t *dfa) if (type == OP_BACK_REF) { - int dest_idx = dfa->edests[node_idx].elems[0]; + Idx dest_idx = dfa->edests[node_idx].elems[0]; if (!re_node_set_contains (&init_nodes, dest_idx)) { - reg_errcode_t err = re_node_set_merge (&init_nodes, - dfa->eclosures - + dest_idx); - if (err != REG_NOERROR) - return err; + reg_errcode_t merge_err + = re_node_set_merge (&init_nodes, dfa->eclosures + dest_idx); + if (merge_err != REG_NOERROR) + return merge_err; i = 0; } } @@ -1074,14 +1093,17 @@ create_initial_state (re_dfa_t *dfa) static void optimize_utf8 (re_dfa_t *dfa) { - int node, i, mb_chars = 0, has_period = 0; + Idx node; + int i; + bool mb_chars = false; + bool has_period = false; for (node = 0; node < dfa->nodes_len; ++node) switch (dfa->nodes[node].type) { case CHARACTER: - if (dfa->nodes[node].opr.c >= 0x80) - mb_chars = 1; + if (dfa->nodes[node].opr.c >= ASCII_CHARS) + mb_chars = true; break; case ANCHOR: switch (dfa->nodes[node].opr.ctx_type) @@ -1099,7 +1121,7 @@ optimize_utf8 (re_dfa_t *dfa) } break; case OP_PERIOD: - has_period = 1; + has_period = true; break; case OP_BACK_REF: case OP_ALT: @@ -1111,11 +1133,18 @@ optimize_utf8 (re_dfa_t *dfa) case COMPLEX_BRACKET: return; case SIMPLE_BRACKET: - /* Just double check. The non-ASCII range starts at 0x80. */ - assert (0x80 % BITSET_WORD_BITS == 0); - for (i = 0x80 / BITSET_WORD_BITS; i < BITSET_WORDS; ++i) - if (dfa->nodes[node].opr.sbcset[i]) - return; + /* Just double check. */ + { + int rshift = (ASCII_CHARS % BITSET_WORD_BITS == 0 + ? 0 + : BITSET_WORD_BITS - ASCII_CHARS % BITSET_WORD_BITS); + for (i = ASCII_CHARS / BITSET_WORD_BITS; i < BITSET_WORDS; ++i) + { + if (dfa->nodes[node].opr.sbcset[i] >> rshift != 0) + return; + rshift = 0; + } + } break; default: abort (); @@ -1125,7 +1154,7 @@ optimize_utf8 (re_dfa_t *dfa) for (node = 0; node < dfa->nodes_len; ++node) { if (dfa->nodes[node].type == CHARACTER - && dfa->nodes[node].opr.c >= 0x80) + && dfa->nodes[node].opr.c >= ASCII_CHARS) dfa->nodes[node].mb_partial = 0; else if (dfa->nodes[node].type == OP_PERIOD) dfa->nodes[node].type = OP_UTF8_PERIOD; @@ -1144,22 +1173,22 @@ optimize_utf8 (re_dfa_t *dfa) static reg_errcode_t analyze (regex_t *preg) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; reg_errcode_t ret; /* Allocate arrays. */ - dfa->nexts = re_malloc (int, dfa->nodes_alloc); - dfa->org_indices = re_malloc (int, dfa->nodes_alloc); + dfa->nexts = re_malloc (Idx, dfa->nodes_alloc); + dfa->org_indices = re_malloc (Idx, dfa->nodes_alloc); dfa->edests = re_malloc (re_node_set, dfa->nodes_alloc); dfa->eclosures = re_malloc (re_node_set, dfa->nodes_alloc); if (BE (dfa->nexts == NULL || dfa->org_indices == NULL || dfa->edests == NULL || dfa->eclosures == NULL, 0)) return REG_ESPACE; - dfa->subexp_map = re_malloc (int, preg->re_nsub); + dfa->subexp_map = re_malloc (Idx, preg->re_nsub); if (dfa->subexp_map != NULL) { - int i; + Idx i; for (i = 0; i < preg->re_nsub; i++) dfa->subexp_map[i] = i; preorder (dfa->str_tree, optimize_subexps, dfa); @@ -1168,7 +1197,7 @@ analyze (regex_t *preg) break; if (i == preg->re_nsub) { - free (dfa->subexp_map); + re_free (dfa->subexp_map); dfa->subexp_map = NULL; } } @@ -1284,7 +1313,7 @@ optimize_subexps (void *extra, bin_tree_t *node) else if (node->token.type == SUBEXP && node->left && node->left->token.type == SUBEXP) { - int other_idx = node->left->token.opr.idx; + Idx other_idx = node->left->token.opr.idx; node->left = node->left->left; if (node->left) @@ -1292,7 +1321,7 @@ optimize_subexps (void *extra, bin_tree_t *node) dfa->subexp_map[other_idx] = dfa->subexp_map[node->token.opr.idx]; if (other_idx < BITSET_WORD_BITS) - dfa->used_bkref_map &= ~((bitset_word_t) 1 << other_idx); + dfa->used_bkref_map &= ~((bitset_word_t) 1 << other_idx); } return REG_NOERROR; @@ -1325,7 +1354,7 @@ lower_subexps (void *extra, bin_tree_t *node) static bin_tree_t * lower_subexp (reg_errcode_t *err, regex_t *preg, bin_tree_t *node) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *body = node->left; bin_tree_t *op, *cls, *tree1, *tree; @@ -1408,7 +1437,7 @@ static reg_errcode_t link_nfa_nodes (void *extra, bin_tree_t *node) { re_dfa_t *dfa = (re_dfa_t *) extra; - int idx = node->node_idx; + Idx idx = node->node_idx; reg_errcode_t err = REG_NOERROR; switch (node->token.type) @@ -1423,7 +1452,7 @@ link_nfa_nodes (void *extra, bin_tree_t *node) case OP_DUP_ASTERISK: case OP_ALT: { - int left, right; + Idx left, right; dfa->has_plural_match = 1; if (node->left != NULL) left = node->left->first->node_idx; @@ -1465,14 +1494,15 @@ link_nfa_nodes (void *extra, bin_tree_t *node) to their own constraint. */ static reg_errcode_t -duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, - int root_node, unsigned int init_constraint) +duplicate_node_closure (re_dfa_t *dfa, Idx top_org_node, Idx top_clone_node, + Idx root_node, unsigned int init_constraint) { - int org_node, clone_node, ret; + Idx org_node, clone_node; + bool ok; unsigned int constraint = init_constraint; for (org_node = top_org_node, clone_node = top_clone_node;;) { - int org_dest, clone_dest; + Idx org_dest, clone_dest; if (dfa->nodes[org_node].type == OP_BACK_REF) { /* If the back reference epsilon-transit, its destination must @@ -1485,8 +1515,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, if (BE (clone_dest == -1, 0)) return REG_ESPACE; dfa->nexts[clone_node] = dfa->nexts[org_node]; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } else if (dfa->edests[org_node].nelem == 0) @@ -1504,12 +1534,12 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, org_dest = dfa->edests[org_node].elems[0]; re_node_set_empty (dfa->edests + clone_node); /* If the node is root_node itself, it means the epsilon closure - has a loop. Then tie it to the destination of the root_node. */ + has a loop. Then tie it to the destination of the root_node. */ if (org_node == root_node && clone_node != org_node) { - ret = re_node_set_insert (dfa->edests + clone_node, org_dest); - if (BE (ret < 0, 0)) - return REG_ESPACE; + ok = re_node_set_insert (dfa->edests + clone_node, org_dest); + if (BE (! ok, 0)) + return REG_ESPACE; break; } /* In case the node has another constraint, append it. */ @@ -1517,8 +1547,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, clone_dest = duplicate_node (dfa, org_dest, constraint); if (BE (clone_dest == -1, 0)) return REG_ESPACE; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } else /* dfa->edests[org_node].nelem == 2 */ @@ -1536,8 +1566,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, clone_dest = duplicate_node (dfa, org_dest, constraint); if (BE (clone_dest == -1, 0)) return REG_ESPACE; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; err = duplicate_node_closure (dfa, org_dest, clone_dest, root_node, constraint); @@ -1548,8 +1578,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, { /* There is a duplicated node which satisfies the constraint, use it to avoid infinite loop. */ - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } @@ -1557,8 +1587,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, clone_dest = duplicate_node (dfa, org_dest, constraint); if (BE (clone_dest == -1, 0)) return REG_ESPACE; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } org_node = org_dest; @@ -1570,11 +1600,11 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, /* Search for a node which is duplicated from the node ORG_NODE, and satisfies the constraint CONSTRAINT. */ -static int -search_duplicated_node (const re_dfa_t *dfa, int org_node, +static Idx +search_duplicated_node (const re_dfa_t *dfa, Idx org_node, unsigned int constraint) { - int idx; + Idx idx; for (idx = dfa->nodes_len - 1; dfa->nodes[idx].duplicated && idx > 0; --idx) { if (org_node == dfa->org_indices[idx] @@ -1588,10 +1618,10 @@ search_duplicated_node (const re_dfa_t *dfa, int org_node, Return the index of the new node, or -1 if insufficient storage is available. */ -static int -duplicate_node (re_dfa_t *dfa, int org_idx, unsigned int constraint) +static Idx +duplicate_node (re_dfa_t *dfa, Idx org_idx, unsigned int constraint) { - int dup_idx = re_dfa_add_node (dfa, dfa->nodes[org_idx]); + Idx dup_idx = re_dfa_add_node (dfa, dfa->nodes[org_idx]); if (BE (dup_idx != -1, 1)) { dfa->nodes[dup_idx].constraint = constraint; @@ -1607,17 +1637,18 @@ duplicate_node (re_dfa_t *dfa, int org_idx, unsigned int constraint) static reg_errcode_t calc_inveclosure (re_dfa_t *dfa) { - int src, idx, ret; + Idx src, idx; + bool ok; for (idx = 0; idx < dfa->nodes_len; ++idx) re_node_set_init_empty (dfa->inveclosures + idx); for (src = 0; src < dfa->nodes_len; ++src) { - int *elems = dfa->eclosures[src].elems; + Idx *elems = dfa->eclosures[src].elems; for (idx = 0; idx < dfa->eclosures[src].nelem; ++idx) { - ret = re_node_set_insert_last (dfa->inveclosures + elems[idx], src); - if (BE (ret == -1, 0)) + ok = re_node_set_insert_last (dfa->inveclosures + elems[idx], src); + if (BE (! ok, 0)) return REG_ESPACE; } } @@ -1630,11 +1661,12 @@ calc_inveclosure (re_dfa_t *dfa) static reg_errcode_t calc_eclosure (re_dfa_t *dfa) { - int node_idx, incomplete; + Idx node_idx; + bool incomplete; #ifdef DEBUG assert (dfa->nodes_len > 0); #endif - incomplete = 0; + incomplete = false; /* For each nodes, calculate epsilon closure. */ for (node_idx = 0; ; ++node_idx) { @@ -1644,7 +1676,7 @@ calc_eclosure (re_dfa_t *dfa) { if (!incomplete) break; - incomplete = 0; + incomplete = false; node_idx = 0; } @@ -1656,13 +1688,13 @@ calc_eclosure (re_dfa_t *dfa) if (dfa->eclosures[node_idx].nelem != 0) continue; /* Calculate epsilon closure of 'node_idx'. */ - err = calc_eclosure_iter (&eclosure_elem, dfa, node_idx, 1); + err = calc_eclosure_iter (&eclosure_elem, dfa, node_idx, true); if (BE (err != REG_NOERROR, 0)) return err; if (dfa->eclosures[node_idx].nelem == 0) { - incomplete = 1; + incomplete = true; re_node_set_free (&eclosure_elem); } } @@ -1672,13 +1704,13 @@ calc_eclosure (re_dfa_t *dfa) /* Calculate epsilon closure of NODE. */ static reg_errcode_t -calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, int node, int root) +calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, Idx node, bool root) { reg_errcode_t err; - int i; + Idx i; re_node_set eclosure; - int ret; - int incomplete = 0; + bool ok; + bool incomplete = false; err = re_node_set_alloc (&eclosure, dfa->edests[node].nelem + 1); if (BE (err != REG_NOERROR, 0)) return err; @@ -1704,19 +1736,19 @@ calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, int node, int root) for (i = 0; i < dfa->edests[node].nelem; ++i) { re_node_set eclosure_elem; - int edest = dfa->edests[node].elems[i]; - /* If calculating the epsilon closure of `edest' is in progress, + Idx edest = dfa->edests[node].elems[i]; + /* If calculating the epsilon closure of 'edest' is in progress, return intermediate result. */ if (dfa->eclosures[edest].nelem == -1) { - incomplete = 1; + incomplete = true; continue; } - /* If we haven't calculated the epsilon closure of `edest' yet, + /* If we haven't calculated the epsilon closure of 'edest' yet, calculate now. Otherwise use calculated epsilon closure. */ if (dfa->eclosures[edest].nelem == 0) { - err = calc_eclosure_iter (&eclosure_elem, dfa, edest, 0); + err = calc_eclosure_iter (&eclosure_elem, dfa, edest, false); if (BE (err != REG_NOERROR, 0)) return err; } @@ -1730,14 +1762,14 @@ calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, int node, int root) the epsilon closure of this node is also incomplete. */ if (dfa->eclosures[edest].nelem == 0) { - incomplete = 1; + incomplete = true; re_node_set_free (&eclosure_elem); } } /* An epsilon closure includes itself. */ - ret = re_node_set_insert (&eclosure, node); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (&eclosure, node); + if (BE (! ok, 0)) return REG_ESPACE; if (incomplete && !root) dfa->eclosures[node].nelem = 0; @@ -2046,16 +2078,18 @@ peek_token_bracket (re_token_t *token, re_string_t *input, reg_syntax_t syntax) case '.': token->type = OP_OPEN_COLL_ELEM; break; + case '=': token->type = OP_OPEN_EQUIV_CLASS; break; + case ':': if (syntax & RE_CHAR_CLASSES) { token->type = OP_OPEN_CHAR_CLASS; break; } - /* else fall through. */ + FALLTHROUGH; default: token->type = CHARACTER; token->opr.c = c; @@ -2099,7 +2133,7 @@ static bin_tree_t * parse (re_string_t *regexp, regex_t *preg, reg_syntax_t syntax, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree, *eor, *root; re_token_t current_token; dfa->syntax = syntax; @@ -2131,10 +2165,11 @@ parse (re_string_t *regexp, regex_t *preg, reg_syntax_t syntax, static bin_tree_t * parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree, *branch = NULL; + bitset_word_t initial_bkref_map = dfa->completed_bkref_map; tree = parse_branch (regexp, preg, token, syntax, nest, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; @@ -2145,6 +2180,8 @@ parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, if (token->type != OP_ALT && token->type != END_OF_RE && (nest == 0 || token->type != OP_CLOSE_SUBEXP)) { + bitset_word_t accumulated_bkref_map = dfa->completed_bkref_map; + dfa->completed_bkref_map = initial_bkref_map; branch = parse_branch (regexp, preg, token, syntax, nest, err); if (BE (*err != REG_NOERROR && branch == NULL, 0)) { @@ -2152,6 +2189,7 @@ parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, postorder (tree, free_tree, NULL); return NULL; } + dfa->completed_bkref_map |= accumulated_bkref_map; } else branch = NULL; @@ -2176,10 +2214,10 @@ parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, static bin_tree_t * parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - bin_tree_t *tree, *exp; - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + bin_tree_t *tree, *expr; + re_dfa_t *dfa = preg->buffer; tree = parse_expression (regexp, preg, token, syntax, nest, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; @@ -2187,19 +2225,19 @@ parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, while (token->type != OP_ALT && token->type != END_OF_RE && (nest == 0 || token->type != OP_CLOSE_SUBEXP)) { - exp = parse_expression (regexp, preg, token, syntax, nest, err); - if (BE (*err != REG_NOERROR && exp == NULL, 0)) + expr = parse_expression (regexp, preg, token, syntax, nest, err); + if (BE (*err != REG_NOERROR && expr == NULL, 0)) { if (tree != NULL) postorder (tree, free_tree, NULL); return NULL; } - if (tree != NULL && exp != NULL) + if (tree != NULL && expr != NULL) { - bin_tree_t *newtree = create_tree (dfa, tree, exp, CONCAT); + bin_tree_t *newtree = create_tree (dfa, tree, expr, CONCAT); if (newtree == NULL) { - postorder (exp, free_tree, NULL); + postorder (expr, free_tree, NULL); postorder (tree, free_tree, NULL); *err = REG_ESPACE; return NULL; @@ -2207,8 +2245,8 @@ parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, tree = newtree; } else if (tree == NULL) - tree = exp; - /* Otherwise exp == NULL, we don't need to create new tree. */ + tree = expr; + /* Otherwise expr == NULL, we don't need to create new tree. */ } return tree; } @@ -2221,9 +2259,9 @@ parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, static bin_tree_t * parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree; switch (token->type) { @@ -2253,16 +2291,19 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, } #endif break; + case OP_OPEN_SUBEXP: tree = parse_sub_exp (regexp, preg, token, syntax, nest + 1, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_OPEN_BRACKET: tree = parse_bracket_exp (regexp, dfa, token, syntax, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_BACK_REF: if (!BE (dfa->completed_bkref_map & (1 << token->opr.idx), 1)) { @@ -2279,13 +2320,14 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, ++dfa->nbackref; dfa->has_mb_node = 1; break; + case OP_OPEN_DUP_NUM: if (syntax & RE_CONTEXT_INVALID_DUP) { *err = REG_BADRPT; return NULL; } - /* FALLTHROUGH */ + FALLTHROUGH; case OP_DUP_ASTERISK: case OP_DUP_PLUS: case OP_DUP_QUESTION: @@ -2299,7 +2341,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, fetch_token (token, regexp, syntax); return parse_expression (regexp, preg, token, syntax, nest, err); } - /* else fall through */ + FALLTHROUGH; case OP_CLOSE_SUBEXP: if ((token->type == OP_CLOSE_SUBEXP) && !(syntax & RE_UNMATCHED_RIGHT_PAREN_ORD)) @@ -2307,7 +2349,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, *err = REG_ERPAREN; return NULL; } - /* else fall through */ + FALLTHROUGH; case OP_CLOSE_DUP_NUM: /* We treat it as a normal character. */ @@ -2322,6 +2364,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, return NULL; } break; + case ANCHOR: if ((token->opr.ctx_type & (WORD_DELIM | NOT_WORD_DELIM | WORD_FIRST | WORD_LAST)) @@ -2366,6 +2409,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, it must not be "<ANCHOR(^)><REPEAT(*)>". */ fetch_token (token, regexp, syntax); return tree; + case OP_PERIOD: tree = create_token_tree (dfa, NULL, NULL, token); if (BE (tree == NULL, 0)) @@ -2376,30 +2420,35 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, if (dfa->mb_cur_max > 1) dfa->has_mb_node = 1; break; + case OP_WORD: case OP_NOTWORD: tree = build_charclass_op (dfa, regexp->trans, - (const unsigned char *) "alnum", - (const unsigned char *) "_", + "alnum", + "_", token->type == OP_NOTWORD, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_SPACE: case OP_NOTSPACE: tree = build_charclass_op (dfa, regexp->trans, - (const unsigned char *) "space", - (const unsigned char *) "", + "space", + "", token->type == OP_NOTSPACE, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_ALT: case END_OF_RE: return NULL; + case BACK_SLASH: *err = REG_EESCAPE; return NULL; + default: /* Must not happen? */ #ifdef DEBUG @@ -2412,7 +2461,8 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, while (token->type == OP_DUP_ASTERISK || token->type == OP_DUP_PLUS || token->type == OP_DUP_QUESTION || token->type == OP_OPEN_DUP_NUM) { - bin_tree_t *dup_tree = parse_dup_op (tree, regexp, dfa, token, syntax, err); + bin_tree_t *dup_tree = parse_dup_op (tree, regexp, dfa, token, + syntax, err); if (BE (*err != REG_NOERROR && dup_tree == NULL, 0)) { if (tree != NULL) @@ -2444,9 +2494,9 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, static bin_tree_t * parse_sub_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree; size_t cur_nsub; cur_nsub = preg->re_nsub++; @@ -2489,7 +2539,7 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, reg_syntax_t syntax, reg_errcode_t *err) { bin_tree_t *tree = NULL, *old_tree = NULL; - int i, start, end, start_idx = re_string_cur_idx (regexp); + Idx i, start, end, start_idx = re_string_cur_idx (regexp); re_token_t start_token = *token; if (token->type == OP_OPEN_DUP_NUM) @@ -2535,12 +2585,19 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, return elem; } - if (BE ((end != -1 && start > end) || token->type != OP_CLOSE_DUP_NUM, 0)) + if (BE ((end != -1 && start > end) + || token->type != OP_CLOSE_DUP_NUM, 0)) { /* First number greater than second. */ *err = REG_BADBR; return NULL; } + + if (BE (RE_DUP_MAX < (end == -1 ? start : end), 0)) + { + *err = REG_ESIZE; + return NULL; + } } else { @@ -2583,26 +2640,35 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, old_tree = NULL; if (elem->token.type == SUBEXP) - postorder (elem, mark_opt_subexp, (void *) (long) elem->token.opr.idx); + { + uintptr_t subidx = elem->token.opr.idx; + postorder (elem, mark_opt_subexp, (void *) subidx); + } - tree = create_tree (dfa, elem, NULL, (end == -1 ? OP_DUP_ASTERISK : OP_ALT)); + tree = create_tree (dfa, elem, NULL, + (end == -1 ? OP_DUP_ASTERISK : OP_ALT)); if (BE (tree == NULL, 0)) goto parse_dup_op_espace; +/* From gnulib's "intprops.h": + True if the arithmetic type T is signed. */ +#define TYPE_SIGNED(t) (! ((t) 0 < (t) -1)) + /* This loop is actually executed only when end != -1, to rewrite <re>{0,n} as (<re>(<re>...<re>?)?)?... We have already created the start+1-th copy. */ - for (i = start + 2; i <= end; ++i) - { - elem = duplicate_tree (elem, dfa); - tree = create_tree (dfa, tree, elem, CONCAT); - if (BE (elem == NULL || tree == NULL, 0)) - goto parse_dup_op_espace; - - tree = create_tree (dfa, tree, NULL, OP_ALT); - if (BE (tree == NULL, 0)) - goto parse_dup_op_espace; - } + if (TYPE_SIGNED (Idx) || end != -1) + for (i = start + 2; i <= end; ++i) + { + elem = duplicate_tree (elem, dfa); + tree = create_tree (dfa, tree, elem, CONCAT); + if (BE (elem == NULL || tree == NULL, 0)) + goto parse_dup_op_espace; + + tree = create_tree (dfa, tree, NULL, OP_ALT); + if (BE (tree == NULL, 0)) + goto parse_dup_op_espace; + } if (old_tree) tree = create_tree (dfa, old_tree, tree, CONCAT); @@ -2619,6 +2685,19 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, #define BRACKET_NAME_BUF_SIZE 32 #ifndef _LIBC + +# ifdef RE_ENABLE_I18N +/* Convert the byte B to the corresponding wide character. In a + unibyte locale, treat B as itself if it is an encoding error. + In a multibyte locale, return WEOF if B is an encoding error. */ +static wint_t +parse_byte (unsigned char b, re_charset_t *mbcset) +{ + wint_t wc = __btowc (b); + return wc == WEOF && !mbcset ? b : wc; +} +#endif + /* Local function for parse_bracket_exp only used in case of NOT _LIBC. Build the range expression which starts from START_ELEM, and ends at END_ELEM. The result are written to MBCSET and SBCSET. @@ -2628,11 +2707,17 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, static reg_errcode_t # ifdef RE_ENABLE_I18N -build_range_exp (bitset_t sbcset, re_charset_t *mbcset, int *range_alloc, - bracket_elem_t *start_elem, bracket_elem_t *end_elem) +build_range_exp (const reg_syntax_t syntax, + bitset_t sbcset, + re_charset_t *mbcset, + Idx *range_alloc, + const bracket_elem_t *start_elem, + const bracket_elem_t *end_elem) # else /* not RE_ENABLE_I18N */ -build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, - bracket_elem_t *end_elem) +build_range_exp (const reg_syntax_t syntax, + bitset_t sbcset, + const bracket_elem_t *start_elem, + const bracket_elem_t *end_elem) # endif /* not RE_ENABLE_I18N */ { unsigned int start_ch, end_ch; @@ -2655,7 +2740,6 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, wchar_t wc; wint_t start_wc; wint_t end_wc; - wchar_t cmp_buf[6] = {L'\0', L'\0', L'\0', L'\0', L'\0', L'\0'}; start_ch = ((start_elem->type == SB_CHAR) ? start_elem->opr.ch : ((start_elem->type == COLL_SYM) ? start_elem->opr.name[0] @@ -2664,14 +2748,12 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, : ((end_elem->type == COLL_SYM) ? end_elem->opr.name[0] : 0)); start_wc = ((start_elem->type == SB_CHAR || start_elem->type == COLL_SYM) - ? __btowc (start_ch) : start_elem->opr.wch); + ? parse_byte (start_ch, mbcset) : start_elem->opr.wch); end_wc = ((end_elem->type == SB_CHAR || end_elem->type == COLL_SYM) - ? __btowc (end_ch) : end_elem->opr.wch); + ? parse_byte (end_ch, mbcset) : end_elem->opr.wch); if (start_wc == WEOF || end_wc == WEOF) return REG_ECOLLATE; - cmp_buf[0] = start_wc; - cmp_buf[4] = end_wc; - if (__wcscoll (cmp_buf, cmp_buf + 4) > 0) + else if (BE ((syntax & RE_NO_EMPTY_RANGES) && start_wc > end_wc, 0)) return REG_ERANGE; /* Got valid collation sequence values, add them as a new entry. @@ -2686,7 +2768,7 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, { /* There is not enough space, need realloc. */ wchar_t *new_array_start, *new_array_end; - int new_nranges; + Idx new_nranges; /* +1 in case of mbcset->nranges is 0. */ new_nranges = 2 * mbcset->nranges + 1; @@ -2698,7 +2780,11 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, new_nranges); if (BE (new_array_start == NULL || new_array_end == NULL, 0)) - return REG_ESPACE; + { + re_free (new_array_start); + re_free (new_array_end); + return REG_ESPACE; + } mbcset->range_starts = new_array_start; mbcset->range_ends = new_array_end; @@ -2712,9 +2798,7 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, /* Build the table for single byte characters. */ for (wc = 0; wc < SBC_MAX; ++wc) { - cmp_buf[2] = wc; - if (__wcscoll (cmp_buf, cmp_buf + 2) <= 0 - && __wcscoll (cmp_buf + 2, cmp_buf + 4) <= 0) + if (start_wc <= wc && wc <= end_wc) bitset_set (sbcset, wc); } } @@ -2749,7 +2833,7 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, static reg_errcode_t # ifdef RE_ENABLE_I18N build_collating_symbol (bitset_t sbcset, re_charset_t *mbcset, - int *coll_sym_alloc, const unsigned char *name) + Idx *coll_sym_alloc, const unsigned char *name) # else /* not RE_ENABLE_I18N */ build_collating_symbol (bitset_t sbcset, const unsigned char *name) # endif /* not RE_ENABLE_I18N */ @@ -2895,6 +2979,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, 0)) return REG_ERANGE; + /* FIXME: Implement rational ranges here, too. */ start_collseq = lookup_collation_sequence_value (start_elem); end_collseq = lookup_collation_sequence_value (end_elem); /* Check start/end collation sequence values. */ @@ -2915,7 +3000,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, /* There is not enough space, need realloc. */ uint32_t *new_array_start; uint32_t *new_array_end; - int new_nranges; + Idx new_nranges; /* +1 in case of mbcset->nranges is 0. */ new_nranges = 2 * mbcset->nranges + 1; @@ -2962,7 +3047,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, auto inline reg_errcode_t __attribute__ ((always_inline)) build_collating_symbol (bitset_t sbcset, re_charset_t *mbcset, - int *coll_sym_alloc, const unsigned char *name) + Idx *coll_sym_alloc, const unsigned char *name) { int32_t elem, idx; size_t name_len = strlen ((const char *) name); @@ -2992,7 +3077,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, { /* Not enough, realloc it. */ /* +1 in case of mbcset->ncoll_syms is 0. */ - int new_coll_sym_alloc = 2 * mbcset->ncoll_syms + 1; + Idx new_coll_sym_alloc = 2 * mbcset->ncoll_syms + 1; /* Use realloc since mbcset->coll_syms is NULL if *alloc == 0. */ int32_t *new_coll_syms = re_realloc (mbcset->coll_syms, int32_t, @@ -3022,13 +3107,13 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, re_bitset_ptr_t sbcset; #ifdef RE_ENABLE_I18N re_charset_t *mbcset; - int coll_sym_alloc = 0, range_alloc = 0, mbchar_alloc = 0; - int equiv_class_alloc = 0, char_class_alloc = 0; + Idx coll_sym_alloc = 0, range_alloc = 0, mbchar_alloc = 0; + Idx equiv_class_alloc = 0, char_class_alloc = 0; #endif /* not RE_ENABLE_I18N */ - int non_match = 0; + bool non_match = false; bin_tree_t *work_tree; int token_len; - int first_round = 1; + bool first_round = true; #ifdef _LIBC collseqmb = (const unsigned char *) _NL_CURRENT (LC_COLLATE, _NL_COLLATE_COLLSEQMB); @@ -3075,7 +3160,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, #ifdef RE_ENABLE_I18N mbcset->non_match = 1; #endif /* not RE_ENABLE_I18N */ - non_match = 1; + non_match = true; if (syntax & RE_HAT_LISTS_NOT_NEWLINE) bitset_set (sbcset, '\n'); re_string_skip_bytes (regexp, token_len); /* Skip a token. */ @@ -3097,7 +3182,8 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, unsigned char start_name_buf[BRACKET_NAME_BUF_SIZE]; unsigned char end_name_buf[BRACKET_NAME_BUF_SIZE]; reg_errcode_t ret; - int token_len2 = 0, is_range_exp = 0; + int token_len2 = 0; + bool is_range_exp = false; re_token_t token2; start_elem.opr.name = start_name_buf; @@ -3109,7 +3195,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, *err = ret; goto parse_bracket_exp_free_return; } - first_round = 0; + first_round = false; /* Get information about the next token. We need it in any case. */ token_len = peek_token_bracket (token, regexp, syntax); @@ -3138,16 +3224,16 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, token->type = CHARACTER; } else - is_range_exp = 1; + is_range_exp = true; } } - if (is_range_exp == 1) + if (is_range_exp == true) { end_elem.opr.name = end_name_buf; end_elem.type = COLL_SYM; ret = parse_bracket_element (&end_elem, regexp, &token2, token_len2, - dfa, syntax, 1); + dfa, syntax, true); if (BE (ret != REG_NOERROR, 0)) { *err = ret; @@ -3161,11 +3247,11 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, &start_elem, &end_elem); #else # ifdef RE_ENABLE_I18N - *err = build_range_exp (sbcset, + *err = build_range_exp (syntax, sbcset, dfa->mb_cur_max > 1 ? mbcset : NULL, &range_alloc, &start_elem, &end_elem); # else - *err = build_range_exp (sbcset, &start_elem, &end_elem); + *err = build_range_exp (syntax, sbcset, &start_elem, &end_elem); # endif #endif /* RE_ENABLE_I18N */ if (BE (*err != REG_NOERROR, 0)) @@ -3220,7 +3306,8 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, #ifdef RE_ENABLE_I18N mbcset, &char_class_alloc, #endif /* RE_ENABLE_I18N */ - start_elem.opr.name, syntax); + (const char *) start_elem.opr.name, + syntax); if (BE (*err != REG_NOERROR, 0)) goto parse_bracket_exp_free_return; break; @@ -3317,7 +3404,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, static reg_errcode_t parse_bracket_element (bracket_elem_t *elem, re_string_t *regexp, re_token_t *token, int token_len, re_dfa_t *dfa, - reg_syntax_t syntax, int accept_hyphen) + reg_syntax_t syntax, bool accept_hyphen) { #ifdef RE_ENABLE_I18N int cur_char_size; @@ -3404,7 +3491,7 @@ parse_bracket_symbol (bracket_elem_t *elem, re_string_t *regexp, static reg_errcode_t #ifdef RE_ENABLE_I18N build_equiv_class (bitset_t sbcset, re_charset_t *mbcset, - int *equiv_class_alloc, const unsigned char *name) + Idx *equiv_class_alloc, const unsigned char *name) #else /* not RE_ENABLE_I18N */ build_equiv_class (bitset_t sbcset, const unsigned char *name) #endif /* not RE_ENABLE_I18N */ @@ -3466,7 +3553,7 @@ build_equiv_class (bitset_t sbcset, const unsigned char *name) { /* Not enough, realloc it. */ /* +1 in case of mbcset->nequiv_classes is 0. */ - int new_equiv_class_alloc = 2 * mbcset->nequiv_classes + 1; + Idx new_equiv_class_alloc = 2 * mbcset->nequiv_classes + 1; /* Use realloc since the array is NULL if *alloc == 0. */ int32_t *new_equiv_classes = re_realloc (mbcset->equiv_classes, int32_t, @@ -3497,15 +3584,15 @@ build_equiv_class (bitset_t sbcset, const unsigned char *name) static reg_errcode_t #ifdef RE_ENABLE_I18N build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, - re_charset_t *mbcset, int *char_class_alloc, - const unsigned char *class_name, reg_syntax_t syntax) + re_charset_t *mbcset, Idx *char_class_alloc, + const char *class_name, reg_syntax_t syntax) #else /* not RE_ENABLE_I18N */ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, - const unsigned char *class_name, reg_syntax_t syntax) + const char *class_name, reg_syntax_t syntax) #endif /* not RE_ENABLE_I18N */ { int i; - const char *name = (const char *) class_name; + const char *name = class_name; /* In case of REG_ICASE "upper" and "lower" match the both of upper and lower cases. */ @@ -3519,7 +3606,7 @@ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, { /* Not enough, realloc it. */ /* +1 in case of mbcset->nchar_classes is 0. */ - int new_char_class_alloc = 2 * mbcset->nchar_classes + 1; + Idx new_char_class_alloc = 2 * mbcset->nchar_classes + 1; /* Use realloc since array is NULL if *alloc == 0. */ wctype_t *new_char_classes = re_realloc (mbcset->char_classes, wctype_t, new_char_class_alloc); @@ -3536,13 +3623,13 @@ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, if (BE (trans != NULL, 0)) \ { \ for (i = 0; i < SBC_MAX; ++i) \ - if (ctype_func (i)) \ + if (ctype_func (i)) \ bitset_set (sbcset, trans[i]); \ } \ else \ { \ for (i = 0; i < SBC_MAX; ++i) \ - if (ctype_func (i)) \ + if (ctype_func (i)) \ bitset_set (sbcset, i); \ } \ } while (0) @@ -3579,40 +3666,35 @@ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, static bin_tree_t * build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, - const unsigned char *class_name, - const unsigned char *extra, int non_match, + const char *class_name, + const char *extra, bool non_match, reg_errcode_t *err) { re_bitset_ptr_t sbcset; #ifdef RE_ENABLE_I18N re_charset_t *mbcset; - int alloc = 0; + Idx alloc = 0; #endif /* not RE_ENABLE_I18N */ reg_errcode_t ret; re_token_t br_token; bin_tree_t *tree; sbcset = (re_bitset_ptr_t) calloc (sizeof (bitset_t), 1); -#ifdef RE_ENABLE_I18N - mbcset = (re_charset_t *) calloc (sizeof (re_charset_t), 1); -#endif /* RE_ENABLE_I18N */ - -#ifdef RE_ENABLE_I18N - if (BE (sbcset == NULL || mbcset == NULL, 0)) -#else /* not RE_ENABLE_I18N */ if (BE (sbcset == NULL, 0)) -#endif /* not RE_ENABLE_I18N */ { *err = REG_ESPACE; return NULL; } - - if (non_match) - { #ifdef RE_ENABLE_I18N - mbcset->non_match = 1; -#endif /* not RE_ENABLE_I18N */ + mbcset = (re_charset_t *) calloc (sizeof (re_charset_t), 1); + if (BE (mbcset == NULL, 0)) + { + re_free (sbcset); + *err = REG_ESPACE; + return NULL; } + mbcset->non_match = non_match; +#endif /* RE_ENABLE_I18N */ /* We don't care the syntax in this case. */ ret = build_charclass (trans, sbcset, @@ -3645,6 +3727,9 @@ build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, #endif /* Build a tree for simple bracket. */ +#if defined GCC_LINT || defined lint + memset (&br_token, 0, sizeof br_token); +#endif br_token.type = SIMPLE_BRACKET; br_token.opr.sbcset = sbcset; tree = create_token_tree (dfa, NULL, NULL, &br_token); @@ -3686,14 +3771,15 @@ build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, } /* This is intended for the expressions like "a{1,3}". - Fetch a number from `input', and return the number. - Return -1, if the number field is empty like "{,1}". - Return -2, If an error is occured. */ + Fetch a number from 'input', and return the number. + Return -1 if the number field is empty like "{,1}". + Return RE_DUP_MAX + 1 if the number field is too large. + Return -2 if an error occurred. */ -static int +static Idx fetch_number (re_string_t *input, re_token_t *token, reg_syntax_t syntax) { - int num = -1; + Idx num = -1; unsigned char c; while (1) { @@ -3704,8 +3790,10 @@ fetch_number (re_string_t *input, re_token_t *token, reg_syntax_t syntax) if (token->type == OP_CLOSE_DUP_NUM || c == ',') break; num = ((token->type != CHARACTER || c < '0' || '9' < c || num == -2) - ? -2 : ((num == -1) ? c - '0' : num * 10 + c - '0')); - num = (num > RE_DUP_MAX) ? -2 : num; + ? -2 + : num == -1 + ? c - '0' + : MIN (RE_DUP_MAX + 1, num * 10 + c - '0')); } return num; } @@ -3735,6 +3823,9 @@ create_tree (re_dfa_t *dfa, bin_tree_t *left, bin_tree_t *right, re_token_type_t type) { re_token_t t; +#if defined GCC_LINT || defined lint + memset (&t, 0, sizeof t); +#endif t.type = type; return create_token_tree (dfa, left, right, &t); } @@ -3779,7 +3870,7 @@ create_token_tree (re_dfa_t *dfa, bin_tree_t *left, bin_tree_t *right, static reg_errcode_t mark_opt_subexp (void *extra, bin_tree_t *node) { - int idx = (int) (long) extra; + Idx idx = (uintptr_t) extra; if (node->token.type == SUBEXP && node->token.opr.idx == idx) node->token.opt_subexp = 1; diff --git a/posix/regex.c b/posix/regex.c index 68b5ef1..d6591e8 100644 --- a/posix/regex.c +++ b/posix/regex.c @@ -15,14 +15,22 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ -#ifdef HAVE_CONFIG_H -#include "config.h" +#ifndef _LIBC +# include <config.h> + +# if (__GNUC__ == 4 && 6 <= __GNUC_MINOR__) || 4 < __GNUC__ +# pragma GCC diagnostic ignored "-Wsuggest-attribute=pure" +# endif +# if (__GNUC__ == 4 && 3 <= __GNUC_MINOR__) || 4 < __GNUC__ +# pragma GCC diagnostic ignored "-Wold-style-definition" +# pragma GCC diagnostic ignored "-Wtype-limits" +# endif #endif -/* Make sure noone compiles this code with a C++ compiler. */ -#ifdef __cplusplus +/* Make sure no one compiles this code with a C++ compiler. */ +#if defined __cplusplus && defined _LIBC # error "This is C code, use a C compiler" #endif @@ -56,9 +64,6 @@ #undefs RE_DUP_MAX and sets it to the right value. */ #include <limits.h> -/* This header defines the MIN and MAX macros. */ -#include <sys/param.h> - #include <regex.h> #include "regex_internal.h" diff --git a/posix/regex.h b/posix/regex.h index e0b8915..32933bc 100644 --- a/posix/regex.h +++ b/posix/regex.h @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ #ifndef _REGEX_H #define _REGEX_H 1 @@ -27,6 +27,36 @@ extern "C" { #endif +/* Define __USE_GNU to declare GNU extensions that violate the + POSIX name space rules. */ +#ifdef _GNU_SOURCE +# define __USE_GNU 1 +#endif + +#ifdef _REGEX_LARGE_OFFSETS + +/* Use types and values that are wide enough to represent signed and + unsigned byte offsets in memory. This currently works only when + the regex code is used outside of the GNU C library; it is not yet + supported within glibc itself, and glibc users should not define + _REGEX_LARGE_OFFSETS. */ + +/* The type of object sizes. */ +typedef size_t __re_size_t; + +/* The type of object sizes, in places where the traditional code + uses unsigned long int. */ +typedef size_t __re_long_size_t; + +#else + +/* The traditional GNU regex implementation mishandles strings longer + than INT_MAX. */ +typedef unsigned int __re_size_t; +typedef unsigned long int __re_long_size_t; + +#endif + /* The following two types have to be signed and unsigned integer type wide enough to hold a value of a pointer. For most ANSI compilers ptrdiff_t and size_t should be likely OK. Still size of these two @@ -108,9 +138,9 @@ typedef unsigned long int reg_syntax_t; If not set, newline is literal. */ # define RE_NEWLINE_ALT (RE_LIMITED_OPS << 1) -/* If this bit is set, then `{...}' defines an interval, and \{ and \} +/* If this bit is set, then '{...}' defines an interval, and \{ and \} are literals. - If not set, then `\{...\}' defines an interval. */ + If not set, then '\{...\}' defines an interval. */ # define RE_NO_BK_BRACES (RE_NEWLINE_ALT << 1) /* If this bit is set, (...) defines a group, and \( and \) are literals. @@ -165,8 +195,8 @@ typedef unsigned long int reg_syntax_t; whether ^ should be special. */ # define RE_CARET_ANCHORS_HERE (RE_ICASE << 1) -/* If this bit is set, then \{ cannot be first in an bre or - immediately after an alternation or begin-group operator. */ +/* If this bit is set, then \{ cannot be first in a regex or + immediately after an alternation, open-group or \} operator. */ # define RE_CONTEXT_INVALID_DUP (RE_CARET_ANCHORS_HERE << 1) /* If this bit is set, then no_sub will be set to 1 during @@ -185,9 +215,9 @@ extern reg_syntax_t re_syntax_options; (The [[[ comments delimit what gets put into the Texinfo file, so don't delete them!) */ /* [[[begin syntaxes]]] */ -#define RE_SYNTAX_EMACS 0 +# define RE_SYNTAX_EMACS 0 -#define RE_SYNTAX_AWK \ +# define RE_SYNTAX_AWK \ (RE_BACKSLASH_ESCAPE_IN_LISTS | RE_DOT_NOT_NULL \ | RE_NO_BK_PARENS | RE_NO_BK_REFS \ | RE_NO_BK_VBAR | RE_NO_EMPTY_RANGES \ @@ -195,52 +225,49 @@ extern reg_syntax_t re_syntax_options; | RE_CHAR_CLASSES \ | RE_UNMATCHED_RIGHT_PAREN_ORD | RE_NO_GNU_OPS) -#define RE_SYNTAX_GNU_AWK \ +# define RE_SYNTAX_GNU_AWK \ ((RE_SYNTAX_POSIX_EXTENDED | RE_BACKSLASH_ESCAPE_IN_LISTS \ | RE_INVALID_INTERVAL_ORD) \ & ~(RE_DOT_NOT_NULL | RE_CONTEXT_INDEP_OPS \ | RE_CONTEXT_INVALID_OPS )) -#define RE_SYNTAX_POSIX_AWK \ +# define RE_SYNTAX_POSIX_AWK \ (RE_SYNTAX_POSIX_EXTENDED | RE_BACKSLASH_ESCAPE_IN_LISTS \ | RE_INTERVALS | RE_NO_GNU_OPS \ | RE_INVALID_INTERVAL_ORD) -#define RE_SYNTAX_GREP \ - (RE_BK_PLUS_QM | RE_CHAR_CLASSES \ - | RE_HAT_LISTS_NOT_NEWLINE | RE_INTERVALS \ - | RE_NEWLINE_ALT) +# define RE_SYNTAX_GREP \ + ((RE_SYNTAX_POSIX_BASIC | RE_NEWLINE_ALT) \ + & ~(RE_CONTEXT_INVALID_DUP | RE_DOT_NOT_NULL)) -#define RE_SYNTAX_EGREP \ - (RE_CHAR_CLASSES | RE_CONTEXT_INDEP_ANCHORS \ - | RE_CONTEXT_INDEP_OPS | RE_HAT_LISTS_NOT_NEWLINE \ - | RE_NEWLINE_ALT | RE_NO_BK_PARENS \ - | RE_NO_BK_VBAR) +# define RE_SYNTAX_EGREP \ + ((RE_SYNTAX_POSIX_EXTENDED | RE_INVALID_INTERVAL_ORD | RE_NEWLINE_ALT) \ + & ~(RE_CONTEXT_INVALID_OPS | RE_DOT_NOT_NULL)) -#define RE_SYNTAX_POSIX_EGREP \ - (RE_SYNTAX_EGREP | RE_INTERVALS | RE_NO_BK_BRACES \ - | RE_INVALID_INTERVAL_ORD) +/* POSIX grep -E behavior is no longer incompatible with GNU. */ +# define RE_SYNTAX_POSIX_EGREP \ + RE_SYNTAX_EGREP /* P1003.2/D11.2, section 4.20.7.1, lines 5078ff. */ -#define RE_SYNTAX_ED RE_SYNTAX_POSIX_BASIC +# define RE_SYNTAX_ED RE_SYNTAX_POSIX_BASIC -#define RE_SYNTAX_SED RE_SYNTAX_POSIX_BASIC +# define RE_SYNTAX_SED RE_SYNTAX_POSIX_BASIC /* Syntax bits common to both basic and extended POSIX regex syntax. */ -#define _RE_SYNTAX_POSIX_COMMON \ +# define _RE_SYNTAX_POSIX_COMMON \ (RE_CHAR_CLASSES | RE_DOT_NEWLINE | RE_DOT_NOT_NULL \ | RE_INTERVALS | RE_NO_EMPTY_RANGES) -#define RE_SYNTAX_POSIX_BASIC \ +# define RE_SYNTAX_POSIX_BASIC \ (_RE_SYNTAX_POSIX_COMMON | RE_BK_PLUS_QM | RE_CONTEXT_INVALID_DUP) /* Differs from ..._POSIX_BASIC only in that RE_BK_PLUS_QM becomes RE_LIMITED_OPS, i.e., \? \+ \| are not recognized. Actually, this isn't minimal, since other operators, such as \`, aren't disabled. */ -#define RE_SYNTAX_POSIX_MINIMAL_BASIC \ +# define RE_SYNTAX_POSIX_MINIMAL_BASIC \ (_RE_SYNTAX_POSIX_COMMON | RE_LIMITED_OPS) -#define RE_SYNTAX_POSIX_EXTENDED \ +# define RE_SYNTAX_POSIX_EXTENDED \ (_RE_SYNTAX_POSIX_COMMON | RE_CONTEXT_INDEP_ANCHORS \ | RE_CONTEXT_INDEP_OPS | RE_NO_BK_BRACES \ | RE_NO_BK_PARENS | RE_NO_BK_VBAR \ @@ -248,25 +275,35 @@ extern reg_syntax_t re_syntax_options; /* Differs from ..._POSIX_EXTENDED in that RE_CONTEXT_INDEP_OPS is removed and RE_NO_BK_REFS is added. */ -#define RE_SYNTAX_POSIX_MINIMAL_EXTENDED \ +# define RE_SYNTAX_POSIX_MINIMAL_EXTENDED \ (_RE_SYNTAX_POSIX_COMMON | RE_CONTEXT_INDEP_ANCHORS \ | RE_CONTEXT_INVALID_OPS | RE_NO_BK_BRACES \ | RE_NO_BK_PARENS | RE_NO_BK_REFS \ | RE_NO_BK_VBAR | RE_UNMATCHED_RIGHT_PAREN_ORD) /* [[[end syntaxes]]] */ - -/* Maximum number of duplicates an interval can allow. Some systems - (erroneously) define this in other header files, but we want our + +/* Maximum number of duplicates an interval can allow. POSIX-conforming + systems might define this in <limits.h>, but we want our value, so remove any previous define. */ +# ifdef _REGEX_INCLUDE_LIMITS_H +# include <limits.h> +# endif # ifdef RE_DUP_MAX # undef RE_DUP_MAX # endif -/* If sizeof(int) == 2, then ((1 << 15) - 1) overflows. */ + +/* RE_DUP_MAX is 2**15 - 1 because an earlier implementation stored + the counter as a 2-byte signed integer. This is no longer true, so + RE_DUP_MAX could be increased to (INT_MAX / 10 - 1), or to + ((SIZE_MAX - 9) / 10) if _REGEX_LARGE_OFFSETS is defined. + However, there would be a huge performance problem if someone + actually used a pattern like a\{214748363\}, so RE_DUP_MAX retains + its historical value. */ # define RE_DUP_MAX (0x7fff) #endif -/* POSIX `cflags' bits (i.e., information for `regcomp'). */ +/* POSIX 'cflags' bits (i.e., information for 'regcomp'). */ /* If this bit is set, then use extended regular expression syntax. If not set, then use basic regular expression syntax. */ @@ -274,19 +311,19 @@ extern reg_syntax_t re_syntax_options; /* If this bit is set, then ignore case when matching. If not set, then case is significant. */ -#define REG_ICASE (REG_EXTENDED << 1) +#define REG_ICASE (1 << 1) /* If this bit is set, then anchors do not match at newline characters in the string. If not set, then anchors do match at newlines. */ -#define REG_NEWLINE (REG_ICASE << 1) +#define REG_NEWLINE (1 << 2) /* If this bit is set, then report only success or fail in regexec. If not set, then returns differ between not matching and errors. */ -#define REG_NOSUB (REG_NEWLINE << 1) +#define REG_NOSUB (1 << 3) -/* POSIX `eflags' bits (i.e., information for regexec). */ +/* POSIX 'eflags' bits (i.e., information for regexec). */ /* If this bit is set, then the beginning-of-line operator doesn't match the beginning of the string (presumably because it's not the @@ -304,41 +341,60 @@ extern reg_syntax_t re_syntax_options; /* If any error codes are removed, changed, or added, update the - `re_error_msg' table in regex.c. */ + '__re_error_msgid' table in regcomp.c. */ + typedef enum { -#if defined _XOPEN_SOURCE || defined __USE_XOPEN2K - REG_ENOSYS = -1, /* This will never happen for this implementation. */ -#endif - - REG_NOERROR = 0, /* Success. */ - REG_NOMATCH, /* Didn't find a match (for regexec). */ + _REG_ENOSYS = -1, /* This will never happen for this implementation. */ + _REG_NOERROR = 0, /* Success. */ + _REG_NOMATCH, /* Didn't find a match (for regexec). */ /* POSIX regcomp return error codes. (In the order listed in the standard.) */ - REG_BADPAT, /* Invalid pattern. */ - REG_ECOLLATE, /* Invalid collating element. */ - REG_ECTYPE, /* Invalid character class name. */ - REG_EESCAPE, /* Trailing backslash. */ - REG_ESUBREG, /* Invalid back reference. */ - REG_EBRACK, /* Unmatched left bracket. */ - REG_EPAREN, /* Parenthesis imbalance. */ - REG_EBRACE, /* Unmatched \{. */ - REG_BADBR, /* Invalid contents of \{\}. */ - REG_ERANGE, /* Invalid range end. */ - REG_ESPACE, /* Ran out of memory. */ - REG_BADRPT, /* No preceding re for repetition op. */ + _REG_BADPAT, /* Invalid pattern. */ + _REG_ECOLLATE, /* Invalid collating element. */ + _REG_ECTYPE, /* Invalid character class name. */ + _REG_EESCAPE, /* Trailing backslash. */ + _REG_ESUBREG, /* Invalid back reference. */ + _REG_EBRACK, /* Unmatched left bracket. */ + _REG_EPAREN, /* Parenthesis imbalance. */ + _REG_EBRACE, /* Unmatched \{. */ + _REG_BADBR, /* Invalid contents of \{\}. */ + _REG_ERANGE, /* Invalid range end. */ + _REG_ESPACE, /* Ran out of memory. */ + _REG_BADRPT, /* No preceding re for repetition op. */ /* Error codes we've added. */ - REG_EEND, /* Premature end. */ - REG_ESIZE, /* Compiled pattern bigger than 2^16 bytes. */ - REG_ERPAREN /* Unmatched ) or \); not returned from regcomp. */ + _REG_EEND, /* Premature end. */ + _REG_ESIZE, /* Too large (e.g., repeat count too large). */ + _REG_ERPAREN /* Unmatched ) or \); not returned from regcomp. */ } reg_errcode_t; + +#if defined _XOPEN_SOURCE || defined __USE_XOPEN2K +# define REG_ENOSYS _REG_ENOSYS +#endif +#define REG_NOERROR _REG_NOERROR +#define REG_NOMATCH _REG_NOMATCH +#define REG_BADPAT _REG_BADPAT +#define REG_ECOLLATE _REG_ECOLLATE +#define REG_ECTYPE _REG_ECTYPE +#define REG_EESCAPE _REG_EESCAPE +#define REG_ESUBREG _REG_ESUBREG +#define REG_EBRACK _REG_EBRACK +#define REG_EPAREN _REG_EPAREN +#define REG_EBRACE _REG_EBRACE +#define REG_BADBR _REG_BADBR +#define REG_ERANGE _REG_ERANGE +#define REG_ESPACE _REG_ESPACE +#define REG_BADRPT _REG_BADRPT +#define REG_EEND _REG_EEND +#define REG_ESIZE _REG_ESIZE +#define REG_ERPAREN _REG_ERPAREN /* This data structure represents a compiled pattern. Before calling - the pattern compiler, the fields `buffer', `allocated', `fastmap', - and `translate' can be set. After the pattern has been compiled, - the fields `re_nsub', `not_bol' and `not_eol' are available. All + the pattern compiler, the fields 'buffer', 'allocated', 'fastmap', + and 'translate' can be set. After the pattern has been compiled, + the fields 're_nsub', 'not_bol' and 'not_eol' are available. All other fields are private to the regex routines. */ #ifndef RE_TRANSLATE_TYPE @@ -356,16 +412,15 @@ typedef enum struct re_pattern_buffer { - /* Space that holds the compiled pattern. It is declared as - `unsigned char *' because its elements are sometimes used as - array indexes. */ - unsigned char *__REPB_PREFIX(buffer); + /* Space that holds the compiled pattern. The type + 'struct re_dfa_t' is private and is not declared here. */ + struct re_dfa_t *__REPB_PREFIX(buffer); - /* Number of bytes to which `buffer' points. */ - unsigned long int __REPB_PREFIX(allocated); + /* Number of bytes to which 'buffer' points. */ + __re_long_size_t __REPB_PREFIX(allocated); - /* Number of bytes actually used in `buffer'. */ - unsigned long int __REPB_PREFIX(used); + /* Number of bytes actually used in 'buffer'. */ + __re_long_size_t __REPB_PREFIX(used); /* Syntax setting with which the pattern was compiled. */ reg_syntax_t __REPB_PREFIX(syntax); @@ -385,13 +440,13 @@ struct re_pattern_buffer size_t re_nsub; /* Zero if this pattern cannot match the empty string, one else. - Well, in truth it's used only in `re_search_2', to see whether or + Well, in truth it's used only in 're_search_2', to see whether or not we should use the fastmap, so we don't set this absolutely - perfectly; see `re_compile_fastmap' (the `duplicate' case). */ + perfectly; see 're_compile_fastmap' (the "duplicate" case). */ unsigned __REPB_PREFIX(can_be_null) : 1; - /* If REGS_UNALLOCATED, allocate space in the `regs' structure - for `max (RE_NREGS, re_nsub + 1)' groups. + /* If REGS_UNALLOCATED, allocate space in the 'regs' structure + for 'max (RE_NREGS, re_nsub + 1)' groups. If REGS_REALLOCATE, reallocate space if necessary. If REGS_FIXED, use what's there. */ #ifdef __USE_GNU @@ -401,11 +456,11 @@ struct re_pattern_buffer #endif unsigned __REPB_PREFIX(regs_allocated) : 2; - /* Set to zero when `regex_compile' compiles a pattern; set to one - by `re_compile_fastmap' if it updates the fastmap. */ + /* Set to zero when 're_compile_pattern' compiles a pattern; set to + one by 're_compile_fastmap' if it updates the fastmap. */ unsigned __REPB_PREFIX(fastmap_accurate) : 1; - /* If set, `re_match_2' does not return information about + /* If set, 're_match_2' does not return information about subexpressions. */ unsigned __REPB_PREFIX(no_sub) : 1; @@ -423,7 +478,17 @@ struct re_pattern_buffer typedef struct re_pattern_buffer regex_t; /* Type for byte offsets within the string. POSIX mandates this. */ +#ifdef _REGEX_LARGE_OFFSETS +/* POSIX 1003.1-2008 requires that regoff_t be at least as wide as + ptrdiff_t and ssize_t. We don't know of any hosts where ptrdiff_t + is wider than ssize_t, so ssize_t is safe. ptrdiff_t is not + visible here, so use ssize_t. */ +typedef ssize_t regoff_t; +#else +/* The traditional GNU regex implementation mishandles strings longer + than INT_MAX. */ typedef int regoff_t; +#endif #ifdef __USE_GNU @@ -431,15 +496,15 @@ typedef int regoff_t; regex.texinfo for a full description of what registers match. */ struct re_registers { - unsigned num_regs; + __re_size_t num_regs; regoff_t *start; regoff_t *end; }; -/* If `regs_allocated' is REGS_UNALLOCATED in the pattern buffer, - `re_match_2' returns information about at least this many registers - the first time a `regs' structure is passed. */ +/* If 'regs_allocated' is REGS_UNALLOCATED in the pattern buffer, + 're_match_2' returns information about at least this many registers + the first time a 'regs' structure is passed. */ # ifndef RE_NREGS # define RE_NREGS 30 # endif @@ -447,7 +512,7 @@ struct re_registers /* POSIX specification for registers. Aside from the different names than - `re_registers', POSIX uses an array of structures, instead of a + 're_registers', POSIX uses an array of structures, instead of a structure of arrays. */ typedef struct { @@ -459,17 +524,17 @@ typedef struct #ifdef __USE_GNU /* Sets the current default syntax to SYNTAX, and return the old syntax. - You can also simply assign to the `re_syntax_options' variable. */ + You can also simply assign to the 're_syntax_options' variable. */ extern reg_syntax_t re_set_syntax (reg_syntax_t __syntax); /* Compile the regular expression PATTERN, with length LENGTH - and syntax given by the global `re_syntax_options', into the buffer + and syntax given by the global 're_syntax_options', into the buffer BUFFER. Return NULL if successful, and an error string if not. - To free the allocated storage, you must call `regfree' on BUFFER. - Note that the translate table must either have been initialised by - `regcomp', with a malloc'ed value, or set to NULL before calling - `regfree'. */ + To free the allocated storage, you must call 'regfree' on BUFFER. + Note that the translate table must either have been initialized by + 'regcomp', with a malloc'ed value, or set to NULL before calling + 'regfree'. */ extern const char *re_compile_pattern (const char *__pattern, size_t __length, struct re_pattern_buffer *__buffer); @@ -485,47 +550,52 @@ extern int re_compile_fastmap (struct re_pattern_buffer *__buffer); characters. Return the starting position of the match, -1 for no match, or -2 for an internal error. Also return register information in REGS (if REGS and BUFFER->no_sub are nonzero). */ -extern int re_search (struct re_pattern_buffer *__buffer, const char *__string, - int __length, int __start, int __range, - struct re_registers *__regs); +extern regoff_t re_search (struct re_pattern_buffer *__buffer, + const char *__String, regoff_t __length, + regoff_t __start, regoff_t __range, + struct re_registers *__regs); -/* Like `re_search', but search in the concatenation of STRING1 and +/* Like 're_search', but search in the concatenation of STRING1 and STRING2. Also, stop searching at index START + STOP. */ -extern int re_search_2 (struct re_pattern_buffer *__buffer, - const char *__string1, int __length1, - const char *__string2, int __length2, int __start, - int __range, struct re_registers *__regs, int __stop); +extern regoff_t re_search_2 (struct re_pattern_buffer *__buffer, + const char *__string1, regoff_t __length1, + const char *__string2, regoff_t __length2, + regoff_t __start, regoff_t __range, + struct re_registers *__regs, + regoff_t __stop); -/* Like `re_search', but return how many characters in STRING the regexp +/* Like 're_search', but return how many characters in STRING the regexp in BUFFER matched, starting at position START. */ -extern int re_match (struct re_pattern_buffer *__buffer, const char *__string, - int __length, int __start, struct re_registers *__regs); +extern regoff_t re_match (struct re_pattern_buffer *__buffer, + const char *__String, regoff_t __length, + regoff_t __start, struct re_registers *__regs); -/* Relates to `re_match' as `re_search_2' relates to `re_search'. */ -extern int re_match_2 (struct re_pattern_buffer *__buffer, - const char *__string1, int __length1, - const char *__string2, int __length2, int __start, - struct re_registers *__regs, int __stop); +/* Relates to 're_match' as 're_search_2' relates to 're_search'. */ +extern regoff_t re_match_2 (struct re_pattern_buffer *__buffer, + const char *__string1, regoff_t __length1, + const char *__string2, regoff_t __length2, + regoff_t __start, struct re_registers *__regs, + regoff_t __stop); /* Set REGS to hold NUM_REGS registers, storing them in STARTS and ENDS. Subsequent matches using BUFFER and REGS will use this memory for recording register information. STARTS and ENDS must be - allocated with malloc, and must each be at least `NUM_REGS * sizeof + allocated with malloc, and must each be at least 'NUM_REGS * sizeof (regoff_t)' bytes long. If NUM_REGS == 0, then subsequent matches should allocate their own register data. Unless this function is called, the first search or match using - PATTERN_BUFFER will allocate its own register data, without + BUFFER will allocate its own register data, without freeing the old data. */ extern void re_set_registers (struct re_pattern_buffer *__buffer, struct re_registers *__regs, - unsigned int __num_regs, + __re_size_t __num_regs, regoff_t *__starts, regoff_t *__ends); #endif /* Use GNU */ @@ -537,39 +607,46 @@ extern int re_exec (const char *); # endif #endif -/* GCC 2.95 and later have "__restrict"; C99 compilers have - "restrict", and "configure" may have defined "restrict". */ -#ifndef __restrict -# if ! (2 < __GNUC__ || (2 == __GNUC__ && 95 <= __GNUC_MINOR__)) -# if defined restrict || 199901L <= __STDC_VERSION__ -# define __restrict restrict -# else -# define __restrict -# endif +/* For plain 'restrict', use glibc's __restrict if defined. + Otherwise, GCC 2.95 and later have "__restrict"; C99 compilers have + "restrict", and "configure" may have defined "restrict". + Other compilers use __restrict, __restrict__, and _Restrict, and + 'configure' might #define 'restrict' to those words, so pick a + different name. */ +#ifndef _Restrict_ +# if defined __restrict || 2 < __GNUC__ + (95 <= __GNUC_MINOR__) +# define _Restrict_ __restrict +# elif 199901L <= __STDC_VERSION__ || defined restrict +# define _Restrict_ restrict +# else +# define _Restrict_ # endif #endif -/* gcc 3.1 and up support the [restrict] syntax. */ -#ifndef __restrict_arr -# if (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 1)) \ - && !defined __GNUG__ -# define __restrict_arr __restrict +/* For [restrict], use glibc's __restrict_arr if available. + Otherwise, GCC 3.1 (not in C++ mode) and C99 support [restrict]. */ +#ifndef _Restrict_arr_ +# ifdef __restrict_arr +# define _Restrict_arr_ __restrict_arr +# elif ((199901L <= __STDC_VERSION__ || 3 < __GNUC__ + (1 <= __GNUC_MINOR__)) \ + && !defined __GNUG__) +# define _Restrict_arr_ _Restrict_ # else -# define __restrict_arr +# define _Restrict_arr_ # endif #endif /* POSIX compatibility. */ -extern int regcomp (regex_t *__restrict __preg, - const char *__restrict __pattern, +extern int regcomp (regex_t *_Restrict_ __preg, + const char *_Restrict_ __pattern, int __cflags); -extern int regexec (const regex_t *__restrict __preg, - const char *__restrict __string, size_t __nmatch, - regmatch_t __pmatch[__restrict_arr], +extern int regexec (const regex_t *_Restrict_ __preg, + const char *_Restrict_ __String, size_t __nmatch, + regmatch_t __pmatch[_Restrict_arr_], int __eflags); -extern size_t regerror (int __errcode, const regex_t *__restrict __preg, - char *__restrict __errbuf, size_t __errbuf_size); +extern size_t regerror (int __errcode, const regex_t *_Restrict_ __preg, + char *_Restrict_ __errbuf, size_t __errbuf_size); extern void regfree (regex_t *__preg); diff --git a/posix/regex_internal.c b/posix/regex_internal.c index 9062083..7f0083b 100644 --- a/posix/regex_internal.c +++ b/posix/regex_internal.c @@ -15,19 +15,29 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ -static void re_string_construct_common (const char *str, int len, +static void re_string_construct_common (const char *str, Idx len, re_string_t *pstr, - RE_TRANSLATE_TYPE trans, int icase, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa); static re_dfastate_t *create_ci_newstate (const re_dfa_t *dfa, const re_node_set *nodes, - unsigned int hash); + re_hashval_t hash); static re_dfastate_t *create_cd_newstate (const re_dfa_t *dfa, const re_node_set *nodes, unsigned int context, - unsigned int hash); + re_hashval_t hash); +static reg_errcode_t re_string_realloc_buffers (re_string_t *pstr, + Idx new_buf_len); +#ifdef RE_ENABLE_I18N +static void build_wcs_buffer (re_string_t *pstr); +static reg_errcode_t build_wcs_upper_buffer (re_string_t *pstr); +#endif /* RE_ENABLE_I18N */ +static void build_upper_buffer (re_string_t *pstr); +static void re_string_translate_buffer (re_string_t *pstr); +static unsigned int re_string_context_at (const re_string_t *input, Idx idx, + int eflags) __attribute__ ((pure)); /* Functions for string operation. */ @@ -36,11 +46,11 @@ static re_dfastate_t *create_cd_newstate (const re_dfa_t *dfa, static reg_errcode_t __attribute_warn_unused_result__ -re_string_allocate (re_string_t *pstr, const char *str, int len, int init_len, - RE_TRANSLATE_TYPE trans, int icase, const re_dfa_t *dfa) +re_string_allocate (re_string_t *pstr, const char *str, Idx len, Idx init_len, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa) { reg_errcode_t ret; - int init_buf_len; + Idx init_buf_len; /* Ensure at least one character fits into the buffers. */ if (init_len < dfa->mb_cur_max) @@ -64,8 +74,8 @@ re_string_allocate (re_string_t *pstr, const char *str, int len, int init_len, static reg_errcode_t __attribute_warn_unused_result__ -re_string_construct (re_string_t *pstr, const char *str, int len, - RE_TRANSLATE_TYPE trans, int icase, const re_dfa_t *dfa) +re_string_construct (re_string_t *pstr, const char *str, Idx len, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa) { reg_errcode_t ret; memset (pstr, '\0', sizeof (re_string_t)); @@ -127,7 +137,7 @@ re_string_construct (re_string_t *pstr, const char *str, int len, static reg_errcode_t __attribute_warn_unused_result__ -re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) +re_string_realloc_buffers (re_string_t *pstr, Idx new_buf_len) { #ifdef RE_ENABLE_I18N if (pstr->mb_cur_max > 1) @@ -135,8 +145,8 @@ re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) wint_t *new_wcs; /* Avoid overflow in realloc. */ - const size_t max_object_size = MAX (sizeof (wint_t), sizeof (int)); - if (BE (SIZE_MAX / max_object_size < new_buf_len, 0)) + const size_t max_object_size = MAX (sizeof (wint_t), sizeof (Idx)); + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) < new_buf_len, 0)) return REG_ESPACE; new_wcs = re_realloc (pstr->wcs, wint_t, new_buf_len); @@ -145,7 +155,7 @@ re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) pstr->wcs = new_wcs; if (pstr->offsets != NULL) { - int *new_offsets = re_realloc (pstr->offsets, int, new_buf_len); + Idx *new_offsets = re_realloc (pstr->offsets, Idx, new_buf_len); if (BE (new_offsets == NULL, 0)) return REG_ESPACE; pstr->offsets = new_offsets; @@ -166,15 +176,15 @@ re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) static void -re_string_construct_common (const char *str, int len, re_string_t *pstr, - RE_TRANSLATE_TYPE trans, int icase, +re_string_construct_common (const char *str, Idx len, re_string_t *pstr, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa) { pstr->raw_mbs = (const unsigned char *) str; pstr->len = len; pstr->raw_len = len; pstr->trans = trans; - pstr->icase = icase ? 1 : 0; + pstr->icase = icase; pstr->mbs_allocated = (trans != NULL || icase); pstr->mb_cur_max = dfa->mb_cur_max; pstr->is_utf8 = dfa->is_utf8; @@ -206,7 +216,7 @@ build_wcs_buffer (re_string_t *pstr) unsigned char buf[64]; #endif mbstate_t prev_st; - int byte_idx, end_idx, remain_len; + Idx byte_idx, end_idx, remain_len; size_t mbclen; /* Build the buffers from pstr->valid_len to either pstr->len or @@ -269,7 +279,7 @@ __attribute_warn_unused_result__ build_wcs_upper_buffer (re_string_t *pstr) { mbstate_t prev_st; - int src_idx, byte_idx, end_idx, remain_len; + Idx src_idx, byte_idx, end_idx, remain_len; size_t mbclen; #ifdef _LIBC char buf[MB_LEN_MAX]; @@ -307,14 +317,13 @@ build_wcs_upper_buffer (re_string_t *pstr) mbclen = __mbrtowc (&wc, ((const char *) pstr->raw_mbs + pstr->raw_mbs_idx + byte_idx), remain_len, &pstr->cur_state); - if (BE (mbclen + 2 > 2, 1)) + if (BE (mbclen < (size_t) -2, 1)) { - wchar_t wcu = wc; - if (__iswlower (wc)) + wchar_t wcu = __towupper (wc); + if (wcu != wc) { size_t mbcdlen; - wcu = __towupper (wc); mbcdlen = __wcrtomb (buf, wcu, &prev_st); if (BE (mbclen == mbcdlen, 1)) memcpy (pstr->mbs + byte_idx, buf, mbclen); @@ -377,14 +386,13 @@ build_wcs_upper_buffer (re_string_t *pstr) else p = (const char *) pstr->raw_mbs + pstr->raw_mbs_idx + src_idx; mbclen = __mbrtowc (&wc, p, remain_len, &pstr->cur_state); - if (BE (mbclen + 2 > 2, 1)) + if (BE (mbclen < (size_t) -2, 1)) { - wchar_t wcu = wc; - if (__iswlower (wc)) + wchar_t wcu = __towupper (wc); + if (wcu != wc) { size_t mbcdlen; - wcu = __towupper (wc); mbcdlen = __wcrtomb ((char *) buf, wcu, &prev_st); if (BE (mbclen == mbcdlen, 1)) memcpy (pstr->mbs + byte_idx, buf, mbclen); @@ -400,7 +408,7 @@ build_wcs_upper_buffer (re_string_t *pstr) if (pstr->offsets == NULL) { - pstr->offsets = re_malloc (int, pstr->bufs_len); + pstr->offsets = re_malloc (Idx, pstr->bufs_len); if (pstr->offsets == NULL) return REG_ESPACE; @@ -483,11 +491,11 @@ build_wcs_upper_buffer (re_string_t *pstr) /* Skip characters until the index becomes greater than NEW_RAW_IDX. Return the index. */ -static int -re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) +static Idx +re_string_skip_chars (re_string_t *pstr, Idx new_raw_idx, wint_t *last_wc) { mbstate_t prev_st; - int rawbuf_idx; + Idx rawbuf_idx; size_t mbclen; wint_t wc = WEOF; @@ -496,11 +504,11 @@ re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) rawbuf_idx < new_raw_idx;) { wchar_t wc2; - int remain_len = pstr->raw_len - rawbuf_idx; + Idx remain_len = pstr->raw_len - rawbuf_idx; prev_st = pstr->cur_state; mbclen = __mbrtowc (&wc2, (const char *) pstr->raw_mbs + rawbuf_idx, remain_len, &pstr->cur_state); - if (BE ((ssize_t) mbclen <= 0, 0)) + if (BE (mbclen == (size_t) -2 || mbclen == (size_t) -1 || mbclen == 0, 0)) { /* We treat these cases as a single byte character. */ if (mbclen == 0 || remain_len == 0) @@ -511,7 +519,7 @@ re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) pstr->cur_state = prev_st; } else - wc = (wint_t) wc2; + wc = wc2; /* Then proceed the next character. */ rawbuf_idx += mbclen; } @@ -526,7 +534,7 @@ re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) static void build_upper_buffer (re_string_t *pstr) { - int char_idx, end_idx; + Idx char_idx, end_idx; end_idx = (pstr->bufs_len > pstr->len) ? pstr->len : pstr->bufs_len; for (char_idx = pstr->valid_len; char_idx < end_idx; ++char_idx) @@ -534,10 +542,7 @@ build_upper_buffer (re_string_t *pstr) int ch = pstr->raw_mbs[pstr->raw_mbs_idx + char_idx]; if (BE (pstr->trans != NULL, 0)) ch = pstr->trans[ch]; - if (islower (ch)) - pstr->mbs[char_idx] = toupper (ch); - else - pstr->mbs[char_idx] = ch; + pstr->mbs[char_idx] = toupper (ch); } pstr->valid_len = char_idx; pstr->valid_raw_len = char_idx; @@ -548,7 +553,7 @@ build_upper_buffer (re_string_t *pstr) static void re_string_translate_buffer (re_string_t *pstr) { - int buf_idx, end_idx; + Idx buf_idx, end_idx; end_idx = (pstr->bufs_len > pstr->len) ? pstr->len : pstr->bufs_len; for (buf_idx = pstr->valid_len; buf_idx < end_idx; ++buf_idx) @@ -567,10 +572,13 @@ re_string_translate_buffer (re_string_t *pstr) static reg_errcode_t __attribute_warn_unused_result__ -re_string_reconstruct (re_string_t *pstr, int idx, int eflags) +re_string_reconstruct (re_string_t *pstr, Idx idx, int eflags) { - int offset = idx - pstr->raw_mbs_idx; - if (BE (offset < 0, 0)) + Idx offset; + + if (BE (pstr->raw_mbs_idx <= idx, 0)) + offset = idx - pstr->raw_mbs_idx; + else { /* Reset buffer. */ #ifdef RE_ENABLE_I18N @@ -599,7 +607,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) #ifdef RE_ENABLE_I18N if (BE (pstr->offsets_needed, 0)) { - int low = 0, high = pstr->valid_len, mid; + Idx low = 0, high = pstr->valid_len, mid; do { mid = (high + low) / 2; @@ -683,7 +691,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) { #ifdef RE_ENABLE_I18N /* No, skip all characters until IDX. */ - int prev_valid_len = pstr->valid_len; + Idx prev_valid_len = pstr->valid_len; if (BE (pstr->offsets_needed, 0)) { @@ -696,7 +704,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) #ifdef RE_ENABLE_I18N if (pstr->mb_cur_max > 1) { - int wcs_idx; + Idx wcs_idx; wint_t wc = WEOF; if (pstr->is_utf8) @@ -726,7 +734,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) { mbstate_t cur_state; wchar_t wc2; - int mlen = raw + pstr->len - p; + Idx mlen = raw + pstr->len - p; unsigned char buf[6]; size_t mbclen; @@ -826,10 +834,11 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) } static unsigned char -__attribute ((pure)) -re_string_peek_byte_case (const re_string_t *pstr, int idx) +__attribute__ ((pure)) +re_string_peek_byte_case (const re_string_t *pstr, Idx idx) { - int ch, off; + int ch; + Idx off; /* Handle the common (easiest) cases first. */ if (BE (!pstr->mbs_allocated, 1)) @@ -870,7 +879,8 @@ re_string_fetch_byte_case (re_string_t *pstr) #ifdef RE_ENABLE_I18N if (pstr->offsets_needed) { - int off, ch; + Idx off; + int ch; /* For tr_TR.UTF-8 [[:islower:]] there is [[: CAPITAL LETTER I WITH DOT lower:]] in mbs. Skip @@ -911,7 +921,7 @@ re_string_destruct (re_string_t *pstr) /* Return the context at IDX in INPUT. */ static unsigned int -re_string_context_at (const re_string_t *input, int idx, int eflags) +re_string_context_at (const re_string_t *input, Idx idx, int eflags) { int c; if (BE (idx < 0, 0)) @@ -925,7 +935,7 @@ re_string_context_at (const re_string_t *input, int idx, int eflags) if (input->mb_cur_max > 1) { wint_t wc; - int wc_idx = idx; + Idx wc_idx = idx; while(input->wcs[wc_idx] == WEOF) { #if defined DEBUG && DEBUG @@ -956,23 +966,23 @@ re_string_context_at (const re_string_t *input, int idx, int eflags) static reg_errcode_t __attribute_warn_unused_result__ -re_node_set_alloc (re_node_set *set, int size) +re_node_set_alloc (re_node_set *set, Idx size) { set->alloc = size; set->nelem = 0; - set->elems = re_malloc (int, size); - if (BE (set->elems == NULL, 0)) + set->elems = re_malloc (Idx, size); + if (BE (set->elems == NULL, 0) && (MALLOC_0_IS_NONNULL || size != 0)) return REG_ESPACE; return REG_NOERROR; } static reg_errcode_t __attribute_warn_unused_result__ -re_node_set_init_1 (re_node_set *set, int elem) +re_node_set_init_1 (re_node_set *set, Idx elem) { set->alloc = 1; set->nelem = 1; - set->elems = re_malloc (int, 1); + set->elems = re_malloc (Idx, 1); if (BE (set->elems == NULL, 0)) { set->alloc = set->nelem = 0; @@ -984,10 +994,10 @@ re_node_set_init_1 (re_node_set *set, int elem) static reg_errcode_t __attribute_warn_unused_result__ -re_node_set_init_2 (re_node_set *set, int elem1, int elem2) +re_node_set_init_2 (re_node_set *set, Idx elem1, Idx elem2) { set->alloc = 2; - set->elems = re_malloc (int, 2); + set->elems = re_malloc (Idx, 2); if (BE (set->elems == NULL, 0)) return REG_ESPACE; if (elem1 == elem2) @@ -1020,13 +1030,13 @@ re_node_set_init_copy (re_node_set *dest, const re_node_set *src) if (src->nelem > 0) { dest->alloc = dest->nelem; - dest->elems = re_malloc (int, dest->alloc); + dest->elems = re_malloc (Idx, dest->alloc); if (BE (dest->elems == NULL, 0)) { dest->alloc = dest->nelem = 0; return REG_ESPACE; } - memcpy (dest->elems, src->elems, src->nelem * sizeof (int)); + memcpy (dest->elems, src->elems, src->nelem * sizeof (Idx)); } else re_node_set_init_empty (dest); @@ -1042,7 +1052,7 @@ __attribute_warn_unused_result__ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, const re_node_set *src2) { - int i1, i2, is, id, delta, sbase; + Idx i1, i2, is, id, delta, sbase; if (src1->nelem == 0 || src2->nelem == 0) return REG_NOERROR; @@ -1050,8 +1060,8 @@ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, conservative estimate. */ if (src1->nelem + src2->nelem + dest->nelem > dest->alloc) { - int new_alloc = src1->nelem + src2->nelem + dest->alloc; - int *new_elems = re_realloc (dest->elems, int, new_alloc); + Idx new_alloc = src1->nelem + src2->nelem + dest->alloc; + Idx *new_elems = re_realloc (dest->elems, Idx, new_alloc); if (BE (new_elems == NULL, 0)) return REG_ESPACE; dest->elems = new_elems; @@ -1073,7 +1083,7 @@ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, --id; if (id < 0 || dest->elems[id] != src1->elems[i1]) - dest->elems[--sbase] = src1->elems[i1]; + dest->elems[--sbase] = src1->elems[i1]; if (--i1 < 0 || --i2 < 0) break; @@ -1120,7 +1130,7 @@ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, } /* Copy remaining SRC elements. */ - memcpy (dest->elems, dest->elems + sbase, delta * sizeof (int)); + memcpy (dest->elems, dest->elems + sbase, delta * sizeof (Idx)); return REG_NOERROR; } @@ -1133,11 +1143,11 @@ __attribute_warn_unused_result__ re_node_set_init_union (re_node_set *dest, const re_node_set *src1, const re_node_set *src2) { - int i1, i2, id; + Idx i1, i2, id; if (src1 != NULL && src1->nelem > 0 && src2 != NULL && src2->nelem > 0) { dest->alloc = src1->nelem + src2->nelem; - dest->elems = re_malloc (int, dest->alloc); + dest->elems = re_malloc (Idx, dest->alloc); if (BE (dest->elems == NULL, 0)) return REG_ESPACE; } @@ -1165,13 +1175,13 @@ re_node_set_init_union (re_node_set *dest, const re_node_set *src1, if (i1 < src1->nelem) { memcpy (dest->elems + id, src1->elems + i1, - (src1->nelem - i1) * sizeof (int)); + (src1->nelem - i1) * sizeof (Idx)); id += src1->nelem - i1; } else if (i2 < src2->nelem) { memcpy (dest->elems + id, src2->elems + i2, - (src2->nelem - i2) * sizeof (int)); + (src2->nelem - i2) * sizeof (Idx)); id += src2->nelem - i2; } dest->nelem = id; @@ -1185,13 +1195,13 @@ static reg_errcode_t __attribute_warn_unused_result__ re_node_set_merge (re_node_set *dest, const re_node_set *src) { - int is, id, sbase, delta; + Idx is, id, sbase, delta; if (src == NULL || src->nelem == 0) return REG_NOERROR; if (dest->alloc < 2 * src->nelem + dest->nelem) { - int new_alloc = 2 * (src->nelem + dest->alloc); - int *new_buffer = re_realloc (dest->elems, int, new_alloc); + Idx new_alloc = 2 * (src->nelem + dest->alloc); + Idx *new_buffer = re_realloc (dest->elems, Idx, new_alloc); if (BE (new_buffer == NULL, 0)) return REG_ESPACE; dest->elems = new_buffer; @@ -1201,7 +1211,7 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) if (BE (dest->nelem == 0, 0)) { dest->nelem = src->nelem; - memcpy (dest->elems, src->elems, src->nelem * sizeof (int)); + memcpy (dest->elems, src->elems, src->nelem * sizeof (Idx)); return REG_NOERROR; } @@ -1222,7 +1232,7 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) { /* If DEST is exhausted, the remaining items of SRC must be unique. */ sbase -= is + 1; - memcpy (dest->elems + sbase, src->elems, (is + 1) * sizeof (int)); + memcpy (dest->elems + sbase, src->elems, (is + 1) * sizeof (Idx)); } id = dest->nelem - 1; @@ -1251,7 +1261,7 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) { /* Copy remaining SRC elements. */ memcpy (dest->elems, dest->elems + sbase, - delta * sizeof (int)); + delta * sizeof (Idx)); break; } } @@ -1262,38 +1272,33 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) /* Insert the new element ELEM to the re_node_set* SET. SET should not already have ELEM. - return -1 if an error is occured, return 1 otherwise. */ + Return true if successful. */ -static int +static bool __attribute_warn_unused_result__ -re_node_set_insert (re_node_set *set, int elem) +re_node_set_insert (re_node_set *set, Idx elem) { - int idx; + Idx idx; /* In case the set is empty. */ if (set->alloc == 0) - { - if (BE (re_node_set_init_1 (set, elem) == REG_NOERROR, 1)) - return 1; - else - return -1; - } + return BE (re_node_set_init_1 (set, elem) == REG_NOERROR, 1); if (BE (set->nelem, 0) == 0) { /* We already guaranteed above that set->alloc != 0. */ set->elems[0] = elem; ++set->nelem; - return 1; + return true; } /* Realloc if we need. */ if (set->alloc == set->nelem) { - int *new_elems; + Idx *new_elems; set->alloc = set->alloc * 2; - new_elems = re_realloc (set->elems, int, set->alloc); + new_elems = re_realloc (set->elems, Idx, set->alloc); if (BE (new_elems == NULL, 0)) - return -1; + return false; set->elems = new_elems; } @@ -1314,56 +1319,56 @@ re_node_set_insert (re_node_set *set, int elem) /* Insert the new element. */ set->elems[idx] = elem; ++set->nelem; - return 1; + return true; } /* Insert the new element ELEM to the re_node_set* SET. SET should not already have any element greater than or equal to ELEM. - Return -1 if an error is occured, return 1 otherwise. */ + Return true if successful. */ -static int +static bool __attribute_warn_unused_result__ -re_node_set_insert_last (re_node_set *set, int elem) +re_node_set_insert_last (re_node_set *set, Idx elem) { /* Realloc if we need. */ if (set->alloc == set->nelem) { - int *new_elems; + Idx *new_elems; set->alloc = (set->alloc + 1) * 2; - new_elems = re_realloc (set->elems, int, set->alloc); + new_elems = re_realloc (set->elems, Idx, set->alloc); if (BE (new_elems == NULL, 0)) - return -1; + return false; set->elems = new_elems; } /* Insert the new element. */ set->elems[set->nelem++] = elem; - return 1; + return true; } /* Compare two node sets SET1 and SET2. - return 1 if SET1 and SET2 are equivalent, return 0 otherwise. */ + Return true if SET1 and SET2 are equivalent. */ -static int -__attribute ((pure)) +static bool +__attribute__ ((pure)) re_node_set_compare (const re_node_set *set1, const re_node_set *set2) { - int i; + Idx i; if (set1 == NULL || set2 == NULL || set1->nelem != set2->nelem) - return 0; + return false; for (i = set1->nelem ; --i >= 0 ; ) if (set1->elems[i] != set2->elems[i]) - return 0; - return 1; + return false; + return true; } /* Return (idx + 1) if SET contains the element ELEM, return 0 otherwise. */ -static int -__attribute ((pure)) -re_node_set_contains (const re_node_set *set, int elem) +static Idx +__attribute__ ((pure)) +re_node_set_contains (const re_node_set *set, Idx elem) { - unsigned int idx, right, mid; + __re_size_t idx, right, mid; if (set->nelem <= 0) return 0; @@ -1382,7 +1387,7 @@ re_node_set_contains (const re_node_set *set, int elem) } static void -re_node_set_remove_at (re_node_set *set, int idx) +re_node_set_remove_at (re_node_set *set, Idx idx) { if (idx < 0 || idx >= set->nelem) return; @@ -1393,37 +1398,42 @@ re_node_set_remove_at (re_node_set *set, int idx) /* Add the token TOKEN to dfa->nodes, and return the index of the token. - Or return -1, if an error will be occured. */ + Or return -1 if an error occurred. */ -static int +static Idx re_dfa_add_node (re_dfa_t *dfa, re_token_t token) { - int type = token.type; if (BE (dfa->nodes_len >= dfa->nodes_alloc, 0)) { size_t new_nodes_alloc = dfa->nodes_alloc * 2; - int *new_nexts, *new_indices; + Idx *new_nexts, *new_indices; re_node_set *new_edests, *new_eclosures; re_token_t *new_nodes; /* Avoid overflows in realloc. */ const size_t max_object_size = MAX (sizeof (re_token_t), MAX (sizeof (re_node_set), - sizeof (int))); - if (BE (SIZE_MAX / max_object_size < new_nodes_alloc, 0)) + sizeof (Idx))); + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) < new_nodes_alloc, 0)) return -1; new_nodes = re_realloc (dfa->nodes, re_token_t, new_nodes_alloc); if (BE (new_nodes == NULL, 0)) return -1; dfa->nodes = new_nodes; - new_nexts = re_realloc (dfa->nexts, int, new_nodes_alloc); - new_indices = re_realloc (dfa->org_indices, int, new_nodes_alloc); + new_nexts = re_realloc (dfa->nexts, Idx, new_nodes_alloc); + new_indices = re_realloc (dfa->org_indices, Idx, new_nodes_alloc); new_edests = re_realloc (dfa->edests, re_node_set, new_nodes_alloc); new_eclosures = re_realloc (dfa->eclosures, re_node_set, new_nodes_alloc); if (BE (new_nexts == NULL || new_indices == NULL || new_edests == NULL || new_eclosures == NULL, 0)) - return -1; + { + re_free (new_nexts); + re_free (new_indices); + re_free (new_edests); + re_free (new_eclosures); + return -1; + } dfa->nexts = new_nexts; dfa->org_indices = new_indices; dfa->edests = new_edests; @@ -1434,7 +1444,8 @@ re_dfa_add_node (re_dfa_t *dfa, re_token_t token) dfa->nodes[dfa->nodes_len].constraint = 0; #ifdef RE_ENABLE_I18N dfa->nodes[dfa->nodes_len].accept_mb = - (type == OP_PERIOD && dfa->mb_cur_max > 1) || type == COMPLEX_BRACKET; + ((token.type == OP_PERIOD && dfa->mb_cur_max > 1) + || token.type == COMPLEX_BRACKET); #endif dfa->nexts[dfa->nodes_len] = -1; re_node_set_init_empty (dfa->edests + dfa->nodes_len); @@ -1442,11 +1453,11 @@ re_dfa_add_node (re_dfa_t *dfa, re_token_t token) return dfa->nodes_len++; } -static inline unsigned int +static re_hashval_t calc_state_hash (const re_node_set *nodes, unsigned int context) { - unsigned int hash = nodes->nelem + context; - int i; + re_hashval_t hash = nodes->nelem + context; + Idx i; for (i = 0 ; i < nodes->nelem ; i++) hash += nodes->elems[i]; return hash; @@ -1466,10 +1477,14 @@ __attribute_warn_unused_result__ re_acquire_state (reg_errcode_t *err, const re_dfa_t *dfa, const re_node_set *nodes) { - unsigned int hash; + re_hashval_t hash; re_dfastate_t *new_state; struct re_state_table_entry *spot; - int i; + Idx i; +#if defined GCC_LINT || defined lint + /* Suppress bogus uninitialized-variable warnings. */ + *err = REG_NOERROR; +#endif if (BE (nodes->nelem == 0, 0)) { *err = REG_NOERROR; @@ -1510,10 +1525,14 @@ __attribute_warn_unused_result__ re_acquire_state_context (reg_errcode_t *err, const re_dfa_t *dfa, const re_node_set *nodes, unsigned int context) { - unsigned int hash; + re_hashval_t hash; re_dfastate_t *new_state; struct re_state_table_entry *spot; - int i; + Idx i; +#if defined GCC_LINT || defined lint + /* Suppress bogus uninitialized-variable warnings. */ + *err = REG_NOERROR; +#endif if (nodes->nelem == 0) { *err = REG_NOERROR; @@ -1530,7 +1549,7 @@ re_acquire_state_context (reg_errcode_t *err, const re_dfa_t *dfa, && re_node_set_compare (state->entrance_nodes, nodes)) return state; } - /* There are no appropriate state in `dfa', create the new one. */ + /* There are no appropriate state in 'dfa', create the new one. */ new_state = create_cd_newstate (dfa, nodes, context, hash); if (BE (new_state == NULL, 0)) *err = REG_ESPACE; @@ -1545,11 +1564,11 @@ re_acquire_state_context (reg_errcode_t *err, const re_dfa_t *dfa, static reg_errcode_t __attribute_warn_unused_result__ register_state (const re_dfa_t *dfa, re_dfastate_t *newstate, - unsigned int hash) + re_hashval_t hash) { struct re_state_table_entry *spot; reg_errcode_t err; - int i; + Idx i; newstate->hash = hash; err = re_node_set_alloc (&newstate->non_eps_nodes, newstate->nodes.nelem); @@ -1557,16 +1576,16 @@ register_state (const re_dfa_t *dfa, re_dfastate_t *newstate, return REG_ESPACE; for (i = 0; i < newstate->nodes.nelem; i++) { - int elem = newstate->nodes.elems[i]; + Idx elem = newstate->nodes.elems[i]; if (!IS_EPSILON_NODE (dfa->nodes[elem].type)) - if (re_node_set_insert_last (&newstate->non_eps_nodes, elem) < 0) + if (! re_node_set_insert_last (&newstate->non_eps_nodes, elem)) return REG_ESPACE; } spot = dfa->state_table + (hash & dfa->state_hash_mask); if (BE (spot->alloc <= spot->num, 0)) { - int new_alloc = 2 * spot->num + 2; + Idx new_alloc = 2 * spot->num + 2; re_dfastate_t **new_array = re_realloc (spot->array, re_dfastate_t *, new_alloc); if (BE (new_array == NULL, 0)) @@ -1600,9 +1619,9 @@ free_state (re_dfastate_t *state) static re_dfastate_t * __attribute_warn_unused_result__ create_ci_newstate (const re_dfa_t *dfa, const re_node_set *nodes, - unsigned int hash) + re_hashval_t hash) { - int i; + Idx i; reg_errcode_t err; re_dfastate_t *newstate; @@ -1650,9 +1669,9 @@ create_ci_newstate (const re_dfa_t *dfa, const re_node_set *nodes, static re_dfastate_t * __attribute_warn_unused_result__ create_cd_newstate (const re_dfa_t *dfa, const re_node_set *nodes, - unsigned int context, unsigned int hash) + unsigned int context, re_hashval_t hash) { - int i, nctx_nodes = 0; + Idx i, nctx_nodes = 0; reg_errcode_t err; re_dfastate_t *newstate; diff --git a/posix/regex_internal.h b/posix/regex_internal.h index 41bf2d3..f38f6b3 100644 --- a/posix/regex_internal.h +++ b/posix/regex_internal.h @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ #ifndef _REGEX_INTERNAL_H #define _REGEX_INTERNAL_H 1 @@ -26,35 +26,60 @@ #include <stdlib.h> #include <string.h> -#if defined HAVE_LANGINFO_H || defined HAVE_LANGINFO_CODESET || defined _LIBC -# include <langinfo.h> -#endif -#if defined HAVE_LOCALE_H || defined _LIBC -# include <locale.h> -#endif -#if defined HAVE_WCHAR_H || defined _LIBC -# include <wchar.h> -#endif /* HAVE_WCHAR_H || _LIBC */ -#if defined HAVE_WCTYPE_H || defined _LIBC -# include <wctype.h> -#endif /* HAVE_WCTYPE_H || _LIBC */ -#if defined HAVE_STDBOOL_H || defined _LIBC -# include <stdbool.h> -#endif /* HAVE_STDBOOL_H || _LIBC */ -#if defined HAVE_STDINT_H || defined _LIBC -# include <stdint.h> -#endif /* HAVE_STDINT_H || _LIBC */ -#if defined _LIBC +#include <langinfo.h> +#include <locale.h> +#include <wchar.h> +#include <wctype.h> +#include <stdbool.h> +#include <stdint.h> + +#ifdef _LIBC # include <libc-lock.h> +# define lock_define(name) __libc_lock_define (, name) +# define lock_init(lock) (__libc_lock_init (lock), 0) +# define lock_fini(lock) ((void) 0) +# define lock_lock(lock) __libc_lock_lock (lock) +# define lock_unlock(lock) __libc_lock_unlock (lock) +#elif defined GNULIB_LOCK && !defined USE_UNLOCKED_IO +# include "glthread/lock.h" + /* Use gl_lock_define if empty macro arguments are known to work. + Otherwise, fall back on less-portable substitutes. */ +# if ((defined __GNUC__ && !defined __STRICT_ANSI__) \ + || (defined __STDC_VERSION__ && 199901L <= __STDC_VERSION__)) +# define lock_define(name) gl_lock_define (, name) +# elif USE_POSIX_THREADS +# define lock_define(name) pthread_mutex_t name; +# elif USE_PTH_THREADS +# define lock_define(name) pth_mutex_t name; +# elif USE_SOLARIS_THREADS +# define lock_define(name) mutex_t name; +# elif USE_WINDOWS_THREADS +# define lock_define(name) gl_lock_t name; +# else +# define lock_define(name) +# endif +# define lock_init(lock) glthread_lock_init (&(lock)) +# define lock_fini(lock) glthread_lock_destroy (&(lock)) +# define lock_lock(lock) glthread_lock_lock (&(lock)) +# define lock_unlock(lock) glthread_lock_unlock (&(lock)) +#elif defined GNULIB_PTHREAD && !defined USE_UNLOCKED_IO +# include <pthread.h> +# define lock_define(name) pthread_mutex_t name; +# define lock_init(lock) pthread_mutex_init (&(lock), 0) +# define lock_fini(lock) pthread_mutex_destroy (&(lock)) +# define lock_lock(lock) pthread_mutex_lock (&(lock)) +# define lock_unlock(lock) pthread_mutex_unlock (&(lock)) #else -# define __libc_lock_define(CLASS,NAME) -# define __libc_lock_init(NAME) do { } while (0) -# define __libc_lock_lock(NAME) do { } while (0) -# define __libc_lock_unlock(NAME) do { } while (0) +# define lock_define(name) +# define lock_init(lock) 0 +# define lock_fini(lock) ((void) 0) + /* The 'dfa' avoids an "unused variable 'dfa'" warning from GCC. */ +# define lock_lock(lock) ((void) dfa) +# define lock_unlock(lock) ((void) 0) #endif /* In case that the system doesn't have isblank(). */ -#if !defined _LIBC && !defined HAVE_ISBLANK && !defined isblank +#if !defined _LIBC && ! (defined isblank || (HAVE_ISBLANK && HAVE_DECL_ISBLANK)) # define isblank(ch) ((ch) == ' ' || (ch) == '\t') #endif @@ -75,6 +100,7 @@ __dcgettext (_libc_intl_domainname, msgid, LC_MESSAGES) # endif #else +# undef gettext # define gettext(msgid) (msgid) #endif @@ -84,23 +110,17 @@ # define gettext_noop(String) String #endif -/* For loser systems without the definition. */ -#ifndef SIZE_MAX -# define SIZE_MAX ((size_t) -1) -#endif - #if (defined MB_CUR_MAX && HAVE_WCTYPE_H && HAVE_ISWCTYPE) || _LIBC # define RE_ENABLE_I18N #endif -#if __GNUC__ >= 3 -# define BE(expr, val) __builtin_expect (expr, val) -#else -# define BE(expr, val) (expr) -#endif +#define BE(expr, val) __builtin_expect (expr, val) + +/* Number of ASCII characters. */ +#define ASCII_CHARS 0x80 -/* Number of single byte character. */ -#define SBC_MAX 256 +/* Number of single byte characters. */ +#define SBC_MAX (UCHAR_MAX + 1) #define COLL_ELEM_LEN_MAX 8 @@ -110,11 +130,15 @@ /* Rename to standard API for using out of glibc. */ #ifndef _LIBC +# undef __wctype +# undef __iswctype # define __wctype wctype +# define __iswalnum iswalnum # define __iswctype iswctype +# define __towlower towlower +# define __towupper towupper # define __btowc btowc # define __mbrtowc mbrtowc -# define __mempcpy mempcpy # define __wcrtomb wcrtomb # define __regfree regfree # define attribute_hidden @@ -124,32 +148,70 @@ # define __attribute__(arg) #endif -extern const char __re_error_msgid[] attribute_hidden; -extern const size_t __re_error_msgid_idx[] attribute_hidden; +#ifndef SSIZE_MAX +# define SSIZE_MAX ((ssize_t) (SIZE_MAX / 2)) +#endif + +/* The type of indexes into strings. This is signed, not size_t, + since the API requires indexes to fit in regoff_t anyway, and using + signed integers makes the code a bit smaller and presumably faster. + The traditional GNU regex implementation uses int for indexes. + The POSIX-compatible implementation uses a possibly-wider type. + The name 'Idx' is three letters to minimize the hassle of + reindenting a lot of regex code that formerly used 'int'. */ +typedef regoff_t Idx; +#ifdef _REGEX_LARGE_OFFSETS +# define IDX_MAX SSIZE_MAX +#else +# define IDX_MAX INT_MAX +#endif + +/* A hash value, suitable for computing hash tables. */ +typedef __re_size_t re_hashval_t; /* An integer used to represent a set of bits. It must be unsigned, and must be at least as wide as unsigned int. */ typedef unsigned long int bitset_word_t; /* All bits set in a bitset_word_t. */ #define BITSET_WORD_MAX ULONG_MAX -/* Number of bits in a bitset_word_t. */ -#define BITSET_WORD_BITS (sizeof (bitset_word_t) * CHAR_BIT) -/* Number of bitset_word_t in a bit_set. */ -#define BITSET_WORDS (SBC_MAX / BITSET_WORD_BITS) + +/* Number of bits in a bitset_word_t. For portability to hosts with + padding bits, do not use '(sizeof (bitset_word_t) * CHAR_BIT)'; + instead, deduce it directly from BITSET_WORD_MAX. Avoid + greater-than-32-bit integers and unconditional shifts by more than + 31 bits, as they're not portable. */ +#if BITSET_WORD_MAX == 0xffffffffUL +# define BITSET_WORD_BITS 32 +#elif BITSET_WORD_MAX >> 31 >> 4 == 1 +# define BITSET_WORD_BITS 36 +#elif BITSET_WORD_MAX >> 31 >> 16 == 1 +# define BITSET_WORD_BITS 48 +#elif BITSET_WORD_MAX >> 31 >> 28 == 1 +# define BITSET_WORD_BITS 60 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 1 == 1 +# define BITSET_WORD_BITS 64 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 9 == 1 +# define BITSET_WORD_BITS 72 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 31 >> 31 >> 3 == 1 +# define BITSET_WORD_BITS 128 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 7 == 1 +# define BITSET_WORD_BITS 256 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 7 > 1 +# define BITSET_WORD_BITS 257 /* any value > SBC_MAX will do here */ +# if BITSET_WORD_BITS <= SBC_MAX +# error "Invalid SBC_MAX" +# endif +#else +# error "Add case for new bitset_word_t size" +#endif + +/* Number of bitset_word_t values in a bitset_t. */ +#define BITSET_WORDS ((SBC_MAX + BITSET_WORD_BITS - 1) / BITSET_WORD_BITS) + typedef bitset_word_t bitset_t[BITSET_WORDS]; typedef bitset_word_t *re_bitset_ptr_t; typedef const bitset_word_t *re_const_bitset_ptr_t; -#define bitset_set(set,i) \ - (set[i / BITSET_WORD_BITS] |= (bitset_word_t) 1 << i % BITSET_WORD_BITS) -#define bitset_clear(set,i) \ - (set[i / BITSET_WORD_BITS] &= ~((bitset_word_t) 1 << i % BITSET_WORD_BITS)) -#define bitset_contain(set,i) \ - (set[i / BITSET_WORD_BITS] & ((bitset_word_t) 1 << i % BITSET_WORD_BITS)) -#define bitset_empty(set) memset (set, '\0', sizeof (bitset_t)) -#define bitset_set_all(set) memset (set, '\xff', sizeof (bitset_t)) -#define bitset_copy(dest,src) memcpy (dest, src, sizeof (bitset_t)) - #define PREV_WORD_CONSTRAINT 0x0001 #define PREV_NOTWORD_CONSTRAINT 0x0002 #define NEXT_WORD_CONSTRAINT 0x0004 @@ -177,9 +239,9 @@ typedef enum typedef struct { - int alloc; - int nelem; - int *elems; + Idx alloc; + Idx nelem; + Idx *elems; } re_node_set; typedef enum @@ -265,19 +327,19 @@ typedef struct unsigned int non_match : 1; /* # of multibyte characters. */ - int nmbchars; + Idx nmbchars; /* # of collating symbols. */ - int ncoll_syms; + Idx ncoll_syms; /* # of equivalence classes. */ - int nequiv_classes; + Idx nequiv_classes; /* # of range expressions. */ - int nranges; + Idx nranges; /* # of character classes. */ - int nchar_classes; + Idx nchar_classes; } re_charset_t; #endif /* RE_ENABLE_I18N */ @@ -290,10 +352,10 @@ typedef struct #ifdef RE_ENABLE_I18N re_charset_t *mbcset; /* for COMPLEX_BRACKET */ #endif /* RE_ENABLE_I18N */ - int idx; /* for BACK_REF */ + Idx idx; /* for BACK_REF */ re_context_type ctx_type; /* for ANCHOR */ } opr; -#if __GNUC__ >= 2 +#if __GNUC__ >= 2 && !defined __STRICT_ANSI__ re_token_type_t type : 8; #else re_token_type_t type; @@ -324,30 +386,30 @@ struct re_string_t #ifdef RE_ENABLE_I18N /* Store the wide character string which is corresponding to MBS. */ wint_t *wcs; - int *offsets; + Idx *offsets; mbstate_t cur_state; #endif /* Index in RAW_MBS. Each character mbs[i] corresponds to raw_mbs[raw_mbs_idx + i]. */ - int raw_mbs_idx; + Idx raw_mbs_idx; /* The length of the valid characters in the buffers. */ - int valid_len; + Idx valid_len; /* The corresponding number of bytes in raw_mbs array. */ - int valid_raw_len; + Idx valid_raw_len; /* The length of the buffers MBS and WCS. */ - int bufs_len; + Idx bufs_len; /* The index in MBS, which is updated by re_string_fetch_byte. */ - int cur_idx; + Idx cur_idx; /* length of RAW_MBS array. */ - int raw_len; + Idx raw_len; /* This is RAW_LEN - RAW_MBS_IDX + VALID_LEN - VALID_RAW_LEN. */ - int len; + Idx len; /* End of the buffer may be shorter than its length in the cases such as re_match_2, re_search_2. Then, we use STOP for end of the buffer instead of LEN. */ - int raw_stop; + Idx raw_stop; /* This is RAW_STOP - RAW_MBS_IDX adjusted through OFFSETS. */ - int stop; + Idx stop; /* The context of mbs[0]. We store the context independently, since the context of mbs[0] may be different from raw_mbs[0], which is @@ -357,7 +419,7 @@ struct re_string_t RE_TRANSLATE_TYPE trans; /* Copy of re_dfa_t's word_char. */ re_const_bitset_ptr_t word_char; - /* 1 if REG_ICASE. */ + /* true if REG_ICASE. */ unsigned char icase; unsigned char is_utf8; unsigned char map_notascii; @@ -373,18 +435,10 @@ typedef struct re_string_t re_string_t; struct re_dfa_t; typedef struct re_dfa_t re_dfa_t; -#if IS_IN (libc) -static reg_errcode_t re_string_realloc_buffers (re_string_t *pstr, - int new_buf_len); -# ifdef RE_ENABLE_I18N -static void build_wcs_buffer (re_string_t *pstr); -static reg_errcode_t build_wcs_upper_buffer (re_string_t *pstr); -# endif /* RE_ENABLE_I18N */ -static void build_upper_buffer (re_string_t *pstr); -static void re_string_translate_buffer (re_string_t *pstr); -static unsigned int re_string_context_at (const re_string_t *input, int idx, - int eflags) __attribute__ ((pure)); +#ifndef _LIBC +# define IS_IN(libc) false #endif + #define re_string_peek_byte(pstr, offset) \ ((pstr)->mbs[(pstr)->cur_idx + offset]) #define re_string_fetch_byte(pstr) \ @@ -402,7 +456,9 @@ static unsigned int re_string_context_at (const re_string_t *input, int idx, #define re_string_skip_bytes(pstr,idx) ((pstr)->cur_idx += (idx)) #define re_string_set_index(pstr,idx) ((pstr)->cur_idx = (idx)) -#include <alloca.h> +#if defined _LIBC || HAVE_ALLOCA +# include <alloca.h> +#endif #ifndef _LIBC # if HAVE_ALLOCA @@ -414,9 +470,24 @@ static unsigned int re_string_context_at (const re_string_t *input, int idx, # else /* alloca is implemented with malloc, so just use malloc. */ # define __libc_use_alloca(n) 0 +# undef alloca +# define alloca(n) malloc (n) # endif #endif +#ifdef _LIBC +# define MALLOC_0_IS_NONNULL 1 +#elif !defined MALLOC_0_IS_NONNULL +# define MALLOC_0_IS_NONNULL 0 +#endif + +#ifndef MAX +# define MAX(a,b) ((a) < (b) ? (b) : (a)) +#endif +#ifndef MIN +# define MIN(a,b) ((a) < (b) ? (a) : (b)) +#endif + #define re_malloc(t,n) ((t *) malloc ((n) * sizeof (t))) #define re_realloc(p,t,n) ((t *) realloc (p, (n) * sizeof (t))) #define re_free(p) free (p) @@ -431,9 +502,9 @@ struct bin_tree_t re_token_t token; - /* `node_idx' is the index in dfa->nodes, if `type' == 0. - Otherwise `type' indicate the type of this node. */ - int node_idx; + /* 'node_idx' is the index in dfa->nodes, if 'type' == 0. + Otherwise 'type' indicate the type of this node. */ + Idx node_idx; }; typedef struct bin_tree_t bin_tree_t; @@ -477,7 +548,7 @@ typedef struct bin_tree_storage_t bin_tree_storage_t; struct re_dfastate_t { - unsigned int hash; + re_hashval_t hash; re_node_set nodes; re_node_set non_eps_nodes; re_node_set inveclosure; @@ -485,9 +556,9 @@ struct re_dfastate_t struct re_dfastate_t **trtable, **word_trtable; unsigned int context : 4; unsigned int halt : 1; - /* If this state can accept `multi byte'. + /* If this state can accept "multi byte". Note that we refer to multibyte characters, and multi character - collating elements as `multi byte'. */ + collating elements as "multi byte". */ unsigned int accept_mb : 1; /* If this state has backreference node(s). */ unsigned int has_backref : 1; @@ -497,8 +568,8 @@ typedef struct re_dfastate_t re_dfastate_t; struct re_state_table_entry { - int num; - int alloc; + Idx num; + Idx alloc; re_dfastate_t **array; }; @@ -506,8 +577,8 @@ struct re_state_table_entry typedef struct { - int next_idx; - int alloc; + Idx next_idx; + Idx alloc; re_dfastate_t **array; } state_array_t; @@ -515,8 +586,8 @@ typedef struct typedef struct { - int node; - int str_idx; /* The position NODE match at. */ + Idx node; + Idx str_idx; /* The position NODE match at. */ state_array_t path; } re_sub_match_last_t; @@ -526,20 +597,20 @@ typedef struct typedef struct { - int str_idx; - int node; + Idx str_idx; + Idx node; state_array_t *path; - int alasts; /* Allocation size of LASTS. */ - int nlasts; /* The number of LASTS. */ + Idx alasts; /* Allocation size of LASTS. */ + Idx nlasts; /* The number of LASTS. */ re_sub_match_last_t **lasts; } re_sub_match_top_t; struct re_backref_cache_entry { - int node; - int str_idx; - int subexp_from; - int subexp_to; + Idx node; + Idx str_idx; + Idx subexp_from; + Idx subexp_to; char more; char unused; unsigned short int eps_reachable_subexps_map; @@ -557,18 +628,18 @@ typedef struct /* EFLAGS of the argument of regexec. */ int eflags; /* Where the matching ends. */ - int match_last; - int last_node; + Idx match_last; + Idx last_node; /* The state log used by the matcher. */ re_dfastate_t **state_log; - int state_log_top; + Idx state_log_top; /* Back reference cache. */ - int nbkref_ents; - int abkref_ents; + Idx nbkref_ents; + Idx abkref_ents; struct re_backref_cache_entry *bkref_ents; int max_mb_elem_len; - int nsub_tops; - int asub_tops; + Idx nsub_tops; + Idx asub_tops; re_sub_match_top_t **sub_tops; } re_match_context_t; @@ -576,23 +647,23 @@ typedef struct { re_dfastate_t **sifted_states; re_dfastate_t **limited_states; - int last_node; - int last_str_idx; + Idx last_node; + Idx last_str_idx; re_node_set limits; } re_sift_context_t; struct re_fail_stack_ent_t { - int idx; - int node; + Idx idx; + Idx node; regmatch_t *regs; re_node_set eps_via_nodes; }; struct re_fail_stack_t { - int num; - int alloc; + Idx num; + Idx alloc; struct re_fail_stack_ent_t *stack; }; @@ -601,8 +672,8 @@ struct re_dfa_t re_token_t *nodes; size_t nodes_alloc; size_t nodes_len; - int *nexts; - int *org_indices; + Idx *nexts; + Idx *org_indices; re_node_set *edests; re_node_set *eclosures; re_node_set *inveclosures; @@ -616,10 +687,10 @@ struct re_dfa_t re_bitset_ptr_t sb_char; int str_tree_storage_idx; - /* number of subexpressions `re_nsub' is in regex_t. */ - unsigned int state_hash_mask; - int init_node; - int nbackref; /* The number of backreference in this dfa. */ + /* number of subexpressions 're_nsub' is in regex_t. */ + re_hashval_t state_hash_mask; + Idx init_node; + Idx nbackref; /* The number of backreference in this dfa. */ /* Bitmap expressing which backreference is used. */ bitset_word_t used_bkref_map; @@ -636,11 +707,11 @@ struct re_dfa_t int mb_cur_max; bitset_t word_char; reg_syntax_t syntax; - int *subexp_map; + Idx *subexp_map; #ifdef DEBUG char* re_str; #endif - __libc_lock_define (, lock) + lock_define (lock) }; #define re_node_set_init_empty(set) memset (set, '\0', sizeof (re_node_set)) @@ -671,16 +742,60 @@ typedef struct } bracket_elem_t; -/* Inline functions for bitset operation. */ -static void __attribute__ ((unused)) +/* Functions for bitset_t operation. */ + +static inline void +bitset_set (bitset_t set, Idx i) +{ + set[i / BITSET_WORD_BITS] |= (bitset_word_t) 1 << i % BITSET_WORD_BITS; +} + +static inline void +bitset_clear (bitset_t set, Idx i) +{ + set[i / BITSET_WORD_BITS] &= ~ ((bitset_word_t) 1 << i % BITSET_WORD_BITS); +} + +static inline bool +bitset_contain (const bitset_t set, Idx i) +{ + return (set[i / BITSET_WORD_BITS] >> i % BITSET_WORD_BITS) & 1; +} + +static inline void +bitset_empty (bitset_t set) +{ + memset (set, '\0', sizeof (bitset_t)); +} + +static inline void +bitset_set_all (bitset_t set) +{ + memset (set, -1, sizeof (bitset_word_t) * (SBC_MAX / BITSET_WORD_BITS)); + if (SBC_MAX % BITSET_WORD_BITS != 0) + set[BITSET_WORDS - 1] = + ((bitset_word_t) 1 << SBC_MAX % BITSET_WORD_BITS) - 1; +} + +static inline void +bitset_copy (bitset_t dest, const bitset_t src) +{ + memcpy (dest, src, sizeof (bitset_t)); +} + +static inline void bitset_not (bitset_t set) { int bitset_i; - for (bitset_i = 0; bitset_i < BITSET_WORDS; ++bitset_i) + for (bitset_i = 0; bitset_i < SBC_MAX / BITSET_WORD_BITS; ++bitset_i) set[bitset_i] = ~set[bitset_i]; + if (SBC_MAX % BITSET_WORD_BITS != 0) + set[BITSET_WORDS - 1] = + ((((bitset_word_t) 1 << SBC_MAX % BITSET_WORD_BITS) - 1) + & ~set[BITSET_WORDS - 1]); } -static void __attribute__ ((unused)) +static inline void bitset_merge (bitset_t dest, const bitset_t src) { int bitset_i; @@ -688,7 +803,7 @@ bitset_merge (bitset_t dest, const bitset_t src) dest[bitset_i] |= src[bitset_i]; } -static void __attribute__ ((unused)) +static inline void bitset_mask (bitset_t dest, const bitset_t src) { int bitset_i; @@ -697,10 +812,10 @@ bitset_mask (bitset_t dest, const bitset_t src) } #ifdef RE_ENABLE_I18N -/* Inline functions for re_string. */ +/* Functions for re_string. */ static int __attribute__ ((pure, unused)) -re_string_char_size_at (const re_string_t *pstr, int idx) +re_string_char_size_at (const re_string_t *pstr, Idx idx) { int byte_idx; if (pstr->mb_cur_max == 1) @@ -713,23 +828,22 @@ re_string_char_size_at (const re_string_t *pstr, int idx) static wint_t __attribute__ ((pure, unused)) -re_string_wchar_at (const re_string_t *pstr, int idx) +re_string_wchar_at (const re_string_t *pstr, Idx idx) { if (pstr->mb_cur_max == 1) return (wint_t) pstr->mbs[idx]; return (wint_t) pstr->wcs[idx]; } -# if IS_IN (libc) -# ifdef _LIBC -# include <locale/weight.h> -# endif +# ifdef _LIBC +# include <locale/weight.h> +# endif static int __attribute__ ((pure, unused)) -re_string_elem_size_at (const re_string_t *pstr, int idx) +re_string_elem_size_at (const re_string_t *pstr, Idx idx) { -# ifdef _LIBC +# ifdef _LIBC const unsigned char *p, *extra; const int32_t *table, *indirect; uint_fast32_t nrules = _NL_CURRENT_WORD (LC_COLLATE, _NL_COLLATE_NRULES); @@ -746,10 +860,34 @@ re_string_elem_size_at (const re_string_t *pstr, int idx) return p - pstr->mbs - idx; } else -# endif /* _LIBC */ +# endif /* _LIBC */ return 1; } -# endif #endif /* RE_ENABLE_I18N */ +#ifndef __GNUC_PREREQ +# if defined __GNUC__ && defined __GNUC_MINOR__ +# define __GNUC_PREREQ(maj, min) \ + ((__GNUC__ << 16) + __GNUC_MINOR__ >= ((maj) << 16) + (min)) +# else +# define __GNUC_PREREQ(maj, min) 0 +# endif +#endif + +#if __GNUC_PREREQ (3,4) +# undef __attribute_warn_unused_result__ +# define __attribute_warn_unused_result__ \ + __attribute__ ((__warn_unused_result__)) +#else +# define __attribute_warn_unused_result__ /* empty */ +#endif + +#ifndef FALLTHROUGH +# if __GNUC__ < 7 +# define FALLTHROUGH ((void) 0) +# else +# define FALLTHROUGH __attribute__ ((__fallthrough__)) +# endif +#endif + #endif /* _REGEX_INTERNAL_H */ diff --git a/posix/regexec.c b/posix/regexec.c index 4b1ab4e..d64b025 100644 --- a/posix/regexec.c +++ b/posix/regexec.c @@ -15,100 +15,98 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <stdint.h> + <https://www.gnu.org/licenses/>. */ static reg_errcode_t match_ctx_init (re_match_context_t *cache, int eflags, - int n); + Idx n); static void match_ctx_clean (re_match_context_t *mctx); static void match_ctx_free (re_match_context_t *cache); -static reg_errcode_t match_ctx_add_entry (re_match_context_t *cache, int node, - int str_idx, int from, int to); -static int search_cur_bkref_entry (const re_match_context_t *mctx, - int str_idx); -static reg_errcode_t match_ctx_add_subtop (re_match_context_t *mctx, int node, - int str_idx); +static reg_errcode_t match_ctx_add_entry (re_match_context_t *cache, Idx node, + Idx str_idx, Idx from, Idx to); +static Idx search_cur_bkref_entry (const re_match_context_t *mctx, Idx str_idx); +static reg_errcode_t match_ctx_add_subtop (re_match_context_t *mctx, Idx node, + Idx str_idx); static re_sub_match_last_t * match_ctx_add_sublast (re_sub_match_top_t *subtop, - int node, int str_idx); + Idx node, Idx str_idx); static void sift_ctx_init (re_sift_context_t *sctx, re_dfastate_t **sifted_sts, - re_dfastate_t **limited_sts, int last_node, - int last_str_idx); + re_dfastate_t **limited_sts, Idx last_node, + Idx last_str_idx); static reg_errcode_t re_search_internal (const regex_t *preg, - const char *string, int length, - int start, int range, int stop, + const char *string, Idx length, + Idx start, Idx last_start, Idx stop, size_t nmatch, regmatch_t pmatch[], int eflags); -static int re_search_2_stub (struct re_pattern_buffer *bufp, - const char *string1, int length1, - const char *string2, int length2, - int start, int range, struct re_registers *regs, - int stop, int ret_len); -static int re_search_stub (struct re_pattern_buffer *bufp, - const char *string, int length, int start, - int range, int stop, struct re_registers *regs, - int ret_len); +static regoff_t re_search_2_stub (struct re_pattern_buffer *bufp, + const char *string1, Idx length1, + const char *string2, Idx length2, + Idx start, regoff_t range, + struct re_registers *regs, + Idx stop, bool ret_len); +static regoff_t re_search_stub (struct re_pattern_buffer *bufp, + const char *string, Idx length, Idx start, + regoff_t range, Idx stop, + struct re_registers *regs, + bool ret_len); static unsigned re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, - int nregs, int regs_allocated); + Idx nregs, int regs_allocated); static reg_errcode_t prune_impossible_nodes (re_match_context_t *mctx); -static int check_matching (re_match_context_t *mctx, int fl_longest_match, - int *p_match_first); -static int check_halt_state_context (const re_match_context_t *mctx, - const re_dfastate_t *state, int idx); +static Idx check_matching (re_match_context_t *mctx, bool fl_longest_match, + Idx *p_match_first); +static Idx check_halt_state_context (const re_match_context_t *mctx, + const re_dfastate_t *state, Idx idx); static void update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, - regmatch_t *prev_idx_match, int cur_node, - int cur_idx, int nmatch); + regmatch_t *prev_idx_match, Idx cur_node, + Idx cur_idx, Idx nmatch); static reg_errcode_t push_fail_stack (struct re_fail_stack_t *fs, - int str_idx, int dest_node, int nregs, + Idx str_idx, Idx dest_node, Idx nregs, regmatch_t *regs, re_node_set *eps_via_nodes); static reg_errcode_t set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, regmatch_t *pmatch, - int fl_backtrack); + bool fl_backtrack); static reg_errcode_t free_fail_stack_return (struct re_fail_stack_t *fs); #ifdef RE_ENABLE_I18N static int sift_states_iter_mb (const re_match_context_t *mctx, re_sift_context_t *sctx, - int node_idx, int str_idx, int max_str_idx); + Idx node_idx, Idx str_idx, Idx max_str_idx); #endif /* RE_ENABLE_I18N */ static reg_errcode_t sift_states_backward (const re_match_context_t *mctx, re_sift_context_t *sctx); static reg_errcode_t build_sifted_states (const re_match_context_t *mctx, - re_sift_context_t *sctx, int str_idx, + re_sift_context_t *sctx, Idx str_idx, re_node_set *cur_dest); static reg_errcode_t update_cur_sifted_state (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, + Idx str_idx, re_node_set *dest_nodes); static reg_errcode_t add_epsilon_src_nodes (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates); -static int check_dst_limits (const re_match_context_t *mctx, - re_node_set *limits, - int dst_node, int dst_idx, int src_node, - int src_idx); +static bool check_dst_limits (const re_match_context_t *mctx, + const re_node_set *limits, + Idx dst_node, Idx dst_idx, Idx src_node, + Idx src_idx); static int check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, - int boundaries, int subexp_idx, - int from_node, int bkref_idx); + int boundaries, Idx subexp_idx, + Idx from_node, Idx bkref_idx); static int check_dst_limits_calc_pos (const re_match_context_t *mctx, - int limit, int subexp_idx, - int node, int str_idx, - int bkref_idx); + Idx limit, Idx subexp_idx, + Idx node, Idx str_idx, + Idx bkref_idx); static reg_errcode_t check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates, re_node_set *limits, struct re_backref_cache_entry *bkref_ents, - int str_idx); + Idx str_idx); static reg_errcode_t sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, - const re_node_set *candidates); + Idx str_idx, const re_node_set *candidates); static reg_errcode_t merge_state_array (const re_dfa_t *dfa, re_dfastate_t **dst, - re_dfastate_t **src, int num); + re_dfastate_t **src, Idx num); static re_dfastate_t *find_recover_state (reg_errcode_t *err, re_match_context_t *mctx); static re_dfastate_t *transit_state (reg_errcode_t *err, @@ -119,7 +117,7 @@ static re_dfastate_t *merge_state_with_log (reg_errcode_t *err, re_dfastate_t *next_state); static reg_errcode_t check_subexp_matching_top (re_match_context_t *mctx, re_node_set *cur_nodes, - int str_idx); + Idx str_idx); #if 0 static re_dfastate_t *transit_state_sb (reg_errcode_t *err, re_match_context_t *mctx, @@ -132,46 +130,46 @@ static reg_errcode_t transit_state_mb (re_match_context_t *mctx, static reg_errcode_t transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes); static reg_errcode_t get_subexp (re_match_context_t *mctx, - int bkref_node, int bkref_str_idx); + Idx bkref_node, Idx bkref_str_idx); static reg_errcode_t get_subexp_sub (re_match_context_t *mctx, const re_sub_match_top_t *sub_top, re_sub_match_last_t *sub_last, - int bkref_node, int bkref_str); -static int find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, - int subexp_idx, int type); + Idx bkref_node, Idx bkref_str); +static Idx find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, + Idx subexp_idx, int type); static reg_errcode_t check_arrival (re_match_context_t *mctx, - state_array_t *path, int top_node, - int top_str, int last_node, int last_str, + state_array_t *path, Idx top_node, + Idx top_str, Idx last_node, Idx last_str, int type); static reg_errcode_t check_arrival_add_next_nodes (re_match_context_t *mctx, - int str_idx, + Idx str_idx, re_node_set *cur_nodes, re_node_set *next_nodes); static reg_errcode_t check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, - int ex_subexp, int type); + Idx ex_subexp, int type); static reg_errcode_t check_arrival_expand_ecl_sub (const re_dfa_t *dfa, re_node_set *dst_nodes, - int target, int ex_subexp, + Idx target, Idx ex_subexp, int type); static reg_errcode_t expand_bkref_cache (re_match_context_t *mctx, - re_node_set *cur_nodes, int cur_str, - int subexp_num, int type); -static int build_trtable (const re_dfa_t *dfa, re_dfastate_t *state); + re_node_set *cur_nodes, Idx cur_str, + Idx subexp_num, int type); +static bool build_trtable (const re_dfa_t *dfa, re_dfastate_t *state); #ifdef RE_ENABLE_I18N -static int check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, - const re_string_t *input, int idx); +static int check_node_accept_bytes (const re_dfa_t *dfa, Idx node_idx, + const re_string_t *input, Idx idx); # ifdef _LIBC static unsigned int find_collation_sequence_value (const unsigned char *mbs, size_t name_len); # endif /* _LIBC */ #endif /* RE_ENABLE_I18N */ -static int group_nodes_into_DFAstates (const re_dfa_t *dfa, +static Idx group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, re_node_set *states_node, bitset_t *states_ch); -static int check_node_accept (const re_match_context_t *mctx, - const re_token_t *node, int idx); +static bool check_node_accept (const re_match_context_t *mctx, + const re_token_t *node, Idx idx); static reg_errcode_t extend_buffers (re_match_context_t *mctx, int min_len); /* Entry point for POSIX code. */ @@ -180,23 +178,23 @@ static reg_errcode_t extend_buffers (re_match_context_t *mctx, int min_len); string STRING. If NMATCH is zero or REG_NOSUB was set in the cflags argument to - `regcomp', we ignore PMATCH. Otherwise, we assume PMATCH has at + 'regcomp', we ignore PMATCH. Otherwise, we assume PMATCH has at least NMATCH elements, and we set them to the offsets of the corresponding matched substrings. - EFLAGS specifies `execution flags' which affect matching: if + EFLAGS specifies "execution flags" which affect matching: if REG_NOTBOL is set, then ^ does not match at the beginning of the string; if REG_NOTEOL is set, then $ does not match at the end. We return 0 if we find a match and REG_NOMATCH if not. */ int -regexec (const regex_t *__restrict preg, const char *__restrict string, +regexec (const regex_t *_Restrict_ preg, const char *_Restrict_ string, size_t nmatch, regmatch_t pmatch[], int eflags) { reg_errcode_t err; - int start, length; - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + Idx start, length; + re_dfa_t *dfa = preg->buffer; if (eflags & ~(REG_NOTBOL | REG_NOTEOL | REG_STARTEND)) return REG_BADPAT; @@ -212,14 +210,14 @@ regexec (const regex_t *__restrict preg, const char *__restrict string, length = strlen (string); } - __libc_lock_lock (dfa->lock); + lock_lock (dfa->lock); if (preg->no_sub) - err = re_search_internal (preg, string, length, start, length - start, + err = re_search_internal (preg, string, length, start, length, length, 0, NULL, eflags); else - err = re_search_internal (preg, string, length, start, length - start, + err = re_search_internal (preg, string, length, start, length, length, nmatch, pmatch, eflags); - __libc_lock_unlock (dfa->lock); + lock_unlock (dfa->lock); return err != REG_NOERROR; } @@ -234,8 +232,8 @@ __typeof__ (__regexec) __compat_regexec; int attribute_compat_text_section -__compat_regexec (const regex_t *__restrict preg, - const char *__restrict string, size_t nmatch, +__compat_regexec (const regex_t *_Restrict_ preg, + const char *_Restrict_ string, size_t nmatch, regmatch_t pmatch[], int eflags) { return regexec (preg, string, nmatch, pmatch, @@ -274,62 +272,81 @@ compat_symbol (libc, __compat_regexec, regexec, GLIBC_2_0); return the position of the start of the match. Return value -1 means no match was found and -2 indicates an internal error. */ -int -re_match (struct re_pattern_buffer *bufp, const char *string, int length, - int start, struct re_registers *regs) +regoff_t +re_match (struct re_pattern_buffer *bufp, const char *string, Idx length, + Idx start, struct re_registers *regs) { - return re_search_stub (bufp, string, length, start, 0, length, regs, 1); + return re_search_stub (bufp, string, length, start, 0, length, regs, true); } #ifdef _LIBC weak_alias (__re_match, re_match) #endif -int -re_search (struct re_pattern_buffer *bufp, const char *string, int length, - int start, int range, struct re_registers *regs) +regoff_t +re_search (struct re_pattern_buffer *bufp, const char *string, Idx length, + Idx start, regoff_t range, struct re_registers *regs) { - return re_search_stub (bufp, string, length, start, range, length, regs, 0); + return re_search_stub (bufp, string, length, start, range, length, regs, + false); } #ifdef _LIBC weak_alias (__re_search, re_search) #endif -int -re_match_2 (struct re_pattern_buffer *bufp, const char *string1, int length1, - const char *string2, int length2, int start, - struct re_registers *regs, int stop) +regoff_t +re_match_2 (struct re_pattern_buffer *bufp, const char *string1, Idx length1, + const char *string2, Idx length2, Idx start, + struct re_registers *regs, Idx stop) { return re_search_2_stub (bufp, string1, length1, string2, length2, - start, 0, regs, stop, 1); + start, 0, regs, stop, true); } #ifdef _LIBC weak_alias (__re_match_2, re_match_2) #endif -int -re_search_2 (struct re_pattern_buffer *bufp, const char *string1, int length1, - const char *string2, int length2, int start, int range, - struct re_registers *regs, int stop) +regoff_t +re_search_2 (struct re_pattern_buffer *bufp, const char *string1, Idx length1, + const char *string2, Idx length2, Idx start, regoff_t range, + struct re_registers *regs, Idx stop) { return re_search_2_stub (bufp, string1, length1, string2, length2, - start, range, regs, stop, 0); + start, range, regs, stop, false); } #ifdef _LIBC weak_alias (__re_search_2, re_search_2) #endif -static int +/* Set *R = A + B. Return true if the answer is mathematically incorrect due + to overflow; in this case, *R is the low order bits of the correct + answer. */ +static inline bool +check_add_overflow_idx (Idx a, Idx b, Idx *r) +{ +#if __GNUC__ >= 5 + return __builtin_add_overflow (a, b, r); +#else + *r = a + b; + if (a < 0 && b < IDX_MAX -a) + return false; + return true; +#endif +} + +static regoff_t re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, - int length1, const char *string2, int length2, int start, - int range, struct re_registers *regs, - int stop, int ret_len) + Idx length1, const char *string2, Idx length2, Idx start, + regoff_t range, struct re_registers *regs, + Idx stop, bool ret_len) { const char *str; - int rval; - int len = length1 + length2; + regoff_t rval; + Idx len; char *s = NULL; - if (BE (length1 < 0 || length2 < 0 || stop < 0 || len < length1, 0)) + if (BE ((length1 < 0 || length2 < 0 || stop < 0 + || check_add_overflow_idx (length1, length2, &len)), + 0)) return -2; /* Concatenate the strings. */ @@ -353,42 +370,45 @@ re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, else str = string1; - rval = re_search_stub (bufp, str, len, start, range, stop, regs, ret_len); + rval = re_search_stub (bufp, str, len, start, range, stop, regs, + ret_len); re_free (s); return rval; } /* The parameters have the same meaning as those of re_search. Additional parameters: - If RET_LEN is nonzero the length of the match is returned (re_match style); + If RET_LEN is true the length of the match is returned (re_match style); otherwise the position of the match is returned. */ -static int -re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, - int start, int range, int stop, struct re_registers *regs, - int ret_len) +static regoff_t +re_search_stub (struct re_pattern_buffer *bufp, const char *string, Idx length, + Idx start, regoff_t range, Idx stop, struct re_registers *regs, + bool ret_len) { reg_errcode_t result; regmatch_t *pmatch; - int nregs, rval; + Idx nregs; + regoff_t rval; int eflags = 0; - re_dfa_t *dfa = (re_dfa_t *) bufp->buffer; + re_dfa_t *dfa = bufp->buffer; + Idx last_start = start + range; /* Check for out-of-range. */ if (BE (start < 0 || start > length, 0)) return -1; - if (BE (start + range > length, 0)) - range = length - start; - else if (BE (start + range < 0, 0)) - range = -start; + if (BE (length < last_start || (0 <= range && last_start < start), 0)) + last_start = length; + else if (BE (last_start < 0 || (range < 0 && start <= last_start), 0)) + last_start = 0; - __libc_lock_lock (dfa->lock); + lock_lock (dfa->lock); eflags |= (bufp->not_bol) ? REG_NOTBOL : 0; eflags |= (bufp->not_eol) ? REG_NOTEOL : 0; /* Compile fastmap if we haven't yet. */ - if (range > 0 && bufp->fastmap != NULL && !bufp->fastmap_accurate) + if (start < last_start && bufp->fastmap != NULL && !bufp->fastmap_accurate) re_compile_fastmap (bufp); if (BE (bufp->no_sub, 0)) @@ -397,8 +417,8 @@ re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, /* We need at least 1 register. */ if (regs == NULL) nregs = 1; - else if (BE (bufp->regs_allocated == REGS_FIXED && - regs->num_regs < bufp->re_nsub + 1, 0)) + else if (BE (bufp->regs_allocated == REGS_FIXED + && regs->num_regs <= bufp->re_nsub, 0)) { nregs = regs->num_regs; if (BE (nregs < 1, 0)) @@ -417,14 +437,14 @@ re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, goto out; } - result = re_search_internal (bufp, string, length, start, range, stop, + result = re_search_internal (bufp, string, length, start, last_start, stop, nregs, pmatch, eflags); rval = 0; - /* I hope we needn't fill ther regs with -1's when no match was found. */ + /* I hope we needn't fill their regs with -1's when no match was found. */ if (result != REG_NOERROR) - rval = -1; + rval = result == REG_NOMATCH ? -1 : -2; else if (regs != NULL) { /* If caller wants register contents data back, copy them. */ @@ -446,18 +466,18 @@ re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, } re_free (pmatch); out: - __libc_lock_unlock (dfa->lock); + lock_unlock (dfa->lock); return rval; } static unsigned -re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, int nregs, +re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, Idx nregs, int regs_allocated) { int rval = REGS_REALLOCATE; - int i; - int need_regs = nregs + 1; - /* We need one extra element beyond `num_regs' for the `-1' marker GNU code + Idx i; + Idx need_regs = nregs + 1; + /* We need one extra element beyond 'num_regs' for the '-1' marker GNU code uses. */ /* Have the register data arrays been allocated? */ @@ -530,7 +550,7 @@ re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, int nregs, void re_set_registers (struct re_pattern_buffer *bufp, struct re_registers *regs, - unsigned num_regs, regoff_t *starts, regoff_t *ends) + __re_size_t num_regs, regoff_t *starts, regoff_t *ends) { if (num_regs) { @@ -543,7 +563,7 @@ re_set_registers (struct re_pattern_buffer *bufp, struct re_registers *regs, { bufp->regs_allocated = REGS_UNALLOCATED; regs->num_regs = 0; - regs->start = regs->end = (regoff_t *) 0; + regs->start = regs->end = NULL; } } #ifdef _LIBC @@ -568,32 +588,38 @@ re_exec (const char *s) /* Searches for a compiled pattern PREG in the string STRING, whose length is LENGTH. NMATCH, PMATCH, and EFLAGS have the same - meaning as with regexec. START, and RANGE have the same meanings - with re_search. + meaning as with regexec. LAST_START is START + RANGE, where + START and RANGE have the same meaning as with re_search. Return REG_NOERROR if we find a match, and REG_NOMATCH if not, otherwise return the error code. Note: We assume front end functions already check ranges. - (START + RANGE >= 0 && START + RANGE <= LENGTH) */ + (0 <= LAST_START && LAST_START <= LENGTH) */ static reg_errcode_t __attribute_warn_unused_result__ -re_search_internal (const regex_t *preg, const char *string, int length, - int start, int range, int stop, size_t nmatch, +re_search_internal (const regex_t *preg, const char *string, Idx length, + Idx start, Idx last_start, Idx stop, size_t nmatch, regmatch_t pmatch[], int eflags) { reg_errcode_t err; - const re_dfa_t *dfa = (const re_dfa_t *) preg->buffer; - int left_lim, right_lim, incr; - int fl_longest_match, match_first, match_kind, match_last = -1; - int extra_nmatch; - int sb, ch; + const re_dfa_t *dfa = preg->buffer; + Idx left_lim, right_lim; + int incr; + bool fl_longest_match; + int match_kind; + Idx match_first; + Idx match_last = -1; + Idx extra_nmatch; + bool sb; + int ch; #if defined _LIBC || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L) re_match_context_t mctx = { .dfa = dfa }; #else re_match_context_t mctx; #endif - char *fastmap = (preg->fastmap != NULL && preg->fastmap_accurate - && range && !preg->can_be_null) ? preg->fastmap : NULL; + char *fastmap = ((preg->fastmap != NULL && preg->fastmap_accurate + && start != last_start && !preg->can_be_null) + ? preg->fastmap : NULL); RE_TRANSLATE_TYPE t = preg->translate; #if !(defined _LIBC || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L)) @@ -612,7 +638,7 @@ re_search_internal (const regex_t *preg, const char *string, int length, #ifdef DEBUG /* We assume front-end functions already check them. */ - assert (start + range >= 0 && start + range <= length); + assert (0 <= last_start && last_start <= length); #endif /* If initial states with non-begbuf contexts have no elements, @@ -623,16 +649,17 @@ re_search_internal (const regex_t *preg, const char *string, int length, && (dfa->init_state_nl->nodes.nelem == 0 || !preg->newline_anchor)) { - if (start != 0 && start + range != 0) - return REG_NOMATCH; - start = range = 0; + if (start != 0 && last_start != 0) + return REG_NOMATCH; + start = last_start = 0; } /* We must check the longest matching, if nmatch > 0. */ fl_longest_match = (nmatch != 0 || dfa->nbackref); err = re_string_allocate (&mctx.input, string, length, dfa->nodes_len + 1, - preg->translate, preg->syntax & RE_ICASE, dfa); + preg->translate, (preg->syntax & RE_ICASE) != 0, + dfa); if (BE (err != REG_NOERROR, 0)) goto free_return; mctx.input.stop = stop; @@ -650,7 +677,8 @@ re_search_internal (const regex_t *preg, const char *string, int length, if (nmatch > 1 || dfa->has_mb_node) { /* Avoid overflow. */ - if (BE (SIZE_MAX / sizeof (re_dfastate_t *) <= mctx.input.bufs_len, 0)) + if (BE ((MIN (IDX_MAX, SIZE_MAX / sizeof (re_dfastate_t *)) + <= mctx.input.bufs_len), 0)) { err = REG_ESPACE; goto free_return; @@ -670,15 +698,15 @@ re_search_internal (const regex_t *preg, const char *string, int length, mctx.input.tip_context = (eflags & REG_NOTBOL) ? CONTEXT_BEGBUF : CONTEXT_NEWLINE | CONTEXT_BEGBUF; - /* Check incrementally whether of not the input string match. */ - incr = (range < 0) ? -1 : 1; - left_lim = (range < 0) ? start + range : start; - right_lim = (range < 0) ? start : start + range; + /* Check incrementally whether the input string matches. */ + incr = (last_start < start) ? -1 : 1; + left_lim = (last_start < start) ? last_start : start; + right_lim = (last_start < start) ? start : last_start; sb = dfa->mb_cur_max == 1; match_kind = (fastmap ? ((sb || !(preg->syntax & RE_ICASE || t) ? 4 : 0) - | (range >= 0 ? 2 : 0) + | (start <= last_start ? 2 : 0) | (t != NULL ? 1 : 0)) : 8); @@ -745,8 +773,8 @@ re_search_internal (const regex_t *preg, const char *string, int length, { /* If MATCH_FIRST is out of the valid range, reconstruct the buffers. */ - unsigned int offset = match_first - mctx.input.raw_mbs_idx; - if (BE (offset >= (unsigned int) mctx.input.valid_raw_len, 0)) + __re_size_t offset = match_first - mctx.input.raw_mbs_idx; + if (BE (offset >= (__re_size_t) mctx.input.valid_raw_len, 0)) { err = re_string_reconstruct (&mctx.input, match_first, eflags); @@ -788,7 +816,7 @@ re_search_internal (const regex_t *preg, const char *string, int length, /* We assume that the matching starts from 0. */ mctx.state_log_top = mctx.nbkref_ents = mctx.max_mb_elem_len = 0; match_last = check_matching (&mctx, fl_longest_match, - range >= 0 ? &match_first : NULL); + start <= last_start ? &match_first : NULL); if (match_last != -1) { if (BE (match_last == -2, 0)) @@ -831,7 +859,7 @@ re_search_internal (const regex_t *preg, const char *string, int length, /* Set pmatch[] if we need. */ if (nmatch > 0) { - int reg_idx; + Idx reg_idx; /* Initialize registers. */ for (reg_idx = 1; reg_idx < nmatch; ++reg_idx) @@ -840,6 +868,9 @@ re_search_internal (const regex_t *preg, const char *string, int length, /* Set the points where matching start/end. */ pmatch[0].rm_so = 0; pmatch[0].rm_eo = mctx.match_last; + /* FIXME: This function should fail if mctx.match_last exceeds + the maximum possible regoff_t value. We need a new error + code REG_OVERFLOW. */ if (!preg->no_sub && nmatch > 1) { @@ -903,7 +934,7 @@ __attribute_warn_unused_result__ prune_impossible_nodes (re_match_context_t *mctx) { const re_dfa_t *const dfa = mctx->dfa; - int halt_node, match_last; + Idx halt_node, match_last; reg_errcode_t ret; re_dfastate_t **sifted_states; re_dfastate_t **lim_states = NULL; @@ -915,7 +946,7 @@ prune_impossible_nodes (re_match_context_t *mctx) halt_node = mctx->last_node; /* Avoid overflow. */ - if (BE (SIZE_MAX / sizeof (re_dfastate_t *) <= match_last, 0)) + if (BE (MIN (IDX_MAX, SIZE_MAX / sizeof (re_dfastate_t *)) <= match_last, 0)) return REG_ESPACE; sifted_states = re_malloc (re_dfastate_t *, match_last + 1); @@ -995,9 +1026,9 @@ prune_impossible_nodes (re_match_context_t *mctx) since initial states may have constraints like "\<", "^", etc.. */ static inline re_dfastate_t * -__attribute ((always_inline)) +__attribute__ ((always_inline)) acquire_init_state_context (reg_errcode_t *err, const re_match_context_t *mctx, - int idx) + Idx idx) { const re_dfa_t *const dfa = mctx->dfa; if (dfa->init_state->has_constraint) @@ -1028,27 +1059,27 @@ acquire_init_state_context (reg_errcode_t *err, const re_match_context_t *mctx, } /* Check whether the regular expression match input string INPUT or not, - and return the index where the matching end, return -1 if not match, - or return -2 in case of an error. + and return the index where the matching end. Return -1 if + there is no match, and return -2 in case of an error. FL_LONGEST_MATCH means we want the POSIX longest matching. If P_MATCH_FIRST is not NULL, and the match fails, it is set to the next place where we may want to try matching. - Note that the matcher assume that the maching starts from the current + Note that the matcher assumes that the matching starts from the current index of the buffer. */ -static int +static Idx __attribute_warn_unused_result__ -check_matching (re_match_context_t *mctx, int fl_longest_match, - int *p_match_first) +check_matching (re_match_context_t *mctx, bool fl_longest_match, + Idx *p_match_first) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int match = 0; - int match_last = -1; - int cur_str_idx = re_string_cur_idx (&mctx->input); + Idx match = 0; + Idx match_last = -1; + Idx cur_str_idx = re_string_cur_idx (&mctx->input); re_dfastate_t *cur_state; - int at_init_state = p_match_first != NULL; - int next_start_idx = cur_str_idx; + bool at_init_state = p_match_first != NULL; + Idx next_start_idx = cur_str_idx; err = REG_NOERROR; cur_state = acquire_init_state_context (&err, mctx, cur_str_idx); @@ -1067,7 +1098,7 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, later. E.g. Processing back references. */ if (BE (dfa->nbackref, 0)) { - at_init_state = 0; + at_init_state = false; err = check_subexp_matching_top (mctx, &cur_state->nodes, 0); if (BE (err != REG_NOERROR, 0)) return err; @@ -1100,7 +1131,7 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, while (!re_string_eoi (&mctx->input)) { re_dfastate_t *old_state = cur_state; - int next_char_idx = re_string_cur_idx (&mctx->input) + 1; + Idx next_char_idx = re_string_cur_idx (&mctx->input) + 1; if ((BE (next_char_idx >= mctx->input.bufs_len, 0) && mctx->input.bufs_len < mctx->input.len) @@ -1138,7 +1169,7 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, if (old_state == cur_state) next_start_idx = next_char_idx; else - at_init_state = 0; + at_init_state = false; } if (cur_state->halt) @@ -1169,29 +1200,29 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, /* Check NODE match the current context. */ -static int -check_halt_node_context (const re_dfa_t *dfa, int node, unsigned int context) +static bool +check_halt_node_context (const re_dfa_t *dfa, Idx node, unsigned int context) { re_token_type_t type = dfa->nodes[node].type; unsigned int constraint = dfa->nodes[node].constraint; if (type != END_OF_RE) - return 0; + return false; if (!constraint) - return 1; + return true; if (NOT_SATISFY_NEXT_CONSTRAINT (constraint, context)) - return 0; - return 1; + return false; + return true; } /* Check the halt state STATE match the current context. Return 0 if not match, if the node, STATE has, is a halt node and match the context, return the node. */ -static int +static Idx check_halt_state_context (const re_match_context_t *mctx, - const re_dfastate_t *state, int idx) + const re_dfastate_t *state, Idx idx) { - int i; + Idx i; unsigned int context; #ifdef DEBUG assert (state->halt); @@ -1205,31 +1236,33 @@ check_halt_state_context (const re_match_context_t *mctx, /* Compute the next node to which "NFA" transit from NODE("NFA" is a NFA corresponding to the DFA). - Return the destination node, and update EPS_VIA_NODES, return -1 in case - of errors. */ + Return the destination node, and update EPS_VIA_NODES; + return -1 in case of errors. */ -static int -proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, - int *pidx, int node, re_node_set *eps_via_nodes, +static Idx +proceed_next_node (const re_match_context_t *mctx, Idx nregs, regmatch_t *regs, + Idx *pidx, Idx node, re_node_set *eps_via_nodes, struct re_fail_stack_t *fs) { const re_dfa_t *const dfa = mctx->dfa; - int i, err; + Idx i; + bool ok; if (IS_EPSILON_NODE (dfa->nodes[node].type)) { re_node_set *cur_nodes = &mctx->state_log[*pidx]->nodes; re_node_set *edests = &dfa->edests[node]; - int dest_node; - err = re_node_set_insert (eps_via_nodes, node); - if (BE (err < 0, 0)) + Idx dest_node; + ok = re_node_set_insert (eps_via_nodes, node); + if (BE (! ok, 0)) return -2; - /* Pick up a valid destination, or return -1 if none is found. */ + /* Pick up a valid destination, or return -1 if none + is found. */ for (dest_node = -1, i = 0; i < edests->nelem; ++i) { - int candidate = edests->elems[i]; + Idx candidate = edests->elems[i]; if (!re_node_set_contains (cur_nodes, candidate)) continue; - if (dest_node == -1) + if (dest_node == -1) dest_node = candidate; else @@ -1253,7 +1286,7 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, } else { - int naccepted = 0; + Idx naccepted = 0; re_token_type_t type = dfa->nodes[node].type; #ifdef RE_ENABLE_I18N @@ -1263,7 +1296,7 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, #endif /* RE_ENABLE_I18N */ if (type == OP_BACK_REF) { - int subexp_idx = dfa->nodes[node].opr.idx + 1; + Idx subexp_idx = dfa->nodes[node].opr.idx + 1; naccepted = regs[subexp_idx].rm_eo - regs[subexp_idx].rm_so; if (fs != NULL) { @@ -1280,9 +1313,9 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, if (naccepted == 0) { - int dest_node; - err = re_node_set_insert (eps_via_nodes, node); - if (BE (err < 0, 0)) + Idx dest_node; + ok = re_node_set_insert (eps_via_nodes, node); + if (BE (! ok, 0)) return -2; dest_node = dfa->edests[node].elems[0]; if (re_node_set_contains (&mctx->state_log[*pidx]->nodes, @@ -1294,7 +1327,7 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, if (naccepted != 0 || check_node_accept (mctx, dfa->nodes + node, *pidx)) { - int dest_node = dfa->nexts[node]; + Idx dest_node = dfa->nexts[node]; *pidx = (naccepted == 0) ? *pidx + 1 : *pidx + naccepted; if (fs && (*pidx > mctx->match_last || mctx->state_log[*pidx] == NULL || !re_node_set_contains (&mctx->state_log[*pidx]->nodes, @@ -1309,16 +1342,16 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, static reg_errcode_t __attribute_warn_unused_result__ -push_fail_stack (struct re_fail_stack_t *fs, int str_idx, int dest_node, - int nregs, regmatch_t *regs, re_node_set *eps_via_nodes) +push_fail_stack (struct re_fail_stack_t *fs, Idx str_idx, Idx dest_node, + Idx nregs, regmatch_t *regs, re_node_set *eps_via_nodes) { reg_errcode_t err; - int num = fs->num++; + Idx num = fs->num++; if (fs->num == fs->alloc) { struct re_fail_stack_ent_t *new_array; - new_array = realloc (fs->stack, (sizeof (struct re_fail_stack_ent_t) - * fs->alloc * 2)); + new_array = re_realloc (fs->stack, struct re_fail_stack_ent_t, + fs->alloc * 2); if (new_array == NULL) return REG_ESPACE; fs->alloc *= 2; @@ -1334,11 +1367,11 @@ push_fail_stack (struct re_fail_stack_t *fs, int str_idx, int dest_node, return err; } -static int -pop_fail_stack (struct re_fail_stack_t *fs, int *pidx, int nregs, +static Idx +pop_fail_stack (struct re_fail_stack_t *fs, Idx *pidx, Idx nregs, regmatch_t *regs, re_node_set *eps_via_nodes) { - int num = --fs->num; + Idx num = --fs->num; assert (num >= 0); *pidx = fs->stack[num].idx; memcpy (regs, fs->stack[num].regs, sizeof (regmatch_t) * nregs); @@ -1356,15 +1389,15 @@ pop_fail_stack (struct re_fail_stack_t *fs, int *pidx, int nregs, static reg_errcode_t __attribute_warn_unused_result__ set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, - regmatch_t *pmatch, int fl_backtrack) + regmatch_t *pmatch, bool fl_backtrack) { - const re_dfa_t *dfa = (const re_dfa_t *) preg->buffer; - int idx, cur_node; + const re_dfa_t *dfa = preg->buffer; + Idx idx, cur_node; re_node_set eps_via_nodes; struct re_fail_stack_t *fs; struct re_fail_stack_t fs_body = { 0, 2, NULL }; regmatch_t *prev_idx_match; - int prev_idx_match_malloced = 0; + bool prev_idx_match_malloced = false; #ifdef DEBUG assert (nmatch > 1); @@ -1393,7 +1426,7 @@ set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, free_fail_stack_return (fs); return REG_ESPACE; } - prev_idx_match_malloced = 1; + prev_idx_match_malloced = true; } memcpy (prev_idx_match, pmatch, sizeof (regmatch_t) * nmatch); @@ -1403,7 +1436,7 @@ set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, if (idx == pmatch[0].rm_eo && cur_node == mctx->last_node) { - int reg_idx; + Idx reg_idx; if (fs) { for (reg_idx = 0; reg_idx < nmatch; ++reg_idx) @@ -1465,7 +1498,7 @@ free_fail_stack_return (struct re_fail_stack_t *fs) { if (fs) { - int fs_idx; + Idx fs_idx; for (fs_idx = 0; fs_idx < fs->num; ++fs_idx) { re_node_set_free (&fs->stack[fs_idx].eps_via_nodes); @@ -1478,12 +1511,12 @@ free_fail_stack_return (struct re_fail_stack_t *fs) static void update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, - regmatch_t *prev_idx_match, int cur_node, int cur_idx, int nmatch) + regmatch_t *prev_idx_match, Idx cur_node, Idx cur_idx, Idx nmatch) { int type = dfa->nodes[cur_node].type; if (type == OP_OPEN_SUBEXP) { - int reg_num = dfa->nodes[cur_node].opr.idx + 1; + Idx reg_num = dfa->nodes[cur_node].opr.idx + 1; /* We are at the first node of this sub expression. */ if (reg_num < nmatch) @@ -1494,7 +1527,7 @@ update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, } else if (type == OP_CLOSE_SUBEXP) { - int reg_num = dfa->nodes[cur_node].opr.idx + 1; + Idx reg_num = dfa->nodes[cur_node].opr.idx + 1; if (reg_num < nmatch) { /* We are at the last node of this sub expression. */ @@ -1528,21 +1561,21 @@ update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, and sift the nodes in each states according to the following rules. Updated state_log will be wrote to STATE_LOG. - Rules: We throw away the Node `a' in the STATE_LOG[STR_IDX] if... + Rules: We throw away the Node 'a' in the STATE_LOG[STR_IDX] if... 1. When STR_IDX == MATCH_LAST(the last index in the state_log): - If `a' isn't the LAST_NODE and `a' can't epsilon transit to - the LAST_NODE, we throw away the node `a'. - 2. When 0 <= STR_IDX < MATCH_LAST and `a' accepts - string `s' and transit to `b': + If 'a' isn't the LAST_NODE and 'a' can't epsilon transit to + the LAST_NODE, we throw away the node 'a'. + 2. When 0 <= STR_IDX < MATCH_LAST and 'a' accepts + string 's' and transit to 'b': i. If 'b' isn't in the STATE_LOG[STR_IDX+strlen('s')], we throw - away the node `a'. + away the node 'a'. ii. If 'b' is in the STATE_LOG[STR_IDX+strlen('s')] but 'b' is - thrown away, we throw away the node `a'. + thrown away, we throw away the node 'a'. 3. When 0 <= STR_IDX < MATCH_LAST and 'a' epsilon transit to 'b': i. If 'b' isn't in the STATE_LOG[STR_IDX], we throw away the - node `a'. + node 'a'. ii. If 'b' is in the STATE_LOG[STR_IDX] but 'b' is thrown away, - we throw away the node `a'. */ + we throw away the node 'a'. */ #define STATE_NODE_CONTAINS(state,node) \ ((state) != NULL && re_node_set_contains (&(state)->nodes, node)) @@ -1552,7 +1585,7 @@ sift_states_backward (const re_match_context_t *mctx, re_sift_context_t *sctx) { reg_errcode_t err; int null_cnt = 0; - int str_idx = sctx->last_str_idx; + Idx str_idx = sctx->last_str_idx; re_node_set cur_dest; #ifdef DEBUG @@ -1607,31 +1640,31 @@ sift_states_backward (const re_match_context_t *mctx, re_sift_context_t *sctx) static reg_errcode_t __attribute_warn_unused_result__ build_sifted_states (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, re_node_set *cur_dest) + Idx str_idx, re_node_set *cur_dest) { const re_dfa_t *const dfa = mctx->dfa; const re_node_set *cur_src = &mctx->state_log[str_idx]->non_eps_nodes; - int i; + Idx i; /* Then build the next sifted state. - We build the next sifted state on `cur_dest', and update - `sifted_states[str_idx]' with `cur_dest'. + We build the next sifted state on 'cur_dest', and update + 'sifted_states[str_idx]' with 'cur_dest'. Note: - `cur_dest' is the sifted state from `state_log[str_idx + 1]'. - `cur_src' points the node_set of the old `state_log[str_idx]' + 'cur_dest' is the sifted state from 'state_log[str_idx + 1]'. + 'cur_src' points the node_set of the old 'state_log[str_idx]' (with the epsilon nodes pre-filtered out). */ for (i = 0; i < cur_src->nelem; i++) { - int prev_node = cur_src->elems[i]; + Idx prev_node = cur_src->elems[i]; int naccepted = 0; - int ret; + bool ok; #ifdef DEBUG re_token_type_t type = dfa->nodes[prev_node].type; assert (!IS_EPSILON_NODE (type)); #endif #ifdef RE_ENABLE_I18N - /* If the node may accept `multi byte'. */ + /* If the node may accept "multi byte". */ if (dfa->nodes[prev_node].accept_mb) naccepted = sift_states_iter_mb (mctx, sctx, prev_node, str_idx, sctx->last_str_idx); @@ -1650,14 +1683,14 @@ build_sifted_states (const re_match_context_t *mctx, re_sift_context_t *sctx, if (sctx->limits.nelem) { - int to_idx = str_idx + naccepted; + Idx to_idx = str_idx + naccepted; if (check_dst_limits (mctx, &sctx->limits, dfa->nexts[prev_node], to_idx, prev_node, str_idx)) continue; } - ret = re_node_set_insert (cur_dest, prev_node); - if (BE (ret == -1, 0)) + ok = re_node_set_insert (cur_dest, prev_node); + if (BE (! ok, 0)) return REG_ESPACE; } @@ -1667,9 +1700,9 @@ build_sifted_states (const re_match_context_t *mctx, re_sift_context_t *sctx, /* Helper functions. */ static reg_errcode_t -clean_state_log_if_needed (re_match_context_t *mctx, int next_state_log_idx) +clean_state_log_if_needed (re_match_context_t *mctx, Idx next_state_log_idx) { - int top = mctx->state_log_top; + Idx top = mctx->state_log_top; if ((next_state_log_idx >= mctx->input.bufs_len && mctx->input.bufs_len < mctx->input.len) @@ -1693,9 +1726,9 @@ clean_state_log_if_needed (re_match_context_t *mctx, int next_state_log_idx) static reg_errcode_t merge_state_array (const re_dfa_t *dfa, re_dfastate_t **dst, - re_dfastate_t **src, int num) + re_dfastate_t **src, Idx num) { - int st_idx; + Idx st_idx; reg_errcode_t err; for (st_idx = 0; st_idx < num; ++st_idx) { @@ -1719,7 +1752,7 @@ merge_state_array (const re_dfa_t *dfa, re_dfastate_t **dst, static reg_errcode_t update_cur_sifted_state (const re_match_context_t *mctx, - re_sift_context_t *sctx, int str_idx, + re_sift_context_t *sctx, Idx str_idx, re_node_set *dest_nodes) { const re_dfa_t *const dfa = mctx->dfa; @@ -1770,7 +1803,7 @@ add_epsilon_src_nodes (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates) { reg_errcode_t err = REG_NOERROR; - int i; + Idx i; re_dfastate_t *state = re_acquire_state (&err, dfa, dest_nodes); if (BE (err != REG_NOERROR, 0)) @@ -1794,23 +1827,23 @@ add_epsilon_src_nodes (const re_dfa_t *dfa, re_node_set *dest_nodes, } static reg_errcode_t -sub_epsilon_src_nodes (const re_dfa_t *dfa, int node, re_node_set *dest_nodes, +sub_epsilon_src_nodes (const re_dfa_t *dfa, Idx node, re_node_set *dest_nodes, const re_node_set *candidates) { - int ecl_idx; + Idx ecl_idx; reg_errcode_t err; re_node_set *inv_eclosure = dfa->inveclosures + node; re_node_set except_nodes; re_node_set_init_empty (&except_nodes); for (ecl_idx = 0; ecl_idx < inv_eclosure->nelem; ++ecl_idx) { - int cur_node = inv_eclosure->elems[ecl_idx]; + Idx cur_node = inv_eclosure->elems[ecl_idx]; if (cur_node == node) continue; if (IS_EPSILON_NODE (dfa->nodes[cur_node].type)) { - int edst1 = dfa->edests[cur_node].elems[0]; - int edst2 = ((dfa->edests[cur_node].nelem > 1) + Idx edst1 = dfa->edests[cur_node].elems[0]; + Idx edst2 = ((dfa->edests[cur_node].nelem > 1) ? dfa->edests[cur_node].elems[1] : -1); if ((!re_node_set_contains (inv_eclosure, edst1) && re_node_set_contains (dest_nodes, edst1)) @@ -1830,10 +1863,10 @@ sub_epsilon_src_nodes (const re_dfa_t *dfa, int node, re_node_set *dest_nodes, } for (ecl_idx = 0; ecl_idx < inv_eclosure->nelem; ++ecl_idx) { - int cur_node = inv_eclosure->elems[ecl_idx]; + Idx cur_node = inv_eclosure->elems[ecl_idx]; if (!re_node_set_contains (&except_nodes, cur_node)) { - int idx = re_node_set_contains (dest_nodes, cur_node) - 1; + Idx idx = re_node_set_contains (dest_nodes, cur_node) - 1; re_node_set_remove_at (dest_nodes, idx); } } @@ -1841,18 +1874,18 @@ sub_epsilon_src_nodes (const re_dfa_t *dfa, int node, re_node_set *dest_nodes, return REG_NOERROR; } -static int -check_dst_limits (const re_match_context_t *mctx, re_node_set *limits, - int dst_node, int dst_idx, int src_node, int src_idx) +static bool +check_dst_limits (const re_match_context_t *mctx, const re_node_set *limits, + Idx dst_node, Idx dst_idx, Idx src_node, Idx src_idx) { const re_dfa_t *const dfa = mctx->dfa; - int lim_idx, src_pos, dst_pos; + Idx lim_idx, src_pos, dst_pos; - int dst_bkref_idx = search_cur_bkref_entry (mctx, dst_idx); - int src_bkref_idx = search_cur_bkref_entry (mctx, src_idx); + Idx dst_bkref_idx = search_cur_bkref_entry (mctx, dst_idx); + Idx src_bkref_idx = search_cur_bkref_entry (mctx, src_idx); for (lim_idx = 0; lim_idx < limits->nelem; ++lim_idx) { - int subexp_idx; + Idx subexp_idx; struct re_backref_cache_entry *ent; ent = mctx->bkref_ents + limits->elems[lim_idx]; subexp_idx = dfa->nodes[ent->node].opr.idx; @@ -1871,24 +1904,24 @@ check_dst_limits (const re_match_context_t *mctx, re_node_set *limits, if (src_pos == dst_pos) continue; /* This is unrelated limitation. */ else - return 1; + return true; } - return 0; + return false; } static int check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, int boundaries, - int subexp_idx, int from_node, int bkref_idx) + Idx subexp_idx, Idx from_node, Idx bkref_idx) { const re_dfa_t *const dfa = mctx->dfa; const re_node_set *eclosures = dfa->eclosures + from_node; - int node_idx; + Idx node_idx; /* Else, we are on the boundary: examine the nodes on the epsilon closure. */ for (node_idx = 0; node_idx < eclosures->nelem; ++node_idx) { - int node = eclosures->elems[node_idx]; + Idx node = eclosures->elems[node_idx]; switch (dfa->nodes[node].type) { case OP_BACK_REF: @@ -1897,7 +1930,8 @@ check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, int boundaries, struct re_backref_cache_entry *ent = mctx->bkref_ents + bkref_idx; do { - int dst, cpos; + Idx dst; + int cpos; if (ent->node != node) continue; @@ -1957,9 +1991,9 @@ check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, int boundaries, } static int -check_dst_limits_calc_pos (const re_match_context_t *mctx, int limit, - int subexp_idx, int from_node, int str_idx, - int bkref_idx) +check_dst_limits_calc_pos (const re_match_context_t *mctx, Idx limit, + Idx subexp_idx, Idx from_node, Idx str_idx, + Idx bkref_idx) { struct re_backref_cache_entry *lim = mctx->bkref_ents + limit; int boundaries; @@ -1988,14 +2022,14 @@ check_dst_limits_calc_pos (const re_match_context_t *mctx, int limit, static reg_errcode_t check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates, re_node_set *limits, - struct re_backref_cache_entry *bkref_ents, int str_idx) + struct re_backref_cache_entry *bkref_ents, Idx str_idx) { reg_errcode_t err; - int node_idx, lim_idx; + Idx node_idx, lim_idx; for (lim_idx = 0; lim_idx < limits->nelem; ++lim_idx) { - int subexp_idx; + Idx subexp_idx; struct re_backref_cache_entry *ent; ent = bkref_ents + limits->elems[lim_idx]; @@ -2005,11 +2039,11 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, subexp_idx = dfa->nodes[ent->node].opr.idx; if (ent->subexp_to == str_idx) { - int ops_node = -1; - int cls_node = -1; + Idx ops_node = -1; + Idx cls_node = -1; for (node_idx = 0; node_idx < dest_nodes->nelem; ++node_idx) { - int node = dest_nodes->elems[node_idx]; + Idx node = dest_nodes->elems[node_idx]; re_token_type_t type = dfa->nodes[node].type; if (type == OP_OPEN_SUBEXP && subexp_idx == dfa->nodes[node].opr.idx) @@ -2033,7 +2067,7 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, if (cls_node >= 0) for (node_idx = 0; node_idx < dest_nodes->nelem; ++node_idx) { - int node = dest_nodes->elems[node_idx]; + Idx node = dest_nodes->elems[node_idx]; if (!re_node_set_contains (dfa->inveclosures + node, cls_node) && !re_node_set_contains (dfa->eclosures + node, @@ -2053,7 +2087,7 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, { for (node_idx = 0; node_idx < dest_nodes->nelem; ++node_idx) { - int node = dest_nodes->elems[node_idx]; + Idx node = dest_nodes->elems[node_idx]; re_token_type_t type = dfa->nodes[node].type; if (type == OP_CLOSE_SUBEXP || type == OP_OPEN_SUBEXP) { @@ -2075,13 +2109,13 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, static reg_errcode_t __attribute_warn_unused_result__ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, const re_node_set *candidates) + Idx str_idx, const re_node_set *candidates) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int node_idx, node; + Idx node_idx, node; re_sift_context_t local_sctx; - int first_idx = search_cur_bkref_entry (mctx, str_idx); + Idx first_idx = search_cur_bkref_entry (mctx, str_idx); if (first_idx == -1) return REG_NOERROR; @@ -2090,7 +2124,7 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, for (node_idx = 0; node_idx < candidates->nelem; ++node_idx) { - int enabled_idx; + Idx enabled_idx; re_token_type_t type; struct re_backref_cache_entry *entry; node = candidates->elems[node_idx]; @@ -2105,10 +2139,10 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, enabled_idx = first_idx; do { - int subexp_len; - int to_idx; - int dst_node; - int ret; + Idx subexp_len; + Idx to_idx; + Idx dst_node; + bool ok; re_dfastate_t *cur_state; if (entry->node != node) @@ -2134,8 +2168,8 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, } local_sctx.last_node = node; local_sctx.last_str_idx = str_idx; - ret = re_node_set_insert (&local_sctx.limits, enabled_idx); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (&local_sctx.limits, enabled_idx); + if (BE (! ok, 0)) { err = REG_ESPACE; goto free_return; @@ -2174,21 +2208,21 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, #ifdef RE_ENABLE_I18N static int sift_states_iter_mb (const re_match_context_t *mctx, re_sift_context_t *sctx, - int node_idx, int str_idx, int max_str_idx) + Idx node_idx, Idx str_idx, Idx max_str_idx) { const re_dfa_t *const dfa = mctx->dfa; int naccepted; - /* Check the node can accept `multi byte'. */ + /* Check the node can accept "multi byte". */ naccepted = check_node_accept_bytes (dfa, node_idx, &mctx->input, str_idx); if (naccepted > 0 && str_idx + naccepted <= max_str_idx && !STATE_NODE_CONTAINS (sctx->sifted_states[str_idx + naccepted], dfa->nexts[node_idx])) - /* The node can't accept the `multi byte', or the + /* The node can't accept the "multi byte", or the destination was already thrown away, then the node - could't accept the current input `multi byte'. */ + could't accept the current input "multi byte". */ naccepted = 0; /* Otherwise, it is sure that the node could accept - `naccepted' bytes input. */ + 'naccepted' bytes input. */ return naccepted; } #endif /* RE_ENABLE_I18N */ @@ -2259,12 +2293,12 @@ transit_state (reg_errcode_t *err, re_match_context_t *mctx, } /* Update the state_log if we need */ -re_dfastate_t * +static re_dfastate_t * merge_state_with_log (reg_errcode_t *err, re_match_context_t *mctx, re_dfastate_t *next_state) { const re_dfa_t *const dfa = mctx->dfa; - int cur_idx = re_string_cur_idx (&mctx->input); + Idx cur_idx = re_string_cur_idx (&mctx->input); if (cur_idx > mctx->state_log_top) { @@ -2337,14 +2371,14 @@ merge_state_with_log (reg_errcode_t *err, re_match_context_t *mctx, /* Skip bytes in the input that correspond to part of a multi-byte match, then look in the log for a state from which to restart matching. */ -re_dfastate_t * +static re_dfastate_t * find_recover_state (reg_errcode_t *err, re_match_context_t *mctx) { re_dfastate_t *cur_state; do { - int max = mctx->state_log_top; - int cur_str_idx = re_string_cur_idx (&mctx->input); + Idx max = mctx->state_log_top; + Idx cur_str_idx = re_string_cur_idx (&mctx->input); do { @@ -2369,10 +2403,10 @@ find_recover_state (reg_errcode_t *err, re_match_context_t *mctx) static reg_errcode_t check_subexp_matching_top (re_match_context_t *mctx, re_node_set *cur_nodes, - int str_idx) + Idx str_idx) { const re_dfa_t *const dfa = mctx->dfa; - int node_idx; + Idx node_idx; reg_errcode_t err; /* TODO: This isn't efficient. @@ -2382,7 +2416,7 @@ check_subexp_matching_top (re_match_context_t *mctx, re_node_set *cur_nodes, E.g. RE: (a){2} */ for (node_idx = 0; node_idx < cur_nodes->nelem; ++node_idx) { - int node = cur_nodes->elems[node_idx]; + Idx node = cur_nodes->elems[node_idx]; if (dfa->nodes[node].type == OP_OPEN_SUBEXP && dfa->nodes[node].opr.idx < BITSET_WORD_BITS && (dfa->used_bkref_map @@ -2407,7 +2441,7 @@ transit_state_sb (reg_errcode_t *err, re_match_context_t *mctx, const re_dfa_t *const dfa = mctx->dfa; re_node_set next_nodes; re_dfastate_t *next_state; - int node_cnt, cur_str_idx = re_string_cur_idx (&mctx->input); + Idx node_cnt, cur_str_idx = re_string_cur_idx (&mctx->input); unsigned int context; *err = re_node_set_alloc (&next_nodes, state->nodes.nelem + 1); @@ -2415,7 +2449,7 @@ transit_state_sb (reg_errcode_t *err, re_match_context_t *mctx, return NULL; for (node_cnt = 0; node_cnt < state->nodes.nelem; ++node_cnt) { - int cur_node = state->nodes.elems[node_cnt]; + Idx cur_node = state->nodes.elems[node_cnt]; if (check_node_accept (mctx, dfa->nodes + cur_node, cur_str_idx)) { *err = re_node_set_merge (&next_nodes, @@ -2444,13 +2478,14 @@ transit_state_mb (re_match_context_t *mctx, re_dfastate_t *pstate) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int i; + Idx i; for (i = 0; i < pstate->nodes.nelem; ++i) { re_node_set dest_nodes, *new_nodes; - int cur_node_idx = pstate->nodes.elems[i]; - int naccepted, dest_idx; + Idx cur_node_idx = pstate->nodes.elems[i]; + int naccepted; + Idx dest_idx; unsigned int context; re_dfastate_t *dest_state; @@ -2473,7 +2508,7 @@ transit_state_mb (re_match_context_t *mctx, re_dfastate_t *pstate) if (naccepted == 0) continue; - /* The node can accepts `naccepted' bytes. */ + /* The node can accepts 'naccepted' bytes. */ dest_idx = re_string_cur_idx (&mctx->input) + naccepted; mctx->max_mb_elem_len = ((mctx->max_mb_elem_len < naccepted) ? naccepted : mctx->max_mb_elem_len); @@ -2513,18 +2548,18 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int i; - int cur_str_idx = re_string_cur_idx (&mctx->input); + Idx i; + Idx cur_str_idx = re_string_cur_idx (&mctx->input); for (i = 0; i < nodes->nelem; ++i) { - int dest_str_idx, prev_nelem, bkc_idx; - int node_idx = nodes->elems[i]; + Idx dest_str_idx, prev_nelem, bkc_idx; + Idx node_idx = nodes->elems[i]; unsigned int context; const re_token_t *node = dfa->nodes + node_idx; re_node_set *new_dest_nodes; - /* Check whether `node' is a backreference or not. */ + /* Check whether 'node' is a backreference or not. */ if (node->type != OP_BACK_REF) continue; @@ -2536,21 +2571,21 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) continue; } - /* `node' is a backreference. + /* 'node' is a backreference. Check the substring which the substring matched. */ bkc_idx = mctx->nbkref_ents; err = get_subexp (mctx, node_idx, cur_str_idx); if (BE (err != REG_NOERROR, 0)) goto free_return; - /* And add the epsilon closures (which is `new_dest_nodes') of + /* And add the epsilon closures (which is 'new_dest_nodes') of the backreference to appropriate state_log. */ #ifdef DEBUG assert (dfa->nexts[node_idx] != -1); #endif for (; bkc_idx < mctx->nbkref_ents; ++bkc_idx) { - int subexp_len; + Idx subexp_len; re_dfastate_t *dest_state; struct re_backref_cache_entry *bkref_ent; bkref_ent = mctx->bkref_ents + bkc_idx; @@ -2567,7 +2602,7 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) dest_state = mctx->state_log[dest_str_idx]; prev_nelem = ((mctx->state_log[cur_str_idx] == NULL) ? 0 : mctx->state_log[cur_str_idx]->nodes.nelem); - /* Add `new_dest_node' to state_log. */ + /* Add 'new_dest_node' to state_log. */ if (dest_state == NULL) { mctx->state_log[dest_str_idx] @@ -2623,13 +2658,13 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) static reg_errcode_t __attribute_warn_unused_result__ -get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) +get_subexp (re_match_context_t *mctx, Idx bkref_node, Idx bkref_str_idx) { const re_dfa_t *const dfa = mctx->dfa; - int subexp_num, sub_top_idx; + Idx subexp_num, sub_top_idx; const char *buf = (const char *) re_string_get_buffer (&mctx->input); /* Return if we have already checked BKREF_NODE at BKREF_STR_IDX. */ - int cache_idx = search_cur_bkref_entry (mctx, bkref_str_idx); + Idx cache_idx = search_cur_bkref_entry (mctx, bkref_str_idx); if (cache_idx != -1) { const struct re_backref_cache_entry *entry @@ -2648,7 +2683,7 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) reg_errcode_t err; re_sub_match_top_t *sub_top = mctx->sub_tops[sub_top_idx]; re_sub_match_last_t *sub_last; - int sub_last_idx, sl_str, bkref_str_off; + Idx sub_last_idx, sl_str, bkref_str_off; if (dfa->nodes[sub_top->node].opr.idx != subexp_num) continue; /* It isn't related. */ @@ -2659,7 +2694,7 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) evaluated. */ for (sub_last_idx = 0; sub_last_idx < sub_top->nlasts; ++sub_last_idx) { - int sl_str_diff; + regoff_t sl_str_diff; sub_last = sub_top->lasts[sub_last_idx]; sl_str_diff = sub_last->str_idx - sl_str; /* The matched string by the sub expression match with the substring @@ -2705,7 +2740,8 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) /* Then, search for the other last nodes of the sub expression. */ for (; sl_str <= bkref_str_idx; ++sl_str) { - int cls_node, sl_str_off; + Idx cls_node; + regoff_t sl_str_off; const re_node_set *nodes; sl_str_off = sl_str - sub_top->str_idx; /* The matched string by the sub expression match with the substring @@ -2772,10 +2808,10 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) static reg_errcode_t get_subexp_sub (re_match_context_t *mctx, const re_sub_match_top_t *sub_top, - re_sub_match_last_t *sub_last, int bkref_node, int bkref_str) + re_sub_match_last_t *sub_last, Idx bkref_node, Idx bkref_str) { reg_errcode_t err; - int to_idx; + Idx to_idx; /* Can the subexpression arrive the back reference? */ err = check_arrival (mctx, &sub_last->path, sub_last->node, sub_last->str_idx, bkref_node, bkref_str, @@ -2798,14 +2834,14 @@ get_subexp_sub (re_match_context_t *mctx, const re_sub_match_top_t *sub_top, nodes. E.g. RE: (a){2} */ -static int +static Idx find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, - int subexp_idx, int type) + Idx subexp_idx, int type) { - int cls_idx; + Idx cls_idx; for (cls_idx = 0; cls_idx < nodes->nelem; ++cls_idx) { - int cls_node = nodes->elems[cls_idx]; + Idx cls_node = nodes->elems[cls_idx]; const re_token_t *node = dfa->nodes + cls_node; if (node->type == type && node->opr.idx == subexp_idx) @@ -2821,12 +2857,12 @@ find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, static reg_errcode_t __attribute_warn_unused_result__ -check_arrival (re_match_context_t *mctx, state_array_t *path, int top_node, - int top_str, int last_node, int last_str, int type) +check_arrival (re_match_context_t *mctx, state_array_t *path, Idx top_node, + Idx top_str, Idx last_node, Idx last_str, int type) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err = REG_NOERROR; - int subexp_num, backup_cur_idx, str_idx, null_cnt; + Idx subexp_num, backup_cur_idx, str_idx, null_cnt; re_dfastate_t *cur_state = NULL; re_node_set *cur_nodes, next_nodes; re_dfastate_t **backup_state_log; @@ -2837,20 +2873,24 @@ check_arrival (re_match_context_t *mctx, state_array_t *path, int top_node, if (BE (path->alloc < last_str + mctx->max_mb_elem_len + 1, 0)) { re_dfastate_t **new_array; - int old_alloc = path->alloc; - path->alloc += last_str + mctx->max_mb_elem_len + 1; - new_array = re_realloc (path->array, re_dfastate_t *, path->alloc); + Idx old_alloc = path->alloc; + Idx incr_alloc = last_str + mctx->max_mb_elem_len + 1; + Idx new_alloc; + if (BE (IDX_MAX - old_alloc < incr_alloc, 0)) + return REG_ESPACE; + new_alloc = old_alloc + incr_alloc; + if (BE (SIZE_MAX / sizeof (re_dfastate_t *) < new_alloc, 0)) + return REG_ESPACE; + new_array = re_realloc (path->array, re_dfastate_t *, new_alloc); if (BE (new_array == NULL, 0)) - { - path->alloc = old_alloc; - return REG_ESPACE; - } + return REG_ESPACE; path->array = new_array; + path->alloc = new_alloc; memset (new_array + old_alloc, '\0', sizeof (re_dfastate_t *) * (path->alloc - old_alloc)); } - str_idx = path->next_idx ?: top_str; + str_idx = path->next_idx ? path->next_idx : top_str; /* Temporary modify MCTX. */ backup_state_log = mctx->state_log; @@ -2982,12 +3022,12 @@ check_arrival (re_match_context_t *mctx, state_array_t *path, int top_node, static reg_errcode_t __attribute_warn_unused_result__ -check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, +check_arrival_add_next_nodes (re_match_context_t *mctx, Idx str_idx, re_node_set *cur_nodes, re_node_set *next_nodes) { const re_dfa_t *const dfa = mctx->dfa; - int result; - int cur_idx; + bool ok; + Idx cur_idx; #ifdef RE_ENABLE_I18N reg_errcode_t err = REG_NOERROR; #endif @@ -2996,13 +3036,13 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, for (cur_idx = 0; cur_idx < cur_nodes->nelem; ++cur_idx) { int naccepted = 0; - int cur_node = cur_nodes->elems[cur_idx]; + Idx cur_node = cur_nodes->elems[cur_idx]; #ifdef DEBUG re_token_type_t type = dfa->nodes[cur_node].type; assert (!IS_EPSILON_NODE (type)); #endif #ifdef RE_ENABLE_I18N - /* If the node may accept `multi byte'. */ + /* If the node may accept "multi byte". */ if (dfa->nodes[cur_node].accept_mb) { naccepted = check_node_accept_bytes (dfa, cur_node, &mctx->input, @@ -3010,8 +3050,8 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, if (naccepted > 1) { re_dfastate_t *dest_state; - int next_node = dfa->nexts[cur_node]; - int next_idx = str_idx + naccepted; + Idx next_node = dfa->nexts[cur_node]; + Idx next_idx = str_idx + naccepted; dest_state = mctx->state_log[next_idx]; re_node_set_empty (&union_set); if (dest_state) @@ -3023,8 +3063,8 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, return err; } } - result = re_node_set_insert (&union_set, next_node); - if (BE (result < 0, 0)) + ok = re_node_set_insert (&union_set, next_node); + if (BE (! ok, 0)) { re_node_set_free (&union_set); return REG_ESPACE; @@ -3043,8 +3083,8 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, if (naccepted || check_node_accept (mctx, dfa->nodes + cur_node, str_idx)) { - result = re_node_set_insert (next_nodes, dfa->nexts[cur_node]); - if (BE (result < 0, 0)) + ok = re_node_set_insert (next_nodes, dfa->nexts[cur_node]); + if (BE (! ok, 0)) { re_node_set_free (&union_set); return REG_ESPACE; @@ -3063,10 +3103,10 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, static reg_errcode_t check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, - int ex_subexp, int type) + Idx ex_subexp, int type) { reg_errcode_t err; - int idx, outside_node; + Idx idx, outside_node; re_node_set new_nodes; #ifdef DEBUG assert (cur_nodes->nelem); @@ -3079,7 +3119,7 @@ check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, for (idx = 0; idx < cur_nodes->nelem; ++idx) { - int cur_node = cur_nodes->elems[idx]; + Idx cur_node = cur_nodes->elems[idx]; const re_node_set *eclosure = dfa->eclosures + cur_node; outside_node = find_subexp_node (dfa, eclosure, ex_subexp, type); if (outside_node == -1) @@ -3116,31 +3156,32 @@ check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, static reg_errcode_t __attribute_warn_unused_result__ check_arrival_expand_ecl_sub (const re_dfa_t *dfa, re_node_set *dst_nodes, - int target, int ex_subexp, int type) + Idx target, Idx ex_subexp, int type) { - int cur_node; + Idx cur_node; for (cur_node = target; !re_node_set_contains (dst_nodes, cur_node);) { - int err; + bool ok; if (dfa->nodes[cur_node].type == type && dfa->nodes[cur_node].opr.idx == ex_subexp) { if (type == OP_CLOSE_SUBEXP) { - err = re_node_set_insert (dst_nodes, cur_node); - if (BE (err == -1, 0)) + ok = re_node_set_insert (dst_nodes, cur_node); + if (BE (! ok, 0)) return REG_ESPACE; } break; } - err = re_node_set_insert (dst_nodes, cur_node); - if (BE (err == -1, 0)) + ok = re_node_set_insert (dst_nodes, cur_node); + if (BE (! ok, 0)) return REG_ESPACE; if (dfa->edests[cur_node].nelem == 0) break; if (dfa->edests[cur_node].nelem == 2) { + reg_errcode_t err; err = check_arrival_expand_ecl_sub (dfa, dst_nodes, dfa->edests[cur_node].elems[1], ex_subexp, type); @@ -3160,11 +3201,11 @@ check_arrival_expand_ecl_sub (const re_dfa_t *dfa, re_node_set *dst_nodes, static reg_errcode_t __attribute_warn_unused_result__ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, - int cur_str, int subexp_num, int type) + Idx cur_str, Idx subexp_num, int type) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int cache_idx_start = search_cur_bkref_entry (mctx, cur_str); + Idx cache_idx_start = search_cur_bkref_entry (mctx, cur_str); struct re_backref_cache_entry *ent; if (cache_idx_start == -1) @@ -3174,7 +3215,7 @@ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, ent = mctx->bkref_ents + cache_idx_start; do { - int to_idx, next_node; + Idx to_idx, next_node; /* Is this entry ENT is appropriate? */ if (!re_node_set_contains (cur_nodes, ent->node)) @@ -3212,14 +3253,14 @@ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, next_node = dfa->nexts[ent->node]; if (mctx->state_log[to_idx]) { - int ret; + bool ok; if (re_node_set_contains (&mctx->state_log[to_idx]->nodes, next_node)) continue; err = re_node_set_init_copy (&union_set, &mctx->state_log[to_idx]->nodes); - ret = re_node_set_insert (&union_set, next_node); - if (BE (err != REG_NOERROR || ret < 0, 0)) + ok = re_node_set_insert (&union_set, next_node); + if (BE (err != REG_NOERROR || ! ok, 0)) { re_node_set_free (&union_set); err = err != REG_NOERROR ? err : REG_ESPACE; @@ -3244,17 +3285,19 @@ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, } /* Build transition table for the state. - Return 1 if succeeded, otherwise return NULL. */ + Return true if successful. */ -static int +static bool build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) { reg_errcode_t err; - int i, j, ch, need_word_trtable = 0; + Idx i, j; + int ch; + bool need_word_trtable = false; bitset_word_t elem, mask; bool dests_node_malloced = false; bool dest_states_malloced = false; - int ndests; /* Number of the destination states from `state'. */ + Idx ndests; /* Number of the destination states from 'state'. */ re_dfastate_t **trtable; re_dfastate_t **dest_states = NULL, **dest_states_word, **dest_states_nl; re_node_set follows, *dests_node; @@ -3268,8 +3311,8 @@ build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) } *dests_alloc; /* We build DFA states which corresponds to the destination nodes - from `state'. `dests_node[i]' represents the nodes which i-th - destination state contains, and `dests_ch[i]' represents the + from 'state'. 'dests_node[i]' represents the nodes which i-th + destination state contains, and 'dests_ch[i]' represents the characters which i-th destination state accepts. */ if (__libc_use_alloca (sizeof (struct dests_alloc))) dests_alloc = (struct dests_alloc *) alloca (sizeof (struct dests_alloc)); @@ -3277,32 +3320,32 @@ build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) { dests_alloc = re_malloc (struct dests_alloc, 1); if (BE (dests_alloc == NULL, 0)) - return 0; + return false; dests_node_malloced = true; } dests_node = dests_alloc->dests_node; dests_ch = dests_alloc->dests_ch; - /* Initialize transiton table. */ + /* Initialize transition table. */ state->word_trtable = state->trtable = NULL; - /* At first, group all nodes belonging to `state' into several + /* At first, group all nodes belonging to 'state' into several destinations. */ ndests = group_nodes_into_DFAstates (dfa, state, dests_node, dests_ch); if (BE (ndests <= 0, 0)) { if (dests_node_malloced) - free (dests_alloc); - /* Return 0 in case of an error, 1 otherwise. */ + re_free (dests_alloc); + /* Return false in case of an error, true otherwise. */ if (ndests == 0) { state->trtable = (re_dfastate_t **) calloc (sizeof (re_dfastate_t *), SBC_MAX); - if (BE (state->trtable == NULL, 0)) - return 0; - return 1; + if (BE (state->trtable == NULL, 0)) + return false; + return true; } - return 0; + return false; } err = re_node_set_alloc (&follows, ndests + 1); @@ -3322,19 +3365,18 @@ build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) alloca (ndests * 3 * sizeof (re_dfastate_t *)); else { - dest_states = (re_dfastate_t **) - malloc (ndests * 3 * sizeof (re_dfastate_t *)); + dest_states = re_malloc (re_dfastate_t *, ndests * 3); if (BE (dest_states == NULL, 0)) { out_free: if (dest_states_malloced) - free (dest_states); + re_free (dest_states); re_node_set_free (&follows); for (i = 0; i < ndests; ++i) re_node_set_free (dests_node + i); if (dests_node_malloced) - free (dests_alloc); - return 0; + re_free (dests_alloc); + return false; } dest_states_malloced = true; } @@ -3345,7 +3387,7 @@ out_free: /* Then build the states for all destinations. */ for (i = 0; i < ndests; ++i) { - int next_node; + Idx next_node; re_node_set_empty (&follows); /* Merge the follows of this destination states. */ for (j = 0; j < dests_node[i].nelem; ++j) @@ -3371,13 +3413,13 @@ out_free: goto out_free; if (dest_states[i] != dest_states_word[i] && dfa->mb_cur_max > 1) - need_word_trtable = 1; + need_word_trtable = true; dest_states_nl[i] = re_acquire_state_context (&err, dfa, &follows, CONTEXT_NEWLINE); if (BE (dest_states_nl[i] == NULL && err != REG_NOERROR, 0)) goto out_free; - } + } else { dest_states_word[i] = dest_states[i]; @@ -3464,16 +3506,16 @@ out_free: } if (dest_states_malloced) - free (dest_states); + re_free (dest_states); re_node_set_free (&follows); for (i = 0; i < ndests; ++i) re_node_set_free (dests_node + i); if (dests_node_malloced) - free (dests_alloc); + re_free (dests_alloc); - return 1; + return true; } /* Group all nodes belonging to STATE into several destinations. @@ -3481,20 +3523,20 @@ out_free: to DESTS_NODE[i] and set the characters accepted by the destination to DEST_CH[i]. This function return the number of destinations. */ -static int +static Idx group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, re_node_set *dests_node, bitset_t *dests_ch) { reg_errcode_t err; - int result; - int i, j, k; - int ndests; /* Number of the destinations from `state'. */ + bool ok; + Idx i, j, k; + Idx ndests; /* Number of the destinations from 'state'. */ bitset_t accepts; /* Characters a node can accept. */ const re_node_set *cur_nodes = &state->nodes; bitset_empty (accepts); ndests = 0; - /* For all the nodes belonging to `state', */ + /* For all the nodes belonging to 'state', */ for (i = 0; i < cur_nodes->nelem; ++i) { re_token_t *node = &dfa->nodes[cur_nodes->elems[i]]; @@ -3524,7 +3566,10 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, #ifdef RE_ENABLE_I18N else if (type == OP_UTF8_PERIOD) { - memset (accepts, '\xff', sizeof (bitset_t) / 2); + if (ASCII_CHARS % BITSET_WORD_BITS == 0) + memset (accepts, -1, ASCII_CHARS / CHAR_BIT); + else + bitset_merge (accepts, utf8_sb_map); if (!(dfa->syntax & RE_DOT_NEWLINE)) bitset_clear (accepts, '\n'); if (dfa->syntax & RE_DOT_NOT_NULL) @@ -3534,7 +3579,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, else continue; - /* Check the `accepts' and sift the characters which are not + /* Check the 'accepts' and sift the characters which are not match it the context. */ if (constraint) { @@ -3593,7 +3638,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, } } - /* Then divide `accepts' into DFA states, or create a new + /* Then divide 'accepts' into DFA states, or create a new state. Above, we make sure that accepts is not empty. */ for (j = 0; j < ndests; ++j) { @@ -3606,7 +3651,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, if (type == CHARACTER && !bitset_contain (dests_ch[j], node->opr.c)) continue; - /* Enumerate the intersection set of this state and `accepts'. */ + /* Enumerate the intersection set of this state and 'accepts'. */ has_intersec = 0; for (k = 0; k < BITSET_WORDS; ++k) has_intersec |= intersec[k] = accepts[k] & dests_ch[j][k]; @@ -3614,7 +3659,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, if (!has_intersec) continue; - /* Then check if this state is a subset of `accepts'. */ + /* Then check if this state is a subset of 'accepts'. */ not_subset = not_consumed = 0; for (k = 0; k < BITSET_WORDS; ++k) { @@ -3622,8 +3667,8 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, not_consumed |= accepts[k] = accepts[k] & ~dests_ch[j][k]; } - /* If this state isn't a subset of `accepts', create a - new group state, which has the `remains'. */ + /* If this state isn't a subset of 'accepts', create a + new group state, which has the 'remains'. */ if (not_subset) { bitset_copy (dests_ch[ndests], remains); @@ -3635,8 +3680,8 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, } /* Put the position in the current group. */ - result = re_node_set_insert (&dests_node[j], cur_nodes->elems[i]); - if (BE (result < 0, 0)) + ok = re_node_set_insert (&dests_node[j], cur_nodes->elems[i]); + if (BE (! ok, 0)) goto error_return; /* If all characters are consumed, go to next node. */ @@ -3662,7 +3707,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, } #ifdef RE_ENABLE_I18N -/* Check how many bytes the node `dfa->nodes[node_idx]' accepts. +/* Check how many bytes the node 'dfa->nodes[node_idx]' accepts. Return the number of the bytes the node accepts. STR_IDX is the current index of the input string. @@ -3675,12 +3720,12 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, # endif static int -check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, - const re_string_t *input, int str_idx) +check_node_accept_bytes (const re_dfa_t *dfa, Idx node_idx, + const re_string_t *input, Idx str_idx) { const re_token_t *node = dfa->nodes + node_idx; int char_len, elem_len; - int i; + Idx i; if (BE (node->type == OP_UTF8_PERIOD, 0)) { @@ -3759,7 +3804,7 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, # ifdef _LIBC const unsigned char *pin = ((const unsigned char *) re_string_get_buffer (input) + str_idx); - int j; + Idx j; uint32_t nrules; # endif /* _LIBC */ int match_len = 0; @@ -3827,6 +3872,7 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, in_collseq = find_collation_sequence_value (pin, elem_len); } /* match with range expression? */ + /* FIXME: Implement rational ranges here, too. */ for (i = 0; i < cset->nranges; ++i) if (cset->range_starts[i] <= in_collseq && in_collseq <= cset->range_ends[i]) @@ -3856,7 +3902,7 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, if (weight_len == weights[equiv_class_idx & 0xffffff] && (idx >> 24) == (equiv_class_idx >> 24)) { - int cnt = 0; + Idx cnt = 0; idx &= 0xffffff; equiv_class_idx &= 0xffffff; @@ -3878,18 +3924,9 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, # endif /* _LIBC */ { /* match with range expression? */ -#if __GNUC__ >= 2 - wchar_t cmp_buf[] = {L'\0', L'\0', wc, L'\0', L'\0', L'\0'}; -#else - wchar_t cmp_buf[] = {L'\0', L'\0', L'\0', L'\0', L'\0', L'\0'}; - cmp_buf[2] = wc; -#endif for (i = 0; i < cset->nranges; ++i) { - cmp_buf[0] = cset->range_starts[i]; - cmp_buf[4] = cset->range_ends[i]; - if (__wcscoll (cmp_buf, cmp_buf + 2) <= 0 - && __wcscoll (cmp_buf + 2, cmp_buf + 4) <= 0) + if (cset->range_starts[i] <= wc && wc <= cset->range_ends[i]) { match_len = char_len; goto check_node_accept_bytes_match; @@ -3936,7 +3973,8 @@ find_collation_sequence_value (const unsigned char *mbs, size_t mbs_len) for (idx = 0; idx < extrasize;) { - int mbs_cnt, found = 0; + int mbs_cnt; + bool found = false; int32_t elem_mbs_len; /* Skip the name of collating element name. */ idx = idx + extra[idx] + 1; @@ -3948,7 +3986,7 @@ find_collation_sequence_value (const unsigned char *mbs, size_t mbs_len) break; if (mbs_cnt == elem_mbs_len) /* Found the entry. */ - found = 1; + found = true; } /* Skip the byte sequence of the collating element. */ idx += elem_mbs_len; @@ -3973,9 +4011,9 @@ find_collation_sequence_value (const unsigned char *mbs, size_t mbs_len) /* Check whether the node accepts the byte which is IDX-th byte of the INPUT. */ -static int +static bool check_node_accept (const re_match_context_t *mctx, const re_token_t *node, - int idx) + Idx idx) { unsigned char ch; ch = re_string_byte_at (&mctx->input, idx); @@ -3983,28 +4021,28 @@ check_node_accept (const re_match_context_t *mctx, const re_token_t *node, { case CHARACTER: if (node->opr.c != ch) - return 0; + return false; break; case SIMPLE_BRACKET: if (!bitset_contain (node->opr.sbcset, ch)) - return 0; + return false; break; #ifdef RE_ENABLE_I18N case OP_UTF8_PERIOD: - if (ch >= 0x80) - return 0; - /* FALLTHROUGH */ + if (ch >= ASCII_CHARS) + return false; + FALLTHROUGH; #endif case OP_PERIOD: if ((ch == '\n' && !(mctx->dfa->syntax & RE_DOT_NEWLINE)) || (ch == '\0' && (mctx->dfa->syntax & RE_DOT_NOT_NULL))) - return 0; + return false; break; default: - return 0; + return false; } if (node->constraint) @@ -4014,10 +4052,10 @@ check_node_accept (const re_match_context_t *mctx, const re_token_t *node, unsigned int context = re_string_context_at (&mctx->input, idx, mctx->eflags); if (NOT_SATISFY_NEXT_CONSTRAINT (node->constraint, context)) - return 0; + return false; } - return 1; + return true; } /* Extend the buffers, if the buffers have run out. */ @@ -4030,10 +4068,11 @@ extend_buffers (re_match_context_t *mctx, int min_len) re_string_t *pstr = &mctx->input; /* Avoid overflow. */ - if (BE (INT_MAX / 2 / sizeof (re_dfastate_t *) <= pstr->bufs_len, 0)) + if (BE (MIN (IDX_MAX, SIZE_MAX / sizeof (re_dfastate_t *)) / 2 + <= pstr->bufs_len, 0)) return REG_ESPACE; - /* Double the lengthes of the buffers, but allocate at least MIN_LEN. */ + /* Double the lengths of the buffers, but allocate at least MIN_LEN. */ ret = re_string_realloc_buffers (pstr, MAX (min_len, MIN (pstr->len, pstr->bufs_len * 2))); @@ -4089,12 +4128,19 @@ extend_buffers (re_match_context_t *mctx, int min_len) static reg_errcode_t __attribute_warn_unused_result__ -match_ctx_init (re_match_context_t *mctx, int eflags, int n) +match_ctx_init (re_match_context_t *mctx, int eflags, Idx n) { mctx->eflags = eflags; mctx->match_last = -1; if (n > 0) { + /* Avoid overflow. */ + size_t max_object_size = + MAX (sizeof (struct re_backref_cache_entry), + sizeof (re_sub_match_top_t *)); + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) < n, 0)) + return REG_ESPACE; + mctx->bkref_ents = re_malloc (struct re_backref_cache_entry, n); mctx->sub_tops = re_malloc (re_sub_match_top_t *, n); if (BE (mctx->bkref_ents == NULL || mctx->sub_tops == NULL, 0)) @@ -4118,10 +4164,10 @@ match_ctx_init (re_match_context_t *mctx, int eflags, int n) static void match_ctx_clean (re_match_context_t *mctx) { - int st_idx; + Idx st_idx; for (st_idx = 0; st_idx < mctx->nsub_tops; ++st_idx) { - int sl_idx; + Idx sl_idx; re_sub_match_top_t *top = mctx->sub_tops[st_idx]; for (sl_idx = 0; sl_idx < top->nlasts; ++sl_idx) { @@ -4135,7 +4181,7 @@ match_ctx_clean (re_match_context_t *mctx) re_free (top->path->array); re_free (top->path); } - free (top); + re_free (top); } mctx->nsub_tops = 0; @@ -4160,8 +4206,8 @@ match_ctx_free (re_match_context_t *mctx) static reg_errcode_t __attribute_warn_unused_result__ -match_ctx_add_entry (re_match_context_t *mctx, int node, int str_idx, int from, - int to) +match_ctx_add_entry (re_match_context_t *mctx, Idx node, Idx str_idx, Idx from, + Idx to) { if (mctx->nbkref_ents >= mctx->abkref_ents) { @@ -4196,7 +4242,7 @@ match_ctx_add_entry (re_match_context_t *mctx, int node, int str_idx, int from, A backreference does not epsilon-transition unless it is empty, so set to all zeros if FROM != TO. */ mctx->bkref_ents[mctx->nbkref_ents].eps_reachable_subexps_map - = (from == to ? ~0 : 0); + = (from == to ? -1 : 0); mctx->bkref_ents[mctx->nbkref_ents++].more = 0; if (mctx->max_mb_elem_len < to - from) @@ -4204,13 +4250,13 @@ match_ctx_add_entry (re_match_context_t *mctx, int node, int str_idx, int from, return REG_NOERROR; } -/* Search for the first entry which has the same str_idx, or -1 if none is +/* Return the first entry with the same str_idx, or -1 if none is found. Note that MCTX->BKREF_ENTS is already sorted by MCTX->STR_IDX. */ -static int -search_cur_bkref_entry (const re_match_context_t *mctx, int str_idx) +static Idx +search_cur_bkref_entry (const re_match_context_t *mctx, Idx str_idx) { - int left, right, mid, last; + Idx left, right, mid, last; last = right = mctx->nbkref_ents; for (left = 0; left < right;) { @@ -4231,7 +4277,7 @@ search_cur_bkref_entry (const re_match_context_t *mctx, int str_idx) static reg_errcode_t __attribute_warn_unused_result__ -match_ctx_add_subtop (re_match_context_t *mctx, int node, int str_idx) +match_ctx_add_subtop (re_match_context_t *mctx, Idx node, Idx str_idx) { #ifdef DEBUG assert (mctx->sub_tops != NULL); @@ -4239,7 +4285,7 @@ match_ctx_add_subtop (re_match_context_t *mctx, int node, int str_idx) #endif if (BE (mctx->nsub_tops == mctx->asub_tops, 0)) { - int new_asub_tops = mctx->asub_tops * 2; + Idx new_asub_tops = mctx->asub_tops * 2; re_sub_match_top_t **new_array = re_realloc (mctx->sub_tops, re_sub_match_top_t *, new_asub_tops); @@ -4260,12 +4306,12 @@ match_ctx_add_subtop (re_match_context_t *mctx, int node, int str_idx) at STR_IDX, whose corresponding OP_OPEN_SUBEXP is SUB_TOP. */ static re_sub_match_last_t * -match_ctx_add_sublast (re_sub_match_top_t *subtop, int node, int str_idx) +match_ctx_add_sublast (re_sub_match_top_t *subtop, Idx node, Idx str_idx) { re_sub_match_last_t *new_entry; if (BE (subtop->nlasts == subtop->alasts, 0)) { - int new_alasts = 2 * subtop->alasts + 1; + Idx new_alasts = 2 * subtop->alasts + 1; re_sub_match_last_t **new_array = re_realloc (subtop->lasts, re_sub_match_last_t *, new_alasts); @@ -4287,7 +4333,7 @@ match_ctx_add_sublast (re_sub_match_top_t *subtop, int node, int str_idx) static void sift_ctx_init (re_sift_context_t *sctx, re_dfastate_t **sifted_sts, - re_dfastate_t **limited_sts, int last_node, int last_str_idx) + re_dfastate_t **limited_sts, Idx last_node, Idx last_str_idx) { sctx->sifted_states = sifted_sts; sctx->limited_states = limited_sts; -- 2.7.4
Adhemerval Zanella wrote: > I really want to make this change independent of intprops.h addition > since previous attempts we tried to add it, some maintainers raised > concerns about its inclusion. As I recall the last time I suggested adding it, the only concern raised was over the licensing boilerplate at the start of the file. I have fixed that in Gnulib, so as far as I know the concerns have been addressed. We can make intprops.h's addition independent by separating its addition into a separate patch, as I proposed in my most-recent email. > For this change I have fixed the points you raised (as you pointed out > the semantic in indeed to wrap over values). I'm afraid the overflow bugs still aren't fixed. Your revised change is not portable to GCC 4 and earlier (or to non-GCC, for Gnulib), since (1) it clearly doesn't work when 'a' is nonnegative and signed integer overflow occurs, and (2) even when 'a' is negative it has undefined behavior with overflow. It's pretty tricky to write this sort of code portably! intprops.h does it right, and I'd really, reaaally rather not debug another copy of it. So let's do things the intprops.h way. > Just to be clear, current glibc regex do accept this pattern with > REG_EXTENDED. And the gnulib code does report an error instead of > accepting it. So it is not clear to me, from glibc side, if we should > continue to accept it or if we should provide old behaviour. This one is an easy call. The ERE (a)|\1 is malformed (in the sense that the \1 doesn't mean anything) and Gnulib is right to reject it. As I recall, current Glibc accepts it but then has undefined behavior that usually does not crash. The Gnulib behavior conforms to the POSIX and glibc spec, and I would say it's clearly better for users, so let's do it.
On 06/29/2018 05:26 PM, Paul Eggert wrote: > As I recall the last time I suggested adding it, the only concern raised > was over the licensing boilerplate at the start of the file. I have > fixed that in Gnulib, so as far as I know the concerns have been > addressed. We can make intprops.h's addition independent by separating > its addition into a separate patch, as I proposed in my most-recent email. Most of intprops.h is untestable due to lack of relevant implementations existing out there on which the code could be tested, so I think it's not maintainable and does not match current glibc quality standards. Florian
On 29/06/2018 12:41, Florian Weimer wrote: > On 06/29/2018 05:26 PM, Paul Eggert wrote: >> As I recall the last time I suggested adding it, the only concern raised was over the licensing boilerplate at the start of the file. I have fixed that in Gnulib, so as far as I know the concerns have been addressed. We can make intprops.h's addition independent by separating its addition into a separate patch, as I proposed in my most-recent email. > > Most of intprops.h is untestable due to lack of relevant implementations existing out there on which the code could be tested, so I think it's not maintainable and does not match current glibc quality standards. > > Florian I tried to check the pos-processed implementation using intprops.h for the snippet: --- bool wrap_int (int a, int b, int *r) { return INT_ADD_WRAPV (a, b, r); } --- And its results are pretty much unreadable [1]. I think I tend to agree with Florian where a simple implementation should be better than import all intprops.h. Paul, could you check what I am missing on the overflow check so we can move forward? I really don't want to get this sync stalled because of intprops.h addition as a pre-requisite. [1] https://paste.ubuntu.com/p/QCr3QQ6X5w/
On 06/29/2018 09:00 AM, Adhemerval Zanella wrote: > Paul, could you check what I am missing on the overflow check so we can > move forward? I really don't want to get this sync stalled because of > intprops.h addition as a pre-requisite. I can add comments to INT_ADD_WRAPV, if that would help. Please ask questions about it; that would help me understand what parts need better commenting. I'm afraid that Florian's comment about testability is a bit of a red herring, as intprops.h is an internal header, not a published one. Glibc typically doesn't have test cases for macros defined in internal include files, and there's no need to test intprops.h directly. Besides, intprops.h and INT_ADD_WRAPV have been tested widely in commonly-used GNU applications like Coreutils, so it's not like I'm proposing anything flaky here. That being said, this use of INT_ADD_WRAPV is unimportant (as far as I know, no Gnu apps use this two-buffer regex searching code any more), so I guess it's OK here to write a simple substitute that's good enough for regex even if it's not robust or general. The goal should be simplicity, portability, and meeting regex's needs. Proposed further Glibc patch attached (it assumes your patch). I've installed a similar patch into Gnulib. I really don't want to get into the habit of doing this sort of thing, though. There's a lot of hard-won integer overflow expertise in intprops.h and it's a waste of time to continue to maintain poor imitations of it. > I tried to check the pos-processed implementation using intprops.h for > the snippet ... And its results are pretty much unreadable. Trying to audit this by looking at the preprocessed output is a bit like auditing glibc by looking at the generated machine code. It can be done, but you can't expect it to be easy. For your particular compiler I expect that the assembly-language output is more readable than the preprocessor output, so if you want to look at low level stuff I suggest looking at assembly language. But really, you should be better off reading the original source (and if it needs more comments, please say where and why and l'll add them). From 614946f802c3a7c2b11be14f85d3ed80f5e7ab57 Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Fri, 29 Jun 2018 15:42:16 -0700 Subject: [PATCH] Merge better with Gnulib * posix/regcomp.c (TYPE_SIGNED): Remove; now done by regex_internal.h. * posix/regex_internal.h [_LIBC]: Do not include intprops.h. (TYPE_SIGNED, INT_ADD_WRAPV) [_LIBC]: New macros. * posix/regexec.c (check_add_overflow_idx): Remove. (re_search_2_stub): Use INT_ADD_WRAPV insead of check_add_overflow_idx. --- ChangeLog | 9 +++++++++ posix/regcomp.c | 4 ---- posix/regex_internal.h | 18 ++++++++++++++++++ posix/regexec.c | 18 +----------------- 4 files changed, 28 insertions(+), 21 deletions(-) diff --git a/ChangeLog b/ChangeLog index c51c651220..4e665873ff 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,12 @@ +2018-06-29 Paul Eggert <eggert@cs.ucla.edu> + + Merge better with Gnulib + * posix/regcomp.c (TYPE_SIGNED): Remove; now done by regex_internal.h. + * posix/regex_internal.h [_LIBC]: Do not include intprops.h. + (TYPE_SIGNED, INT_ADD_WRAPV) [_LIBC]: New macros. + * posix/regexec.c (check_add_overflow_idx): Remove. + (re_search_2_stub): Use INT_ADD_WRAPV insead of check_add_overflow_idx. + 2018-06-29 Adhemerval Zanella <adhemerval.zanella@linaro.org> [BZ #23233] diff --git a/posix/regcomp.c b/posix/regcomp.c index 6aaa54f66d..7b5ddaad0c 100644 --- a/posix/regcomp.c +++ b/posix/regcomp.c @@ -2650,10 +2650,6 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, if (BE (tree == NULL, 0)) goto parse_dup_op_espace; -/* From gnulib's "intprops.h": - True if the arithmetic type T is signed. */ -#define TYPE_SIGNED(t) (! ((t) 0 < (t) -1)) - /* This loop is actually executed only when end != -1, to rewrite <re>{0,n} as (<re>(<re>...<re>?)?)?... We have already created the start+1-th copy. */ diff --git a/posix/regex_internal.h b/posix/regex_internal.h index f38f6b3d80..3b836ed206 100644 --- a/posix/regex_internal.h +++ b/posix/regex_internal.h @@ -33,6 +33,24 @@ #include <stdbool.h> #include <stdint.h> +/* Properties of integers. Although Gnulib has intprops.h, glibc does + without for now. */ +#ifndef _LIBC +# include "intprops.h" +#else +/* True if the real type T is signed. */ +# define TYPE_SIGNED(t) (! ((t) 0 < (t) -1)) + +/* True if adding the nonnegative Idx values A and B would overflow. + If false, set *R to A + B. A, B, and R may be evaluated more than + once, or zero times. Although this is not a full implementation of + Gnulib INT_ADD_WRAPV, it is good enough for glibc regex code. + FIXME: This implementation is a fragile stopgap, and this file would + be simpler and more robust if intprops.h were migrated into glibc. */ +# define INT_ADD_WRAPV(a, b, r) \ + (IDX_MAX - (a) < (b) ? true : (*(r) = (a) + (b), false)) +#endif + #ifdef _LIBC # include <libc-lock.h> # define lock_define(name) __libc_lock_define (, name) diff --git a/posix/regexec.c b/posix/regexec.c index d64b025d12..63aef97535 100644 --- a/posix/regexec.c +++ b/posix/regexec.c @@ -317,22 +317,6 @@ re_search_2 (struct re_pattern_buffer *bufp, const char *string1, Idx length1, weak_alias (__re_search_2, re_search_2) #endif -/* Set *R = A + B. Return true if the answer is mathematically incorrect due - to overflow; in this case, *R is the low order bits of the correct - answer. */ -static inline bool -check_add_overflow_idx (Idx a, Idx b, Idx *r) -{ -#if __GNUC__ >= 5 - return __builtin_add_overflow (a, b, r); -#else - *r = a + b; - if (a < 0 && b < IDX_MAX -a) - return false; - return true; -#endif -} - static regoff_t re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, Idx length1, const char *string2, Idx length2, Idx start, @@ -345,7 +329,7 @@ re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, char *s = NULL; if (BE ((length1 < 0 || length2 < 0 || stop < 0 - || check_add_overflow_idx (length1, length2, &len)), + || INT_ADD_WRAPV (length1, length2, &len)), 0)) return -2; -- 2.17.1
On 29/06/2018 20:10, Paul Eggert wrote: > On 06/29/2018 09:00 AM, Adhemerval Zanella wrote: >> Paul, could you check what I am missing on the overflow check so we can >> move forward? I really don't want to get this sync stalled because of >> intprops.h addition as a pre-requisite. > > I can add comments to INT_ADD_WRAPV, if that would help. Please ask questions about it; that would help me understand what parts need better commenting. I think we can get back to intprops.h addition on next 2.29 release. > > I'm afraid that Florian's comment about testability is a bit of a red herring, as intprops.h is an internal header, not a published one. Glibc typically doesn't have test cases for macros defined in internal include files, and there's no need to test intprops.h directly. Besides, intprops.h and INT_ADD_WRAPV have been tested widely in commonly-used GNU applications like Coreutils, so it's not like I'm proposing anything flaky here. We do have internal testing and situation for internal interfaces testing are improving over the releases. But it is not clear to me what exactly Florian opposes in intprops.h code and which parts he thinks would be untestable using glibc testsuite (the header contains some code which is not directly applicable to glibc, like various non-gnu compiler bugs support, but I don't think it should prevent its inclusion). > > That being said, this use of INT_ADD_WRAPV is unimportant (as far as I know, no Gnu apps use this two-buffer regex searching code any more), so I guess it's OK here to write a simple substitute that's good enough for regex even if it's not robust or general. The goal should be simplicity, portability, and meeting regex's needs. Proposed further Glibc patch attached (it assumes your patch). I've installed a similar patch into Gnulib. Thanks, I sync in the patch I intend to push upstream. > > I really don't want to get into the habit of doing this sort of thing, though. There's a lot of hard-won integer overflow expertise in intprops.h and it's a waste of time to continue to maintain poor imitations of it. I tend to agree, I just did not want to drag the regex sync indefinitely due this specific code detail. > > >> I tried to check the pos-processed implementation using intprops.h for >> the snippet ... And its results are pretty much unreadable. > Trying to audit this by looking at the preprocessed output is a bit like auditing glibc by looking at the generated machine code. It can be done, but you can't expect it to be easy. For your particular compiler I expect that the assembly-language output is more readable than the preprocessor output, so if you want to look at low level stuff I suggest looking at assembly language. But really, you should be better off reading the original source (and if it needs more comments, please say where and why and l'll add them). > > I followed your suggestion and reading intprops.h code I indeed couldn't find anything that should prevents us to include it on GLIBC (also mostly of the complexity of the pos-processed code came from generic use, so I take back it is 'unreadable'). Thanks for your review and work on gnulib side Paul, in attachments is the patch I indeed to push upstream on GLIBC. Basically it is just new tests for the referenced bugzillas, and some tests additions from previous discussions. From e88cdae7e032e0be122d7cf4741a3de86402cd00 Mon Sep 17 00:00:00 2001 From: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Wed, 20 Dec 2017 09:47:44 -0200 Subject: [PATCH] posix: Sync gnulib regex implementation This patch syncs regex code with gnulib (commit 0ee5212). Only two changes in GLIBC regex testing are required: 1. posix/bug-regex28.c: as previously discussed [1] the change of expected results on the pattern should be safe. 2. posix/PCRE.tests: the ERE (a)|\1 is malformed (in the sense that the \1 doesn't mean anything) and although current GLIBC accepts it has undefined behavior. This patch removes the specific test. This sync contains some patches from thread 'Regex: Make libc regex more usable outside GLIBC.' [2] which have been pushed upstream in gnulib. This patches also fixes some regex issues (BZ #23233, BZ #21163, BZ #18986, BZ #13762) and I did not add testcases for both #23233 and #13762 because I couldn't think a simple way to trigger the expected failure path to trigger them. Checked on x86_64-linux-gnu and i686-linux-gnu. [BZ #23233] [BZ #21163] [BZ #18986] [BZ #13762] * posix/Makefile (tests): Add bug-regex37 and bug-regex38. * posix/PCRE.tests: Remove invalid test. * posix/bug-regex28.c: Fix expected values for used syntax. * posix/bug-regex37.c: New file. * posix/bug-regex38.c: Likewise. * posix/regcomp.c: Sync with gnulib. * posix/regex.c: Likewise. * posix/regex.h: Likewise. * posix/regex_internal.c: Likewise. * posix/regex_internal.h: Likewise. * posix/regexec.c: Likewise. [1] https://sourceware.org/ml/libc-alpha/2017-12/msg00807.html [2] https://sourceware.org/ml/libc-alpha/2017-12/msg00237.html --- ChangeLog | 18 + posix/Makefile | 3 +- posix/PCRE.tests | 13 - posix/bug-regex28.c | 46 +-- posix/bug-regex37.c | 32 ++ posix/bug-regex38.c | 32 ++ posix/regcomp.c | 597 +++++++++++++++++-------------- posix/regex.c | 21 +- posix/regex.h | 335 +++++++++++------- posix/regex_internal.c | 295 ++++++++-------- posix/regex_internal.h | 442 +++++++++++++++-------- posix/regexec.c | 936 +++++++++++++++++++++++++------------------------ 12 files changed, 1609 insertions(+), 1161 deletions(-) create mode 100644 posix/bug-regex37.c create mode 100644 posix/bug-regex38.c diff --git a/ChangeLog b/ChangeLog index 25b90b0..7c50230 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,21 @@ +2018-07-03 Adhemerval Zanella <adhemerval.zanella@linaro.org> + + [BZ #23233] + [BZ #21163] + [BZ #18986] + [BZ #13762] + * posix/Makefile (tests): Add bug-regex37 and bug-regex38. + * posix/PCRE.tests: Remove invalid test. + * posix/bug-regex28.c: Fix expected values for used syntax. + * posix/bug-regex37.c: New file. + * posix/bug-regex38.c: Likewise. + * posix/regcomp.c: Sync with gnulib. + * posix/regex.c: Likewise. + * posix/regex.h: Likewise. + * posix/regex_internal.c: Likewise. + * posix/regex_internal.h: Likewise. + * posix/regexec.c: Likewise. + 2018-07-03 Florian Weimer <fweimer@redhat.com> [BZ #23363] diff --git a/posix/Makefile b/posix/Makefile index 989e42e..00c6284 100644 --- a/posix/Makefile +++ b/posix/Makefile @@ -95,7 +95,8 @@ tests := test-errno tstgetopt testfnm runtests runptests \ tst-posix_spawn-fd tst-posix_spawn-setsid \ tst-posix_fadvise tst-posix_fadvise64 \ tst-sysconf-empty-chroot tst-glob_symlinks tst-fexecve \ - tst-glob-tilde test-ssize-max tst-spawn4 + tst-glob-tilde test-ssize-max tst-spawn4 bug-regex37 \ + bug-regex38 tests-internal := bug-regex5 bug-regex20 bug-regex33 \ tst-rfc3484 tst-rfc3484-2 tst-rfc3484-3 \ tst-glob_lstat_compat tst-spawn4-compat diff --git a/posix/PCRE.tests b/posix/PCRE.tests index 0fb9cad..da63b86 100644 --- a/posix/PCRE.tests +++ b/posix/PCRE.tests @@ -1774,19 +1774,6 @@ No match 0: abcabc 1: abc -/(a)|\1/ - a - 0: a - 1: a - *** Failers - 0: a - 1: a - ab - 0: a - 1: a - x -No match - /abc/i ABC 0: ABC diff --git a/posix/bug-regex28.c b/posix/bug-regex28.c index 5353edf..ba263b2 100644 --- a/posix/bug-regex28.c +++ b/posix/bug-regex28.c @@ -21,18 +21,22 @@ #include <stdio.h> #include <string.h> +#include <support/test-driver.h> +#include <support/check.h> + struct tests { const char *regex; const char *string; reg_syntax_t syntax; int retval; -} tests[] = { +}; +static const struct tests tests[] = { #define EGREP RE_SYNTAX_EGREP #define EGREP_NL (RE_SYNTAX_EGREP | RE_DOT_NEWLINE) & ~RE_HAT_LISTS_NOT_NEWLINE - { "a.b", "a\nb", EGREP, -1 }, + { "a.b", "a\nb", EGREP, 0 }, { "a.b", "a\nb", EGREP_NL, 0 }, - { "a[^x]b", "a\nb", EGREP, -1 }, + { "a[^x]b", "a\nb", EGREP, 0 }, { "a[^x]b", "a\nb", EGREP_NL, 0 }, /* While \S and \W are internally handled as [^[:space:]] and [^[:alnum:]_], RE_HAT_LISTS_NOT_NEWLINE did not make any difference, so ensure @@ -42,33 +46,33 @@ struct tests { "a\\Wb", "a\nb", EGREP, 0 }, { "a\\Wb", "a\nb", EGREP_NL, 0 } }; +static const size_t tests_size = sizeof (tests) / sizeof (tests[0]); -int -main (void) +static int +do_test (void) { struct re_pattern_buffer r; - size_t i; - int ret = 0; - for (i = 0; i < sizeof (tests) / sizeof (tests[i]); ++i) + for (size_t i = 0; i < tests_size; i++) { re_set_syntax (tests[i].syntax); memset (&r, 0, sizeof (r)); - if (re_compile_pattern (tests[i].regex, strlen (tests[i].regex), &r)) - { - printf ("re_compile_pattern %zd failed\n", i); - ret = 1; - continue; - } + const char *re = re_compile_pattern (tests[i].regex, + strlen (tests[i].regex), &r); + TEST_VERIFY (re == NULL); + if (re != NULL) + continue; + size_t len = strlen (tests[i].string); int rv = re_search (&r, tests[i].string, len, 0, len, NULL); - if (rv != tests[i].retval) - { - printf ("re_search %zd unexpected value %d != %d\n", - i, rv, tests[i].retval); - ret = 1; - } + TEST_VERIFY (rv == tests[i].retval); + if (test_verbose > 0) + printf ("info: i=%zu rv=%d expected=%d\n", i, rv, tests[i].retval); + regfree (&r); } - return ret; + + return 0; } + +#include <support/test-driver.c> diff --git a/posix/bug-regex37.c b/posix/bug-regex37.c new file mode 100644 index 0000000..87a0916 --- /dev/null +++ b/posix/bug-regex37.c @@ -0,0 +1,32 @@ +/* Test regcomp return for invalid expression (BZ #21163). + Copyright (C) 2018 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <regex.h> + +#include <support/check.h> + +static int +do_test (void) +{ + char const pattern[] = "()*)|\\1)*"; + regex_t r; + TEST_VERIFY_EXIT (regcomp (&r, pattern, REG_EXTENDED) == REG_ESUBREG); + return 0; +} + +#include <support/test-driver.c> diff --git a/posix/bug-regex38.c b/posix/bug-regex38.c new file mode 100644 index 0000000..cb0eb7d --- /dev/null +++ b/posix/bug-regex38.c @@ -0,0 +1,32 @@ +/* Diagnose invalid back-reference in the ERE '()|\1' (BZ #18986). + Copyright (C) 2018 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <regex.h> + +#include <support/check.h> + +static int +do_test (void) +{ + char const pattern[] = "0|()0|\\1|0"; + regex_t r; + TEST_VERIFY_EXIT (regcomp (&r, pattern, REG_EXTENDED) == REG_ESUBREG); + return 0; +} + +#include <support/test-driver.c> diff --git a/posix/regcomp.c b/posix/regcomp.c index f5c09fe..7b5ddaa 100644 --- a/posix/regcomp.c +++ b/posix/regcomp.c @@ -15,9 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <stdint.h> + <https://www.gnu.org/licenses/>. */ #ifdef _LIBC # include <locale/weight.h> @@ -51,14 +49,14 @@ static bin_tree_t *lower_subexp (reg_errcode_t *err, regex_t *preg, static reg_errcode_t calc_first (void *extra, bin_tree_t *node); static reg_errcode_t calc_next (void *extra, bin_tree_t *node); static reg_errcode_t link_nfa_nodes (void *extra, bin_tree_t *node); -static int duplicate_node (re_dfa_t *dfa, int org_idx, unsigned int constraint); -static int search_duplicated_node (const re_dfa_t *dfa, int org_node, +static Idx duplicate_node (re_dfa_t *dfa, Idx org_idx, unsigned int constraint); +static Idx search_duplicated_node (const re_dfa_t *dfa, Idx org_node, unsigned int constraint); static reg_errcode_t calc_eclosure (re_dfa_t *dfa); static reg_errcode_t calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, - int node, int root); + Idx node, bool root); static reg_errcode_t calc_inveclosure (re_dfa_t *dfa); -static int fetch_number (re_string_t *input, re_token_t *token, +static Idx fetch_number (re_string_t *input, re_token_t *token, reg_syntax_t syntax); static int peek_token (re_token_t *token, re_string_t *input, reg_syntax_t syntax); @@ -66,16 +64,16 @@ static bin_tree_t *parse (re_string_t *regexp, regex_t *preg, reg_syntax_t syntax, reg_errcode_t *err); static bin_tree_t *parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_sub_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_dup_op (bin_tree_t *dup_elem, re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, reg_syntax_t syntax, reg_errcode_t *err); @@ -87,34 +85,34 @@ static reg_errcode_t parse_bracket_element (bracket_elem_t *elem, re_token_t *token, int token_len, re_dfa_t *dfa, reg_syntax_t syntax, - int accept_hyphen); + bool accept_hyphen); static reg_errcode_t parse_bracket_symbol (bracket_elem_t *elem, re_string_t *regexp, re_token_t *token); #ifdef RE_ENABLE_I18N static reg_errcode_t build_equiv_class (bitset_t sbcset, re_charset_t *mbcset, - int *equiv_class_alloc, + Idx *equiv_class_alloc, const unsigned char *name); static reg_errcode_t build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, re_charset_t *mbcset, - int *char_class_alloc, - const unsigned char *class_name, + Idx *char_class_alloc, + const char *class_name, reg_syntax_t syntax); #else /* not RE_ENABLE_I18N */ static reg_errcode_t build_equiv_class (bitset_t sbcset, const unsigned char *name); static reg_errcode_t build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, - const unsigned char *class_name, + const char *class_name, reg_syntax_t syntax); #endif /* not RE_ENABLE_I18N */ static bin_tree_t *build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, - const unsigned char *class_name, - const unsigned char *extra, - int non_match, reg_errcode_t *err); + const char *class_name, + const char *extra, + bool non_match, reg_errcode_t *err); static bin_tree_t *create_tree (re_dfa_t *dfa, bin_tree_t *left, bin_tree_t *right, re_token_type_t type); @@ -131,7 +129,7 @@ static reg_errcode_t mark_opt_subexp (void *extra, bin_tree_t *node); POSIX doesn't require that we do anything for REG_NOERROR, but why not be nice? */ -const char __re_error_msgid[] attribute_hidden = +static const char __re_error_msgid[] = { #define REG_NOERROR_IDX 0 gettext_noop ("Success") /* REG_NOERROR */ @@ -155,9 +153,9 @@ const char __re_error_msgid[] attribute_hidden = gettext_noop ("Invalid back reference") /* REG_ESUBREG */ "\0" #define REG_EBRACK_IDX (REG_ESUBREG_IDX + sizeof "Invalid back reference") - gettext_noop ("Unmatched [ or [^") /* REG_EBRACK */ + gettext_noop ("Unmatched [, [^, [:, [., or [=") /* REG_EBRACK */ "\0" -#define REG_EPAREN_IDX (REG_EBRACK_IDX + sizeof "Unmatched [ or [^") +#define REG_EPAREN_IDX (REG_EBRACK_IDX + sizeof "Unmatched [, [^, [:, [., or [=") gettext_noop ("Unmatched ( or \\(") /* REG_EPAREN */ "\0" #define REG_EBRACE_IDX (REG_EPAREN_IDX + sizeof "Unmatched ( or \\(") @@ -185,7 +183,7 @@ const char __re_error_msgid[] attribute_hidden = gettext_noop ("Unmatched ) or \\)") /* REG_ERPAREN */ }; -const size_t __re_error_msgid_idx[] attribute_hidden = +static const size_t __re_error_msgid_idx[] = { REG_NOERROR_IDX, REG_NOMATCH_IDX, @@ -269,7 +267,7 @@ weak_alias (__re_set_syntax, re_set_syntax) int re_compile_fastmap (struct re_pattern_buffer *bufp) { - re_dfa_t *dfa = (re_dfa_t *) bufp->buffer; + re_dfa_t *dfa = bufp->buffer; char *fastmap = bufp->fastmap; memset (fastmap, '\0', sizeof (char) * SBC_MAX); @@ -303,12 +301,12 @@ static void re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, char *fastmap) { - re_dfa_t *dfa = (re_dfa_t *) bufp->buffer; - int node_cnt; - int icase = (dfa->mb_cur_max == 1 && (bufp->syntax & RE_ICASE)); + re_dfa_t *dfa = bufp->buffer; + Idx node_cnt; + bool icase = (dfa->mb_cur_max == 1 && (bufp->syntax & RE_ICASE)); for (node_cnt = 0; node_cnt < init_state->nodes.nelem; ++node_cnt) { - int node = init_state->nodes.elems[node_cnt]; + Idx node = init_state->nodes.elems[node_cnt]; re_token_type_t type = dfa->nodes[node].type; if (type == CHARACTER) @@ -317,7 +315,8 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, #ifdef RE_ENABLE_I18N if ((bufp->syntax & RE_ICASE) && dfa->mb_cur_max > 1) { - unsigned char *buf = alloca (dfa->mb_cur_max), *p; + unsigned char buf[MB_LEN_MAX]; + unsigned char *p; wchar_t wc; mbstate_t state; @@ -332,7 +331,7 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, &state) == p - buf && (__wcrtomb ((char *) buf, __towlower (wc), &state) != (size_t) -1)) - re_set_fastmap (fastmap, 0, buf[0]); + re_set_fastmap (fastmap, false, buf[0]); } #endif } @@ -352,7 +351,7 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, else if (type == COMPLEX_BRACKET) { re_charset_t *cset = dfa->nodes[node].opr.mbcset; - int i; + Idx i; # ifdef _LIBC /* See if we have to try all bytes which start multiple collation @@ -465,7 +464,7 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, the return codes and their meanings.) */ int -regcomp (regex_t *__restrict preg, const char *__restrict pattern, int cflags) +regcomp (regex_t *_Restrict_ preg, const char *_Restrict_ pattern, int cflags) { reg_errcode_t ret; reg_syntax_t syntax = ((cflags & REG_EXTENDED) ? RE_SYNTAX_POSIX_EXTENDED @@ -525,7 +524,7 @@ weak_alias (__regcomp, regcomp) from either regcomp or regexec. We don't use PREG here. */ size_t -regerror (int errcode, const regex_t *__restrict preg, char *__restrict errbuf, +regerror (int errcode, const regex_t *_Restrict_ preg, char *_Restrict_ errbuf, size_t errbuf_size) { const char *msg; @@ -546,17 +545,13 @@ regerror (int errcode, const regex_t *__restrict preg, char *__restrict errbuf, if (BE (errbuf_size != 0, 1)) { + size_t cpy_size = msg_size; if (BE (msg_size > errbuf_size, 0)) { -#if defined HAVE_MEMPCPY || defined _LIBC - *((char *) __mempcpy (errbuf, msg, errbuf_size - 1)) = '\0'; -#else - memcpy (errbuf, msg, errbuf_size - 1); - errbuf[errbuf_size - 1] = 0; -#endif + cpy_size = errbuf_size - 1; + errbuf[cpy_size] = '\0'; } - else - memcpy (errbuf, msg, msg_size); + memcpy (errbuf, msg, cpy_size); } return msg_size; @@ -574,7 +569,23 @@ weak_alias (__regerror, regerror) static const bitset_t utf8_sb_map = { /* Set the first 128 bits. */ +# if defined __GNUC__ && !defined __STRICT_ANSI__ [0 ... 0x80 / BITSET_WORD_BITS - 1] = BITSET_WORD_MAX +# else +# if 4 * BITSET_WORD_BITS < ASCII_CHARS +# error "bitset_word_t is narrower than 32 bits" +# elif 3 * BITSET_WORD_BITS < ASCII_CHARS + BITSET_WORD_MAX, BITSET_WORD_MAX, BITSET_WORD_MAX, +# elif 2 * BITSET_WORD_BITS < ASCII_CHARS + BITSET_WORD_MAX, BITSET_WORD_MAX, +# elif 1 * BITSET_WORD_BITS < ASCII_CHARS + BITSET_WORD_MAX, +# endif + (BITSET_WORD_MAX + >> (SBC_MAX % BITSET_WORD_BITS == 0 + ? 0 + : BITSET_WORD_BITS - SBC_MAX % BITSET_WORD_BITS)) +# endif }; #endif @@ -582,7 +593,7 @@ static const bitset_t utf8_sb_map = static void free_dfa_content (re_dfa_t *dfa) { - int i, j; + Idx i, j; if (dfa->nodes) for (i = 0; i < dfa->nodes_len; ++i) @@ -632,9 +643,12 @@ free_dfa_content (re_dfa_t *dfa) void regfree (regex_t *preg) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; if (BE (dfa != NULL, 1)) - free_dfa_content (dfa); + { + lock_fini (dfa->lock); + free_dfa_content (dfa); + } preg->buffer = NULL; preg->allocated = 0; @@ -687,7 +701,7 @@ re_comp (const char *s) if (re_comp_buf.fastmap == NULL) { - re_comp_buf.fastmap = (char *) malloc (SBC_MAX); + re_comp_buf.fastmap = re_malloc (char, SBC_MAX); if (re_comp_buf.fastmap == NULL) return (char *) gettext (__re_error_msgid + __re_error_msgid_idx[(int) REG_ESPACE]); @@ -704,7 +718,7 @@ re_comp (const char *s) if (!ret) return NULL; - /* Yes, we're discarding `const' here if !HAVE_LIBINTL. */ + /* Yes, we're discarding 'const' here if !HAVE_LIBINTL. */ return (char *) gettext (__re_error_msgid + __re_error_msgid_idx[(int) ret]); } @@ -739,7 +753,7 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, preg->regs_allocated = REGS_UNALLOCATED; /* Initialize the dfa. */ - dfa = (re_dfa_t *) preg->buffer; + dfa = preg->buffer; if (BE (preg->allocated < sizeof (re_dfa_t), 0)) { /* If zero allocated, but buffer is non-null, try to realloc @@ -750,11 +764,13 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, if (dfa == NULL) return REG_ESPACE; preg->allocated = sizeof (re_dfa_t); - preg->buffer = (unsigned char *) dfa; + preg->buffer = dfa; } preg->used = sizeof (re_dfa_t); err = init_dfa (dfa, length); + if (BE (err == REG_NOERROR && lock_init (dfa->lock) != 0, 0)) + err = REG_ESPACE; if (BE (err != REG_NOERROR, 0)) { free_dfa_content (dfa); @@ -768,15 +784,14 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, strncpy (dfa->re_str, pattern, length + 1); #endif - __libc_lock_init (dfa->lock); - err = re_string_construct (®exp, pattern, length, preg->translate, - syntax & RE_ICASE, dfa); + (syntax & RE_ICASE) != 0, dfa); if (BE (err != REG_NOERROR, 0)) { re_compile_internal_free_return: free_workarea_compile (preg); re_string_destruct (®exp); + lock_fini (dfa->lock); free_dfa_content (dfa); preg->buffer = NULL; preg->allocated = 0; @@ -809,6 +824,7 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, if (BE (err != REG_NOERROR, 0)) { + lock_fini (dfa->lock); free_dfa_content (dfa); preg->buffer = NULL; preg->allocated = 0; @@ -823,18 +839,32 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, static reg_errcode_t init_dfa (re_dfa_t *dfa, size_t pat_len) { - unsigned int table_size; + __re_size_t table_size; #ifndef _LIBC - char *codeset_name; + const char *codeset_name; +#endif +#ifdef RE_ENABLE_I18N + size_t max_i18n_object_size = MAX (sizeof (wchar_t), sizeof (wctype_t)); +#else + size_t max_i18n_object_size = 0; #endif + size_t max_object_size = + MAX (sizeof (struct re_state_table_entry), + MAX (sizeof (re_token_t), + MAX (sizeof (re_node_set), + MAX (sizeof (regmatch_t), + max_i18n_object_size)))); memset (dfa, '\0', sizeof (re_dfa_t)); /* Force allocation of str_tree_storage the first time. */ dfa->str_tree_storage_idx = BIN_TREE_STORAGE_SIZE; - /* Avoid overflows. */ - if (pat_len == SIZE_MAX) + /* Avoid overflows. The extra "/ 2" is for the table_size doubling + calculation below, and for similar doubling calculations + elsewhere. And it's <= rather than <, because some of the + doubling calculations add 1 afterwards. */ + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) / 2 <= pat_len, 0)) return REG_ESPACE; dfa->nodes_alloc = pat_len + 1; @@ -856,22 +886,11 @@ init_dfa (re_dfa_t *dfa, size_t pat_len) dfa->map_notascii = (_NL_CURRENT_WORD (LC_CTYPE, _NL_CTYPE_MAP_TO_NONASCII) != 0); #else -# ifdef HAVE_LANGINFO_CODESET codeset_name = nl_langinfo (CODESET); -# else - codeset_name = getenv ("LC_ALL"); - if (codeset_name == NULL || codeset_name[0] == '\0') - codeset_name = getenv ("LC_CTYPE"); - if (codeset_name == NULL || codeset_name[0] == '\0') - codeset_name = getenv ("LANG"); - if (codeset_name == NULL) - codeset_name = ""; - else if (strchr (codeset_name, '.') != NULL) - codeset_name = strchr (codeset_name, '.') + 1; -# endif - - if (strcasecmp (codeset_name, "UTF-8") == 0 - || strcasecmp (codeset_name, "UTF8") == 0) + if ((codeset_name[0] == 'U' || codeset_name[0] == 'u') + && (codeset_name[1] == 'T' || codeset_name[1] == 't') + && (codeset_name[2] == 'F' || codeset_name[2] == 'f') + && strcmp (codeset_name + 3 + (codeset_name[3] == '-'), "8") == 0) dfa->is_utf8 = 1; /* We check exhaustively in the loop below if this charset is a @@ -920,9 +939,10 @@ init_dfa (re_dfa_t *dfa, size_t pat_len) static void init_word_char (re_dfa_t *dfa) { - dfa->word_ops_used = 1; int i = 0; + int j; int ch = 0; + dfa->word_ops_used = 1; if (BE (dfa->map_notascii == 0, 1)) { /* Avoid uint32_t and uint64_t as some non-GCC platforms lack @@ -959,7 +979,7 @@ init_word_char (re_dfa_t *dfa) general_case: for (; i < BITSET_WORDS; ++i) - for (int j = 0; j < BITSET_WORD_BITS; ++j, ++ch) + for (j = 0; j < BITSET_WORD_BITS; ++j, ++ch) if (isalnum (ch) || ch == '_') dfa->word_char[i] |= (bitset_word_t) 1 << j; } @@ -969,7 +989,7 @@ init_word_char (re_dfa_t *dfa) static void free_workarea_compile (regex_t *preg) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_storage_t *storage, *next; for (storage = dfa->str_tree_storage; storage; storage = next) { @@ -988,7 +1008,7 @@ free_workarea_compile (regex_t *preg) static reg_errcode_t create_initial_state (re_dfa_t *dfa) { - int first, i; + Idx first, i; reg_errcode_t err; re_node_set init_nodes; @@ -1007,10 +1027,10 @@ create_initial_state (re_dfa_t *dfa) if (dfa->nbackref > 0) for (i = 0; i < init_nodes.nelem; ++i) { - int node_idx = init_nodes.elems[i]; + Idx node_idx = init_nodes.elems[i]; re_token_type_t type = dfa->nodes[node_idx].type; - int clexp_idx; + Idx clexp_idx; if (type != OP_BACK_REF) continue; for (clexp_idx = 0; clexp_idx < init_nodes.nelem; ++clexp_idx) @@ -1026,14 +1046,13 @@ create_initial_state (re_dfa_t *dfa) if (type == OP_BACK_REF) { - int dest_idx = dfa->edests[node_idx].elems[0]; + Idx dest_idx = dfa->edests[node_idx].elems[0]; if (!re_node_set_contains (&init_nodes, dest_idx)) { - reg_errcode_t err = re_node_set_merge (&init_nodes, - dfa->eclosures - + dest_idx); - if (err != REG_NOERROR) - return err; + reg_errcode_t merge_err + = re_node_set_merge (&init_nodes, dfa->eclosures + dest_idx); + if (merge_err != REG_NOERROR) + return merge_err; i = 0; } } @@ -1074,14 +1093,17 @@ create_initial_state (re_dfa_t *dfa) static void optimize_utf8 (re_dfa_t *dfa) { - int node, i, mb_chars = 0, has_period = 0; + Idx node; + int i; + bool mb_chars = false; + bool has_period = false; for (node = 0; node < dfa->nodes_len; ++node) switch (dfa->nodes[node].type) { case CHARACTER: - if (dfa->nodes[node].opr.c >= 0x80) - mb_chars = 1; + if (dfa->nodes[node].opr.c >= ASCII_CHARS) + mb_chars = true; break; case ANCHOR: switch (dfa->nodes[node].opr.ctx_type) @@ -1099,7 +1121,7 @@ optimize_utf8 (re_dfa_t *dfa) } break; case OP_PERIOD: - has_period = 1; + has_period = true; break; case OP_BACK_REF: case OP_ALT: @@ -1111,11 +1133,18 @@ optimize_utf8 (re_dfa_t *dfa) case COMPLEX_BRACKET: return; case SIMPLE_BRACKET: - /* Just double check. The non-ASCII range starts at 0x80. */ - assert (0x80 % BITSET_WORD_BITS == 0); - for (i = 0x80 / BITSET_WORD_BITS; i < BITSET_WORDS; ++i) - if (dfa->nodes[node].opr.sbcset[i]) - return; + /* Just double check. */ + { + int rshift = (ASCII_CHARS % BITSET_WORD_BITS == 0 + ? 0 + : BITSET_WORD_BITS - ASCII_CHARS % BITSET_WORD_BITS); + for (i = ASCII_CHARS / BITSET_WORD_BITS; i < BITSET_WORDS; ++i) + { + if (dfa->nodes[node].opr.sbcset[i] >> rshift != 0) + return; + rshift = 0; + } + } break; default: abort (); @@ -1125,7 +1154,7 @@ optimize_utf8 (re_dfa_t *dfa) for (node = 0; node < dfa->nodes_len; ++node) { if (dfa->nodes[node].type == CHARACTER - && dfa->nodes[node].opr.c >= 0x80) + && dfa->nodes[node].opr.c >= ASCII_CHARS) dfa->nodes[node].mb_partial = 0; else if (dfa->nodes[node].type == OP_PERIOD) dfa->nodes[node].type = OP_UTF8_PERIOD; @@ -1144,22 +1173,22 @@ optimize_utf8 (re_dfa_t *dfa) static reg_errcode_t analyze (regex_t *preg) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; reg_errcode_t ret; /* Allocate arrays. */ - dfa->nexts = re_malloc (int, dfa->nodes_alloc); - dfa->org_indices = re_malloc (int, dfa->nodes_alloc); + dfa->nexts = re_malloc (Idx, dfa->nodes_alloc); + dfa->org_indices = re_malloc (Idx, dfa->nodes_alloc); dfa->edests = re_malloc (re_node_set, dfa->nodes_alloc); dfa->eclosures = re_malloc (re_node_set, dfa->nodes_alloc); if (BE (dfa->nexts == NULL || dfa->org_indices == NULL || dfa->edests == NULL || dfa->eclosures == NULL, 0)) return REG_ESPACE; - dfa->subexp_map = re_malloc (int, preg->re_nsub); + dfa->subexp_map = re_malloc (Idx, preg->re_nsub); if (dfa->subexp_map != NULL) { - int i; + Idx i; for (i = 0; i < preg->re_nsub; i++) dfa->subexp_map[i] = i; preorder (dfa->str_tree, optimize_subexps, dfa); @@ -1168,7 +1197,7 @@ analyze (regex_t *preg) break; if (i == preg->re_nsub) { - free (dfa->subexp_map); + re_free (dfa->subexp_map); dfa->subexp_map = NULL; } } @@ -1284,7 +1313,7 @@ optimize_subexps (void *extra, bin_tree_t *node) else if (node->token.type == SUBEXP && node->left && node->left->token.type == SUBEXP) { - int other_idx = node->left->token.opr.idx; + Idx other_idx = node->left->token.opr.idx; node->left = node->left->left; if (node->left) @@ -1292,7 +1321,7 @@ optimize_subexps (void *extra, bin_tree_t *node) dfa->subexp_map[other_idx] = dfa->subexp_map[node->token.opr.idx]; if (other_idx < BITSET_WORD_BITS) - dfa->used_bkref_map &= ~((bitset_word_t) 1 << other_idx); + dfa->used_bkref_map &= ~((bitset_word_t) 1 << other_idx); } return REG_NOERROR; @@ -1325,7 +1354,7 @@ lower_subexps (void *extra, bin_tree_t *node) static bin_tree_t * lower_subexp (reg_errcode_t *err, regex_t *preg, bin_tree_t *node) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *body = node->left; bin_tree_t *op, *cls, *tree1, *tree; @@ -1408,7 +1437,7 @@ static reg_errcode_t link_nfa_nodes (void *extra, bin_tree_t *node) { re_dfa_t *dfa = (re_dfa_t *) extra; - int idx = node->node_idx; + Idx idx = node->node_idx; reg_errcode_t err = REG_NOERROR; switch (node->token.type) @@ -1423,7 +1452,7 @@ link_nfa_nodes (void *extra, bin_tree_t *node) case OP_DUP_ASTERISK: case OP_ALT: { - int left, right; + Idx left, right; dfa->has_plural_match = 1; if (node->left != NULL) left = node->left->first->node_idx; @@ -1465,14 +1494,15 @@ link_nfa_nodes (void *extra, bin_tree_t *node) to their own constraint. */ static reg_errcode_t -duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, - int root_node, unsigned int init_constraint) +duplicate_node_closure (re_dfa_t *dfa, Idx top_org_node, Idx top_clone_node, + Idx root_node, unsigned int init_constraint) { - int org_node, clone_node, ret; + Idx org_node, clone_node; + bool ok; unsigned int constraint = init_constraint; for (org_node = top_org_node, clone_node = top_clone_node;;) { - int org_dest, clone_dest; + Idx org_dest, clone_dest; if (dfa->nodes[org_node].type == OP_BACK_REF) { /* If the back reference epsilon-transit, its destination must @@ -1485,8 +1515,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, if (BE (clone_dest == -1, 0)) return REG_ESPACE; dfa->nexts[clone_node] = dfa->nexts[org_node]; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } else if (dfa->edests[org_node].nelem == 0) @@ -1504,12 +1534,12 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, org_dest = dfa->edests[org_node].elems[0]; re_node_set_empty (dfa->edests + clone_node); /* If the node is root_node itself, it means the epsilon closure - has a loop. Then tie it to the destination of the root_node. */ + has a loop. Then tie it to the destination of the root_node. */ if (org_node == root_node && clone_node != org_node) { - ret = re_node_set_insert (dfa->edests + clone_node, org_dest); - if (BE (ret < 0, 0)) - return REG_ESPACE; + ok = re_node_set_insert (dfa->edests + clone_node, org_dest); + if (BE (! ok, 0)) + return REG_ESPACE; break; } /* In case the node has another constraint, append it. */ @@ -1517,8 +1547,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, clone_dest = duplicate_node (dfa, org_dest, constraint); if (BE (clone_dest == -1, 0)) return REG_ESPACE; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } else /* dfa->edests[org_node].nelem == 2 */ @@ -1536,8 +1566,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, clone_dest = duplicate_node (dfa, org_dest, constraint); if (BE (clone_dest == -1, 0)) return REG_ESPACE; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; err = duplicate_node_closure (dfa, org_dest, clone_dest, root_node, constraint); @@ -1548,8 +1578,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, { /* There is a duplicated node which satisfies the constraint, use it to avoid infinite loop. */ - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } @@ -1557,8 +1587,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, clone_dest = duplicate_node (dfa, org_dest, constraint); if (BE (clone_dest == -1, 0)) return REG_ESPACE; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } org_node = org_dest; @@ -1570,11 +1600,11 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, /* Search for a node which is duplicated from the node ORG_NODE, and satisfies the constraint CONSTRAINT. */ -static int -search_duplicated_node (const re_dfa_t *dfa, int org_node, +static Idx +search_duplicated_node (const re_dfa_t *dfa, Idx org_node, unsigned int constraint) { - int idx; + Idx idx; for (idx = dfa->nodes_len - 1; dfa->nodes[idx].duplicated && idx > 0; --idx) { if (org_node == dfa->org_indices[idx] @@ -1588,10 +1618,10 @@ search_duplicated_node (const re_dfa_t *dfa, int org_node, Return the index of the new node, or -1 if insufficient storage is available. */ -static int -duplicate_node (re_dfa_t *dfa, int org_idx, unsigned int constraint) +static Idx +duplicate_node (re_dfa_t *dfa, Idx org_idx, unsigned int constraint) { - int dup_idx = re_dfa_add_node (dfa, dfa->nodes[org_idx]); + Idx dup_idx = re_dfa_add_node (dfa, dfa->nodes[org_idx]); if (BE (dup_idx != -1, 1)) { dfa->nodes[dup_idx].constraint = constraint; @@ -1607,17 +1637,18 @@ duplicate_node (re_dfa_t *dfa, int org_idx, unsigned int constraint) static reg_errcode_t calc_inveclosure (re_dfa_t *dfa) { - int src, idx, ret; + Idx src, idx; + bool ok; for (idx = 0; idx < dfa->nodes_len; ++idx) re_node_set_init_empty (dfa->inveclosures + idx); for (src = 0; src < dfa->nodes_len; ++src) { - int *elems = dfa->eclosures[src].elems; + Idx *elems = dfa->eclosures[src].elems; for (idx = 0; idx < dfa->eclosures[src].nelem; ++idx) { - ret = re_node_set_insert_last (dfa->inveclosures + elems[idx], src); - if (BE (ret == -1, 0)) + ok = re_node_set_insert_last (dfa->inveclosures + elems[idx], src); + if (BE (! ok, 0)) return REG_ESPACE; } } @@ -1630,11 +1661,12 @@ calc_inveclosure (re_dfa_t *dfa) static reg_errcode_t calc_eclosure (re_dfa_t *dfa) { - int node_idx, incomplete; + Idx node_idx; + bool incomplete; #ifdef DEBUG assert (dfa->nodes_len > 0); #endif - incomplete = 0; + incomplete = false; /* For each nodes, calculate epsilon closure. */ for (node_idx = 0; ; ++node_idx) { @@ -1644,7 +1676,7 @@ calc_eclosure (re_dfa_t *dfa) { if (!incomplete) break; - incomplete = 0; + incomplete = false; node_idx = 0; } @@ -1656,13 +1688,13 @@ calc_eclosure (re_dfa_t *dfa) if (dfa->eclosures[node_idx].nelem != 0) continue; /* Calculate epsilon closure of 'node_idx'. */ - err = calc_eclosure_iter (&eclosure_elem, dfa, node_idx, 1); + err = calc_eclosure_iter (&eclosure_elem, dfa, node_idx, true); if (BE (err != REG_NOERROR, 0)) return err; if (dfa->eclosures[node_idx].nelem == 0) { - incomplete = 1; + incomplete = true; re_node_set_free (&eclosure_elem); } } @@ -1672,13 +1704,13 @@ calc_eclosure (re_dfa_t *dfa) /* Calculate epsilon closure of NODE. */ static reg_errcode_t -calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, int node, int root) +calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, Idx node, bool root) { reg_errcode_t err; - int i; + Idx i; re_node_set eclosure; - int ret; - int incomplete = 0; + bool ok; + bool incomplete = false; err = re_node_set_alloc (&eclosure, dfa->edests[node].nelem + 1); if (BE (err != REG_NOERROR, 0)) return err; @@ -1704,19 +1736,19 @@ calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, int node, int root) for (i = 0; i < dfa->edests[node].nelem; ++i) { re_node_set eclosure_elem; - int edest = dfa->edests[node].elems[i]; - /* If calculating the epsilon closure of `edest' is in progress, + Idx edest = dfa->edests[node].elems[i]; + /* If calculating the epsilon closure of 'edest' is in progress, return intermediate result. */ if (dfa->eclosures[edest].nelem == -1) { - incomplete = 1; + incomplete = true; continue; } - /* If we haven't calculated the epsilon closure of `edest' yet, + /* If we haven't calculated the epsilon closure of 'edest' yet, calculate now. Otherwise use calculated epsilon closure. */ if (dfa->eclosures[edest].nelem == 0) { - err = calc_eclosure_iter (&eclosure_elem, dfa, edest, 0); + err = calc_eclosure_iter (&eclosure_elem, dfa, edest, false); if (BE (err != REG_NOERROR, 0)) return err; } @@ -1730,14 +1762,14 @@ calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, int node, int root) the epsilon closure of this node is also incomplete. */ if (dfa->eclosures[edest].nelem == 0) { - incomplete = 1; + incomplete = true; re_node_set_free (&eclosure_elem); } } /* An epsilon closure includes itself. */ - ret = re_node_set_insert (&eclosure, node); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (&eclosure, node); + if (BE (! ok, 0)) return REG_ESPACE; if (incomplete && !root) dfa->eclosures[node].nelem = 0; @@ -2046,16 +2078,18 @@ peek_token_bracket (re_token_t *token, re_string_t *input, reg_syntax_t syntax) case '.': token->type = OP_OPEN_COLL_ELEM; break; + case '=': token->type = OP_OPEN_EQUIV_CLASS; break; + case ':': if (syntax & RE_CHAR_CLASSES) { token->type = OP_OPEN_CHAR_CLASS; break; } - /* else fall through. */ + FALLTHROUGH; default: token->type = CHARACTER; token->opr.c = c; @@ -2099,7 +2133,7 @@ static bin_tree_t * parse (re_string_t *regexp, regex_t *preg, reg_syntax_t syntax, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree, *eor, *root; re_token_t current_token; dfa->syntax = syntax; @@ -2131,10 +2165,11 @@ parse (re_string_t *regexp, regex_t *preg, reg_syntax_t syntax, static bin_tree_t * parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree, *branch = NULL; + bitset_word_t initial_bkref_map = dfa->completed_bkref_map; tree = parse_branch (regexp, preg, token, syntax, nest, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; @@ -2145,6 +2180,8 @@ parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, if (token->type != OP_ALT && token->type != END_OF_RE && (nest == 0 || token->type != OP_CLOSE_SUBEXP)) { + bitset_word_t accumulated_bkref_map = dfa->completed_bkref_map; + dfa->completed_bkref_map = initial_bkref_map; branch = parse_branch (regexp, preg, token, syntax, nest, err); if (BE (*err != REG_NOERROR && branch == NULL, 0)) { @@ -2152,6 +2189,7 @@ parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, postorder (tree, free_tree, NULL); return NULL; } + dfa->completed_bkref_map |= accumulated_bkref_map; } else branch = NULL; @@ -2176,10 +2214,10 @@ parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, static bin_tree_t * parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - bin_tree_t *tree, *exp; - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + bin_tree_t *tree, *expr; + re_dfa_t *dfa = preg->buffer; tree = parse_expression (regexp, preg, token, syntax, nest, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; @@ -2187,19 +2225,19 @@ parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, while (token->type != OP_ALT && token->type != END_OF_RE && (nest == 0 || token->type != OP_CLOSE_SUBEXP)) { - exp = parse_expression (regexp, preg, token, syntax, nest, err); - if (BE (*err != REG_NOERROR && exp == NULL, 0)) + expr = parse_expression (regexp, preg, token, syntax, nest, err); + if (BE (*err != REG_NOERROR && expr == NULL, 0)) { if (tree != NULL) postorder (tree, free_tree, NULL); return NULL; } - if (tree != NULL && exp != NULL) + if (tree != NULL && expr != NULL) { - bin_tree_t *newtree = create_tree (dfa, tree, exp, CONCAT); + bin_tree_t *newtree = create_tree (dfa, tree, expr, CONCAT); if (newtree == NULL) { - postorder (exp, free_tree, NULL); + postorder (expr, free_tree, NULL); postorder (tree, free_tree, NULL); *err = REG_ESPACE; return NULL; @@ -2207,8 +2245,8 @@ parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, tree = newtree; } else if (tree == NULL) - tree = exp; - /* Otherwise exp == NULL, we don't need to create new tree. */ + tree = expr; + /* Otherwise expr == NULL, we don't need to create new tree. */ } return tree; } @@ -2221,9 +2259,9 @@ parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, static bin_tree_t * parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree; switch (token->type) { @@ -2253,16 +2291,19 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, } #endif break; + case OP_OPEN_SUBEXP: tree = parse_sub_exp (regexp, preg, token, syntax, nest + 1, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_OPEN_BRACKET: tree = parse_bracket_exp (regexp, dfa, token, syntax, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_BACK_REF: if (!BE (dfa->completed_bkref_map & (1 << token->opr.idx), 1)) { @@ -2279,13 +2320,14 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, ++dfa->nbackref; dfa->has_mb_node = 1; break; + case OP_OPEN_DUP_NUM: if (syntax & RE_CONTEXT_INVALID_DUP) { *err = REG_BADRPT; return NULL; } - /* FALLTHROUGH */ + FALLTHROUGH; case OP_DUP_ASTERISK: case OP_DUP_PLUS: case OP_DUP_QUESTION: @@ -2299,7 +2341,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, fetch_token (token, regexp, syntax); return parse_expression (regexp, preg, token, syntax, nest, err); } - /* else fall through */ + FALLTHROUGH; case OP_CLOSE_SUBEXP: if ((token->type == OP_CLOSE_SUBEXP) && !(syntax & RE_UNMATCHED_RIGHT_PAREN_ORD)) @@ -2307,7 +2349,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, *err = REG_ERPAREN; return NULL; } - /* else fall through */ + FALLTHROUGH; case OP_CLOSE_DUP_NUM: /* We treat it as a normal character. */ @@ -2322,6 +2364,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, return NULL; } break; + case ANCHOR: if ((token->opr.ctx_type & (WORD_DELIM | NOT_WORD_DELIM | WORD_FIRST | WORD_LAST)) @@ -2366,6 +2409,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, it must not be "<ANCHOR(^)><REPEAT(*)>". */ fetch_token (token, regexp, syntax); return tree; + case OP_PERIOD: tree = create_token_tree (dfa, NULL, NULL, token); if (BE (tree == NULL, 0)) @@ -2376,30 +2420,35 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, if (dfa->mb_cur_max > 1) dfa->has_mb_node = 1; break; + case OP_WORD: case OP_NOTWORD: tree = build_charclass_op (dfa, regexp->trans, - (const unsigned char *) "alnum", - (const unsigned char *) "_", + "alnum", + "_", token->type == OP_NOTWORD, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_SPACE: case OP_NOTSPACE: tree = build_charclass_op (dfa, regexp->trans, - (const unsigned char *) "space", - (const unsigned char *) "", + "space", + "", token->type == OP_NOTSPACE, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_ALT: case END_OF_RE: return NULL; + case BACK_SLASH: *err = REG_EESCAPE; return NULL; + default: /* Must not happen? */ #ifdef DEBUG @@ -2412,7 +2461,8 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, while (token->type == OP_DUP_ASTERISK || token->type == OP_DUP_PLUS || token->type == OP_DUP_QUESTION || token->type == OP_OPEN_DUP_NUM) { - bin_tree_t *dup_tree = parse_dup_op (tree, regexp, dfa, token, syntax, err); + bin_tree_t *dup_tree = parse_dup_op (tree, regexp, dfa, token, + syntax, err); if (BE (*err != REG_NOERROR && dup_tree == NULL, 0)) { if (tree != NULL) @@ -2444,9 +2494,9 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, static bin_tree_t * parse_sub_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree; size_t cur_nsub; cur_nsub = preg->re_nsub++; @@ -2489,7 +2539,7 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, reg_syntax_t syntax, reg_errcode_t *err) { bin_tree_t *tree = NULL, *old_tree = NULL; - int i, start, end, start_idx = re_string_cur_idx (regexp); + Idx i, start, end, start_idx = re_string_cur_idx (regexp); re_token_t start_token = *token; if (token->type == OP_OPEN_DUP_NUM) @@ -2535,12 +2585,19 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, return elem; } - if (BE ((end != -1 && start > end) || token->type != OP_CLOSE_DUP_NUM, 0)) + if (BE ((end != -1 && start > end) + || token->type != OP_CLOSE_DUP_NUM, 0)) { /* First number greater than second. */ *err = REG_BADBR; return NULL; } + + if (BE (RE_DUP_MAX < (end == -1 ? start : end), 0)) + { + *err = REG_ESIZE; + return NULL; + } } else { @@ -2583,26 +2640,31 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, old_tree = NULL; if (elem->token.type == SUBEXP) - postorder (elem, mark_opt_subexp, (void *) (long) elem->token.opr.idx); + { + uintptr_t subidx = elem->token.opr.idx; + postorder (elem, mark_opt_subexp, (void *) subidx); + } - tree = create_tree (dfa, elem, NULL, (end == -1 ? OP_DUP_ASTERISK : OP_ALT)); + tree = create_tree (dfa, elem, NULL, + (end == -1 ? OP_DUP_ASTERISK : OP_ALT)); if (BE (tree == NULL, 0)) goto parse_dup_op_espace; /* This loop is actually executed only when end != -1, to rewrite <re>{0,n} as (<re>(<re>...<re>?)?)?... We have already created the start+1-th copy. */ - for (i = start + 2; i <= end; ++i) - { - elem = duplicate_tree (elem, dfa); - tree = create_tree (dfa, tree, elem, CONCAT); - if (BE (elem == NULL || tree == NULL, 0)) - goto parse_dup_op_espace; - - tree = create_tree (dfa, tree, NULL, OP_ALT); - if (BE (tree == NULL, 0)) - goto parse_dup_op_espace; - } + if (TYPE_SIGNED (Idx) || end != -1) + for (i = start + 2; i <= end; ++i) + { + elem = duplicate_tree (elem, dfa); + tree = create_tree (dfa, tree, elem, CONCAT); + if (BE (elem == NULL || tree == NULL, 0)) + goto parse_dup_op_espace; + + tree = create_tree (dfa, tree, NULL, OP_ALT); + if (BE (tree == NULL, 0)) + goto parse_dup_op_espace; + } if (old_tree) tree = create_tree (dfa, old_tree, tree, CONCAT); @@ -2619,6 +2681,19 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, #define BRACKET_NAME_BUF_SIZE 32 #ifndef _LIBC + +# ifdef RE_ENABLE_I18N +/* Convert the byte B to the corresponding wide character. In a + unibyte locale, treat B as itself if it is an encoding error. + In a multibyte locale, return WEOF if B is an encoding error. */ +static wint_t +parse_byte (unsigned char b, re_charset_t *mbcset) +{ + wint_t wc = __btowc (b); + return wc == WEOF && !mbcset ? b : wc; +} +#endif + /* Local function for parse_bracket_exp only used in case of NOT _LIBC. Build the range expression which starts from START_ELEM, and ends at END_ELEM. The result are written to MBCSET and SBCSET. @@ -2628,11 +2703,17 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, static reg_errcode_t # ifdef RE_ENABLE_I18N -build_range_exp (bitset_t sbcset, re_charset_t *mbcset, int *range_alloc, - bracket_elem_t *start_elem, bracket_elem_t *end_elem) +build_range_exp (const reg_syntax_t syntax, + bitset_t sbcset, + re_charset_t *mbcset, + Idx *range_alloc, + const bracket_elem_t *start_elem, + const bracket_elem_t *end_elem) # else /* not RE_ENABLE_I18N */ -build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, - bracket_elem_t *end_elem) +build_range_exp (const reg_syntax_t syntax, + bitset_t sbcset, + const bracket_elem_t *start_elem, + const bracket_elem_t *end_elem) # endif /* not RE_ENABLE_I18N */ { unsigned int start_ch, end_ch; @@ -2655,7 +2736,6 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, wchar_t wc; wint_t start_wc; wint_t end_wc; - wchar_t cmp_buf[6] = {L'\0', L'\0', L'\0', L'\0', L'\0', L'\0'}; start_ch = ((start_elem->type == SB_CHAR) ? start_elem->opr.ch : ((start_elem->type == COLL_SYM) ? start_elem->opr.name[0] @@ -2664,14 +2744,12 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, : ((end_elem->type == COLL_SYM) ? end_elem->opr.name[0] : 0)); start_wc = ((start_elem->type == SB_CHAR || start_elem->type == COLL_SYM) - ? __btowc (start_ch) : start_elem->opr.wch); + ? parse_byte (start_ch, mbcset) : start_elem->opr.wch); end_wc = ((end_elem->type == SB_CHAR || end_elem->type == COLL_SYM) - ? __btowc (end_ch) : end_elem->opr.wch); + ? parse_byte (end_ch, mbcset) : end_elem->opr.wch); if (start_wc == WEOF || end_wc == WEOF) return REG_ECOLLATE; - cmp_buf[0] = start_wc; - cmp_buf[4] = end_wc; - if (__wcscoll (cmp_buf, cmp_buf + 4) > 0) + else if (BE ((syntax & RE_NO_EMPTY_RANGES) && start_wc > end_wc, 0)) return REG_ERANGE; /* Got valid collation sequence values, add them as a new entry. @@ -2686,7 +2764,7 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, { /* There is not enough space, need realloc. */ wchar_t *new_array_start, *new_array_end; - int new_nranges; + Idx new_nranges; /* +1 in case of mbcset->nranges is 0. */ new_nranges = 2 * mbcset->nranges + 1; @@ -2698,7 +2776,11 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, new_nranges); if (BE (new_array_start == NULL || new_array_end == NULL, 0)) - return REG_ESPACE; + { + re_free (new_array_start); + re_free (new_array_end); + return REG_ESPACE; + } mbcset->range_starts = new_array_start; mbcset->range_ends = new_array_end; @@ -2712,9 +2794,7 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, /* Build the table for single byte characters. */ for (wc = 0; wc < SBC_MAX; ++wc) { - cmp_buf[2] = wc; - if (__wcscoll (cmp_buf, cmp_buf + 2) <= 0 - && __wcscoll (cmp_buf + 2, cmp_buf + 4) <= 0) + if (start_wc <= wc && wc <= end_wc) bitset_set (sbcset, wc); } } @@ -2749,7 +2829,7 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, static reg_errcode_t # ifdef RE_ENABLE_I18N build_collating_symbol (bitset_t sbcset, re_charset_t *mbcset, - int *coll_sym_alloc, const unsigned char *name) + Idx *coll_sym_alloc, const unsigned char *name) # else /* not RE_ENABLE_I18N */ build_collating_symbol (bitset_t sbcset, const unsigned char *name) # endif /* not RE_ENABLE_I18N */ @@ -2895,6 +2975,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, 0)) return REG_ERANGE; + /* FIXME: Implement rational ranges here, too. */ start_collseq = lookup_collation_sequence_value (start_elem); end_collseq = lookup_collation_sequence_value (end_elem); /* Check start/end collation sequence values. */ @@ -2915,7 +2996,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, /* There is not enough space, need realloc. */ uint32_t *new_array_start; uint32_t *new_array_end; - int new_nranges; + Idx new_nranges; /* +1 in case of mbcset->nranges is 0. */ new_nranges = 2 * mbcset->nranges + 1; @@ -2962,7 +3043,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, auto inline reg_errcode_t __attribute__ ((always_inline)) build_collating_symbol (bitset_t sbcset, re_charset_t *mbcset, - int *coll_sym_alloc, const unsigned char *name) + Idx *coll_sym_alloc, const unsigned char *name) { int32_t elem, idx; size_t name_len = strlen ((const char *) name); @@ -2992,7 +3073,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, { /* Not enough, realloc it. */ /* +1 in case of mbcset->ncoll_syms is 0. */ - int new_coll_sym_alloc = 2 * mbcset->ncoll_syms + 1; + Idx new_coll_sym_alloc = 2 * mbcset->ncoll_syms + 1; /* Use realloc since mbcset->coll_syms is NULL if *alloc == 0. */ int32_t *new_coll_syms = re_realloc (mbcset->coll_syms, int32_t, @@ -3022,13 +3103,13 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, re_bitset_ptr_t sbcset; #ifdef RE_ENABLE_I18N re_charset_t *mbcset; - int coll_sym_alloc = 0, range_alloc = 0, mbchar_alloc = 0; - int equiv_class_alloc = 0, char_class_alloc = 0; + Idx coll_sym_alloc = 0, range_alloc = 0, mbchar_alloc = 0; + Idx equiv_class_alloc = 0, char_class_alloc = 0; #endif /* not RE_ENABLE_I18N */ - int non_match = 0; + bool non_match = false; bin_tree_t *work_tree; int token_len; - int first_round = 1; + bool first_round = true; #ifdef _LIBC collseqmb = (const unsigned char *) _NL_CURRENT (LC_COLLATE, _NL_COLLATE_COLLSEQMB); @@ -3075,7 +3156,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, #ifdef RE_ENABLE_I18N mbcset->non_match = 1; #endif /* not RE_ENABLE_I18N */ - non_match = 1; + non_match = true; if (syntax & RE_HAT_LISTS_NOT_NEWLINE) bitset_set (sbcset, '\n'); re_string_skip_bytes (regexp, token_len); /* Skip a token. */ @@ -3097,7 +3178,8 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, unsigned char start_name_buf[BRACKET_NAME_BUF_SIZE]; unsigned char end_name_buf[BRACKET_NAME_BUF_SIZE]; reg_errcode_t ret; - int token_len2 = 0, is_range_exp = 0; + int token_len2 = 0; + bool is_range_exp = false; re_token_t token2; start_elem.opr.name = start_name_buf; @@ -3109,7 +3191,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, *err = ret; goto parse_bracket_exp_free_return; } - first_round = 0; + first_round = false; /* Get information about the next token. We need it in any case. */ token_len = peek_token_bracket (token, regexp, syntax); @@ -3138,16 +3220,16 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, token->type = CHARACTER; } else - is_range_exp = 1; + is_range_exp = true; } } - if (is_range_exp == 1) + if (is_range_exp == true) { end_elem.opr.name = end_name_buf; end_elem.type = COLL_SYM; ret = parse_bracket_element (&end_elem, regexp, &token2, token_len2, - dfa, syntax, 1); + dfa, syntax, true); if (BE (ret != REG_NOERROR, 0)) { *err = ret; @@ -3161,11 +3243,11 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, &start_elem, &end_elem); #else # ifdef RE_ENABLE_I18N - *err = build_range_exp (sbcset, + *err = build_range_exp (syntax, sbcset, dfa->mb_cur_max > 1 ? mbcset : NULL, &range_alloc, &start_elem, &end_elem); # else - *err = build_range_exp (sbcset, &start_elem, &end_elem); + *err = build_range_exp (syntax, sbcset, &start_elem, &end_elem); # endif #endif /* RE_ENABLE_I18N */ if (BE (*err != REG_NOERROR, 0)) @@ -3220,7 +3302,8 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, #ifdef RE_ENABLE_I18N mbcset, &char_class_alloc, #endif /* RE_ENABLE_I18N */ - start_elem.opr.name, syntax); + (const char *) start_elem.opr.name, + syntax); if (BE (*err != REG_NOERROR, 0)) goto parse_bracket_exp_free_return; break; @@ -3317,7 +3400,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, static reg_errcode_t parse_bracket_element (bracket_elem_t *elem, re_string_t *regexp, re_token_t *token, int token_len, re_dfa_t *dfa, - reg_syntax_t syntax, int accept_hyphen) + reg_syntax_t syntax, bool accept_hyphen) { #ifdef RE_ENABLE_I18N int cur_char_size; @@ -3404,7 +3487,7 @@ parse_bracket_symbol (bracket_elem_t *elem, re_string_t *regexp, static reg_errcode_t #ifdef RE_ENABLE_I18N build_equiv_class (bitset_t sbcset, re_charset_t *mbcset, - int *equiv_class_alloc, const unsigned char *name) + Idx *equiv_class_alloc, const unsigned char *name) #else /* not RE_ENABLE_I18N */ build_equiv_class (bitset_t sbcset, const unsigned char *name) #endif /* not RE_ENABLE_I18N */ @@ -3466,7 +3549,7 @@ build_equiv_class (bitset_t sbcset, const unsigned char *name) { /* Not enough, realloc it. */ /* +1 in case of mbcset->nequiv_classes is 0. */ - int new_equiv_class_alloc = 2 * mbcset->nequiv_classes + 1; + Idx new_equiv_class_alloc = 2 * mbcset->nequiv_classes + 1; /* Use realloc since the array is NULL if *alloc == 0. */ int32_t *new_equiv_classes = re_realloc (mbcset->equiv_classes, int32_t, @@ -3497,15 +3580,15 @@ build_equiv_class (bitset_t sbcset, const unsigned char *name) static reg_errcode_t #ifdef RE_ENABLE_I18N build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, - re_charset_t *mbcset, int *char_class_alloc, - const unsigned char *class_name, reg_syntax_t syntax) + re_charset_t *mbcset, Idx *char_class_alloc, + const char *class_name, reg_syntax_t syntax) #else /* not RE_ENABLE_I18N */ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, - const unsigned char *class_name, reg_syntax_t syntax) + const char *class_name, reg_syntax_t syntax) #endif /* not RE_ENABLE_I18N */ { int i; - const char *name = (const char *) class_name; + const char *name = class_name; /* In case of REG_ICASE "upper" and "lower" match the both of upper and lower cases. */ @@ -3519,7 +3602,7 @@ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, { /* Not enough, realloc it. */ /* +1 in case of mbcset->nchar_classes is 0. */ - int new_char_class_alloc = 2 * mbcset->nchar_classes + 1; + Idx new_char_class_alloc = 2 * mbcset->nchar_classes + 1; /* Use realloc since array is NULL if *alloc == 0. */ wctype_t *new_char_classes = re_realloc (mbcset->char_classes, wctype_t, new_char_class_alloc); @@ -3536,13 +3619,13 @@ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, if (BE (trans != NULL, 0)) \ { \ for (i = 0; i < SBC_MAX; ++i) \ - if (ctype_func (i)) \ + if (ctype_func (i)) \ bitset_set (sbcset, trans[i]); \ } \ else \ { \ for (i = 0; i < SBC_MAX; ++i) \ - if (ctype_func (i)) \ + if (ctype_func (i)) \ bitset_set (sbcset, i); \ } \ } while (0) @@ -3579,40 +3662,35 @@ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, static bin_tree_t * build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, - const unsigned char *class_name, - const unsigned char *extra, int non_match, + const char *class_name, + const char *extra, bool non_match, reg_errcode_t *err) { re_bitset_ptr_t sbcset; #ifdef RE_ENABLE_I18N re_charset_t *mbcset; - int alloc = 0; + Idx alloc = 0; #endif /* not RE_ENABLE_I18N */ reg_errcode_t ret; re_token_t br_token; bin_tree_t *tree; sbcset = (re_bitset_ptr_t) calloc (sizeof (bitset_t), 1); -#ifdef RE_ENABLE_I18N - mbcset = (re_charset_t *) calloc (sizeof (re_charset_t), 1); -#endif /* RE_ENABLE_I18N */ - -#ifdef RE_ENABLE_I18N - if (BE (sbcset == NULL || mbcset == NULL, 0)) -#else /* not RE_ENABLE_I18N */ if (BE (sbcset == NULL, 0)) -#endif /* not RE_ENABLE_I18N */ { *err = REG_ESPACE; return NULL; } - - if (non_match) - { #ifdef RE_ENABLE_I18N - mbcset->non_match = 1; -#endif /* not RE_ENABLE_I18N */ + mbcset = (re_charset_t *) calloc (sizeof (re_charset_t), 1); + if (BE (mbcset == NULL, 0)) + { + re_free (sbcset); + *err = REG_ESPACE; + return NULL; } + mbcset->non_match = non_match; +#endif /* RE_ENABLE_I18N */ /* We don't care the syntax in this case. */ ret = build_charclass (trans, sbcset, @@ -3645,6 +3723,9 @@ build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, #endif /* Build a tree for simple bracket. */ +#if defined GCC_LINT || defined lint + memset (&br_token, 0, sizeof br_token); +#endif br_token.type = SIMPLE_BRACKET; br_token.opr.sbcset = sbcset; tree = create_token_tree (dfa, NULL, NULL, &br_token); @@ -3686,14 +3767,15 @@ build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, } /* This is intended for the expressions like "a{1,3}". - Fetch a number from `input', and return the number. - Return -1, if the number field is empty like "{,1}". - Return -2, If an error is occured. */ + Fetch a number from 'input', and return the number. + Return -1 if the number field is empty like "{,1}". + Return RE_DUP_MAX + 1 if the number field is too large. + Return -2 if an error occurred. */ -static int +static Idx fetch_number (re_string_t *input, re_token_t *token, reg_syntax_t syntax) { - int num = -1; + Idx num = -1; unsigned char c; while (1) { @@ -3704,8 +3786,10 @@ fetch_number (re_string_t *input, re_token_t *token, reg_syntax_t syntax) if (token->type == OP_CLOSE_DUP_NUM || c == ',') break; num = ((token->type != CHARACTER || c < '0' || '9' < c || num == -2) - ? -2 : ((num == -1) ? c - '0' : num * 10 + c - '0')); - num = (num > RE_DUP_MAX) ? -2 : num; + ? -2 + : num == -1 + ? c - '0' + : MIN (RE_DUP_MAX + 1, num * 10 + c - '0')); } return num; } @@ -3735,6 +3819,9 @@ create_tree (re_dfa_t *dfa, bin_tree_t *left, bin_tree_t *right, re_token_type_t type) { re_token_t t; +#if defined GCC_LINT || defined lint + memset (&t, 0, sizeof t); +#endif t.type = type; return create_token_tree (dfa, left, right, &t); } @@ -3779,7 +3866,7 @@ create_token_tree (re_dfa_t *dfa, bin_tree_t *left, bin_tree_t *right, static reg_errcode_t mark_opt_subexp (void *extra, bin_tree_t *node) { - int idx = (int) (long) extra; + Idx idx = (uintptr_t) extra; if (node->token.type == SUBEXP && node->token.opr.idx == idx) node->token.opt_subexp = 1; diff --git a/posix/regex.c b/posix/regex.c index 68b5ef1..d6591e8 100644 --- a/posix/regex.c +++ b/posix/regex.c @@ -15,14 +15,22 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ -#ifdef HAVE_CONFIG_H -#include "config.h" +#ifndef _LIBC +# include <config.h> + +# if (__GNUC__ == 4 && 6 <= __GNUC_MINOR__) || 4 < __GNUC__ +# pragma GCC diagnostic ignored "-Wsuggest-attribute=pure" +# endif +# if (__GNUC__ == 4 && 3 <= __GNUC_MINOR__) || 4 < __GNUC__ +# pragma GCC diagnostic ignored "-Wold-style-definition" +# pragma GCC diagnostic ignored "-Wtype-limits" +# endif #endif -/* Make sure noone compiles this code with a C++ compiler. */ -#ifdef __cplusplus +/* Make sure no one compiles this code with a C++ compiler. */ +#if defined __cplusplus && defined _LIBC # error "This is C code, use a C compiler" #endif @@ -56,9 +64,6 @@ #undefs RE_DUP_MAX and sets it to the right value. */ #include <limits.h> -/* This header defines the MIN and MAX macros. */ -#include <sys/param.h> - #include <regex.h> #include "regex_internal.h" diff --git a/posix/regex.h b/posix/regex.h index e0b8915..32933bc 100644 --- a/posix/regex.h +++ b/posix/regex.h @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ #ifndef _REGEX_H #define _REGEX_H 1 @@ -27,6 +27,36 @@ extern "C" { #endif +/* Define __USE_GNU to declare GNU extensions that violate the + POSIX name space rules. */ +#ifdef _GNU_SOURCE +# define __USE_GNU 1 +#endif + +#ifdef _REGEX_LARGE_OFFSETS + +/* Use types and values that are wide enough to represent signed and + unsigned byte offsets in memory. This currently works only when + the regex code is used outside of the GNU C library; it is not yet + supported within glibc itself, and glibc users should not define + _REGEX_LARGE_OFFSETS. */ + +/* The type of object sizes. */ +typedef size_t __re_size_t; + +/* The type of object sizes, in places where the traditional code + uses unsigned long int. */ +typedef size_t __re_long_size_t; + +#else + +/* The traditional GNU regex implementation mishandles strings longer + than INT_MAX. */ +typedef unsigned int __re_size_t; +typedef unsigned long int __re_long_size_t; + +#endif + /* The following two types have to be signed and unsigned integer type wide enough to hold a value of a pointer. For most ANSI compilers ptrdiff_t and size_t should be likely OK. Still size of these two @@ -108,9 +138,9 @@ typedef unsigned long int reg_syntax_t; If not set, newline is literal. */ # define RE_NEWLINE_ALT (RE_LIMITED_OPS << 1) -/* If this bit is set, then `{...}' defines an interval, and \{ and \} +/* If this bit is set, then '{...}' defines an interval, and \{ and \} are literals. - If not set, then `\{...\}' defines an interval. */ + If not set, then '\{...\}' defines an interval. */ # define RE_NO_BK_BRACES (RE_NEWLINE_ALT << 1) /* If this bit is set, (...) defines a group, and \( and \) are literals. @@ -165,8 +195,8 @@ typedef unsigned long int reg_syntax_t; whether ^ should be special. */ # define RE_CARET_ANCHORS_HERE (RE_ICASE << 1) -/* If this bit is set, then \{ cannot be first in an bre or - immediately after an alternation or begin-group operator. */ +/* If this bit is set, then \{ cannot be first in a regex or + immediately after an alternation, open-group or \} operator. */ # define RE_CONTEXT_INVALID_DUP (RE_CARET_ANCHORS_HERE << 1) /* If this bit is set, then no_sub will be set to 1 during @@ -185,9 +215,9 @@ extern reg_syntax_t re_syntax_options; (The [[[ comments delimit what gets put into the Texinfo file, so don't delete them!) */ /* [[[begin syntaxes]]] */ -#define RE_SYNTAX_EMACS 0 +# define RE_SYNTAX_EMACS 0 -#define RE_SYNTAX_AWK \ +# define RE_SYNTAX_AWK \ (RE_BACKSLASH_ESCAPE_IN_LISTS | RE_DOT_NOT_NULL \ | RE_NO_BK_PARENS | RE_NO_BK_REFS \ | RE_NO_BK_VBAR | RE_NO_EMPTY_RANGES \ @@ -195,52 +225,49 @@ extern reg_syntax_t re_syntax_options; | RE_CHAR_CLASSES \ | RE_UNMATCHED_RIGHT_PAREN_ORD | RE_NO_GNU_OPS) -#define RE_SYNTAX_GNU_AWK \ +# define RE_SYNTAX_GNU_AWK \ ((RE_SYNTAX_POSIX_EXTENDED | RE_BACKSLASH_ESCAPE_IN_LISTS \ | RE_INVALID_INTERVAL_ORD) \ & ~(RE_DOT_NOT_NULL | RE_CONTEXT_INDEP_OPS \ | RE_CONTEXT_INVALID_OPS )) -#define RE_SYNTAX_POSIX_AWK \ +# define RE_SYNTAX_POSIX_AWK \ (RE_SYNTAX_POSIX_EXTENDED | RE_BACKSLASH_ESCAPE_IN_LISTS \ | RE_INTERVALS | RE_NO_GNU_OPS \ | RE_INVALID_INTERVAL_ORD) -#define RE_SYNTAX_GREP \ - (RE_BK_PLUS_QM | RE_CHAR_CLASSES \ - | RE_HAT_LISTS_NOT_NEWLINE | RE_INTERVALS \ - | RE_NEWLINE_ALT) +# define RE_SYNTAX_GREP \ + ((RE_SYNTAX_POSIX_BASIC | RE_NEWLINE_ALT) \ + & ~(RE_CONTEXT_INVALID_DUP | RE_DOT_NOT_NULL)) -#define RE_SYNTAX_EGREP \ - (RE_CHAR_CLASSES | RE_CONTEXT_INDEP_ANCHORS \ - | RE_CONTEXT_INDEP_OPS | RE_HAT_LISTS_NOT_NEWLINE \ - | RE_NEWLINE_ALT | RE_NO_BK_PARENS \ - | RE_NO_BK_VBAR) +# define RE_SYNTAX_EGREP \ + ((RE_SYNTAX_POSIX_EXTENDED | RE_INVALID_INTERVAL_ORD | RE_NEWLINE_ALT) \ + & ~(RE_CONTEXT_INVALID_OPS | RE_DOT_NOT_NULL)) -#define RE_SYNTAX_POSIX_EGREP \ - (RE_SYNTAX_EGREP | RE_INTERVALS | RE_NO_BK_BRACES \ - | RE_INVALID_INTERVAL_ORD) +/* POSIX grep -E behavior is no longer incompatible with GNU. */ +# define RE_SYNTAX_POSIX_EGREP \ + RE_SYNTAX_EGREP /* P1003.2/D11.2, section 4.20.7.1, lines 5078ff. */ -#define RE_SYNTAX_ED RE_SYNTAX_POSIX_BASIC +# define RE_SYNTAX_ED RE_SYNTAX_POSIX_BASIC -#define RE_SYNTAX_SED RE_SYNTAX_POSIX_BASIC +# define RE_SYNTAX_SED RE_SYNTAX_POSIX_BASIC /* Syntax bits common to both basic and extended POSIX regex syntax. */ -#define _RE_SYNTAX_POSIX_COMMON \ +# define _RE_SYNTAX_POSIX_COMMON \ (RE_CHAR_CLASSES | RE_DOT_NEWLINE | RE_DOT_NOT_NULL \ | RE_INTERVALS | RE_NO_EMPTY_RANGES) -#define RE_SYNTAX_POSIX_BASIC \ +# define RE_SYNTAX_POSIX_BASIC \ (_RE_SYNTAX_POSIX_COMMON | RE_BK_PLUS_QM | RE_CONTEXT_INVALID_DUP) /* Differs from ..._POSIX_BASIC only in that RE_BK_PLUS_QM becomes RE_LIMITED_OPS, i.e., \? \+ \| are not recognized. Actually, this isn't minimal, since other operators, such as \`, aren't disabled. */ -#define RE_SYNTAX_POSIX_MINIMAL_BASIC \ +# define RE_SYNTAX_POSIX_MINIMAL_BASIC \ (_RE_SYNTAX_POSIX_COMMON | RE_LIMITED_OPS) -#define RE_SYNTAX_POSIX_EXTENDED \ +# define RE_SYNTAX_POSIX_EXTENDED \ (_RE_SYNTAX_POSIX_COMMON | RE_CONTEXT_INDEP_ANCHORS \ | RE_CONTEXT_INDEP_OPS | RE_NO_BK_BRACES \ | RE_NO_BK_PARENS | RE_NO_BK_VBAR \ @@ -248,25 +275,35 @@ extern reg_syntax_t re_syntax_options; /* Differs from ..._POSIX_EXTENDED in that RE_CONTEXT_INDEP_OPS is removed and RE_NO_BK_REFS is added. */ -#define RE_SYNTAX_POSIX_MINIMAL_EXTENDED \ +# define RE_SYNTAX_POSIX_MINIMAL_EXTENDED \ (_RE_SYNTAX_POSIX_COMMON | RE_CONTEXT_INDEP_ANCHORS \ | RE_CONTEXT_INVALID_OPS | RE_NO_BK_BRACES \ | RE_NO_BK_PARENS | RE_NO_BK_REFS \ | RE_NO_BK_VBAR | RE_UNMATCHED_RIGHT_PAREN_ORD) /* [[[end syntaxes]]] */ - -/* Maximum number of duplicates an interval can allow. Some systems - (erroneously) define this in other header files, but we want our + +/* Maximum number of duplicates an interval can allow. POSIX-conforming + systems might define this in <limits.h>, but we want our value, so remove any previous define. */ +# ifdef _REGEX_INCLUDE_LIMITS_H +# include <limits.h> +# endif # ifdef RE_DUP_MAX # undef RE_DUP_MAX # endif -/* If sizeof(int) == 2, then ((1 << 15) - 1) overflows. */ + +/* RE_DUP_MAX is 2**15 - 1 because an earlier implementation stored + the counter as a 2-byte signed integer. This is no longer true, so + RE_DUP_MAX could be increased to (INT_MAX / 10 - 1), or to + ((SIZE_MAX - 9) / 10) if _REGEX_LARGE_OFFSETS is defined. + However, there would be a huge performance problem if someone + actually used a pattern like a\{214748363\}, so RE_DUP_MAX retains + its historical value. */ # define RE_DUP_MAX (0x7fff) #endif -/* POSIX `cflags' bits (i.e., information for `regcomp'). */ +/* POSIX 'cflags' bits (i.e., information for 'regcomp'). */ /* If this bit is set, then use extended regular expression syntax. If not set, then use basic regular expression syntax. */ @@ -274,19 +311,19 @@ extern reg_syntax_t re_syntax_options; /* If this bit is set, then ignore case when matching. If not set, then case is significant. */ -#define REG_ICASE (REG_EXTENDED << 1) +#define REG_ICASE (1 << 1) /* If this bit is set, then anchors do not match at newline characters in the string. If not set, then anchors do match at newlines. */ -#define REG_NEWLINE (REG_ICASE << 1) +#define REG_NEWLINE (1 << 2) /* If this bit is set, then report only success or fail in regexec. If not set, then returns differ between not matching and errors. */ -#define REG_NOSUB (REG_NEWLINE << 1) +#define REG_NOSUB (1 << 3) -/* POSIX `eflags' bits (i.e., information for regexec). */ +/* POSIX 'eflags' bits (i.e., information for regexec). */ /* If this bit is set, then the beginning-of-line operator doesn't match the beginning of the string (presumably because it's not the @@ -304,41 +341,60 @@ extern reg_syntax_t re_syntax_options; /* If any error codes are removed, changed, or added, update the - `re_error_msg' table in regex.c. */ + '__re_error_msgid' table in regcomp.c. */ + typedef enum { -#if defined _XOPEN_SOURCE || defined __USE_XOPEN2K - REG_ENOSYS = -1, /* This will never happen for this implementation. */ -#endif - - REG_NOERROR = 0, /* Success. */ - REG_NOMATCH, /* Didn't find a match (for regexec). */ + _REG_ENOSYS = -1, /* This will never happen for this implementation. */ + _REG_NOERROR = 0, /* Success. */ + _REG_NOMATCH, /* Didn't find a match (for regexec). */ /* POSIX regcomp return error codes. (In the order listed in the standard.) */ - REG_BADPAT, /* Invalid pattern. */ - REG_ECOLLATE, /* Invalid collating element. */ - REG_ECTYPE, /* Invalid character class name. */ - REG_EESCAPE, /* Trailing backslash. */ - REG_ESUBREG, /* Invalid back reference. */ - REG_EBRACK, /* Unmatched left bracket. */ - REG_EPAREN, /* Parenthesis imbalance. */ - REG_EBRACE, /* Unmatched \{. */ - REG_BADBR, /* Invalid contents of \{\}. */ - REG_ERANGE, /* Invalid range end. */ - REG_ESPACE, /* Ran out of memory. */ - REG_BADRPT, /* No preceding re for repetition op. */ + _REG_BADPAT, /* Invalid pattern. */ + _REG_ECOLLATE, /* Invalid collating element. */ + _REG_ECTYPE, /* Invalid character class name. */ + _REG_EESCAPE, /* Trailing backslash. */ + _REG_ESUBREG, /* Invalid back reference. */ + _REG_EBRACK, /* Unmatched left bracket. */ + _REG_EPAREN, /* Parenthesis imbalance. */ + _REG_EBRACE, /* Unmatched \{. */ + _REG_BADBR, /* Invalid contents of \{\}. */ + _REG_ERANGE, /* Invalid range end. */ + _REG_ESPACE, /* Ran out of memory. */ + _REG_BADRPT, /* No preceding re for repetition op. */ /* Error codes we've added. */ - REG_EEND, /* Premature end. */ - REG_ESIZE, /* Compiled pattern bigger than 2^16 bytes. */ - REG_ERPAREN /* Unmatched ) or \); not returned from regcomp. */ + _REG_EEND, /* Premature end. */ + _REG_ESIZE, /* Too large (e.g., repeat count too large). */ + _REG_ERPAREN /* Unmatched ) or \); not returned from regcomp. */ } reg_errcode_t; + +#if defined _XOPEN_SOURCE || defined __USE_XOPEN2K +# define REG_ENOSYS _REG_ENOSYS +#endif +#define REG_NOERROR _REG_NOERROR +#define REG_NOMATCH _REG_NOMATCH +#define REG_BADPAT _REG_BADPAT +#define REG_ECOLLATE _REG_ECOLLATE +#define REG_ECTYPE _REG_ECTYPE +#define REG_EESCAPE _REG_EESCAPE +#define REG_ESUBREG _REG_ESUBREG +#define REG_EBRACK _REG_EBRACK +#define REG_EPAREN _REG_EPAREN +#define REG_EBRACE _REG_EBRACE +#define REG_BADBR _REG_BADBR +#define REG_ERANGE _REG_ERANGE +#define REG_ESPACE _REG_ESPACE +#define REG_BADRPT _REG_BADRPT +#define REG_EEND _REG_EEND +#define REG_ESIZE _REG_ESIZE +#define REG_ERPAREN _REG_ERPAREN /* This data structure represents a compiled pattern. Before calling - the pattern compiler, the fields `buffer', `allocated', `fastmap', - and `translate' can be set. After the pattern has been compiled, - the fields `re_nsub', `not_bol' and `not_eol' are available. All + the pattern compiler, the fields 'buffer', 'allocated', 'fastmap', + and 'translate' can be set. After the pattern has been compiled, + the fields 're_nsub', 'not_bol' and 'not_eol' are available. All other fields are private to the regex routines. */ #ifndef RE_TRANSLATE_TYPE @@ -356,16 +412,15 @@ typedef enum struct re_pattern_buffer { - /* Space that holds the compiled pattern. It is declared as - `unsigned char *' because its elements are sometimes used as - array indexes. */ - unsigned char *__REPB_PREFIX(buffer); + /* Space that holds the compiled pattern. The type + 'struct re_dfa_t' is private and is not declared here. */ + struct re_dfa_t *__REPB_PREFIX(buffer); - /* Number of bytes to which `buffer' points. */ - unsigned long int __REPB_PREFIX(allocated); + /* Number of bytes to which 'buffer' points. */ + __re_long_size_t __REPB_PREFIX(allocated); - /* Number of bytes actually used in `buffer'. */ - unsigned long int __REPB_PREFIX(used); + /* Number of bytes actually used in 'buffer'. */ + __re_long_size_t __REPB_PREFIX(used); /* Syntax setting with which the pattern was compiled. */ reg_syntax_t __REPB_PREFIX(syntax); @@ -385,13 +440,13 @@ struct re_pattern_buffer size_t re_nsub; /* Zero if this pattern cannot match the empty string, one else. - Well, in truth it's used only in `re_search_2', to see whether or + Well, in truth it's used only in 're_search_2', to see whether or not we should use the fastmap, so we don't set this absolutely - perfectly; see `re_compile_fastmap' (the `duplicate' case). */ + perfectly; see 're_compile_fastmap' (the "duplicate" case). */ unsigned __REPB_PREFIX(can_be_null) : 1; - /* If REGS_UNALLOCATED, allocate space in the `regs' structure - for `max (RE_NREGS, re_nsub + 1)' groups. + /* If REGS_UNALLOCATED, allocate space in the 'regs' structure + for 'max (RE_NREGS, re_nsub + 1)' groups. If REGS_REALLOCATE, reallocate space if necessary. If REGS_FIXED, use what's there. */ #ifdef __USE_GNU @@ -401,11 +456,11 @@ struct re_pattern_buffer #endif unsigned __REPB_PREFIX(regs_allocated) : 2; - /* Set to zero when `regex_compile' compiles a pattern; set to one - by `re_compile_fastmap' if it updates the fastmap. */ + /* Set to zero when 're_compile_pattern' compiles a pattern; set to + one by 're_compile_fastmap' if it updates the fastmap. */ unsigned __REPB_PREFIX(fastmap_accurate) : 1; - /* If set, `re_match_2' does not return information about + /* If set, 're_match_2' does not return information about subexpressions. */ unsigned __REPB_PREFIX(no_sub) : 1; @@ -423,7 +478,17 @@ struct re_pattern_buffer typedef struct re_pattern_buffer regex_t; /* Type for byte offsets within the string. POSIX mandates this. */ +#ifdef _REGEX_LARGE_OFFSETS +/* POSIX 1003.1-2008 requires that regoff_t be at least as wide as + ptrdiff_t and ssize_t. We don't know of any hosts where ptrdiff_t + is wider than ssize_t, so ssize_t is safe. ptrdiff_t is not + visible here, so use ssize_t. */ +typedef ssize_t regoff_t; +#else +/* The traditional GNU regex implementation mishandles strings longer + than INT_MAX. */ typedef int regoff_t; +#endif #ifdef __USE_GNU @@ -431,15 +496,15 @@ typedef int regoff_t; regex.texinfo for a full description of what registers match. */ struct re_registers { - unsigned num_regs; + __re_size_t num_regs; regoff_t *start; regoff_t *end; }; -/* If `regs_allocated' is REGS_UNALLOCATED in the pattern buffer, - `re_match_2' returns information about at least this many registers - the first time a `regs' structure is passed. */ +/* If 'regs_allocated' is REGS_UNALLOCATED in the pattern buffer, + 're_match_2' returns information about at least this many registers + the first time a 'regs' structure is passed. */ # ifndef RE_NREGS # define RE_NREGS 30 # endif @@ -447,7 +512,7 @@ struct re_registers /* POSIX specification for registers. Aside from the different names than - `re_registers', POSIX uses an array of structures, instead of a + 're_registers', POSIX uses an array of structures, instead of a structure of arrays. */ typedef struct { @@ -459,17 +524,17 @@ typedef struct #ifdef __USE_GNU /* Sets the current default syntax to SYNTAX, and return the old syntax. - You can also simply assign to the `re_syntax_options' variable. */ + You can also simply assign to the 're_syntax_options' variable. */ extern reg_syntax_t re_set_syntax (reg_syntax_t __syntax); /* Compile the regular expression PATTERN, with length LENGTH - and syntax given by the global `re_syntax_options', into the buffer + and syntax given by the global 're_syntax_options', into the buffer BUFFER. Return NULL if successful, and an error string if not. - To free the allocated storage, you must call `regfree' on BUFFER. - Note that the translate table must either have been initialised by - `regcomp', with a malloc'ed value, or set to NULL before calling - `regfree'. */ + To free the allocated storage, you must call 'regfree' on BUFFER. + Note that the translate table must either have been initialized by + 'regcomp', with a malloc'ed value, or set to NULL before calling + 'regfree'. */ extern const char *re_compile_pattern (const char *__pattern, size_t __length, struct re_pattern_buffer *__buffer); @@ -485,47 +550,52 @@ extern int re_compile_fastmap (struct re_pattern_buffer *__buffer); characters. Return the starting position of the match, -1 for no match, or -2 for an internal error. Also return register information in REGS (if REGS and BUFFER->no_sub are nonzero). */ -extern int re_search (struct re_pattern_buffer *__buffer, const char *__string, - int __length, int __start, int __range, - struct re_registers *__regs); +extern regoff_t re_search (struct re_pattern_buffer *__buffer, + const char *__String, regoff_t __length, + regoff_t __start, regoff_t __range, + struct re_registers *__regs); -/* Like `re_search', but search in the concatenation of STRING1 and +/* Like 're_search', but search in the concatenation of STRING1 and STRING2. Also, stop searching at index START + STOP. */ -extern int re_search_2 (struct re_pattern_buffer *__buffer, - const char *__string1, int __length1, - const char *__string2, int __length2, int __start, - int __range, struct re_registers *__regs, int __stop); +extern regoff_t re_search_2 (struct re_pattern_buffer *__buffer, + const char *__string1, regoff_t __length1, + const char *__string2, regoff_t __length2, + regoff_t __start, regoff_t __range, + struct re_registers *__regs, + regoff_t __stop); -/* Like `re_search', but return how many characters in STRING the regexp +/* Like 're_search', but return how many characters in STRING the regexp in BUFFER matched, starting at position START. */ -extern int re_match (struct re_pattern_buffer *__buffer, const char *__string, - int __length, int __start, struct re_registers *__regs); +extern regoff_t re_match (struct re_pattern_buffer *__buffer, + const char *__String, regoff_t __length, + regoff_t __start, struct re_registers *__regs); -/* Relates to `re_match' as `re_search_2' relates to `re_search'. */ -extern int re_match_2 (struct re_pattern_buffer *__buffer, - const char *__string1, int __length1, - const char *__string2, int __length2, int __start, - struct re_registers *__regs, int __stop); +/* Relates to 're_match' as 're_search_2' relates to 're_search'. */ +extern regoff_t re_match_2 (struct re_pattern_buffer *__buffer, + const char *__string1, regoff_t __length1, + const char *__string2, regoff_t __length2, + regoff_t __start, struct re_registers *__regs, + regoff_t __stop); /* Set REGS to hold NUM_REGS registers, storing them in STARTS and ENDS. Subsequent matches using BUFFER and REGS will use this memory for recording register information. STARTS and ENDS must be - allocated with malloc, and must each be at least `NUM_REGS * sizeof + allocated with malloc, and must each be at least 'NUM_REGS * sizeof (regoff_t)' bytes long. If NUM_REGS == 0, then subsequent matches should allocate their own register data. Unless this function is called, the first search or match using - PATTERN_BUFFER will allocate its own register data, without + BUFFER will allocate its own register data, without freeing the old data. */ extern void re_set_registers (struct re_pattern_buffer *__buffer, struct re_registers *__regs, - unsigned int __num_regs, + __re_size_t __num_regs, regoff_t *__starts, regoff_t *__ends); #endif /* Use GNU */ @@ -537,39 +607,46 @@ extern int re_exec (const char *); # endif #endif -/* GCC 2.95 and later have "__restrict"; C99 compilers have - "restrict", and "configure" may have defined "restrict". */ -#ifndef __restrict -# if ! (2 < __GNUC__ || (2 == __GNUC__ && 95 <= __GNUC_MINOR__)) -# if defined restrict || 199901L <= __STDC_VERSION__ -# define __restrict restrict -# else -# define __restrict -# endif +/* For plain 'restrict', use glibc's __restrict if defined. + Otherwise, GCC 2.95 and later have "__restrict"; C99 compilers have + "restrict", and "configure" may have defined "restrict". + Other compilers use __restrict, __restrict__, and _Restrict, and + 'configure' might #define 'restrict' to those words, so pick a + different name. */ +#ifndef _Restrict_ +# if defined __restrict || 2 < __GNUC__ + (95 <= __GNUC_MINOR__) +# define _Restrict_ __restrict +# elif 199901L <= __STDC_VERSION__ || defined restrict +# define _Restrict_ restrict +# else +# define _Restrict_ # endif #endif -/* gcc 3.1 and up support the [restrict] syntax. */ -#ifndef __restrict_arr -# if (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 1)) \ - && !defined __GNUG__ -# define __restrict_arr __restrict +/* For [restrict], use glibc's __restrict_arr if available. + Otherwise, GCC 3.1 (not in C++ mode) and C99 support [restrict]. */ +#ifndef _Restrict_arr_ +# ifdef __restrict_arr +# define _Restrict_arr_ __restrict_arr +# elif ((199901L <= __STDC_VERSION__ || 3 < __GNUC__ + (1 <= __GNUC_MINOR__)) \ + && !defined __GNUG__) +# define _Restrict_arr_ _Restrict_ # else -# define __restrict_arr +# define _Restrict_arr_ # endif #endif /* POSIX compatibility. */ -extern int regcomp (regex_t *__restrict __preg, - const char *__restrict __pattern, +extern int regcomp (regex_t *_Restrict_ __preg, + const char *_Restrict_ __pattern, int __cflags); -extern int regexec (const regex_t *__restrict __preg, - const char *__restrict __string, size_t __nmatch, - regmatch_t __pmatch[__restrict_arr], +extern int regexec (const regex_t *_Restrict_ __preg, + const char *_Restrict_ __String, size_t __nmatch, + regmatch_t __pmatch[_Restrict_arr_], int __eflags); -extern size_t regerror (int __errcode, const regex_t *__restrict __preg, - char *__restrict __errbuf, size_t __errbuf_size); +extern size_t regerror (int __errcode, const regex_t *_Restrict_ __preg, + char *_Restrict_ __errbuf, size_t __errbuf_size); extern void regfree (regex_t *__preg); diff --git a/posix/regex_internal.c b/posix/regex_internal.c index 9062083..7f0083b 100644 --- a/posix/regex_internal.c +++ b/posix/regex_internal.c @@ -15,19 +15,29 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ -static void re_string_construct_common (const char *str, int len, +static void re_string_construct_common (const char *str, Idx len, re_string_t *pstr, - RE_TRANSLATE_TYPE trans, int icase, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa); static re_dfastate_t *create_ci_newstate (const re_dfa_t *dfa, const re_node_set *nodes, - unsigned int hash); + re_hashval_t hash); static re_dfastate_t *create_cd_newstate (const re_dfa_t *dfa, const re_node_set *nodes, unsigned int context, - unsigned int hash); + re_hashval_t hash); +static reg_errcode_t re_string_realloc_buffers (re_string_t *pstr, + Idx new_buf_len); +#ifdef RE_ENABLE_I18N +static void build_wcs_buffer (re_string_t *pstr); +static reg_errcode_t build_wcs_upper_buffer (re_string_t *pstr); +#endif /* RE_ENABLE_I18N */ +static void build_upper_buffer (re_string_t *pstr); +static void re_string_translate_buffer (re_string_t *pstr); +static unsigned int re_string_context_at (const re_string_t *input, Idx idx, + int eflags) __attribute__ ((pure)); /* Functions for string operation. */ @@ -36,11 +46,11 @@ static re_dfastate_t *create_cd_newstate (const re_dfa_t *dfa, static reg_errcode_t __attribute_warn_unused_result__ -re_string_allocate (re_string_t *pstr, const char *str, int len, int init_len, - RE_TRANSLATE_TYPE trans, int icase, const re_dfa_t *dfa) +re_string_allocate (re_string_t *pstr, const char *str, Idx len, Idx init_len, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa) { reg_errcode_t ret; - int init_buf_len; + Idx init_buf_len; /* Ensure at least one character fits into the buffers. */ if (init_len < dfa->mb_cur_max) @@ -64,8 +74,8 @@ re_string_allocate (re_string_t *pstr, const char *str, int len, int init_len, static reg_errcode_t __attribute_warn_unused_result__ -re_string_construct (re_string_t *pstr, const char *str, int len, - RE_TRANSLATE_TYPE trans, int icase, const re_dfa_t *dfa) +re_string_construct (re_string_t *pstr, const char *str, Idx len, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa) { reg_errcode_t ret; memset (pstr, '\0', sizeof (re_string_t)); @@ -127,7 +137,7 @@ re_string_construct (re_string_t *pstr, const char *str, int len, static reg_errcode_t __attribute_warn_unused_result__ -re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) +re_string_realloc_buffers (re_string_t *pstr, Idx new_buf_len) { #ifdef RE_ENABLE_I18N if (pstr->mb_cur_max > 1) @@ -135,8 +145,8 @@ re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) wint_t *new_wcs; /* Avoid overflow in realloc. */ - const size_t max_object_size = MAX (sizeof (wint_t), sizeof (int)); - if (BE (SIZE_MAX / max_object_size < new_buf_len, 0)) + const size_t max_object_size = MAX (sizeof (wint_t), sizeof (Idx)); + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) < new_buf_len, 0)) return REG_ESPACE; new_wcs = re_realloc (pstr->wcs, wint_t, new_buf_len); @@ -145,7 +155,7 @@ re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) pstr->wcs = new_wcs; if (pstr->offsets != NULL) { - int *new_offsets = re_realloc (pstr->offsets, int, new_buf_len); + Idx *new_offsets = re_realloc (pstr->offsets, Idx, new_buf_len); if (BE (new_offsets == NULL, 0)) return REG_ESPACE; pstr->offsets = new_offsets; @@ -166,15 +176,15 @@ re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) static void -re_string_construct_common (const char *str, int len, re_string_t *pstr, - RE_TRANSLATE_TYPE trans, int icase, +re_string_construct_common (const char *str, Idx len, re_string_t *pstr, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa) { pstr->raw_mbs = (const unsigned char *) str; pstr->len = len; pstr->raw_len = len; pstr->trans = trans; - pstr->icase = icase ? 1 : 0; + pstr->icase = icase; pstr->mbs_allocated = (trans != NULL || icase); pstr->mb_cur_max = dfa->mb_cur_max; pstr->is_utf8 = dfa->is_utf8; @@ -206,7 +216,7 @@ build_wcs_buffer (re_string_t *pstr) unsigned char buf[64]; #endif mbstate_t prev_st; - int byte_idx, end_idx, remain_len; + Idx byte_idx, end_idx, remain_len; size_t mbclen; /* Build the buffers from pstr->valid_len to either pstr->len or @@ -269,7 +279,7 @@ __attribute_warn_unused_result__ build_wcs_upper_buffer (re_string_t *pstr) { mbstate_t prev_st; - int src_idx, byte_idx, end_idx, remain_len; + Idx src_idx, byte_idx, end_idx, remain_len; size_t mbclen; #ifdef _LIBC char buf[MB_LEN_MAX]; @@ -307,14 +317,13 @@ build_wcs_upper_buffer (re_string_t *pstr) mbclen = __mbrtowc (&wc, ((const char *) pstr->raw_mbs + pstr->raw_mbs_idx + byte_idx), remain_len, &pstr->cur_state); - if (BE (mbclen + 2 > 2, 1)) + if (BE (mbclen < (size_t) -2, 1)) { - wchar_t wcu = wc; - if (__iswlower (wc)) + wchar_t wcu = __towupper (wc); + if (wcu != wc) { size_t mbcdlen; - wcu = __towupper (wc); mbcdlen = __wcrtomb (buf, wcu, &prev_st); if (BE (mbclen == mbcdlen, 1)) memcpy (pstr->mbs + byte_idx, buf, mbclen); @@ -377,14 +386,13 @@ build_wcs_upper_buffer (re_string_t *pstr) else p = (const char *) pstr->raw_mbs + pstr->raw_mbs_idx + src_idx; mbclen = __mbrtowc (&wc, p, remain_len, &pstr->cur_state); - if (BE (mbclen + 2 > 2, 1)) + if (BE (mbclen < (size_t) -2, 1)) { - wchar_t wcu = wc; - if (__iswlower (wc)) + wchar_t wcu = __towupper (wc); + if (wcu != wc) { size_t mbcdlen; - wcu = __towupper (wc); mbcdlen = __wcrtomb ((char *) buf, wcu, &prev_st); if (BE (mbclen == mbcdlen, 1)) memcpy (pstr->mbs + byte_idx, buf, mbclen); @@ -400,7 +408,7 @@ build_wcs_upper_buffer (re_string_t *pstr) if (pstr->offsets == NULL) { - pstr->offsets = re_malloc (int, pstr->bufs_len); + pstr->offsets = re_malloc (Idx, pstr->bufs_len); if (pstr->offsets == NULL) return REG_ESPACE; @@ -483,11 +491,11 @@ build_wcs_upper_buffer (re_string_t *pstr) /* Skip characters until the index becomes greater than NEW_RAW_IDX. Return the index. */ -static int -re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) +static Idx +re_string_skip_chars (re_string_t *pstr, Idx new_raw_idx, wint_t *last_wc) { mbstate_t prev_st; - int rawbuf_idx; + Idx rawbuf_idx; size_t mbclen; wint_t wc = WEOF; @@ -496,11 +504,11 @@ re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) rawbuf_idx < new_raw_idx;) { wchar_t wc2; - int remain_len = pstr->raw_len - rawbuf_idx; + Idx remain_len = pstr->raw_len - rawbuf_idx; prev_st = pstr->cur_state; mbclen = __mbrtowc (&wc2, (const char *) pstr->raw_mbs + rawbuf_idx, remain_len, &pstr->cur_state); - if (BE ((ssize_t) mbclen <= 0, 0)) + if (BE (mbclen == (size_t) -2 || mbclen == (size_t) -1 || mbclen == 0, 0)) { /* We treat these cases as a single byte character. */ if (mbclen == 0 || remain_len == 0) @@ -511,7 +519,7 @@ re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) pstr->cur_state = prev_st; } else - wc = (wint_t) wc2; + wc = wc2; /* Then proceed the next character. */ rawbuf_idx += mbclen; } @@ -526,7 +534,7 @@ re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) static void build_upper_buffer (re_string_t *pstr) { - int char_idx, end_idx; + Idx char_idx, end_idx; end_idx = (pstr->bufs_len > pstr->len) ? pstr->len : pstr->bufs_len; for (char_idx = pstr->valid_len; char_idx < end_idx; ++char_idx) @@ -534,10 +542,7 @@ build_upper_buffer (re_string_t *pstr) int ch = pstr->raw_mbs[pstr->raw_mbs_idx + char_idx]; if (BE (pstr->trans != NULL, 0)) ch = pstr->trans[ch]; - if (islower (ch)) - pstr->mbs[char_idx] = toupper (ch); - else - pstr->mbs[char_idx] = ch; + pstr->mbs[char_idx] = toupper (ch); } pstr->valid_len = char_idx; pstr->valid_raw_len = char_idx; @@ -548,7 +553,7 @@ build_upper_buffer (re_string_t *pstr) static void re_string_translate_buffer (re_string_t *pstr) { - int buf_idx, end_idx; + Idx buf_idx, end_idx; end_idx = (pstr->bufs_len > pstr->len) ? pstr->len : pstr->bufs_len; for (buf_idx = pstr->valid_len; buf_idx < end_idx; ++buf_idx) @@ -567,10 +572,13 @@ re_string_translate_buffer (re_string_t *pstr) static reg_errcode_t __attribute_warn_unused_result__ -re_string_reconstruct (re_string_t *pstr, int idx, int eflags) +re_string_reconstruct (re_string_t *pstr, Idx idx, int eflags) { - int offset = idx - pstr->raw_mbs_idx; - if (BE (offset < 0, 0)) + Idx offset; + + if (BE (pstr->raw_mbs_idx <= idx, 0)) + offset = idx - pstr->raw_mbs_idx; + else { /* Reset buffer. */ #ifdef RE_ENABLE_I18N @@ -599,7 +607,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) #ifdef RE_ENABLE_I18N if (BE (pstr->offsets_needed, 0)) { - int low = 0, high = pstr->valid_len, mid; + Idx low = 0, high = pstr->valid_len, mid; do { mid = (high + low) / 2; @@ -683,7 +691,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) { #ifdef RE_ENABLE_I18N /* No, skip all characters until IDX. */ - int prev_valid_len = pstr->valid_len; + Idx prev_valid_len = pstr->valid_len; if (BE (pstr->offsets_needed, 0)) { @@ -696,7 +704,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) #ifdef RE_ENABLE_I18N if (pstr->mb_cur_max > 1) { - int wcs_idx; + Idx wcs_idx; wint_t wc = WEOF; if (pstr->is_utf8) @@ -726,7 +734,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) { mbstate_t cur_state; wchar_t wc2; - int mlen = raw + pstr->len - p; + Idx mlen = raw + pstr->len - p; unsigned char buf[6]; size_t mbclen; @@ -826,10 +834,11 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) } static unsigned char -__attribute ((pure)) -re_string_peek_byte_case (const re_string_t *pstr, int idx) +__attribute__ ((pure)) +re_string_peek_byte_case (const re_string_t *pstr, Idx idx) { - int ch, off; + int ch; + Idx off; /* Handle the common (easiest) cases first. */ if (BE (!pstr->mbs_allocated, 1)) @@ -870,7 +879,8 @@ re_string_fetch_byte_case (re_string_t *pstr) #ifdef RE_ENABLE_I18N if (pstr->offsets_needed) { - int off, ch; + Idx off; + int ch; /* For tr_TR.UTF-8 [[:islower:]] there is [[: CAPITAL LETTER I WITH DOT lower:]] in mbs. Skip @@ -911,7 +921,7 @@ re_string_destruct (re_string_t *pstr) /* Return the context at IDX in INPUT. */ static unsigned int -re_string_context_at (const re_string_t *input, int idx, int eflags) +re_string_context_at (const re_string_t *input, Idx idx, int eflags) { int c; if (BE (idx < 0, 0)) @@ -925,7 +935,7 @@ re_string_context_at (const re_string_t *input, int idx, int eflags) if (input->mb_cur_max > 1) { wint_t wc; - int wc_idx = idx; + Idx wc_idx = idx; while(input->wcs[wc_idx] == WEOF) { #if defined DEBUG && DEBUG @@ -956,23 +966,23 @@ re_string_context_at (const re_string_t *input, int idx, int eflags) static reg_errcode_t __attribute_warn_unused_result__ -re_node_set_alloc (re_node_set *set, int size) +re_node_set_alloc (re_node_set *set, Idx size) { set->alloc = size; set->nelem = 0; - set->elems = re_malloc (int, size); - if (BE (set->elems == NULL, 0)) + set->elems = re_malloc (Idx, size); + if (BE (set->elems == NULL, 0) && (MALLOC_0_IS_NONNULL || size != 0)) return REG_ESPACE; return REG_NOERROR; } static reg_errcode_t __attribute_warn_unused_result__ -re_node_set_init_1 (re_node_set *set, int elem) +re_node_set_init_1 (re_node_set *set, Idx elem) { set->alloc = 1; set->nelem = 1; - set->elems = re_malloc (int, 1); + set->elems = re_malloc (Idx, 1); if (BE (set->elems == NULL, 0)) { set->alloc = set->nelem = 0; @@ -984,10 +994,10 @@ re_node_set_init_1 (re_node_set *set, int elem) static reg_errcode_t __attribute_warn_unused_result__ -re_node_set_init_2 (re_node_set *set, int elem1, int elem2) +re_node_set_init_2 (re_node_set *set, Idx elem1, Idx elem2) { set->alloc = 2; - set->elems = re_malloc (int, 2); + set->elems = re_malloc (Idx, 2); if (BE (set->elems == NULL, 0)) return REG_ESPACE; if (elem1 == elem2) @@ -1020,13 +1030,13 @@ re_node_set_init_copy (re_node_set *dest, const re_node_set *src) if (src->nelem > 0) { dest->alloc = dest->nelem; - dest->elems = re_malloc (int, dest->alloc); + dest->elems = re_malloc (Idx, dest->alloc); if (BE (dest->elems == NULL, 0)) { dest->alloc = dest->nelem = 0; return REG_ESPACE; } - memcpy (dest->elems, src->elems, src->nelem * sizeof (int)); + memcpy (dest->elems, src->elems, src->nelem * sizeof (Idx)); } else re_node_set_init_empty (dest); @@ -1042,7 +1052,7 @@ __attribute_warn_unused_result__ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, const re_node_set *src2) { - int i1, i2, is, id, delta, sbase; + Idx i1, i2, is, id, delta, sbase; if (src1->nelem == 0 || src2->nelem == 0) return REG_NOERROR; @@ -1050,8 +1060,8 @@ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, conservative estimate. */ if (src1->nelem + src2->nelem + dest->nelem > dest->alloc) { - int new_alloc = src1->nelem + src2->nelem + dest->alloc; - int *new_elems = re_realloc (dest->elems, int, new_alloc); + Idx new_alloc = src1->nelem + src2->nelem + dest->alloc; + Idx *new_elems = re_realloc (dest->elems, Idx, new_alloc); if (BE (new_elems == NULL, 0)) return REG_ESPACE; dest->elems = new_elems; @@ -1073,7 +1083,7 @@ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, --id; if (id < 0 || dest->elems[id] != src1->elems[i1]) - dest->elems[--sbase] = src1->elems[i1]; + dest->elems[--sbase] = src1->elems[i1]; if (--i1 < 0 || --i2 < 0) break; @@ -1120,7 +1130,7 @@ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, } /* Copy remaining SRC elements. */ - memcpy (dest->elems, dest->elems + sbase, delta * sizeof (int)); + memcpy (dest->elems, dest->elems + sbase, delta * sizeof (Idx)); return REG_NOERROR; } @@ -1133,11 +1143,11 @@ __attribute_warn_unused_result__ re_node_set_init_union (re_node_set *dest, const re_node_set *src1, const re_node_set *src2) { - int i1, i2, id; + Idx i1, i2, id; if (src1 != NULL && src1->nelem > 0 && src2 != NULL && src2->nelem > 0) { dest->alloc = src1->nelem + src2->nelem; - dest->elems = re_malloc (int, dest->alloc); + dest->elems = re_malloc (Idx, dest->alloc); if (BE (dest->elems == NULL, 0)) return REG_ESPACE; } @@ -1165,13 +1175,13 @@ re_node_set_init_union (re_node_set *dest, const re_node_set *src1, if (i1 < src1->nelem) { memcpy (dest->elems + id, src1->elems + i1, - (src1->nelem - i1) * sizeof (int)); + (src1->nelem - i1) * sizeof (Idx)); id += src1->nelem - i1; } else if (i2 < src2->nelem) { memcpy (dest->elems + id, src2->elems + i2, - (src2->nelem - i2) * sizeof (int)); + (src2->nelem - i2) * sizeof (Idx)); id += src2->nelem - i2; } dest->nelem = id; @@ -1185,13 +1195,13 @@ static reg_errcode_t __attribute_warn_unused_result__ re_node_set_merge (re_node_set *dest, const re_node_set *src) { - int is, id, sbase, delta; + Idx is, id, sbase, delta; if (src == NULL || src->nelem == 0) return REG_NOERROR; if (dest->alloc < 2 * src->nelem + dest->nelem) { - int new_alloc = 2 * (src->nelem + dest->alloc); - int *new_buffer = re_realloc (dest->elems, int, new_alloc); + Idx new_alloc = 2 * (src->nelem + dest->alloc); + Idx *new_buffer = re_realloc (dest->elems, Idx, new_alloc); if (BE (new_buffer == NULL, 0)) return REG_ESPACE; dest->elems = new_buffer; @@ -1201,7 +1211,7 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) if (BE (dest->nelem == 0, 0)) { dest->nelem = src->nelem; - memcpy (dest->elems, src->elems, src->nelem * sizeof (int)); + memcpy (dest->elems, src->elems, src->nelem * sizeof (Idx)); return REG_NOERROR; } @@ -1222,7 +1232,7 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) { /* If DEST is exhausted, the remaining items of SRC must be unique. */ sbase -= is + 1; - memcpy (dest->elems + sbase, src->elems, (is + 1) * sizeof (int)); + memcpy (dest->elems + sbase, src->elems, (is + 1) * sizeof (Idx)); } id = dest->nelem - 1; @@ -1251,7 +1261,7 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) { /* Copy remaining SRC elements. */ memcpy (dest->elems, dest->elems + sbase, - delta * sizeof (int)); + delta * sizeof (Idx)); break; } } @@ -1262,38 +1272,33 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) /* Insert the new element ELEM to the re_node_set* SET. SET should not already have ELEM. - return -1 if an error is occured, return 1 otherwise. */ + Return true if successful. */ -static int +static bool __attribute_warn_unused_result__ -re_node_set_insert (re_node_set *set, int elem) +re_node_set_insert (re_node_set *set, Idx elem) { - int idx; + Idx idx; /* In case the set is empty. */ if (set->alloc == 0) - { - if (BE (re_node_set_init_1 (set, elem) == REG_NOERROR, 1)) - return 1; - else - return -1; - } + return BE (re_node_set_init_1 (set, elem) == REG_NOERROR, 1); if (BE (set->nelem, 0) == 0) { /* We already guaranteed above that set->alloc != 0. */ set->elems[0] = elem; ++set->nelem; - return 1; + return true; } /* Realloc if we need. */ if (set->alloc == set->nelem) { - int *new_elems; + Idx *new_elems; set->alloc = set->alloc * 2; - new_elems = re_realloc (set->elems, int, set->alloc); + new_elems = re_realloc (set->elems, Idx, set->alloc); if (BE (new_elems == NULL, 0)) - return -1; + return false; set->elems = new_elems; } @@ -1314,56 +1319,56 @@ re_node_set_insert (re_node_set *set, int elem) /* Insert the new element. */ set->elems[idx] = elem; ++set->nelem; - return 1; + return true; } /* Insert the new element ELEM to the re_node_set* SET. SET should not already have any element greater than or equal to ELEM. - Return -1 if an error is occured, return 1 otherwise. */ + Return true if successful. */ -static int +static bool __attribute_warn_unused_result__ -re_node_set_insert_last (re_node_set *set, int elem) +re_node_set_insert_last (re_node_set *set, Idx elem) { /* Realloc if we need. */ if (set->alloc == set->nelem) { - int *new_elems; + Idx *new_elems; set->alloc = (set->alloc + 1) * 2; - new_elems = re_realloc (set->elems, int, set->alloc); + new_elems = re_realloc (set->elems, Idx, set->alloc); if (BE (new_elems == NULL, 0)) - return -1; + return false; set->elems = new_elems; } /* Insert the new element. */ set->elems[set->nelem++] = elem; - return 1; + return true; } /* Compare two node sets SET1 and SET2. - return 1 if SET1 and SET2 are equivalent, return 0 otherwise. */ + Return true if SET1 and SET2 are equivalent. */ -static int -__attribute ((pure)) +static bool +__attribute__ ((pure)) re_node_set_compare (const re_node_set *set1, const re_node_set *set2) { - int i; + Idx i; if (set1 == NULL || set2 == NULL || set1->nelem != set2->nelem) - return 0; + return false; for (i = set1->nelem ; --i >= 0 ; ) if (set1->elems[i] != set2->elems[i]) - return 0; - return 1; + return false; + return true; } /* Return (idx + 1) if SET contains the element ELEM, return 0 otherwise. */ -static int -__attribute ((pure)) -re_node_set_contains (const re_node_set *set, int elem) +static Idx +__attribute__ ((pure)) +re_node_set_contains (const re_node_set *set, Idx elem) { - unsigned int idx, right, mid; + __re_size_t idx, right, mid; if (set->nelem <= 0) return 0; @@ -1382,7 +1387,7 @@ re_node_set_contains (const re_node_set *set, int elem) } static void -re_node_set_remove_at (re_node_set *set, int idx) +re_node_set_remove_at (re_node_set *set, Idx idx) { if (idx < 0 || idx >= set->nelem) return; @@ -1393,37 +1398,42 @@ re_node_set_remove_at (re_node_set *set, int idx) /* Add the token TOKEN to dfa->nodes, and return the index of the token. - Or return -1, if an error will be occured. */ + Or return -1 if an error occurred. */ -static int +static Idx re_dfa_add_node (re_dfa_t *dfa, re_token_t token) { - int type = token.type; if (BE (dfa->nodes_len >= dfa->nodes_alloc, 0)) { size_t new_nodes_alloc = dfa->nodes_alloc * 2; - int *new_nexts, *new_indices; + Idx *new_nexts, *new_indices; re_node_set *new_edests, *new_eclosures; re_token_t *new_nodes; /* Avoid overflows in realloc. */ const size_t max_object_size = MAX (sizeof (re_token_t), MAX (sizeof (re_node_set), - sizeof (int))); - if (BE (SIZE_MAX / max_object_size < new_nodes_alloc, 0)) + sizeof (Idx))); + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) < new_nodes_alloc, 0)) return -1; new_nodes = re_realloc (dfa->nodes, re_token_t, new_nodes_alloc); if (BE (new_nodes == NULL, 0)) return -1; dfa->nodes = new_nodes; - new_nexts = re_realloc (dfa->nexts, int, new_nodes_alloc); - new_indices = re_realloc (dfa->org_indices, int, new_nodes_alloc); + new_nexts = re_realloc (dfa->nexts, Idx, new_nodes_alloc); + new_indices = re_realloc (dfa->org_indices, Idx, new_nodes_alloc); new_edests = re_realloc (dfa->edests, re_node_set, new_nodes_alloc); new_eclosures = re_realloc (dfa->eclosures, re_node_set, new_nodes_alloc); if (BE (new_nexts == NULL || new_indices == NULL || new_edests == NULL || new_eclosures == NULL, 0)) - return -1; + { + re_free (new_nexts); + re_free (new_indices); + re_free (new_edests); + re_free (new_eclosures); + return -1; + } dfa->nexts = new_nexts; dfa->org_indices = new_indices; dfa->edests = new_edests; @@ -1434,7 +1444,8 @@ re_dfa_add_node (re_dfa_t *dfa, re_token_t token) dfa->nodes[dfa->nodes_len].constraint = 0; #ifdef RE_ENABLE_I18N dfa->nodes[dfa->nodes_len].accept_mb = - (type == OP_PERIOD && dfa->mb_cur_max > 1) || type == COMPLEX_BRACKET; + ((token.type == OP_PERIOD && dfa->mb_cur_max > 1) + || token.type == COMPLEX_BRACKET); #endif dfa->nexts[dfa->nodes_len] = -1; re_node_set_init_empty (dfa->edests + dfa->nodes_len); @@ -1442,11 +1453,11 @@ re_dfa_add_node (re_dfa_t *dfa, re_token_t token) return dfa->nodes_len++; } -static inline unsigned int +static re_hashval_t calc_state_hash (const re_node_set *nodes, unsigned int context) { - unsigned int hash = nodes->nelem + context; - int i; + re_hashval_t hash = nodes->nelem + context; + Idx i; for (i = 0 ; i < nodes->nelem ; i++) hash += nodes->elems[i]; return hash; @@ -1466,10 +1477,14 @@ __attribute_warn_unused_result__ re_acquire_state (reg_errcode_t *err, const re_dfa_t *dfa, const re_node_set *nodes) { - unsigned int hash; + re_hashval_t hash; re_dfastate_t *new_state; struct re_state_table_entry *spot; - int i; + Idx i; +#if defined GCC_LINT || defined lint + /* Suppress bogus uninitialized-variable warnings. */ + *err = REG_NOERROR; +#endif if (BE (nodes->nelem == 0, 0)) { *err = REG_NOERROR; @@ -1510,10 +1525,14 @@ __attribute_warn_unused_result__ re_acquire_state_context (reg_errcode_t *err, const re_dfa_t *dfa, const re_node_set *nodes, unsigned int context) { - unsigned int hash; + re_hashval_t hash; re_dfastate_t *new_state; struct re_state_table_entry *spot; - int i; + Idx i; +#if defined GCC_LINT || defined lint + /* Suppress bogus uninitialized-variable warnings. */ + *err = REG_NOERROR; +#endif if (nodes->nelem == 0) { *err = REG_NOERROR; @@ -1530,7 +1549,7 @@ re_acquire_state_context (reg_errcode_t *err, const re_dfa_t *dfa, && re_node_set_compare (state->entrance_nodes, nodes)) return state; } - /* There are no appropriate state in `dfa', create the new one. */ + /* There are no appropriate state in 'dfa', create the new one. */ new_state = create_cd_newstate (dfa, nodes, context, hash); if (BE (new_state == NULL, 0)) *err = REG_ESPACE; @@ -1545,11 +1564,11 @@ re_acquire_state_context (reg_errcode_t *err, const re_dfa_t *dfa, static reg_errcode_t __attribute_warn_unused_result__ register_state (const re_dfa_t *dfa, re_dfastate_t *newstate, - unsigned int hash) + re_hashval_t hash) { struct re_state_table_entry *spot; reg_errcode_t err; - int i; + Idx i; newstate->hash = hash; err = re_node_set_alloc (&newstate->non_eps_nodes, newstate->nodes.nelem); @@ -1557,16 +1576,16 @@ register_state (const re_dfa_t *dfa, re_dfastate_t *newstate, return REG_ESPACE; for (i = 0; i < newstate->nodes.nelem; i++) { - int elem = newstate->nodes.elems[i]; + Idx elem = newstate->nodes.elems[i]; if (!IS_EPSILON_NODE (dfa->nodes[elem].type)) - if (re_node_set_insert_last (&newstate->non_eps_nodes, elem) < 0) + if (! re_node_set_insert_last (&newstate->non_eps_nodes, elem)) return REG_ESPACE; } spot = dfa->state_table + (hash & dfa->state_hash_mask); if (BE (spot->alloc <= spot->num, 0)) { - int new_alloc = 2 * spot->num + 2; + Idx new_alloc = 2 * spot->num + 2; re_dfastate_t **new_array = re_realloc (spot->array, re_dfastate_t *, new_alloc); if (BE (new_array == NULL, 0)) @@ -1600,9 +1619,9 @@ free_state (re_dfastate_t *state) static re_dfastate_t * __attribute_warn_unused_result__ create_ci_newstate (const re_dfa_t *dfa, const re_node_set *nodes, - unsigned int hash) + re_hashval_t hash) { - int i; + Idx i; reg_errcode_t err; re_dfastate_t *newstate; @@ -1650,9 +1669,9 @@ create_ci_newstate (const re_dfa_t *dfa, const re_node_set *nodes, static re_dfastate_t * __attribute_warn_unused_result__ create_cd_newstate (const re_dfa_t *dfa, const re_node_set *nodes, - unsigned int context, unsigned int hash) + unsigned int context, re_hashval_t hash) { - int i, nctx_nodes = 0; + Idx i, nctx_nodes = 0; reg_errcode_t err; re_dfastate_t *newstate; diff --git a/posix/regex_internal.h b/posix/regex_internal.h index 41bf2d3..3b836ed 100644 --- a/posix/regex_internal.h +++ b/posix/regex_internal.h @@ -15,7 +15,7 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ + <https://www.gnu.org/licenses/>. */ #ifndef _REGEX_INTERNAL_H #define _REGEX_INTERNAL_H 1 @@ -26,35 +26,78 @@ #include <stdlib.h> #include <string.h> -#if defined HAVE_LANGINFO_H || defined HAVE_LANGINFO_CODESET || defined _LIBC -# include <langinfo.h> -#endif -#if defined HAVE_LOCALE_H || defined _LIBC -# include <locale.h> +#include <langinfo.h> +#include <locale.h> +#include <wchar.h> +#include <wctype.h> +#include <stdbool.h> +#include <stdint.h> + +/* Properties of integers. Although Gnulib has intprops.h, glibc does + without for now. */ +#ifndef _LIBC +# include "intprops.h" +#else +/* True if the real type T is signed. */ +# define TYPE_SIGNED(t) (! ((t) 0 < (t) -1)) + +/* True if adding the nonnegative Idx values A and B would overflow. + If false, set *R to A + B. A, B, and R may be evaluated more than + once, or zero times. Although this is not a full implementation of + Gnulib INT_ADD_WRAPV, it is good enough for glibc regex code. + FIXME: This implementation is a fragile stopgap, and this file would + be simpler and more robust if intprops.h were migrated into glibc. */ +# define INT_ADD_WRAPV(a, b, r) \ + (IDX_MAX - (a) < (b) ? true : (*(r) = (a) + (b), false)) #endif -#if defined HAVE_WCHAR_H || defined _LIBC -# include <wchar.h> -#endif /* HAVE_WCHAR_H || _LIBC */ -#if defined HAVE_WCTYPE_H || defined _LIBC -# include <wctype.h> -#endif /* HAVE_WCTYPE_H || _LIBC */ -#if defined HAVE_STDBOOL_H || defined _LIBC -# include <stdbool.h> -#endif /* HAVE_STDBOOL_H || _LIBC */ -#if defined HAVE_STDINT_H || defined _LIBC -# include <stdint.h> -#endif /* HAVE_STDINT_H || _LIBC */ -#if defined _LIBC + +#ifdef _LIBC # include <libc-lock.h> +# define lock_define(name) __libc_lock_define (, name) +# define lock_init(lock) (__libc_lock_init (lock), 0) +# define lock_fini(lock) ((void) 0) +# define lock_lock(lock) __libc_lock_lock (lock) +# define lock_unlock(lock) __libc_lock_unlock (lock) +#elif defined GNULIB_LOCK && !defined USE_UNLOCKED_IO +# include "glthread/lock.h" + /* Use gl_lock_define if empty macro arguments are known to work. + Otherwise, fall back on less-portable substitutes. */ +# if ((defined __GNUC__ && !defined __STRICT_ANSI__) \ + || (defined __STDC_VERSION__ && 199901L <= __STDC_VERSION__)) +# define lock_define(name) gl_lock_define (, name) +# elif USE_POSIX_THREADS +# define lock_define(name) pthread_mutex_t name; +# elif USE_PTH_THREADS +# define lock_define(name) pth_mutex_t name; +# elif USE_SOLARIS_THREADS +# define lock_define(name) mutex_t name; +# elif USE_WINDOWS_THREADS +# define lock_define(name) gl_lock_t name; +# else +# define lock_define(name) +# endif +# define lock_init(lock) glthread_lock_init (&(lock)) +# define lock_fini(lock) glthread_lock_destroy (&(lock)) +# define lock_lock(lock) glthread_lock_lock (&(lock)) +# define lock_unlock(lock) glthread_lock_unlock (&(lock)) +#elif defined GNULIB_PTHREAD && !defined USE_UNLOCKED_IO +# include <pthread.h> +# define lock_define(name) pthread_mutex_t name; +# define lock_init(lock) pthread_mutex_init (&(lock), 0) +# define lock_fini(lock) pthread_mutex_destroy (&(lock)) +# define lock_lock(lock) pthread_mutex_lock (&(lock)) +# define lock_unlock(lock) pthread_mutex_unlock (&(lock)) #else -# define __libc_lock_define(CLASS,NAME) -# define __libc_lock_init(NAME) do { } while (0) -# define __libc_lock_lock(NAME) do { } while (0) -# define __libc_lock_unlock(NAME) do { } while (0) +# define lock_define(name) +# define lock_init(lock) 0 +# define lock_fini(lock) ((void) 0) + /* The 'dfa' avoids an "unused variable 'dfa'" warning from GCC. */ +# define lock_lock(lock) ((void) dfa) +# define lock_unlock(lock) ((void) 0) #endif /* In case that the system doesn't have isblank(). */ -#if !defined _LIBC && !defined HAVE_ISBLANK && !defined isblank +#if !defined _LIBC && ! (defined isblank || (HAVE_ISBLANK && HAVE_DECL_ISBLANK)) # define isblank(ch) ((ch) == ' ' || (ch) == '\t') #endif @@ -75,6 +118,7 @@ __dcgettext (_libc_intl_domainname, msgid, LC_MESSAGES) # endif #else +# undef gettext # define gettext(msgid) (msgid) #endif @@ -84,23 +128,17 @@ # define gettext_noop(String) String #endif -/* For loser systems without the definition. */ -#ifndef SIZE_MAX -# define SIZE_MAX ((size_t) -1) -#endif - #if (defined MB_CUR_MAX && HAVE_WCTYPE_H && HAVE_ISWCTYPE) || _LIBC # define RE_ENABLE_I18N #endif -#if __GNUC__ >= 3 -# define BE(expr, val) __builtin_expect (expr, val) -#else -# define BE(expr, val) (expr) -#endif +#define BE(expr, val) __builtin_expect (expr, val) -/* Number of single byte character. */ -#define SBC_MAX 256 +/* Number of ASCII characters. */ +#define ASCII_CHARS 0x80 + +/* Number of single byte characters. */ +#define SBC_MAX (UCHAR_MAX + 1) #define COLL_ELEM_LEN_MAX 8 @@ -110,11 +148,15 @@ /* Rename to standard API for using out of glibc. */ #ifndef _LIBC +# undef __wctype +# undef __iswctype # define __wctype wctype +# define __iswalnum iswalnum # define __iswctype iswctype +# define __towlower towlower +# define __towupper towupper # define __btowc btowc # define __mbrtowc mbrtowc -# define __mempcpy mempcpy # define __wcrtomb wcrtomb # define __regfree regfree # define attribute_hidden @@ -124,32 +166,70 @@ # define __attribute__(arg) #endif -extern const char __re_error_msgid[] attribute_hidden; -extern const size_t __re_error_msgid_idx[] attribute_hidden; +#ifndef SSIZE_MAX +# define SSIZE_MAX ((ssize_t) (SIZE_MAX / 2)) +#endif + +/* The type of indexes into strings. This is signed, not size_t, + since the API requires indexes to fit in regoff_t anyway, and using + signed integers makes the code a bit smaller and presumably faster. + The traditional GNU regex implementation uses int for indexes. + The POSIX-compatible implementation uses a possibly-wider type. + The name 'Idx' is three letters to minimize the hassle of + reindenting a lot of regex code that formerly used 'int'. */ +typedef regoff_t Idx; +#ifdef _REGEX_LARGE_OFFSETS +# define IDX_MAX SSIZE_MAX +#else +# define IDX_MAX INT_MAX +#endif + +/* A hash value, suitable for computing hash tables. */ +typedef __re_size_t re_hashval_t; /* An integer used to represent a set of bits. It must be unsigned, and must be at least as wide as unsigned int. */ typedef unsigned long int bitset_word_t; /* All bits set in a bitset_word_t. */ #define BITSET_WORD_MAX ULONG_MAX -/* Number of bits in a bitset_word_t. */ -#define BITSET_WORD_BITS (sizeof (bitset_word_t) * CHAR_BIT) -/* Number of bitset_word_t in a bit_set. */ -#define BITSET_WORDS (SBC_MAX / BITSET_WORD_BITS) + +/* Number of bits in a bitset_word_t. For portability to hosts with + padding bits, do not use '(sizeof (bitset_word_t) * CHAR_BIT)'; + instead, deduce it directly from BITSET_WORD_MAX. Avoid + greater-than-32-bit integers and unconditional shifts by more than + 31 bits, as they're not portable. */ +#if BITSET_WORD_MAX == 0xffffffffUL +# define BITSET_WORD_BITS 32 +#elif BITSET_WORD_MAX >> 31 >> 4 == 1 +# define BITSET_WORD_BITS 36 +#elif BITSET_WORD_MAX >> 31 >> 16 == 1 +# define BITSET_WORD_BITS 48 +#elif BITSET_WORD_MAX >> 31 >> 28 == 1 +# define BITSET_WORD_BITS 60 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 1 == 1 +# define BITSET_WORD_BITS 64 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 9 == 1 +# define BITSET_WORD_BITS 72 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 31 >> 31 >> 3 == 1 +# define BITSET_WORD_BITS 128 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 7 == 1 +# define BITSET_WORD_BITS 256 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 7 > 1 +# define BITSET_WORD_BITS 257 /* any value > SBC_MAX will do here */ +# if BITSET_WORD_BITS <= SBC_MAX +# error "Invalid SBC_MAX" +# endif +#else +# error "Add case for new bitset_word_t size" +#endif + +/* Number of bitset_word_t values in a bitset_t. */ +#define BITSET_WORDS ((SBC_MAX + BITSET_WORD_BITS - 1) / BITSET_WORD_BITS) + typedef bitset_word_t bitset_t[BITSET_WORDS]; typedef bitset_word_t *re_bitset_ptr_t; typedef const bitset_word_t *re_const_bitset_ptr_t; -#define bitset_set(set,i) \ - (set[i / BITSET_WORD_BITS] |= (bitset_word_t) 1 << i % BITSET_WORD_BITS) -#define bitset_clear(set,i) \ - (set[i / BITSET_WORD_BITS] &= ~((bitset_word_t) 1 << i % BITSET_WORD_BITS)) -#define bitset_contain(set,i) \ - (set[i / BITSET_WORD_BITS] & ((bitset_word_t) 1 << i % BITSET_WORD_BITS)) -#define bitset_empty(set) memset (set, '\0', sizeof (bitset_t)) -#define bitset_set_all(set) memset (set, '\xff', sizeof (bitset_t)) -#define bitset_copy(dest,src) memcpy (dest, src, sizeof (bitset_t)) - #define PREV_WORD_CONSTRAINT 0x0001 #define PREV_NOTWORD_CONSTRAINT 0x0002 #define NEXT_WORD_CONSTRAINT 0x0004 @@ -177,9 +257,9 @@ typedef enum typedef struct { - int alloc; - int nelem; - int *elems; + Idx alloc; + Idx nelem; + Idx *elems; } re_node_set; typedef enum @@ -265,19 +345,19 @@ typedef struct unsigned int non_match : 1; /* # of multibyte characters. */ - int nmbchars; + Idx nmbchars; /* # of collating symbols. */ - int ncoll_syms; + Idx ncoll_syms; /* # of equivalence classes. */ - int nequiv_classes; + Idx nequiv_classes; /* # of range expressions. */ - int nranges; + Idx nranges; /* # of character classes. */ - int nchar_classes; + Idx nchar_classes; } re_charset_t; #endif /* RE_ENABLE_I18N */ @@ -290,10 +370,10 @@ typedef struct #ifdef RE_ENABLE_I18N re_charset_t *mbcset; /* for COMPLEX_BRACKET */ #endif /* RE_ENABLE_I18N */ - int idx; /* for BACK_REF */ + Idx idx; /* for BACK_REF */ re_context_type ctx_type; /* for ANCHOR */ } opr; -#if __GNUC__ >= 2 +#if __GNUC__ >= 2 && !defined __STRICT_ANSI__ re_token_type_t type : 8; #else re_token_type_t type; @@ -324,30 +404,30 @@ struct re_string_t #ifdef RE_ENABLE_I18N /* Store the wide character string which is corresponding to MBS. */ wint_t *wcs; - int *offsets; + Idx *offsets; mbstate_t cur_state; #endif /* Index in RAW_MBS. Each character mbs[i] corresponds to raw_mbs[raw_mbs_idx + i]. */ - int raw_mbs_idx; + Idx raw_mbs_idx; /* The length of the valid characters in the buffers. */ - int valid_len; + Idx valid_len; /* The corresponding number of bytes in raw_mbs array. */ - int valid_raw_len; + Idx valid_raw_len; /* The length of the buffers MBS and WCS. */ - int bufs_len; + Idx bufs_len; /* The index in MBS, which is updated by re_string_fetch_byte. */ - int cur_idx; + Idx cur_idx; /* length of RAW_MBS array. */ - int raw_len; + Idx raw_len; /* This is RAW_LEN - RAW_MBS_IDX + VALID_LEN - VALID_RAW_LEN. */ - int len; + Idx len; /* End of the buffer may be shorter than its length in the cases such as re_match_2, re_search_2. Then, we use STOP for end of the buffer instead of LEN. */ - int raw_stop; + Idx raw_stop; /* This is RAW_STOP - RAW_MBS_IDX adjusted through OFFSETS. */ - int stop; + Idx stop; /* The context of mbs[0]. We store the context independently, since the context of mbs[0] may be different from raw_mbs[0], which is @@ -357,7 +437,7 @@ struct re_string_t RE_TRANSLATE_TYPE trans; /* Copy of re_dfa_t's word_char. */ re_const_bitset_ptr_t word_char; - /* 1 if REG_ICASE. */ + /* true if REG_ICASE. */ unsigned char icase; unsigned char is_utf8; unsigned char map_notascii; @@ -373,18 +453,10 @@ typedef struct re_string_t re_string_t; struct re_dfa_t; typedef struct re_dfa_t re_dfa_t; -#if IS_IN (libc) -static reg_errcode_t re_string_realloc_buffers (re_string_t *pstr, - int new_buf_len); -# ifdef RE_ENABLE_I18N -static void build_wcs_buffer (re_string_t *pstr); -static reg_errcode_t build_wcs_upper_buffer (re_string_t *pstr); -# endif /* RE_ENABLE_I18N */ -static void build_upper_buffer (re_string_t *pstr); -static void re_string_translate_buffer (re_string_t *pstr); -static unsigned int re_string_context_at (const re_string_t *input, int idx, - int eflags) __attribute__ ((pure)); +#ifndef _LIBC +# define IS_IN(libc) false #endif + #define re_string_peek_byte(pstr, offset) \ ((pstr)->mbs[(pstr)->cur_idx + offset]) #define re_string_fetch_byte(pstr) \ @@ -402,7 +474,9 @@ static unsigned int re_string_context_at (const re_string_t *input, int idx, #define re_string_skip_bytes(pstr,idx) ((pstr)->cur_idx += (idx)) #define re_string_set_index(pstr,idx) ((pstr)->cur_idx = (idx)) -#include <alloca.h> +#if defined _LIBC || HAVE_ALLOCA +# include <alloca.h> +#endif #ifndef _LIBC # if HAVE_ALLOCA @@ -414,9 +488,24 @@ static unsigned int re_string_context_at (const re_string_t *input, int idx, # else /* alloca is implemented with malloc, so just use malloc. */ # define __libc_use_alloca(n) 0 +# undef alloca +# define alloca(n) malloc (n) # endif #endif +#ifdef _LIBC +# define MALLOC_0_IS_NONNULL 1 +#elif !defined MALLOC_0_IS_NONNULL +# define MALLOC_0_IS_NONNULL 0 +#endif + +#ifndef MAX +# define MAX(a,b) ((a) < (b) ? (b) : (a)) +#endif +#ifndef MIN +# define MIN(a,b) ((a) < (b) ? (a) : (b)) +#endif + #define re_malloc(t,n) ((t *) malloc ((n) * sizeof (t))) #define re_realloc(p,t,n) ((t *) realloc (p, (n) * sizeof (t))) #define re_free(p) free (p) @@ -431,9 +520,9 @@ struct bin_tree_t re_token_t token; - /* `node_idx' is the index in dfa->nodes, if `type' == 0. - Otherwise `type' indicate the type of this node. */ - int node_idx; + /* 'node_idx' is the index in dfa->nodes, if 'type' == 0. + Otherwise 'type' indicate the type of this node. */ + Idx node_idx; }; typedef struct bin_tree_t bin_tree_t; @@ -477,7 +566,7 @@ typedef struct bin_tree_storage_t bin_tree_storage_t; struct re_dfastate_t { - unsigned int hash; + re_hashval_t hash; re_node_set nodes; re_node_set non_eps_nodes; re_node_set inveclosure; @@ -485,9 +574,9 @@ struct re_dfastate_t struct re_dfastate_t **trtable, **word_trtable; unsigned int context : 4; unsigned int halt : 1; - /* If this state can accept `multi byte'. + /* If this state can accept "multi byte". Note that we refer to multibyte characters, and multi character - collating elements as `multi byte'. */ + collating elements as "multi byte". */ unsigned int accept_mb : 1; /* If this state has backreference node(s). */ unsigned int has_backref : 1; @@ -497,8 +586,8 @@ typedef struct re_dfastate_t re_dfastate_t; struct re_state_table_entry { - int num; - int alloc; + Idx num; + Idx alloc; re_dfastate_t **array; }; @@ -506,8 +595,8 @@ struct re_state_table_entry typedef struct { - int next_idx; - int alloc; + Idx next_idx; + Idx alloc; re_dfastate_t **array; } state_array_t; @@ -515,8 +604,8 @@ typedef struct typedef struct { - int node; - int str_idx; /* The position NODE match at. */ + Idx node; + Idx str_idx; /* The position NODE match at. */ state_array_t path; } re_sub_match_last_t; @@ -526,20 +615,20 @@ typedef struct typedef struct { - int str_idx; - int node; + Idx str_idx; + Idx node; state_array_t *path; - int alasts; /* Allocation size of LASTS. */ - int nlasts; /* The number of LASTS. */ + Idx alasts; /* Allocation size of LASTS. */ + Idx nlasts; /* The number of LASTS. */ re_sub_match_last_t **lasts; } re_sub_match_top_t; struct re_backref_cache_entry { - int node; - int str_idx; - int subexp_from; - int subexp_to; + Idx node; + Idx str_idx; + Idx subexp_from; + Idx subexp_to; char more; char unused; unsigned short int eps_reachable_subexps_map; @@ -557,18 +646,18 @@ typedef struct /* EFLAGS of the argument of regexec. */ int eflags; /* Where the matching ends. */ - int match_last; - int last_node; + Idx match_last; + Idx last_node; /* The state log used by the matcher. */ re_dfastate_t **state_log; - int state_log_top; + Idx state_log_top; /* Back reference cache. */ - int nbkref_ents; - int abkref_ents; + Idx nbkref_ents; + Idx abkref_ents; struct re_backref_cache_entry *bkref_ents; int max_mb_elem_len; - int nsub_tops; - int asub_tops; + Idx nsub_tops; + Idx asub_tops; re_sub_match_top_t **sub_tops; } re_match_context_t; @@ -576,23 +665,23 @@ typedef struct { re_dfastate_t **sifted_states; re_dfastate_t **limited_states; - int last_node; - int last_str_idx; + Idx last_node; + Idx last_str_idx; re_node_set limits; } re_sift_context_t; struct re_fail_stack_ent_t { - int idx; - int node; + Idx idx; + Idx node; regmatch_t *regs; re_node_set eps_via_nodes; }; struct re_fail_stack_t { - int num; - int alloc; + Idx num; + Idx alloc; struct re_fail_stack_ent_t *stack; }; @@ -601,8 +690,8 @@ struct re_dfa_t re_token_t *nodes; size_t nodes_alloc; size_t nodes_len; - int *nexts; - int *org_indices; + Idx *nexts; + Idx *org_indices; re_node_set *edests; re_node_set *eclosures; re_node_set *inveclosures; @@ -616,10 +705,10 @@ struct re_dfa_t re_bitset_ptr_t sb_char; int str_tree_storage_idx; - /* number of subexpressions `re_nsub' is in regex_t. */ - unsigned int state_hash_mask; - int init_node; - int nbackref; /* The number of backreference in this dfa. */ + /* number of subexpressions 're_nsub' is in regex_t. */ + re_hashval_t state_hash_mask; + Idx init_node; + Idx nbackref; /* The number of backreference in this dfa. */ /* Bitmap expressing which backreference is used. */ bitset_word_t used_bkref_map; @@ -636,11 +725,11 @@ struct re_dfa_t int mb_cur_max; bitset_t word_char; reg_syntax_t syntax; - int *subexp_map; + Idx *subexp_map; #ifdef DEBUG char* re_str; #endif - __libc_lock_define (, lock) + lock_define (lock) }; #define re_node_set_init_empty(set) memset (set, '\0', sizeof (re_node_set)) @@ -671,16 +760,60 @@ typedef struct } bracket_elem_t; -/* Inline functions for bitset operation. */ -static void __attribute__ ((unused)) +/* Functions for bitset_t operation. */ + +static inline void +bitset_set (bitset_t set, Idx i) +{ + set[i / BITSET_WORD_BITS] |= (bitset_word_t) 1 << i % BITSET_WORD_BITS; +} + +static inline void +bitset_clear (bitset_t set, Idx i) +{ + set[i / BITSET_WORD_BITS] &= ~ ((bitset_word_t) 1 << i % BITSET_WORD_BITS); +} + +static inline bool +bitset_contain (const bitset_t set, Idx i) +{ + return (set[i / BITSET_WORD_BITS] >> i % BITSET_WORD_BITS) & 1; +} + +static inline void +bitset_empty (bitset_t set) +{ + memset (set, '\0', sizeof (bitset_t)); +} + +static inline void +bitset_set_all (bitset_t set) +{ + memset (set, -1, sizeof (bitset_word_t) * (SBC_MAX / BITSET_WORD_BITS)); + if (SBC_MAX % BITSET_WORD_BITS != 0) + set[BITSET_WORDS - 1] = + ((bitset_word_t) 1 << SBC_MAX % BITSET_WORD_BITS) - 1; +} + +static inline void +bitset_copy (bitset_t dest, const bitset_t src) +{ + memcpy (dest, src, sizeof (bitset_t)); +} + +static inline void bitset_not (bitset_t set) { int bitset_i; - for (bitset_i = 0; bitset_i < BITSET_WORDS; ++bitset_i) + for (bitset_i = 0; bitset_i < SBC_MAX / BITSET_WORD_BITS; ++bitset_i) set[bitset_i] = ~set[bitset_i]; + if (SBC_MAX % BITSET_WORD_BITS != 0) + set[BITSET_WORDS - 1] = + ((((bitset_word_t) 1 << SBC_MAX % BITSET_WORD_BITS) - 1) + & ~set[BITSET_WORDS - 1]); } -static void __attribute__ ((unused)) +static inline void bitset_merge (bitset_t dest, const bitset_t src) { int bitset_i; @@ -688,7 +821,7 @@ bitset_merge (bitset_t dest, const bitset_t src) dest[bitset_i] |= src[bitset_i]; } -static void __attribute__ ((unused)) +static inline void bitset_mask (bitset_t dest, const bitset_t src) { int bitset_i; @@ -697,10 +830,10 @@ bitset_mask (bitset_t dest, const bitset_t src) } #ifdef RE_ENABLE_I18N -/* Inline functions for re_string. */ +/* Functions for re_string. */ static int __attribute__ ((pure, unused)) -re_string_char_size_at (const re_string_t *pstr, int idx) +re_string_char_size_at (const re_string_t *pstr, Idx idx) { int byte_idx; if (pstr->mb_cur_max == 1) @@ -713,23 +846,22 @@ re_string_char_size_at (const re_string_t *pstr, int idx) static wint_t __attribute__ ((pure, unused)) -re_string_wchar_at (const re_string_t *pstr, int idx) +re_string_wchar_at (const re_string_t *pstr, Idx idx) { if (pstr->mb_cur_max == 1) return (wint_t) pstr->mbs[idx]; return (wint_t) pstr->wcs[idx]; } -# if IS_IN (libc) -# ifdef _LIBC -# include <locale/weight.h> -# endif +# ifdef _LIBC +# include <locale/weight.h> +# endif static int __attribute__ ((pure, unused)) -re_string_elem_size_at (const re_string_t *pstr, int idx) +re_string_elem_size_at (const re_string_t *pstr, Idx idx) { -# ifdef _LIBC +# ifdef _LIBC const unsigned char *p, *extra; const int32_t *table, *indirect; uint_fast32_t nrules = _NL_CURRENT_WORD (LC_COLLATE, _NL_COLLATE_NRULES); @@ -746,10 +878,34 @@ re_string_elem_size_at (const re_string_t *pstr, int idx) return p - pstr->mbs - idx; } else -# endif /* _LIBC */ +# endif /* _LIBC */ return 1; } -# endif #endif /* RE_ENABLE_I18N */ +#ifndef __GNUC_PREREQ +# if defined __GNUC__ && defined __GNUC_MINOR__ +# define __GNUC_PREREQ(maj, min) \ + ((__GNUC__ << 16) + __GNUC_MINOR__ >= ((maj) << 16) + (min)) +# else +# define __GNUC_PREREQ(maj, min) 0 +# endif +#endif + +#if __GNUC_PREREQ (3,4) +# undef __attribute_warn_unused_result__ +# define __attribute_warn_unused_result__ \ + __attribute__ ((__warn_unused_result__)) +#else +# define __attribute_warn_unused_result__ /* empty */ +#endif + +#ifndef FALLTHROUGH +# if __GNUC__ < 7 +# define FALLTHROUGH ((void) 0) +# else +# define FALLTHROUGH __attribute__ ((__fallthrough__)) +# endif +#endif + #endif /* _REGEX_INTERNAL_H */ diff --git a/posix/regexec.c b/posix/regexec.c index 4b1ab4e..63aef97 100644 --- a/posix/regexec.c +++ b/posix/regexec.c @@ -15,100 +15,98 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <stdint.h> + <https://www.gnu.org/licenses/>. */ static reg_errcode_t match_ctx_init (re_match_context_t *cache, int eflags, - int n); + Idx n); static void match_ctx_clean (re_match_context_t *mctx); static void match_ctx_free (re_match_context_t *cache); -static reg_errcode_t match_ctx_add_entry (re_match_context_t *cache, int node, - int str_idx, int from, int to); -static int search_cur_bkref_entry (const re_match_context_t *mctx, - int str_idx); -static reg_errcode_t match_ctx_add_subtop (re_match_context_t *mctx, int node, - int str_idx); +static reg_errcode_t match_ctx_add_entry (re_match_context_t *cache, Idx node, + Idx str_idx, Idx from, Idx to); +static Idx search_cur_bkref_entry (const re_match_context_t *mctx, Idx str_idx); +static reg_errcode_t match_ctx_add_subtop (re_match_context_t *mctx, Idx node, + Idx str_idx); static re_sub_match_last_t * match_ctx_add_sublast (re_sub_match_top_t *subtop, - int node, int str_idx); + Idx node, Idx str_idx); static void sift_ctx_init (re_sift_context_t *sctx, re_dfastate_t **sifted_sts, - re_dfastate_t **limited_sts, int last_node, - int last_str_idx); + re_dfastate_t **limited_sts, Idx last_node, + Idx last_str_idx); static reg_errcode_t re_search_internal (const regex_t *preg, - const char *string, int length, - int start, int range, int stop, + const char *string, Idx length, + Idx start, Idx last_start, Idx stop, size_t nmatch, regmatch_t pmatch[], int eflags); -static int re_search_2_stub (struct re_pattern_buffer *bufp, - const char *string1, int length1, - const char *string2, int length2, - int start, int range, struct re_registers *regs, - int stop, int ret_len); -static int re_search_stub (struct re_pattern_buffer *bufp, - const char *string, int length, int start, - int range, int stop, struct re_registers *regs, - int ret_len); +static regoff_t re_search_2_stub (struct re_pattern_buffer *bufp, + const char *string1, Idx length1, + const char *string2, Idx length2, + Idx start, regoff_t range, + struct re_registers *regs, + Idx stop, bool ret_len); +static regoff_t re_search_stub (struct re_pattern_buffer *bufp, + const char *string, Idx length, Idx start, + regoff_t range, Idx stop, + struct re_registers *regs, + bool ret_len); static unsigned re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, - int nregs, int regs_allocated); + Idx nregs, int regs_allocated); static reg_errcode_t prune_impossible_nodes (re_match_context_t *mctx); -static int check_matching (re_match_context_t *mctx, int fl_longest_match, - int *p_match_first); -static int check_halt_state_context (const re_match_context_t *mctx, - const re_dfastate_t *state, int idx); +static Idx check_matching (re_match_context_t *mctx, bool fl_longest_match, + Idx *p_match_first); +static Idx check_halt_state_context (const re_match_context_t *mctx, + const re_dfastate_t *state, Idx idx); static void update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, - regmatch_t *prev_idx_match, int cur_node, - int cur_idx, int nmatch); + regmatch_t *prev_idx_match, Idx cur_node, + Idx cur_idx, Idx nmatch); static reg_errcode_t push_fail_stack (struct re_fail_stack_t *fs, - int str_idx, int dest_node, int nregs, + Idx str_idx, Idx dest_node, Idx nregs, regmatch_t *regs, re_node_set *eps_via_nodes); static reg_errcode_t set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, regmatch_t *pmatch, - int fl_backtrack); + bool fl_backtrack); static reg_errcode_t free_fail_stack_return (struct re_fail_stack_t *fs); #ifdef RE_ENABLE_I18N static int sift_states_iter_mb (const re_match_context_t *mctx, re_sift_context_t *sctx, - int node_idx, int str_idx, int max_str_idx); + Idx node_idx, Idx str_idx, Idx max_str_idx); #endif /* RE_ENABLE_I18N */ static reg_errcode_t sift_states_backward (const re_match_context_t *mctx, re_sift_context_t *sctx); static reg_errcode_t build_sifted_states (const re_match_context_t *mctx, - re_sift_context_t *sctx, int str_idx, + re_sift_context_t *sctx, Idx str_idx, re_node_set *cur_dest); static reg_errcode_t update_cur_sifted_state (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, + Idx str_idx, re_node_set *dest_nodes); static reg_errcode_t add_epsilon_src_nodes (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates); -static int check_dst_limits (const re_match_context_t *mctx, - re_node_set *limits, - int dst_node, int dst_idx, int src_node, - int src_idx); +static bool check_dst_limits (const re_match_context_t *mctx, + const re_node_set *limits, + Idx dst_node, Idx dst_idx, Idx src_node, + Idx src_idx); static int check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, - int boundaries, int subexp_idx, - int from_node, int bkref_idx); + int boundaries, Idx subexp_idx, + Idx from_node, Idx bkref_idx); static int check_dst_limits_calc_pos (const re_match_context_t *mctx, - int limit, int subexp_idx, - int node, int str_idx, - int bkref_idx); + Idx limit, Idx subexp_idx, + Idx node, Idx str_idx, + Idx bkref_idx); static reg_errcode_t check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates, re_node_set *limits, struct re_backref_cache_entry *bkref_ents, - int str_idx); + Idx str_idx); static reg_errcode_t sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, - const re_node_set *candidates); + Idx str_idx, const re_node_set *candidates); static reg_errcode_t merge_state_array (const re_dfa_t *dfa, re_dfastate_t **dst, - re_dfastate_t **src, int num); + re_dfastate_t **src, Idx num); static re_dfastate_t *find_recover_state (reg_errcode_t *err, re_match_context_t *mctx); static re_dfastate_t *transit_state (reg_errcode_t *err, @@ -119,7 +117,7 @@ static re_dfastate_t *merge_state_with_log (reg_errcode_t *err, re_dfastate_t *next_state); static reg_errcode_t check_subexp_matching_top (re_match_context_t *mctx, re_node_set *cur_nodes, - int str_idx); + Idx str_idx); #if 0 static re_dfastate_t *transit_state_sb (reg_errcode_t *err, re_match_context_t *mctx, @@ -132,46 +130,46 @@ static reg_errcode_t transit_state_mb (re_match_context_t *mctx, static reg_errcode_t transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes); static reg_errcode_t get_subexp (re_match_context_t *mctx, - int bkref_node, int bkref_str_idx); + Idx bkref_node, Idx bkref_str_idx); static reg_errcode_t get_subexp_sub (re_match_context_t *mctx, const re_sub_match_top_t *sub_top, re_sub_match_last_t *sub_last, - int bkref_node, int bkref_str); -static int find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, - int subexp_idx, int type); + Idx bkref_node, Idx bkref_str); +static Idx find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, + Idx subexp_idx, int type); static reg_errcode_t check_arrival (re_match_context_t *mctx, - state_array_t *path, int top_node, - int top_str, int last_node, int last_str, + state_array_t *path, Idx top_node, + Idx top_str, Idx last_node, Idx last_str, int type); static reg_errcode_t check_arrival_add_next_nodes (re_match_context_t *mctx, - int str_idx, + Idx str_idx, re_node_set *cur_nodes, re_node_set *next_nodes); static reg_errcode_t check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, - int ex_subexp, int type); + Idx ex_subexp, int type); static reg_errcode_t check_arrival_expand_ecl_sub (const re_dfa_t *dfa, re_node_set *dst_nodes, - int target, int ex_subexp, + Idx target, Idx ex_subexp, int type); static reg_errcode_t expand_bkref_cache (re_match_context_t *mctx, - re_node_set *cur_nodes, int cur_str, - int subexp_num, int type); -static int build_trtable (const re_dfa_t *dfa, re_dfastate_t *state); + re_node_set *cur_nodes, Idx cur_str, + Idx subexp_num, int type); +static bool build_trtable (const re_dfa_t *dfa, re_dfastate_t *state); #ifdef RE_ENABLE_I18N -static int check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, - const re_string_t *input, int idx); +static int check_node_accept_bytes (const re_dfa_t *dfa, Idx node_idx, + const re_string_t *input, Idx idx); # ifdef _LIBC static unsigned int find_collation_sequence_value (const unsigned char *mbs, size_t name_len); # endif /* _LIBC */ #endif /* RE_ENABLE_I18N */ -static int group_nodes_into_DFAstates (const re_dfa_t *dfa, +static Idx group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, re_node_set *states_node, bitset_t *states_ch); -static int check_node_accept (const re_match_context_t *mctx, - const re_token_t *node, int idx); +static bool check_node_accept (const re_match_context_t *mctx, + const re_token_t *node, Idx idx); static reg_errcode_t extend_buffers (re_match_context_t *mctx, int min_len); /* Entry point for POSIX code. */ @@ -180,23 +178,23 @@ static reg_errcode_t extend_buffers (re_match_context_t *mctx, int min_len); string STRING. If NMATCH is zero or REG_NOSUB was set in the cflags argument to - `regcomp', we ignore PMATCH. Otherwise, we assume PMATCH has at + 'regcomp', we ignore PMATCH. Otherwise, we assume PMATCH has at least NMATCH elements, and we set them to the offsets of the corresponding matched substrings. - EFLAGS specifies `execution flags' which affect matching: if + EFLAGS specifies "execution flags" which affect matching: if REG_NOTBOL is set, then ^ does not match at the beginning of the string; if REG_NOTEOL is set, then $ does not match at the end. We return 0 if we find a match and REG_NOMATCH if not. */ int -regexec (const regex_t *__restrict preg, const char *__restrict string, +regexec (const regex_t *_Restrict_ preg, const char *_Restrict_ string, size_t nmatch, regmatch_t pmatch[], int eflags) { reg_errcode_t err; - int start, length; - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + Idx start, length; + re_dfa_t *dfa = preg->buffer; if (eflags & ~(REG_NOTBOL | REG_NOTEOL | REG_STARTEND)) return REG_BADPAT; @@ -212,14 +210,14 @@ regexec (const regex_t *__restrict preg, const char *__restrict string, length = strlen (string); } - __libc_lock_lock (dfa->lock); + lock_lock (dfa->lock); if (preg->no_sub) - err = re_search_internal (preg, string, length, start, length - start, + err = re_search_internal (preg, string, length, start, length, length, 0, NULL, eflags); else - err = re_search_internal (preg, string, length, start, length - start, + err = re_search_internal (preg, string, length, start, length, length, nmatch, pmatch, eflags); - __libc_lock_unlock (dfa->lock); + lock_unlock (dfa->lock); return err != REG_NOERROR; } @@ -234,8 +232,8 @@ __typeof__ (__regexec) __compat_regexec; int attribute_compat_text_section -__compat_regexec (const regex_t *__restrict preg, - const char *__restrict string, size_t nmatch, +__compat_regexec (const regex_t *_Restrict_ preg, + const char *_Restrict_ string, size_t nmatch, regmatch_t pmatch[], int eflags) { return regexec (preg, string, nmatch, pmatch, @@ -274,62 +272,65 @@ compat_symbol (libc, __compat_regexec, regexec, GLIBC_2_0); return the position of the start of the match. Return value -1 means no match was found and -2 indicates an internal error. */ -int -re_match (struct re_pattern_buffer *bufp, const char *string, int length, - int start, struct re_registers *regs) +regoff_t +re_match (struct re_pattern_buffer *bufp, const char *string, Idx length, + Idx start, struct re_registers *regs) { - return re_search_stub (bufp, string, length, start, 0, length, regs, 1); + return re_search_stub (bufp, string, length, start, 0, length, regs, true); } #ifdef _LIBC weak_alias (__re_match, re_match) #endif -int -re_search (struct re_pattern_buffer *bufp, const char *string, int length, - int start, int range, struct re_registers *regs) +regoff_t +re_search (struct re_pattern_buffer *bufp, const char *string, Idx length, + Idx start, regoff_t range, struct re_registers *regs) { - return re_search_stub (bufp, string, length, start, range, length, regs, 0); + return re_search_stub (bufp, string, length, start, range, length, regs, + false); } #ifdef _LIBC weak_alias (__re_search, re_search) #endif -int -re_match_2 (struct re_pattern_buffer *bufp, const char *string1, int length1, - const char *string2, int length2, int start, - struct re_registers *regs, int stop) +regoff_t +re_match_2 (struct re_pattern_buffer *bufp, const char *string1, Idx length1, + const char *string2, Idx length2, Idx start, + struct re_registers *regs, Idx stop) { return re_search_2_stub (bufp, string1, length1, string2, length2, - start, 0, regs, stop, 1); + start, 0, regs, stop, true); } #ifdef _LIBC weak_alias (__re_match_2, re_match_2) #endif -int -re_search_2 (struct re_pattern_buffer *bufp, const char *string1, int length1, - const char *string2, int length2, int start, int range, - struct re_registers *regs, int stop) +regoff_t +re_search_2 (struct re_pattern_buffer *bufp, const char *string1, Idx length1, + const char *string2, Idx length2, Idx start, regoff_t range, + struct re_registers *regs, Idx stop) { return re_search_2_stub (bufp, string1, length1, string2, length2, - start, range, regs, stop, 0); + start, range, regs, stop, false); } #ifdef _LIBC weak_alias (__re_search_2, re_search_2) #endif -static int +static regoff_t re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, - int length1, const char *string2, int length2, int start, - int range, struct re_registers *regs, - int stop, int ret_len) + Idx length1, const char *string2, Idx length2, Idx start, + regoff_t range, struct re_registers *regs, + Idx stop, bool ret_len) { const char *str; - int rval; - int len = length1 + length2; + regoff_t rval; + Idx len; char *s = NULL; - if (BE (length1 < 0 || length2 < 0 || stop < 0 || len < length1, 0)) + if (BE ((length1 < 0 || length2 < 0 || stop < 0 + || INT_ADD_WRAPV (length1, length2, &len)), + 0)) return -2; /* Concatenate the strings. */ @@ -353,42 +354,45 @@ re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, else str = string1; - rval = re_search_stub (bufp, str, len, start, range, stop, regs, ret_len); + rval = re_search_stub (bufp, str, len, start, range, stop, regs, + ret_len); re_free (s); return rval; } /* The parameters have the same meaning as those of re_search. Additional parameters: - If RET_LEN is nonzero the length of the match is returned (re_match style); + If RET_LEN is true the length of the match is returned (re_match style); otherwise the position of the match is returned. */ -static int -re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, - int start, int range, int stop, struct re_registers *regs, - int ret_len) +static regoff_t +re_search_stub (struct re_pattern_buffer *bufp, const char *string, Idx length, + Idx start, regoff_t range, Idx stop, struct re_registers *regs, + bool ret_len) { reg_errcode_t result; regmatch_t *pmatch; - int nregs, rval; + Idx nregs; + regoff_t rval; int eflags = 0; - re_dfa_t *dfa = (re_dfa_t *) bufp->buffer; + re_dfa_t *dfa = bufp->buffer; + Idx last_start = start + range; /* Check for out-of-range. */ if (BE (start < 0 || start > length, 0)) return -1; - if (BE (start + range > length, 0)) - range = length - start; - else if (BE (start + range < 0, 0)) - range = -start; + if (BE (length < last_start || (0 <= range && last_start < start), 0)) + last_start = length; + else if (BE (last_start < 0 || (range < 0 && start <= last_start), 0)) + last_start = 0; - __libc_lock_lock (dfa->lock); + lock_lock (dfa->lock); eflags |= (bufp->not_bol) ? REG_NOTBOL : 0; eflags |= (bufp->not_eol) ? REG_NOTEOL : 0; /* Compile fastmap if we haven't yet. */ - if (range > 0 && bufp->fastmap != NULL && !bufp->fastmap_accurate) + if (start < last_start && bufp->fastmap != NULL && !bufp->fastmap_accurate) re_compile_fastmap (bufp); if (BE (bufp->no_sub, 0)) @@ -397,8 +401,8 @@ re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, /* We need at least 1 register. */ if (regs == NULL) nregs = 1; - else if (BE (bufp->regs_allocated == REGS_FIXED && - regs->num_regs < bufp->re_nsub + 1, 0)) + else if (BE (bufp->regs_allocated == REGS_FIXED + && regs->num_regs <= bufp->re_nsub, 0)) { nregs = regs->num_regs; if (BE (nregs < 1, 0)) @@ -417,14 +421,14 @@ re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, goto out; } - result = re_search_internal (bufp, string, length, start, range, stop, + result = re_search_internal (bufp, string, length, start, last_start, stop, nregs, pmatch, eflags); rval = 0; - /* I hope we needn't fill ther regs with -1's when no match was found. */ + /* I hope we needn't fill their regs with -1's when no match was found. */ if (result != REG_NOERROR) - rval = -1; + rval = result == REG_NOMATCH ? -1 : -2; else if (regs != NULL) { /* If caller wants register contents data back, copy them. */ @@ -446,18 +450,18 @@ re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, } re_free (pmatch); out: - __libc_lock_unlock (dfa->lock); + lock_unlock (dfa->lock); return rval; } static unsigned -re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, int nregs, +re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, Idx nregs, int regs_allocated) { int rval = REGS_REALLOCATE; - int i; - int need_regs = nregs + 1; - /* We need one extra element beyond `num_regs' for the `-1' marker GNU code + Idx i; + Idx need_regs = nregs + 1; + /* We need one extra element beyond 'num_regs' for the '-1' marker GNU code uses. */ /* Have the register data arrays been allocated? */ @@ -530,7 +534,7 @@ re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, int nregs, void re_set_registers (struct re_pattern_buffer *bufp, struct re_registers *regs, - unsigned num_regs, regoff_t *starts, regoff_t *ends) + __re_size_t num_regs, regoff_t *starts, regoff_t *ends) { if (num_regs) { @@ -543,7 +547,7 @@ re_set_registers (struct re_pattern_buffer *bufp, struct re_registers *regs, { bufp->regs_allocated = REGS_UNALLOCATED; regs->num_regs = 0; - regs->start = regs->end = (regoff_t *) 0; + regs->start = regs->end = NULL; } } #ifdef _LIBC @@ -568,32 +572,38 @@ re_exec (const char *s) /* Searches for a compiled pattern PREG in the string STRING, whose length is LENGTH. NMATCH, PMATCH, and EFLAGS have the same - meaning as with regexec. START, and RANGE have the same meanings - with re_search. + meaning as with regexec. LAST_START is START + RANGE, where + START and RANGE have the same meaning as with re_search. Return REG_NOERROR if we find a match, and REG_NOMATCH if not, otherwise return the error code. Note: We assume front end functions already check ranges. - (START + RANGE >= 0 && START + RANGE <= LENGTH) */ + (0 <= LAST_START && LAST_START <= LENGTH) */ static reg_errcode_t __attribute_warn_unused_result__ -re_search_internal (const regex_t *preg, const char *string, int length, - int start, int range, int stop, size_t nmatch, +re_search_internal (const regex_t *preg, const char *string, Idx length, + Idx start, Idx last_start, Idx stop, size_t nmatch, regmatch_t pmatch[], int eflags) { reg_errcode_t err; - const re_dfa_t *dfa = (const re_dfa_t *) preg->buffer; - int left_lim, right_lim, incr; - int fl_longest_match, match_first, match_kind, match_last = -1; - int extra_nmatch; - int sb, ch; + const re_dfa_t *dfa = preg->buffer; + Idx left_lim, right_lim; + int incr; + bool fl_longest_match; + int match_kind; + Idx match_first; + Idx match_last = -1; + Idx extra_nmatch; + bool sb; + int ch; #if defined _LIBC || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L) re_match_context_t mctx = { .dfa = dfa }; #else re_match_context_t mctx; #endif - char *fastmap = (preg->fastmap != NULL && preg->fastmap_accurate - && range && !preg->can_be_null) ? preg->fastmap : NULL; + char *fastmap = ((preg->fastmap != NULL && preg->fastmap_accurate + && start != last_start && !preg->can_be_null) + ? preg->fastmap : NULL); RE_TRANSLATE_TYPE t = preg->translate; #if !(defined _LIBC || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L)) @@ -612,7 +622,7 @@ re_search_internal (const regex_t *preg, const char *string, int length, #ifdef DEBUG /* We assume front-end functions already check them. */ - assert (start + range >= 0 && start + range <= length); + assert (0 <= last_start && last_start <= length); #endif /* If initial states with non-begbuf contexts have no elements, @@ -623,16 +633,17 @@ re_search_internal (const regex_t *preg, const char *string, int length, && (dfa->init_state_nl->nodes.nelem == 0 || !preg->newline_anchor)) { - if (start != 0 && start + range != 0) - return REG_NOMATCH; - start = range = 0; + if (start != 0 && last_start != 0) + return REG_NOMATCH; + start = last_start = 0; } /* We must check the longest matching, if nmatch > 0. */ fl_longest_match = (nmatch != 0 || dfa->nbackref); err = re_string_allocate (&mctx.input, string, length, dfa->nodes_len + 1, - preg->translate, preg->syntax & RE_ICASE, dfa); + preg->translate, (preg->syntax & RE_ICASE) != 0, + dfa); if (BE (err != REG_NOERROR, 0)) goto free_return; mctx.input.stop = stop; @@ -650,7 +661,8 @@ re_search_internal (const regex_t *preg, const char *string, int length, if (nmatch > 1 || dfa->has_mb_node) { /* Avoid overflow. */ - if (BE (SIZE_MAX / sizeof (re_dfastate_t *) <= mctx.input.bufs_len, 0)) + if (BE ((MIN (IDX_MAX, SIZE_MAX / sizeof (re_dfastate_t *)) + <= mctx.input.bufs_len), 0)) { err = REG_ESPACE; goto free_return; @@ -670,15 +682,15 @@ re_search_internal (const regex_t *preg, const char *string, int length, mctx.input.tip_context = (eflags & REG_NOTBOL) ? CONTEXT_BEGBUF : CONTEXT_NEWLINE | CONTEXT_BEGBUF; - /* Check incrementally whether of not the input string match. */ - incr = (range < 0) ? -1 : 1; - left_lim = (range < 0) ? start + range : start; - right_lim = (range < 0) ? start : start + range; + /* Check incrementally whether the input string matches. */ + incr = (last_start < start) ? -1 : 1; + left_lim = (last_start < start) ? last_start : start; + right_lim = (last_start < start) ? start : last_start; sb = dfa->mb_cur_max == 1; match_kind = (fastmap ? ((sb || !(preg->syntax & RE_ICASE || t) ? 4 : 0) - | (range >= 0 ? 2 : 0) + | (start <= last_start ? 2 : 0) | (t != NULL ? 1 : 0)) : 8); @@ -745,8 +757,8 @@ re_search_internal (const regex_t *preg, const char *string, int length, { /* If MATCH_FIRST is out of the valid range, reconstruct the buffers. */ - unsigned int offset = match_first - mctx.input.raw_mbs_idx; - if (BE (offset >= (unsigned int) mctx.input.valid_raw_len, 0)) + __re_size_t offset = match_first - mctx.input.raw_mbs_idx; + if (BE (offset >= (__re_size_t) mctx.input.valid_raw_len, 0)) { err = re_string_reconstruct (&mctx.input, match_first, eflags); @@ -788,7 +800,7 @@ re_search_internal (const regex_t *preg, const char *string, int length, /* We assume that the matching starts from 0. */ mctx.state_log_top = mctx.nbkref_ents = mctx.max_mb_elem_len = 0; match_last = check_matching (&mctx, fl_longest_match, - range >= 0 ? &match_first : NULL); + start <= last_start ? &match_first : NULL); if (match_last != -1) { if (BE (match_last == -2, 0)) @@ -831,7 +843,7 @@ re_search_internal (const regex_t *preg, const char *string, int length, /* Set pmatch[] if we need. */ if (nmatch > 0) { - int reg_idx; + Idx reg_idx; /* Initialize registers. */ for (reg_idx = 1; reg_idx < nmatch; ++reg_idx) @@ -840,6 +852,9 @@ re_search_internal (const regex_t *preg, const char *string, int length, /* Set the points where matching start/end. */ pmatch[0].rm_so = 0; pmatch[0].rm_eo = mctx.match_last; + /* FIXME: This function should fail if mctx.match_last exceeds + the maximum possible regoff_t value. We need a new error + code REG_OVERFLOW. */ if (!preg->no_sub && nmatch > 1) { @@ -903,7 +918,7 @@ __attribute_warn_unused_result__ prune_impossible_nodes (re_match_context_t *mctx) { const re_dfa_t *const dfa = mctx->dfa; - int halt_node, match_last; + Idx halt_node, match_last; reg_errcode_t ret; re_dfastate_t **sifted_states; re_dfastate_t **lim_states = NULL; @@ -915,7 +930,7 @@ prune_impossible_nodes (re_match_context_t *mctx) halt_node = mctx->last_node; /* Avoid overflow. */ - if (BE (SIZE_MAX / sizeof (re_dfastate_t *) <= match_last, 0)) + if (BE (MIN (IDX_MAX, SIZE_MAX / sizeof (re_dfastate_t *)) <= match_last, 0)) return REG_ESPACE; sifted_states = re_malloc (re_dfastate_t *, match_last + 1); @@ -995,9 +1010,9 @@ prune_impossible_nodes (re_match_context_t *mctx) since initial states may have constraints like "\<", "^", etc.. */ static inline re_dfastate_t * -__attribute ((always_inline)) +__attribute__ ((always_inline)) acquire_init_state_context (reg_errcode_t *err, const re_match_context_t *mctx, - int idx) + Idx idx) { const re_dfa_t *const dfa = mctx->dfa; if (dfa->init_state->has_constraint) @@ -1028,27 +1043,27 @@ acquire_init_state_context (reg_errcode_t *err, const re_match_context_t *mctx, } /* Check whether the regular expression match input string INPUT or not, - and return the index where the matching end, return -1 if not match, - or return -2 in case of an error. + and return the index where the matching end. Return -1 if + there is no match, and return -2 in case of an error. FL_LONGEST_MATCH means we want the POSIX longest matching. If P_MATCH_FIRST is not NULL, and the match fails, it is set to the next place where we may want to try matching. - Note that the matcher assume that the maching starts from the current + Note that the matcher assumes that the matching starts from the current index of the buffer. */ -static int +static Idx __attribute_warn_unused_result__ -check_matching (re_match_context_t *mctx, int fl_longest_match, - int *p_match_first) +check_matching (re_match_context_t *mctx, bool fl_longest_match, + Idx *p_match_first) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int match = 0; - int match_last = -1; - int cur_str_idx = re_string_cur_idx (&mctx->input); + Idx match = 0; + Idx match_last = -1; + Idx cur_str_idx = re_string_cur_idx (&mctx->input); re_dfastate_t *cur_state; - int at_init_state = p_match_first != NULL; - int next_start_idx = cur_str_idx; + bool at_init_state = p_match_first != NULL; + Idx next_start_idx = cur_str_idx; err = REG_NOERROR; cur_state = acquire_init_state_context (&err, mctx, cur_str_idx); @@ -1067,7 +1082,7 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, later. E.g. Processing back references. */ if (BE (dfa->nbackref, 0)) { - at_init_state = 0; + at_init_state = false; err = check_subexp_matching_top (mctx, &cur_state->nodes, 0); if (BE (err != REG_NOERROR, 0)) return err; @@ -1100,7 +1115,7 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, while (!re_string_eoi (&mctx->input)) { re_dfastate_t *old_state = cur_state; - int next_char_idx = re_string_cur_idx (&mctx->input) + 1; + Idx next_char_idx = re_string_cur_idx (&mctx->input) + 1; if ((BE (next_char_idx >= mctx->input.bufs_len, 0) && mctx->input.bufs_len < mctx->input.len) @@ -1138,7 +1153,7 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, if (old_state == cur_state) next_start_idx = next_char_idx; else - at_init_state = 0; + at_init_state = false; } if (cur_state->halt) @@ -1169,29 +1184,29 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, /* Check NODE match the current context. */ -static int -check_halt_node_context (const re_dfa_t *dfa, int node, unsigned int context) +static bool +check_halt_node_context (const re_dfa_t *dfa, Idx node, unsigned int context) { re_token_type_t type = dfa->nodes[node].type; unsigned int constraint = dfa->nodes[node].constraint; if (type != END_OF_RE) - return 0; + return false; if (!constraint) - return 1; + return true; if (NOT_SATISFY_NEXT_CONSTRAINT (constraint, context)) - return 0; - return 1; + return false; + return true; } /* Check the halt state STATE match the current context. Return 0 if not match, if the node, STATE has, is a halt node and match the context, return the node. */ -static int +static Idx check_halt_state_context (const re_match_context_t *mctx, - const re_dfastate_t *state, int idx) + const re_dfastate_t *state, Idx idx) { - int i; + Idx i; unsigned int context; #ifdef DEBUG assert (state->halt); @@ -1205,31 +1220,33 @@ check_halt_state_context (const re_match_context_t *mctx, /* Compute the next node to which "NFA" transit from NODE("NFA" is a NFA corresponding to the DFA). - Return the destination node, and update EPS_VIA_NODES, return -1 in case - of errors. */ + Return the destination node, and update EPS_VIA_NODES; + return -1 in case of errors. */ -static int -proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, - int *pidx, int node, re_node_set *eps_via_nodes, +static Idx +proceed_next_node (const re_match_context_t *mctx, Idx nregs, regmatch_t *regs, + Idx *pidx, Idx node, re_node_set *eps_via_nodes, struct re_fail_stack_t *fs) { const re_dfa_t *const dfa = mctx->dfa; - int i, err; + Idx i; + bool ok; if (IS_EPSILON_NODE (dfa->nodes[node].type)) { re_node_set *cur_nodes = &mctx->state_log[*pidx]->nodes; re_node_set *edests = &dfa->edests[node]; - int dest_node; - err = re_node_set_insert (eps_via_nodes, node); - if (BE (err < 0, 0)) + Idx dest_node; + ok = re_node_set_insert (eps_via_nodes, node); + if (BE (! ok, 0)) return -2; - /* Pick up a valid destination, or return -1 if none is found. */ + /* Pick up a valid destination, or return -1 if none + is found. */ for (dest_node = -1, i = 0; i < edests->nelem; ++i) { - int candidate = edests->elems[i]; + Idx candidate = edests->elems[i]; if (!re_node_set_contains (cur_nodes, candidate)) continue; - if (dest_node == -1) + if (dest_node == -1) dest_node = candidate; else @@ -1253,7 +1270,7 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, } else { - int naccepted = 0; + Idx naccepted = 0; re_token_type_t type = dfa->nodes[node].type; #ifdef RE_ENABLE_I18N @@ -1263,7 +1280,7 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, #endif /* RE_ENABLE_I18N */ if (type == OP_BACK_REF) { - int subexp_idx = dfa->nodes[node].opr.idx + 1; + Idx subexp_idx = dfa->nodes[node].opr.idx + 1; naccepted = regs[subexp_idx].rm_eo - regs[subexp_idx].rm_so; if (fs != NULL) { @@ -1280,9 +1297,9 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, if (naccepted == 0) { - int dest_node; - err = re_node_set_insert (eps_via_nodes, node); - if (BE (err < 0, 0)) + Idx dest_node; + ok = re_node_set_insert (eps_via_nodes, node); + if (BE (! ok, 0)) return -2; dest_node = dfa->edests[node].elems[0]; if (re_node_set_contains (&mctx->state_log[*pidx]->nodes, @@ -1294,7 +1311,7 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, if (naccepted != 0 || check_node_accept (mctx, dfa->nodes + node, *pidx)) { - int dest_node = dfa->nexts[node]; + Idx dest_node = dfa->nexts[node]; *pidx = (naccepted == 0) ? *pidx + 1 : *pidx + naccepted; if (fs && (*pidx > mctx->match_last || mctx->state_log[*pidx] == NULL || !re_node_set_contains (&mctx->state_log[*pidx]->nodes, @@ -1309,16 +1326,16 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, static reg_errcode_t __attribute_warn_unused_result__ -push_fail_stack (struct re_fail_stack_t *fs, int str_idx, int dest_node, - int nregs, regmatch_t *regs, re_node_set *eps_via_nodes) +push_fail_stack (struct re_fail_stack_t *fs, Idx str_idx, Idx dest_node, + Idx nregs, regmatch_t *regs, re_node_set *eps_via_nodes) { reg_errcode_t err; - int num = fs->num++; + Idx num = fs->num++; if (fs->num == fs->alloc) { struct re_fail_stack_ent_t *new_array; - new_array = realloc (fs->stack, (sizeof (struct re_fail_stack_ent_t) - * fs->alloc * 2)); + new_array = re_realloc (fs->stack, struct re_fail_stack_ent_t, + fs->alloc * 2); if (new_array == NULL) return REG_ESPACE; fs->alloc *= 2; @@ -1334,11 +1351,11 @@ push_fail_stack (struct re_fail_stack_t *fs, int str_idx, int dest_node, return err; } -static int -pop_fail_stack (struct re_fail_stack_t *fs, int *pidx, int nregs, +static Idx +pop_fail_stack (struct re_fail_stack_t *fs, Idx *pidx, Idx nregs, regmatch_t *regs, re_node_set *eps_via_nodes) { - int num = --fs->num; + Idx num = --fs->num; assert (num >= 0); *pidx = fs->stack[num].idx; memcpy (regs, fs->stack[num].regs, sizeof (regmatch_t) * nregs); @@ -1356,15 +1373,15 @@ pop_fail_stack (struct re_fail_stack_t *fs, int *pidx, int nregs, static reg_errcode_t __attribute_warn_unused_result__ set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, - regmatch_t *pmatch, int fl_backtrack) + regmatch_t *pmatch, bool fl_backtrack) { - const re_dfa_t *dfa = (const re_dfa_t *) preg->buffer; - int idx, cur_node; + const re_dfa_t *dfa = preg->buffer; + Idx idx, cur_node; re_node_set eps_via_nodes; struct re_fail_stack_t *fs; struct re_fail_stack_t fs_body = { 0, 2, NULL }; regmatch_t *prev_idx_match; - int prev_idx_match_malloced = 0; + bool prev_idx_match_malloced = false; #ifdef DEBUG assert (nmatch > 1); @@ -1393,7 +1410,7 @@ set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, free_fail_stack_return (fs); return REG_ESPACE; } - prev_idx_match_malloced = 1; + prev_idx_match_malloced = true; } memcpy (prev_idx_match, pmatch, sizeof (regmatch_t) * nmatch); @@ -1403,7 +1420,7 @@ set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, if (idx == pmatch[0].rm_eo && cur_node == mctx->last_node) { - int reg_idx; + Idx reg_idx; if (fs) { for (reg_idx = 0; reg_idx < nmatch; ++reg_idx) @@ -1465,7 +1482,7 @@ free_fail_stack_return (struct re_fail_stack_t *fs) { if (fs) { - int fs_idx; + Idx fs_idx; for (fs_idx = 0; fs_idx < fs->num; ++fs_idx) { re_node_set_free (&fs->stack[fs_idx].eps_via_nodes); @@ -1478,12 +1495,12 @@ free_fail_stack_return (struct re_fail_stack_t *fs) static void update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, - regmatch_t *prev_idx_match, int cur_node, int cur_idx, int nmatch) + regmatch_t *prev_idx_match, Idx cur_node, Idx cur_idx, Idx nmatch) { int type = dfa->nodes[cur_node].type; if (type == OP_OPEN_SUBEXP) { - int reg_num = dfa->nodes[cur_node].opr.idx + 1; + Idx reg_num = dfa->nodes[cur_node].opr.idx + 1; /* We are at the first node of this sub expression. */ if (reg_num < nmatch) @@ -1494,7 +1511,7 @@ update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, } else if (type == OP_CLOSE_SUBEXP) { - int reg_num = dfa->nodes[cur_node].opr.idx + 1; + Idx reg_num = dfa->nodes[cur_node].opr.idx + 1; if (reg_num < nmatch) { /* We are at the last node of this sub expression. */ @@ -1528,21 +1545,21 @@ update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, and sift the nodes in each states according to the following rules. Updated state_log will be wrote to STATE_LOG. - Rules: We throw away the Node `a' in the STATE_LOG[STR_IDX] if... + Rules: We throw away the Node 'a' in the STATE_LOG[STR_IDX] if... 1. When STR_IDX == MATCH_LAST(the last index in the state_log): - If `a' isn't the LAST_NODE and `a' can't epsilon transit to - the LAST_NODE, we throw away the node `a'. - 2. When 0 <= STR_IDX < MATCH_LAST and `a' accepts - string `s' and transit to `b': + If 'a' isn't the LAST_NODE and 'a' can't epsilon transit to + the LAST_NODE, we throw away the node 'a'. + 2. When 0 <= STR_IDX < MATCH_LAST and 'a' accepts + string 's' and transit to 'b': i. If 'b' isn't in the STATE_LOG[STR_IDX+strlen('s')], we throw - away the node `a'. + away the node 'a'. ii. If 'b' is in the STATE_LOG[STR_IDX+strlen('s')] but 'b' is - thrown away, we throw away the node `a'. + thrown away, we throw away the node 'a'. 3. When 0 <= STR_IDX < MATCH_LAST and 'a' epsilon transit to 'b': i. If 'b' isn't in the STATE_LOG[STR_IDX], we throw away the - node `a'. + node 'a'. ii. If 'b' is in the STATE_LOG[STR_IDX] but 'b' is thrown away, - we throw away the node `a'. */ + we throw away the node 'a'. */ #define STATE_NODE_CONTAINS(state,node) \ ((state) != NULL && re_node_set_contains (&(state)->nodes, node)) @@ -1552,7 +1569,7 @@ sift_states_backward (const re_match_context_t *mctx, re_sift_context_t *sctx) { reg_errcode_t err; int null_cnt = 0; - int str_idx = sctx->last_str_idx; + Idx str_idx = sctx->last_str_idx; re_node_set cur_dest; #ifdef DEBUG @@ -1607,31 +1624,31 @@ sift_states_backward (const re_match_context_t *mctx, re_sift_context_t *sctx) static reg_errcode_t __attribute_warn_unused_result__ build_sifted_states (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, re_node_set *cur_dest) + Idx str_idx, re_node_set *cur_dest) { const re_dfa_t *const dfa = mctx->dfa; const re_node_set *cur_src = &mctx->state_log[str_idx]->non_eps_nodes; - int i; + Idx i; /* Then build the next sifted state. - We build the next sifted state on `cur_dest', and update - `sifted_states[str_idx]' with `cur_dest'. + We build the next sifted state on 'cur_dest', and update + 'sifted_states[str_idx]' with 'cur_dest'. Note: - `cur_dest' is the sifted state from `state_log[str_idx + 1]'. - `cur_src' points the node_set of the old `state_log[str_idx]' + 'cur_dest' is the sifted state from 'state_log[str_idx + 1]'. + 'cur_src' points the node_set of the old 'state_log[str_idx]' (with the epsilon nodes pre-filtered out). */ for (i = 0; i < cur_src->nelem; i++) { - int prev_node = cur_src->elems[i]; + Idx prev_node = cur_src->elems[i]; int naccepted = 0; - int ret; + bool ok; #ifdef DEBUG re_token_type_t type = dfa->nodes[prev_node].type; assert (!IS_EPSILON_NODE (type)); #endif #ifdef RE_ENABLE_I18N - /* If the node may accept `multi byte'. */ + /* If the node may accept "multi byte". */ if (dfa->nodes[prev_node].accept_mb) naccepted = sift_states_iter_mb (mctx, sctx, prev_node, str_idx, sctx->last_str_idx); @@ -1650,14 +1667,14 @@ build_sifted_states (const re_match_context_t *mctx, re_sift_context_t *sctx, if (sctx->limits.nelem) { - int to_idx = str_idx + naccepted; + Idx to_idx = str_idx + naccepted; if (check_dst_limits (mctx, &sctx->limits, dfa->nexts[prev_node], to_idx, prev_node, str_idx)) continue; } - ret = re_node_set_insert (cur_dest, prev_node); - if (BE (ret == -1, 0)) + ok = re_node_set_insert (cur_dest, prev_node); + if (BE (! ok, 0)) return REG_ESPACE; } @@ -1667,9 +1684,9 @@ build_sifted_states (const re_match_context_t *mctx, re_sift_context_t *sctx, /* Helper functions. */ static reg_errcode_t -clean_state_log_if_needed (re_match_context_t *mctx, int next_state_log_idx) +clean_state_log_if_needed (re_match_context_t *mctx, Idx next_state_log_idx) { - int top = mctx->state_log_top; + Idx top = mctx->state_log_top; if ((next_state_log_idx >= mctx->input.bufs_len && mctx->input.bufs_len < mctx->input.len) @@ -1693,9 +1710,9 @@ clean_state_log_if_needed (re_match_context_t *mctx, int next_state_log_idx) static reg_errcode_t merge_state_array (const re_dfa_t *dfa, re_dfastate_t **dst, - re_dfastate_t **src, int num) + re_dfastate_t **src, Idx num) { - int st_idx; + Idx st_idx; reg_errcode_t err; for (st_idx = 0; st_idx < num; ++st_idx) { @@ -1719,7 +1736,7 @@ merge_state_array (const re_dfa_t *dfa, re_dfastate_t **dst, static reg_errcode_t update_cur_sifted_state (const re_match_context_t *mctx, - re_sift_context_t *sctx, int str_idx, + re_sift_context_t *sctx, Idx str_idx, re_node_set *dest_nodes) { const re_dfa_t *const dfa = mctx->dfa; @@ -1770,7 +1787,7 @@ add_epsilon_src_nodes (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates) { reg_errcode_t err = REG_NOERROR; - int i; + Idx i; re_dfastate_t *state = re_acquire_state (&err, dfa, dest_nodes); if (BE (err != REG_NOERROR, 0)) @@ -1794,23 +1811,23 @@ add_epsilon_src_nodes (const re_dfa_t *dfa, re_node_set *dest_nodes, } static reg_errcode_t -sub_epsilon_src_nodes (const re_dfa_t *dfa, int node, re_node_set *dest_nodes, +sub_epsilon_src_nodes (const re_dfa_t *dfa, Idx node, re_node_set *dest_nodes, const re_node_set *candidates) { - int ecl_idx; + Idx ecl_idx; reg_errcode_t err; re_node_set *inv_eclosure = dfa->inveclosures + node; re_node_set except_nodes; re_node_set_init_empty (&except_nodes); for (ecl_idx = 0; ecl_idx < inv_eclosure->nelem; ++ecl_idx) { - int cur_node = inv_eclosure->elems[ecl_idx]; + Idx cur_node = inv_eclosure->elems[ecl_idx]; if (cur_node == node) continue; if (IS_EPSILON_NODE (dfa->nodes[cur_node].type)) { - int edst1 = dfa->edests[cur_node].elems[0]; - int edst2 = ((dfa->edests[cur_node].nelem > 1) + Idx edst1 = dfa->edests[cur_node].elems[0]; + Idx edst2 = ((dfa->edests[cur_node].nelem > 1) ? dfa->edests[cur_node].elems[1] : -1); if ((!re_node_set_contains (inv_eclosure, edst1) && re_node_set_contains (dest_nodes, edst1)) @@ -1830,10 +1847,10 @@ sub_epsilon_src_nodes (const re_dfa_t *dfa, int node, re_node_set *dest_nodes, } for (ecl_idx = 0; ecl_idx < inv_eclosure->nelem; ++ecl_idx) { - int cur_node = inv_eclosure->elems[ecl_idx]; + Idx cur_node = inv_eclosure->elems[ecl_idx]; if (!re_node_set_contains (&except_nodes, cur_node)) { - int idx = re_node_set_contains (dest_nodes, cur_node) - 1; + Idx idx = re_node_set_contains (dest_nodes, cur_node) - 1; re_node_set_remove_at (dest_nodes, idx); } } @@ -1841,18 +1858,18 @@ sub_epsilon_src_nodes (const re_dfa_t *dfa, int node, re_node_set *dest_nodes, return REG_NOERROR; } -static int -check_dst_limits (const re_match_context_t *mctx, re_node_set *limits, - int dst_node, int dst_idx, int src_node, int src_idx) +static bool +check_dst_limits (const re_match_context_t *mctx, const re_node_set *limits, + Idx dst_node, Idx dst_idx, Idx src_node, Idx src_idx) { const re_dfa_t *const dfa = mctx->dfa; - int lim_idx, src_pos, dst_pos; + Idx lim_idx, src_pos, dst_pos; - int dst_bkref_idx = search_cur_bkref_entry (mctx, dst_idx); - int src_bkref_idx = search_cur_bkref_entry (mctx, src_idx); + Idx dst_bkref_idx = search_cur_bkref_entry (mctx, dst_idx); + Idx src_bkref_idx = search_cur_bkref_entry (mctx, src_idx); for (lim_idx = 0; lim_idx < limits->nelem; ++lim_idx) { - int subexp_idx; + Idx subexp_idx; struct re_backref_cache_entry *ent; ent = mctx->bkref_ents + limits->elems[lim_idx]; subexp_idx = dfa->nodes[ent->node].opr.idx; @@ -1871,24 +1888,24 @@ check_dst_limits (const re_match_context_t *mctx, re_node_set *limits, if (src_pos == dst_pos) continue; /* This is unrelated limitation. */ else - return 1; + return true; } - return 0; + return false; } static int check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, int boundaries, - int subexp_idx, int from_node, int bkref_idx) + Idx subexp_idx, Idx from_node, Idx bkref_idx) { const re_dfa_t *const dfa = mctx->dfa; const re_node_set *eclosures = dfa->eclosures + from_node; - int node_idx; + Idx node_idx; /* Else, we are on the boundary: examine the nodes on the epsilon closure. */ for (node_idx = 0; node_idx < eclosures->nelem; ++node_idx) { - int node = eclosures->elems[node_idx]; + Idx node = eclosures->elems[node_idx]; switch (dfa->nodes[node].type) { case OP_BACK_REF: @@ -1897,7 +1914,8 @@ check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, int boundaries, struct re_backref_cache_entry *ent = mctx->bkref_ents + bkref_idx; do { - int dst, cpos; + Idx dst; + int cpos; if (ent->node != node) continue; @@ -1957,9 +1975,9 @@ check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, int boundaries, } static int -check_dst_limits_calc_pos (const re_match_context_t *mctx, int limit, - int subexp_idx, int from_node, int str_idx, - int bkref_idx) +check_dst_limits_calc_pos (const re_match_context_t *mctx, Idx limit, + Idx subexp_idx, Idx from_node, Idx str_idx, + Idx bkref_idx) { struct re_backref_cache_entry *lim = mctx->bkref_ents + limit; int boundaries; @@ -1988,14 +2006,14 @@ check_dst_limits_calc_pos (const re_match_context_t *mctx, int limit, static reg_errcode_t check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates, re_node_set *limits, - struct re_backref_cache_entry *bkref_ents, int str_idx) + struct re_backref_cache_entry *bkref_ents, Idx str_idx) { reg_errcode_t err; - int node_idx, lim_idx; + Idx node_idx, lim_idx; for (lim_idx = 0; lim_idx < limits->nelem; ++lim_idx) { - int subexp_idx; + Idx subexp_idx; struct re_backref_cache_entry *ent; ent = bkref_ents + limits->elems[lim_idx]; @@ -2005,11 +2023,11 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, subexp_idx = dfa->nodes[ent->node].opr.idx; if (ent->subexp_to == str_idx) { - int ops_node = -1; - int cls_node = -1; + Idx ops_node = -1; + Idx cls_node = -1; for (node_idx = 0; node_idx < dest_nodes->nelem; ++node_idx) { - int node = dest_nodes->elems[node_idx]; + Idx node = dest_nodes->elems[node_idx]; re_token_type_t type = dfa->nodes[node].type; if (type == OP_OPEN_SUBEXP && subexp_idx == dfa->nodes[node].opr.idx) @@ -2033,7 +2051,7 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, if (cls_node >= 0) for (node_idx = 0; node_idx < dest_nodes->nelem; ++node_idx) { - int node = dest_nodes->elems[node_idx]; + Idx node = dest_nodes->elems[node_idx]; if (!re_node_set_contains (dfa->inveclosures + node, cls_node) && !re_node_set_contains (dfa->eclosures + node, @@ -2053,7 +2071,7 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, { for (node_idx = 0; node_idx < dest_nodes->nelem; ++node_idx) { - int node = dest_nodes->elems[node_idx]; + Idx node = dest_nodes->elems[node_idx]; re_token_type_t type = dfa->nodes[node].type; if (type == OP_CLOSE_SUBEXP || type == OP_OPEN_SUBEXP) { @@ -2075,13 +2093,13 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, static reg_errcode_t __attribute_warn_unused_result__ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, const re_node_set *candidates) + Idx str_idx, const re_node_set *candidates) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int node_idx, node; + Idx node_idx, node; re_sift_context_t local_sctx; - int first_idx = search_cur_bkref_entry (mctx, str_idx); + Idx first_idx = search_cur_bkref_entry (mctx, str_idx); if (first_idx == -1) return REG_NOERROR; @@ -2090,7 +2108,7 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, for (node_idx = 0; node_idx < candidates->nelem; ++node_idx) { - int enabled_idx; + Idx enabled_idx; re_token_type_t type; struct re_backref_cache_entry *entry; node = candidates->elems[node_idx]; @@ -2105,10 +2123,10 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, enabled_idx = first_idx; do { - int subexp_len; - int to_idx; - int dst_node; - int ret; + Idx subexp_len; + Idx to_idx; + Idx dst_node; + bool ok; re_dfastate_t *cur_state; if (entry->node != node) @@ -2134,8 +2152,8 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, } local_sctx.last_node = node; local_sctx.last_str_idx = str_idx; - ret = re_node_set_insert (&local_sctx.limits, enabled_idx); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (&local_sctx.limits, enabled_idx); + if (BE (! ok, 0)) { err = REG_ESPACE; goto free_return; @@ -2174,21 +2192,21 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, #ifdef RE_ENABLE_I18N static int sift_states_iter_mb (const re_match_context_t *mctx, re_sift_context_t *sctx, - int node_idx, int str_idx, int max_str_idx) + Idx node_idx, Idx str_idx, Idx max_str_idx) { const re_dfa_t *const dfa = mctx->dfa; int naccepted; - /* Check the node can accept `multi byte'. */ + /* Check the node can accept "multi byte". */ naccepted = check_node_accept_bytes (dfa, node_idx, &mctx->input, str_idx); if (naccepted > 0 && str_idx + naccepted <= max_str_idx && !STATE_NODE_CONTAINS (sctx->sifted_states[str_idx + naccepted], dfa->nexts[node_idx])) - /* The node can't accept the `multi byte', or the + /* The node can't accept the "multi byte", or the destination was already thrown away, then the node - could't accept the current input `multi byte'. */ + could't accept the current input "multi byte". */ naccepted = 0; /* Otherwise, it is sure that the node could accept - `naccepted' bytes input. */ + 'naccepted' bytes input. */ return naccepted; } #endif /* RE_ENABLE_I18N */ @@ -2259,12 +2277,12 @@ transit_state (reg_errcode_t *err, re_match_context_t *mctx, } /* Update the state_log if we need */ -re_dfastate_t * +static re_dfastate_t * merge_state_with_log (reg_errcode_t *err, re_match_context_t *mctx, re_dfastate_t *next_state) { const re_dfa_t *const dfa = mctx->dfa; - int cur_idx = re_string_cur_idx (&mctx->input); + Idx cur_idx = re_string_cur_idx (&mctx->input); if (cur_idx > mctx->state_log_top) { @@ -2337,14 +2355,14 @@ merge_state_with_log (reg_errcode_t *err, re_match_context_t *mctx, /* Skip bytes in the input that correspond to part of a multi-byte match, then look in the log for a state from which to restart matching. */ -re_dfastate_t * +static re_dfastate_t * find_recover_state (reg_errcode_t *err, re_match_context_t *mctx) { re_dfastate_t *cur_state; do { - int max = mctx->state_log_top; - int cur_str_idx = re_string_cur_idx (&mctx->input); + Idx max = mctx->state_log_top; + Idx cur_str_idx = re_string_cur_idx (&mctx->input); do { @@ -2369,10 +2387,10 @@ find_recover_state (reg_errcode_t *err, re_match_context_t *mctx) static reg_errcode_t check_subexp_matching_top (re_match_context_t *mctx, re_node_set *cur_nodes, - int str_idx) + Idx str_idx) { const re_dfa_t *const dfa = mctx->dfa; - int node_idx; + Idx node_idx; reg_errcode_t err; /* TODO: This isn't efficient. @@ -2382,7 +2400,7 @@ check_subexp_matching_top (re_match_context_t *mctx, re_node_set *cur_nodes, E.g. RE: (a){2} */ for (node_idx = 0; node_idx < cur_nodes->nelem; ++node_idx) { - int node = cur_nodes->elems[node_idx]; + Idx node = cur_nodes->elems[node_idx]; if (dfa->nodes[node].type == OP_OPEN_SUBEXP && dfa->nodes[node].opr.idx < BITSET_WORD_BITS && (dfa->used_bkref_map @@ -2407,7 +2425,7 @@ transit_state_sb (reg_errcode_t *err, re_match_context_t *mctx, const re_dfa_t *const dfa = mctx->dfa; re_node_set next_nodes; re_dfastate_t *next_state; - int node_cnt, cur_str_idx = re_string_cur_idx (&mctx->input); + Idx node_cnt, cur_str_idx = re_string_cur_idx (&mctx->input); unsigned int context; *err = re_node_set_alloc (&next_nodes, state->nodes.nelem + 1); @@ -2415,7 +2433,7 @@ transit_state_sb (reg_errcode_t *err, re_match_context_t *mctx, return NULL; for (node_cnt = 0; node_cnt < state->nodes.nelem; ++node_cnt) { - int cur_node = state->nodes.elems[node_cnt]; + Idx cur_node = state->nodes.elems[node_cnt]; if (check_node_accept (mctx, dfa->nodes + cur_node, cur_str_idx)) { *err = re_node_set_merge (&next_nodes, @@ -2444,13 +2462,14 @@ transit_state_mb (re_match_context_t *mctx, re_dfastate_t *pstate) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int i; + Idx i; for (i = 0; i < pstate->nodes.nelem; ++i) { re_node_set dest_nodes, *new_nodes; - int cur_node_idx = pstate->nodes.elems[i]; - int naccepted, dest_idx; + Idx cur_node_idx = pstate->nodes.elems[i]; + int naccepted; + Idx dest_idx; unsigned int context; re_dfastate_t *dest_state; @@ -2473,7 +2492,7 @@ transit_state_mb (re_match_context_t *mctx, re_dfastate_t *pstate) if (naccepted == 0) continue; - /* The node can accepts `naccepted' bytes. */ + /* The node can accepts 'naccepted' bytes. */ dest_idx = re_string_cur_idx (&mctx->input) + naccepted; mctx->max_mb_elem_len = ((mctx->max_mb_elem_len < naccepted) ? naccepted : mctx->max_mb_elem_len); @@ -2513,18 +2532,18 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int i; - int cur_str_idx = re_string_cur_idx (&mctx->input); + Idx i; + Idx cur_str_idx = re_string_cur_idx (&mctx->input); for (i = 0; i < nodes->nelem; ++i) { - int dest_str_idx, prev_nelem, bkc_idx; - int node_idx = nodes->elems[i]; + Idx dest_str_idx, prev_nelem, bkc_idx; + Idx node_idx = nodes->elems[i]; unsigned int context; const re_token_t *node = dfa->nodes + node_idx; re_node_set *new_dest_nodes; - /* Check whether `node' is a backreference or not. */ + /* Check whether 'node' is a backreference or not. */ if (node->type != OP_BACK_REF) continue; @@ -2536,21 +2555,21 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) continue; } - /* `node' is a backreference. + /* 'node' is a backreference. Check the substring which the substring matched. */ bkc_idx = mctx->nbkref_ents; err = get_subexp (mctx, node_idx, cur_str_idx); if (BE (err != REG_NOERROR, 0)) goto free_return; - /* And add the epsilon closures (which is `new_dest_nodes') of + /* And add the epsilon closures (which is 'new_dest_nodes') of the backreference to appropriate state_log. */ #ifdef DEBUG assert (dfa->nexts[node_idx] != -1); #endif for (; bkc_idx < mctx->nbkref_ents; ++bkc_idx) { - int subexp_len; + Idx subexp_len; re_dfastate_t *dest_state; struct re_backref_cache_entry *bkref_ent; bkref_ent = mctx->bkref_ents + bkc_idx; @@ -2567,7 +2586,7 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) dest_state = mctx->state_log[dest_str_idx]; prev_nelem = ((mctx->state_log[cur_str_idx] == NULL) ? 0 : mctx->state_log[cur_str_idx]->nodes.nelem); - /* Add `new_dest_node' to state_log. */ + /* Add 'new_dest_node' to state_log. */ if (dest_state == NULL) { mctx->state_log[dest_str_idx] @@ -2623,13 +2642,13 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) static reg_errcode_t __attribute_warn_unused_result__ -get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) +get_subexp (re_match_context_t *mctx, Idx bkref_node, Idx bkref_str_idx) { const re_dfa_t *const dfa = mctx->dfa; - int subexp_num, sub_top_idx; + Idx subexp_num, sub_top_idx; const char *buf = (const char *) re_string_get_buffer (&mctx->input); /* Return if we have already checked BKREF_NODE at BKREF_STR_IDX. */ - int cache_idx = search_cur_bkref_entry (mctx, bkref_str_idx); + Idx cache_idx = search_cur_bkref_entry (mctx, bkref_str_idx); if (cache_idx != -1) { const struct re_backref_cache_entry *entry @@ -2648,7 +2667,7 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) reg_errcode_t err; re_sub_match_top_t *sub_top = mctx->sub_tops[sub_top_idx]; re_sub_match_last_t *sub_last; - int sub_last_idx, sl_str, bkref_str_off; + Idx sub_last_idx, sl_str, bkref_str_off; if (dfa->nodes[sub_top->node].opr.idx != subexp_num) continue; /* It isn't related. */ @@ -2659,7 +2678,7 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) evaluated. */ for (sub_last_idx = 0; sub_last_idx < sub_top->nlasts; ++sub_last_idx) { - int sl_str_diff; + regoff_t sl_str_diff; sub_last = sub_top->lasts[sub_last_idx]; sl_str_diff = sub_last->str_idx - sl_str; /* The matched string by the sub expression match with the substring @@ -2705,7 +2724,8 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) /* Then, search for the other last nodes of the sub expression. */ for (; sl_str <= bkref_str_idx; ++sl_str) { - int cls_node, sl_str_off; + Idx cls_node; + regoff_t sl_str_off; const re_node_set *nodes; sl_str_off = sl_str - sub_top->str_idx; /* The matched string by the sub expression match with the substring @@ -2772,10 +2792,10 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) static reg_errcode_t get_subexp_sub (re_match_context_t *mctx, const re_sub_match_top_t *sub_top, - re_sub_match_last_t *sub_last, int bkref_node, int bkref_str) + re_sub_match_last_t *sub_last, Idx bkref_node, Idx bkref_str) { reg_errcode_t err; - int to_idx; + Idx to_idx; /* Can the subexpression arrive the back reference? */ err = check_arrival (mctx, &sub_last->path, sub_last->node, sub_last->str_idx, bkref_node, bkref_str, @@ -2798,14 +2818,14 @@ get_subexp_sub (re_match_context_t *mctx, const re_sub_match_top_t *sub_top, nodes. E.g. RE: (a){2} */ -static int +static Idx find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, - int subexp_idx, int type) + Idx subexp_idx, int type) { - int cls_idx; + Idx cls_idx; for (cls_idx = 0; cls_idx < nodes->nelem; ++cls_idx) { - int cls_node = nodes->elems[cls_idx]; + Idx cls_node = nodes->elems[cls_idx]; const re_token_t *node = dfa->nodes + cls_node; if (node->type == type && node->opr.idx == subexp_idx) @@ -2821,12 +2841,12 @@ find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, static reg_errcode_t __attribute_warn_unused_result__ -check_arrival (re_match_context_t *mctx, state_array_t *path, int top_node, - int top_str, int last_node, int last_str, int type) +check_arrival (re_match_context_t *mctx, state_array_t *path, Idx top_node, + Idx top_str, Idx last_node, Idx last_str, int type) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err = REG_NOERROR; - int subexp_num, backup_cur_idx, str_idx, null_cnt; + Idx subexp_num, backup_cur_idx, str_idx, null_cnt; re_dfastate_t *cur_state = NULL; re_node_set *cur_nodes, next_nodes; re_dfastate_t **backup_state_log; @@ -2837,20 +2857,24 @@ check_arrival (re_match_context_t *mctx, state_array_t *path, int top_node, if (BE (path->alloc < last_str + mctx->max_mb_elem_len + 1, 0)) { re_dfastate_t **new_array; - int old_alloc = path->alloc; - path->alloc += last_str + mctx->max_mb_elem_len + 1; - new_array = re_realloc (path->array, re_dfastate_t *, path->alloc); + Idx old_alloc = path->alloc; + Idx incr_alloc = last_str + mctx->max_mb_elem_len + 1; + Idx new_alloc; + if (BE (IDX_MAX - old_alloc < incr_alloc, 0)) + return REG_ESPACE; + new_alloc = old_alloc + incr_alloc; + if (BE (SIZE_MAX / sizeof (re_dfastate_t *) < new_alloc, 0)) + return REG_ESPACE; + new_array = re_realloc (path->array, re_dfastate_t *, new_alloc); if (BE (new_array == NULL, 0)) - { - path->alloc = old_alloc; - return REG_ESPACE; - } + return REG_ESPACE; path->array = new_array; + path->alloc = new_alloc; memset (new_array + old_alloc, '\0', sizeof (re_dfastate_t *) * (path->alloc - old_alloc)); } - str_idx = path->next_idx ?: top_str; + str_idx = path->next_idx ? path->next_idx : top_str; /* Temporary modify MCTX. */ backup_state_log = mctx->state_log; @@ -2982,12 +3006,12 @@ check_arrival (re_match_context_t *mctx, state_array_t *path, int top_node, static reg_errcode_t __attribute_warn_unused_result__ -check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, +check_arrival_add_next_nodes (re_match_context_t *mctx, Idx str_idx, re_node_set *cur_nodes, re_node_set *next_nodes) { const re_dfa_t *const dfa = mctx->dfa; - int result; - int cur_idx; + bool ok; + Idx cur_idx; #ifdef RE_ENABLE_I18N reg_errcode_t err = REG_NOERROR; #endif @@ -2996,13 +3020,13 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, for (cur_idx = 0; cur_idx < cur_nodes->nelem; ++cur_idx) { int naccepted = 0; - int cur_node = cur_nodes->elems[cur_idx]; + Idx cur_node = cur_nodes->elems[cur_idx]; #ifdef DEBUG re_token_type_t type = dfa->nodes[cur_node].type; assert (!IS_EPSILON_NODE (type)); #endif #ifdef RE_ENABLE_I18N - /* If the node may accept `multi byte'. */ + /* If the node may accept "multi byte". */ if (dfa->nodes[cur_node].accept_mb) { naccepted = check_node_accept_bytes (dfa, cur_node, &mctx->input, @@ -3010,8 +3034,8 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, if (naccepted > 1) { re_dfastate_t *dest_state; - int next_node = dfa->nexts[cur_node]; - int next_idx = str_idx + naccepted; + Idx next_node = dfa->nexts[cur_node]; + Idx next_idx = str_idx + naccepted; dest_state = mctx->state_log[next_idx]; re_node_set_empty (&union_set); if (dest_state) @@ -3023,8 +3047,8 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, return err; } } - result = re_node_set_insert (&union_set, next_node); - if (BE (result < 0, 0)) + ok = re_node_set_insert (&union_set, next_node); + if (BE (! ok, 0)) { re_node_set_free (&union_set); return REG_ESPACE; @@ -3043,8 +3067,8 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, if (naccepted || check_node_accept (mctx, dfa->nodes + cur_node, str_idx)) { - result = re_node_set_insert (next_nodes, dfa->nexts[cur_node]); - if (BE (result < 0, 0)) + ok = re_node_set_insert (next_nodes, dfa->nexts[cur_node]); + if (BE (! ok, 0)) { re_node_set_free (&union_set); return REG_ESPACE; @@ -3063,10 +3087,10 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, static reg_errcode_t check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, - int ex_subexp, int type) + Idx ex_subexp, int type) { reg_errcode_t err; - int idx, outside_node; + Idx idx, outside_node; re_node_set new_nodes; #ifdef DEBUG assert (cur_nodes->nelem); @@ -3079,7 +3103,7 @@ check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, for (idx = 0; idx < cur_nodes->nelem; ++idx) { - int cur_node = cur_nodes->elems[idx]; + Idx cur_node = cur_nodes->elems[idx]; const re_node_set *eclosure = dfa->eclosures + cur_node; outside_node = find_subexp_node (dfa, eclosure, ex_subexp, type); if (outside_node == -1) @@ -3116,31 +3140,32 @@ check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, static reg_errcode_t __attribute_warn_unused_result__ check_arrival_expand_ecl_sub (const re_dfa_t *dfa, re_node_set *dst_nodes, - int target, int ex_subexp, int type) + Idx target, Idx ex_subexp, int type) { - int cur_node; + Idx cur_node; for (cur_node = target; !re_node_set_contains (dst_nodes, cur_node);) { - int err; + bool ok; if (dfa->nodes[cur_node].type == type && dfa->nodes[cur_node].opr.idx == ex_subexp) { if (type == OP_CLOSE_SUBEXP) { - err = re_node_set_insert (dst_nodes, cur_node); - if (BE (err == -1, 0)) + ok = re_node_set_insert (dst_nodes, cur_node); + if (BE (! ok, 0)) return REG_ESPACE; } break; } - err = re_node_set_insert (dst_nodes, cur_node); - if (BE (err == -1, 0)) + ok = re_node_set_insert (dst_nodes, cur_node); + if (BE (! ok, 0)) return REG_ESPACE; if (dfa->edests[cur_node].nelem == 0) break; if (dfa->edests[cur_node].nelem == 2) { + reg_errcode_t err; err = check_arrival_expand_ecl_sub (dfa, dst_nodes, dfa->edests[cur_node].elems[1], ex_subexp, type); @@ -3160,11 +3185,11 @@ check_arrival_expand_ecl_sub (const re_dfa_t *dfa, re_node_set *dst_nodes, static reg_errcode_t __attribute_warn_unused_result__ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, - int cur_str, int subexp_num, int type) + Idx cur_str, Idx subexp_num, int type) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int cache_idx_start = search_cur_bkref_entry (mctx, cur_str); + Idx cache_idx_start = search_cur_bkref_entry (mctx, cur_str); struct re_backref_cache_entry *ent; if (cache_idx_start == -1) @@ -3174,7 +3199,7 @@ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, ent = mctx->bkref_ents + cache_idx_start; do { - int to_idx, next_node; + Idx to_idx, next_node; /* Is this entry ENT is appropriate? */ if (!re_node_set_contains (cur_nodes, ent->node)) @@ -3212,14 +3237,14 @@ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, next_node = dfa->nexts[ent->node]; if (mctx->state_log[to_idx]) { - int ret; + bool ok; if (re_node_set_contains (&mctx->state_log[to_idx]->nodes, next_node)) continue; err = re_node_set_init_copy (&union_set, &mctx->state_log[to_idx]->nodes); - ret = re_node_set_insert (&union_set, next_node); - if (BE (err != REG_NOERROR || ret < 0, 0)) + ok = re_node_set_insert (&union_set, next_node); + if (BE (err != REG_NOERROR || ! ok, 0)) { re_node_set_free (&union_set); err = err != REG_NOERROR ? err : REG_ESPACE; @@ -3244,17 +3269,19 @@ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, } /* Build transition table for the state. - Return 1 if succeeded, otherwise return NULL. */ + Return true if successful. */ -static int +static bool build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) { reg_errcode_t err; - int i, j, ch, need_word_trtable = 0; + Idx i, j; + int ch; + bool need_word_trtable = false; bitset_word_t elem, mask; bool dests_node_malloced = false; bool dest_states_malloced = false; - int ndests; /* Number of the destination states from `state'. */ + Idx ndests; /* Number of the destination states from 'state'. */ re_dfastate_t **trtable; re_dfastate_t **dest_states = NULL, **dest_states_word, **dest_states_nl; re_node_set follows, *dests_node; @@ -3268,8 +3295,8 @@ build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) } *dests_alloc; /* We build DFA states which corresponds to the destination nodes - from `state'. `dests_node[i]' represents the nodes which i-th - destination state contains, and `dests_ch[i]' represents the + from 'state'. 'dests_node[i]' represents the nodes which i-th + destination state contains, and 'dests_ch[i]' represents the characters which i-th destination state accepts. */ if (__libc_use_alloca (sizeof (struct dests_alloc))) dests_alloc = (struct dests_alloc *) alloca (sizeof (struct dests_alloc)); @@ -3277,32 +3304,32 @@ build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) { dests_alloc = re_malloc (struct dests_alloc, 1); if (BE (dests_alloc == NULL, 0)) - return 0; + return false; dests_node_malloced = true; } dests_node = dests_alloc->dests_node; dests_ch = dests_alloc->dests_ch; - /* Initialize transiton table. */ + /* Initialize transition table. */ state->word_trtable = state->trtable = NULL; - /* At first, group all nodes belonging to `state' into several + /* At first, group all nodes belonging to 'state' into several destinations. */ ndests = group_nodes_into_DFAstates (dfa, state, dests_node, dests_ch); if (BE (ndests <= 0, 0)) { if (dests_node_malloced) - free (dests_alloc); - /* Return 0 in case of an error, 1 otherwise. */ + re_free (dests_alloc); + /* Return false in case of an error, true otherwise. */ if (ndests == 0) { state->trtable = (re_dfastate_t **) calloc (sizeof (re_dfastate_t *), SBC_MAX); - if (BE (state->trtable == NULL, 0)) - return 0; - return 1; + if (BE (state->trtable == NULL, 0)) + return false; + return true; } - return 0; + return false; } err = re_node_set_alloc (&follows, ndests + 1); @@ -3322,19 +3349,18 @@ build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) alloca (ndests * 3 * sizeof (re_dfastate_t *)); else { - dest_states = (re_dfastate_t **) - malloc (ndests * 3 * sizeof (re_dfastate_t *)); + dest_states = re_malloc (re_dfastate_t *, ndests * 3); if (BE (dest_states == NULL, 0)) { out_free: if (dest_states_malloced) - free (dest_states); + re_free (dest_states); re_node_set_free (&follows); for (i = 0; i < ndests; ++i) re_node_set_free (dests_node + i); if (dests_node_malloced) - free (dests_alloc); - return 0; + re_free (dests_alloc); + return false; } dest_states_malloced = true; } @@ -3345,7 +3371,7 @@ out_free: /* Then build the states for all destinations. */ for (i = 0; i < ndests; ++i) { - int next_node; + Idx next_node; re_node_set_empty (&follows); /* Merge the follows of this destination states. */ for (j = 0; j < dests_node[i].nelem; ++j) @@ -3371,13 +3397,13 @@ out_free: goto out_free; if (dest_states[i] != dest_states_word[i] && dfa->mb_cur_max > 1) - need_word_trtable = 1; + need_word_trtable = true; dest_states_nl[i] = re_acquire_state_context (&err, dfa, &follows, CONTEXT_NEWLINE); if (BE (dest_states_nl[i] == NULL && err != REG_NOERROR, 0)) goto out_free; - } + } else { dest_states_word[i] = dest_states[i]; @@ -3464,16 +3490,16 @@ out_free: } if (dest_states_malloced) - free (dest_states); + re_free (dest_states); re_node_set_free (&follows); for (i = 0; i < ndests; ++i) re_node_set_free (dests_node + i); if (dests_node_malloced) - free (dests_alloc); + re_free (dests_alloc); - return 1; + return true; } /* Group all nodes belonging to STATE into several destinations. @@ -3481,20 +3507,20 @@ out_free: to DESTS_NODE[i] and set the characters accepted by the destination to DEST_CH[i]. This function return the number of destinations. */ -static int +static Idx group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, re_node_set *dests_node, bitset_t *dests_ch) { reg_errcode_t err; - int result; - int i, j, k; - int ndests; /* Number of the destinations from `state'. */ + bool ok; + Idx i, j, k; + Idx ndests; /* Number of the destinations from 'state'. */ bitset_t accepts; /* Characters a node can accept. */ const re_node_set *cur_nodes = &state->nodes; bitset_empty (accepts); ndests = 0; - /* For all the nodes belonging to `state', */ + /* For all the nodes belonging to 'state', */ for (i = 0; i < cur_nodes->nelem; ++i) { re_token_t *node = &dfa->nodes[cur_nodes->elems[i]]; @@ -3524,7 +3550,10 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, #ifdef RE_ENABLE_I18N else if (type == OP_UTF8_PERIOD) { - memset (accepts, '\xff', sizeof (bitset_t) / 2); + if (ASCII_CHARS % BITSET_WORD_BITS == 0) + memset (accepts, -1, ASCII_CHARS / CHAR_BIT); + else + bitset_merge (accepts, utf8_sb_map); if (!(dfa->syntax & RE_DOT_NEWLINE)) bitset_clear (accepts, '\n'); if (dfa->syntax & RE_DOT_NOT_NULL) @@ -3534,7 +3563,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, else continue; - /* Check the `accepts' and sift the characters which are not + /* Check the 'accepts' and sift the characters which are not match it the context. */ if (constraint) { @@ -3593,7 +3622,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, } } - /* Then divide `accepts' into DFA states, or create a new + /* Then divide 'accepts' into DFA states, or create a new state. Above, we make sure that accepts is not empty. */ for (j = 0; j < ndests; ++j) { @@ -3606,7 +3635,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, if (type == CHARACTER && !bitset_contain (dests_ch[j], node->opr.c)) continue; - /* Enumerate the intersection set of this state and `accepts'. */ + /* Enumerate the intersection set of this state and 'accepts'. */ has_intersec = 0; for (k = 0; k < BITSET_WORDS; ++k) has_intersec |= intersec[k] = accepts[k] & dests_ch[j][k]; @@ -3614,7 +3643,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, if (!has_intersec) continue; - /* Then check if this state is a subset of `accepts'. */ + /* Then check if this state is a subset of 'accepts'. */ not_subset = not_consumed = 0; for (k = 0; k < BITSET_WORDS; ++k) { @@ -3622,8 +3651,8 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, not_consumed |= accepts[k] = accepts[k] & ~dests_ch[j][k]; } - /* If this state isn't a subset of `accepts', create a - new group state, which has the `remains'. */ + /* If this state isn't a subset of 'accepts', create a + new group state, which has the 'remains'. */ if (not_subset) { bitset_copy (dests_ch[ndests], remains); @@ -3635,8 +3664,8 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, } /* Put the position in the current group. */ - result = re_node_set_insert (&dests_node[j], cur_nodes->elems[i]); - if (BE (result < 0, 0)) + ok = re_node_set_insert (&dests_node[j], cur_nodes->elems[i]); + if (BE (! ok, 0)) goto error_return; /* If all characters are consumed, go to next node. */ @@ -3662,7 +3691,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, } #ifdef RE_ENABLE_I18N -/* Check how many bytes the node `dfa->nodes[node_idx]' accepts. +/* Check how many bytes the node 'dfa->nodes[node_idx]' accepts. Return the number of the bytes the node accepts. STR_IDX is the current index of the input string. @@ -3675,12 +3704,12 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, # endif static int -check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, - const re_string_t *input, int str_idx) +check_node_accept_bytes (const re_dfa_t *dfa, Idx node_idx, + const re_string_t *input, Idx str_idx) { const re_token_t *node = dfa->nodes + node_idx; int char_len, elem_len; - int i; + Idx i; if (BE (node->type == OP_UTF8_PERIOD, 0)) { @@ -3759,7 +3788,7 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, # ifdef _LIBC const unsigned char *pin = ((const unsigned char *) re_string_get_buffer (input) + str_idx); - int j; + Idx j; uint32_t nrules; # endif /* _LIBC */ int match_len = 0; @@ -3827,6 +3856,7 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, in_collseq = find_collation_sequence_value (pin, elem_len); } /* match with range expression? */ + /* FIXME: Implement rational ranges here, too. */ for (i = 0; i < cset->nranges; ++i) if (cset->range_starts[i] <= in_collseq && in_collseq <= cset->range_ends[i]) @@ -3856,7 +3886,7 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, if (weight_len == weights[equiv_class_idx & 0xffffff] && (idx >> 24) == (equiv_class_idx >> 24)) { - int cnt = 0; + Idx cnt = 0; idx &= 0xffffff; equiv_class_idx &= 0xffffff; @@ -3878,18 +3908,9 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, # endif /* _LIBC */ { /* match with range expression? */ -#if __GNUC__ >= 2 - wchar_t cmp_buf[] = {L'\0', L'\0', wc, L'\0', L'\0', L'\0'}; -#else - wchar_t cmp_buf[] = {L'\0', L'\0', L'\0', L'\0', L'\0', L'\0'}; - cmp_buf[2] = wc; -#endif for (i = 0; i < cset->nranges; ++i) { - cmp_buf[0] = cset->range_starts[i]; - cmp_buf[4] = cset->range_ends[i]; - if (__wcscoll (cmp_buf, cmp_buf + 2) <= 0 - && __wcscoll (cmp_buf + 2, cmp_buf + 4) <= 0) + if (cset->range_starts[i] <= wc && wc <= cset->range_ends[i]) { match_len = char_len; goto check_node_accept_bytes_match; @@ -3936,7 +3957,8 @@ find_collation_sequence_value (const unsigned char *mbs, size_t mbs_len) for (idx = 0; idx < extrasize;) { - int mbs_cnt, found = 0; + int mbs_cnt; + bool found = false; int32_t elem_mbs_len; /* Skip the name of collating element name. */ idx = idx + extra[idx] + 1; @@ -3948,7 +3970,7 @@ find_collation_sequence_value (const unsigned char *mbs, size_t mbs_len) break; if (mbs_cnt == elem_mbs_len) /* Found the entry. */ - found = 1; + found = true; } /* Skip the byte sequence of the collating element. */ idx += elem_mbs_len; @@ -3973,9 +3995,9 @@ find_collation_sequence_value (const unsigned char *mbs, size_t mbs_len) /* Check whether the node accepts the byte which is IDX-th byte of the INPUT. */ -static int +static bool check_node_accept (const re_match_context_t *mctx, const re_token_t *node, - int idx) + Idx idx) { unsigned char ch; ch = re_string_byte_at (&mctx->input, idx); @@ -3983,28 +4005,28 @@ check_node_accept (const re_match_context_t *mctx, const re_token_t *node, { case CHARACTER: if (node->opr.c != ch) - return 0; + return false; break; case SIMPLE_BRACKET: if (!bitset_contain (node->opr.sbcset, ch)) - return 0; + return false; break; #ifdef RE_ENABLE_I18N case OP_UTF8_PERIOD: - if (ch >= 0x80) - return 0; - /* FALLTHROUGH */ + if (ch >= ASCII_CHARS) + return false; + FALLTHROUGH; #endif case OP_PERIOD: if ((ch == '\n' && !(mctx->dfa->syntax & RE_DOT_NEWLINE)) || (ch == '\0' && (mctx->dfa->syntax & RE_DOT_NOT_NULL))) - return 0; + return false; break; default: - return 0; + return false; } if (node->constraint) @@ -4014,10 +4036,10 @@ check_node_accept (const re_match_context_t *mctx, const re_token_t *node, unsigned int context = re_string_context_at (&mctx->input, idx, mctx->eflags); if (NOT_SATISFY_NEXT_CONSTRAINT (node->constraint, context)) - return 0; + return false; } - return 1; + return true; } /* Extend the buffers, if the buffers have run out. */ @@ -4030,10 +4052,11 @@ extend_buffers (re_match_context_t *mctx, int min_len) re_string_t *pstr = &mctx->input; /* Avoid overflow. */ - if (BE (INT_MAX / 2 / sizeof (re_dfastate_t *) <= pstr->bufs_len, 0)) + if (BE (MIN (IDX_MAX, SIZE_MAX / sizeof (re_dfastate_t *)) / 2 + <= pstr->bufs_len, 0)) return REG_ESPACE; - /* Double the lengthes of the buffers, but allocate at least MIN_LEN. */ + /* Double the lengths of the buffers, but allocate at least MIN_LEN. */ ret = re_string_realloc_buffers (pstr, MAX (min_len, MIN (pstr->len, pstr->bufs_len * 2))); @@ -4089,12 +4112,19 @@ extend_buffers (re_match_context_t *mctx, int min_len) static reg_errcode_t __attribute_warn_unused_result__ -match_ctx_init (re_match_context_t *mctx, int eflags, int n) +match_ctx_init (re_match_context_t *mctx, int eflags, Idx n) { mctx->eflags = eflags; mctx->match_last = -1; if (n > 0) { + /* Avoid overflow. */ + size_t max_object_size = + MAX (sizeof (struct re_backref_cache_entry), + sizeof (re_sub_match_top_t *)); + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) < n, 0)) + return REG_ESPACE; + mctx->bkref_ents = re_malloc (struct re_backref_cache_entry, n); mctx->sub_tops = re_malloc (re_sub_match_top_t *, n); if (BE (mctx->bkref_ents == NULL || mctx->sub_tops == NULL, 0)) @@ -4118,10 +4148,10 @@ match_ctx_init (re_match_context_t *mctx, int eflags, int n) static void match_ctx_clean (re_match_context_t *mctx) { - int st_idx; + Idx st_idx; for (st_idx = 0; st_idx < mctx->nsub_tops; ++st_idx) { - int sl_idx; + Idx sl_idx; re_sub_match_top_t *top = mctx->sub_tops[st_idx]; for (sl_idx = 0; sl_idx < top->nlasts; ++sl_idx) { @@ -4135,7 +4165,7 @@ match_ctx_clean (re_match_context_t *mctx) re_free (top->path->array); re_free (top->path); } - free (top); + re_free (top); } mctx->nsub_tops = 0; @@ -4160,8 +4190,8 @@ match_ctx_free (re_match_context_t *mctx) static reg_errcode_t __attribute_warn_unused_result__ -match_ctx_add_entry (re_match_context_t *mctx, int node, int str_idx, int from, - int to) +match_ctx_add_entry (re_match_context_t *mctx, Idx node, Idx str_idx, Idx from, + Idx to) { if (mctx->nbkref_ents >= mctx->abkref_ents) { @@ -4196,7 +4226,7 @@ match_ctx_add_entry (re_match_context_t *mctx, int node, int str_idx, int from, A backreference does not epsilon-transition unless it is empty, so set to all zeros if FROM != TO. */ mctx->bkref_ents[mctx->nbkref_ents].eps_reachable_subexps_map - = (from == to ? ~0 : 0); + = (from == to ? -1 : 0); mctx->bkref_ents[mctx->nbkref_ents++].more = 0; if (mctx->max_mb_elem_len < to - from) @@ -4204,13 +4234,13 @@ match_ctx_add_entry (re_match_context_t *mctx, int node, int str_idx, int from, return REG_NOERROR; } -/* Search for the first entry which has the same str_idx, or -1 if none is +/* Return the first entry with the same str_idx, or -1 if none is found. Note that MCTX->BKREF_ENTS is already sorted by MCTX->STR_IDX. */ -static int -search_cur_bkref_entry (const re_match_context_t *mctx, int str_idx) +static Idx +search_cur_bkref_entry (const re_match_context_t *mctx, Idx str_idx) { - int left, right, mid, last; + Idx left, right, mid, last; last = right = mctx->nbkref_ents; for (left = 0; left < right;) { @@ -4231,7 +4261,7 @@ search_cur_bkref_entry (const re_match_context_t *mctx, int str_idx) static reg_errcode_t __attribute_warn_unused_result__ -match_ctx_add_subtop (re_match_context_t *mctx, int node, int str_idx) +match_ctx_add_subtop (re_match_context_t *mctx, Idx node, Idx str_idx) { #ifdef DEBUG assert (mctx->sub_tops != NULL); @@ -4239,7 +4269,7 @@ match_ctx_add_subtop (re_match_context_t *mctx, int node, int str_idx) #endif if (BE (mctx->nsub_tops == mctx->asub_tops, 0)) { - int new_asub_tops = mctx->asub_tops * 2; + Idx new_asub_tops = mctx->asub_tops * 2; re_sub_match_top_t **new_array = re_realloc (mctx->sub_tops, re_sub_match_top_t *, new_asub_tops); @@ -4260,12 +4290,12 @@ match_ctx_add_subtop (re_match_context_t *mctx, int node, int str_idx) at STR_IDX, whose corresponding OP_OPEN_SUBEXP is SUB_TOP. */ static re_sub_match_last_t * -match_ctx_add_sublast (re_sub_match_top_t *subtop, int node, int str_idx) +match_ctx_add_sublast (re_sub_match_top_t *subtop, Idx node, Idx str_idx) { re_sub_match_last_t *new_entry; if (BE (subtop->nlasts == subtop->alasts, 0)) { - int new_alasts = 2 * subtop->alasts + 1; + Idx new_alasts = 2 * subtop->alasts + 1; re_sub_match_last_t **new_array = re_realloc (subtop->lasts, re_sub_match_last_t *, new_alasts); @@ -4287,7 +4317,7 @@ match_ctx_add_sublast (re_sub_match_top_t *subtop, int node, int str_idx) static void sift_ctx_init (re_sift_context_t *sctx, re_dfastate_t **sifted_sts, - re_dfastate_t **limited_sts, int last_node, int last_str_idx) + re_dfastate_t **limited_sts, Idx last_node, Idx last_str_idx) { sctx->sifted_states = sifted_sts; sctx->limited_states = limited_sts; -- 2.7.4
Adhemerval Zanella wrote:
> Thanks, I sync in the patch I intend to push upstream.
That patch looks good. Thanks again for all your work on this.
diff --git a/posix/Makefile b/posix/Makefile index 989e42e..00c6284 100644 --- a/posix/Makefile +++ b/posix/Makefile @@ -95,7 +95,8 @@ tests := test-errno tstgetopt testfnm runtests runptests \ tst-posix_spawn-fd tst-posix_spawn-setsid \ tst-posix_fadvise tst-posix_fadvise64 \ tst-sysconf-empty-chroot tst-glob_symlinks tst-fexecve \ - tst-glob-tilde test-ssize-max tst-spawn4 + tst-glob-tilde test-ssize-max tst-spawn4 bug-regex37 \ + bug-regex38 tests-internal := bug-regex5 bug-regex20 bug-regex33 \ tst-rfc3484 tst-rfc3484-2 tst-rfc3484-3 \ tst-glob_lstat_compat tst-spawn4-compat diff --git a/posix/PCRE.tests b/posix/PCRE.tests index 0fb9cad..da63b86 100644 --- a/posix/PCRE.tests +++ b/posix/PCRE.tests @@ -1774,19 +1774,6 @@ No match 0: abcabc 1: abc -/(a)|\1/ - a - 0: a - 1: a - *** Failers - 0: a - 1: a - ab - 0: a - 1: a - x -No match - /abc/i ABC 0: ABC diff --git a/posix/bug-regex28.c b/posix/bug-regex28.c index 5353edf..ba263b2 100644 --- a/posix/bug-regex28.c +++ b/posix/bug-regex28.c @@ -21,18 +21,22 @@ #include <stdio.h> #include <string.h> +#include <support/test-driver.h> +#include <support/check.h> + struct tests { const char *regex; const char *string; reg_syntax_t syntax; int retval; -} tests[] = { +}; +static const struct tests tests[] = { #define EGREP RE_SYNTAX_EGREP #define EGREP_NL (RE_SYNTAX_EGREP | RE_DOT_NEWLINE) & ~RE_HAT_LISTS_NOT_NEWLINE - { "a.b", "a\nb", EGREP, -1 }, + { "a.b", "a\nb", EGREP, 0 }, { "a.b", "a\nb", EGREP_NL, 0 }, - { "a[^x]b", "a\nb", EGREP, -1 }, + { "a[^x]b", "a\nb", EGREP, 0 }, { "a[^x]b", "a\nb", EGREP_NL, 0 }, /* While \S and \W are internally handled as [^[:space:]] and [^[:alnum:]_], RE_HAT_LISTS_NOT_NEWLINE did not make any difference, so ensure @@ -42,33 +46,33 @@ struct tests { "a\\Wb", "a\nb", EGREP, 0 }, { "a\\Wb", "a\nb", EGREP_NL, 0 } }; +static const size_t tests_size = sizeof (tests) / sizeof (tests[0]); -int -main (void) +static int +do_test (void) { struct re_pattern_buffer r; - size_t i; - int ret = 0; - for (i = 0; i < sizeof (tests) / sizeof (tests[i]); ++i) + for (size_t i = 0; i < tests_size; i++) { re_set_syntax (tests[i].syntax); memset (&r, 0, sizeof (r)); - if (re_compile_pattern (tests[i].regex, strlen (tests[i].regex), &r)) - { - printf ("re_compile_pattern %zd failed\n", i); - ret = 1; - continue; - } + const char *re = re_compile_pattern (tests[i].regex, + strlen (tests[i].regex), &r); + TEST_VERIFY (re == NULL); + if (re != NULL) + continue; + size_t len = strlen (tests[i].string); int rv = re_search (&r, tests[i].string, len, 0, len, NULL); - if (rv != tests[i].retval) - { - printf ("re_search %zd unexpected value %d != %d\n", - i, rv, tests[i].retval); - ret = 1; - } + TEST_VERIFY (rv == tests[i].retval); + if (test_verbose > 0) + printf ("info: i=%zu rv=%d expected=%d\n", i, rv, tests[i].retval); + regfree (&r); } - return ret; + + return 0; } + +#include <support/test-driver.c> diff --git a/posix/bug-regex37.c b/posix/bug-regex37.c new file mode 100644 index 0000000..87a0916 --- /dev/null +++ b/posix/bug-regex37.c @@ -0,0 +1,32 @@ +/* Test regcomp return for invalid expression (BZ #21163). + Copyright (C) 2018 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <regex.h> + +#include <support/check.h> + +static int +do_test (void) +{ + char const pattern[] = "()*)|\\1)*"; + regex_t r; + TEST_VERIFY_EXIT (regcomp (&r, pattern, REG_EXTENDED) == REG_ESUBREG); + return 0; +} + +#include <support/test-driver.c> diff --git a/posix/bug-regex38.c b/posix/bug-regex38.c new file mode 100644 index 0000000..cb0eb7d --- /dev/null +++ b/posix/bug-regex38.c @@ -0,0 +1,32 @@ +/* Diagnose invalid back-reference in the ERE '()|\1' (BZ #18986). + Copyright (C) 2018 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <regex.h> + +#include <support/check.h> + +static int +do_test (void) +{ + char const pattern[] = "0|()0|\\1|0"; + regex_t r; + TEST_VERIFY_EXIT (regcomp (&r, pattern, REG_EXTENDED) == REG_ESUBREG); + return 0; +} + +#include <support/test-driver.c> diff --git a/posix/regcomp.c b/posix/regcomp.c index f5c09fe..8431268 100644 --- a/posix/regcomp.c +++ b/posix/regcomp.c @@ -17,8 +17,6 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <stdint.h> - #ifdef _LIBC # include <locale/weight.h> #endif @@ -51,14 +49,14 @@ static bin_tree_t *lower_subexp (reg_errcode_t *err, regex_t *preg, static reg_errcode_t calc_first (void *extra, bin_tree_t *node); static reg_errcode_t calc_next (void *extra, bin_tree_t *node); static reg_errcode_t link_nfa_nodes (void *extra, bin_tree_t *node); -static int duplicate_node (re_dfa_t *dfa, int org_idx, unsigned int constraint); -static int search_duplicated_node (const re_dfa_t *dfa, int org_node, +static Idx duplicate_node (re_dfa_t *dfa, Idx org_idx, unsigned int constraint); +static Idx search_duplicated_node (const re_dfa_t *dfa, Idx org_node, unsigned int constraint); static reg_errcode_t calc_eclosure (re_dfa_t *dfa); static reg_errcode_t calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, - int node, int root); + Idx node, bool root); static reg_errcode_t calc_inveclosure (re_dfa_t *dfa); -static int fetch_number (re_string_t *input, re_token_t *token, +static Idx fetch_number (re_string_t *input, re_token_t *token, reg_syntax_t syntax); static int peek_token (re_token_t *token, re_string_t *input, reg_syntax_t syntax); @@ -66,16 +64,16 @@ static bin_tree_t *parse (re_string_t *regexp, regex_t *preg, reg_syntax_t syntax, reg_errcode_t *err); static bin_tree_t *parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_sub_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, reg_syntax_t syntax, - int nest, reg_errcode_t *err); + Idx nest, reg_errcode_t *err); static bin_tree_t *parse_dup_op (bin_tree_t *dup_elem, re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, reg_syntax_t syntax, reg_errcode_t *err); @@ -87,34 +85,34 @@ static reg_errcode_t parse_bracket_element (bracket_elem_t *elem, re_token_t *token, int token_len, re_dfa_t *dfa, reg_syntax_t syntax, - int accept_hyphen); + bool accept_hyphen); static reg_errcode_t parse_bracket_symbol (bracket_elem_t *elem, re_string_t *regexp, re_token_t *token); #ifdef RE_ENABLE_I18N static reg_errcode_t build_equiv_class (bitset_t sbcset, re_charset_t *mbcset, - int *equiv_class_alloc, + Idx *equiv_class_alloc, const unsigned char *name); static reg_errcode_t build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, re_charset_t *mbcset, - int *char_class_alloc, - const unsigned char *class_name, + Idx *char_class_alloc, + const char *class_name, reg_syntax_t syntax); #else /* not RE_ENABLE_I18N */ static reg_errcode_t build_equiv_class (bitset_t sbcset, const unsigned char *name); static reg_errcode_t build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, - const unsigned char *class_name, + const char *class_name, reg_syntax_t syntax); #endif /* not RE_ENABLE_I18N */ static bin_tree_t *build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, - const unsigned char *class_name, - const unsigned char *extra, - int non_match, reg_errcode_t *err); + const char *class_name, + const char *extra, + bool non_match, reg_errcode_t *err); static bin_tree_t *create_tree (re_dfa_t *dfa, bin_tree_t *left, bin_tree_t *right, re_token_type_t type); @@ -131,7 +129,7 @@ static reg_errcode_t mark_opt_subexp (void *extra, bin_tree_t *node); POSIX doesn't require that we do anything for REG_NOERROR, but why not be nice? */ -const char __re_error_msgid[] attribute_hidden = +static const char __re_error_msgid[] = { #define REG_NOERROR_IDX 0 gettext_noop ("Success") /* REG_NOERROR */ @@ -155,9 +153,9 @@ const char __re_error_msgid[] attribute_hidden = gettext_noop ("Invalid back reference") /* REG_ESUBREG */ "\0" #define REG_EBRACK_IDX (REG_ESUBREG_IDX + sizeof "Invalid back reference") - gettext_noop ("Unmatched [ or [^") /* REG_EBRACK */ + gettext_noop ("Unmatched [, [^, [:, [., or [=") /* REG_EBRACK */ "\0" -#define REG_EPAREN_IDX (REG_EBRACK_IDX + sizeof "Unmatched [ or [^") +#define REG_EPAREN_IDX (REG_EBRACK_IDX + sizeof "Unmatched [, [^, [:, [., or [=") gettext_noop ("Unmatched ( or \\(") /* REG_EPAREN */ "\0" #define REG_EBRACE_IDX (REG_EPAREN_IDX + sizeof "Unmatched ( or \\(") @@ -185,7 +183,7 @@ const char __re_error_msgid[] attribute_hidden = gettext_noop ("Unmatched ) or \\)") /* REG_ERPAREN */ }; -const size_t __re_error_msgid_idx[] attribute_hidden = +static const size_t __re_error_msgid_idx[] = { REG_NOERROR_IDX, REG_NOMATCH_IDX, @@ -269,7 +267,7 @@ weak_alias (__re_set_syntax, re_set_syntax) int re_compile_fastmap (struct re_pattern_buffer *bufp) { - re_dfa_t *dfa = (re_dfa_t *) bufp->buffer; + re_dfa_t *dfa = bufp->buffer; char *fastmap = bufp->fastmap; memset (fastmap, '\0', sizeof (char) * SBC_MAX); @@ -303,12 +301,12 @@ static void re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, char *fastmap) { - re_dfa_t *dfa = (re_dfa_t *) bufp->buffer; - int node_cnt; - int icase = (dfa->mb_cur_max == 1 && (bufp->syntax & RE_ICASE)); + re_dfa_t *dfa = bufp->buffer; + Idx node_cnt; + bool icase = (dfa->mb_cur_max == 1 && (bufp->syntax & RE_ICASE)); for (node_cnt = 0; node_cnt < init_state->nodes.nelem; ++node_cnt) { - int node = init_state->nodes.elems[node_cnt]; + Idx node = init_state->nodes.elems[node_cnt]; re_token_type_t type = dfa->nodes[node].type; if (type == CHARACTER) @@ -317,7 +315,8 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, #ifdef RE_ENABLE_I18N if ((bufp->syntax & RE_ICASE) && dfa->mb_cur_max > 1) { - unsigned char *buf = alloca (dfa->mb_cur_max), *p; + unsigned char buf[MB_LEN_MAX]; + unsigned char *p; wchar_t wc; mbstate_t state; @@ -332,7 +331,7 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, &state) == p - buf && (__wcrtomb ((char *) buf, __towlower (wc), &state) != (size_t) -1)) - re_set_fastmap (fastmap, 0, buf[0]); + re_set_fastmap (fastmap, false, buf[0]); } #endif } @@ -352,7 +351,7 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, else if (type == COMPLEX_BRACKET) { re_charset_t *cset = dfa->nodes[node].opr.mbcset; - int i; + Idx i; # ifdef _LIBC /* See if we have to try all bytes which start multiple collation @@ -465,7 +464,7 @@ re_compile_fastmap_iter (regex_t *bufp, const re_dfastate_t *init_state, the return codes and their meanings.) */ int -regcomp (regex_t *__restrict preg, const char *__restrict pattern, int cflags) +regcomp (regex_t *_Restrict_ preg, const char *_Restrict_ pattern, int cflags) { reg_errcode_t ret; reg_syntax_t syntax = ((cflags & REG_EXTENDED) ? RE_SYNTAX_POSIX_EXTENDED @@ -525,7 +524,7 @@ weak_alias (__regcomp, regcomp) from either regcomp or regexec. We don't use PREG here. */ size_t -regerror (int errcode, const regex_t *__restrict preg, char *__restrict errbuf, +regerror (int errcode, const regex_t *_Restrict_ preg, char *_Restrict_ errbuf, size_t errbuf_size) { const char *msg; @@ -546,17 +545,13 @@ regerror (int errcode, const regex_t *__restrict preg, char *__restrict errbuf, if (BE (errbuf_size != 0, 1)) { + size_t cpy_size = msg_size; if (BE (msg_size > errbuf_size, 0)) { -#if defined HAVE_MEMPCPY || defined _LIBC - *((char *) __mempcpy (errbuf, msg, errbuf_size - 1)) = '\0'; -#else - memcpy (errbuf, msg, errbuf_size - 1); - errbuf[errbuf_size - 1] = 0; -#endif + cpy_size = errbuf_size - 1; + errbuf[cpy_size] = '\0'; } - else - memcpy (errbuf, msg, msg_size); + memcpy (errbuf, msg, cpy_size); } return msg_size; @@ -574,7 +569,23 @@ weak_alias (__regerror, regerror) static const bitset_t utf8_sb_map = { /* Set the first 128 bits. */ +# if defined __GNUC__ && !defined __STRICT_ANSI__ [0 ... 0x80 / BITSET_WORD_BITS - 1] = BITSET_WORD_MAX +# else +# if 4 * BITSET_WORD_BITS < ASCII_CHARS +# error "bitset_word_t is narrower than 32 bits" +# elif 3 * BITSET_WORD_BITS < ASCII_CHARS + BITSET_WORD_MAX, BITSET_WORD_MAX, BITSET_WORD_MAX, +# elif 2 * BITSET_WORD_BITS < ASCII_CHARS + BITSET_WORD_MAX, BITSET_WORD_MAX, +# elif 1 * BITSET_WORD_BITS < ASCII_CHARS + BITSET_WORD_MAX, +# endif + (BITSET_WORD_MAX + >> (SBC_MAX % BITSET_WORD_BITS == 0 + ? 0 + : BITSET_WORD_BITS - SBC_MAX % BITSET_WORD_BITS)) +# endif }; #endif @@ -582,7 +593,7 @@ static const bitset_t utf8_sb_map = static void free_dfa_content (re_dfa_t *dfa) { - int i, j; + Idx i, j; if (dfa->nodes) for (i = 0; i < dfa->nodes_len; ++i) @@ -632,9 +643,12 @@ free_dfa_content (re_dfa_t *dfa) void regfree (regex_t *preg) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; if (BE (dfa != NULL, 1)) - free_dfa_content (dfa); + { + lock_fini (dfa->lock); + free_dfa_content (dfa); + } preg->buffer = NULL; preg->allocated = 0; @@ -687,7 +701,7 @@ re_comp (const char *s) if (re_comp_buf.fastmap == NULL) { - re_comp_buf.fastmap = (char *) malloc (SBC_MAX); + re_comp_buf.fastmap = re_malloc (char, SBC_MAX); if (re_comp_buf.fastmap == NULL) return (char *) gettext (__re_error_msgid + __re_error_msgid_idx[(int) REG_ESPACE]); @@ -704,7 +718,7 @@ re_comp (const char *s) if (!ret) return NULL; - /* Yes, we're discarding `const' here if !HAVE_LIBINTL. */ + /* Yes, we're discarding 'const' here if !HAVE_LIBINTL. */ return (char *) gettext (__re_error_msgid + __re_error_msgid_idx[(int) ret]); } @@ -739,7 +753,7 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, preg->regs_allocated = REGS_UNALLOCATED; /* Initialize the dfa. */ - dfa = (re_dfa_t *) preg->buffer; + dfa = preg->buffer; if (BE (preg->allocated < sizeof (re_dfa_t), 0)) { /* If zero allocated, but buffer is non-null, try to realloc @@ -750,11 +764,13 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, if (dfa == NULL) return REG_ESPACE; preg->allocated = sizeof (re_dfa_t); - preg->buffer = (unsigned char *) dfa; + preg->buffer = dfa; } preg->used = sizeof (re_dfa_t); err = init_dfa (dfa, length); + if (BE (err == REG_NOERROR && lock_init (dfa->lock) != 0, 0)) + err = REG_ESPACE; if (BE (err != REG_NOERROR, 0)) { free_dfa_content (dfa); @@ -771,12 +787,13 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, __libc_lock_init (dfa->lock); err = re_string_construct (®exp, pattern, length, preg->translate, - syntax & RE_ICASE, dfa); + (syntax & RE_ICASE) != 0, dfa); if (BE (err != REG_NOERROR, 0)) { re_compile_internal_free_return: free_workarea_compile (preg); re_string_destruct (®exp); + lock_fini (dfa->lock); free_dfa_content (dfa); preg->buffer = NULL; preg->allocated = 0; @@ -809,6 +826,7 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, if (BE (err != REG_NOERROR, 0)) { + lock_fini (dfa->lock); free_dfa_content (dfa); preg->buffer = NULL; preg->allocated = 0; @@ -823,18 +841,32 @@ re_compile_internal (regex_t *preg, const char * pattern, size_t length, static reg_errcode_t init_dfa (re_dfa_t *dfa, size_t pat_len) { - unsigned int table_size; + __re_size_t table_size; #ifndef _LIBC - char *codeset_name; + const char *codeset_name; +#endif +#ifdef RE_ENABLE_I18N + size_t max_i18n_object_size = MAX (sizeof (wchar_t), sizeof (wctype_t)); +#else + size_t max_i18n_object_size = 0; #endif + size_t max_object_size = + MAX (sizeof (struct re_state_table_entry), + MAX (sizeof (re_token_t), + MAX (sizeof (re_node_set), + MAX (sizeof (regmatch_t), + max_i18n_object_size)))); memset (dfa, '\0', sizeof (re_dfa_t)); /* Force allocation of str_tree_storage the first time. */ dfa->str_tree_storage_idx = BIN_TREE_STORAGE_SIZE; - /* Avoid overflows. */ - if (pat_len == SIZE_MAX) + /* Avoid overflows. The extra "/ 2" is for the table_size doubling + calculation below, and for similar doubling calculations + elsewhere. And it's <= rather than <, because some of the + doubling calculations add 1 afterwards. */ + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) / 2 <= pat_len, 0)) return REG_ESPACE; dfa->nodes_alloc = pat_len + 1; @@ -856,22 +888,11 @@ init_dfa (re_dfa_t *dfa, size_t pat_len) dfa->map_notascii = (_NL_CURRENT_WORD (LC_CTYPE, _NL_CTYPE_MAP_TO_NONASCII) != 0); #else -# ifdef HAVE_LANGINFO_CODESET codeset_name = nl_langinfo (CODESET); -# else - codeset_name = getenv ("LC_ALL"); - if (codeset_name == NULL || codeset_name[0] == '\0') - codeset_name = getenv ("LC_CTYPE"); - if (codeset_name == NULL || codeset_name[0] == '\0') - codeset_name = getenv ("LANG"); - if (codeset_name == NULL) - codeset_name = ""; - else if (strchr (codeset_name, '.') != NULL) - codeset_name = strchr (codeset_name, '.') + 1; -# endif - - if (strcasecmp (codeset_name, "UTF-8") == 0 - || strcasecmp (codeset_name, "UTF8") == 0) + if ((codeset_name[0] == 'U' || codeset_name[0] == 'u') + && (codeset_name[1] == 'T' || codeset_name[1] == 't') + && (codeset_name[2] == 'F' || codeset_name[2] == 'f') + && strcmp (codeset_name + 3 + (codeset_name[3] == '-'), "8") == 0) dfa->is_utf8 = 1; /* We check exhaustively in the loop below if this charset is a @@ -920,9 +941,10 @@ init_dfa (re_dfa_t *dfa, size_t pat_len) static void init_word_char (re_dfa_t *dfa) { - dfa->word_ops_used = 1; int i = 0; + int j; int ch = 0; + dfa->word_ops_used = 1; if (BE (dfa->map_notascii == 0, 1)) { /* Avoid uint32_t and uint64_t as some non-GCC platforms lack @@ -959,7 +981,7 @@ init_word_char (re_dfa_t *dfa) general_case: for (; i < BITSET_WORDS; ++i) - for (int j = 0; j < BITSET_WORD_BITS; ++j, ++ch) + for (j = 0; j < BITSET_WORD_BITS; ++j, ++ch) if (isalnum (ch) || ch == '_') dfa->word_char[i] |= (bitset_word_t) 1 << j; } @@ -969,7 +991,7 @@ init_word_char (re_dfa_t *dfa) static void free_workarea_compile (regex_t *preg) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_storage_t *storage, *next; for (storage = dfa->str_tree_storage; storage; storage = next) { @@ -988,7 +1010,7 @@ free_workarea_compile (regex_t *preg) static reg_errcode_t create_initial_state (re_dfa_t *dfa) { - int first, i; + Idx first, i; reg_errcode_t err; re_node_set init_nodes; @@ -1007,10 +1029,10 @@ create_initial_state (re_dfa_t *dfa) if (dfa->nbackref > 0) for (i = 0; i < init_nodes.nelem; ++i) { - int node_idx = init_nodes.elems[i]; + Idx node_idx = init_nodes.elems[i]; re_token_type_t type = dfa->nodes[node_idx].type; - int clexp_idx; + Idx clexp_idx; if (type != OP_BACK_REF) continue; for (clexp_idx = 0; clexp_idx < init_nodes.nelem; ++clexp_idx) @@ -1026,14 +1048,13 @@ create_initial_state (re_dfa_t *dfa) if (type == OP_BACK_REF) { - int dest_idx = dfa->edests[node_idx].elems[0]; + Idx dest_idx = dfa->edests[node_idx].elems[0]; if (!re_node_set_contains (&init_nodes, dest_idx)) { - reg_errcode_t err = re_node_set_merge (&init_nodes, - dfa->eclosures - + dest_idx); - if (err != REG_NOERROR) - return err; + reg_errcode_t merge_err + = re_node_set_merge (&init_nodes, dfa->eclosures + dest_idx); + if (merge_err != REG_NOERROR) + return merge_err; i = 0; } } @@ -1074,14 +1095,17 @@ create_initial_state (re_dfa_t *dfa) static void optimize_utf8 (re_dfa_t *dfa) { - int node, i, mb_chars = 0, has_period = 0; + Idx node; + int i; + bool mb_chars = false; + bool has_period = false; for (node = 0; node < dfa->nodes_len; ++node) switch (dfa->nodes[node].type) { case CHARACTER: - if (dfa->nodes[node].opr.c >= 0x80) - mb_chars = 1; + if (dfa->nodes[node].opr.c >= ASCII_CHARS) + mb_chars = true; break; case ANCHOR: switch (dfa->nodes[node].opr.ctx_type) @@ -1099,7 +1123,7 @@ optimize_utf8 (re_dfa_t *dfa) } break; case OP_PERIOD: - has_period = 1; + has_period = true; break; case OP_BACK_REF: case OP_ALT: @@ -1111,11 +1135,18 @@ optimize_utf8 (re_dfa_t *dfa) case COMPLEX_BRACKET: return; case SIMPLE_BRACKET: - /* Just double check. The non-ASCII range starts at 0x80. */ - assert (0x80 % BITSET_WORD_BITS == 0); - for (i = 0x80 / BITSET_WORD_BITS; i < BITSET_WORDS; ++i) - if (dfa->nodes[node].opr.sbcset[i]) - return; + /* Just double check. */ + { + int rshift = (ASCII_CHARS % BITSET_WORD_BITS == 0 + ? 0 + : BITSET_WORD_BITS - ASCII_CHARS % BITSET_WORD_BITS); + for (i = ASCII_CHARS / BITSET_WORD_BITS; i < BITSET_WORDS; ++i) + { + if (dfa->nodes[node].opr.sbcset[i] >> rshift != 0) + return; + rshift = 0; + } + } break; default: abort (); @@ -1125,7 +1156,7 @@ optimize_utf8 (re_dfa_t *dfa) for (node = 0; node < dfa->nodes_len; ++node) { if (dfa->nodes[node].type == CHARACTER - && dfa->nodes[node].opr.c >= 0x80) + && dfa->nodes[node].opr.c >= ASCII_CHARS) dfa->nodes[node].mb_partial = 0; else if (dfa->nodes[node].type == OP_PERIOD) dfa->nodes[node].type = OP_UTF8_PERIOD; @@ -1144,22 +1175,22 @@ optimize_utf8 (re_dfa_t *dfa) static reg_errcode_t analyze (regex_t *preg) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; reg_errcode_t ret; /* Allocate arrays. */ - dfa->nexts = re_malloc (int, dfa->nodes_alloc); - dfa->org_indices = re_malloc (int, dfa->nodes_alloc); + dfa->nexts = re_malloc (Idx, dfa->nodes_alloc); + dfa->org_indices = re_malloc (Idx, dfa->nodes_alloc); dfa->edests = re_malloc (re_node_set, dfa->nodes_alloc); dfa->eclosures = re_malloc (re_node_set, dfa->nodes_alloc); if (BE (dfa->nexts == NULL || dfa->org_indices == NULL || dfa->edests == NULL || dfa->eclosures == NULL, 0)) return REG_ESPACE; - dfa->subexp_map = re_malloc (int, preg->re_nsub); + dfa->subexp_map = re_malloc (Idx, preg->re_nsub); if (dfa->subexp_map != NULL) { - int i; + Idx i; for (i = 0; i < preg->re_nsub; i++) dfa->subexp_map[i] = i; preorder (dfa->str_tree, optimize_subexps, dfa); @@ -1168,7 +1199,7 @@ analyze (regex_t *preg) break; if (i == preg->re_nsub) { - free (dfa->subexp_map); + re_free (dfa->subexp_map); dfa->subexp_map = NULL; } } @@ -1284,7 +1315,7 @@ optimize_subexps (void *extra, bin_tree_t *node) else if (node->token.type == SUBEXP && node->left && node->left->token.type == SUBEXP) { - int other_idx = node->left->token.opr.idx; + Idx other_idx = node->left->token.opr.idx; node->left = node->left->left; if (node->left) @@ -1292,7 +1323,7 @@ optimize_subexps (void *extra, bin_tree_t *node) dfa->subexp_map[other_idx] = dfa->subexp_map[node->token.opr.idx]; if (other_idx < BITSET_WORD_BITS) - dfa->used_bkref_map &= ~((bitset_word_t) 1 << other_idx); + dfa->used_bkref_map &= ~((bitset_word_t) 1 << other_idx); } return REG_NOERROR; @@ -1325,7 +1356,7 @@ lower_subexps (void *extra, bin_tree_t *node) static bin_tree_t * lower_subexp (reg_errcode_t *err, regex_t *preg, bin_tree_t *node) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *body = node->left; bin_tree_t *op, *cls, *tree1, *tree; @@ -1408,7 +1439,7 @@ static reg_errcode_t link_nfa_nodes (void *extra, bin_tree_t *node) { re_dfa_t *dfa = (re_dfa_t *) extra; - int idx = node->node_idx; + Idx idx = node->node_idx; reg_errcode_t err = REG_NOERROR; switch (node->token.type) @@ -1423,7 +1454,7 @@ link_nfa_nodes (void *extra, bin_tree_t *node) case OP_DUP_ASTERISK: case OP_ALT: { - int left, right; + Idx left, right; dfa->has_plural_match = 1; if (node->left != NULL) left = node->left->first->node_idx; @@ -1465,14 +1496,15 @@ link_nfa_nodes (void *extra, bin_tree_t *node) to their own constraint. */ static reg_errcode_t -duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, - int root_node, unsigned int init_constraint) +duplicate_node_closure (re_dfa_t *dfa, Idx top_org_node, Idx top_clone_node, + Idx root_node, unsigned int init_constraint) { - int org_node, clone_node, ret; + Idx org_node, clone_node; + bool ok; unsigned int constraint = init_constraint; for (org_node = top_org_node, clone_node = top_clone_node;;) { - int org_dest, clone_dest; + Idx org_dest, clone_dest; if (dfa->nodes[org_node].type == OP_BACK_REF) { /* If the back reference epsilon-transit, its destination must @@ -1485,8 +1517,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, if (BE (clone_dest == -1, 0)) return REG_ESPACE; dfa->nexts[clone_node] = dfa->nexts[org_node]; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } else if (dfa->edests[org_node].nelem == 0) @@ -1504,12 +1536,12 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, org_dest = dfa->edests[org_node].elems[0]; re_node_set_empty (dfa->edests + clone_node); /* If the node is root_node itself, it means the epsilon closure - has a loop. Then tie it to the destination of the root_node. */ + has a loop. Then tie it to the destination of the root_node. */ if (org_node == root_node && clone_node != org_node) { - ret = re_node_set_insert (dfa->edests + clone_node, org_dest); - if (BE (ret < 0, 0)) - return REG_ESPACE; + ok = re_node_set_insert (dfa->edests + clone_node, org_dest); + if (BE (! ok, 0)) + return REG_ESPACE; break; } /* In case the node has another constraint, append it. */ @@ -1517,8 +1549,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, clone_dest = duplicate_node (dfa, org_dest, constraint); if (BE (clone_dest == -1, 0)) return REG_ESPACE; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } else /* dfa->edests[org_node].nelem == 2 */ @@ -1536,8 +1568,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, clone_dest = duplicate_node (dfa, org_dest, constraint); if (BE (clone_dest == -1, 0)) return REG_ESPACE; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; err = duplicate_node_closure (dfa, org_dest, clone_dest, root_node, constraint); @@ -1548,8 +1580,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, { /* There is a duplicated node which satisfies the constraint, use it to avoid infinite loop. */ - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } @@ -1557,8 +1589,8 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, clone_dest = duplicate_node (dfa, org_dest, constraint); if (BE (clone_dest == -1, 0)) return REG_ESPACE; - ret = re_node_set_insert (dfa->edests + clone_node, clone_dest); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (dfa->edests + clone_node, clone_dest); + if (BE (! ok, 0)) return REG_ESPACE; } org_node = org_dest; @@ -1570,11 +1602,11 @@ duplicate_node_closure (re_dfa_t *dfa, int top_org_node, int top_clone_node, /* Search for a node which is duplicated from the node ORG_NODE, and satisfies the constraint CONSTRAINT. */ -static int -search_duplicated_node (const re_dfa_t *dfa, int org_node, +static Idx +search_duplicated_node (const re_dfa_t *dfa, Idx org_node, unsigned int constraint) { - int idx; + Idx idx; for (idx = dfa->nodes_len - 1; dfa->nodes[idx].duplicated && idx > 0; --idx) { if (org_node == dfa->org_indices[idx] @@ -1588,10 +1620,10 @@ search_duplicated_node (const re_dfa_t *dfa, int org_node, Return the index of the new node, or -1 if insufficient storage is available. */ -static int -duplicate_node (re_dfa_t *dfa, int org_idx, unsigned int constraint) +static Idx +duplicate_node (re_dfa_t *dfa, Idx org_idx, unsigned int constraint) { - int dup_idx = re_dfa_add_node (dfa, dfa->nodes[org_idx]); + Idx dup_idx = re_dfa_add_node (dfa, dfa->nodes[org_idx]); if (BE (dup_idx != -1, 1)) { dfa->nodes[dup_idx].constraint = constraint; @@ -1607,17 +1639,18 @@ duplicate_node (re_dfa_t *dfa, int org_idx, unsigned int constraint) static reg_errcode_t calc_inveclosure (re_dfa_t *dfa) { - int src, idx, ret; + Idx src, idx; + bool ok; for (idx = 0; idx < dfa->nodes_len; ++idx) re_node_set_init_empty (dfa->inveclosures + idx); for (src = 0; src < dfa->nodes_len; ++src) { - int *elems = dfa->eclosures[src].elems; + Idx *elems = dfa->eclosures[src].elems; for (idx = 0; idx < dfa->eclosures[src].nelem; ++idx) { - ret = re_node_set_insert_last (dfa->inveclosures + elems[idx], src); - if (BE (ret == -1, 0)) + ok = re_node_set_insert_last (dfa->inveclosures + elems[idx], src); + if (BE (! ok, 0)) return REG_ESPACE; } } @@ -1630,11 +1663,12 @@ calc_inveclosure (re_dfa_t *dfa) static reg_errcode_t calc_eclosure (re_dfa_t *dfa) { - int node_idx, incomplete; + Idx node_idx; + bool incomplete; #ifdef DEBUG assert (dfa->nodes_len > 0); #endif - incomplete = 0; + incomplete = false; /* For each nodes, calculate epsilon closure. */ for (node_idx = 0; ; ++node_idx) { @@ -1644,7 +1678,7 @@ calc_eclosure (re_dfa_t *dfa) { if (!incomplete) break; - incomplete = 0; + incomplete = false; node_idx = 0; } @@ -1656,13 +1690,13 @@ calc_eclosure (re_dfa_t *dfa) if (dfa->eclosures[node_idx].nelem != 0) continue; /* Calculate epsilon closure of 'node_idx'. */ - err = calc_eclosure_iter (&eclosure_elem, dfa, node_idx, 1); + err = calc_eclosure_iter (&eclosure_elem, dfa, node_idx, true); if (BE (err != REG_NOERROR, 0)) return err; if (dfa->eclosures[node_idx].nelem == 0) { - incomplete = 1; + incomplete = true; re_node_set_free (&eclosure_elem); } } @@ -1672,13 +1706,13 @@ calc_eclosure (re_dfa_t *dfa) /* Calculate epsilon closure of NODE. */ static reg_errcode_t -calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, int node, int root) +calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, Idx node, bool root) { reg_errcode_t err; - int i; + Idx i; re_node_set eclosure; - int ret; - int incomplete = 0; + bool ok; + bool incomplete = false; err = re_node_set_alloc (&eclosure, dfa->edests[node].nelem + 1); if (BE (err != REG_NOERROR, 0)) return err; @@ -1704,19 +1738,19 @@ calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, int node, int root) for (i = 0; i < dfa->edests[node].nelem; ++i) { re_node_set eclosure_elem; - int edest = dfa->edests[node].elems[i]; - /* If calculating the epsilon closure of `edest' is in progress, + Idx edest = dfa->edests[node].elems[i]; + /* If calculating the epsilon closure of 'edest' is in progress, return intermediate result. */ if (dfa->eclosures[edest].nelem == -1) { - incomplete = 1; + incomplete = true; continue; } - /* If we haven't calculated the epsilon closure of `edest' yet, + /* If we haven't calculated the epsilon closure of 'edest' yet, calculate now. Otherwise use calculated epsilon closure. */ if (dfa->eclosures[edest].nelem == 0) { - err = calc_eclosure_iter (&eclosure_elem, dfa, edest, 0); + err = calc_eclosure_iter (&eclosure_elem, dfa, edest, false); if (BE (err != REG_NOERROR, 0)) return err; } @@ -1730,14 +1764,14 @@ calc_eclosure_iter (re_node_set *new_set, re_dfa_t *dfa, int node, int root) the epsilon closure of this node is also incomplete. */ if (dfa->eclosures[edest].nelem == 0) { - incomplete = 1; + incomplete = true; re_node_set_free (&eclosure_elem); } } /* An epsilon closure includes itself. */ - ret = re_node_set_insert (&eclosure, node); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (&eclosure, node); + if (BE (! ok, 0)) return REG_ESPACE; if (incomplete && !root) dfa->eclosures[node].nelem = 0; @@ -2046,16 +2080,18 @@ peek_token_bracket (re_token_t *token, re_string_t *input, reg_syntax_t syntax) case '.': token->type = OP_OPEN_COLL_ELEM; break; + case '=': token->type = OP_OPEN_EQUIV_CLASS; break; + case ':': if (syntax & RE_CHAR_CLASSES) { token->type = OP_OPEN_CHAR_CLASS; break; } - /* else fall through. */ + FALLTHROUGH; default: token->type = CHARACTER; token->opr.c = c; @@ -2099,7 +2135,7 @@ static bin_tree_t * parse (re_string_t *regexp, regex_t *preg, reg_syntax_t syntax, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree, *eor, *root; re_token_t current_token; dfa->syntax = syntax; @@ -2131,10 +2167,11 @@ parse (re_string_t *regexp, regex_t *preg, reg_syntax_t syntax, static bin_tree_t * parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree, *branch = NULL; + bitset_word_t initial_bkref_map = dfa->completed_bkref_map; tree = parse_branch (regexp, preg, token, syntax, nest, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; @@ -2145,6 +2182,8 @@ parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, if (token->type != OP_ALT && token->type != END_OF_RE && (nest == 0 || token->type != OP_CLOSE_SUBEXP)) { + bitset_word_t accumulated_bkref_map = dfa->completed_bkref_map; + dfa->completed_bkref_map = initial_bkref_map; branch = parse_branch (regexp, preg, token, syntax, nest, err); if (BE (*err != REG_NOERROR && branch == NULL, 0)) { @@ -2152,6 +2191,7 @@ parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, postorder (tree, free_tree, NULL); return NULL; } + dfa->completed_bkref_map |= accumulated_bkref_map; } else branch = NULL; @@ -2176,10 +2216,10 @@ parse_reg_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, static bin_tree_t * parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - bin_tree_t *tree, *exp; - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + bin_tree_t *tree, *expr; + re_dfa_t *dfa = preg->buffer; tree = parse_expression (regexp, preg, token, syntax, nest, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; @@ -2187,19 +2227,19 @@ parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, while (token->type != OP_ALT && token->type != END_OF_RE && (nest == 0 || token->type != OP_CLOSE_SUBEXP)) { - exp = parse_expression (regexp, preg, token, syntax, nest, err); - if (BE (*err != REG_NOERROR && exp == NULL, 0)) + expr = parse_expression (regexp, preg, token, syntax, nest, err); + if (BE (*err != REG_NOERROR && expr == NULL, 0)) { if (tree != NULL) postorder (tree, free_tree, NULL); return NULL; } - if (tree != NULL && exp != NULL) + if (tree != NULL && expr != NULL) { - bin_tree_t *newtree = create_tree (dfa, tree, exp, CONCAT); + bin_tree_t *newtree = create_tree (dfa, tree, expr, CONCAT); if (newtree == NULL) { - postorder (exp, free_tree, NULL); + postorder (expr, free_tree, NULL); postorder (tree, free_tree, NULL); *err = REG_ESPACE; return NULL; @@ -2207,8 +2247,8 @@ parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, tree = newtree; } else if (tree == NULL) - tree = exp; - /* Otherwise exp == NULL, we don't need to create new tree. */ + tree = expr; + /* Otherwise expr == NULL, we don't need to create new tree. */ } return tree; } @@ -2221,9 +2261,9 @@ parse_branch (re_string_t *regexp, regex_t *preg, re_token_t *token, static bin_tree_t * parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree; switch (token->type) { @@ -2253,16 +2293,19 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, } #endif break; + case OP_OPEN_SUBEXP: tree = parse_sub_exp (regexp, preg, token, syntax, nest + 1, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_OPEN_BRACKET: tree = parse_bracket_exp (regexp, dfa, token, syntax, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_BACK_REF: if (!BE (dfa->completed_bkref_map & (1 << token->opr.idx), 1)) { @@ -2279,13 +2322,14 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, ++dfa->nbackref; dfa->has_mb_node = 1; break; + case OP_OPEN_DUP_NUM: if (syntax & RE_CONTEXT_INVALID_DUP) { *err = REG_BADRPT; return NULL; } - /* FALLTHROUGH */ + FALLTHROUGH; case OP_DUP_ASTERISK: case OP_DUP_PLUS: case OP_DUP_QUESTION: @@ -2299,7 +2343,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, fetch_token (token, regexp, syntax); return parse_expression (regexp, preg, token, syntax, nest, err); } - /* else fall through */ + FALLTHROUGH; case OP_CLOSE_SUBEXP: if ((token->type == OP_CLOSE_SUBEXP) && !(syntax & RE_UNMATCHED_RIGHT_PAREN_ORD)) @@ -2307,7 +2351,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, *err = REG_ERPAREN; return NULL; } - /* else fall through */ + FALLTHROUGH; case OP_CLOSE_DUP_NUM: /* We treat it as a normal character. */ @@ -2322,6 +2366,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, return NULL; } break; + case ANCHOR: if ((token->opr.ctx_type & (WORD_DELIM | NOT_WORD_DELIM | WORD_FIRST | WORD_LAST)) @@ -2366,6 +2411,7 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, it must not be "<ANCHOR(^)><REPEAT(*)>". */ fetch_token (token, regexp, syntax); return tree; + case OP_PERIOD: tree = create_token_tree (dfa, NULL, NULL, token); if (BE (tree == NULL, 0)) @@ -2376,30 +2422,35 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, if (dfa->mb_cur_max > 1) dfa->has_mb_node = 1; break; + case OP_WORD: case OP_NOTWORD: tree = build_charclass_op (dfa, regexp->trans, - (const unsigned char *) "alnum", - (const unsigned char *) "_", + "alnum", + "_", token->type == OP_NOTWORD, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_SPACE: case OP_NOTSPACE: tree = build_charclass_op (dfa, regexp->trans, - (const unsigned char *) "space", - (const unsigned char *) "", + "space", + "", token->type == OP_NOTSPACE, err); if (BE (*err != REG_NOERROR && tree == NULL, 0)) return NULL; break; + case OP_ALT: case END_OF_RE: return NULL; + case BACK_SLASH: *err = REG_EESCAPE; return NULL; + default: /* Must not happen? */ #ifdef DEBUG @@ -2412,7 +2463,8 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, while (token->type == OP_DUP_ASTERISK || token->type == OP_DUP_PLUS || token->type == OP_DUP_QUESTION || token->type == OP_OPEN_DUP_NUM) { - bin_tree_t *dup_tree = parse_dup_op (tree, regexp, dfa, token, syntax, err); + bin_tree_t *dup_tree = parse_dup_op (tree, regexp, dfa, token, + syntax, err); if (BE (*err != REG_NOERROR && dup_tree == NULL, 0)) { if (tree != NULL) @@ -2444,9 +2496,9 @@ parse_expression (re_string_t *regexp, regex_t *preg, re_token_t *token, static bin_tree_t * parse_sub_exp (re_string_t *regexp, regex_t *preg, re_token_t *token, - reg_syntax_t syntax, int nest, reg_errcode_t *err) + reg_syntax_t syntax, Idx nest, reg_errcode_t *err) { - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + re_dfa_t *dfa = preg->buffer; bin_tree_t *tree; size_t cur_nsub; cur_nsub = preg->re_nsub++; @@ -2489,7 +2541,7 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, reg_syntax_t syntax, reg_errcode_t *err) { bin_tree_t *tree = NULL, *old_tree = NULL; - int i, start, end, start_idx = re_string_cur_idx (regexp); + Idx i, start, end, start_idx = re_string_cur_idx (regexp); re_token_t start_token = *token; if (token->type == OP_OPEN_DUP_NUM) @@ -2535,12 +2587,19 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, return elem; } - if (BE ((end != -1 && start > end) || token->type != OP_CLOSE_DUP_NUM, 0)) + if (BE ((end != -1 && start > end) + || token->type != OP_CLOSE_DUP_NUM, 0)) { /* First number greater than second. */ *err = REG_BADBR; return NULL; } + + if (BE (RE_DUP_MAX < (end == -1 ? start : end), 0)) + { + *err = REG_ESIZE; + return NULL; + } } else { @@ -2583,26 +2642,35 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, old_tree = NULL; if (elem->token.type == SUBEXP) - postorder (elem, mark_opt_subexp, (void *) (long) elem->token.opr.idx); + { + uintptr_t subidx = elem->token.opr.idx; + postorder (elem, mark_opt_subexp, (void *) subidx); + } - tree = create_tree (dfa, elem, NULL, (end == -1 ? OP_DUP_ASTERISK : OP_ALT)); + tree = create_tree (dfa, elem, NULL, + (end == -1 ? OP_DUP_ASTERISK : OP_ALT)); if (BE (tree == NULL, 0)) goto parse_dup_op_espace; +/* From gnulib's "intprops.h": + True if the arithmetic type T is signed. */ +#define TYPE_SIGNED(t) (! ((t) 0 < (t) -1)) + /* This loop is actually executed only when end != -1, to rewrite <re>{0,n} as (<re>(<re>...<re>?)?)?... We have already created the start+1-th copy. */ - for (i = start + 2; i <= end; ++i) - { - elem = duplicate_tree (elem, dfa); - tree = create_tree (dfa, tree, elem, CONCAT); - if (BE (elem == NULL || tree == NULL, 0)) - goto parse_dup_op_espace; - - tree = create_tree (dfa, tree, NULL, OP_ALT); - if (BE (tree == NULL, 0)) - goto parse_dup_op_espace; - } + if (TYPE_SIGNED (Idx) || end != -1) + for (i = start + 2; i <= end; ++i) + { + elem = duplicate_tree (elem, dfa); + tree = create_tree (dfa, tree, elem, CONCAT); + if (BE (elem == NULL || tree == NULL, 0)) + goto parse_dup_op_espace; + + tree = create_tree (dfa, tree, NULL, OP_ALT); + if (BE (tree == NULL, 0)) + goto parse_dup_op_espace; + } if (old_tree) tree = create_tree (dfa, old_tree, tree, CONCAT); @@ -2619,6 +2687,19 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, #define BRACKET_NAME_BUF_SIZE 32 #ifndef _LIBC + +# ifdef RE_ENABLE_I18N +/* Convert the byte B to the corresponding wide character. In a + unibyte locale, treat B as itself if it is an encoding error. + In a multibyte locale, return WEOF if B is an encoding error. */ +static wint_t +parse_byte (unsigned char b, re_charset_t *mbcset) +{ + wint_t wc = __btowc (b); + return wc == WEOF && !mbcset ? b : wc; +} +#endif + /* Local function for parse_bracket_exp only used in case of NOT _LIBC. Build the range expression which starts from START_ELEM, and ends at END_ELEM. The result are written to MBCSET and SBCSET. @@ -2628,11 +2709,17 @@ parse_dup_op (bin_tree_t *elem, re_string_t *regexp, re_dfa_t *dfa, static reg_errcode_t # ifdef RE_ENABLE_I18N -build_range_exp (bitset_t sbcset, re_charset_t *mbcset, int *range_alloc, - bracket_elem_t *start_elem, bracket_elem_t *end_elem) +build_range_exp (const reg_syntax_t syntax, + bitset_t sbcset, + re_charset_t *mbcset, + Idx *range_alloc, + const bracket_elem_t *start_elem, + const bracket_elem_t *end_elem) # else /* not RE_ENABLE_I18N */ -build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, - bracket_elem_t *end_elem) +build_range_exp (const reg_syntax_t syntax, + bitset_t sbcset, + const bracket_elem_t *start_elem, + const bracket_elem_t *end_elem) # endif /* not RE_ENABLE_I18N */ { unsigned int start_ch, end_ch; @@ -2655,7 +2742,6 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, wchar_t wc; wint_t start_wc; wint_t end_wc; - wchar_t cmp_buf[6] = {L'\0', L'\0', L'\0', L'\0', L'\0', L'\0'}; start_ch = ((start_elem->type == SB_CHAR) ? start_elem->opr.ch : ((start_elem->type == COLL_SYM) ? start_elem->opr.name[0] @@ -2664,14 +2750,12 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, : ((end_elem->type == COLL_SYM) ? end_elem->opr.name[0] : 0)); start_wc = ((start_elem->type == SB_CHAR || start_elem->type == COLL_SYM) - ? __btowc (start_ch) : start_elem->opr.wch); + ? parse_byte (start_ch, mbcset) : start_elem->opr.wch); end_wc = ((end_elem->type == SB_CHAR || end_elem->type == COLL_SYM) - ? __btowc (end_ch) : end_elem->opr.wch); + ? parse_byte (end_ch, mbcset) : end_elem->opr.wch); if (start_wc == WEOF || end_wc == WEOF) return REG_ECOLLATE; - cmp_buf[0] = start_wc; - cmp_buf[4] = end_wc; - if (__wcscoll (cmp_buf, cmp_buf + 4) > 0) + else if (BE ((syntax & RE_NO_EMPTY_RANGES) && start_wc > end_wc, 0)) return REG_ERANGE; /* Got valid collation sequence values, add them as a new entry. @@ -2686,7 +2770,7 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, { /* There is not enough space, need realloc. */ wchar_t *new_array_start, *new_array_end; - int new_nranges; + Idx new_nranges; /* +1 in case of mbcset->nranges is 0. */ new_nranges = 2 * mbcset->nranges + 1; @@ -2698,7 +2782,11 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, new_nranges); if (BE (new_array_start == NULL || new_array_end == NULL, 0)) - return REG_ESPACE; + { + re_free (new_array_start); + re_free (new_array_end); + return REG_ESPACE; + } mbcset->range_starts = new_array_start; mbcset->range_ends = new_array_end; @@ -2712,9 +2800,7 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, /* Build the table for single byte characters. */ for (wc = 0; wc < SBC_MAX; ++wc) { - cmp_buf[2] = wc; - if (__wcscoll (cmp_buf, cmp_buf + 2) <= 0 - && __wcscoll (cmp_buf + 2, cmp_buf + 4) <= 0) + if (start_wc <= wc && wc <= end_wc) bitset_set (sbcset, wc); } } @@ -2749,7 +2835,7 @@ build_range_exp (bitset_t sbcset, bracket_elem_t *start_elem, static reg_errcode_t # ifdef RE_ENABLE_I18N build_collating_symbol (bitset_t sbcset, re_charset_t *mbcset, - int *coll_sym_alloc, const unsigned char *name) + Idx *coll_sym_alloc, const unsigned char *name) # else /* not RE_ENABLE_I18N */ build_collating_symbol (bitset_t sbcset, const unsigned char *name) # endif /* not RE_ENABLE_I18N */ @@ -2895,6 +2981,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, 0)) return REG_ERANGE; + /* FIXME: Implement rational ranges here, too. */ start_collseq = lookup_collation_sequence_value (start_elem); end_collseq = lookup_collation_sequence_value (end_elem); /* Check start/end collation sequence values. */ @@ -2915,7 +3002,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, /* There is not enough space, need realloc. */ uint32_t *new_array_start; uint32_t *new_array_end; - int new_nranges; + Idx new_nranges; /* +1 in case of mbcset->nranges is 0. */ new_nranges = 2 * mbcset->nranges + 1; @@ -2962,7 +3049,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, auto inline reg_errcode_t __attribute__ ((always_inline)) build_collating_symbol (bitset_t sbcset, re_charset_t *mbcset, - int *coll_sym_alloc, const unsigned char *name) + Idx *coll_sym_alloc, const unsigned char *name) { int32_t elem, idx; size_t name_len = strlen ((const char *) name); @@ -2992,7 +3079,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, { /* Not enough, realloc it. */ /* +1 in case of mbcset->ncoll_syms is 0. */ - int new_coll_sym_alloc = 2 * mbcset->ncoll_syms + 1; + Idx new_coll_sym_alloc = 2 * mbcset->ncoll_syms + 1; /* Use realloc since mbcset->coll_syms is NULL if *alloc == 0. */ int32_t *new_coll_syms = re_realloc (mbcset->coll_syms, int32_t, @@ -3022,13 +3109,13 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, re_bitset_ptr_t sbcset; #ifdef RE_ENABLE_I18N re_charset_t *mbcset; - int coll_sym_alloc = 0, range_alloc = 0, mbchar_alloc = 0; - int equiv_class_alloc = 0, char_class_alloc = 0; + Idx coll_sym_alloc = 0, range_alloc = 0, mbchar_alloc = 0; + Idx equiv_class_alloc = 0, char_class_alloc = 0; #endif /* not RE_ENABLE_I18N */ - int non_match = 0; + bool non_match = false; bin_tree_t *work_tree; int token_len; - int first_round = 1; + bool first_round = true; #ifdef _LIBC collseqmb = (const unsigned char *) _NL_CURRENT (LC_COLLATE, _NL_COLLATE_COLLSEQMB); @@ -3075,7 +3162,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, #ifdef RE_ENABLE_I18N mbcset->non_match = 1; #endif /* not RE_ENABLE_I18N */ - non_match = 1; + non_match = true; if (syntax & RE_HAT_LISTS_NOT_NEWLINE) bitset_set (sbcset, '\n'); re_string_skip_bytes (regexp, token_len); /* Skip a token. */ @@ -3097,7 +3184,8 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, unsigned char start_name_buf[BRACKET_NAME_BUF_SIZE]; unsigned char end_name_buf[BRACKET_NAME_BUF_SIZE]; reg_errcode_t ret; - int token_len2 = 0, is_range_exp = 0; + int token_len2 = 0; + bool is_range_exp = false; re_token_t token2; start_elem.opr.name = start_name_buf; @@ -3109,7 +3197,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, *err = ret; goto parse_bracket_exp_free_return; } - first_round = 0; + first_round = false; /* Get information about the next token. We need it in any case. */ token_len = peek_token_bracket (token, regexp, syntax); @@ -3138,16 +3226,16 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, token->type = CHARACTER; } else - is_range_exp = 1; + is_range_exp = true; } } - if (is_range_exp == 1) + if (is_range_exp == true) { end_elem.opr.name = end_name_buf; end_elem.type = COLL_SYM; ret = parse_bracket_element (&end_elem, regexp, &token2, token_len2, - dfa, syntax, 1); + dfa, syntax, true); if (BE (ret != REG_NOERROR, 0)) { *err = ret; @@ -3161,11 +3249,11 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, &start_elem, &end_elem); #else # ifdef RE_ENABLE_I18N - *err = build_range_exp (sbcset, + *err = build_range_exp (syntax, sbcset, dfa->mb_cur_max > 1 ? mbcset : NULL, &range_alloc, &start_elem, &end_elem); # else - *err = build_range_exp (sbcset, &start_elem, &end_elem); + *err = build_range_exp (syntax, sbcset, &start_elem, &end_elem); # endif #endif /* RE_ENABLE_I18N */ if (BE (*err != REG_NOERROR, 0)) @@ -3220,7 +3308,8 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, #ifdef RE_ENABLE_I18N mbcset, &char_class_alloc, #endif /* RE_ENABLE_I18N */ - start_elem.opr.name, syntax); + (const char *) start_elem.opr.name, + syntax); if (BE (*err != REG_NOERROR, 0)) goto parse_bracket_exp_free_return; break; @@ -3317,7 +3406,7 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token, static reg_errcode_t parse_bracket_element (bracket_elem_t *elem, re_string_t *regexp, re_token_t *token, int token_len, re_dfa_t *dfa, - reg_syntax_t syntax, int accept_hyphen) + reg_syntax_t syntax, bool accept_hyphen) { #ifdef RE_ENABLE_I18N int cur_char_size; @@ -3404,7 +3493,7 @@ parse_bracket_symbol (bracket_elem_t *elem, re_string_t *regexp, static reg_errcode_t #ifdef RE_ENABLE_I18N build_equiv_class (bitset_t sbcset, re_charset_t *mbcset, - int *equiv_class_alloc, const unsigned char *name) + Idx *equiv_class_alloc, const unsigned char *name) #else /* not RE_ENABLE_I18N */ build_equiv_class (bitset_t sbcset, const unsigned char *name) #endif /* not RE_ENABLE_I18N */ @@ -3466,7 +3555,7 @@ build_equiv_class (bitset_t sbcset, const unsigned char *name) { /* Not enough, realloc it. */ /* +1 in case of mbcset->nequiv_classes is 0. */ - int new_equiv_class_alloc = 2 * mbcset->nequiv_classes + 1; + Idx new_equiv_class_alloc = 2 * mbcset->nequiv_classes + 1; /* Use realloc since the array is NULL if *alloc == 0. */ int32_t *new_equiv_classes = re_realloc (mbcset->equiv_classes, int32_t, @@ -3497,15 +3586,15 @@ build_equiv_class (bitset_t sbcset, const unsigned char *name) static reg_errcode_t #ifdef RE_ENABLE_I18N build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, - re_charset_t *mbcset, int *char_class_alloc, - const unsigned char *class_name, reg_syntax_t syntax) + re_charset_t *mbcset, Idx *char_class_alloc, + const char *class_name, reg_syntax_t syntax) #else /* not RE_ENABLE_I18N */ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, - const unsigned char *class_name, reg_syntax_t syntax) + const char *class_name, reg_syntax_t syntax) #endif /* not RE_ENABLE_I18N */ { int i; - const char *name = (const char *) class_name; + const char *name = class_name; /* In case of REG_ICASE "upper" and "lower" match the both of upper and lower cases. */ @@ -3519,7 +3608,7 @@ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, { /* Not enough, realloc it. */ /* +1 in case of mbcset->nchar_classes is 0. */ - int new_char_class_alloc = 2 * mbcset->nchar_classes + 1; + Idx new_char_class_alloc = 2 * mbcset->nchar_classes + 1; /* Use realloc since array is NULL if *alloc == 0. */ wctype_t *new_char_classes = re_realloc (mbcset->char_classes, wctype_t, new_char_class_alloc); @@ -3536,13 +3625,13 @@ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, if (BE (trans != NULL, 0)) \ { \ for (i = 0; i < SBC_MAX; ++i) \ - if (ctype_func (i)) \ + if (ctype_func (i)) \ bitset_set (sbcset, trans[i]); \ } \ else \ { \ for (i = 0; i < SBC_MAX; ++i) \ - if (ctype_func (i)) \ + if (ctype_func (i)) \ bitset_set (sbcset, i); \ } \ } while (0) @@ -3579,40 +3668,35 @@ build_charclass (RE_TRANSLATE_TYPE trans, bitset_t sbcset, static bin_tree_t * build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, - const unsigned char *class_name, - const unsigned char *extra, int non_match, + const char *class_name, + const char *extra, bool non_match, reg_errcode_t *err) { re_bitset_ptr_t sbcset; #ifdef RE_ENABLE_I18N re_charset_t *mbcset; - int alloc = 0; + Idx alloc = 0; #endif /* not RE_ENABLE_I18N */ reg_errcode_t ret; re_token_t br_token; bin_tree_t *tree; sbcset = (re_bitset_ptr_t) calloc (sizeof (bitset_t), 1); -#ifdef RE_ENABLE_I18N - mbcset = (re_charset_t *) calloc (sizeof (re_charset_t), 1); -#endif /* RE_ENABLE_I18N */ - -#ifdef RE_ENABLE_I18N - if (BE (sbcset == NULL || mbcset == NULL, 0)) -#else /* not RE_ENABLE_I18N */ if (BE (sbcset == NULL, 0)) -#endif /* not RE_ENABLE_I18N */ { *err = REG_ESPACE; return NULL; } - - if (non_match) - { #ifdef RE_ENABLE_I18N - mbcset->non_match = 1; -#endif /* not RE_ENABLE_I18N */ + mbcset = (re_charset_t *) calloc (sizeof (re_charset_t), 1); + if (BE (mbcset == NULL, 0)) + { + re_free (sbcset); + *err = REG_ESPACE; + return NULL; } + mbcset->non_match = non_match; +#endif /* RE_ENABLE_I18N */ /* We don't care the syntax in this case. */ ret = build_charclass (trans, sbcset, @@ -3645,6 +3729,9 @@ build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, #endif /* Build a tree for simple bracket. */ +#if defined GCC_LINT || defined lint + memset (&br_token, 0, sizeof br_token); +#endif br_token.type = SIMPLE_BRACKET; br_token.opr.sbcset = sbcset; tree = create_token_tree (dfa, NULL, NULL, &br_token); @@ -3686,14 +3773,15 @@ build_charclass_op (re_dfa_t *dfa, RE_TRANSLATE_TYPE trans, } /* This is intended for the expressions like "a{1,3}". - Fetch a number from `input', and return the number. - Return -1, if the number field is empty like "{,1}". - Return -2, If an error is occured. */ + Fetch a number from 'input', and return the number. + Return -1 if the number field is empty like "{,1}". + Return RE_DUP_MAX + 1 if the number field is too large. + Return -2 if an error occurred. */ -static int +static Idx fetch_number (re_string_t *input, re_token_t *token, reg_syntax_t syntax) { - int num = -1; + Idx num = -1; unsigned char c; while (1) { @@ -3704,8 +3792,10 @@ fetch_number (re_string_t *input, re_token_t *token, reg_syntax_t syntax) if (token->type == OP_CLOSE_DUP_NUM || c == ',') break; num = ((token->type != CHARACTER || c < '0' || '9' < c || num == -2) - ? -2 : ((num == -1) ? c - '0' : num * 10 + c - '0')); - num = (num > RE_DUP_MAX) ? -2 : num; + ? -2 + : num == -1 + ? c - '0' + : MIN (RE_DUP_MAX + 1, num * 10 + c - '0')); } return num; } @@ -3735,6 +3825,9 @@ create_tree (re_dfa_t *dfa, bin_tree_t *left, bin_tree_t *right, re_token_type_t type) { re_token_t t; +#if defined GCC_LINT || defined lint + memset (&t, 0, sizeof t); +#endif t.type = type; return create_token_tree (dfa, left, right, &t); } @@ -3779,7 +3872,7 @@ create_token_tree (re_dfa_t *dfa, bin_tree_t *left, bin_tree_t *right, static reg_errcode_t mark_opt_subexp (void *extra, bin_tree_t *node) { - int idx = (int) (long) extra; + Idx idx = (uintptr_t) extra; if (node->token.type == SUBEXP && node->token.opr.idx == idx) node->token.opt_subexp = 1; diff --git a/posix/regex.c b/posix/regex.c index 68b5ef1..cfadc6b 100644 --- a/posix/regex.c +++ b/posix/regex.c @@ -17,12 +17,20 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#ifdef HAVE_CONFIG_H -#include "config.h" +#ifndef _LIBC +# include <config.h> + +# if (__GNUC__ == 4 && 6 <= __GNUC_MINOR__) || 4 < __GNUC__ +# pragma GCC diagnostic ignored "-Wsuggest-attribute=pure" +# endif +# if (__GNUC__ == 4 && 3 <= __GNUC_MINOR__) || 4 < __GNUC__ +# pragma GCC diagnostic ignored "-Wold-style-definition" +# pragma GCC diagnostic ignored "-Wtype-limits" +# endif #endif -/* Make sure noone compiles this code with a C++ compiler. */ -#ifdef __cplusplus +/* Make sure no one compiles this code with a C++ compiler. */ +#if defined __cplusplus && defined _LIBC # error "This is C code, use a C compiler" #endif @@ -56,9 +64,6 @@ #undefs RE_DUP_MAX and sets it to the right value. */ #include <limits.h> -/* This header defines the MIN and MAX macros. */ -#include <sys/param.h> - #include <regex.h> #include "regex_internal.h" diff --git a/posix/regex.h b/posix/regex.h index e0b8915..eef2629 100644 --- a/posix/regex.h +++ b/posix/regex.h @@ -27,6 +27,36 @@ extern "C" { #endif +/* Define __USE_GNU to declare GNU extensions that violate the + POSIX name space rules. */ +#ifdef _GNU_SOURCE +# define __USE_GNU 1 +#endif + +#ifdef _REGEX_LARGE_OFFSETS + +/* Use types and values that are wide enough to represent signed and + unsigned byte offsets in memory. This currently works only when + the regex code is used outside of the GNU C library; it is not yet + supported within glibc itself, and glibc users should not define + _REGEX_LARGE_OFFSETS. */ + +/* The type of object sizes. */ +typedef size_t __re_size_t; + +/* The type of object sizes, in places where the traditional code + uses unsigned long int. */ +typedef size_t __re_long_size_t; + +#else + +/* The traditional GNU regex implementation mishandles strings longer + than INT_MAX. */ +typedef unsigned int __re_size_t; +typedef unsigned long int __re_long_size_t; + +#endif + /* The following two types have to be signed and unsigned integer type wide enough to hold a value of a pointer. For most ANSI compilers ptrdiff_t and size_t should be likely OK. Still size of these two @@ -108,9 +138,9 @@ typedef unsigned long int reg_syntax_t; If not set, newline is literal. */ # define RE_NEWLINE_ALT (RE_LIMITED_OPS << 1) -/* If this bit is set, then `{...}' defines an interval, and \{ and \} +/* If this bit is set, then '{...}' defines an interval, and \{ and \} are literals. - If not set, then `\{...\}' defines an interval. */ + If not set, then '\{...\}' defines an interval. */ # define RE_NO_BK_BRACES (RE_NEWLINE_ALT << 1) /* If this bit is set, (...) defines a group, and \( and \) are literals. @@ -165,8 +195,8 @@ typedef unsigned long int reg_syntax_t; whether ^ should be special. */ # define RE_CARET_ANCHORS_HERE (RE_ICASE << 1) -/* If this bit is set, then \{ cannot be first in an bre or - immediately after an alternation or begin-group operator. */ +/* If this bit is set, then \{ cannot be first in a regex or + immediately after an alternation, open-group or \} operator. */ # define RE_CONTEXT_INVALID_DUP (RE_CARET_ANCHORS_HERE << 1) /* If this bit is set, then no_sub will be set to 1 during @@ -185,9 +215,9 @@ extern reg_syntax_t re_syntax_options; (The [[[ comments delimit what gets put into the Texinfo file, so don't delete them!) */ /* [[[begin syntaxes]]] */ -#define RE_SYNTAX_EMACS 0 +# define RE_SYNTAX_EMACS 0 -#define RE_SYNTAX_AWK \ +# define RE_SYNTAX_AWK \ (RE_BACKSLASH_ESCAPE_IN_LISTS | RE_DOT_NOT_NULL \ | RE_NO_BK_PARENS | RE_NO_BK_REFS \ | RE_NO_BK_VBAR | RE_NO_EMPTY_RANGES \ @@ -195,52 +225,49 @@ extern reg_syntax_t re_syntax_options; | RE_CHAR_CLASSES \ | RE_UNMATCHED_RIGHT_PAREN_ORD | RE_NO_GNU_OPS) -#define RE_SYNTAX_GNU_AWK \ +# define RE_SYNTAX_GNU_AWK \ ((RE_SYNTAX_POSIX_EXTENDED | RE_BACKSLASH_ESCAPE_IN_LISTS \ | RE_INVALID_INTERVAL_ORD) \ & ~(RE_DOT_NOT_NULL | RE_CONTEXT_INDEP_OPS \ | RE_CONTEXT_INVALID_OPS )) -#define RE_SYNTAX_POSIX_AWK \ +# define RE_SYNTAX_POSIX_AWK \ (RE_SYNTAX_POSIX_EXTENDED | RE_BACKSLASH_ESCAPE_IN_LISTS \ | RE_INTERVALS | RE_NO_GNU_OPS \ | RE_INVALID_INTERVAL_ORD) -#define RE_SYNTAX_GREP \ - (RE_BK_PLUS_QM | RE_CHAR_CLASSES \ - | RE_HAT_LISTS_NOT_NEWLINE | RE_INTERVALS \ - | RE_NEWLINE_ALT) +# define RE_SYNTAX_GREP \ + ((RE_SYNTAX_POSIX_BASIC | RE_NEWLINE_ALT) \ + & ~(RE_CONTEXT_INVALID_DUP | RE_DOT_NOT_NULL)) -#define RE_SYNTAX_EGREP \ - (RE_CHAR_CLASSES | RE_CONTEXT_INDEP_ANCHORS \ - | RE_CONTEXT_INDEP_OPS | RE_HAT_LISTS_NOT_NEWLINE \ - | RE_NEWLINE_ALT | RE_NO_BK_PARENS \ - | RE_NO_BK_VBAR) +# define RE_SYNTAX_EGREP \ + ((RE_SYNTAX_POSIX_EXTENDED | RE_INVALID_INTERVAL_ORD | RE_NEWLINE_ALT) \ + & ~(RE_CONTEXT_INVALID_OPS | RE_DOT_NOT_NULL)) -#define RE_SYNTAX_POSIX_EGREP \ - (RE_SYNTAX_EGREP | RE_INTERVALS | RE_NO_BK_BRACES \ - | RE_INVALID_INTERVAL_ORD) +/* POSIX grep -E behavior is no longer incompatible with GNU. */ +# define RE_SYNTAX_POSIX_EGREP \ + RE_SYNTAX_EGREP /* P1003.2/D11.2, section 4.20.7.1, lines 5078ff. */ -#define RE_SYNTAX_ED RE_SYNTAX_POSIX_BASIC +# define RE_SYNTAX_ED RE_SYNTAX_POSIX_BASIC -#define RE_SYNTAX_SED RE_SYNTAX_POSIX_BASIC +# define RE_SYNTAX_SED RE_SYNTAX_POSIX_BASIC /* Syntax bits common to both basic and extended POSIX regex syntax. */ -#define _RE_SYNTAX_POSIX_COMMON \ +# define _RE_SYNTAX_POSIX_COMMON \ (RE_CHAR_CLASSES | RE_DOT_NEWLINE | RE_DOT_NOT_NULL \ | RE_INTERVALS | RE_NO_EMPTY_RANGES) -#define RE_SYNTAX_POSIX_BASIC \ +# define RE_SYNTAX_POSIX_BASIC \ (_RE_SYNTAX_POSIX_COMMON | RE_BK_PLUS_QM | RE_CONTEXT_INVALID_DUP) /* Differs from ..._POSIX_BASIC only in that RE_BK_PLUS_QM becomes RE_LIMITED_OPS, i.e., \? \+ \| are not recognized. Actually, this isn't minimal, since other operators, such as \`, aren't disabled. */ -#define RE_SYNTAX_POSIX_MINIMAL_BASIC \ +# define RE_SYNTAX_POSIX_MINIMAL_BASIC \ (_RE_SYNTAX_POSIX_COMMON | RE_LIMITED_OPS) -#define RE_SYNTAX_POSIX_EXTENDED \ +# define RE_SYNTAX_POSIX_EXTENDED \ (_RE_SYNTAX_POSIX_COMMON | RE_CONTEXT_INDEP_ANCHORS \ | RE_CONTEXT_INDEP_OPS | RE_NO_BK_BRACES \ | RE_NO_BK_PARENS | RE_NO_BK_VBAR \ @@ -248,25 +275,35 @@ extern reg_syntax_t re_syntax_options; /* Differs from ..._POSIX_EXTENDED in that RE_CONTEXT_INDEP_OPS is removed and RE_NO_BK_REFS is added. */ -#define RE_SYNTAX_POSIX_MINIMAL_EXTENDED \ +# define RE_SYNTAX_POSIX_MINIMAL_EXTENDED \ (_RE_SYNTAX_POSIX_COMMON | RE_CONTEXT_INDEP_ANCHORS \ | RE_CONTEXT_INVALID_OPS | RE_NO_BK_BRACES \ | RE_NO_BK_PARENS | RE_NO_BK_REFS \ | RE_NO_BK_VBAR | RE_UNMATCHED_RIGHT_PAREN_ORD) /* [[[end syntaxes]]] */ - -/* Maximum number of duplicates an interval can allow. Some systems - (erroneously) define this in other header files, but we want our + +/* Maximum number of duplicates an interval can allow. POSIX-conforming + systems might define this in <limits.h>, but we want our value, so remove any previous define. */ +# ifdef _REGEX_INCLUDE_LIMITS_H +# include <limits.h> +# endif # ifdef RE_DUP_MAX # undef RE_DUP_MAX # endif -/* If sizeof(int) == 2, then ((1 << 15) - 1) overflows. */ + +/* RE_DUP_MAX is 2**15 - 1 because an earlier implementation stored + the counter as a 2-byte signed integer. This is no longer true, so + RE_DUP_MAX could be increased to (INT_MAX / 10 - 1), or to + ((SIZE_MAX - 9) / 10) if _REGEX_LARGE_OFFSETS is defined. + However, there would be a huge performance problem if someone + actually used a pattern like a\{214748363\}, so RE_DUP_MAX retains + its historical value. */ # define RE_DUP_MAX (0x7fff) #endif -/* POSIX `cflags' bits (i.e., information for `regcomp'). */ +/* POSIX 'cflags' bits (i.e., information for 'regcomp'). */ /* If this bit is set, then use extended regular expression syntax. If not set, then use basic regular expression syntax. */ @@ -274,19 +311,19 @@ extern reg_syntax_t re_syntax_options; /* If this bit is set, then ignore case when matching. If not set, then case is significant. */ -#define REG_ICASE (REG_EXTENDED << 1) +#define REG_ICASE (1 << 1) /* If this bit is set, then anchors do not match at newline characters in the string. If not set, then anchors do match at newlines. */ -#define REG_NEWLINE (REG_ICASE << 1) +#define REG_NEWLINE (1 << 2) /* If this bit is set, then report only success or fail in regexec. If not set, then returns differ between not matching and errors. */ -#define REG_NOSUB (REG_NEWLINE << 1) +#define REG_NOSUB (1 << 3) -/* POSIX `eflags' bits (i.e., information for regexec). */ +/* POSIX 'eflags' bits (i.e., information for regexec). */ /* If this bit is set, then the beginning-of-line operator doesn't match the beginning of the string (presumably because it's not the @@ -304,41 +341,60 @@ extern reg_syntax_t re_syntax_options; /* If any error codes are removed, changed, or added, update the - `re_error_msg' table in regex.c. */ + '__re_error_msgid' table in regcomp.c. */ + typedef enum { -#if defined _XOPEN_SOURCE || defined __USE_XOPEN2K - REG_ENOSYS = -1, /* This will never happen for this implementation. */ -#endif - - REG_NOERROR = 0, /* Success. */ - REG_NOMATCH, /* Didn't find a match (for regexec). */ + _REG_ENOSYS = -1, /* This will never happen for this implementation. */ + _REG_NOERROR = 0, /* Success. */ + _REG_NOMATCH, /* Didn't find a match (for regexec). */ /* POSIX regcomp return error codes. (In the order listed in the standard.) */ - REG_BADPAT, /* Invalid pattern. */ - REG_ECOLLATE, /* Invalid collating element. */ - REG_ECTYPE, /* Invalid character class name. */ - REG_EESCAPE, /* Trailing backslash. */ - REG_ESUBREG, /* Invalid back reference. */ - REG_EBRACK, /* Unmatched left bracket. */ - REG_EPAREN, /* Parenthesis imbalance. */ - REG_EBRACE, /* Unmatched \{. */ - REG_BADBR, /* Invalid contents of \{\}. */ - REG_ERANGE, /* Invalid range end. */ - REG_ESPACE, /* Ran out of memory. */ - REG_BADRPT, /* No preceding re for repetition op. */ + _REG_BADPAT, /* Invalid pattern. */ + _REG_ECOLLATE, /* Invalid collating element. */ + _REG_ECTYPE, /* Invalid character class name. */ + _REG_EESCAPE, /* Trailing backslash. */ + _REG_ESUBREG, /* Invalid back reference. */ + _REG_EBRACK, /* Unmatched left bracket. */ + _REG_EPAREN, /* Parenthesis imbalance. */ + _REG_EBRACE, /* Unmatched \{. */ + _REG_BADBR, /* Invalid contents of \{\}. */ + _REG_ERANGE, /* Invalid range end. */ + _REG_ESPACE, /* Ran out of memory. */ + _REG_BADRPT, /* No preceding re for repetition op. */ /* Error codes we've added. */ - REG_EEND, /* Premature end. */ - REG_ESIZE, /* Compiled pattern bigger than 2^16 bytes. */ - REG_ERPAREN /* Unmatched ) or \); not returned from regcomp. */ + _REG_EEND, /* Premature end. */ + _REG_ESIZE, /* Too large (e.g., repeat count too large). */ + _REG_ERPAREN /* Unmatched ) or \); not returned from regcomp. */ } reg_errcode_t; + +#if defined _XOPEN_SOURCE || defined __USE_XOPEN2K +# define REG_ENOSYS _REG_ENOSYS +#endif +#define REG_NOERROR _REG_NOERROR +#define REG_NOMATCH _REG_NOMATCH +#define REG_BADPAT _REG_BADPAT +#define REG_ECOLLATE _REG_ECOLLATE +#define REG_ECTYPE _REG_ECTYPE +#define REG_EESCAPE _REG_EESCAPE +#define REG_ESUBREG _REG_ESUBREG +#define REG_EBRACK _REG_EBRACK +#define REG_EPAREN _REG_EPAREN +#define REG_EBRACE _REG_EBRACE +#define REG_BADBR _REG_BADBR +#define REG_ERANGE _REG_ERANGE +#define REG_ESPACE _REG_ESPACE +#define REG_BADRPT _REG_BADRPT +#define REG_EEND _REG_EEND +#define REG_ESIZE _REG_ESIZE +#define REG_ERPAREN _REG_ERPAREN /* This data structure represents a compiled pattern. Before calling - the pattern compiler, the fields `buffer', `allocated', `fastmap', - and `translate' can be set. After the pattern has been compiled, - the fields `re_nsub', `not_bol' and `not_eol' are available. All + the pattern compiler, the fields 'buffer', 'allocated', 'fastmap', + and 'translate' can be set. After the pattern has been compiled, + the fields 're_nsub', 'not_bol' and 'not_eol' are available. All other fields are private to the regex routines. */ #ifndef RE_TRANSLATE_TYPE @@ -356,16 +412,15 @@ typedef enum struct re_pattern_buffer { - /* Space that holds the compiled pattern. It is declared as - `unsigned char *' because its elements are sometimes used as - array indexes. */ - unsigned char *__REPB_PREFIX(buffer); + /* Space that holds the compiled pattern. The type + 'struct re_dfa_t' is private and is not declared here. */ + struct re_dfa_t *__REPB_PREFIX(buffer); - /* Number of bytes to which `buffer' points. */ - unsigned long int __REPB_PREFIX(allocated); + /* Number of bytes to which 'buffer' points. */ + __re_long_size_t __REPB_PREFIX(allocated); - /* Number of bytes actually used in `buffer'. */ - unsigned long int __REPB_PREFIX(used); + /* Number of bytes actually used in 'buffer'. */ + __re_long_size_t __REPB_PREFIX(used); /* Syntax setting with which the pattern was compiled. */ reg_syntax_t __REPB_PREFIX(syntax); @@ -385,13 +440,13 @@ struct re_pattern_buffer size_t re_nsub; /* Zero if this pattern cannot match the empty string, one else. - Well, in truth it's used only in `re_search_2', to see whether or + Well, in truth it's used only in 're_search_2', to see whether or not we should use the fastmap, so we don't set this absolutely - perfectly; see `re_compile_fastmap' (the `duplicate' case). */ + perfectly; see 're_compile_fastmap' (the "duplicate" case). */ unsigned __REPB_PREFIX(can_be_null) : 1; - /* If REGS_UNALLOCATED, allocate space in the `regs' structure - for `max (RE_NREGS, re_nsub + 1)' groups. + /* If REGS_UNALLOCATED, allocate space in the 'regs' structure + for 'max (RE_NREGS, re_nsub + 1)' groups. If REGS_REALLOCATE, reallocate space if necessary. If REGS_FIXED, use what's there. */ #ifdef __USE_GNU @@ -401,11 +456,11 @@ struct re_pattern_buffer #endif unsigned __REPB_PREFIX(regs_allocated) : 2; - /* Set to zero when `regex_compile' compiles a pattern; set to one - by `re_compile_fastmap' if it updates the fastmap. */ + /* Set to zero when 're_compile_pattern' compiles a pattern; set to + one by 're_compile_fastmap' if it updates the fastmap. */ unsigned __REPB_PREFIX(fastmap_accurate) : 1; - /* If set, `re_match_2' does not return information about + /* If set, 're_match_2' does not return information about subexpressions. */ unsigned __REPB_PREFIX(no_sub) : 1; @@ -423,7 +478,17 @@ struct re_pattern_buffer typedef struct re_pattern_buffer regex_t; /* Type for byte offsets within the string. POSIX mandates this. */ +#ifdef _REGEX_LARGE_OFFSETS +/* POSIX 1003.1-2008 requires that regoff_t be at least as wide as + ptrdiff_t and ssize_t. We don't know of any hosts where ptrdiff_t + is wider than ssize_t, so ssize_t is safe. ptrdiff_t is not + visible here, so use ssize_t. */ +typedef ssize_t regoff_t; +#else +/* The traditional GNU regex implementation mishandles strings longer + than INT_MAX. */ typedef int regoff_t; +#endif #ifdef __USE_GNU @@ -431,15 +496,15 @@ typedef int regoff_t; regex.texinfo for a full description of what registers match. */ struct re_registers { - unsigned num_regs; + __re_size_t num_regs; regoff_t *start; regoff_t *end; }; -/* If `regs_allocated' is REGS_UNALLOCATED in the pattern buffer, - `re_match_2' returns information about at least this many registers - the first time a `regs' structure is passed. */ +/* If 'regs_allocated' is REGS_UNALLOCATED in the pattern buffer, + 're_match_2' returns information about at least this many registers + the first time a 'regs' structure is passed. */ # ifndef RE_NREGS # define RE_NREGS 30 # endif @@ -447,7 +512,7 @@ struct re_registers /* POSIX specification for registers. Aside from the different names than - `re_registers', POSIX uses an array of structures, instead of a + 're_registers', POSIX uses an array of structures, instead of a structure of arrays. */ typedef struct { @@ -459,17 +524,17 @@ typedef struct #ifdef __USE_GNU /* Sets the current default syntax to SYNTAX, and return the old syntax. - You can also simply assign to the `re_syntax_options' variable. */ + You can also simply assign to the 're_syntax_options' variable. */ extern reg_syntax_t re_set_syntax (reg_syntax_t __syntax); /* Compile the regular expression PATTERN, with length LENGTH - and syntax given by the global `re_syntax_options', into the buffer + and syntax given by the global 're_syntax_options', into the buffer BUFFER. Return NULL if successful, and an error string if not. - To free the allocated storage, you must call `regfree' on BUFFER. - Note that the translate table must either have been initialised by - `regcomp', with a malloc'ed value, or set to NULL before calling - `regfree'. */ + To free the allocated storage, you must call 'regfree' on BUFFER. + Note that the translate table must either have been initialized by + 'regcomp', with a malloc'ed value, or set to NULL before calling + 'regfree'. */ extern const char *re_compile_pattern (const char *__pattern, size_t __length, struct re_pattern_buffer *__buffer); @@ -485,47 +550,52 @@ extern int re_compile_fastmap (struct re_pattern_buffer *__buffer); characters. Return the starting position of the match, -1 for no match, or -2 for an internal error. Also return register information in REGS (if REGS and BUFFER->no_sub are nonzero). */ -extern int re_search (struct re_pattern_buffer *__buffer, const char *__string, - int __length, int __start, int __range, - struct re_registers *__regs); +extern regoff_t re_search (struct re_pattern_buffer *__buffer, + const char *__String, regoff_t __length, + regoff_t __start, regoff_t __range, + struct re_registers *__regs); -/* Like `re_search', but search in the concatenation of STRING1 and +/* Like 're_search', but search in the concatenation of STRING1 and STRING2. Also, stop searching at index START + STOP. */ -extern int re_search_2 (struct re_pattern_buffer *__buffer, - const char *__string1, int __length1, - const char *__string2, int __length2, int __start, - int __range, struct re_registers *__regs, int __stop); +extern regoff_t re_search_2 (struct re_pattern_buffer *__buffer, + const char *__string1, regoff_t __length1, + const char *__string2, regoff_t __length2, + regoff_t __start, regoff_t __range, + struct re_registers *__regs, + regoff_t __stop); -/* Like `re_search', but return how many characters in STRING the regexp +/* Like 're_search', but return how many characters in STRING the regexp in BUFFER matched, starting at position START. */ -extern int re_match (struct re_pattern_buffer *__buffer, const char *__string, - int __length, int __start, struct re_registers *__regs); +extern regoff_t re_match (struct re_pattern_buffer *__buffer, + const char *__String, regoff_t __length, + regoff_t __start, struct re_registers *__regs); -/* Relates to `re_match' as `re_search_2' relates to `re_search'. */ -extern int re_match_2 (struct re_pattern_buffer *__buffer, - const char *__string1, int __length1, - const char *__string2, int __length2, int __start, - struct re_registers *__regs, int __stop); +/* Relates to 're_match' as 're_search_2' relates to 're_search'. */ +extern regoff_t re_match_2 (struct re_pattern_buffer *__buffer, + const char *__string1, regoff_t __length1, + const char *__string2, regoff_t __length2, + regoff_t __start, struct re_registers *__regs, + regoff_t __stop); /* Set REGS to hold NUM_REGS registers, storing them in STARTS and ENDS. Subsequent matches using BUFFER and REGS will use this memory for recording register information. STARTS and ENDS must be - allocated with malloc, and must each be at least `NUM_REGS * sizeof + allocated with malloc, and must each be at least 'NUM_REGS * sizeof (regoff_t)' bytes long. If NUM_REGS == 0, then subsequent matches should allocate their own register data. Unless this function is called, the first search or match using - PATTERN_BUFFER will allocate its own register data, without + BUFFER will allocate its own register data, without freeing the old data. */ extern void re_set_registers (struct re_pattern_buffer *__buffer, struct re_registers *__regs, - unsigned int __num_regs, + __re_size_t __num_regs, regoff_t *__starts, regoff_t *__ends); #endif /* Use GNU */ @@ -538,38 +608,32 @@ extern int re_exec (const char *); #endif /* GCC 2.95 and later have "__restrict"; C99 compilers have - "restrict", and "configure" may have defined "restrict". */ -#ifndef __restrict -# if ! (2 < __GNUC__ || (2 == __GNUC__ && 95 <= __GNUC_MINOR__)) -# if defined restrict || 199901L <= __STDC_VERSION__ -# define __restrict restrict -# else -# define __restrict -# endif -# endif + "restrict", and "configure" may have defined "restrict". + Other compilers use __restrict, __restrict__, and _Restrict, and + 'configure' might #define 'restrict' to those words, so pick a + different name. */ +#ifndef _Restrict_ +# define _Restrict_ __restrict #endif -/* gcc 3.1 and up support the [restrict] syntax. */ -#ifndef __restrict_arr -# if (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 1)) \ - && !defined __GNUG__ -# define __restrict_arr __restrict -# else -# define __restrict_arr -# endif +/* gcc 3.1 and up support the [restrict] syntax. Don't trust + sys/cdefs.h's definition of __restrict_arr, though, as it + mishandles gcc -ansi -pedantic. */ +#ifndef _Restrict_arr_ +# define _Restrict_arr_ __restrict_arr #endif /* POSIX compatibility. */ -extern int regcomp (regex_t *__restrict __preg, - const char *__restrict __pattern, +extern int regcomp (regex_t *_Restrict_ __preg, + const char *_Restrict_ __pattern, int __cflags); -extern int regexec (const regex_t *__restrict __preg, - const char *__restrict __string, size_t __nmatch, - regmatch_t __pmatch[__restrict_arr], +extern int regexec (const regex_t *_Restrict_ __preg, + const char *_Restrict_ __String, size_t __nmatch, + regmatch_t __pmatch[_Restrict_arr_], int __eflags); -extern size_t regerror (int __errcode, const regex_t *__restrict __preg, - char *__restrict __errbuf, size_t __errbuf_size); +extern size_t regerror (int __errcode, const regex_t *_Restrict_ __preg, + char *_Restrict_ __errbuf, size_t __errbuf_size); extern void regfree (regex_t *__preg); diff --git a/posix/regex_internal.c b/posix/regex_internal.c index 9062083..2a0c015 100644 --- a/posix/regex_internal.c +++ b/posix/regex_internal.c @@ -17,17 +17,27 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -static void re_string_construct_common (const char *str, int len, +static void re_string_construct_common (const char *str, Idx len, re_string_t *pstr, - RE_TRANSLATE_TYPE trans, int icase, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa); static re_dfastate_t *create_ci_newstate (const re_dfa_t *dfa, const re_node_set *nodes, - unsigned int hash); + re_hashval_t hash); static re_dfastate_t *create_cd_newstate (const re_dfa_t *dfa, const re_node_set *nodes, unsigned int context, - unsigned int hash); + re_hashval_t hash); +static reg_errcode_t re_string_realloc_buffers (re_string_t *pstr, + Idx new_buf_len); +#ifdef RE_ENABLE_I18N +static void build_wcs_buffer (re_string_t *pstr); +static reg_errcode_t build_wcs_upper_buffer (re_string_t *pstr); +#endif /* RE_ENABLE_I18N */ +static void build_upper_buffer (re_string_t *pstr); +static void re_string_translate_buffer (re_string_t *pstr); +static unsigned int re_string_context_at (const re_string_t *input, Idx idx, + int eflags) __attribute__ ((pure)); /* Functions for string operation. */ @@ -36,11 +46,11 @@ static re_dfastate_t *create_cd_newstate (const re_dfa_t *dfa, static reg_errcode_t __attribute_warn_unused_result__ -re_string_allocate (re_string_t *pstr, const char *str, int len, int init_len, - RE_TRANSLATE_TYPE trans, int icase, const re_dfa_t *dfa) +re_string_allocate (re_string_t *pstr, const char *str, Idx len, Idx init_len, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa) { reg_errcode_t ret; - int init_buf_len; + Idx init_buf_len; /* Ensure at least one character fits into the buffers. */ if (init_len < dfa->mb_cur_max) @@ -64,8 +74,8 @@ re_string_allocate (re_string_t *pstr, const char *str, int len, int init_len, static reg_errcode_t __attribute_warn_unused_result__ -re_string_construct (re_string_t *pstr, const char *str, int len, - RE_TRANSLATE_TYPE trans, int icase, const re_dfa_t *dfa) +re_string_construct (re_string_t *pstr, const char *str, Idx len, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa) { reg_errcode_t ret; memset (pstr, '\0', sizeof (re_string_t)); @@ -127,7 +137,7 @@ re_string_construct (re_string_t *pstr, const char *str, int len, static reg_errcode_t __attribute_warn_unused_result__ -re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) +re_string_realloc_buffers (re_string_t *pstr, Idx new_buf_len) { #ifdef RE_ENABLE_I18N if (pstr->mb_cur_max > 1) @@ -135,8 +145,8 @@ re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) wint_t *new_wcs; /* Avoid overflow in realloc. */ - const size_t max_object_size = MAX (sizeof (wint_t), sizeof (int)); - if (BE (SIZE_MAX / max_object_size < new_buf_len, 0)) + const size_t max_object_size = MAX (sizeof (wint_t), sizeof (Idx)); + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) < new_buf_len, 0)) return REG_ESPACE; new_wcs = re_realloc (pstr->wcs, wint_t, new_buf_len); @@ -145,7 +155,7 @@ re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) pstr->wcs = new_wcs; if (pstr->offsets != NULL) { - int *new_offsets = re_realloc (pstr->offsets, int, new_buf_len); + Idx *new_offsets = re_realloc (pstr->offsets, Idx, new_buf_len); if (BE (new_offsets == NULL, 0)) return REG_ESPACE; pstr->offsets = new_offsets; @@ -166,15 +176,15 @@ re_string_realloc_buffers (re_string_t *pstr, int new_buf_len) static void -re_string_construct_common (const char *str, int len, re_string_t *pstr, - RE_TRANSLATE_TYPE trans, int icase, +re_string_construct_common (const char *str, Idx len, re_string_t *pstr, + RE_TRANSLATE_TYPE trans, bool icase, const re_dfa_t *dfa) { pstr->raw_mbs = (const unsigned char *) str; pstr->len = len; pstr->raw_len = len; pstr->trans = trans; - pstr->icase = icase ? 1 : 0; + pstr->icase = icase; pstr->mbs_allocated = (trans != NULL || icase); pstr->mb_cur_max = dfa->mb_cur_max; pstr->is_utf8 = dfa->is_utf8; @@ -206,7 +216,7 @@ build_wcs_buffer (re_string_t *pstr) unsigned char buf[64]; #endif mbstate_t prev_st; - int byte_idx, end_idx, remain_len; + Idx byte_idx, end_idx, remain_len; size_t mbclen; /* Build the buffers from pstr->valid_len to either pstr->len or @@ -269,7 +279,7 @@ __attribute_warn_unused_result__ build_wcs_upper_buffer (re_string_t *pstr) { mbstate_t prev_st; - int src_idx, byte_idx, end_idx, remain_len; + Idx src_idx, byte_idx, end_idx, remain_len; size_t mbclen; #ifdef _LIBC char buf[MB_LEN_MAX]; @@ -307,14 +317,13 @@ build_wcs_upper_buffer (re_string_t *pstr) mbclen = __mbrtowc (&wc, ((const char *) pstr->raw_mbs + pstr->raw_mbs_idx + byte_idx), remain_len, &pstr->cur_state); - if (BE (mbclen + 2 > 2, 1)) + if (BE (mbclen < (size_t) -2, 1)) { - wchar_t wcu = wc; - if (__iswlower (wc)) + wchar_t wcu = __towupper (wc); + if (wcu != wc) { size_t mbcdlen; - wcu = __towupper (wc); mbcdlen = __wcrtomb (buf, wcu, &prev_st); if (BE (mbclen == mbcdlen, 1)) memcpy (pstr->mbs + byte_idx, buf, mbclen); @@ -377,14 +386,13 @@ build_wcs_upper_buffer (re_string_t *pstr) else p = (const char *) pstr->raw_mbs + pstr->raw_mbs_idx + src_idx; mbclen = __mbrtowc (&wc, p, remain_len, &pstr->cur_state); - if (BE (mbclen + 2 > 2, 1)) + if (BE (mbclen < (size_t) -2, 1)) { - wchar_t wcu = wc; - if (__iswlower (wc)) + wchar_t wcu = __towupper (wc); + if (wcu != wc) { size_t mbcdlen; - wcu = __towupper (wc); mbcdlen = __wcrtomb ((char *) buf, wcu, &prev_st); if (BE (mbclen == mbcdlen, 1)) memcpy (pstr->mbs + byte_idx, buf, mbclen); @@ -400,7 +408,7 @@ build_wcs_upper_buffer (re_string_t *pstr) if (pstr->offsets == NULL) { - pstr->offsets = re_malloc (int, pstr->bufs_len); + pstr->offsets = re_malloc (Idx, pstr->bufs_len); if (pstr->offsets == NULL) return REG_ESPACE; @@ -483,11 +491,11 @@ build_wcs_upper_buffer (re_string_t *pstr) /* Skip characters until the index becomes greater than NEW_RAW_IDX. Return the index. */ -static int -re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) +static Idx +re_string_skip_chars (re_string_t *pstr, Idx new_raw_idx, wint_t *last_wc) { mbstate_t prev_st; - int rawbuf_idx; + Idx rawbuf_idx; size_t mbclen; wint_t wc = WEOF; @@ -496,11 +504,11 @@ re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) rawbuf_idx < new_raw_idx;) { wchar_t wc2; - int remain_len = pstr->raw_len - rawbuf_idx; + Idx remain_len = pstr->raw_len - rawbuf_idx; prev_st = pstr->cur_state; mbclen = __mbrtowc (&wc2, (const char *) pstr->raw_mbs + rawbuf_idx, remain_len, &pstr->cur_state); - if (BE ((ssize_t) mbclen <= 0, 0)) + if (BE (mbclen == (size_t) -2 || mbclen == (size_t) -1 || mbclen == 0, 0)) { /* We treat these cases as a single byte character. */ if (mbclen == 0 || remain_len == 0) @@ -511,7 +519,7 @@ re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) pstr->cur_state = prev_st; } else - wc = (wint_t) wc2; + wc = wc2; /* Then proceed the next character. */ rawbuf_idx += mbclen; } @@ -526,7 +534,7 @@ re_string_skip_chars (re_string_t *pstr, int new_raw_idx, wint_t *last_wc) static void build_upper_buffer (re_string_t *pstr) { - int char_idx, end_idx; + Idx char_idx, end_idx; end_idx = (pstr->bufs_len > pstr->len) ? pstr->len : pstr->bufs_len; for (char_idx = pstr->valid_len; char_idx < end_idx; ++char_idx) @@ -534,10 +542,7 @@ build_upper_buffer (re_string_t *pstr) int ch = pstr->raw_mbs[pstr->raw_mbs_idx + char_idx]; if (BE (pstr->trans != NULL, 0)) ch = pstr->trans[ch]; - if (islower (ch)) - pstr->mbs[char_idx] = toupper (ch); - else - pstr->mbs[char_idx] = ch; + pstr->mbs[char_idx] = toupper (ch); } pstr->valid_len = char_idx; pstr->valid_raw_len = char_idx; @@ -548,7 +553,7 @@ build_upper_buffer (re_string_t *pstr) static void re_string_translate_buffer (re_string_t *pstr) { - int buf_idx, end_idx; + Idx buf_idx, end_idx; end_idx = (pstr->bufs_len > pstr->len) ? pstr->len : pstr->bufs_len; for (buf_idx = pstr->valid_len; buf_idx < end_idx; ++buf_idx) @@ -567,10 +572,13 @@ re_string_translate_buffer (re_string_t *pstr) static reg_errcode_t __attribute_warn_unused_result__ -re_string_reconstruct (re_string_t *pstr, int idx, int eflags) +re_string_reconstruct (re_string_t *pstr, Idx idx, int eflags) { - int offset = idx - pstr->raw_mbs_idx; - if (BE (offset < 0, 0)) + Idx offset; + + if (BE (pstr->raw_mbs_idx <= idx, 0)) + offset = idx - pstr->raw_mbs_idx; + else { /* Reset buffer. */ #ifdef RE_ENABLE_I18N @@ -599,7 +607,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) #ifdef RE_ENABLE_I18N if (BE (pstr->offsets_needed, 0)) { - int low = 0, high = pstr->valid_len, mid; + Idx low = 0, high = pstr->valid_len, mid; do { mid = (high + low) / 2; @@ -683,7 +691,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) { #ifdef RE_ENABLE_I18N /* No, skip all characters until IDX. */ - int prev_valid_len = pstr->valid_len; + Idx prev_valid_len = pstr->valid_len; if (BE (pstr->offsets_needed, 0)) { @@ -696,7 +704,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) #ifdef RE_ENABLE_I18N if (pstr->mb_cur_max > 1) { - int wcs_idx; + Idx wcs_idx; wint_t wc = WEOF; if (pstr->is_utf8) @@ -726,7 +734,7 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) { mbstate_t cur_state; wchar_t wc2; - int mlen = raw + pstr->len - p; + Idx mlen = raw + pstr->len - p; unsigned char buf[6]; size_t mbclen; @@ -826,10 +834,11 @@ re_string_reconstruct (re_string_t *pstr, int idx, int eflags) } static unsigned char -__attribute ((pure)) -re_string_peek_byte_case (const re_string_t *pstr, int idx) +__attribute__ ((pure)) +re_string_peek_byte_case (const re_string_t *pstr, Idx idx) { - int ch, off; + int ch; + Idx off; /* Handle the common (easiest) cases first. */ if (BE (!pstr->mbs_allocated, 1)) @@ -870,7 +879,8 @@ re_string_fetch_byte_case (re_string_t *pstr) #ifdef RE_ENABLE_I18N if (pstr->offsets_needed) { - int off, ch; + Idx off; + int ch; /* For tr_TR.UTF-8 [[:islower:]] there is [[: CAPITAL LETTER I WITH DOT lower:]] in mbs. Skip @@ -911,7 +921,7 @@ re_string_destruct (re_string_t *pstr) /* Return the context at IDX in INPUT. */ static unsigned int -re_string_context_at (const re_string_t *input, int idx, int eflags) +re_string_context_at (const re_string_t *input, Idx idx, int eflags) { int c; if (BE (idx < 0, 0)) @@ -925,7 +935,7 @@ re_string_context_at (const re_string_t *input, int idx, int eflags) if (input->mb_cur_max > 1) { wint_t wc; - int wc_idx = idx; + Idx wc_idx = idx; while(input->wcs[wc_idx] == WEOF) { #if defined DEBUG && DEBUG @@ -956,23 +966,23 @@ re_string_context_at (const re_string_t *input, int idx, int eflags) static reg_errcode_t __attribute_warn_unused_result__ -re_node_set_alloc (re_node_set *set, int size) +re_node_set_alloc (re_node_set *set, Idx size) { set->alloc = size; set->nelem = 0; - set->elems = re_malloc (int, size); - if (BE (set->elems == NULL, 0)) + set->elems = re_malloc (Idx, size); + if (BE (set->elems == NULL, 0) && (MALLOC_0_IS_NONNULL || size != 0)) return REG_ESPACE; return REG_NOERROR; } static reg_errcode_t __attribute_warn_unused_result__ -re_node_set_init_1 (re_node_set *set, int elem) +re_node_set_init_1 (re_node_set *set, Idx elem) { set->alloc = 1; set->nelem = 1; - set->elems = re_malloc (int, 1); + set->elems = re_malloc (Idx, 1); if (BE (set->elems == NULL, 0)) { set->alloc = set->nelem = 0; @@ -984,10 +994,10 @@ re_node_set_init_1 (re_node_set *set, int elem) static reg_errcode_t __attribute_warn_unused_result__ -re_node_set_init_2 (re_node_set *set, int elem1, int elem2) +re_node_set_init_2 (re_node_set *set, Idx elem1, Idx elem2) { set->alloc = 2; - set->elems = re_malloc (int, 2); + set->elems = re_malloc (Idx, 2); if (BE (set->elems == NULL, 0)) return REG_ESPACE; if (elem1 == elem2) @@ -1020,13 +1030,13 @@ re_node_set_init_copy (re_node_set *dest, const re_node_set *src) if (src->nelem > 0) { dest->alloc = dest->nelem; - dest->elems = re_malloc (int, dest->alloc); + dest->elems = re_malloc (Idx, dest->alloc); if (BE (dest->elems == NULL, 0)) { dest->alloc = dest->nelem = 0; return REG_ESPACE; } - memcpy (dest->elems, src->elems, src->nelem * sizeof (int)); + memcpy (dest->elems, src->elems, src->nelem * sizeof (Idx)); } else re_node_set_init_empty (dest); @@ -1042,7 +1052,7 @@ __attribute_warn_unused_result__ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, const re_node_set *src2) { - int i1, i2, is, id, delta, sbase; + Idx i1, i2, is, id, delta, sbase; if (src1->nelem == 0 || src2->nelem == 0) return REG_NOERROR; @@ -1050,8 +1060,8 @@ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, conservative estimate. */ if (src1->nelem + src2->nelem + dest->nelem > dest->alloc) { - int new_alloc = src1->nelem + src2->nelem + dest->alloc; - int *new_elems = re_realloc (dest->elems, int, new_alloc); + Idx new_alloc = src1->nelem + src2->nelem + dest->alloc; + Idx *new_elems = re_realloc (dest->elems, Idx, new_alloc); if (BE (new_elems == NULL, 0)) return REG_ESPACE; dest->elems = new_elems; @@ -1073,7 +1083,7 @@ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, --id; if (id < 0 || dest->elems[id] != src1->elems[i1]) - dest->elems[--sbase] = src1->elems[i1]; + dest->elems[--sbase] = src1->elems[i1]; if (--i1 < 0 || --i2 < 0) break; @@ -1120,7 +1130,7 @@ re_node_set_add_intersect (re_node_set *dest, const re_node_set *src1, } /* Copy remaining SRC elements. */ - memcpy (dest->elems, dest->elems + sbase, delta * sizeof (int)); + memcpy (dest->elems, dest->elems + sbase, delta * sizeof (Idx)); return REG_NOERROR; } @@ -1133,11 +1143,11 @@ __attribute_warn_unused_result__ re_node_set_init_union (re_node_set *dest, const re_node_set *src1, const re_node_set *src2) { - int i1, i2, id; + Idx i1, i2, id; if (src1 != NULL && src1->nelem > 0 && src2 != NULL && src2->nelem > 0) { dest->alloc = src1->nelem + src2->nelem; - dest->elems = re_malloc (int, dest->alloc); + dest->elems = re_malloc (Idx, dest->alloc); if (BE (dest->elems == NULL, 0)) return REG_ESPACE; } @@ -1165,13 +1175,13 @@ re_node_set_init_union (re_node_set *dest, const re_node_set *src1, if (i1 < src1->nelem) { memcpy (dest->elems + id, src1->elems + i1, - (src1->nelem - i1) * sizeof (int)); + (src1->nelem - i1) * sizeof (Idx)); id += src1->nelem - i1; } else if (i2 < src2->nelem) { memcpy (dest->elems + id, src2->elems + i2, - (src2->nelem - i2) * sizeof (int)); + (src2->nelem - i2) * sizeof (Idx)); id += src2->nelem - i2; } dest->nelem = id; @@ -1185,13 +1195,13 @@ static reg_errcode_t __attribute_warn_unused_result__ re_node_set_merge (re_node_set *dest, const re_node_set *src) { - int is, id, sbase, delta; + Idx is, id, sbase, delta; if (src == NULL || src->nelem == 0) return REG_NOERROR; if (dest->alloc < 2 * src->nelem + dest->nelem) { - int new_alloc = 2 * (src->nelem + dest->alloc); - int *new_buffer = re_realloc (dest->elems, int, new_alloc); + Idx new_alloc = 2 * (src->nelem + dest->alloc); + Idx *new_buffer = re_realloc (dest->elems, Idx, new_alloc); if (BE (new_buffer == NULL, 0)) return REG_ESPACE; dest->elems = new_buffer; @@ -1201,7 +1211,7 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) if (BE (dest->nelem == 0, 0)) { dest->nelem = src->nelem; - memcpy (dest->elems, src->elems, src->nelem * sizeof (int)); + memcpy (dest->elems, src->elems, src->nelem * sizeof (Idx)); return REG_NOERROR; } @@ -1222,7 +1232,7 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) { /* If DEST is exhausted, the remaining items of SRC must be unique. */ sbase -= is + 1; - memcpy (dest->elems + sbase, src->elems, (is + 1) * sizeof (int)); + memcpy (dest->elems + sbase, src->elems, (is + 1) * sizeof (Idx)); } id = dest->nelem - 1; @@ -1251,7 +1261,7 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) { /* Copy remaining SRC elements. */ memcpy (dest->elems, dest->elems + sbase, - delta * sizeof (int)); + delta * sizeof (Idx)); break; } } @@ -1262,38 +1272,33 @@ re_node_set_merge (re_node_set *dest, const re_node_set *src) /* Insert the new element ELEM to the re_node_set* SET. SET should not already have ELEM. - return -1 if an error is occured, return 1 otherwise. */ + Return true if successful. */ -static int +static bool __attribute_warn_unused_result__ -re_node_set_insert (re_node_set *set, int elem) +re_node_set_insert (re_node_set *set, Idx elem) { - int idx; + Idx idx; /* In case the set is empty. */ if (set->alloc == 0) - { - if (BE (re_node_set_init_1 (set, elem) == REG_NOERROR, 1)) - return 1; - else - return -1; - } + return BE (re_node_set_init_1 (set, elem) == REG_NOERROR, 1); if (BE (set->nelem, 0) == 0) { /* We already guaranteed above that set->alloc != 0. */ set->elems[0] = elem; ++set->nelem; - return 1; + return true; } /* Realloc if we need. */ if (set->alloc == set->nelem) { - int *new_elems; + Idx *new_elems; set->alloc = set->alloc * 2; - new_elems = re_realloc (set->elems, int, set->alloc); + new_elems = re_realloc (set->elems, Idx, set->alloc); if (BE (new_elems == NULL, 0)) - return -1; + return false; set->elems = new_elems; } @@ -1314,56 +1319,56 @@ re_node_set_insert (re_node_set *set, int elem) /* Insert the new element. */ set->elems[idx] = elem; ++set->nelem; - return 1; + return true; } /* Insert the new element ELEM to the re_node_set* SET. SET should not already have any element greater than or equal to ELEM. - Return -1 if an error is occured, return 1 otherwise. */ + Return true if successful. */ -static int +static bool __attribute_warn_unused_result__ -re_node_set_insert_last (re_node_set *set, int elem) +re_node_set_insert_last (re_node_set *set, Idx elem) { /* Realloc if we need. */ if (set->alloc == set->nelem) { - int *new_elems; + Idx *new_elems; set->alloc = (set->alloc + 1) * 2; - new_elems = re_realloc (set->elems, int, set->alloc); + new_elems = re_realloc (set->elems, Idx, set->alloc); if (BE (new_elems == NULL, 0)) - return -1; + return false; set->elems = new_elems; } /* Insert the new element. */ set->elems[set->nelem++] = elem; - return 1; + return true; } /* Compare two node sets SET1 and SET2. - return 1 if SET1 and SET2 are equivalent, return 0 otherwise. */ + Return true if SET1 and SET2 are equivalent. */ -static int -__attribute ((pure)) +static bool +__attribute__ ((pure)) re_node_set_compare (const re_node_set *set1, const re_node_set *set2) { - int i; + Idx i; if (set1 == NULL || set2 == NULL || set1->nelem != set2->nelem) - return 0; + return false; for (i = set1->nelem ; --i >= 0 ; ) if (set1->elems[i] != set2->elems[i]) - return 0; - return 1; + return false; + return true; } /* Return (idx + 1) if SET contains the element ELEM, return 0 otherwise. */ -static int -__attribute ((pure)) -re_node_set_contains (const re_node_set *set, int elem) +static Idx +__attribute__ ((pure)) +re_node_set_contains (const re_node_set *set, Idx elem) { - unsigned int idx, right, mid; + __re_size_t idx, right, mid; if (set->nelem <= 0) return 0; @@ -1382,7 +1387,7 @@ re_node_set_contains (const re_node_set *set, int elem) } static void -re_node_set_remove_at (re_node_set *set, int idx) +re_node_set_remove_at (re_node_set *set, Idx idx) { if (idx < 0 || idx >= set->nelem) return; @@ -1393,37 +1398,42 @@ re_node_set_remove_at (re_node_set *set, int idx) /* Add the token TOKEN to dfa->nodes, and return the index of the token. - Or return -1, if an error will be occured. */ + Or return -1 if an error occurred. */ -static int +static Idx re_dfa_add_node (re_dfa_t *dfa, re_token_t token) { - int type = token.type; if (BE (dfa->nodes_len >= dfa->nodes_alloc, 0)) { size_t new_nodes_alloc = dfa->nodes_alloc * 2; - int *new_nexts, *new_indices; + Idx *new_nexts, *new_indices; re_node_set *new_edests, *new_eclosures; re_token_t *new_nodes; /* Avoid overflows in realloc. */ const size_t max_object_size = MAX (sizeof (re_token_t), MAX (sizeof (re_node_set), - sizeof (int))); - if (BE (SIZE_MAX / max_object_size < new_nodes_alloc, 0)) + sizeof (Idx))); + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) < new_nodes_alloc, 0)) return -1; new_nodes = re_realloc (dfa->nodes, re_token_t, new_nodes_alloc); if (BE (new_nodes == NULL, 0)) return -1; dfa->nodes = new_nodes; - new_nexts = re_realloc (dfa->nexts, int, new_nodes_alloc); - new_indices = re_realloc (dfa->org_indices, int, new_nodes_alloc); + new_nexts = re_realloc (dfa->nexts, Idx, new_nodes_alloc); + new_indices = re_realloc (dfa->org_indices, Idx, new_nodes_alloc); new_edests = re_realloc (dfa->edests, re_node_set, new_nodes_alloc); new_eclosures = re_realloc (dfa->eclosures, re_node_set, new_nodes_alloc); if (BE (new_nexts == NULL || new_indices == NULL || new_edests == NULL || new_eclosures == NULL, 0)) - return -1; + { + re_free (new_nexts); + re_free (new_indices); + re_free (new_edests); + re_free (new_eclosures); + return -1; + } dfa->nexts = new_nexts; dfa->org_indices = new_indices; dfa->edests = new_edests; @@ -1434,7 +1444,8 @@ re_dfa_add_node (re_dfa_t *dfa, re_token_t token) dfa->nodes[dfa->nodes_len].constraint = 0; #ifdef RE_ENABLE_I18N dfa->nodes[dfa->nodes_len].accept_mb = - (type == OP_PERIOD && dfa->mb_cur_max > 1) || type == COMPLEX_BRACKET; + ((token.type == OP_PERIOD && dfa->mb_cur_max > 1) + || token.type == COMPLEX_BRACKET); #endif dfa->nexts[dfa->nodes_len] = -1; re_node_set_init_empty (dfa->edests + dfa->nodes_len); @@ -1442,11 +1453,11 @@ re_dfa_add_node (re_dfa_t *dfa, re_token_t token) return dfa->nodes_len++; } -static inline unsigned int +static re_hashval_t calc_state_hash (const re_node_set *nodes, unsigned int context) { - unsigned int hash = nodes->nelem + context; - int i; + re_hashval_t hash = nodes->nelem + context; + Idx i; for (i = 0 ; i < nodes->nelem ; i++) hash += nodes->elems[i]; return hash; @@ -1466,10 +1477,14 @@ __attribute_warn_unused_result__ re_acquire_state (reg_errcode_t *err, const re_dfa_t *dfa, const re_node_set *nodes) { - unsigned int hash; + re_hashval_t hash; re_dfastate_t *new_state; struct re_state_table_entry *spot; - int i; + Idx i; +#if defined GCC_LINT || defined lint + /* Suppress bogus uninitialized-variable warnings. */ + *err = REG_NOERROR; +#endif if (BE (nodes->nelem == 0, 0)) { *err = REG_NOERROR; @@ -1510,10 +1525,14 @@ __attribute_warn_unused_result__ re_acquire_state_context (reg_errcode_t *err, const re_dfa_t *dfa, const re_node_set *nodes, unsigned int context) { - unsigned int hash; + re_hashval_t hash; re_dfastate_t *new_state; struct re_state_table_entry *spot; - int i; + Idx i; +#if defined GCC_LINT || defined lint + /* Suppress bogus uninitialized-variable warnings. */ + *err = REG_NOERROR; +#endif if (nodes->nelem == 0) { *err = REG_NOERROR; @@ -1530,7 +1549,7 @@ re_acquire_state_context (reg_errcode_t *err, const re_dfa_t *dfa, && re_node_set_compare (state->entrance_nodes, nodes)) return state; } - /* There are no appropriate state in `dfa', create the new one. */ + /* There are no appropriate state in 'dfa', create the new one. */ new_state = create_cd_newstate (dfa, nodes, context, hash); if (BE (new_state == NULL, 0)) *err = REG_ESPACE; @@ -1545,11 +1564,11 @@ re_acquire_state_context (reg_errcode_t *err, const re_dfa_t *dfa, static reg_errcode_t __attribute_warn_unused_result__ register_state (const re_dfa_t *dfa, re_dfastate_t *newstate, - unsigned int hash) + re_hashval_t hash) { struct re_state_table_entry *spot; reg_errcode_t err; - int i; + Idx i; newstate->hash = hash; err = re_node_set_alloc (&newstate->non_eps_nodes, newstate->nodes.nelem); @@ -1557,16 +1576,16 @@ register_state (const re_dfa_t *dfa, re_dfastate_t *newstate, return REG_ESPACE; for (i = 0; i < newstate->nodes.nelem; i++) { - int elem = newstate->nodes.elems[i]; + Idx elem = newstate->nodes.elems[i]; if (!IS_EPSILON_NODE (dfa->nodes[elem].type)) - if (re_node_set_insert_last (&newstate->non_eps_nodes, elem) < 0) + if (! re_node_set_insert_last (&newstate->non_eps_nodes, elem)) return REG_ESPACE; } spot = dfa->state_table + (hash & dfa->state_hash_mask); if (BE (spot->alloc <= spot->num, 0)) { - int new_alloc = 2 * spot->num + 2; + Idx new_alloc = 2 * spot->num + 2; re_dfastate_t **new_array = re_realloc (spot->array, re_dfastate_t *, new_alloc); if (BE (new_array == NULL, 0)) @@ -1600,9 +1619,9 @@ free_state (re_dfastate_t *state) static re_dfastate_t * __attribute_warn_unused_result__ create_ci_newstate (const re_dfa_t *dfa, const re_node_set *nodes, - unsigned int hash) + re_hashval_t hash) { - int i; + Idx i; reg_errcode_t err; re_dfastate_t *newstate; @@ -1650,9 +1669,9 @@ create_ci_newstate (const re_dfa_t *dfa, const re_node_set *nodes, static re_dfastate_t * __attribute_warn_unused_result__ create_cd_newstate (const re_dfa_t *dfa, const re_node_set *nodes, - unsigned int context, unsigned int hash) + unsigned int context, re_hashval_t hash) { - int i, nctx_nodes = 0; + Idx i, nctx_nodes = 0; reg_errcode_t err; re_dfastate_t *newstate; diff --git a/posix/regex_internal.h b/posix/regex_internal.h index 41bf2d3..e621b3e 100644 --- a/posix/regex_internal.h +++ b/posix/regex_internal.h @@ -26,35 +26,60 @@ #include <stdlib.h> #include <string.h> -#if defined HAVE_LANGINFO_H || defined HAVE_LANGINFO_CODESET || defined _LIBC -# include <langinfo.h> -#endif -#if defined HAVE_LOCALE_H || defined _LIBC -# include <locale.h> -#endif -#if defined HAVE_WCHAR_H || defined _LIBC -# include <wchar.h> -#endif /* HAVE_WCHAR_H || _LIBC */ -#if defined HAVE_WCTYPE_H || defined _LIBC -# include <wctype.h> -#endif /* HAVE_WCTYPE_H || _LIBC */ -#if defined HAVE_STDBOOL_H || defined _LIBC -# include <stdbool.h> -#endif /* HAVE_STDBOOL_H || _LIBC */ -#if defined HAVE_STDINT_H || defined _LIBC -# include <stdint.h> -#endif /* HAVE_STDINT_H || _LIBC */ -#if defined _LIBC +#include <langinfo.h> +#include <locale.h> +#include <wchar.h> +#include <wctype.h> +#include <stdbool.h> +#include <stdint.h> + +#ifdef _LIBC # include <libc-lock.h> +# define lock_define(name) __libc_lock_define (, name) +# define lock_init(lock) (__libc_lock_init (lock), 0) +# define lock_fini(lock) +# define lock_lock(lock) __libc_lock_lock (lock) +# define lock_unlock(lock) __libc_lock_unlock (lock) +#elif defined GNULIB_LOCK && !defined USE_UNLOCKED_IO +# include "glthread/lock.h" + /* Use gl_lock_define if empty macro arguments are known to work. + Otherwise, fall back on less-portable substitutes. */ +# if ((defined __GNUC__ && !defined __STRICT_ANSI__) \ + || (defined __STDC_VERSION__ && 199901L <= __STDC_VERSION__)) +# define lock_define(name) gl_lock_define (, name) +# elif USE_POSIX_THREADS +# define lock_define(name) pthread_mutex_t name; +# elif USE_PTH_THREADS +# define lock_define(name) pth_mutex_t name; +# elif USE_SOLARIS_THREADS +# define lock_define(name) mutex_t name; +# elif USE_WINDOWS_THREADS +# define lock_define(name) gl_lock_t name; +# else +# define lock_define(name) +# endif +# define lock_init(lock) glthread_lock_init (&(lock)) +# define lock_fini(lock) glthread_lock_destroy (&(lock)) +# define lock_lock(lock) glthread_lock_lock (&(lock)) +# define lock_unlock(lock) glthread_lock_unlock (&(lock)) +#elif defined GNULIB_PTHREAD && !defined USE_UNLOCKED_IO +# include <pthread.h> +# define lock_define(name) pthread_mutex_t name; +# define lock_init(lock) pthread_mutex_init (&(lock), 0) +# define lock_fini(lock) pthread_mutex_destroy (&(lock)) +# define lock_lock(lock) pthread_mutex_lock (&(lock)) +# define lock_unlock(lock) pthread_mutex_unlock (&(lock)) #else -# define __libc_lock_define(CLASS,NAME) -# define __libc_lock_init(NAME) do { } while (0) -# define __libc_lock_lock(NAME) do { } while (0) -# define __libc_lock_unlock(NAME) do { } while (0) +# define lock_define(name) +# define lock_init(lock) 0 +# define lock_fini(lock) ((void) 0) + /* The 'dfa' avoids an "unused variable 'dfa'" warning from GCC. */ +# define lock_lock(lock) ((void) dfa) +# define lock_unlock(lock) ((void) 0) #endif /* In case that the system doesn't have isblank(). */ -#if !defined _LIBC && !defined HAVE_ISBLANK && !defined isblank +#if !defined _LIBC && ! (defined isblank || (HAVE_ISBLANK && HAVE_DECL_ISBLANK)) # define isblank(ch) ((ch) == ' ' || (ch) == '\t') #endif @@ -75,6 +100,7 @@ __dcgettext (_libc_intl_domainname, msgid, LC_MESSAGES) # endif #else +# undef gettext # define gettext(msgid) (msgid) #endif @@ -84,23 +110,17 @@ # define gettext_noop(String) String #endif -/* For loser systems without the definition. */ -#ifndef SIZE_MAX -# define SIZE_MAX ((size_t) -1) -#endif - #if (defined MB_CUR_MAX && HAVE_WCTYPE_H && HAVE_ISWCTYPE) || _LIBC # define RE_ENABLE_I18N #endif -#if __GNUC__ >= 3 -# define BE(expr, val) __builtin_expect (expr, val) -#else -# define BE(expr, val) (expr) -#endif +#define BE(expr, val) __builtin_expect (expr, val) + +/* Number of ASCII characters. */ +#define ASCII_CHARS 0x80 -/* Number of single byte character. */ -#define SBC_MAX 256 +/* Number of single byte characters. */ +#define SBC_MAX (UCHAR_MAX + 1) #define COLL_ELEM_LEN_MAX 8 @@ -110,11 +130,15 @@ /* Rename to standard API for using out of glibc. */ #ifndef _LIBC +# undef __wctype +# undef __iswctype # define __wctype wctype +# define __iswalnum iswalnum # define __iswctype iswctype +# define __towlower towlower +# define __towupper towupper # define __btowc btowc # define __mbrtowc mbrtowc -# define __mempcpy mempcpy # define __wcrtomb wcrtomb # define __regfree regfree # define attribute_hidden @@ -124,32 +148,70 @@ # define __attribute__(arg) #endif -extern const char __re_error_msgid[] attribute_hidden; -extern const size_t __re_error_msgid_idx[] attribute_hidden; +#ifndef SSIZE_MAX +# define SSIZE_MAX ((ssize_t) (SIZE_MAX / 2)) +#endif + +/* The type of indexes into strings. This is signed, not size_t, + since the API requires indexes to fit in regoff_t anyway, and using + signed integers makes the code a bit smaller and presumably faster. + The traditional GNU regex implementation uses int for indexes. + The POSIX-compatible implementation uses a possibly-wider type. + The name 'Idx' is three letters to minimize the hassle of + reindenting a lot of regex code that formerly used 'int'. */ +typedef regoff_t Idx; +#ifdef _REGEX_LARGE_OFFSETS +# define IDX_MAX SSIZE_MAX +#else +# define IDX_MAX INT_MAX +#endif + +/* A hash value, suitable for computing hash tables. */ +typedef __re_size_t re_hashval_t; /* An integer used to represent a set of bits. It must be unsigned, and must be at least as wide as unsigned int. */ typedef unsigned long int bitset_word_t; /* All bits set in a bitset_word_t. */ #define BITSET_WORD_MAX ULONG_MAX -/* Number of bits in a bitset_word_t. */ -#define BITSET_WORD_BITS (sizeof (bitset_word_t) * CHAR_BIT) -/* Number of bitset_word_t in a bit_set. */ -#define BITSET_WORDS (SBC_MAX / BITSET_WORD_BITS) + +/* Number of bits in a bitset_word_t. For portability to hosts with + padding bits, do not use '(sizeof (bitset_word_t) * CHAR_BIT)'; + instead, deduce it directly from BITSET_WORD_MAX. Avoid + greater-than-32-bit integers and unconditional shifts by more than + 31 bits, as they're not portable. */ +#if BITSET_WORD_MAX == 0xffffffffUL +# define BITSET_WORD_BITS 32 +#elif BITSET_WORD_MAX >> 31 >> 4 == 1 +# define BITSET_WORD_BITS 36 +#elif BITSET_WORD_MAX >> 31 >> 16 == 1 +# define BITSET_WORD_BITS 48 +#elif BITSET_WORD_MAX >> 31 >> 28 == 1 +# define BITSET_WORD_BITS 60 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 1 == 1 +# define BITSET_WORD_BITS 64 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 9 == 1 +# define BITSET_WORD_BITS 72 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 31 >> 31 >> 3 == 1 +# define BITSET_WORD_BITS 128 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 7 == 1 +# define BITSET_WORD_BITS 256 +#elif BITSET_WORD_MAX >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 31 >> 7 > 1 +# define BITSET_WORD_BITS 257 /* any value > SBC_MAX will do here */ +# if BITSET_WORD_BITS <= SBC_MAX +# error "Invalid SBC_MAX" +# endif +#else +# error "Add case for new bitset_word_t size" +#endif + +/* Number of bitset_word_t values in a bitset_t. */ +#define BITSET_WORDS ((SBC_MAX + BITSET_WORD_BITS - 1) / BITSET_WORD_BITS) + typedef bitset_word_t bitset_t[BITSET_WORDS]; typedef bitset_word_t *re_bitset_ptr_t; typedef const bitset_word_t *re_const_bitset_ptr_t; -#define bitset_set(set,i) \ - (set[i / BITSET_WORD_BITS] |= (bitset_word_t) 1 << i % BITSET_WORD_BITS) -#define bitset_clear(set,i) \ - (set[i / BITSET_WORD_BITS] &= ~((bitset_word_t) 1 << i % BITSET_WORD_BITS)) -#define bitset_contain(set,i) \ - (set[i / BITSET_WORD_BITS] & ((bitset_word_t) 1 << i % BITSET_WORD_BITS)) -#define bitset_empty(set) memset (set, '\0', sizeof (bitset_t)) -#define bitset_set_all(set) memset (set, '\xff', sizeof (bitset_t)) -#define bitset_copy(dest,src) memcpy (dest, src, sizeof (bitset_t)) - #define PREV_WORD_CONSTRAINT 0x0001 #define PREV_NOTWORD_CONSTRAINT 0x0002 #define NEXT_WORD_CONSTRAINT 0x0004 @@ -177,9 +239,9 @@ typedef enum typedef struct { - int alloc; - int nelem; - int *elems; + Idx alloc; + Idx nelem; + Idx *elems; } re_node_set; typedef enum @@ -265,19 +327,19 @@ typedef struct unsigned int non_match : 1; /* # of multibyte characters. */ - int nmbchars; + Idx nmbchars; /* # of collating symbols. */ - int ncoll_syms; + Idx ncoll_syms; /* # of equivalence classes. */ - int nequiv_classes; + Idx nequiv_classes; /* # of range expressions. */ - int nranges; + Idx nranges; /* # of character classes. */ - int nchar_classes; + Idx nchar_classes; } re_charset_t; #endif /* RE_ENABLE_I18N */ @@ -290,10 +352,10 @@ typedef struct #ifdef RE_ENABLE_I18N re_charset_t *mbcset; /* for COMPLEX_BRACKET */ #endif /* RE_ENABLE_I18N */ - int idx; /* for BACK_REF */ + Idx idx; /* for BACK_REF */ re_context_type ctx_type; /* for ANCHOR */ } opr; -#if __GNUC__ >= 2 +#if __GNUC__ >= 2 && !defined __STRICT_ANSI__ re_token_type_t type : 8; #else re_token_type_t type; @@ -324,30 +386,30 @@ struct re_string_t #ifdef RE_ENABLE_I18N /* Store the wide character string which is corresponding to MBS. */ wint_t *wcs; - int *offsets; + Idx *offsets; mbstate_t cur_state; #endif /* Index in RAW_MBS. Each character mbs[i] corresponds to raw_mbs[raw_mbs_idx + i]. */ - int raw_mbs_idx; + Idx raw_mbs_idx; /* The length of the valid characters in the buffers. */ - int valid_len; + Idx valid_len; /* The corresponding number of bytes in raw_mbs array. */ - int valid_raw_len; + Idx valid_raw_len; /* The length of the buffers MBS and WCS. */ - int bufs_len; + Idx bufs_len; /* The index in MBS, which is updated by re_string_fetch_byte. */ - int cur_idx; + Idx cur_idx; /* length of RAW_MBS array. */ - int raw_len; + Idx raw_len; /* This is RAW_LEN - RAW_MBS_IDX + VALID_LEN - VALID_RAW_LEN. */ - int len; + Idx len; /* End of the buffer may be shorter than its length in the cases such as re_match_2, re_search_2. Then, we use STOP for end of the buffer instead of LEN. */ - int raw_stop; + Idx raw_stop; /* This is RAW_STOP - RAW_MBS_IDX adjusted through OFFSETS. */ - int stop; + Idx stop; /* The context of mbs[0]. We store the context independently, since the context of mbs[0] may be different from raw_mbs[0], which is @@ -357,7 +419,7 @@ struct re_string_t RE_TRANSLATE_TYPE trans; /* Copy of re_dfa_t's word_char. */ re_const_bitset_ptr_t word_char; - /* 1 if REG_ICASE. */ + /* true if REG_ICASE. */ unsigned char icase; unsigned char is_utf8; unsigned char map_notascii; @@ -373,18 +435,10 @@ typedef struct re_string_t re_string_t; struct re_dfa_t; typedef struct re_dfa_t re_dfa_t; -#if IS_IN (libc) -static reg_errcode_t re_string_realloc_buffers (re_string_t *pstr, - int new_buf_len); -# ifdef RE_ENABLE_I18N -static void build_wcs_buffer (re_string_t *pstr); -static reg_errcode_t build_wcs_upper_buffer (re_string_t *pstr); -# endif /* RE_ENABLE_I18N */ -static void build_upper_buffer (re_string_t *pstr); -static void re_string_translate_buffer (re_string_t *pstr); -static unsigned int re_string_context_at (const re_string_t *input, int idx, - int eflags) __attribute__ ((pure)); +#ifndef _LIBC +# define IS_IN(libc) false #endif + #define re_string_peek_byte(pstr, offset) \ ((pstr)->mbs[(pstr)->cur_idx + offset]) #define re_string_fetch_byte(pstr) \ @@ -402,7 +456,9 @@ static unsigned int re_string_context_at (const re_string_t *input, int idx, #define re_string_skip_bytes(pstr,idx) ((pstr)->cur_idx += (idx)) #define re_string_set_index(pstr,idx) ((pstr)->cur_idx = (idx)) -#include <alloca.h> +#if defined _LIBC || HAVE_ALLOCA +# include <alloca.h> +#endif #ifndef _LIBC # if HAVE_ALLOCA @@ -414,9 +470,24 @@ static unsigned int re_string_context_at (const re_string_t *input, int idx, # else /* alloca is implemented with malloc, so just use malloc. */ # define __libc_use_alloca(n) 0 +# undef alloca +# define alloca(n) malloc (n) # endif #endif +#ifdef _LIBC +# define MALLOC_0_IS_NONNULL 1 +#elif !defined MALLOC_0_IS_NONNULL +# define MALLOC_0_IS_NONNULL 0 +#endif + +#ifndef MAX +# define MAX(a,b) ((a) < (b) ? (b) : (a)) +#endif +#ifndef MIN +# define MIN(a,b) ((a) < (b) ? (a) : (b)) +#endif + #define re_malloc(t,n) ((t *) malloc ((n) * sizeof (t))) #define re_realloc(p,t,n) ((t *) realloc (p, (n) * sizeof (t))) #define re_free(p) free (p) @@ -431,9 +502,9 @@ struct bin_tree_t re_token_t token; - /* `node_idx' is the index in dfa->nodes, if `type' == 0. - Otherwise `type' indicate the type of this node. */ - int node_idx; + /* 'node_idx' is the index in dfa->nodes, if 'type' == 0. + Otherwise 'type' indicate the type of this node. */ + Idx node_idx; }; typedef struct bin_tree_t bin_tree_t; @@ -477,7 +548,7 @@ typedef struct bin_tree_storage_t bin_tree_storage_t; struct re_dfastate_t { - unsigned int hash; + re_hashval_t hash; re_node_set nodes; re_node_set non_eps_nodes; re_node_set inveclosure; @@ -485,9 +556,9 @@ struct re_dfastate_t struct re_dfastate_t **trtable, **word_trtable; unsigned int context : 4; unsigned int halt : 1; - /* If this state can accept `multi byte'. + /* If this state can accept "multi byte". Note that we refer to multibyte characters, and multi character - collating elements as `multi byte'. */ + collating elements as "multi byte". */ unsigned int accept_mb : 1; /* If this state has backreference node(s). */ unsigned int has_backref : 1; @@ -497,8 +568,8 @@ typedef struct re_dfastate_t re_dfastate_t; struct re_state_table_entry { - int num; - int alloc; + Idx num; + Idx alloc; re_dfastate_t **array; }; @@ -506,8 +577,8 @@ struct re_state_table_entry typedef struct { - int next_idx; - int alloc; + Idx next_idx; + Idx alloc; re_dfastate_t **array; } state_array_t; @@ -515,8 +586,8 @@ typedef struct typedef struct { - int node; - int str_idx; /* The position NODE match at. */ + Idx node; + Idx str_idx; /* The position NODE match at. */ state_array_t path; } re_sub_match_last_t; @@ -526,20 +597,20 @@ typedef struct typedef struct { - int str_idx; - int node; + Idx str_idx; + Idx node; state_array_t *path; - int alasts; /* Allocation size of LASTS. */ - int nlasts; /* The number of LASTS. */ + Idx alasts; /* Allocation size of LASTS. */ + Idx nlasts; /* The number of LASTS. */ re_sub_match_last_t **lasts; } re_sub_match_top_t; struct re_backref_cache_entry { - int node; - int str_idx; - int subexp_from; - int subexp_to; + Idx node; + Idx str_idx; + Idx subexp_from; + Idx subexp_to; char more; char unused; unsigned short int eps_reachable_subexps_map; @@ -557,18 +628,18 @@ typedef struct /* EFLAGS of the argument of regexec. */ int eflags; /* Where the matching ends. */ - int match_last; - int last_node; + Idx match_last; + Idx last_node; /* The state log used by the matcher. */ re_dfastate_t **state_log; - int state_log_top; + Idx state_log_top; /* Back reference cache. */ - int nbkref_ents; - int abkref_ents; + Idx nbkref_ents; + Idx abkref_ents; struct re_backref_cache_entry *bkref_ents; int max_mb_elem_len; - int nsub_tops; - int asub_tops; + Idx nsub_tops; + Idx asub_tops; re_sub_match_top_t **sub_tops; } re_match_context_t; @@ -576,23 +647,23 @@ typedef struct { re_dfastate_t **sifted_states; re_dfastate_t **limited_states; - int last_node; - int last_str_idx; + Idx last_node; + Idx last_str_idx; re_node_set limits; } re_sift_context_t; struct re_fail_stack_ent_t { - int idx; - int node; + Idx idx; + Idx node; regmatch_t *regs; re_node_set eps_via_nodes; }; struct re_fail_stack_t { - int num; - int alloc; + Idx num; + Idx alloc; struct re_fail_stack_ent_t *stack; }; @@ -601,8 +672,8 @@ struct re_dfa_t re_token_t *nodes; size_t nodes_alloc; size_t nodes_len; - int *nexts; - int *org_indices; + Idx *nexts; + Idx *org_indices; re_node_set *edests; re_node_set *eclosures; re_node_set *inveclosures; @@ -616,10 +687,10 @@ struct re_dfa_t re_bitset_ptr_t sb_char; int str_tree_storage_idx; - /* number of subexpressions `re_nsub' is in regex_t. */ - unsigned int state_hash_mask; - int init_node; - int nbackref; /* The number of backreference in this dfa. */ + /* number of subexpressions 're_nsub' is in regex_t. */ + re_hashval_t state_hash_mask; + Idx init_node; + Idx nbackref; /* The number of backreference in this dfa. */ /* Bitmap expressing which backreference is used. */ bitset_word_t used_bkref_map; @@ -636,11 +707,11 @@ struct re_dfa_t int mb_cur_max; bitset_t word_char; reg_syntax_t syntax; - int *subexp_map; + Idx *subexp_map; #ifdef DEBUG char* re_str; #endif - __libc_lock_define (, lock) + lock_define (lock) }; #define re_node_set_init_empty(set) memset (set, '\0', sizeof (re_node_set)) @@ -671,16 +742,60 @@ typedef struct } bracket_elem_t; -/* Inline functions for bitset operation. */ -static void __attribute__ ((unused)) +/* Functions for bitset_t operation. */ + +static inline void +bitset_set (bitset_t set, Idx i) +{ + set[i / BITSET_WORD_BITS] |= (bitset_word_t) 1 << i % BITSET_WORD_BITS; +} + +static inline void +bitset_clear (bitset_t set, Idx i) +{ + set[i / BITSET_WORD_BITS] &= ~ ((bitset_word_t) 1 << i % BITSET_WORD_BITS); +} + +static inline bool +bitset_contain (const bitset_t set, Idx i) +{ + return (set[i / BITSET_WORD_BITS] >> i % BITSET_WORD_BITS) & 1; +} + +static inline void +bitset_empty (bitset_t set) +{ + memset (set, '\0', sizeof (bitset_t)); +} + +static inline void +bitset_set_all (bitset_t set) +{ + memset (set, -1, sizeof (bitset_word_t) * (SBC_MAX / BITSET_WORD_BITS)); + if (SBC_MAX % BITSET_WORD_BITS != 0) + set[BITSET_WORDS - 1] = + ((bitset_word_t) 1 << SBC_MAX % BITSET_WORD_BITS) - 1; +} + +static inline void +bitset_copy (bitset_t dest, const bitset_t src) +{ + memcpy (dest, src, sizeof (bitset_t)); +} + +static inline void bitset_not (bitset_t set) { int bitset_i; - for (bitset_i = 0; bitset_i < BITSET_WORDS; ++bitset_i) + for (bitset_i = 0; bitset_i < SBC_MAX / BITSET_WORD_BITS; ++bitset_i) set[bitset_i] = ~set[bitset_i]; + if (SBC_MAX % BITSET_WORD_BITS != 0) + set[BITSET_WORDS - 1] = + ((((bitset_word_t) 1 << SBC_MAX % BITSET_WORD_BITS) - 1) + & ~set[BITSET_WORDS - 1]); } -static void __attribute__ ((unused)) +static inline void bitset_merge (bitset_t dest, const bitset_t src) { int bitset_i; @@ -688,7 +803,7 @@ bitset_merge (bitset_t dest, const bitset_t src) dest[bitset_i] |= src[bitset_i]; } -static void __attribute__ ((unused)) +static inline void bitset_mask (bitset_t dest, const bitset_t src) { int bitset_i; @@ -697,10 +812,10 @@ bitset_mask (bitset_t dest, const bitset_t src) } #ifdef RE_ENABLE_I18N -/* Inline functions for re_string. */ +/* Functions for re_string. */ static int __attribute__ ((pure, unused)) -re_string_char_size_at (const re_string_t *pstr, int idx) +re_string_char_size_at (const re_string_t *pstr, Idx idx) { int byte_idx; if (pstr->mb_cur_max == 1) @@ -713,23 +828,22 @@ re_string_char_size_at (const re_string_t *pstr, int idx) static wint_t __attribute__ ((pure, unused)) -re_string_wchar_at (const re_string_t *pstr, int idx) +re_string_wchar_at (const re_string_t *pstr, Idx idx) { if (pstr->mb_cur_max == 1) return (wint_t) pstr->mbs[idx]; return (wint_t) pstr->wcs[idx]; } -# if IS_IN (libc) -# ifdef _LIBC -# include <locale/weight.h> -# endif +# ifdef _LIBC +# include <locale/weight.h> +# endif static int __attribute__ ((pure, unused)) -re_string_elem_size_at (const re_string_t *pstr, int idx) +re_string_elem_size_at (const re_string_t *pstr, Idx idx) { -# ifdef _LIBC +# ifdef _LIBC const unsigned char *p, *extra; const int32_t *table, *indirect; uint_fast32_t nrules = _NL_CURRENT_WORD (LC_COLLATE, _NL_COLLATE_NRULES); @@ -746,10 +860,34 @@ re_string_elem_size_at (const re_string_t *pstr, int idx) return p - pstr->mbs - idx; } else -# endif /* _LIBC */ +# endif /* _LIBC */ return 1; } -# endif #endif /* RE_ENABLE_I18N */ +#ifndef __GNUC_PREREQ +# if defined __GNUC__ && defined __GNUC_MINOR__ +# define __GNUC_PREREQ(maj, min) \ + ((__GNUC__ << 16) + __GNUC_MINOR__ >= ((maj) << 16) + (min)) +# else +# define __GNUC_PREREQ(maj, min) 0 +# endif +#endif + +#if __GNUC_PREREQ (3,4) +# undef __attribute_warn_unused_result__ +# define __attribute_warn_unused_result__ \ + __attribute__ ((__warn_unused_result__)) +#else +# define __attribute_warn_unused_result__ /* empty */ +#endif + +#ifndef FALLTHROUGH +# if __GNUC__ < 7 +# define FALLTHROUGH ((void) 0) +# else +# define FALLTHROUGH __attribute__ ((__fallthrough__)) +# endif +#endif + #endif /* _REGEX_INTERNAL_H */ diff --git a/posix/regexec.c b/posix/regexec.c index 4b1ab4e..573fc87 100644 --- a/posix/regexec.c +++ b/posix/regexec.c @@ -17,98 +17,96 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <stdint.h> - static reg_errcode_t match_ctx_init (re_match_context_t *cache, int eflags, - int n); + Idx n); static void match_ctx_clean (re_match_context_t *mctx); static void match_ctx_free (re_match_context_t *cache); -static reg_errcode_t match_ctx_add_entry (re_match_context_t *cache, int node, - int str_idx, int from, int to); -static int search_cur_bkref_entry (const re_match_context_t *mctx, - int str_idx); -static reg_errcode_t match_ctx_add_subtop (re_match_context_t *mctx, int node, - int str_idx); +static reg_errcode_t match_ctx_add_entry (re_match_context_t *cache, Idx node, + Idx str_idx, Idx from, Idx to); +static Idx search_cur_bkref_entry (const re_match_context_t *mctx, Idx str_idx); +static reg_errcode_t match_ctx_add_subtop (re_match_context_t *mctx, Idx node, + Idx str_idx); static re_sub_match_last_t * match_ctx_add_sublast (re_sub_match_top_t *subtop, - int node, int str_idx); + Idx node, Idx str_idx); static void sift_ctx_init (re_sift_context_t *sctx, re_dfastate_t **sifted_sts, - re_dfastate_t **limited_sts, int last_node, - int last_str_idx); + re_dfastate_t **limited_sts, Idx last_node, + Idx last_str_idx); static reg_errcode_t re_search_internal (const regex_t *preg, - const char *string, int length, - int start, int range, int stop, + const char *string, Idx length, + Idx start, Idx last_start, Idx stop, size_t nmatch, regmatch_t pmatch[], int eflags); -static int re_search_2_stub (struct re_pattern_buffer *bufp, - const char *string1, int length1, - const char *string2, int length2, - int start, int range, struct re_registers *regs, - int stop, int ret_len); -static int re_search_stub (struct re_pattern_buffer *bufp, - const char *string, int length, int start, - int range, int stop, struct re_registers *regs, - int ret_len); +static regoff_t re_search_2_stub (struct re_pattern_buffer *bufp, + const char *string1, Idx length1, + const char *string2, Idx length2, + Idx start, regoff_t range, + struct re_registers *regs, + Idx stop, bool ret_len); +static regoff_t re_search_stub (struct re_pattern_buffer *bufp, + const char *string, Idx length, Idx start, + regoff_t range, Idx stop, + struct re_registers *regs, + bool ret_len); static unsigned re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, - int nregs, int regs_allocated); + Idx nregs, int regs_allocated); static reg_errcode_t prune_impossible_nodes (re_match_context_t *mctx); -static int check_matching (re_match_context_t *mctx, int fl_longest_match, - int *p_match_first); -static int check_halt_state_context (const re_match_context_t *mctx, - const re_dfastate_t *state, int idx); +static Idx check_matching (re_match_context_t *mctx, bool fl_longest_match, + Idx *p_match_first); +static Idx check_halt_state_context (const re_match_context_t *mctx, + const re_dfastate_t *state, Idx idx); static void update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, - regmatch_t *prev_idx_match, int cur_node, - int cur_idx, int nmatch); + regmatch_t *prev_idx_match, Idx cur_node, + Idx cur_idx, Idx nmatch); static reg_errcode_t push_fail_stack (struct re_fail_stack_t *fs, - int str_idx, int dest_node, int nregs, + Idx str_idx, Idx dest_node, Idx nregs, regmatch_t *regs, re_node_set *eps_via_nodes); static reg_errcode_t set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, regmatch_t *pmatch, - int fl_backtrack); + bool fl_backtrack); static reg_errcode_t free_fail_stack_return (struct re_fail_stack_t *fs); #ifdef RE_ENABLE_I18N static int sift_states_iter_mb (const re_match_context_t *mctx, re_sift_context_t *sctx, - int node_idx, int str_idx, int max_str_idx); + Idx node_idx, Idx str_idx, Idx max_str_idx); #endif /* RE_ENABLE_I18N */ static reg_errcode_t sift_states_backward (const re_match_context_t *mctx, re_sift_context_t *sctx); static reg_errcode_t build_sifted_states (const re_match_context_t *mctx, - re_sift_context_t *sctx, int str_idx, + re_sift_context_t *sctx, Idx str_idx, re_node_set *cur_dest); static reg_errcode_t update_cur_sifted_state (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, + Idx str_idx, re_node_set *dest_nodes); static reg_errcode_t add_epsilon_src_nodes (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates); -static int check_dst_limits (const re_match_context_t *mctx, - re_node_set *limits, - int dst_node, int dst_idx, int src_node, - int src_idx); +static bool check_dst_limits (const re_match_context_t *mctx, + const re_node_set *limits, + Idx dst_node, Idx dst_idx, Idx src_node, + Idx src_idx); static int check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, - int boundaries, int subexp_idx, - int from_node, int bkref_idx); + int boundaries, Idx subexp_idx, + Idx from_node, Idx bkref_idx); static int check_dst_limits_calc_pos (const re_match_context_t *mctx, - int limit, int subexp_idx, - int node, int str_idx, - int bkref_idx); + Idx limit, Idx subexp_idx, + Idx node, Idx str_idx, + Idx bkref_idx); static reg_errcode_t check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates, re_node_set *limits, struct re_backref_cache_entry *bkref_ents, - int str_idx); + Idx str_idx); static reg_errcode_t sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, - const re_node_set *candidates); + Idx str_idx, const re_node_set *candidates); static reg_errcode_t merge_state_array (const re_dfa_t *dfa, re_dfastate_t **dst, - re_dfastate_t **src, int num); + re_dfastate_t **src, Idx num); static re_dfastate_t *find_recover_state (reg_errcode_t *err, re_match_context_t *mctx); static re_dfastate_t *transit_state (reg_errcode_t *err, @@ -119,7 +117,7 @@ static re_dfastate_t *merge_state_with_log (reg_errcode_t *err, re_dfastate_t *next_state); static reg_errcode_t check_subexp_matching_top (re_match_context_t *mctx, re_node_set *cur_nodes, - int str_idx); + Idx str_idx); #if 0 static re_dfastate_t *transit_state_sb (reg_errcode_t *err, re_match_context_t *mctx, @@ -132,46 +130,46 @@ static reg_errcode_t transit_state_mb (re_match_context_t *mctx, static reg_errcode_t transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes); static reg_errcode_t get_subexp (re_match_context_t *mctx, - int bkref_node, int bkref_str_idx); + Idx bkref_node, Idx bkref_str_idx); static reg_errcode_t get_subexp_sub (re_match_context_t *mctx, const re_sub_match_top_t *sub_top, re_sub_match_last_t *sub_last, - int bkref_node, int bkref_str); -static int find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, - int subexp_idx, int type); + Idx bkref_node, Idx bkref_str); +static Idx find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, + Idx subexp_idx, int type); static reg_errcode_t check_arrival (re_match_context_t *mctx, - state_array_t *path, int top_node, - int top_str, int last_node, int last_str, + state_array_t *path, Idx top_node, + Idx top_str, Idx last_node, Idx last_str, int type); static reg_errcode_t check_arrival_add_next_nodes (re_match_context_t *mctx, - int str_idx, + Idx str_idx, re_node_set *cur_nodes, re_node_set *next_nodes); static reg_errcode_t check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, - int ex_subexp, int type); + Idx ex_subexp, int type); static reg_errcode_t check_arrival_expand_ecl_sub (const re_dfa_t *dfa, re_node_set *dst_nodes, - int target, int ex_subexp, + Idx target, Idx ex_subexp, int type); static reg_errcode_t expand_bkref_cache (re_match_context_t *mctx, - re_node_set *cur_nodes, int cur_str, - int subexp_num, int type); -static int build_trtable (const re_dfa_t *dfa, re_dfastate_t *state); + re_node_set *cur_nodes, Idx cur_str, + Idx subexp_num, int type); +static bool build_trtable (const re_dfa_t *dfa, re_dfastate_t *state); #ifdef RE_ENABLE_I18N -static int check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, - const re_string_t *input, int idx); +static int check_node_accept_bytes (const re_dfa_t *dfa, Idx node_idx, + const re_string_t *input, Idx idx); # ifdef _LIBC static unsigned int find_collation_sequence_value (const unsigned char *mbs, size_t name_len); # endif /* _LIBC */ #endif /* RE_ENABLE_I18N */ -static int group_nodes_into_DFAstates (const re_dfa_t *dfa, +static Idx group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, re_node_set *states_node, bitset_t *states_ch); -static int check_node_accept (const re_match_context_t *mctx, - const re_token_t *node, int idx); +static bool check_node_accept (const re_match_context_t *mctx, + const re_token_t *node, Idx idx); static reg_errcode_t extend_buffers (re_match_context_t *mctx, int min_len); /* Entry point for POSIX code. */ @@ -180,23 +178,23 @@ static reg_errcode_t extend_buffers (re_match_context_t *mctx, int min_len); string STRING. If NMATCH is zero or REG_NOSUB was set in the cflags argument to - `regcomp', we ignore PMATCH. Otherwise, we assume PMATCH has at + 'regcomp', we ignore PMATCH. Otherwise, we assume PMATCH has at least NMATCH elements, and we set them to the offsets of the corresponding matched substrings. - EFLAGS specifies `execution flags' which affect matching: if + EFLAGS specifies "execution flags" which affect matching: if REG_NOTBOL is set, then ^ does not match at the beginning of the string; if REG_NOTEOL is set, then $ does not match at the end. We return 0 if we find a match and REG_NOMATCH if not. */ int -regexec (const regex_t *__restrict preg, const char *__restrict string, +regexec (const regex_t *_Restrict_ preg, const char *_Restrict_ string, size_t nmatch, regmatch_t pmatch[], int eflags) { reg_errcode_t err; - int start, length; - re_dfa_t *dfa = (re_dfa_t *) preg->buffer; + Idx start, length; + re_dfa_t *dfa = preg->buffer; if (eflags & ~(REG_NOTBOL | REG_NOTEOL | REG_STARTEND)) return REG_BADPAT; @@ -212,14 +210,14 @@ regexec (const regex_t *__restrict preg, const char *__restrict string, length = strlen (string); } - __libc_lock_lock (dfa->lock); + lock_lock (dfa->lock); if (preg->no_sub) - err = re_search_internal (preg, string, length, start, length - start, + err = re_search_internal (preg, string, length, start, length, length, 0, NULL, eflags); else - err = re_search_internal (preg, string, length, start, length - start, + err = re_search_internal (preg, string, length, start, length, length, nmatch, pmatch, eflags); - __libc_lock_unlock (dfa->lock); + lock_unlock (dfa->lock); return err != REG_NOERROR; } @@ -234,8 +232,8 @@ __typeof__ (__regexec) __compat_regexec; int attribute_compat_text_section -__compat_regexec (const regex_t *__restrict preg, - const char *__restrict string, size_t nmatch, +__compat_regexec (const regex_t *_Restrict_ preg, + const char *_Restrict_ string, size_t nmatch, regmatch_t pmatch[], int eflags) { return regexec (preg, string, nmatch, pmatch, @@ -274,62 +272,82 @@ compat_symbol (libc, __compat_regexec, regexec, GLIBC_2_0); return the position of the start of the match. Return value -1 means no match was found and -2 indicates an internal error. */ -int -re_match (struct re_pattern_buffer *bufp, const char *string, int length, - int start, struct re_registers *regs) +regoff_t +re_match (struct re_pattern_buffer *bufp, const char *string, Idx length, + Idx start, struct re_registers *regs) { - return re_search_stub (bufp, string, length, start, 0, length, regs, 1); + return re_search_stub (bufp, string, length, start, 0, length, regs, true); } #ifdef _LIBC weak_alias (__re_match, re_match) #endif -int -re_search (struct re_pattern_buffer *bufp, const char *string, int length, - int start, int range, struct re_registers *regs) +regoff_t +re_search (struct re_pattern_buffer *bufp, const char *string, Idx length, + Idx start, regoff_t range, struct re_registers *regs) { - return re_search_stub (bufp, string, length, start, range, length, regs, 0); + return re_search_stub (bufp, string, length, start, range, length, regs, + false); } #ifdef _LIBC weak_alias (__re_search, re_search) #endif -int -re_match_2 (struct re_pattern_buffer *bufp, const char *string1, int length1, - const char *string2, int length2, int start, - struct re_registers *regs, int stop) +regoff_t +re_match_2 (struct re_pattern_buffer *bufp, const char *string1, Idx length1, + const char *string2, Idx length2, Idx start, + struct re_registers *regs, Idx stop) { return re_search_2_stub (bufp, string1, length1, string2, length2, - start, 0, regs, stop, 1); + start, 0, regs, stop, true); } #ifdef _LIBC weak_alias (__re_match_2, re_match_2) #endif -int -re_search_2 (struct re_pattern_buffer *bufp, const char *string1, int length1, - const char *string2, int length2, int start, int range, - struct re_registers *regs, int stop) +regoff_t +re_search_2 (struct re_pattern_buffer *bufp, const char *string1, Idx length1, + const char *string2, Idx length2, Idx start, regoff_t range, + struct re_registers *regs, Idx stop) { return re_search_2_stub (bufp, string1, length1, string2, length2, - start, range, regs, stop, 0); + start, range, regs, stop, false); } #ifdef _LIBC weak_alias (__re_search_2, re_search_2) #endif -static int +/* Set *R = A + B. Return true if the answer is mathematically incorrect due + to overflow; in this case, *R is the low order bits of the correct + answer. */ +static inline bool +check_add_overflow_idx (Idx a, Idx b, Idx *r) +{ +#if __GNUC__ >= 5 + return __builtin_add_overflow (a, b, r); +#else + if ((a > 0 && b > SSIZE_MAX - a) + || (a < 0 && b < SSIZE_MIN -a) + return false; + *r = a + b; + return true; +#endif +} + +static regoff_t re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, - int length1, const char *string2, int length2, int start, - int range, struct re_registers *regs, - int stop, int ret_len) + Idx length1, const char *string2, Idx length2, Idx start, + regoff_t range, struct re_registers *regs, + Idx stop, bool ret_len) { const char *str; - int rval; - int len = length1 + length2; + regoff_t rval; + Idx len; char *s = NULL; - if (BE (length1 < 0 || length2 < 0 || stop < 0 || len < length1, 0)) + if (BE ((length1 < 0 || length2 < 0 || stop < 0 + || check_add_overflow_idx (length1, length2, &len)), + 0)) return -2; /* Concatenate the strings. */ @@ -353,42 +371,45 @@ re_search_2_stub (struct re_pattern_buffer *bufp, const char *string1, else str = string1; - rval = re_search_stub (bufp, str, len, start, range, stop, regs, ret_len); + rval = re_search_stub (bufp, str, len, start, range, stop, regs, + ret_len); re_free (s); return rval; } /* The parameters have the same meaning as those of re_search. Additional parameters: - If RET_LEN is nonzero the length of the match is returned (re_match style); + If RET_LEN is true the length of the match is returned (re_match style); otherwise the position of the match is returned. */ -static int -re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, - int start, int range, int stop, struct re_registers *regs, - int ret_len) +static regoff_t +re_search_stub (struct re_pattern_buffer *bufp, const char *string, Idx length, + Idx start, regoff_t range, Idx stop, struct re_registers *regs, + bool ret_len) { reg_errcode_t result; regmatch_t *pmatch; - int nregs, rval; + Idx nregs; + regoff_t rval; int eflags = 0; - re_dfa_t *dfa = (re_dfa_t *) bufp->buffer; + re_dfa_t *dfa = bufp->buffer; + Idx last_start = start + range; /* Check for out-of-range. */ if (BE (start < 0 || start > length, 0)) return -1; - if (BE (start + range > length, 0)) - range = length - start; - else if (BE (start + range < 0, 0)) - range = -start; + if (BE (length < last_start || (0 <= range && last_start < start), 0)) + last_start = length; + else if (BE (last_start < 0 || (range < 0 && start <= last_start), 0)) + last_start = 0; - __libc_lock_lock (dfa->lock); + lock_lock (dfa->lock); eflags |= (bufp->not_bol) ? REG_NOTBOL : 0; eflags |= (bufp->not_eol) ? REG_NOTEOL : 0; /* Compile fastmap if we haven't yet. */ - if (range > 0 && bufp->fastmap != NULL && !bufp->fastmap_accurate) + if (start < last_start && bufp->fastmap != NULL && !bufp->fastmap_accurate) re_compile_fastmap (bufp); if (BE (bufp->no_sub, 0)) @@ -397,8 +418,8 @@ re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, /* We need at least 1 register. */ if (regs == NULL) nregs = 1; - else if (BE (bufp->regs_allocated == REGS_FIXED && - regs->num_regs < bufp->re_nsub + 1, 0)) + else if (BE (bufp->regs_allocated == REGS_FIXED + && regs->num_regs <= bufp->re_nsub, 0)) { nregs = regs->num_regs; if (BE (nregs < 1, 0)) @@ -417,14 +438,14 @@ re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, goto out; } - result = re_search_internal (bufp, string, length, start, range, stop, + result = re_search_internal (bufp, string, length, start, last_start, stop, nregs, pmatch, eflags); rval = 0; - /* I hope we needn't fill ther regs with -1's when no match was found. */ + /* I hope we needn't fill their regs with -1's when no match was found. */ if (result != REG_NOERROR) - rval = -1; + rval = result == REG_NOMATCH ? -1 : -2; else if (regs != NULL) { /* If caller wants register contents data back, copy them. */ @@ -446,18 +467,18 @@ re_search_stub (struct re_pattern_buffer *bufp, const char *string, int length, } re_free (pmatch); out: - __libc_lock_unlock (dfa->lock); + lock_unlock (dfa->lock); return rval; } static unsigned -re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, int nregs, +re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, Idx nregs, int regs_allocated) { int rval = REGS_REALLOCATE; - int i; - int need_regs = nregs + 1; - /* We need one extra element beyond `num_regs' for the `-1' marker GNU code + Idx i; + Idx need_regs = nregs + 1; + /* We need one extra element beyond 'num_regs' for the '-1' marker GNU code uses. */ /* Have the register data arrays been allocated? */ @@ -530,7 +551,7 @@ re_copy_regs (struct re_registers *regs, regmatch_t *pmatch, int nregs, void re_set_registers (struct re_pattern_buffer *bufp, struct re_registers *regs, - unsigned num_regs, regoff_t *starts, regoff_t *ends) + __re_size_t num_regs, regoff_t *starts, regoff_t *ends) { if (num_regs) { @@ -543,7 +564,7 @@ re_set_registers (struct re_pattern_buffer *bufp, struct re_registers *regs, { bufp->regs_allocated = REGS_UNALLOCATED; regs->num_regs = 0; - regs->start = regs->end = (regoff_t *) 0; + regs->start = regs->end = NULL; } } #ifdef _LIBC @@ -568,32 +589,38 @@ re_exec (const char *s) /* Searches for a compiled pattern PREG in the string STRING, whose length is LENGTH. NMATCH, PMATCH, and EFLAGS have the same - meaning as with regexec. START, and RANGE have the same meanings - with re_search. + meaning as with regexec. LAST_START is START + RANGE, where + START and RANGE have the same meaning as with re_search. Return REG_NOERROR if we find a match, and REG_NOMATCH if not, otherwise return the error code. Note: We assume front end functions already check ranges. - (START + RANGE >= 0 && START + RANGE <= LENGTH) */ + (0 <= LAST_START && LAST_START <= LENGTH) */ static reg_errcode_t __attribute_warn_unused_result__ -re_search_internal (const regex_t *preg, const char *string, int length, - int start, int range, int stop, size_t nmatch, +re_search_internal (const regex_t *preg, const char *string, Idx length, + Idx start, Idx last_start, Idx stop, size_t nmatch, regmatch_t pmatch[], int eflags) { reg_errcode_t err; - const re_dfa_t *dfa = (const re_dfa_t *) preg->buffer; - int left_lim, right_lim, incr; - int fl_longest_match, match_first, match_kind, match_last = -1; - int extra_nmatch; - int sb, ch; + const re_dfa_t *dfa = preg->buffer; + Idx left_lim, right_lim; + int incr; + bool fl_longest_match; + int match_kind; + Idx match_first; + Idx match_last = -1; + Idx extra_nmatch; + bool sb; + int ch; #if defined _LIBC || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L) re_match_context_t mctx = { .dfa = dfa }; #else re_match_context_t mctx; #endif - char *fastmap = (preg->fastmap != NULL && preg->fastmap_accurate - && range && !preg->can_be_null) ? preg->fastmap : NULL; + char *fastmap = ((preg->fastmap != NULL && preg->fastmap_accurate + && start != last_start && !preg->can_be_null) + ? preg->fastmap : NULL); RE_TRANSLATE_TYPE t = preg->translate; #if !(defined _LIBC || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L)) @@ -612,7 +639,7 @@ re_search_internal (const regex_t *preg, const char *string, int length, #ifdef DEBUG /* We assume front-end functions already check them. */ - assert (start + range >= 0 && start + range <= length); + assert (0 <= last_start && last_start <= length); #endif /* If initial states with non-begbuf contexts have no elements, @@ -623,16 +650,17 @@ re_search_internal (const regex_t *preg, const char *string, int length, && (dfa->init_state_nl->nodes.nelem == 0 || !preg->newline_anchor)) { - if (start != 0 && start + range != 0) - return REG_NOMATCH; - start = range = 0; + if (start != 0 && last_start != 0) + return REG_NOMATCH; + start = last_start = 0; } /* We must check the longest matching, if nmatch > 0. */ fl_longest_match = (nmatch != 0 || dfa->nbackref); err = re_string_allocate (&mctx.input, string, length, dfa->nodes_len + 1, - preg->translate, preg->syntax & RE_ICASE, dfa); + preg->translate, (preg->syntax & RE_ICASE) != 0, + dfa); if (BE (err != REG_NOERROR, 0)) goto free_return; mctx.input.stop = stop; @@ -650,7 +678,8 @@ re_search_internal (const regex_t *preg, const char *string, int length, if (nmatch > 1 || dfa->has_mb_node) { /* Avoid overflow. */ - if (BE (SIZE_MAX / sizeof (re_dfastate_t *) <= mctx.input.bufs_len, 0)) + if (BE ((MIN (IDX_MAX, SIZE_MAX / sizeof (re_dfastate_t *)) + <= mctx.input.bufs_len), 0)) { err = REG_ESPACE; goto free_return; @@ -670,15 +699,15 @@ re_search_internal (const regex_t *preg, const char *string, int length, mctx.input.tip_context = (eflags & REG_NOTBOL) ? CONTEXT_BEGBUF : CONTEXT_NEWLINE | CONTEXT_BEGBUF; - /* Check incrementally whether of not the input string match. */ - incr = (range < 0) ? -1 : 1; - left_lim = (range < 0) ? start + range : start; - right_lim = (range < 0) ? start : start + range; + /* Check incrementally whether the input string matches. */ + incr = (last_start < start) ? -1 : 1; + left_lim = (last_start < start) ? last_start : start; + right_lim = (last_start < start) ? start : last_start; sb = dfa->mb_cur_max == 1; match_kind = (fastmap ? ((sb || !(preg->syntax & RE_ICASE || t) ? 4 : 0) - | (range >= 0 ? 2 : 0) + | (start <= last_start ? 2 : 0) | (t != NULL ? 1 : 0)) : 8); @@ -745,8 +774,8 @@ re_search_internal (const regex_t *preg, const char *string, int length, { /* If MATCH_FIRST is out of the valid range, reconstruct the buffers. */ - unsigned int offset = match_first - mctx.input.raw_mbs_idx; - if (BE (offset >= (unsigned int) mctx.input.valid_raw_len, 0)) + __re_size_t offset = match_first - mctx.input.raw_mbs_idx; + if (BE (offset >= (__re_size_t) mctx.input.valid_raw_len, 0)) { err = re_string_reconstruct (&mctx.input, match_first, eflags); @@ -788,7 +817,7 @@ re_search_internal (const regex_t *preg, const char *string, int length, /* We assume that the matching starts from 0. */ mctx.state_log_top = mctx.nbkref_ents = mctx.max_mb_elem_len = 0; match_last = check_matching (&mctx, fl_longest_match, - range >= 0 ? &match_first : NULL); + start <= last_start ? &match_first : NULL); if (match_last != -1) { if (BE (match_last == -2, 0)) @@ -831,7 +860,7 @@ re_search_internal (const regex_t *preg, const char *string, int length, /* Set pmatch[] if we need. */ if (nmatch > 0) { - int reg_idx; + Idx reg_idx; /* Initialize registers. */ for (reg_idx = 1; reg_idx < nmatch; ++reg_idx) @@ -840,6 +869,9 @@ re_search_internal (const regex_t *preg, const char *string, int length, /* Set the points where matching start/end. */ pmatch[0].rm_so = 0; pmatch[0].rm_eo = mctx.match_last; + /* FIXME: This function should fail if mctx.match_last exceeds + the maximum possible regoff_t value. We need a new error + code REG_OVERFLOW. */ if (!preg->no_sub && nmatch > 1) { @@ -903,7 +935,7 @@ __attribute_warn_unused_result__ prune_impossible_nodes (re_match_context_t *mctx) { const re_dfa_t *const dfa = mctx->dfa; - int halt_node, match_last; + Idx halt_node, match_last; reg_errcode_t ret; re_dfastate_t **sifted_states; re_dfastate_t **lim_states = NULL; @@ -915,7 +947,7 @@ prune_impossible_nodes (re_match_context_t *mctx) halt_node = mctx->last_node; /* Avoid overflow. */ - if (BE (SIZE_MAX / sizeof (re_dfastate_t *) <= match_last, 0)) + if (BE (MIN (IDX_MAX, SIZE_MAX / sizeof (re_dfastate_t *)) <= match_last, 0)) return REG_ESPACE; sifted_states = re_malloc (re_dfastate_t *, match_last + 1); @@ -995,9 +1027,9 @@ prune_impossible_nodes (re_match_context_t *mctx) since initial states may have constraints like "\<", "^", etc.. */ static inline re_dfastate_t * -__attribute ((always_inline)) +__attribute__ ((always_inline)) acquire_init_state_context (reg_errcode_t *err, const re_match_context_t *mctx, - int idx) + Idx idx) { const re_dfa_t *const dfa = mctx->dfa; if (dfa->init_state->has_constraint) @@ -1028,27 +1060,27 @@ acquire_init_state_context (reg_errcode_t *err, const re_match_context_t *mctx, } /* Check whether the regular expression match input string INPUT or not, - and return the index where the matching end, return -1 if not match, - or return -2 in case of an error. + and return the index where the matching end. Return -1 if + there is no match, and return -2 in case of an error. FL_LONGEST_MATCH means we want the POSIX longest matching. If P_MATCH_FIRST is not NULL, and the match fails, it is set to the next place where we may want to try matching. - Note that the matcher assume that the maching starts from the current + Note that the matcher assumes that the matching starts from the current index of the buffer. */ -static int +static Idx __attribute_warn_unused_result__ -check_matching (re_match_context_t *mctx, int fl_longest_match, - int *p_match_first) +check_matching (re_match_context_t *mctx, bool fl_longest_match, + Idx *p_match_first) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int match = 0; - int match_last = -1; - int cur_str_idx = re_string_cur_idx (&mctx->input); + Idx match = 0; + Idx match_last = -1; + Idx cur_str_idx = re_string_cur_idx (&mctx->input); re_dfastate_t *cur_state; - int at_init_state = p_match_first != NULL; - int next_start_idx = cur_str_idx; + bool at_init_state = p_match_first != NULL; + Idx next_start_idx = cur_str_idx; err = REG_NOERROR; cur_state = acquire_init_state_context (&err, mctx, cur_str_idx); @@ -1067,7 +1099,7 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, later. E.g. Processing back references. */ if (BE (dfa->nbackref, 0)) { - at_init_state = 0; + at_init_state = false; err = check_subexp_matching_top (mctx, &cur_state->nodes, 0); if (BE (err != REG_NOERROR, 0)) return err; @@ -1100,7 +1132,7 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, while (!re_string_eoi (&mctx->input)) { re_dfastate_t *old_state = cur_state; - int next_char_idx = re_string_cur_idx (&mctx->input) + 1; + Idx next_char_idx = re_string_cur_idx (&mctx->input) + 1; if ((BE (next_char_idx >= mctx->input.bufs_len, 0) && mctx->input.bufs_len < mctx->input.len) @@ -1138,7 +1170,7 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, if (old_state == cur_state) next_start_idx = next_char_idx; else - at_init_state = 0; + at_init_state = false; } if (cur_state->halt) @@ -1169,29 +1201,29 @@ check_matching (re_match_context_t *mctx, int fl_longest_match, /* Check NODE match the current context. */ -static int -check_halt_node_context (const re_dfa_t *dfa, int node, unsigned int context) +static bool +check_halt_node_context (const re_dfa_t *dfa, Idx node, unsigned int context) { re_token_type_t type = dfa->nodes[node].type; unsigned int constraint = dfa->nodes[node].constraint; if (type != END_OF_RE) - return 0; + return false; if (!constraint) - return 1; + return true; if (NOT_SATISFY_NEXT_CONSTRAINT (constraint, context)) - return 0; - return 1; + return false; + return true; } /* Check the halt state STATE match the current context. Return 0 if not match, if the node, STATE has, is a halt node and match the context, return the node. */ -static int +static Idx check_halt_state_context (const re_match_context_t *mctx, - const re_dfastate_t *state, int idx) + const re_dfastate_t *state, Idx idx) { - int i; + Idx i; unsigned int context; #ifdef DEBUG assert (state->halt); @@ -1205,31 +1237,33 @@ check_halt_state_context (const re_match_context_t *mctx, /* Compute the next node to which "NFA" transit from NODE("NFA" is a NFA corresponding to the DFA). - Return the destination node, and update EPS_VIA_NODES, return -1 in case - of errors. */ + Return the destination node, and update EPS_VIA_NODES; + return -1 in case of errors. */ -static int -proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, - int *pidx, int node, re_node_set *eps_via_nodes, +static Idx +proceed_next_node (const re_match_context_t *mctx, Idx nregs, regmatch_t *regs, + Idx *pidx, Idx node, re_node_set *eps_via_nodes, struct re_fail_stack_t *fs) { const re_dfa_t *const dfa = mctx->dfa; - int i, err; + Idx i; + bool ok; if (IS_EPSILON_NODE (dfa->nodes[node].type)) { re_node_set *cur_nodes = &mctx->state_log[*pidx]->nodes; re_node_set *edests = &dfa->edests[node]; - int dest_node; - err = re_node_set_insert (eps_via_nodes, node); - if (BE (err < 0, 0)) + Idx dest_node; + ok = re_node_set_insert (eps_via_nodes, node); + if (BE (! ok, 0)) return -2; - /* Pick up a valid destination, or return -1 if none is found. */ + /* Pick up a valid destination, or return -1 if none + is found. */ for (dest_node = -1, i = 0; i < edests->nelem; ++i) { - int candidate = edests->elems[i]; + Idx candidate = edests->elems[i]; if (!re_node_set_contains (cur_nodes, candidate)) continue; - if (dest_node == -1) + if (dest_node == -1) dest_node = candidate; else @@ -1253,7 +1287,7 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, } else { - int naccepted = 0; + Idx naccepted = 0; re_token_type_t type = dfa->nodes[node].type; #ifdef RE_ENABLE_I18N @@ -1263,7 +1297,7 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, #endif /* RE_ENABLE_I18N */ if (type == OP_BACK_REF) { - int subexp_idx = dfa->nodes[node].opr.idx + 1; + Idx subexp_idx = dfa->nodes[node].opr.idx + 1; naccepted = regs[subexp_idx].rm_eo - regs[subexp_idx].rm_so; if (fs != NULL) { @@ -1280,9 +1314,9 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, if (naccepted == 0) { - int dest_node; - err = re_node_set_insert (eps_via_nodes, node); - if (BE (err < 0, 0)) + Idx dest_node; + ok = re_node_set_insert (eps_via_nodes, node); + if (BE (! ok, 0)) return -2; dest_node = dfa->edests[node].elems[0]; if (re_node_set_contains (&mctx->state_log[*pidx]->nodes, @@ -1294,7 +1328,7 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, if (naccepted != 0 || check_node_accept (mctx, dfa->nodes + node, *pidx)) { - int dest_node = dfa->nexts[node]; + Idx dest_node = dfa->nexts[node]; *pidx = (naccepted == 0) ? *pidx + 1 : *pidx + naccepted; if (fs && (*pidx > mctx->match_last || mctx->state_log[*pidx] == NULL || !re_node_set_contains (&mctx->state_log[*pidx]->nodes, @@ -1309,16 +1343,16 @@ proceed_next_node (const re_match_context_t *mctx, int nregs, regmatch_t *regs, static reg_errcode_t __attribute_warn_unused_result__ -push_fail_stack (struct re_fail_stack_t *fs, int str_idx, int dest_node, - int nregs, regmatch_t *regs, re_node_set *eps_via_nodes) +push_fail_stack (struct re_fail_stack_t *fs, Idx str_idx, Idx dest_node, + Idx nregs, regmatch_t *regs, re_node_set *eps_via_nodes) { reg_errcode_t err; - int num = fs->num++; + Idx num = fs->num++; if (fs->num == fs->alloc) { struct re_fail_stack_ent_t *new_array; - new_array = realloc (fs->stack, (sizeof (struct re_fail_stack_ent_t) - * fs->alloc * 2)); + new_array = re_realloc (fs->stack, struct re_fail_stack_ent_t, + fs->alloc * 2); if (new_array == NULL) return REG_ESPACE; fs->alloc *= 2; @@ -1334,11 +1368,11 @@ push_fail_stack (struct re_fail_stack_t *fs, int str_idx, int dest_node, return err; } -static int -pop_fail_stack (struct re_fail_stack_t *fs, int *pidx, int nregs, +static Idx +pop_fail_stack (struct re_fail_stack_t *fs, Idx *pidx, Idx nregs, regmatch_t *regs, re_node_set *eps_via_nodes) { - int num = --fs->num; + Idx num = --fs->num; assert (num >= 0); *pidx = fs->stack[num].idx; memcpy (regs, fs->stack[num].regs, sizeof (regmatch_t) * nregs); @@ -1356,15 +1390,15 @@ pop_fail_stack (struct re_fail_stack_t *fs, int *pidx, int nregs, static reg_errcode_t __attribute_warn_unused_result__ set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, - regmatch_t *pmatch, int fl_backtrack) + regmatch_t *pmatch, bool fl_backtrack) { - const re_dfa_t *dfa = (const re_dfa_t *) preg->buffer; - int idx, cur_node; + const re_dfa_t *dfa = preg->buffer; + Idx idx, cur_node; re_node_set eps_via_nodes; struct re_fail_stack_t *fs; struct re_fail_stack_t fs_body = { 0, 2, NULL }; regmatch_t *prev_idx_match; - int prev_idx_match_malloced = 0; + bool prev_idx_match_malloced = false; #ifdef DEBUG assert (nmatch > 1); @@ -1393,7 +1427,7 @@ set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, free_fail_stack_return (fs); return REG_ESPACE; } - prev_idx_match_malloced = 1; + prev_idx_match_malloced = true; } memcpy (prev_idx_match, pmatch, sizeof (regmatch_t) * nmatch); @@ -1403,7 +1437,7 @@ set_regs (const regex_t *preg, const re_match_context_t *mctx, size_t nmatch, if (idx == pmatch[0].rm_eo && cur_node == mctx->last_node) { - int reg_idx; + Idx reg_idx; if (fs) { for (reg_idx = 0; reg_idx < nmatch; ++reg_idx) @@ -1465,7 +1499,7 @@ free_fail_stack_return (struct re_fail_stack_t *fs) { if (fs) { - int fs_idx; + Idx fs_idx; for (fs_idx = 0; fs_idx < fs->num; ++fs_idx) { re_node_set_free (&fs->stack[fs_idx].eps_via_nodes); @@ -1478,12 +1512,12 @@ free_fail_stack_return (struct re_fail_stack_t *fs) static void update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, - regmatch_t *prev_idx_match, int cur_node, int cur_idx, int nmatch) + regmatch_t *prev_idx_match, Idx cur_node, Idx cur_idx, Idx nmatch) { int type = dfa->nodes[cur_node].type; if (type == OP_OPEN_SUBEXP) { - int reg_num = dfa->nodes[cur_node].opr.idx + 1; + Idx reg_num = dfa->nodes[cur_node].opr.idx + 1; /* We are at the first node of this sub expression. */ if (reg_num < nmatch) @@ -1494,7 +1528,7 @@ update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, } else if (type == OP_CLOSE_SUBEXP) { - int reg_num = dfa->nodes[cur_node].opr.idx + 1; + Idx reg_num = dfa->nodes[cur_node].opr.idx + 1; if (reg_num < nmatch) { /* We are at the last node of this sub expression. */ @@ -1528,21 +1562,21 @@ update_regs (const re_dfa_t *dfa, regmatch_t *pmatch, and sift the nodes in each states according to the following rules. Updated state_log will be wrote to STATE_LOG. - Rules: We throw away the Node `a' in the STATE_LOG[STR_IDX] if... + Rules: We throw away the Node 'a' in the STATE_LOG[STR_IDX] if... 1. When STR_IDX == MATCH_LAST(the last index in the state_log): - If `a' isn't the LAST_NODE and `a' can't epsilon transit to - the LAST_NODE, we throw away the node `a'. - 2. When 0 <= STR_IDX < MATCH_LAST and `a' accepts - string `s' and transit to `b': + If 'a' isn't the LAST_NODE and 'a' can't epsilon transit to + the LAST_NODE, we throw away the node 'a'. + 2. When 0 <= STR_IDX < MATCH_LAST and 'a' accepts + string 's' and transit to 'b': i. If 'b' isn't in the STATE_LOG[STR_IDX+strlen('s')], we throw - away the node `a'. + away the node 'a'. ii. If 'b' is in the STATE_LOG[STR_IDX+strlen('s')] but 'b' is - thrown away, we throw away the node `a'. + thrown away, we throw away the node 'a'. 3. When 0 <= STR_IDX < MATCH_LAST and 'a' epsilon transit to 'b': i. If 'b' isn't in the STATE_LOG[STR_IDX], we throw away the - node `a'. + node 'a'. ii. If 'b' is in the STATE_LOG[STR_IDX] but 'b' is thrown away, - we throw away the node `a'. */ + we throw away the node 'a'. */ #define STATE_NODE_CONTAINS(state,node) \ ((state) != NULL && re_node_set_contains (&(state)->nodes, node)) @@ -1552,7 +1586,7 @@ sift_states_backward (const re_match_context_t *mctx, re_sift_context_t *sctx) { reg_errcode_t err; int null_cnt = 0; - int str_idx = sctx->last_str_idx; + Idx str_idx = sctx->last_str_idx; re_node_set cur_dest; #ifdef DEBUG @@ -1607,31 +1641,31 @@ sift_states_backward (const re_match_context_t *mctx, re_sift_context_t *sctx) static reg_errcode_t __attribute_warn_unused_result__ build_sifted_states (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, re_node_set *cur_dest) + Idx str_idx, re_node_set *cur_dest) { const re_dfa_t *const dfa = mctx->dfa; const re_node_set *cur_src = &mctx->state_log[str_idx]->non_eps_nodes; - int i; + Idx i; /* Then build the next sifted state. - We build the next sifted state on `cur_dest', and update - `sifted_states[str_idx]' with `cur_dest'. + We build the next sifted state on 'cur_dest', and update + 'sifted_states[str_idx]' with 'cur_dest'. Note: - `cur_dest' is the sifted state from `state_log[str_idx + 1]'. - `cur_src' points the node_set of the old `state_log[str_idx]' + 'cur_dest' is the sifted state from 'state_log[str_idx + 1]'. + 'cur_src' points the node_set of the old 'state_log[str_idx]' (with the epsilon nodes pre-filtered out). */ for (i = 0; i < cur_src->nelem; i++) { - int prev_node = cur_src->elems[i]; + Idx prev_node = cur_src->elems[i]; int naccepted = 0; - int ret; + bool ok; #ifdef DEBUG re_token_type_t type = dfa->nodes[prev_node].type; assert (!IS_EPSILON_NODE (type)); #endif #ifdef RE_ENABLE_I18N - /* If the node may accept `multi byte'. */ + /* If the node may accept "multi byte". */ if (dfa->nodes[prev_node].accept_mb) naccepted = sift_states_iter_mb (mctx, sctx, prev_node, str_idx, sctx->last_str_idx); @@ -1650,14 +1684,14 @@ build_sifted_states (const re_match_context_t *mctx, re_sift_context_t *sctx, if (sctx->limits.nelem) { - int to_idx = str_idx + naccepted; + Idx to_idx = str_idx + naccepted; if (check_dst_limits (mctx, &sctx->limits, dfa->nexts[prev_node], to_idx, prev_node, str_idx)) continue; } - ret = re_node_set_insert (cur_dest, prev_node); - if (BE (ret == -1, 0)) + ok = re_node_set_insert (cur_dest, prev_node); + if (BE (! ok, 0)) return REG_ESPACE; } @@ -1667,9 +1701,9 @@ build_sifted_states (const re_match_context_t *mctx, re_sift_context_t *sctx, /* Helper functions. */ static reg_errcode_t -clean_state_log_if_needed (re_match_context_t *mctx, int next_state_log_idx) +clean_state_log_if_needed (re_match_context_t *mctx, Idx next_state_log_idx) { - int top = mctx->state_log_top; + Idx top = mctx->state_log_top; if ((next_state_log_idx >= mctx->input.bufs_len && mctx->input.bufs_len < mctx->input.len) @@ -1693,9 +1727,9 @@ clean_state_log_if_needed (re_match_context_t *mctx, int next_state_log_idx) static reg_errcode_t merge_state_array (const re_dfa_t *dfa, re_dfastate_t **dst, - re_dfastate_t **src, int num) + re_dfastate_t **src, Idx num) { - int st_idx; + Idx st_idx; reg_errcode_t err; for (st_idx = 0; st_idx < num; ++st_idx) { @@ -1719,7 +1753,7 @@ merge_state_array (const re_dfa_t *dfa, re_dfastate_t **dst, static reg_errcode_t update_cur_sifted_state (const re_match_context_t *mctx, - re_sift_context_t *sctx, int str_idx, + re_sift_context_t *sctx, Idx str_idx, re_node_set *dest_nodes) { const re_dfa_t *const dfa = mctx->dfa; @@ -1770,7 +1804,7 @@ add_epsilon_src_nodes (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates) { reg_errcode_t err = REG_NOERROR; - int i; + Idx i; re_dfastate_t *state = re_acquire_state (&err, dfa, dest_nodes); if (BE (err != REG_NOERROR, 0)) @@ -1794,23 +1828,23 @@ add_epsilon_src_nodes (const re_dfa_t *dfa, re_node_set *dest_nodes, } static reg_errcode_t -sub_epsilon_src_nodes (const re_dfa_t *dfa, int node, re_node_set *dest_nodes, +sub_epsilon_src_nodes (const re_dfa_t *dfa, Idx node, re_node_set *dest_nodes, const re_node_set *candidates) { - int ecl_idx; + Idx ecl_idx; reg_errcode_t err; re_node_set *inv_eclosure = dfa->inveclosures + node; re_node_set except_nodes; re_node_set_init_empty (&except_nodes); for (ecl_idx = 0; ecl_idx < inv_eclosure->nelem; ++ecl_idx) { - int cur_node = inv_eclosure->elems[ecl_idx]; + Idx cur_node = inv_eclosure->elems[ecl_idx]; if (cur_node == node) continue; if (IS_EPSILON_NODE (dfa->nodes[cur_node].type)) { - int edst1 = dfa->edests[cur_node].elems[0]; - int edst2 = ((dfa->edests[cur_node].nelem > 1) + Idx edst1 = dfa->edests[cur_node].elems[0]; + Idx edst2 = ((dfa->edests[cur_node].nelem > 1) ? dfa->edests[cur_node].elems[1] : -1); if ((!re_node_set_contains (inv_eclosure, edst1) && re_node_set_contains (dest_nodes, edst1)) @@ -1830,10 +1864,10 @@ sub_epsilon_src_nodes (const re_dfa_t *dfa, int node, re_node_set *dest_nodes, } for (ecl_idx = 0; ecl_idx < inv_eclosure->nelem; ++ecl_idx) { - int cur_node = inv_eclosure->elems[ecl_idx]; + Idx cur_node = inv_eclosure->elems[ecl_idx]; if (!re_node_set_contains (&except_nodes, cur_node)) { - int idx = re_node_set_contains (dest_nodes, cur_node) - 1; + Idx idx = re_node_set_contains (dest_nodes, cur_node) - 1; re_node_set_remove_at (dest_nodes, idx); } } @@ -1841,18 +1875,18 @@ sub_epsilon_src_nodes (const re_dfa_t *dfa, int node, re_node_set *dest_nodes, return REG_NOERROR; } -static int -check_dst_limits (const re_match_context_t *mctx, re_node_set *limits, - int dst_node, int dst_idx, int src_node, int src_idx) +static bool +check_dst_limits (const re_match_context_t *mctx, const re_node_set *limits, + Idx dst_node, Idx dst_idx, Idx src_node, Idx src_idx) { const re_dfa_t *const dfa = mctx->dfa; - int lim_idx, src_pos, dst_pos; + Idx lim_idx, src_pos, dst_pos; - int dst_bkref_idx = search_cur_bkref_entry (mctx, dst_idx); - int src_bkref_idx = search_cur_bkref_entry (mctx, src_idx); + Idx dst_bkref_idx = search_cur_bkref_entry (mctx, dst_idx); + Idx src_bkref_idx = search_cur_bkref_entry (mctx, src_idx); for (lim_idx = 0; lim_idx < limits->nelem; ++lim_idx) { - int subexp_idx; + Idx subexp_idx; struct re_backref_cache_entry *ent; ent = mctx->bkref_ents + limits->elems[lim_idx]; subexp_idx = dfa->nodes[ent->node].opr.idx; @@ -1871,24 +1905,24 @@ check_dst_limits (const re_match_context_t *mctx, re_node_set *limits, if (src_pos == dst_pos) continue; /* This is unrelated limitation. */ else - return 1; + return true; } - return 0; + return false; } static int check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, int boundaries, - int subexp_idx, int from_node, int bkref_idx) + Idx subexp_idx, Idx from_node, Idx bkref_idx) { const re_dfa_t *const dfa = mctx->dfa; const re_node_set *eclosures = dfa->eclosures + from_node; - int node_idx; + Idx node_idx; /* Else, we are on the boundary: examine the nodes on the epsilon closure. */ for (node_idx = 0; node_idx < eclosures->nelem; ++node_idx) { - int node = eclosures->elems[node_idx]; + Idx node = eclosures->elems[node_idx]; switch (dfa->nodes[node].type) { case OP_BACK_REF: @@ -1897,7 +1931,8 @@ check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, int boundaries, struct re_backref_cache_entry *ent = mctx->bkref_ents + bkref_idx; do { - int dst, cpos; + Idx dst; + int cpos; if (ent->node != node) continue; @@ -1957,9 +1992,9 @@ check_dst_limits_calc_pos_1 (const re_match_context_t *mctx, int boundaries, } static int -check_dst_limits_calc_pos (const re_match_context_t *mctx, int limit, - int subexp_idx, int from_node, int str_idx, - int bkref_idx) +check_dst_limits_calc_pos (const re_match_context_t *mctx, Idx limit, + Idx subexp_idx, Idx from_node, Idx str_idx, + Idx bkref_idx) { struct re_backref_cache_entry *lim = mctx->bkref_ents + limit; int boundaries; @@ -1988,14 +2023,14 @@ check_dst_limits_calc_pos (const re_match_context_t *mctx, int limit, static reg_errcode_t check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, const re_node_set *candidates, re_node_set *limits, - struct re_backref_cache_entry *bkref_ents, int str_idx) + struct re_backref_cache_entry *bkref_ents, Idx str_idx) { reg_errcode_t err; - int node_idx, lim_idx; + Idx node_idx, lim_idx; for (lim_idx = 0; lim_idx < limits->nelem; ++lim_idx) { - int subexp_idx; + Idx subexp_idx; struct re_backref_cache_entry *ent; ent = bkref_ents + limits->elems[lim_idx]; @@ -2005,11 +2040,11 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, subexp_idx = dfa->nodes[ent->node].opr.idx; if (ent->subexp_to == str_idx) { - int ops_node = -1; - int cls_node = -1; + Idx ops_node = -1; + Idx cls_node = -1; for (node_idx = 0; node_idx < dest_nodes->nelem; ++node_idx) { - int node = dest_nodes->elems[node_idx]; + Idx node = dest_nodes->elems[node_idx]; re_token_type_t type = dfa->nodes[node].type; if (type == OP_OPEN_SUBEXP && subexp_idx == dfa->nodes[node].opr.idx) @@ -2033,7 +2068,7 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, if (cls_node >= 0) for (node_idx = 0; node_idx < dest_nodes->nelem; ++node_idx) { - int node = dest_nodes->elems[node_idx]; + Idx node = dest_nodes->elems[node_idx]; if (!re_node_set_contains (dfa->inveclosures + node, cls_node) && !re_node_set_contains (dfa->eclosures + node, @@ -2053,7 +2088,7 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, { for (node_idx = 0; node_idx < dest_nodes->nelem; ++node_idx) { - int node = dest_nodes->elems[node_idx]; + Idx node = dest_nodes->elems[node_idx]; re_token_type_t type = dfa->nodes[node].type; if (type == OP_CLOSE_SUBEXP || type == OP_OPEN_SUBEXP) { @@ -2075,13 +2110,13 @@ check_subexp_limits (const re_dfa_t *dfa, re_node_set *dest_nodes, static reg_errcode_t __attribute_warn_unused_result__ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, - int str_idx, const re_node_set *candidates) + Idx str_idx, const re_node_set *candidates) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int node_idx, node; + Idx node_idx, node; re_sift_context_t local_sctx; - int first_idx = search_cur_bkref_entry (mctx, str_idx); + Idx first_idx = search_cur_bkref_entry (mctx, str_idx); if (first_idx == -1) return REG_NOERROR; @@ -2090,7 +2125,7 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, for (node_idx = 0; node_idx < candidates->nelem; ++node_idx) { - int enabled_idx; + Idx enabled_idx; re_token_type_t type; struct re_backref_cache_entry *entry; node = candidates->elems[node_idx]; @@ -2105,10 +2140,10 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, enabled_idx = first_idx; do { - int subexp_len; - int to_idx; - int dst_node; - int ret; + Idx subexp_len; + Idx to_idx; + Idx dst_node; + bool ok; re_dfastate_t *cur_state; if (entry->node != node) @@ -2134,8 +2169,8 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, } local_sctx.last_node = node; local_sctx.last_str_idx = str_idx; - ret = re_node_set_insert (&local_sctx.limits, enabled_idx); - if (BE (ret < 0, 0)) + ok = re_node_set_insert (&local_sctx.limits, enabled_idx); + if (BE (! ok, 0)) { err = REG_ESPACE; goto free_return; @@ -2174,21 +2209,21 @@ sift_states_bkref (const re_match_context_t *mctx, re_sift_context_t *sctx, #ifdef RE_ENABLE_I18N static int sift_states_iter_mb (const re_match_context_t *mctx, re_sift_context_t *sctx, - int node_idx, int str_idx, int max_str_idx) + Idx node_idx, Idx str_idx, Idx max_str_idx) { const re_dfa_t *const dfa = mctx->dfa; int naccepted; - /* Check the node can accept `multi byte'. */ + /* Check the node can accept "multi byte". */ naccepted = check_node_accept_bytes (dfa, node_idx, &mctx->input, str_idx); if (naccepted > 0 && str_idx + naccepted <= max_str_idx && !STATE_NODE_CONTAINS (sctx->sifted_states[str_idx + naccepted], dfa->nexts[node_idx])) - /* The node can't accept the `multi byte', or the + /* The node can't accept the "multi byte", or the destination was already thrown away, then the node - could't accept the current input `multi byte'. */ + could't accept the current input "multi byte". */ naccepted = 0; /* Otherwise, it is sure that the node could accept - `naccepted' bytes input. */ + 'naccepted' bytes input. */ return naccepted; } #endif /* RE_ENABLE_I18N */ @@ -2259,12 +2294,12 @@ transit_state (reg_errcode_t *err, re_match_context_t *mctx, } /* Update the state_log if we need */ -re_dfastate_t * +static re_dfastate_t * merge_state_with_log (reg_errcode_t *err, re_match_context_t *mctx, re_dfastate_t *next_state) { const re_dfa_t *const dfa = mctx->dfa; - int cur_idx = re_string_cur_idx (&mctx->input); + Idx cur_idx = re_string_cur_idx (&mctx->input); if (cur_idx > mctx->state_log_top) { @@ -2337,14 +2372,14 @@ merge_state_with_log (reg_errcode_t *err, re_match_context_t *mctx, /* Skip bytes in the input that correspond to part of a multi-byte match, then look in the log for a state from which to restart matching. */ -re_dfastate_t * +static re_dfastate_t * find_recover_state (reg_errcode_t *err, re_match_context_t *mctx) { re_dfastate_t *cur_state; do { - int max = mctx->state_log_top; - int cur_str_idx = re_string_cur_idx (&mctx->input); + Idx max = mctx->state_log_top; + Idx cur_str_idx = re_string_cur_idx (&mctx->input); do { @@ -2369,10 +2404,10 @@ find_recover_state (reg_errcode_t *err, re_match_context_t *mctx) static reg_errcode_t check_subexp_matching_top (re_match_context_t *mctx, re_node_set *cur_nodes, - int str_idx) + Idx str_idx) { const re_dfa_t *const dfa = mctx->dfa; - int node_idx; + Idx node_idx; reg_errcode_t err; /* TODO: This isn't efficient. @@ -2382,7 +2417,7 @@ check_subexp_matching_top (re_match_context_t *mctx, re_node_set *cur_nodes, E.g. RE: (a){2} */ for (node_idx = 0; node_idx < cur_nodes->nelem; ++node_idx) { - int node = cur_nodes->elems[node_idx]; + Idx node = cur_nodes->elems[node_idx]; if (dfa->nodes[node].type == OP_OPEN_SUBEXP && dfa->nodes[node].opr.idx < BITSET_WORD_BITS && (dfa->used_bkref_map @@ -2407,7 +2442,7 @@ transit_state_sb (reg_errcode_t *err, re_match_context_t *mctx, const re_dfa_t *const dfa = mctx->dfa; re_node_set next_nodes; re_dfastate_t *next_state; - int node_cnt, cur_str_idx = re_string_cur_idx (&mctx->input); + Idx node_cnt, cur_str_idx = re_string_cur_idx (&mctx->input); unsigned int context; *err = re_node_set_alloc (&next_nodes, state->nodes.nelem + 1); @@ -2415,7 +2450,7 @@ transit_state_sb (reg_errcode_t *err, re_match_context_t *mctx, return NULL; for (node_cnt = 0; node_cnt < state->nodes.nelem; ++node_cnt) { - int cur_node = state->nodes.elems[node_cnt]; + Idx cur_node = state->nodes.elems[node_cnt]; if (check_node_accept (mctx, dfa->nodes + cur_node, cur_str_idx)) { *err = re_node_set_merge (&next_nodes, @@ -2444,13 +2479,14 @@ transit_state_mb (re_match_context_t *mctx, re_dfastate_t *pstate) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int i; + Idx i; for (i = 0; i < pstate->nodes.nelem; ++i) { re_node_set dest_nodes, *new_nodes; - int cur_node_idx = pstate->nodes.elems[i]; - int naccepted, dest_idx; + Idx cur_node_idx = pstate->nodes.elems[i]; + int naccepted; + Idx dest_idx; unsigned int context; re_dfastate_t *dest_state; @@ -2473,7 +2509,7 @@ transit_state_mb (re_match_context_t *mctx, re_dfastate_t *pstate) if (naccepted == 0) continue; - /* The node can accepts `naccepted' bytes. */ + /* The node can accepts 'naccepted' bytes. */ dest_idx = re_string_cur_idx (&mctx->input) + naccepted; mctx->max_mb_elem_len = ((mctx->max_mb_elem_len < naccepted) ? naccepted : mctx->max_mb_elem_len); @@ -2513,18 +2549,18 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int i; - int cur_str_idx = re_string_cur_idx (&mctx->input); + Idx i; + Idx cur_str_idx = re_string_cur_idx (&mctx->input); for (i = 0; i < nodes->nelem; ++i) { - int dest_str_idx, prev_nelem, bkc_idx; - int node_idx = nodes->elems[i]; + Idx dest_str_idx, prev_nelem, bkc_idx; + Idx node_idx = nodes->elems[i]; unsigned int context; const re_token_t *node = dfa->nodes + node_idx; re_node_set *new_dest_nodes; - /* Check whether `node' is a backreference or not. */ + /* Check whether 'node' is a backreference or not. */ if (node->type != OP_BACK_REF) continue; @@ -2536,21 +2572,21 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) continue; } - /* `node' is a backreference. + /* 'node' is a backreference. Check the substring which the substring matched. */ bkc_idx = mctx->nbkref_ents; err = get_subexp (mctx, node_idx, cur_str_idx); if (BE (err != REG_NOERROR, 0)) goto free_return; - /* And add the epsilon closures (which is `new_dest_nodes') of + /* And add the epsilon closures (which is 'new_dest_nodes') of the backreference to appropriate state_log. */ #ifdef DEBUG assert (dfa->nexts[node_idx] != -1); #endif for (; bkc_idx < mctx->nbkref_ents; ++bkc_idx) { - int subexp_len; + Idx subexp_len; re_dfastate_t *dest_state; struct re_backref_cache_entry *bkref_ent; bkref_ent = mctx->bkref_ents + bkc_idx; @@ -2567,7 +2603,7 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) dest_state = mctx->state_log[dest_str_idx]; prev_nelem = ((mctx->state_log[cur_str_idx] == NULL) ? 0 : mctx->state_log[cur_str_idx]->nodes.nelem); - /* Add `new_dest_node' to state_log. */ + /* Add 'new_dest_node' to state_log. */ if (dest_state == NULL) { mctx->state_log[dest_str_idx] @@ -2623,13 +2659,13 @@ transit_state_bkref (re_match_context_t *mctx, const re_node_set *nodes) static reg_errcode_t __attribute_warn_unused_result__ -get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) +get_subexp (re_match_context_t *mctx, Idx bkref_node, Idx bkref_str_idx) { const re_dfa_t *const dfa = mctx->dfa; - int subexp_num, sub_top_idx; + Idx subexp_num, sub_top_idx; const char *buf = (const char *) re_string_get_buffer (&mctx->input); /* Return if we have already checked BKREF_NODE at BKREF_STR_IDX. */ - int cache_idx = search_cur_bkref_entry (mctx, bkref_str_idx); + Idx cache_idx = search_cur_bkref_entry (mctx, bkref_str_idx); if (cache_idx != -1) { const struct re_backref_cache_entry *entry @@ -2648,7 +2684,7 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) reg_errcode_t err; re_sub_match_top_t *sub_top = mctx->sub_tops[sub_top_idx]; re_sub_match_last_t *sub_last; - int sub_last_idx, sl_str, bkref_str_off; + Idx sub_last_idx, sl_str, bkref_str_off; if (dfa->nodes[sub_top->node].opr.idx != subexp_num) continue; /* It isn't related. */ @@ -2659,7 +2695,7 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) evaluated. */ for (sub_last_idx = 0; sub_last_idx < sub_top->nlasts; ++sub_last_idx) { - int sl_str_diff; + regoff_t sl_str_diff; sub_last = sub_top->lasts[sub_last_idx]; sl_str_diff = sub_last->str_idx - sl_str; /* The matched string by the sub expression match with the substring @@ -2705,7 +2741,8 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) /* Then, search for the other last nodes of the sub expression. */ for (; sl_str <= bkref_str_idx; ++sl_str) { - int cls_node, sl_str_off; + Idx cls_node; + regoff_t sl_str_off; const re_node_set *nodes; sl_str_off = sl_str - sub_top->str_idx; /* The matched string by the sub expression match with the substring @@ -2772,10 +2809,10 @@ get_subexp (re_match_context_t *mctx, int bkref_node, int bkref_str_idx) static reg_errcode_t get_subexp_sub (re_match_context_t *mctx, const re_sub_match_top_t *sub_top, - re_sub_match_last_t *sub_last, int bkref_node, int bkref_str) + re_sub_match_last_t *sub_last, Idx bkref_node, Idx bkref_str) { reg_errcode_t err; - int to_idx; + Idx to_idx; /* Can the subexpression arrive the back reference? */ err = check_arrival (mctx, &sub_last->path, sub_last->node, sub_last->str_idx, bkref_node, bkref_str, @@ -2798,14 +2835,14 @@ get_subexp_sub (re_match_context_t *mctx, const re_sub_match_top_t *sub_top, nodes. E.g. RE: (a){2} */ -static int +static Idx find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, - int subexp_idx, int type) + Idx subexp_idx, int type) { - int cls_idx; + Idx cls_idx; for (cls_idx = 0; cls_idx < nodes->nelem; ++cls_idx) { - int cls_node = nodes->elems[cls_idx]; + Idx cls_node = nodes->elems[cls_idx]; const re_token_t *node = dfa->nodes + cls_node; if (node->type == type && node->opr.idx == subexp_idx) @@ -2821,12 +2858,12 @@ find_subexp_node (const re_dfa_t *dfa, const re_node_set *nodes, static reg_errcode_t __attribute_warn_unused_result__ -check_arrival (re_match_context_t *mctx, state_array_t *path, int top_node, - int top_str, int last_node, int last_str, int type) +check_arrival (re_match_context_t *mctx, state_array_t *path, Idx top_node, + Idx top_str, Idx last_node, Idx last_str, int type) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err = REG_NOERROR; - int subexp_num, backup_cur_idx, str_idx, null_cnt; + Idx subexp_num, backup_cur_idx, str_idx, null_cnt; re_dfastate_t *cur_state = NULL; re_node_set *cur_nodes, next_nodes; re_dfastate_t **backup_state_log; @@ -2837,20 +2874,24 @@ check_arrival (re_match_context_t *mctx, state_array_t *path, int top_node, if (BE (path->alloc < last_str + mctx->max_mb_elem_len + 1, 0)) { re_dfastate_t **new_array; - int old_alloc = path->alloc; - path->alloc += last_str + mctx->max_mb_elem_len + 1; - new_array = re_realloc (path->array, re_dfastate_t *, path->alloc); + Idx old_alloc = path->alloc; + Idx incr_alloc = last_str + mctx->max_mb_elem_len + 1; + Idx new_alloc; + if (BE (IDX_MAX - old_alloc < incr_alloc, 0)) + return REG_ESPACE; + new_alloc = old_alloc + incr_alloc; + if (BE (SIZE_MAX / sizeof (re_dfastate_t *) < new_alloc, 0)) + return REG_ESPACE; + new_array = re_realloc (path->array, re_dfastate_t *, new_alloc); if (BE (new_array == NULL, 0)) - { - path->alloc = old_alloc; - return REG_ESPACE; - } + return REG_ESPACE; path->array = new_array; + path->alloc = new_alloc; memset (new_array + old_alloc, '\0', sizeof (re_dfastate_t *) * (path->alloc - old_alloc)); } - str_idx = path->next_idx ?: top_str; + str_idx = path->next_idx ? path->next_idx : top_str; /* Temporary modify MCTX. */ backup_state_log = mctx->state_log; @@ -2982,12 +3023,12 @@ check_arrival (re_match_context_t *mctx, state_array_t *path, int top_node, static reg_errcode_t __attribute_warn_unused_result__ -check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, +check_arrival_add_next_nodes (re_match_context_t *mctx, Idx str_idx, re_node_set *cur_nodes, re_node_set *next_nodes) { const re_dfa_t *const dfa = mctx->dfa; - int result; - int cur_idx; + bool ok; + Idx cur_idx; #ifdef RE_ENABLE_I18N reg_errcode_t err = REG_NOERROR; #endif @@ -2996,13 +3037,13 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, for (cur_idx = 0; cur_idx < cur_nodes->nelem; ++cur_idx) { int naccepted = 0; - int cur_node = cur_nodes->elems[cur_idx]; + Idx cur_node = cur_nodes->elems[cur_idx]; #ifdef DEBUG re_token_type_t type = dfa->nodes[cur_node].type; assert (!IS_EPSILON_NODE (type)); #endif #ifdef RE_ENABLE_I18N - /* If the node may accept `multi byte'. */ + /* If the node may accept "multi byte". */ if (dfa->nodes[cur_node].accept_mb) { naccepted = check_node_accept_bytes (dfa, cur_node, &mctx->input, @@ -3010,8 +3051,8 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, if (naccepted > 1) { re_dfastate_t *dest_state; - int next_node = dfa->nexts[cur_node]; - int next_idx = str_idx + naccepted; + Idx next_node = dfa->nexts[cur_node]; + Idx next_idx = str_idx + naccepted; dest_state = mctx->state_log[next_idx]; re_node_set_empty (&union_set); if (dest_state) @@ -3023,8 +3064,8 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, return err; } } - result = re_node_set_insert (&union_set, next_node); - if (BE (result < 0, 0)) + ok = re_node_set_insert (&union_set, next_node); + if (BE (! ok, 0)) { re_node_set_free (&union_set); return REG_ESPACE; @@ -3043,8 +3084,8 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, if (naccepted || check_node_accept (mctx, dfa->nodes + cur_node, str_idx)) { - result = re_node_set_insert (next_nodes, dfa->nexts[cur_node]); - if (BE (result < 0, 0)) + ok = re_node_set_insert (next_nodes, dfa->nexts[cur_node]); + if (BE (! ok, 0)) { re_node_set_free (&union_set); return REG_ESPACE; @@ -3063,10 +3104,10 @@ check_arrival_add_next_nodes (re_match_context_t *mctx, int str_idx, static reg_errcode_t check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, - int ex_subexp, int type) + Idx ex_subexp, int type) { reg_errcode_t err; - int idx, outside_node; + Idx idx, outside_node; re_node_set new_nodes; #ifdef DEBUG assert (cur_nodes->nelem); @@ -3079,7 +3120,7 @@ check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, for (idx = 0; idx < cur_nodes->nelem; ++idx) { - int cur_node = cur_nodes->elems[idx]; + Idx cur_node = cur_nodes->elems[idx]; const re_node_set *eclosure = dfa->eclosures + cur_node; outside_node = find_subexp_node (dfa, eclosure, ex_subexp, type); if (outside_node == -1) @@ -3116,31 +3157,32 @@ check_arrival_expand_ecl (const re_dfa_t *dfa, re_node_set *cur_nodes, static reg_errcode_t __attribute_warn_unused_result__ check_arrival_expand_ecl_sub (const re_dfa_t *dfa, re_node_set *dst_nodes, - int target, int ex_subexp, int type) + Idx target, Idx ex_subexp, int type) { - int cur_node; + Idx cur_node; for (cur_node = target; !re_node_set_contains (dst_nodes, cur_node);) { - int err; + bool ok; if (dfa->nodes[cur_node].type == type && dfa->nodes[cur_node].opr.idx == ex_subexp) { if (type == OP_CLOSE_SUBEXP) { - err = re_node_set_insert (dst_nodes, cur_node); - if (BE (err == -1, 0)) + ok = re_node_set_insert (dst_nodes, cur_node); + if (BE (! ok, 0)) return REG_ESPACE; } break; } - err = re_node_set_insert (dst_nodes, cur_node); - if (BE (err == -1, 0)) + ok = re_node_set_insert (dst_nodes, cur_node); + if (BE (! ok, 0)) return REG_ESPACE; if (dfa->edests[cur_node].nelem == 0) break; if (dfa->edests[cur_node].nelem == 2) { + reg_errcode_t err; err = check_arrival_expand_ecl_sub (dfa, dst_nodes, dfa->edests[cur_node].elems[1], ex_subexp, type); @@ -3160,11 +3202,11 @@ check_arrival_expand_ecl_sub (const re_dfa_t *dfa, re_node_set *dst_nodes, static reg_errcode_t __attribute_warn_unused_result__ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, - int cur_str, int subexp_num, int type) + Idx cur_str, Idx subexp_num, int type) { const re_dfa_t *const dfa = mctx->dfa; reg_errcode_t err; - int cache_idx_start = search_cur_bkref_entry (mctx, cur_str); + Idx cache_idx_start = search_cur_bkref_entry (mctx, cur_str); struct re_backref_cache_entry *ent; if (cache_idx_start == -1) @@ -3174,7 +3216,7 @@ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, ent = mctx->bkref_ents + cache_idx_start; do { - int to_idx, next_node; + Idx to_idx, next_node; /* Is this entry ENT is appropriate? */ if (!re_node_set_contains (cur_nodes, ent->node)) @@ -3212,14 +3254,14 @@ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, next_node = dfa->nexts[ent->node]; if (mctx->state_log[to_idx]) { - int ret; + bool ok; if (re_node_set_contains (&mctx->state_log[to_idx]->nodes, next_node)) continue; err = re_node_set_init_copy (&union_set, &mctx->state_log[to_idx]->nodes); - ret = re_node_set_insert (&union_set, next_node); - if (BE (err != REG_NOERROR || ret < 0, 0)) + ok = re_node_set_insert (&union_set, next_node); + if (BE (err != REG_NOERROR || ! ok, 0)) { re_node_set_free (&union_set); err = err != REG_NOERROR ? err : REG_ESPACE; @@ -3244,17 +3286,19 @@ expand_bkref_cache (re_match_context_t *mctx, re_node_set *cur_nodes, } /* Build transition table for the state. - Return 1 if succeeded, otherwise return NULL. */ + Return true if successful. */ -static int +static bool build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) { reg_errcode_t err; - int i, j, ch, need_word_trtable = 0; + Idx i, j; + int ch; + bool need_word_trtable = false; bitset_word_t elem, mask; bool dests_node_malloced = false; bool dest_states_malloced = false; - int ndests; /* Number of the destination states from `state'. */ + Idx ndests; /* Number of the destination states from 'state'. */ re_dfastate_t **trtable; re_dfastate_t **dest_states = NULL, **dest_states_word, **dest_states_nl; re_node_set follows, *dests_node; @@ -3268,8 +3312,8 @@ build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) } *dests_alloc; /* We build DFA states which corresponds to the destination nodes - from `state'. `dests_node[i]' represents the nodes which i-th - destination state contains, and `dests_ch[i]' represents the + from 'state'. 'dests_node[i]' represents the nodes which i-th + destination state contains, and 'dests_ch[i]' represents the characters which i-th destination state accepts. */ if (__libc_use_alloca (sizeof (struct dests_alloc))) dests_alloc = (struct dests_alloc *) alloca (sizeof (struct dests_alloc)); @@ -3277,32 +3321,32 @@ build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) { dests_alloc = re_malloc (struct dests_alloc, 1); if (BE (dests_alloc == NULL, 0)) - return 0; + return false; dests_node_malloced = true; } dests_node = dests_alloc->dests_node; dests_ch = dests_alloc->dests_ch; - /* Initialize transiton table. */ + /* Initialize transition table. */ state->word_trtable = state->trtable = NULL; - /* At first, group all nodes belonging to `state' into several + /* At first, group all nodes belonging to 'state' into several destinations. */ ndests = group_nodes_into_DFAstates (dfa, state, dests_node, dests_ch); if (BE (ndests <= 0, 0)) { if (dests_node_malloced) - free (dests_alloc); - /* Return 0 in case of an error, 1 otherwise. */ + re_free (dests_alloc); + /* Return false in case of an error, true otherwise. */ if (ndests == 0) { state->trtable = (re_dfastate_t **) calloc (sizeof (re_dfastate_t *), SBC_MAX); - if (BE (state->trtable == NULL, 0)) - return 0; - return 1; + if (BE (state->trtable == NULL, 0)) + return false; + return true; } - return 0; + return false; } err = re_node_set_alloc (&follows, ndests + 1); @@ -3322,19 +3366,18 @@ build_trtable (const re_dfa_t *dfa, re_dfastate_t *state) alloca (ndests * 3 * sizeof (re_dfastate_t *)); else { - dest_states = (re_dfastate_t **) - malloc (ndests * 3 * sizeof (re_dfastate_t *)); + dest_states = re_malloc (re_dfastate_t *, ndests * 3); if (BE (dest_states == NULL, 0)) { out_free: if (dest_states_malloced) - free (dest_states); + re_free (dest_states); re_node_set_free (&follows); for (i = 0; i < ndests; ++i) re_node_set_free (dests_node + i); if (dests_node_malloced) - free (dests_alloc); - return 0; + re_free (dests_alloc); + return false; } dest_states_malloced = true; } @@ -3345,7 +3388,7 @@ out_free: /* Then build the states for all destinations. */ for (i = 0; i < ndests; ++i) { - int next_node; + Idx next_node; re_node_set_empty (&follows); /* Merge the follows of this destination states. */ for (j = 0; j < dests_node[i].nelem; ++j) @@ -3371,13 +3414,13 @@ out_free: goto out_free; if (dest_states[i] != dest_states_word[i] && dfa->mb_cur_max > 1) - need_word_trtable = 1; + need_word_trtable = true; dest_states_nl[i] = re_acquire_state_context (&err, dfa, &follows, CONTEXT_NEWLINE); if (BE (dest_states_nl[i] == NULL && err != REG_NOERROR, 0)) goto out_free; - } + } else { dest_states_word[i] = dest_states[i]; @@ -3464,16 +3507,16 @@ out_free: } if (dest_states_malloced) - free (dest_states); + re_free (dest_states); re_node_set_free (&follows); for (i = 0; i < ndests; ++i) re_node_set_free (dests_node + i); if (dests_node_malloced) - free (dests_alloc); + re_free (dests_alloc); - return 1; + return true; } /* Group all nodes belonging to STATE into several destinations. @@ -3481,20 +3524,20 @@ out_free: to DESTS_NODE[i] and set the characters accepted by the destination to DEST_CH[i]. This function return the number of destinations. */ -static int +static Idx group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, re_node_set *dests_node, bitset_t *dests_ch) { reg_errcode_t err; - int result; - int i, j, k; - int ndests; /* Number of the destinations from `state'. */ + bool ok; + Idx i, j, k; + Idx ndests; /* Number of the destinations from 'state'. */ bitset_t accepts; /* Characters a node can accept. */ const re_node_set *cur_nodes = &state->nodes; bitset_empty (accepts); ndests = 0; - /* For all the nodes belonging to `state', */ + /* For all the nodes belonging to 'state', */ for (i = 0; i < cur_nodes->nelem; ++i) { re_token_t *node = &dfa->nodes[cur_nodes->elems[i]]; @@ -3524,7 +3567,10 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, #ifdef RE_ENABLE_I18N else if (type == OP_UTF8_PERIOD) { - memset (accepts, '\xff', sizeof (bitset_t) / 2); + if (ASCII_CHARS % BITSET_WORD_BITS == 0) + memset (accepts, -1, ASCII_CHARS / CHAR_BIT); + else + bitset_merge (accepts, utf8_sb_map); if (!(dfa->syntax & RE_DOT_NEWLINE)) bitset_clear (accepts, '\n'); if (dfa->syntax & RE_DOT_NOT_NULL) @@ -3534,7 +3580,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, else continue; - /* Check the `accepts' and sift the characters which are not + /* Check the 'accepts' and sift the characters which are not match it the context. */ if (constraint) { @@ -3593,7 +3639,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, } } - /* Then divide `accepts' into DFA states, or create a new + /* Then divide 'accepts' into DFA states, or create a new state. Above, we make sure that accepts is not empty. */ for (j = 0; j < ndests; ++j) { @@ -3606,7 +3652,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, if (type == CHARACTER && !bitset_contain (dests_ch[j], node->opr.c)) continue; - /* Enumerate the intersection set of this state and `accepts'. */ + /* Enumerate the intersection set of this state and 'accepts'. */ has_intersec = 0; for (k = 0; k < BITSET_WORDS; ++k) has_intersec |= intersec[k] = accepts[k] & dests_ch[j][k]; @@ -3614,7 +3660,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, if (!has_intersec) continue; - /* Then check if this state is a subset of `accepts'. */ + /* Then check if this state is a subset of 'accepts'. */ not_subset = not_consumed = 0; for (k = 0; k < BITSET_WORDS; ++k) { @@ -3622,8 +3668,8 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, not_consumed |= accepts[k] = accepts[k] & ~dests_ch[j][k]; } - /* If this state isn't a subset of `accepts', create a - new group state, which has the `remains'. */ + /* If this state isn't a subset of 'accepts', create a + new group state, which has the 'remains'. */ if (not_subset) { bitset_copy (dests_ch[ndests], remains); @@ -3635,8 +3681,8 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, } /* Put the position in the current group. */ - result = re_node_set_insert (&dests_node[j], cur_nodes->elems[i]); - if (BE (result < 0, 0)) + ok = re_node_set_insert (&dests_node[j], cur_nodes->elems[i]); + if (BE (! ok, 0)) goto error_return; /* If all characters are consumed, go to next node. */ @@ -3662,7 +3708,7 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, } #ifdef RE_ENABLE_I18N -/* Check how many bytes the node `dfa->nodes[node_idx]' accepts. +/* Check how many bytes the node 'dfa->nodes[node_idx]' accepts. Return the number of the bytes the node accepts. STR_IDX is the current index of the input string. @@ -3675,12 +3721,12 @@ group_nodes_into_DFAstates (const re_dfa_t *dfa, const re_dfastate_t *state, # endif static int -check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, - const re_string_t *input, int str_idx) +check_node_accept_bytes (const re_dfa_t *dfa, Idx node_idx, + const re_string_t *input, Idx str_idx) { const re_token_t *node = dfa->nodes + node_idx; int char_len, elem_len; - int i; + Idx i; if (BE (node->type == OP_UTF8_PERIOD, 0)) { @@ -3759,7 +3805,7 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, # ifdef _LIBC const unsigned char *pin = ((const unsigned char *) re_string_get_buffer (input) + str_idx); - int j; + Idx j; uint32_t nrules; # endif /* _LIBC */ int match_len = 0; @@ -3827,6 +3873,7 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, in_collseq = find_collation_sequence_value (pin, elem_len); } /* match with range expression? */ + /* FIXME: Implement rational ranges here, too. */ for (i = 0; i < cset->nranges; ++i) if (cset->range_starts[i] <= in_collseq && in_collseq <= cset->range_ends[i]) @@ -3856,7 +3903,7 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, if (weight_len == weights[equiv_class_idx & 0xffffff] && (idx >> 24) == (equiv_class_idx >> 24)) { - int cnt = 0; + Idx cnt = 0; idx &= 0xffffff; equiv_class_idx &= 0xffffff; @@ -3878,18 +3925,9 @@ check_node_accept_bytes (const re_dfa_t *dfa, int node_idx, # endif /* _LIBC */ { /* match with range expression? */ -#if __GNUC__ >= 2 - wchar_t cmp_buf[] = {L'\0', L'\0', wc, L'\0', L'\0', L'\0'}; -#else - wchar_t cmp_buf[] = {L'\0', L'\0', L'\0', L'\0', L'\0', L'\0'}; - cmp_buf[2] = wc; -#endif for (i = 0; i < cset->nranges; ++i) { - cmp_buf[0] = cset->range_starts[i]; - cmp_buf[4] = cset->range_ends[i]; - if (__wcscoll (cmp_buf, cmp_buf + 2) <= 0 - && __wcscoll (cmp_buf + 2, cmp_buf + 4) <= 0) + if (cset->range_starts[i] <= wc && wc <= cset->range_ends[i]) { match_len = char_len; goto check_node_accept_bytes_match; @@ -3936,7 +3974,8 @@ find_collation_sequence_value (const unsigned char *mbs, size_t mbs_len) for (idx = 0; idx < extrasize;) { - int mbs_cnt, found = 0; + int mbs_cnt; + bool found = false; int32_t elem_mbs_len; /* Skip the name of collating element name. */ idx = idx + extra[idx] + 1; @@ -3948,7 +3987,7 @@ find_collation_sequence_value (const unsigned char *mbs, size_t mbs_len) break; if (mbs_cnt == elem_mbs_len) /* Found the entry. */ - found = 1; + found = true; } /* Skip the byte sequence of the collating element. */ idx += elem_mbs_len; @@ -3973,9 +4012,9 @@ find_collation_sequence_value (const unsigned char *mbs, size_t mbs_len) /* Check whether the node accepts the byte which is IDX-th byte of the INPUT. */ -static int +static bool check_node_accept (const re_match_context_t *mctx, const re_token_t *node, - int idx) + Idx idx) { unsigned char ch; ch = re_string_byte_at (&mctx->input, idx); @@ -3983,28 +4022,28 @@ check_node_accept (const re_match_context_t *mctx, const re_token_t *node, { case CHARACTER: if (node->opr.c != ch) - return 0; + return false; break; case SIMPLE_BRACKET: if (!bitset_contain (node->opr.sbcset, ch)) - return 0; + return false; break; #ifdef RE_ENABLE_I18N case OP_UTF8_PERIOD: - if (ch >= 0x80) - return 0; - /* FALLTHROUGH */ + if (ch >= ASCII_CHARS) + return false; + FALLTHROUGH; #endif case OP_PERIOD: if ((ch == '\n' && !(mctx->dfa->syntax & RE_DOT_NEWLINE)) || (ch == '\0' && (mctx->dfa->syntax & RE_DOT_NOT_NULL))) - return 0; + return false; break; default: - return 0; + return false; } if (node->constraint) @@ -4014,10 +4053,10 @@ check_node_accept (const re_match_context_t *mctx, const re_token_t *node, unsigned int context = re_string_context_at (&mctx->input, idx, mctx->eflags); if (NOT_SATISFY_NEXT_CONSTRAINT (node->constraint, context)) - return 0; + return false; } - return 1; + return true; } /* Extend the buffers, if the buffers have run out. */ @@ -4030,10 +4069,11 @@ extend_buffers (re_match_context_t *mctx, int min_len) re_string_t *pstr = &mctx->input; /* Avoid overflow. */ - if (BE (INT_MAX / 2 / sizeof (re_dfastate_t *) <= pstr->bufs_len, 0)) + if (BE (MIN (IDX_MAX, SIZE_MAX / sizeof (re_dfastate_t *)) / 2 + <= pstr->bufs_len, 0)) return REG_ESPACE; - /* Double the lengthes of the buffers, but allocate at least MIN_LEN. */ + /* Double the lengths of the buffers, but allocate at least MIN_LEN. */ ret = re_string_realloc_buffers (pstr, MAX (min_len, MIN (pstr->len, pstr->bufs_len * 2))); @@ -4089,12 +4129,19 @@ extend_buffers (re_match_context_t *mctx, int min_len) static reg_errcode_t __attribute_warn_unused_result__ -match_ctx_init (re_match_context_t *mctx, int eflags, int n) +match_ctx_init (re_match_context_t *mctx, int eflags, Idx n) { mctx->eflags = eflags; mctx->match_last = -1; if (n > 0) { + /* Avoid overflow. */ + size_t max_object_size = + MAX (sizeof (struct re_backref_cache_entry), + sizeof (re_sub_match_top_t *)); + if (BE (MIN (IDX_MAX, SIZE_MAX / max_object_size) < n, 0)) + return REG_ESPACE; + mctx->bkref_ents = re_malloc (struct re_backref_cache_entry, n); mctx->sub_tops = re_malloc (re_sub_match_top_t *, n); if (BE (mctx->bkref_ents == NULL || mctx->sub_tops == NULL, 0)) @@ -4118,10 +4165,10 @@ match_ctx_init (re_match_context_t *mctx, int eflags, int n) static void match_ctx_clean (re_match_context_t *mctx) { - int st_idx; + Idx st_idx; for (st_idx = 0; st_idx < mctx->nsub_tops; ++st_idx) { - int sl_idx; + Idx sl_idx; re_sub_match_top_t *top = mctx->sub_tops[st_idx]; for (sl_idx = 0; sl_idx < top->nlasts; ++sl_idx) { @@ -4135,7 +4182,7 @@ match_ctx_clean (re_match_context_t *mctx) re_free (top->path->array); re_free (top->path); } - free (top); + re_free (top); } mctx->nsub_tops = 0; @@ -4160,8 +4207,8 @@ match_ctx_free (re_match_context_t *mctx) static reg_errcode_t __attribute_warn_unused_result__ -match_ctx_add_entry (re_match_context_t *mctx, int node, int str_idx, int from, - int to) +match_ctx_add_entry (re_match_context_t *mctx, Idx node, Idx str_idx, Idx from, + Idx to) { if (mctx->nbkref_ents >= mctx->abkref_ents) { @@ -4196,7 +4243,7 @@ match_ctx_add_entry (re_match_context_t *mctx, int node, int str_idx, int from, A backreference does not epsilon-transition unless it is empty, so set to all zeros if FROM != TO. */ mctx->bkref_ents[mctx->nbkref_ents].eps_reachable_subexps_map - = (from == to ? ~0 : 0); + = (from == to ? -1 : 0); mctx->bkref_ents[mctx->nbkref_ents++].more = 0; if (mctx->max_mb_elem_len < to - from) @@ -4204,13 +4251,13 @@ match_ctx_add_entry (re_match_context_t *mctx, int node, int str_idx, int from, return REG_NOERROR; } -/* Search for the first entry which has the same str_idx, or -1 if none is +/* Return the first entry with the same str_idx, or -1 if none is found. Note that MCTX->BKREF_ENTS is already sorted by MCTX->STR_IDX. */ -static int -search_cur_bkref_entry (const re_match_context_t *mctx, int str_idx) +static Idx +search_cur_bkref_entry (const re_match_context_t *mctx, Idx str_idx) { - int left, right, mid, last; + Idx left, right, mid, last; last = right = mctx->nbkref_ents; for (left = 0; left < right;) { @@ -4231,7 +4278,7 @@ search_cur_bkref_entry (const re_match_context_t *mctx, int str_idx) static reg_errcode_t __attribute_warn_unused_result__ -match_ctx_add_subtop (re_match_context_t *mctx, int node, int str_idx) +match_ctx_add_subtop (re_match_context_t *mctx, Idx node, Idx str_idx) { #ifdef DEBUG assert (mctx->sub_tops != NULL); @@ -4239,7 +4286,7 @@ match_ctx_add_subtop (re_match_context_t *mctx, int node, int str_idx) #endif if (BE (mctx->nsub_tops == mctx->asub_tops, 0)) { - int new_asub_tops = mctx->asub_tops * 2; + Idx new_asub_tops = mctx->asub_tops * 2; re_sub_match_top_t **new_array = re_realloc (mctx->sub_tops, re_sub_match_top_t *, new_asub_tops); @@ -4260,12 +4307,12 @@ match_ctx_add_subtop (re_match_context_t *mctx, int node, int str_idx) at STR_IDX, whose corresponding OP_OPEN_SUBEXP is SUB_TOP. */ static re_sub_match_last_t * -match_ctx_add_sublast (re_sub_match_top_t *subtop, int node, int str_idx) +match_ctx_add_sublast (re_sub_match_top_t *subtop, Idx node, Idx str_idx) { re_sub_match_last_t *new_entry; if (BE (subtop->nlasts == subtop->alasts, 0)) { - int new_alasts = 2 * subtop->alasts + 1; + Idx new_alasts = 2 * subtop->alasts + 1; re_sub_match_last_t **new_array = re_realloc (subtop->lasts, re_sub_match_last_t *, new_alasts); @@ -4287,7 +4334,7 @@ match_ctx_add_sublast (re_sub_match_top_t *subtop, int node, int str_idx) static void sift_ctx_init (re_sift_context_t *sctx, re_dfastate_t **sifted_sts, - re_dfastate_t **limited_sts, int last_node, int last_str_idx) + re_dfastate_t **limited_sts, Idx last_node, Idx last_str_idx) { sctx->sifted_states = sifted_sts; sctx->limited_states = limited_sts;
This patch syncs regex code with gnulib (commit 16aa5a2). The main differences are: * regcomp: add a __libc_lock_init init for dfa lock. * regexec: instead of use intprops.h and its macros, this patch the check_add_overflow_idx function to check for addition overflow. The function need to be defined on the file because the file will be build multiple times wit 'Idx' type being redefined (I think intprops.h addition should be done in a separate patch). * regex.h: Remove _Restrict_ and _Restrict_arr_ definition based on __STDC_VERSION__ because its usage leads to failures on posix/check-installed-headers-c{xx}. * regex_internal.h: Define lock_fini to empty macro because setting to 0 lead to build issues (error: statement with no effect [-Werror=unused-value]). Also the prototypes of 're_string_realloc_buffers', 'build_wcs_buffer', 'build_wcs_upper_buffer', 'build_upper_buffer', 're_string_translate_buffer', 're_string_context_at' are kept in its implementation counterpart because of internal tests building. The same internal testing requires the 'bitset_*' macros to be 'static inline' instead of just 'static' plus attribute used in gnulib code. Only two changes in GLIBC regex testing are required: 1. posix/bug-regex28.c: as previously discussed [1] the change of expected results on the pattern should be safe. 2. posix/PCRE.tests: the test '(a)|\1' uses a backreference along with group creation and I am not sure if it is the correct behavior to accept it with regcomp (REG_EXTENDED). The GNU grep accepts it with ERE option though. This sync contains some patches from thread 'Regex: Make libc regex more usable outside GLIBC.' [2] which have been pushed upstream in gnulib. This patches also fixes some regex issues (BZ #23233, BZ #21163, BZ #18986, BZ #13762) and I did not add testcases for both #23233 and #13762 because I couldn't think a simple way to trigger the expected failure path to trigger them. Checked on x86_64-linux-gnu and i686-linux-gnu. [BZ #23233] [BZ #21163] [BZ #18986] [BZ #13762] * posix/Makefile (tests): Add bug-regex37 and bug-regex38. * posix/PCRE.tests: Remove invalid test. * posix/bug-regex28.c: Fix expected values for used syntax. * posix/bug-regex37.c: New file. * posix/bug-regex38.c: Likewise. * posix/regcomp.c: Sync with gnulib. * posix/regex.c: Likewise. * posix/regex.h: Likewise. * posix/regex_internal.c: Likewise. * posix/regex_internal.h: Likewise. * posix/regexec.c: Likewise. [1] https://sourceware.org/ml/libc-alpha/2017-12/msg00807.html [2] https://sourceware.org/ml/libc-alpha/2017-12/msg00237.html --- ChangeLog | 18 + posix/Makefile | 3 +- posix/PCRE.tests | 13 - posix/bug-regex28.c | 46 +-- posix/bug-regex37.c | 32 ++ posix/bug-regex38.c | 32 ++ posix/regcomp.c | 597 ++++++++++++++++++------------- posix/regex.c | 19 +- posix/regex.h | 324 ++++++++++------- posix/regex_internal.c | 293 ++++++++------- posix/regex_internal.h | 424 ++++++++++++++-------- posix/regexec.c | 951 ++++++++++++++++++++++++++----------------------- 12 files changed, 1596 insertions(+), 1156 deletions(-) create mode 100644 posix/bug-regex37.c create mode 100644 posix/bug-regex38.c -- 2.7.4