PHPRegex Optimizer

Rewrites regex patterns into shorter equivalents and modernizes old syntax; equivalence can be proven, opt-in, by regex-automata.

Documentation: php-regex.com — the API reference documents optimize(), and the correctness contracts spell out what each rewrite must prove.

Features

• [0-9] becomes \d; character class members are sorted, merged and deduplicated
• Repeated items collapse into a quantifier, /aaaa/ becoming /a{4}/, from min_quantifier_count repetitions on
• Alternatives sharing a prefix share it, /abc|abd/ becoming /ab(?:c|d)/ — opt-in
• A quantifier becomes possessive where nothing after it can take back what it matched, /[0-9]+;/ becoming /\d++;/ — opt-in
• Each rewrite can be proven by the automata engine to write the same $matches, or at least to match the same strings, before it is offered — opt-in
• RedosRepairer repairs a pattern open to catastrophic backtracking, /href="([^"]+)*"/ becoming /href="([^"]*)"/, each repair proven linear and, when the automata can tell, proven to match the same subjects
• A Modernizer visitor cleans legacy syntax: unneeded escapes dropped, redundant non-capturing groups unwrapped, \1 written \g{1}
• A pattern that cannot be improved comes back unchanged, with an empty change log

Installation

composer require php-regex/regex-optimizer

Requires PHP 8.1+. MIT licensed. php-regex/regex-parser, and the php-regex/regex-automata engine the equivalence proof runs on, are installed with it. This package is part of PHPRegex, released with its siblings under one version number.

Configuration

optimize() takes an OptimizerOptions value, or an array it reads. Array keys are the snake_case names below; regex.json and the PHPStan parameters spell the same options in camelCase (OptimizerOptions::fromCamelCaseArray()).

Option	Type	Default	What it does
digits	bool	true	[0-9] becomes \d
word	bool	true	[a-zA-Z0-9_] becomes \w where the two match alike: never today, see below
ranges	bool	true	runs of consecutive characters become ranges
canonicalize_char_classes	bool	true	members of a class are sorted and merged
possessive	bool	false	a quantifier becomes possessive where nothing after it can take the match back
factorize	bool	false	alternatives sharing a prefix share it
min_quantifier_count	int	4	how many repetitions of an item become a quantifier, at least 2
verify_with_automata	bool	true	a rewrite is only offered when the automata prove it writes the same $matches, or, where the match solver does not read it, when it reads as the pattern does or each part it changed is proven alone

possessive and factorize are off by default: a backreference can make a possessive quantifier change what the pattern matches, and factorization makes an /x pattern harder to read. verify_with_automata is on, as it is for the lint's suggestions, the PHPStan rule and the editor's quick fix: a rewrite it disproves, or cannot prove within a budget of its own, is withdrawn, the original pattern returned. The proof builds automata for both patterns: on some 600 patterns gathered from open-source projects it made optimize() about two and a half times as slow on average, about a second for the slowest, and withdrew none of the rewrites. Turned off, the rewrites ship as the rewriter writes them, but one that adds a possessive quantifier or an atomic group, which is proven whatever the option says.

A rewrite holds whatever locale the program runs under. Without u, PHP builds the tables of \w, \s and the POSIX classes from LC_CTYPE once a program calls setlocale(): under a Latin-1 locale \w takes "\xE9", which [a-zA-Z0-9_] does not, and with u it takes every letter. Nothing in a pattern says which locale it runs under, so word writes no \w today. [0-9] becomes \d under every locale, as the C library's digits are the ten ASCII ones, but not under u or (*UCP), where \d matches every decimal digit.

Under /x the rewrite keeps the pattern as its author laid it out: only the parts it changes are written again, in place, and the newlines, indents and comments around them stay. The lint suggests for such a pattern what optimize() gives it with factorize off, whatever its options say, so the alternatives keep the order and the lines the author gave them.

A rewrite compiles wherever the original does, for the target the parser judges for: the automata read what a pattern matches, never whether PCRE compiles it, so a rewrite PCRE would refuse, as (?<=abc|abde) factorized into (?<=ab(?:c|de)) for PHP 8.2, whose PCRE2 refuses a lookbehind branch of two lengths, is withdrawn whatever the options say. A DEFINE group keeps its quantifier as written, and a run of the same item is counted up to 65535, the largest count PCRE takes.

An unknown key or a value of the wrong type throws InvalidRegexOptionException. The 1.x names autoPossessify and allowAlternationFactorization were renamed possessive and factorize: the old names meet the same unknown-option error as any typo.

Usage

use PHPRegex\Optimizer\Optimizer;
use PHPRegex\Parser\RegexParser;

$optimizer = new Optimizer(RegexParser::create());

$result = $optimizer->optimize('/[0-9][0-9][0-9][0-9]/');

echo $result->optimized;   // '/\d{4}/'
echo $result->changes[0];  // 'Wrote [0-9] as \d.'

Each rewrite is opt-out: turn off the ones you do not want, keep the rest.

$result = $optimizer->optimize('/[abc][0-9]/', ['digits' => false]);

echo $result->optimized;  // '/[a-c][0-9]/'

The same options as a value object, turning on the two off-by-default rewrites:

use PHPRegex\Optimizer\OptimizerOptions;

$options = new OptimizerOptions(possessive: true, factorize: true);

echo $optimizer->optimize('/[0-9]+;/', $options)->optimized;  // '/\d++;/'
echo $optimizer->optimize('/abc|abd/', $options)->optimized;  // '/ab(?:c|d)/'

The instance keeps the solver its automata checks ask: keep one instance for a batch of patterns rather than building one per call. The check of a rewrite builds no DFA to keep between calls, so the constructor takes the parser alone.

To take the rewrite unproven, faster:

$result = $optimizer->optimize('/[0-9][0-9][0-9][0-9]/', ['verify_with_automata' => false]);

echo $result->optimized;  // '/\d{4}/'

The Modernizer visitor rewrites old syntax on the AST; render its result with the parser's printer:

use PHPRegex\Optimizer\Modernizer;
use PHPRegex\Parser\Printer\PatternPrinter;
use PHPRegex\Parser\RegexParser;

$parser = RegexParser::create();
$modernized = $parser->parse('/a\-(?:[0-9])/')->accept(new Modernizer());

echo $modernized->accept(PatternPrinter::asWritten());  // '/a-\d/'

[\t\n\x0B\f\r ] becomes \s only when the Modernizer is told the pattern runs under the C or POSIX locale, new Modernizer('C'): without u, most other locales give \s the no-break space, byte 0xA0, which the class does not hold.

Documentation

• API reference — optimize() in full: both option spellings and OptimizationResult
• Correctness contracts — what the optimizer guarantees, and when it declines a rewrite
• Backward compatibility promise — the public surface of this package and what stays stable within 2.x
• Feature support matrix — which constructs each analyzer, the optimizer included, handles

Resources

• Upgrading from 1.x — moving from yoeunes/regex-parser 1.3 to 2.0, every change that needs one on your side
• The batteries-included facade: regex-toolkit
• The equivalence engine: regex-automata
• Documentation
• Changelog
• Report issues and send pull requests in the main PHPRegex repository

Sponsors

If PHPRegex saves you time, consider sponsoring its maintenance.

License

MIT. See LICENSE.