Skip to content

Shadow MutatingScope, ClassReflection and ExpressionResult natively - #6509

Merged
ondrejmirtes merged 9 commits into
2.3.xfrom
turbo-native-scope
Sep 22, 2026
Merged

Shadow MutatingScope, ClassReflection and ExpressionResult natively#6509
ondrejmirtes merged 9 commits into
2.3.xfrom
turbo-native-scope

Conversation

@ondrejmirtes

@ondrejmirtes ondrejmirtes commented Sep 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Makes MutatingScope, ClassReflection, ExpressionResult, PhpClassReflectionExtension and the variable-flow helpers native classes in a production run. Apart from the #[ShadowedByTurboExtension] / #[ReferencedByTurboExtension] attributes, the PHP twins are unchanged.

MutatingScope and ClassReflection: port, then flip

Each of these two classes lands in two commits. The first ports every method, in the twin's file order, and registers the class with the new reg::Class::shadowDifferentialOnly(): the class is declared only under the prefixed activation of the differential tests, never under the real name, so production keeps the PHP twin. The second commit flips it to shadow(): it adds the #[ShadowedByTurboExtension] attribute and moves the declarations from the hand-kept *Declarations.h into the generated header. side-by-side.php skips differential-only classes, since no attribute names them yet.

Each port is checked differentially against its twin:

  • tests/scope-family.php rebuilds real walk scopes on both sides and compares 64,669 observations over 80 scopes. It was mutation-checked with 39 deliberate native bugs; 37 were caught, and the other two sit in arms the twin cannot reach either.
  • tests/reflection-family.php builds 68 class reflections (fixture classes, stubbed built-ins, and withTypes/withVariances/asFinal derivations) as twin and native, and compares 41,140 observations, memo slots included.
  • tests/php-class-reflection-family.php covers PhpClassReflectionExtension: every public method over inherited, trait, magic, promoted, hooked, attributed, enum, interface and signature-mapped members, in and out of class scope, plus the member-cache memo and its eviction.

ClassReflectionAccess.cpp is deleted. Its slot readers are re-exported from ClassReflection.cpp as direct calls into the native bodies, so the Type-kernel call sites keep their spelling. The flip also exposed a latent bug: the typeGetClassReflection() shortcut ran ObjectType's body for a GenericObjectType, which overrides it. It now dispatches through the receiver's own op entry.

Other native classes

  • ExpressionResult (29 methods). Its state lives in the twin's property slots and its constructor keeps the twin's exact arginfo, so Nette still generates ExpressionResultFactory from it.
  • VolatileExpressionHelper, VariableFlow, VariableFlowBuilder and VariableLivenessResolver. The resolver keeps its per-call state in the twin's exact array shapes, so every set union and every key and insertion order matches the twin's.
  • PhpClassReflectionExtension, the member factory behind every ClassReflection::has*()/get*() call (hundreds of thousands of calls per self-analysis). Its hot path answers hasMethod()/hasProperty()/getMethod() from the BetterReflection adapter's member memos instead of three PHP frames. In 8 interleaved ABBA pairs this is −1.16% user CPU, with byte-identical output.
  • ExpressionResultStorageStack.

Accessor hops

Several PHP frames stood between the native scope and values it already owns. They are now read from property slots:

  • ExpressionResultStorageStack::getCurrent(): 1.0M calls per self-analysis.
  • LazyClassReflectionExtensionRegistryProvider::getRegistry() and the registry getter behind it: 1.3M calls each.
  • LazyInternalScopeFactory::create(): 1.45M calls.

The PHP methods still answer while a memo is unset, for any other implementation, and under the prefixed differential activation.

Verification

Each commit builds under -Wall -Wextra -Werror, keeps the generated declarations current, and passes side-by-side.php, signature-parity.php and smoke.php (including the differential scope and reflection families) on its own. On the last commit, with the extension loaded and active, make tests, make phpstan, make cs and make lint are clean, and a strict profile-guided g++-15 build passes.

🤖 Generated with Claude Code

https://claude.ai/code/session_017MvPby652L7wUqGAHEiEcN

@ondrejmirtes
ondrejmirtes force-pushed the turbo-native-scope branch 3 times, most recently from 06160e7 to fe72499 Compare September 22, 2026 08:47
ondrejmirtes and others added 9 commits September 22, 2026 11:04
@ondrejmirtes @claude
Analyser helper ports: VolatileExpressionHelper, VariableFlow, Variab…
d14a749 
…leFlowBuilder, VariableLivenessResolver

Shadow VolatileExpressionHelper, VariableFlow and VariableFlowBuilder natively

VolatileExpressionHelper works on the scope's by-reference expression tables
in place, separating a table only when an entry is actually removed (an eager
zpp separation would duplicate the whole table on every no-op call).
VariableFlow is declared abstract (new reg::Class::abstract_()) with the
readonly $kind slot its PHP subclasses fill through the protected constructor;
its factories instantiate the PHP flow classes through the class map (new
keys: the four flow classes, VariableWrite, VariableWriteOffset, List_,
StaticPropertyFetch). VariableFlowBuilder reads the node structure from the
node properties and takes stored results from the native
ExpressionResultStorage directly (pt_expression_result_storage_find; the PHP
collaborators ExpressionResult, ArgsResult, MutatingScope and
VariableWriteOffset stay engine calls). reg.h gains by-ref array, nullable
variadic object, nullable bool and public string / private array constant
builders.

Shadow VariableLivenessResolver natively

The resolver's only public entry point is the static resolve(): the native
class keeps the twin's per-call state on the C stack as PHP arrays in the
twin's exact shapes (so every set union, key order and insertion order is
the twin's), walks the PHP flow classes through their property slots, reads
VariableWrite through its slots (pt_variable_write_slots_of; the getters for
anything else) and answers the catch-clause Type queries through the Type
ops. The PHP VariableWritesNode is instantiated through the class map (new
key variableWritesNode).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MShHKdUB19w38vboJLPKXy
@ondrejmirtes @claude
Shadow ExpressionResult natively
1183b25 
Ports PHPStan\Analyser\ExpressionResult (29 methods) to turbo-ext/src/
ExpressionResult.cpp. The state lives in the twin's property slots in
declaration order and the constructor keeps the twin's exact arginfo, so
Nette still generates ExpressionResultFactory from it; the type/specify/
create callbacks, MutatingScope and the extensions collection are called
through the engine, while the Type queries, late-resolvable resolution and
the void->null projection go through the native Type ops and TypeTraverser.
VariableFlowBuilder now reads a result's flow through the new
pt_expression_result_variable_flow() slot reader instead of a method call.
Adds the class-map keys typeSpecifierContext, voidToNullTraverser,
issetabilityResolution, issetabilityLinkInfo and nullsafeMethodCall, and
lets reg::callableArg() be nullable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MShHKdUB19w38vboJLPKXy
@ondrejmirtes @claude
Port ClassReflection natively behind a differential-only plan
77409a0 
Ports every method of PHPStan\Reflection\ClassReflection to
turbo-ext/src/ClassReflection.cpp, in the twin's file order: the class
layout (54 instance slots in the twin's declaration order plus the static
$resolvingTypeAliasImports; the `string|false|null = false` memo slots are
TypedFalse properties), the 19-parameter constructor with the twin's exact
service class names, direct C++ $this dispatch (the twin is final; the
final flag joins the plan with the flip), the ancestors, the enum
methods, traits, the constants family, the type-alias machinery,
deprecation/internal/immutable/consistent-constructor/named-arguments, the
attribute-class flags, the generics machinery (template type maps,
call-site variance maps, withTypes/withVariances/asFinal), the resolved
PHPDoc and its tag getters, resolved mixin types and allowed subtypes.

The Type kernel is reached natively, with instanceof against a shadowed
Type class accepting the twin under the prefixed harness. The
VerbosityLevel / TemplateTypeVariance singletons and the two statics that
classify a Type by its class entry, TemplateTypeScope::createWithClass()
and TemplateTypeFactory::fromTemplateTag(), go through the classes' real
names (kernelSingleton() / kernelStatic()), which under the prefixed
differential activation are the PHP ones. TypehintHelper::
decideTypeFromReflection() gained a native export so the twin's
ClassReflection parameter type never sees the native $this. Twelve
class-map keys support the port: initializerExprContext, the
Extended*/Wrapped* method and property reflections, enumCaseReflection,
reflectionEnumBackedCase, realClassClassConstantReflection, typeAlias,
circularTypeAliasDefinitionException, argumentsNormalizer and arg.

The plan is differential-only: reg::Class::shadowDifferentialOnly(), a new
ShadowPlan flag Shadow.cpp honours, declares the class only under the
prefixed activation of the differential tests, never under the real name,
so production keeps the PHP twin and the ClassReflectionAccess.cpp slot
readers until the flip. side-by-side.php skips such a class, as no
attribute names it yet. The generator renders declarations for shadowed
classes only, so the port's generated declarations are kept next to it in
turbo-ext/src/ClassReflectionDeclarations.h.

tests/reflection-family.php rebuilds real class reflections (fixture +
stubbed built-ins + withTypes/withVariances/asFinal derivations) as twin
and native over duck-typed collaborators - the class reflection factory,
the PHPDoc inheritance resolver and stand-ins for the class-map classes
that receive $this - and compares 41,140 observations over 68 samples,
memo slots included; the fixture covers type aliases, documented/
templated/deprecated constants, mixin, sealed, property/method tags and
attribute classes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MShHKdUB19w38vboJLPKXy
Claude-Session: https://claude.ai/code/session_017MvPby652L7wUqGAHEiEcN
@ondrejmirtes @claude
Shadow ClassReflection natively
36e2928 
PHPStan\Reflection\ClassReflection is the native class in a production run
now: declareClass() (final, like the twin) + shadow(),
ClassReflectionDeclarations.h to the generator's generated/ClassReflection.h,
and the PT_CLASS_CLASS_REFLECTION class-map key gone.

ClassReflectionAccess.cpp is deleted with it. Its seven readers used to
answer from the twin's memo slots by name; they are re-exported from
ClassReflection.cpp as a direct call into the native body (the PHP method
for a foreign object), so the 44 Type-kernel call sites keep their spelling.
The two MutatingScope scope readers and their slot cache moved to
ScopeContext.cpp, where pt_class_reflection_access_rinit() became
pt_scope_access_rinit().

typeGetClassReflection() shortcut through pt_object_type_get_class_reflection()
ran ObjectType's body for a GenericObjectType, which overrides it; only the
flip made that reachable. It dispatches through the receiver's own
PT_OP_GET_CLASS_REFLECTION entry now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MShHKdUB19w38vboJLPKXy
@ondrejmirtes @claude
Shadow PhpClassReflectionExtension natively
8712c9b 
PHPStan\Reflection\Php\PhpClassReflectionExtension is the member factory
behind every ClassReflection::has*()/get*() call: hasMethod() 807K,
hasNativeMethod() 746K, touchMemberCacheKey() 399K, getNativeMethod() 292K,
hasProperty() 156K and getNativeProperty() 78K times in a self-analysis of
src/Analyser, src/Rules and src/Type. The whole class is ported (the nine
public methods natively, the member construction below them as C++), and
its hot path is absorbed on both sides: the service's own bodies and
self-calls, ClassReflection::getCacheKey() and ::getNativeReflection()
through the pt_class_reflection_*() direct calls, the shared member
LRU natively, and new readers of the BetterReflection adapter's member
memos, which answer hasMethod()/hasProperty()/getMethod() out of
ReflectionClass::$cachedMethods/$cachedProperties instead of three PHP
frames (lazy population, the empty name and a missing method stay the
library's, through its own methods).

Values that reach a PHP collaborator's concrete typed parameter or an
instanceof it performs - TrinaryLogic, TemplateTypeMap,
TemplateTypeVariance, MixedType and what TypehintHelper and TypeCombinator
build - are created through the classes' real names, which are the native
classes in a production run and the PHP twins under the prefixed
differential activation.

tests/php-class-reflection-family.php builds both sides from the
container's own collaborators with named arguments and compares every
public method over inherited, trait, magic, promoted, hooked, attributed,
enum, interface and signature-mapped members, in and out of class scope,
plus the member-cache memo and its eviction.

Interleaved A/B on src/Analyser + src/Rules + src/Type, user CPU, 8 ABBA
pairs: 41.03s -> 40.55s, -1.16% (median pair delta -1.22%); analysis output
byte-identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MShHKdUB19w38vboJLPKXy
@ondrejmirtes @claude
Port MutatingScope natively behind a differential-only plan
7240ad6 
Ports every method PHPStan\Analyser\MutatingScope declares to
turbo-ext/src/MutatingScope.cpp, in the twin's file order:

- the class layout (37 declared slots in the twin's order), the
  33-parameter constructor and Z_OBJCE-polymorphic $this dispatch, with
  named handlers for the methods NodeCallbackScope overrides so a
  $this-dispatch can identify the native body;
- the type resolution core: getType() with the ScopeOps memo and
  tracked-holder direct entries and the NodeScopeResolver guard statics,
  resolveType() and the handler-node resolution, the
  ExpressionResultStorage stack accessors, the template-argument frame and
  constraints, getNativeType() / getKeepVoidType() / promoteNativeTypes(),
  resolveName() / resolveTypeByName() and getTypeFromValue();
- the in-function-call stack and the enter* families (classes, traits,
  class methods, property hooks, functions, namespaces, anonymous and
  arrow functions, closure binding);
- the assignment and invalidation family and the specification
  machinery, where the twin starts chaining scopes: foreign-scope helpers
  read another scope's tables by slot when it is the native class and by
  name otherwise;
- the narrowing application (applySpecifiedTypes() with its deferred
  augments and conditional-holder recipes) and the scope merges;
- the closure and loop scopes, the loop fixed-point (generalizeWith()
  and generalizeType()'s seven buckets), the scope comparison, the
  visibility queries, debug(), the union-filtering member lookups and the
  node-callback entries.

ScopeOps gains direct entries for the memo, the invalidation, the merges
and the conditional-expression bookkeeping, StaticTypeFactory for the
offset-accessible types, TypeUtils for resolveLateResolvableTypes(), and
ExpressionResultStorage for duplicate(); UnionType::filterTypes() gets
its PHP callable from one pt_type_native_callback() holder. The class-map
keys the port needs are added, the PHP classes among them carrying
#[ReferencedByTurboExtension].

The plan is differential-only (reg::Class::shadowDifferentialOnly()):
the class is declared only by the prefixed activation of the differential
tests, so production keeps the PHP twin until the flip. The generator
renders declarations for shadowed classes only, so the port's generated
declarations are kept next to it in turbo-ext/src/MutatingScopeDeclarations.h.

tests/scope-family.php rebuilds real walk scopes on both sides over a
recording scope factory - the enter* producers' create() arguments, a
digest of the scope a chaining method answered, adjacent per-side scopes
whose tables and conditional expressions are built per side for the
merges and the generalization, and a Base/Child/Sibling hierarchy for the
visibility queries - and compares 64,669 observations over 80 scopes.
Mutation-checked with 39 deliberate native bugs: 37 caught, and the other
two sit in arms the twin cannot reach either.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MShHKdUB19w38vboJLPKXy
Claude-Session: https://claude.ai/code/session_017MvPby652L7wUqGAHEiEcN
@ondrejmirtes @claude
Shadow MutatingScope natively
82ceaf5 
PHPStan\Analyser\MutatingScope is the native class in a production run now:
declareClass() (implementing Scope, NodeCallbackInvoker, CollectedDataEmitter)
+ shadow() instead of shadowDifferentialOnly(), #[ShadowedByTurboExtension]
on the twin, the declarations moving from MutatingScopeDeclarations.h to the
generator's generated/MutatingScope.h, and the PT_CLASS_MUTATING_SCOPE
class-map key gone.

Its only user, ScopeContext.cpp's scopeClassReflectionSlot(), reads
pt_ce_mutating_scope directly; the class map was the only thing there that
could throw, so the helper's `bool &error` out-parameter and the two
UNEXPECTED(error) arms of pt_scope_is_in_class() / pt_scope_get_class_reflection()
go with it.

scope-family.php keeps comparing both sides under the prefix — the type
barrier the prefix creates is unchanged by the flip — but its
reflection-driven delegate generation is gone: every public method of the
twin is native, so it produced nothing. smoke.php registers the class in
$covered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MShHKdUB19w38vboJLPKXy
@ondrejmirtes @claude
Absorb the accessor hops the native scope and class reflection make
d2409b8 
Four PHP frames stood between the native MutatingScope / ClassReflection
and values they already owned, each reached per call through the
pt_type_call path:

- ExpressionResultStorageStack::getCurrent() (1.0M calls per
  self-analysis run, plus push/pop) - the class is small and final, so it
  is now shadowed natively, and MutatingScope::currentStorage() reads the
  top of the stack out of the list property instead of calling the method.

- LazyClassReflectionExtensionRegistryProvider::getRegistry() (1.3M)
  followed by a ClassReflectionExtensionRegistry getter (1.3M) - the
  provider builds its registry once and keeps it in $registry for the
  rest of the run (it drops the container reference right after), and the
  registry is a final value class whose slots only its constructor
  writes. ReflectionAccess.cpp therefore reads both hops out of the
  property slots, exactly as it already does for the memoizing reflection
  provider: the methods still answer while the memo is null, for a
  provider or registry of any other class, and for a slot the
  constructor never wrote.

- LazyInternalScopeFactory::create() (1.45M) - once its `??=` service
  memos are filled, create() is those slots plus a `new`, which
  MutatingScope::factoryCreate() now does itself. The method answers for
  the first create() of each factory, for any other InternalScopeFactory
  implementation, for arguments its signature would reject, and whenever
  the classes it would instantiate are not the native ones - which is
  what the prefixed differential activation sets up, so the tests keep
  exercising the twin.

Hit rates over a src/Analyser + src/Reflection run: 499,649 native vs 351
method calls for create(), 299,999 vs 1 for the registry members, 400,000
vs 0 for the storage stack.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MShHKdUB19w38vboJLPKXy
@ondrejmirtes
Bump expected turbo version
cd47e80
@ondrejmirtes
ondrejmirtes merged commit cd47e80 into 2.3.x Sep 22, 2026
1 check passed
@ondrejmirtes
ondrejmirtes deleted the turbo-native-scope branch September 22, 2026 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ondrejmirtes