From ccd75c6b625877346c9dc22dce3299d04b3a8524 Mon Sep 17 00:00:00 2001 From: said Date: Wed, 23 Sep 2026 22:53:52 +0100 Subject: [PATCH 01/47] codex: integrate Open MPI mpi_f08 through generated contracts --- CHANGELOG.md | 10 +- docs/user/guide/wrapping-modules.md | 3 + docs/user/reference/cli-commands.md | 19 +- docs/user/reference/pyi-format.md | 12 +- docs/user/reference/python-api.md | 4 +- docs/user/tutorials/openmpi-f08.md | 98 ++++++++ mkdocs.yml | 3 +- prik/cli.py | 12 +- prik/codegen/c/binding.py | 75 ++++-- prik/codegen/fortran/bridge.py | 85 +++++-- prik/contracts/__init__.py | 4 + prik/parsers/fortran/parser.py | 28 +++ prik/pipeline/build.py | 30 +-- prik/pipeline/wrapper.py | 46 +--- prik/planning/entrypoints.py | 1 + prik/planning/models.py | 10 +- prik/planning/planner.py | 82 +++---- prik/policy/completion.py | 12 +- prik/policy/construction.py | 28 ++- prik/policy/contract_imports.py | 11 + prik/policy/exports.py | 36 ++- prik/policy/models.py | 21 +- prik/printers/pyi.py | 9 +- prik/runtime/native_support/prik_binding.h | 5 + prik/semantics/fortran2ir.py | 30 ++- prik/semantics/fortran_exports.py | 225 ++++++++++++++---- prik/semantics/models.py | 2 + prik/semantics/pyi2ir.py | 52 +++- .../fixtures/native/assumed_type_calls.f90 | 5 + .../fixtures/runtime/openmpi_basic.py | 64 +++++ .../end_to_end/test_assumed_type_calls.py | 14 ++ .../end_to_end/test_openmpi_f08.py | 154 ++++++++++++ .../native/export_selection_facade/facade.f90 | 3 + .../native/export_selection_facade/owner.f90 | 12 + .../test_fortran_export_symbol_workflow.py | 53 +++++ .../test_fortran_export_symbol_selection.py | 78 +++++- .../test_declaring_namespace_publication.py | 20 +- .../test_module_array_view_lowering.py | 2 +- .../parsing/test_project_scope_models.py | 18 ++ .../policy/test_module_variable_policy.py | 6 +- 40 files changed, 1063 insertions(+), 319 deletions(-) create mode 100644 docs/user/tutorials/openmpi-f08.md create mode 100644 tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py create mode 100644 tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py create mode 100644 tests/fortran/functions/end_to_end/fixtures/native/export_selection_facade/facade.f90 create mode 100644 tests/fortran/functions/end_to_end/fixtures/native/export_selection_facade/owner.f90 diff --git a/CHANGELOG.md b/CHANGELOG.md index fef42eeea..ab99f2f67 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,10 +18,12 @@ release tags add a leading `v` to the package version. retain buffers for nonblocking operations. - `--export-symbols` and `build_fortran_extension(export_symbols=...)` accept - module-qualified Fortran procedure identities. Generated contracts and - source builds publish only that reviewed function surface while retaining - callback, type, and other declaration dependencies required by its - signatures. + module-qualified Fortran procedures and variables, including symbols + re-exported by a public facade. Generated contracts retain the selected + access module, required type declarations, and native scalar storage views. +- The Open MPI `mpi_f08` tutorial and opt-in two-rank integration test build a + wrapper from a restricted generated `.pyi` against a matching prebuilt + Open MPI installation and exercise NumPy communication and in-place reduction. - The PRIMA example links five derivative-free solvers against one statically compiled `libprimaf` archive through a generated semantic contract and runs in the real-library portability matrix. Its guide includes a reproducible diff --git a/docs/user/guide/wrapping-modules.md b/docs/user/guide/wrapping-modules.md index 7c7201b6d..892acde75 100644 --- a/docs/user/guide/wrapping-modules.md +++ b/docs/user/guide/wrapping-modules.md @@ -136,6 +136,9 @@ Assign elements or slices, rather than rebinding the module variable. Fixed-shape module arrays and derived-type array fields expose live views whether or not their declarations include `target`. Logical array dtypes follow the [data-type mapping](data-types.md#scalar-type-mapping). +An addressable scalar module variable declared with `bind(C)` is also a live +rank-zero NumPy view. Use `value[()]` to read or update it; pass the view itself +when a `TYPE(*)` dummy needs its original native storage. - Allocatable module arrays use the `Allocatable[T[...]]` API. - Allocation, lifetime, NumPy views, and mutation rules are covered in the storage and objects section. diff --git a/docs/user/reference/cli-commands.md b/docs/user/reference/cli-commands.md index ed27c4d10..cc702d0a2 100644 --- a/docs/user/reference/cli-commands.md +++ b/docs/user/reference/cli-commands.md @@ -380,7 +380,7 @@ locations come from the template's own output. ## Source export selection -`--export-symbols FILE` selects the exact function surface to convert from +`--export-symbols FILE` selects the exact symbol surface to convert from native source. It is available for C and Fortran source commands, including source builds, `semantics`, and `generate --pyi`. A generated contract records the corresponding Python names in `__all__`; when building that contract, @@ -394,23 +394,24 @@ vendor_open vendor_close ``` -Fortran module procedures use a case-insensitive, module-qualified identity: +Fortran module procedures and module variables use a case-insensitive, +module-qualified identity: ```text bobyqa_mod::bobyqa cobyla_mod::cobyla +state_mod::counter ``` -Qualification keeps procedures with the same spelling in different modules +Qualification keeps symbols with the same spelling in different modules distinct. The module side must name a declared Fortran `module`, not a file-level external-procedure group. Every listed identity must resolve to -exactly one reachable function. Empty files, invalid or repeated identities, -unknown declarations, and names that do not denote functions fail the command. +one public procedure or variable. Empty files, invalid or repeated identities, +and unknown or private declarations fail the command. -Fortran extraction retains declarations needed to express the selected -signatures, such as callback prototypes and derived types, without publishing -them as additional callable functions. Unselected procedures and unrelated -modules are omitted from the generated contract. The positional inputs remain +Fortran extraction retains declarations needed to express selected signatures +and variable types, such as callback prototypes and derived types, without +publishing unrelated declarations. The positional inputs remain the native source universe used to resolve those dependencies and, for a source build, the implementation sources compiled unless `--no-compile-input-sources` is selected. diff --git a/docs/user/reference/pyi-format.md b/docs/user/reference/pyi-format.md index c288d526f..106c3d40c 100644 --- a/docs/user/reference/pyi-format.md +++ b/docs/user/reference/pyi-format.md @@ -305,6 +305,9 @@ def update(value: Float64[()]) -> None: ... ``` The leaf filename supplies the native module. No placement decorator is needed. +When a public Fortran facade re-exports a procedure from a supporting module, +`@native_module("facade_name")` selects the module the bridge imports. Generated +contracts retain this access route, so building from the `.pyi` uses the facade. ### Standalone Procedures @@ -403,6 +406,9 @@ scale: Float64 = 2.0 Fortran module variables can be buildable getters, setters, constants, wrapped objects, or descriptor handles according to their completed policy. A literal default on supported mutable scalar state is an import-time native initializer. +`Annotated[Int32, NativeStorage]` describes a scalar module variable whose +Python value is a live rank-zero NumPy view over its native storage. Use this +marker when rebuilding from a contract must preserve the variable's address. C global declarations can be represented for inspection, but current C wrapper builds reject native global state. C functions remain the supported runtime @@ -554,6 +560,7 @@ Python declaration and native callable names differ. | `@private` | Function or method | Shared: declaration remains available to contract dependencies but is not exported. | | `@bind("symbol")` | Function, method, constructor, prototype, or destructor | Shared: select a different native name. | | `@native_abi("c")` | Function, method, or prototype | Fortran only: original declaration is `bind(C)`. | +| `@native_module("module_name")` | Module-level function | Fortran only: import the procedure through this native module. | | `@standalone` | Module-level function | Fortran only: native procedure is outside a module. | | `@native_call([...], result=...)` | Function, method, or constructor | Shared: state the complete native argument order and optional native result mapping. | | `@overload("specific", generic=...)` | Function or method | Shared: add one exact candidate to a generated Python overload set. | @@ -604,6 +611,8 @@ The linked concrete declaration owns `@native_call`. An overload-level `@bind(...)` selects a public native generic when the specific itself is not the link target. Runtime dispatch distinguishes exact scalar dtype, array element dtype and rank, or wrapped class; it does not use implicit numeric coercion. +An `AnyNative` choice-buffer argument can appear in a generic with one selected +candidate; its concrete wrapper validates the actual storage at the call. The optional `generic=` string preserves a Fortran operator spelling when the Python method name is ambiguous, such as `.eqv.` versus `==`. @@ -927,6 +936,7 @@ type. Metadata falls into four groups. | `PointerAssociation("runtime")` | Pointer association is runtime state. | Fortran pointer facts. | | `SourceName("native-name")` | Preserve a native name that the Python identifier cannot represent. | Shared. | | `FortranAllocatable` | Fortran allocatable source provenance. | Loaded source contracts. | +| `NativeStorage` | Scalar module variable uses its existing native address. | Prebuilt Fortran library state exposed as a live rank-zero NumPy view. | | `CAnonymousMember` | Field is the containing member for a nested anonymous C aggregate. | C inspection contracts. | ### Constraints @@ -1100,7 +1110,7 @@ valid and whether it is buildable. | Prototype direction | `In`, `Out`, `InOut` | | Native-call helpers | `Arg`, `Hidden`, `IsPresent`, `Len`, `Pass`, `Return`, `Value`, `Work` | | Exact C scalar helpers | `CBool`, `CChar`, `CSignedChar`, `CUnsignedChar`, `CShort`, `CUnsignedShort`, `CInt`, `CUnsignedInt`, `CLong`, `CUnsignedLong`, `CLongLong`, `CUnsignedLongLong`, `CFloat`, `CDouble`, `CLongDouble`, `CFloatComplex`, `CDoubleComplex`, `CLongDoubleComplex` | -| Decorators | `abstract`, `abstractmethod`, `bind`, `destroy`, `native_abi`, `native_call`, `nogil`, `overload`, `private`, `prototype`, `pure`, `raises`, `standalone` | +| Decorators | `abstract`, `abstractmethod`, `bind`, `destroy`, `native_abi`, `native_call`, `native_module`, `nogil`, `overload`, `private`, `prototype`, `pure`, `raises`, `standalone` | `staticmethod` is supported Python syntax for methods but is not exported by `prik.contracts`. diff --git a/docs/user/reference/python-api.md b/docs/user/reference/python-api.md index 28c10ef03..835d91371 100644 --- a/docs/user/reference/python-api.md +++ b/docs/user/reference/python-api.md @@ -100,8 +100,8 @@ Once you author or generate a semantic `.pyi` contract, that contract's own `__all__` states the public surface and `export_symbols` no longer applies. `build_fortran_extension` accepts the same option with module-qualified native -procedure identities. PRIK retains signature dependencies while publishing -only the selected procedures: +procedure and variable identities. PRIK retains signature and type +dependencies while publishing only the selected symbols: ```python from prik import build_fortran_extension diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md new file mode 100644 index 000000000..4d76959ba --- /dev/null +++ b/docs/user/tutorials/openmpi-f08.md @@ -0,0 +1,98 @@ +--- +title: Wrap a prebuilt Open MPI mpi_f08 installation +description: Generate a restricted Fortran contract and build a Python extension against Open MPI +audience: users +prerequisites: configured Open MPI source tree and matching installed Open MPI development files +related: ../guide/wrapping-modules.md, ../reference/cli-commands.md, ../reference/pyi-format.md +status: maintained +publication: reviewed +--- + +# Wrap Open MPI `mpi_f08` + +Use a configured Open MPI source tree and the corresponding installed Open MPI +toolchain. PRIK reads the interface and type sources to generate a contract; +the extension compiles against the installed modules and libraries. + +Set `PRIK_OPENMPI_SOURCE` to the matching Open MPI source root and +`PRIK_OPENMPI_BUILD` to its configured build root. The build must contain its +generated Fortran includes, including `configure-fortran-output.h` and +`sizeof_f08.h`. Confirm that `mpifort --showme:version` reports the same +Open MPI version as the source tree. + +Select the public facade's small initial API: + +```bash +cat > exports.txt <<'EOF' +mpi_f08::MPI_Init +mpi_f08::MPI_Finalize +mpi_f08::MPI_Comm_rank +mpi_f08::MPI_Comm_size +mpi_f08::MPI_Barrier +mpi_f08::MPI_Send +mpi_f08::MPI_Recv +mpi_f08::MPI_Allreduce +mpi_f08::MPI_COMM_WORLD +mpi_f08::MPI_INT +mpi_f08::MPI_DOUBLE_PRECISION +mpi_f08::MPI_SUM +mpi_f08::MPI_IN_PLACE +mpi_f08::MPI_STATUS_IGNORE +EOF + +python3 -m prik generate --pyi \ + "$PRIK_OPENMPI_SOURCE/ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-types.F90" \ + "$PRIK_OPENMPI_SOURCE/ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-interfaces.F90" \ + "$PRIK_OPENMPI_SOURCE/ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90" \ + --export-symbols exports.txt --out contract --compiler mpifort \ + -I "$PRIK_OPENMPI_BUILD" \ + -I "$PRIK_OPENMPI_BUILD/ompi/mpi/fortran/use-mpi-f08" \ + -I "$PRIK_OPENMPI_SOURCE" \ + -I "$PRIK_OPENMPI_BUILD/ompi/include" \ + -I "$PRIK_OPENMPI_SOURCE/ompi/include" +``` + +Build from the generated `contract/__init__.pyi`. Query the installed wrapper +compiler for its module paths and link arguments: + +```python +import shlex +import subprocess + +from prik.pipeline.build import NativeLinkItem, build_pyi_extension + + +def show(option): + return shlex.split(subprocess.check_output(["mpifort", option], text=True)) + + +compile_flags = show("--showme:compile") +build_pyi_extension( + "contract/__init__.pyi", + input_compiler=show("--showme:command")[0], + native_include_dirs=[flag[2:] for flag in compile_flags if flag.startswith("-I")], + native_link_items=[NativeLinkItem("linker_argument", flag) for flag in show("--showme:link")], + native_linker_language="fortran", + output_name="prik_openmpi_f08", + output_dir="build/openmpi", +) +``` + +Run a Python program under the matching Open MPI launcher. The selected +functions live in `prik_openmpi_f08.mpi_f08`; Fortran `Int32` arguments such as +counts and ranks use `numpy.int32` values. NumPy arrays provide the storage for +choice buffers. In the tested Open MPI 4.1.2 configuration, `mpi_in_place` is +a concrete integer module object exposed as a live rank-zero NumPy view, so +pass that view directly to `mpi_allreduce`. +A two-rank example lives at +`tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py`. +Run it with `PYTHONPATH=build/openmpi orterun -n 2 python3` followed by that +path, or use the matching Open MPI `mpirun` launcher. It exercises +send/receive, ordinary and in-place all-reduce, and the selected communicator +and datatype objects. + +The choice-buffer ABI follows the configured interface declaration: +`TYPE(*), DIMENSION(*)` passes a raw address, while assumed-shape and +assumed-rank declarations pass a C descriptor. Arrays of derived MPI handles, +persistent callbacks, and nonblocking buffer-lifetime management are outside +this initial surface. diff --git a/mkdocs.yml b/mkdocs.yml index 73e5ccdae..626fb8131 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -52,7 +52,8 @@ nav: - Strings: user/guide/strings.md - Wrapping Functions: user/guide/wrapping-functions.md - Wrapping Subroutines: user/guide/wrapping-subroutines.md - - Wrapping Modules: user/guide/wrapping-modules.md + - Wrapping Modules: user/guide/wrapping-modules.md + - Open MPI mpi_f08: user/tutorials/openmpi-f08.md - Optional Arguments: user/guide/optional-arguments.md - Generic Interfaces (Overloading): user/guide/generic-interfaces.md - Wrapping Derived Types: user/guide/wrapping-derived-types.md diff --git a/prik/cli.py b/prik/cli.py index de50cfcd3..6f1461e19 100644 --- a/prik/cli.py +++ b/prik/cli.py @@ -524,9 +524,9 @@ def _converted_semantic_files( if language != "fortran" or export_symbols is None: return _ConvertedSemanticSources(tuple(converted_files), available_modules) - from prik.semantics.fortran_exports import select_fortran_export_functions + from prik.semantics.fortran_exports import select_fortran_export_symbols - selection = select_fortran_export_functions(available_modules, export_symbols) + selection = select_fortran_export_symbols(available_modules, export_symbols) selected_by_source = { id(source): selected for source, selected in zip(selection.primary_sources, selection.primary_modules, strict=True) @@ -1202,7 +1202,7 @@ def _validate_c_main_options(args: argparse.Namespace, parser: argparse.Argument def _read_export_symbols(path: str | Path, *, language: str) -> tuple[str, ...]: - """Read one fail-closed source-function allowlist from a UTF-8 file.""" + """Read one fail-closed source-symbol allowlist from a UTF-8 file.""" source = Path(path) try: lines = source.read_text(encoding="utf-8").splitlines() @@ -1232,8 +1232,8 @@ def _read_export_symbols(path: str | Path, *, language: str) -> tuple[str, ...]: valid = False else: valid = True - label = "Fortran module procedure identity" - duplicate_label = "Fortran module procedure identity" + label = "Fortran module symbol identity" + duplicate_label = "Fortran module symbol identity" if not valid: raise ValueError(f"Invalid {label} in --export-symbols file {source}:{line_number}: {symbol!r}") identity = symbol if language == "c" else symbol.casefold() @@ -1246,7 +1246,7 @@ def _read_export_symbols(path: str | Path, *, language: str) -> tuple[str, ...]: locations[identity] = line_number symbols.append(symbol) if not symbols: - label = "C function names" if language == "c" else "Fortran module procedure identities" + label = "C function names" if language == "c" else "Fortran module symbol identities" raise ValueError(f"--export-symbols file contains no {label}: {source}") return tuple(symbols) diff --git a/prik/codegen/c/binding.py b/prik/codegen/c/binding.py index 27cd28fb6..aee9efa35 100644 --- a/prik/codegen/c/binding.py +++ b/prik/codegen/c/binding.py @@ -41,7 +41,7 @@ DerivedWriteback, DirectResultABI, ModuleObjectAccessMechanism, - ModuleArrayAddressMechanism, + ModuleStorageAddressMechanism, ModuleGetterAction, NativeArrayDescriptorAttribute, NativeArrayDescriptorKind, @@ -218,6 +218,7 @@ class _COverloadDispatch: ModuleGetterAction.NATIVE_CONSTANT_VALUE: "Builds a Python object from the compiler-evaluated constant.", ModuleGetterAction.NATIVE_CONSTANT_ARRAY_VALUE: "Copies the parameter array into one read-only NumPy array.", ModuleGetterAction.DIRECT_VALUE: "Builds a Python scalar from the current native value.", + ModuleGetterAction.NATIVE_SCALAR_VIEW: "Wraps live native scalar storage in a rank-zero NumPy view.", ModuleGetterAction.CHARACTER_VALUE: "Decodes the fixed-width native characters into a Python str.", ModuleGetterAction.NULLABLE_SNAPSHOT: "Returns a detached copy, or None when the native value holds nothing.", ModuleGetterAction.BORROWED_ARRAY_VIEW: "Wraps the native storage in a live NumPy array without copying.", @@ -729,7 +730,8 @@ def _requires_address_capture(self, plan: ModulePlan) -> bool: plain module object, which is likewise not a target. """ return any( - variable.array_address is ModuleArrayAddressMechanism.CAPTURED_ADDRESS for variable in self._variables(plan) + variable.storage_address is ModuleStorageAddressMechanism.CAPTURED_ADDRESS + for variable in self._variables(plan) ) or any( member.field.access is DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR for variable in self._variables(plan) @@ -4526,7 +4528,11 @@ def _module_array_owner_variables(self, plan: ModulePlan) -> tuple[ModuleVariabl variable for variable in self._variables(plan) if variable.binding.getter_action - in {ModuleGetterAction.BORROWED_ARRAY_VIEW, ModuleGetterAction.NATIVE_ARRAY_HANDLE} + in { + ModuleGetterAction.BORROWED_ARRAY_VIEW, + ModuleGetterAction.NATIVE_ARRAY_HANDLE, + ModuleGetterAction.NATIVE_SCALAR_VIEW, + } ) # Borrowed module native-array-handle operations. @@ -6121,6 +6127,8 @@ def _lower_module_getter(self, plan: ModuleVariablePlan) -> tuple[CFunction, ... return self._lower_module_getter_constant_value(plan) case ModuleGetterAction.DIRECT_VALUE: return self._lower_module_getter_direct_value(plan) + case ModuleGetterAction.NATIVE_SCALAR_VIEW: + return self._lower_module_getter_native_scalar_view(plan) case ModuleGetterAction.CHARACTER_VALUE: return self._lower_module_getter_character_value(plan) case ModuleGetterAction.NULLABLE_SNAPSHOT: @@ -6165,6 +6173,27 @@ def _lower_module_getter_direct_value(self, plan: ModuleVariablePlan) -> tuple[C ), ) + def _lower_module_getter_native_scalar_view(self, plan: ModuleVariablePlan) -> tuple[CFunction, ...]: + """Expose live scalar module storage as a rank-zero NumPy view.""" + scalar = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) + owner = self._module_native_array_owner_name(plan) + return ( + CFunction( + self._module_getter_name(plan), + "PyObject *", + storage="static", + body=( + CDeclaration("data", "void *", CodeExpression(f"{self._module_bridge_getter_name(plan)}()")), + CDeclaration( + "result", + "PyObject *", + CodeExpression(f"PyArray_SimpleNewFromData(0, NULL, {scalar.array_numpy_type}, data)"), + ), + *self._ordinary_array_field_owner_nodes("result", owner), + ), + ), + ) + def _module_character_length(self, plan: ModuleVariablePlan) -> int: """Return the declared width one character module accessor copies.""" length = plan.character_length @@ -15224,24 +15253,29 @@ def _overload_dispatch_function(self, dispatch: _COverloadDispatch) -> CFunction body.append(CDeclaration("user_nargs", "Py_ssize_t", CodeExpression("nargs"))) body.append(CDeclaration("candidate_id", "int", CodeExpression("-1"))) body.extend(self._overload_special_case_nodes(overload, dispatch.receiver)) - body.extend( - CIf( - CodeExpression( - "candidate_id < 0 && (" - + self._overload_candidate_condition( - matches, - positional_offset=positional_offset, - ) - + ")" - ), - body=(CExpressionStatement(CodeExpression(f"candidate_id = {candidate_id}")),), - ) - for candidate_id, matches in zip( - overload.candidate_ids, - overload.candidate_matches, - strict=True, + if overload.direct_single_candidate: + # The sole wrapper validates open-ended native actuals after + # policy has selected direct dispatch. + body.append(CExpressionStatement(CodeExpression(f"candidate_id = {overload.candidate_ids[0]}"))) + else: + body.extend( + CIf( + CodeExpression( + "candidate_id < 0 && (" + + self._overload_candidate_condition( + matches, + positional_offset=positional_offset, + ) + + ")" + ), + body=(CExpressionStatement(CodeExpression(f"candidate_id = {candidate_id}")),), + ) + for candidate_id, matches in zip( + overload.candidate_ids, + overload.candidate_matches, + strict=True, + ) ) - ) cases = tuple( self._overload_candidate_case( dispatch, @@ -15933,6 +15967,7 @@ def _module_native_array_owner_nodes( if variable.binding.getter_action not in { ModuleGetterAction.BORROWED_ARRAY_VIEW, ModuleGetterAction.NATIVE_ARRAY_HANDLE, + ModuleGetterAction.NATIVE_SCALAR_VIEW, }: continue owner = self._module_native_array_owner_name(variable) diff --git a/prik/codegen/fortran/bridge.py b/prik/codegen/fortran/bridge.py index 4d720480b..bde2e9708 100644 --- a/prik/codegen/fortran/bridge.py +++ b/prik/codegen/fortran/bridge.py @@ -41,7 +41,7 @@ DeclarationCallableAction, DirectResultABI, ExternalDeclarationMode, - ModuleArrayAddressMechanism, + ModuleStorageAddressMechanism, ModuleGetterAction, ModuleObjectAccessMechanism, CharacterLocalRelease, @@ -118,6 +118,7 @@ # The C identity function that reports a non-target module array's base # address. The binding defines it; the bridge declares and calls it. _MODULE_ARRAY_CAPTURE_NAME = "prik_capture_address" +_MODULE_SCALAR_CAPTURE_NAME = "prik_capture_scalar_address" # The binding answers these from the live descriptor the handle's entry point # supplies, so the bridge emits no procedure of its own for them. @@ -1804,7 +1805,7 @@ def _derived_scoped_internal_procedures( FortranParameter("address", "type(c_ptr)", ("value",)), FortranParameter("context", "type(c_ptr)", ("value",)), ), - result_name="status", + result_name="prik_consumer_status", result_type="integer(c_int)", bind_c=True, body=( @@ -1813,9 +1814,9 @@ def _derived_scoped_internal_procedures( body=( FortranCall("c_f_pointer", (CodeExpression("address"), CodeExpression(name))), FortranCall(next_step, ()), - FortranAssignment("status", CodeExpression("0_c_int")), + FortranAssignment("prik_consumer_status", CodeExpression("0_c_int")), ), - else_body=(FortranAssignment("status", CodeExpression("1_c_int")),), + else_body=(FortranAssignment("prik_consumer_status", CodeExpression("1_c_int")),), ), ), ) @@ -2691,6 +2692,8 @@ def _lower_module_getter(self, plan: ModuleVariablePlan) -> tuple[FortranFunctio return self._lower_module_getter_constant_array_value(plan) case ModuleGetterAction.DIRECT_VALUE: return self._lower_module_getter_direct_value(plan) + case ModuleGetterAction.NATIVE_SCALAR_VIEW: + return self._lower_module_getter_native_scalar_view(plan) case ModuleGetterAction.CHARACTER_VALUE: return self._lower_module_getter_character_value(plan) case ModuleGetterAction.NULLABLE_SNAPSHOT: @@ -3435,6 +3438,20 @@ def _lower_module_getter_direct_value(self, plan: ModuleVariablePlan) -> tuple[F ), ) + def _lower_module_getter_native_scalar_view(self, plan: ModuleVariablePlan) -> tuple[FortranFunction, ...]: + """Return the original module scalar's address without forming a copy.""" + name = self._module_bridge_getter_name(plan) + native = self._native_variable_name(plan) + return ( + FortranFunction( + name=name, + result_name="result", + result_type="type(c_ptr)", + bind_name=name, + body=(FortranAssignment("result", CodeExpression(f"{_MODULE_SCALAR_CAPTURE_NAME}({native})")),), + ), + ) + def _module_character_length(self, plan: ModuleVariablePlan) -> int: """Return the declared width one character module accessor copies.""" length = plan.character_length @@ -3606,10 +3623,10 @@ def _lower_module_getter_borrowed_array_view( @staticmethod def _module_array_address(plan: ModuleVariablePlan, native: str) -> str: """Return the address expression selected by the completed mechanism.""" - mechanism = plan.array_address - if mechanism is ModuleArrayAddressMechanism.TARGET_ADDRESS: + mechanism = plan.storage_address + if mechanism is ModuleStorageAddressMechanism.TARGET_ADDRESS: return f"c_loc({native})" - if mechanism is ModuleArrayAddressMechanism.CAPTURED_ADDRESS: + if mechanism is ModuleStorageAddressMechanism.CAPTURED_ADDRESS: return f"{_MODULE_ARRAY_CAPTURE_NAME}({native})" raise ValueError(f"Module array view {plan.owner_path!r} has no completed address mechanism: {mechanism!r}") @@ -3622,7 +3639,8 @@ def _requires_address_capture(self, plan: ModulePlan) -> bool: plain module object, which is likewise not a target. """ return any( - variable.array_address is ModuleArrayAddressMechanism.CAPTURED_ADDRESS for variable in self._variables(plan) + variable.storage_address is ModuleStorageAddressMechanism.CAPTURED_ADDRESS + for variable in self._variables(plan) ) or any( member.field.access is DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR for variable in self._derived_member_proxy_variables(plan) @@ -3641,21 +3659,44 @@ def _module_array_capture_interfaces(self, plan: ModulePlan) -> tuple[FortranInt """ if not self._requires_address_capture(plan): return () - return ( - FortranInterface( - ( - FortranInterfaceProcedure( - name=_MODULE_ARRAY_CAPTURE_NAME, - imports=("c_ptr",), - parameters=(FortranParameter("base", "type(*)", ("dimension(*)",)),), - result_name="address", - result_type="type(c_ptr)", - bind_name=_MODULE_ARRAY_CAPTURE_NAME, - bind_c=True, - ), + procedures = [] + if any( + variable.storage_address is ModuleStorageAddressMechanism.CAPTURED_ADDRESS + and variable.bridge.native_getter_action is ModuleGetterAction.NATIVE_SCALAR_VIEW + for variable in self._variables(plan) + ): + procedures.append( + FortranInterfaceProcedure( + name=_MODULE_SCALAR_CAPTURE_NAME, + imports=("c_ptr",), + parameters=(FortranParameter("base", "type(*)"),), + result_name="address", + result_type="type(c_ptr)", + bind_name=_MODULE_SCALAR_CAPTURE_NAME, + bind_c=True, ) - ), - ) + ) + if any( + variable.storage_address is ModuleStorageAddressMechanism.CAPTURED_ADDRESS + and variable.bridge.native_getter_action is ModuleGetterAction.BORROWED_ARRAY_VIEW + for variable in self._variables(plan) + ) or any( + member.field.access is DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR + for variable in self._derived_member_proxy_variables(plan) + for member in variable.derived.member_paths + ): + procedures.append( + FortranInterfaceProcedure( + name=_MODULE_ARRAY_CAPTURE_NAME, + imports=("c_ptr",), + parameters=(FortranParameter("base", "type(*)", ("dimension(*)",)),), + result_name="address", + result_type="type(c_ptr)", + bind_name=_MODULE_ARRAY_CAPTURE_NAME, + bind_c=True, + ) + ) + return (FortranInterface(tuple(procedures)),) def _lower_module_getter_nullable_snapshot( self, diff --git a/prik/contracts/__init__.py b/prik/contracts/__init__.py index f22e8364c..02735c8e5 100644 --- a/prik/contracts/__init__.py +++ b/prik/contracts/__init__.py @@ -236,6 +236,7 @@ def apply(target): FortranAllocatable = _ContractExpression() Immutable = _ContractExpression() MaybeUnallocated = _ContractExpression() +NativeStorage = _ContractExpression() ORDER_ANY = _ContractExpression() ORDER_C = _ContractExpression() ORDER_F = _ContractExpression() @@ -329,6 +330,7 @@ def destroy(target): bind = _decorator nogil = _decorator native_abi = _decorator +native_module = _decorator native_call = _decorator overload = _decorator prototype = _decorator @@ -394,6 +396,7 @@ def destroy(target): "Matrix", "AnyNative", "MaybeUnallocated", + "NativeStorage", "Opaque", "OpaqueHandle", "ORDER_ANY", @@ -430,6 +433,7 @@ def destroy(target): "destroy", "nogil", "native_abi", + "native_module", "native_call", "overload", "prototype", diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index 9325a67ae..433a12592 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -363,6 +363,7 @@ class _Declaration: pointer: bool = False target: bool = False protected: bool = False + bind_c: bool = False contiguous: bool = False asynchronous: bool = False external: bool = False @@ -3500,6 +3501,11 @@ def _parse_module_like_spec_line( target.uses.append(parsed_use) return + legacy_parameter = _REGEX["legacy_parameter"].match(stripped) + if legacy_parameter: + self._record_scope_legacy_parameters(target, legacy_parameter.group("body")) + return + if _REGEX["derived_type"].match(stripped): parsed_type = self._source_unit_scanner.parse_derived_type_start(stripped) raise FortranParseError( @@ -3530,6 +3536,24 @@ def _parse_module_like_spec_line( return self._raise_unsupported_module_like_declaration(target, stripped, filename, lineno, source_line) + @staticmethod + def _record_scope_legacy_parameters(target, assignments: str) -> None: + """Apply a separate PARAMETER statement to its module-like declarations.""" + variables = {variable.name.casefold(): variable for variable in target.variables} + for assignment in split_csv(assignments): + if "=" not in assignment: + continue + name, expression = (part.strip() for part in assignment.split("=", 1)) + variable = variables.get(name.casefold()) + if variable is None: + variable = FortranArgument(name=name, base_type=FortranParser._infer_implicit_base_type(name)) + target.variables.append(variable) + variables[name.casefold()] = variable + variable.is_parameter = True + variable.value = FortranParser._normalize_parameter_value(expression) + variable.symbolic_value = expression + variable.value_type = "expression" + def _raise_unsupported_openmp_declaration(self, target, line, filename, lineno, source_line) -> None: """Raise the stable diagnostic for an unsupported OpenMP declaration. @@ -4412,6 +4436,8 @@ def _apply_declaration_attributes( elif lowered in {"public", "private"}: declaration.visibility = lowered declaration.explicit_visibility = lowered + elif re.match(r"bind\s*\(\s*c\b", lowered): + declaration.bind_c = True elif lowered.startswith("dimension") and "(" in attribute and ")" in attribute: shape = split_csv(attribute[attribute.find("(") + 1 : attribute.rfind(")")]) declaration.shape = shape @@ -4493,6 +4519,8 @@ def _apply_internal_type_metadata(arg: FortranVariable, declaration: _Declaratio arg._fortran_polymorphic = True if declaration.protected: arg._fortran_protected = True + if declaration.bind_c: + arg._fortran_bind_c = True @staticmethod def _split_dim_bounds(dim: str) -> tuple[str | None, str | None]: diff --git a/prik/pipeline/build.py b/prik/pipeline/build.py index bc400693a..0f095e494 100644 --- a/prik/pipeline/build.py +++ b/prik/pipeline/build.py @@ -2071,6 +2071,7 @@ def _apply_pyi_python_exports(entry: Path, modules_by_path: dict[Path, SemanticM for declaration, entity_kind in ( *((item, "derived_type") for item in module.classes), *((item, "procedure") for item in module.functions), + *((item, "generic") for item in module.overload_sets), ): exports = _declaration_exports(declaration) if len(exports) < 2: @@ -2092,7 +2093,6 @@ def _apply_pyi_python_exports(entry: Path, modules_by_path: dict[Path, SemanticM ) ) exports[:] = [primary] - _reject_unsupported_republication(path, module, home) def _pyi_export_tree( @@ -2237,30 +2237,6 @@ def _merge_export_child(tree: _PyiExportNode, name: str, child: _PyiExportNode, ) -def _reject_unsupported_republication( - path: Path, - module: SemanticModule, - home: tuple[str, ...] | None, -) -> None: - """Refuse a generic published outside the namespace declaring it. - - A module variable has a dedicated publication plan that routes every - namespace to one native variable plan. A generic remains a dispatch - surface rather than one bindable object, so it cannot be republished. - """ - for declaration, kind in ((item, "generic") for item in module.overload_sets): - exports = _declaration_exports(declaration) - relocated = [export for export in exports if home is None or tuple(export["namespace"]) != home] - if not relocated: - continue - declaring = "" if home is None else (".".join(home) or "") - namespaces = ", ".join(".".join(export["namespace"]) or "" for export in relocated) - raise ValueError( - f"{path}: {kind} {declaration.name!r} is declared in {declaring} and published in " - f"{namespaces}; this kind is publishable only by the namespace declaring it" - ) - - def _namespace_by_contract(tree: _PyiExportNode, entry: Path) -> dict[Path, tuple[str, ...]]: """Return the Python namespace each contract's own declarations live in. @@ -3558,9 +3534,9 @@ def _fortran_wrapper_module( assume_intent_in_scalars=assume_intent_in_scalars, ) if export_symbols is not None: - from prik.semantics.fortran_exports import select_fortran_export_functions + from prik.semantics.fortran_exports import select_fortran_export_symbols - selection = select_fortran_export_functions(modules, export_symbols) + selection = select_fortran_export_symbols(modules, export_symbols) for context_module in selection.context_modules: context_module.exported_names = [] modules = list(selection.available_modules) diff --git a/prik/pipeline/wrapper.py b/prik/pipeline/wrapper.py index 648cada95..453719023 100644 --- a/prik/pipeline/wrapper.py +++ b/prik/pipeline/wrapper.py @@ -132,6 +132,7 @@ WrapperPlanDiagnostic, ) from prik.planning.entrypoints import build_generated_support_procedure_projection +from prik.planning.planner import requires_cfi_header from prik.printers import CSourcePrinter, FortranSourcePrinter __all__ = ("GeneratedSource", "GeneratedWrapper", "WrapperGenerator") @@ -534,54 +535,13 @@ def _required_header_diagnostics(self, plan: ModulePlan) -> tuple[WrapperPlanDia if handle is not None ) expected_headers = list(self._native_array_required_headers(handles)) - if ( - any( - field.access - in { - DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR, - DerivedFieldAccessMechanism.NATIVE_ARRAY_HANDLE, - } - for namespace in plan.namespaces - for derived in namespace.derived_types - for field in derived.fields - ) - or self._accepts_array_handle_actual(plan) - or self._uses_array_descriptor_abi(plan) - ): + if requires_cfi_header(plan.namespaces): expected_headers.append(NATIVE_ARRAY_POINTER_C_DESCRIPTOR_HEADER) expected = tuple(dict.fromkeys(expected_headers)) if plan.required_headers == expected: return () return (self._diagnostic(plan.owner_path, "inconsistent-required-headers", plan.required_headers),) - @staticmethod - def _accepts_array_handle_actual(plan: ModulePlan) -> bool: - """Return whether an ordinary array argument accepts an array handle. - - The storage such a handle names is reached through its descriptor, so - the module needs the interop header even when nothing else in it does. - Only a Fortran argument accepts a handle, so the accepted sources are - the whole test. - """ - accepts = {NativeArraySourceKind.ALLOCATABLE_HANDLE, NativeArraySourceKind.POINTER_HANDLE} - return any( - argument.native_array_actual is not None - and accepts.intersection(argument.native_array_actual.accepted_sources) - for namespace in plan.namespaces - for function in namespace.functions - for argument in function.arguments - ) - - @staticmethod - def _uses_array_descriptor_abi(plan: ModulePlan) -> bool: - """Return whether an ordinary argument uses the standard descriptor ABI.""" - return any( - argument.array is not None and argument.array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR - for namespace in plan.namespaces - for function in namespace.functions - for argument in function.arguments - ) - def _namespace_native_array_handles( self, namespace: NamespacePlan, @@ -1345,7 +1305,7 @@ def _module_borrowed_array_view_diagnostics( diagnostics.append(self._diagnostic(plan.owner_path, "missing-module-array-getter-role", None)) # The route to the array's base address is a policy decision. Bridge # lowering reads it; it must never fall back to one when it is absent. - if plan.array_address is None: + if plan.storage_address is None: diagnostics.append(self._diagnostic(plan.owner_path, "missing-module-array-address-mechanism", None)) if plan.bridge.native_assignment is not AssignmentMode.NONE: diagnostics.append( diff --git a/prik/planning/entrypoints.py b/prik/planning/entrypoints.py index 4cf0f9733..3d25df4ca 100644 --- a/prik/planning/entrypoints.py +++ b/prik/planning/entrypoints.py @@ -1133,6 +1133,7 @@ def _primary_module_variable_operations(self, variable): elif variable.bridge.native_getter_action in { ModuleGetterAction.NULLABLE_SNAPSHOT, ModuleGetterAction.DERIVED_OBJECT, + ModuleGetterAction.NATIVE_SCALAR_VIEW, }: parameters = () result = self._opaque_result() diff --git a/prik/planning/models.py b/prik/planning/models.py index d04ffcfe0..f35f92642 100644 --- a/prik/planning/models.py +++ b/prik/planning/models.py @@ -66,7 +66,7 @@ DirectResultABI, DeclarationCallableAction, ExternalDeclarationMode, - ModuleArrayAddressMechanism, + ModuleStorageAddressMechanism, ModuleGetterAction, ModuleObjectAccessMechanism, NativeArrayDescriptorInterop, @@ -451,6 +451,7 @@ class OverloadPlan(StageRecord): candidate_passed_objects: tuple[bool, ...] unsupported_extra_argument_message: str | None = None identity_receiver_shortcut: bool = False + direct_single_candidate: bool = False docstring: str | None = None @@ -818,10 +819,9 @@ class ModuleVariablePlan(StageRecord): derived: DerivedModuleObjectPlan | None = None character_length: int | None = None docstring: str | None = None - # Present only for a borrowed fixed-array view. Both backends read it: the - # bridge to reach the address, the binding to define the C helper that one - # of the two mechanisms calls. It is therefore a shared fact, not a facet. - array_address: ModuleArrayAddressMechanism | None = None + # The original address of borrowed scalar or fixed-array module storage. + # Both backends read this shared fact when an address capture is required. + storage_address: ModuleStorageAddressMechanism | None = None @dataclass diff --git a/prik/planning/planner.py b/prik/planning/planner.py index 8e4ca2c85..c619af766 100644 --- a/prik/planning/planner.py +++ b/prik/planning/planner.py @@ -158,7 +158,32 @@ ) # Re-export reaches Python only where the published name is one exported object. -_ALIASABLE_REEXPORT_KINDS = frozenset({"procedure", "derived_type"}) +_ALIASABLE_REEXPORT_KINDS = frozenset({"procedure", "generic", "derived_type"}) + + +def requires_cfi_header(namespaces: tuple[NamespacePlan, ...]) -> bool: + """Read completed plans that require the standard C descriptor header.""" + accepts = {NativeArraySourceKind.ALLOCATABLE_HANDLE, NativeArraySourceKind.POINTER_HANDLE} + descriptor_fields = { + DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR, + DerivedFieldAccessMechanism.NATIVE_ARRAY_HANDLE, + } + return any( + field.access in descriptor_fields + for namespace in namespaces + for derived in namespace.derived_types + for field in derived.fields + ) or any( + argument.datatype_family is DatatypeFamily.ASSUMED_NATIVE + or (argument.array is not None and argument.array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR) + or ( + argument.native_array_actual is not None + and bool(accepts.intersection(argument.native_array_actual.accepted_sources)) + ) + for namespace in namespaces + for function in namespace.functions + for argument in function.arguments + ) _DATATYPE_FAMILIES = { @@ -670,7 +695,7 @@ def _exported_declaration_name( """ wanted = source_name.casefold() published: str | None = None - for declaration in (*module.functions, *module.classes): + for declaration in (*module.functions, *module.overload_sets, *module.classes): if getattr(declaration, "visibility", "public") != "public": continue native = str(getattr(declaration, "native_name", "") or declaration.name).casefold() @@ -1088,6 +1113,7 @@ def _overload_plan( candidate_passed_objects=tuple(candidate.passed_object for candidate in policy.candidates), unsupported_extra_argument_message=policy.unsupported_extra_argument_message, identity_receiver_shortcut=policy.identity_receiver_shortcut, + direct_single_candidate=policy.direct_single_candidate, ) def _class_callable_name(self, type_identity: tuple[str, str], name: str) -> str: @@ -1372,7 +1398,7 @@ def _module_variable_plan( native_assignment=policy.native_assignment, ), character_length=policy.character_length, - array_address=policy.array_address, + storage_address=policy.storage_address, array=self._array_plan(policy.array, policy.owner_path), native_array_handle=self._native_array_handle_plan(policy.native_array_handle, policy.owner_path), derived=( @@ -2819,58 +2845,10 @@ def _required_headers( if handle is not None ) headers = list(self._native_array_headers(handles)) - if ( - self._requires_derived_descriptor_header(namespaces) - or self._accepts_array_handle_actual(namespaces) - or self._uses_array_descriptor_abi(namespaces) - or any( - argument.datatype_family is DatatypeFamily.ASSUMED_NATIVE - for namespace in namespaces - for function in namespace.functions - for argument in function.arguments - ) - ): + if requires_cfi_header(namespaces): headers.append(NATIVE_ARRAY_POINTER_C_DESCRIPTOR_HEADER) return tuple(dict.fromkeys(headers)) - @staticmethod - def _uses_array_descriptor_abi(namespaces: tuple[NamespacePlan, ...]) -> bool: - """Return whether an ordinary argument uses the standard descriptor ABI.""" - return any( - argument.array is not None and argument.array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR - for namespace in namespaces - for function in namespace.functions - for argument in function.arguments - ) - - @staticmethod - def _accepts_array_handle_actual(namespaces: tuple[NamespacePlan, ...]) -> bool: - """Return whether an ordinary array argument accepts an array handle. - - The storage such a handle names is reached through its descriptor, so a - module whose ordinary array dummies accept one needs the interop header - even when nothing else about the module does. Only a Fortran argument - accepts one, so no separate language test is needed here. - """ - accepts = {NativeArraySourceKind.ALLOCATABLE_HANDLE, NativeArraySourceKind.POINTER_HANDLE} - return any( - argument.native_array_actual is not None - and accepts.intersection(argument.native_array_actual.accepted_sources) - for namespace in namespaces - for function in namespace.functions - for argument in function.arguments - ) - - @staticmethod - def _requires_derived_descriptor_header(namespaces: tuple[NamespacePlan, ...]) -> bool: - """Return whether one derived field uses a standard C descriptor callback.""" - descriptor_access = { - DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR, - DerivedFieldAccessMechanism.NATIVE_ARRAY_HANDLE, - } - fields = (field for namespace in namespaces for derived in namespace.derived_types for field in derived.fields) - return any(field.access in descriptor_access for field in fields) - def _native_array_headers(self, handles: tuple[NativeArrayHandlePlan, ...]) -> tuple[str, ...]: """Deduplicate planned handle headers in encounter order.""" return tuple(dict.fromkeys(header for handle in handles for header in handle.required_headers)) diff --git a/prik/policy/completion.py b/prik/policy/completion.py index c6e8ec3e5..d0756441e 100644 --- a/prik/policy/completion.py +++ b/prik/policy/completion.py @@ -903,7 +903,17 @@ def _complete_overload_policy( builtin_signatures = tuple(_overload_candidate_builtin_signature(candidate.arguments) for candidate in candidates) if len(set(builtin_signatures)) != len(builtin_signatures): blockers.append(f"overload {overload.owner_path!r} has overlapping reflected scalar signatures") - return replace(overload, candidates=tuple(candidates), blockers=tuple(dict.fromkeys(blockers))) + has_open_native_actual = any( + argument.semantic_type_name == "AnyNative" for candidate in candidates for argument in candidate.arguments + ) + if has_open_native_actual and len(candidates) > 1: + blockers.append(f"overload {overload.owner_path!r} cannot select among open native actuals") + return replace( + overload, + candidates=tuple(candidates), + blockers=tuple(dict.fromkeys(blockers)), + direct_single_candidate=has_open_native_actual and len(candidates) == 1, + ) def _overload_candidate_signature(arguments: tuple[OverloadArgumentPolicy, ...]) -> tuple: diff --git a/prik/policy/construction.py b/prik/policy/construction.py index fa95ebc3c..78c26071f 100644 --- a/prik/policy/construction.py +++ b/prik/policy/construction.py @@ -92,7 +92,7 @@ CallbackThreadAction, CallbackGILAction, CallbackFatalAction, - ModuleArrayAddressMechanism, + ModuleStorageAddressMechanism, ModuleGetterAction, ModuleObjectAccessMechanism, DerivedFieldAccessMechanism, @@ -1153,13 +1153,13 @@ def _ordinary_array_module_variable_policy( supported=not blockers, blockers=tuple(blockers), array=array, - array_address=address, + storage_address=address, ) def _ordinary_array_module_address_mechanism( variable: models.SemanticVariable, -) -> ModuleArrayAddressMechanism: +) -> ModuleStorageAddressMechanism: """Select how the bridge obtains one fixed module array's base address. Addressable storage names itself directly. An ordinary declaration cannot, @@ -1168,8 +1168,8 @@ def _ordinary_array_module_address_mechanism( storage; only the route to its address differs. """ if variable.semantic_type.metadata.get("aliased"): - return ModuleArrayAddressMechanism.TARGET_ADDRESS - return ModuleArrayAddressMechanism.CAPTURED_ADDRESS + return ModuleStorageAddressMechanism.TARGET_ADDRESS + return ModuleStorageAddressMechanism.CAPTURED_ADDRESS def _constant_array_module_variable_policy( @@ -1271,6 +1271,11 @@ def _scalar_module_variable_policy( ), supported=not blockers, blockers=tuple(blockers), + storage_address=( + ModuleStorageAddressMechanism.CAPTURED_ADDRESS + if getter_action is ModuleGetterAction.NATIVE_SCALAR_VIEW + else None + ), ) @@ -3242,6 +3247,7 @@ def _argument_policy( if argument.semantic_type.storage is not None and argument.semantic_type.storage.array is not None and argument.semantic_type.storage.array.contiguous + and argument.semantic_type.storage.array.category != "assumed_size" else None, ) if attr is not None @@ -6987,6 +6993,8 @@ def _scalar_module_getter_blockers( blockers.append("module variable getter is not a supported scalar policy") elif getter.codegen_action not in supported_getter_actions: blockers.append(f"module variable getter action {getter.codegen_action.value!r} is unsupported") + if getter_action is ModuleGetterAction.NATIVE_SCALAR_VIEW and variable.semantic_type.name == "String": + blockers.append("native scalar storage view requires a primitive numeric type") return tuple(blockers) @@ -7085,6 +7093,8 @@ def _scalar_module_getter_action( # A character value cannot cross the C ABI by value, so it copies # through a fixed-width byte buffer the way a character field does. return ModuleGetterAction.CHARACTER_VALUE + if variable.semantic_type.metadata.get("native_storage"): + return ModuleGetterAction.NATIVE_SCALAR_VIEW return ModuleGetterAction.DIRECT_VALUE @@ -7588,7 +7598,7 @@ def _array_handoff_policy( axes = tuple(str(item) for item in array.axes) flatten_python_storage = _array_handoff_flattens_python_storage(array) minimum_rank, maximum_rank = _array_handoff_rank_bounds(rank, array.category, flatten_python_storage) - if semantic_type.name == "AnyNative" and array.category == "assumed_rank": + if semantic_type.name == "AnyNative" and array.category in {"assumed_rank", "assumed_size"}: minimum_rank = 0 order = _array_handoff_order(array.order, array.category) entrypoint_abi = _array_entrypoint_abi( @@ -8228,7 +8238,11 @@ def _native_module(function: models.SemanticFunction, owner_path: str) -> str | """Return the completed native module scope for non-standalone procedures.""" if _is_standalone(function): return None - return str(function.origin.native_scope or owner_path.split(".", maxsplit=1)[0]) + return str( + function.metadata.get(models.NATIVE_ACCESS_MODULE_METADATA) + or function.origin.native_scope + or owner_path.split(".", maxsplit=1)[0] + ) def _native_is_subroutine(function: models.SemanticFunction) -> bool: diff --git a/prik/policy/contract_imports.py b/prik/policy/contract_imports.py index bd5581b90..2f7fa0b78 100644 --- a/prik/policy/contract_imports.py +++ b/prik/policy/contract_imports.py @@ -99,8 +99,19 @@ def bindings(self) -> list[str | models.SemanticImport]: ) for origin, source, local, written, kind in sorted(set(self._references())): self._bind(origin, source, local, written=written, verbatim=kind in {"prototype", "namespace"}) + self._complete_qualified_type_names() return self._statements + def _complete_qualified_type_names(self) -> None: + """Write procedure-local type names using the declaring contract's spelling.""" + for semantic_type in models._module_semantic_types(self._module): + reference = imported_type_reference(semantic_type) + if reference is None or not reference.procedure_local: + continue + name = contract_name_for_source(self._completed.get(reference.module.casefold()), reference.name) + if name is not None: + semantic_type.metadata[models.CONTRACT_NAME_METADATA] = f"{reference.module}.{name}" + def _stated(self, statement: str | models.SemanticImport) -> None: """Carry one import the module states itself.""" if isinstance(statement, str) or not statement.items: diff --git a/prik/policy/exports.py b/prik/policy/exports.py index 3c58a6fb6..1972d1123 100644 --- a/prik/policy/exports.py +++ b/prik/policy/exports.py @@ -47,7 +47,9 @@ def _stated_export_names(module: models.SemanticModule) -> set[str] | None: """ if module.exported_names is None: return None - return {str(name) for name in module.exported_names} + return { + str(name) for name in module.metadata.get(models.NATIVE_STATED_EXPORT_NAMES_METADATA, module.exported_names) + } def complete_python_export_policy( @@ -69,6 +71,8 @@ def complete_python_export_policy( default, and completing one would publish what the contract declined to. """ contract_named = bool(module.metadata.get(PYI_LOADED_METADATA)) + if not contract_named and module.exported_names is not None: + module.metadata.setdefault(models.NATIVE_STATED_EXPORT_NAMES_METADATA, list(module.exported_names)) complete_reexport_publication_policy(module, contract_named=contract_named) stated = _stated_export_names(module) naming = NamingPolicy( @@ -112,14 +116,29 @@ def complete_python_export_policy( strict_wrapper_names=strict_wrapper_names, contract_named=contract_named, ) + if not contract_named and module.exported_names is not None: + _complete_source_export_names(module) + + +def _complete_source_export_names(module: models.SemanticModule) -> None: + """Write the already-selected source surface with completed Python names.""" + selected = _stated_export_names(module) + module.exported_names = [ + models.completed_contract_name(owner) + for owner in _module_export_owners(module) + if str(owner.name) in selected and getattr(owner, "visibility", "public") == "public" + ] + module.exported_names.extend( + reexport.python_name + for reexport in module.reexports + if reexport.local_name in selected and reexport.publishes_to_python() + ) #: Entity kinds a second namespace cannot publish, whatever it may reach. #: -#: A generic dispatcher has no single object another namespace can bind, so it -#: is published where it is declared and nowhere else. An intrinsic module's -#: name has no declaration at all, so nothing is there to publish. -UNPUBLISHABLE_REEXPORT_KINDS = frozenset({"generic", "intrinsic"}) +#: An intrinsic module's name has no declaration a Python namespace can bind. +UNPUBLISHABLE_REEXPORT_KINDS = frozenset({"intrinsic"}) def complete_reexport_publication_policy( @@ -135,10 +154,9 @@ def complete_reexport_publication_policy( export surface, so every re-export record constructed from that surface is published. - A generic is reachable through the importing module like any other name, - but it dispatches rather than naming one object, so PRIK publishes it in - its declaring namespace alone. That is a publication decision, settled here - once, rather than an accessibility one. + A generic dispatcher is one Python callable in its declaring namespace, + which an importing namespace can publish through the same alias path as + an ordinary procedure. """ if contract_named is None: contract_named = bool(module.metadata.get(PYI_LOADED_METADATA)) diff --git a/prik/policy/models.py b/prik/policy/models.py index 3b8e93596..23e457f02 100644 --- a/prik/policy/models.py +++ b/prik/policy/models.py @@ -329,6 +329,7 @@ class ModuleGetterAction(str, Enum): NATIVE_CONSTANT_VALUE = "native_constant_value" NATIVE_CONSTANT_ARRAY_VALUE = "native_constant_array_value" DIRECT_VALUE = "direct_value" + NATIVE_SCALAR_VIEW = "native_scalar_view" CHARACTER_VALUE = "character_value" NULLABLE_SNAPSHOT = "nullable_snapshot" BORROWED_ARRAY_VIEW = "borrowed_array_view" @@ -336,18 +337,13 @@ class ModuleGetterAction(str, Enum): DERIVED_OBJECT = "derived_object" -class ModuleArrayAddressMechanism(str, Enum): - """Completed native mechanism that yields a fixed module array's base address. +class ModuleStorageAddressMechanism(str, Enum): + """Completed route to the original storage of a module scalar or array. - ``TARGET_ADDRESS`` applies to storage the declaration made addressable, where - ``c_loc`` names the array directly. ``CAPTURED_ADDRESS`` applies to an - ordinary array without that attribute: ``c_loc`` cannot name it, so the whole - array is handed to ``prik_capture_address``, a ``bind(C)`` primitive whose - assumed-type assumed-size dummy receives the bare base address. The - Fortran side forms no pointer and claims no target. The captured address is - valid for as long as the module variable keeps its storage, which the Fortran - standard does not guarantee across the program's lifetime; see the module - variable guide for the responsibility that carries. + ``TARGET_ADDRESS`` uses ``c_loc`` on a target array. ``CAPTURED_ADDRESS`` + passes a non-target scalar or array to a ``bind(C)`` identity procedure, + which returns its original address without copying. The captured address + remains valid only while the module variable keeps that storage. """ TARGET_ADDRESS = "target_address" @@ -742,6 +738,7 @@ class OverloadPolicy: blockers: tuple[str, ...] = () unsupported_extra_argument_message: str | None = None identity_receiver_shortcut: bool = False + direct_single_candidate: bool = False @dataclass(frozen=True) @@ -990,7 +987,7 @@ class ModuleVariablePolicy: blockers: tuple[str, ...] = () character_length: int | None = None array: ArrayHandoffPolicy | None = None - array_address: ModuleArrayAddressMechanism | None = None + storage_address: ModuleStorageAddressMechanism | None = None native_array_handle: NativeArrayHandleWrapperPolicy | None = None derived: DerivedModuleObjectPolicy | None = None diff --git a/prik/printers/pyi.py b/prik/printers/pyi.py index 92e156bd5..909a02841 100644 --- a/prik/printers/pyi.py +++ b/prik/printers/pyi.py @@ -46,6 +46,7 @@ OVERLOAD_KIND_METADATA, OVERLOAD_TARGET_METADATA, NATIVE_BY_VALUE_METADATA, + NATIVE_ACCESS_MODULE_METADATA, PYTHON_BOUND_POSITION_METADATA, PYTHON_METHOD_NAME_METADATA, PYTHON_STATIC_METADATA, @@ -1082,6 +1083,8 @@ def _emit_typed_name( semantic_type = self._without_constant_constraint(arg.semantic_type) type_text = self._visit(semantic_type, context) annotation_metadata = [] + if arg.semantic_type.metadata.get("native_storage") and arg.origin.source_kind == "variable": + annotation_metadata.append(context.contract("NativeStorage")) if original_name is not None: annotation_metadata.append(f"{context.contract('SourceName')}({json.dumps(original_name)})") if annotation_metadata: @@ -1970,7 +1973,9 @@ def _decorators( and any(str(attribute).casefold() == "pure" for attribute in func.metadata.get("fortran_attributes", ())) ): decorators.append(f"{indent}@{context.contract('pure')}") - if not func.metadata.get(OVERLOAD_TARGET_METADATA) and self._requires_native_call(func): + overload_target = func.metadata.get(OVERLOAD_TARGET_METADATA) + is_specific_declaration = not overload_target or str(func.name).casefold() == str(overload_target).casefold() + if is_specific_declaration and self._requires_native_call(func): decorators.append( f"{indent}{self._native_call(self._pyi_projection(func), context, self._native_result_projection(func), func)}" ) @@ -2000,6 +2005,8 @@ def _identity_decorators( decorators.append(f"{indent}@staticmethod") if func.metadata.get(DEFERRED_BINDING_METADATA): decorators.append(f"{indent}@{context.contract('abstractmethod')}") + if native_module := func.metadata.get(NATIVE_ACCESS_MODULE_METADATA): + decorators.append(f"{indent}@{context.contract('native_module')}({json.dumps(str(native_module))})") is_native_c_abi = func.origin.source_language == "fortran" and func.origin.native_abi == "c" is_overload = bool(func.metadata.get(OVERLOAD_TARGET_METADATA)) if is_native_c_abi and not is_overload: diff --git a/prik/runtime/native_support/prik_binding.h b/prik/runtime/native_support/prik_binding.h index 16c2c77cb..49b75f539 100644 --- a/prik/runtime/native_support/prik_binding.h +++ b/prik/runtime/native_support/prik_binding.h @@ -211,6 +211,11 @@ void *prik_capture_address(void *base) { return base; } + +void *prik_capture_scalar_address(void *base) +{ + return base; +} #endif /* diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index 02630776f..3880a71bc 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -506,6 +506,16 @@ def _visit_FortranVariable( declaration_arrays=declaration_arrays, ) + @staticmethod + def _has_native_scalar_storage(var: FortranVariable) -> bool: + """Identify concrete interoperable module storage with a stable address.""" + return ( + getattr(var, "_fortran_bind_c", False) + and var.rank == 0 + and not var.is_parameter + and var.base_type.casefold() in {"integer", "real", "complex", "logical"} + ) + def _convert_variable_type( self, var: FortranVariable, @@ -541,6 +551,8 @@ def _convert_variable_type( if getattr(var, "target", False): metadata["aliased"] = True metadata["fortran_target"] = True + if self._has_native_scalar_storage(var): + metadata["native_storage"] = True if getattr(var, "_fortran_protected", False): metadata["fortran_protected"] = True if getattr(var, "pointer", False): @@ -3128,20 +3140,20 @@ def _declared_specific(procedure: SemanticFunction) -> tuple[str, str]: ).key @staticmethod - def _bind_private_specifics_through_generic( + def _bind_unreachable_specifics_through_generic( overload_set: ProcedureOverloadSet, targets: list[_SpecificProcedure], lookup: dict[tuple[str, str], SemanticFunction], generic_name: str, + interface_body_targets: set[tuple[str, str]], ) -> None: - """Bind each private specific through the generic name that reaches it. + """Bind specifics without a module name through their reachable generic. - A specific its declaring module keeps private is unreachable by its own - name, while the generic -- or, for a constructor, the type name -- is - public and resolves to the same procedure. + A private module procedure and a procedure declared only inside an + interface body are both callable through the generic name. """ for target, candidate in zip(targets, overload_set.procedures, strict=True): - if lookup[target.key].visibility == "private": + if lookup[target.key].visibility == "private" or target.key in interface_body_targets: candidate.native_name = generic_name candidate.metadata[BIND_TARGET_METADATA] = generic_name @@ -3211,11 +3223,12 @@ def _module_overload_sets( # constructor, so its specifics become the class's own # `__init__` overload set rather than a module generic. constructor_set = self._normal_overload_set("__init__", procedures) - self._bind_private_specifics_through_generic( + self._bind_unreachable_specifics_through_generic( constructor_set, target_names, own_lookup | inline_lookup | inherited_lookup, interface.name, + set(inline_lookup), ) self._merge_overload_sets(constructor_class.overload_sets, [constructor_set]) self._mark_constructor_specifics(procedures, own_lookup, interface.name) @@ -3228,11 +3241,12 @@ def _module_overload_sets( else module.name, visibility=self._symbol_visibility(module, interface.name), ) - self._bind_private_specifics_through_generic( + self._bind_unreachable_specifics_through_generic( overload_set, target_names, own_lookup | inline_lookup | inherited_lookup, interface.name, + set(inline_lookup), ) overload_sets.append(overload_set) continue diff --git a/prik/semantics/fortran_exports.py b/prik/semantics/fortran_exports.py index f2d5f73a1..9e13b5cb2 100644 --- a/prik/semantics/fortran_exports.py +++ b/prik/semantics/fortran_exports.py @@ -7,11 +7,19 @@ from dataclasses import dataclass import re -from prik.semantics.models import ProcedureOverloadSet, SemanticFunction, SemanticModule +from prik.semantics.metadata import BIND_TARGET_METADATA +from prik.semantics.models import ( + EXTERNAL_TYPE_REF_METADATA, + NATIVE_ACCESS_MODULE_METADATA, + ProcedureOverloadSet, + SemanticFunction, + SemanticModule, + _semantic_type_tree, +) _FORTRAN_IDENTIFIER = r"[A-Za-z][A-Za-z0-9_]*" -_FORTRAN_EXPORT_RE = re.compile(rf"^(?P{_FORTRAN_IDENTIFIER})::(?P{_FORTRAN_IDENTIFIER})$") +_FORTRAN_EXPORT_RE = re.compile(rf"^(?P{_FORTRAN_IDENTIFIER})::(?P{_FORTRAN_IDENTIFIER})$") @dataclass(frozen=True) @@ -28,23 +36,22 @@ def available_modules(self) -> tuple[SemanticModule, ...]: def parse_fortran_export_identity(value: str) -> tuple[str, str]: - """Return one case-folded ``module::procedure`` identity or raise.""" + """Return one case-folded ``module::symbol`` identity or raise.""" match = _FORTRAN_EXPORT_RE.fullmatch(str(value)) if match is None: - raise ValueError(f"invalid Fortran procedure identity: {value}") - return match.group("module").casefold(), match.group("procedure").casefold() + raise ValueError(f"invalid Fortran module symbol identity: {value}") + return match.group("module").casefold(), match.group("symbol").casefold() -def select_fortran_export_functions( +def select_fortran_export_symbols( modules: Iterable[SemanticModule], symbols: Iterable[str], ) -> FortranExportSelection: - """Select exact module procedures with their semantic source context. + """Select exact module procedures and variables with semantic source context. Selection is expressed in native identities before policy names anything. - Primary module copies contain only the requested callable declarations; - their classes, prototypes, variables, and reexports remain available as - signature facts but are not added to the stated public surface. Other + Primary module copies contain only requested procedures and variables; + classes and prototypes remain available as signature facts. Other source modules remain available as context. Contract-import policy decides which of them the generated contract needs to emit. """ @@ -55,35 +62,26 @@ def select_fortran_export_functions( ) requested = _validated_fortran_export_symbols(symbols) module_index = {_native_module_name(module): module for module in source_modules} - callable_index, non_callable_index = _fortran_export_candidates(source_modules) - _validate_fortran_export_resolution(requested, callable_index, non_callable_index, module_index) + selectable, non_selectable = _fortran_export_candidates(source_modules) + _validate_fortran_export_resolution(requested, selectable, non_selectable, module_index) - selected = set(requested) - primary_names = {module_name for module_name, _procedure_name in requested} + selected, access_modules = _selection_routes(requested, module_index) + primary_names = {module_name for module_name, _symbol_name in selected} primary_sources = [] primary_modules = [] for module in source_modules: module_name = _native_module_name(module) if module_name not in primary_names: continue - selected_module = deepcopy(module) - selected_module.functions = [ - function - for function in selected_module.functions - if (module_name, _native_procedure_name(function)) in selected - ] - selected_module.overload_sets = [ - overload - for overload in selected_module.overload_sets - if (module_name, _native_procedure_name(overload)) in selected - ] - selected_module.exported_names = [ - declaration.name for declaration in (*selected_module.functions, *selected_module.overload_sets) - ] + selected_module = _select_module_surface(module, selected, set(requested), access_modules) primary_sources.append(module) primary_modules.append(selected_module) - # Root selection owns only the requested callable surface. Contract-import + required_types = _required_type_identities(primary_modules, selected) + for module in primary_modules: + _retain_required_types(module, module_index[_native_module_name(module)], required_types) + + # Root selection owns only the requested symbol surface. Contract-import # completion already owns which available modules selected declarations # actually name, including private callback prototypes and imported types # that are not Fortran reexports. Keep the remaining modules available and @@ -93,10 +91,136 @@ def select_fortran_export_functions( return FortranExportSelection(tuple(primary_sources), tuple(primary_modules), context_modules) +def _selection_routes(requested, module_index): + """Resolve selected facade names to their declaring identities and access routes.""" + selected = set(requested) + access_modules: dict[tuple[str, str], str] = {} + for module_name, symbol_name in requested: + for reexport in module_index[module_name].reexports: + if reexport.local_name.casefold() == symbol_name: + identity = (reexport.origin_module.casefold(), reexport.source_name.casefold()) + selected.add(identity) + access_modules[identity] = module_name + return selected, access_modules + + +def _select_module_surface(module, selected, requested, access_modules): + """Retain selected declarations while keeping generic specifics private to them.""" + selected_module = deepcopy(module) + module_name = _native_module_name(module) + selected_module.overload_sets = [ + overload + for overload in selected_module.overload_sets + if (module_name, _native_symbol_name(overload)) in selected + ] + _retain_selected_procedures(selected_module, module_name, selected) + _route_selected_callables(selected_module, module_name, access_modules) + selected_module.variables = [ + variable for variable in selected_module.variables if (module_name, _native_symbol_name(variable)) in selected + ] + selected_module.reexports = [ + reexport for reexport in selected_module.reexports if (module_name, reexport.local_name.casefold()) in requested + ] + for reexport in selected_module.reexports: + reexport.explicitly_public = True + reexport.python_exported = None + selected_module.exported_names = [ + declaration.name + for declaration in (*selected_module.functions, *selected_module.overload_sets, *selected_module.variables) + if (module_name, _native_symbol_name(declaration)) in selected + ] + selected_module.exported_names.extend(reexport.local_name for reexport in selected_module.reexports) + return selected_module + + +def _retain_selected_procedures(module, module_name, selected): + """Keep named procedures and the specifics of each selected generic.""" + specifics = { + _native_symbol_name(procedure): procedure + for overload in module.overload_sets + for procedure in overload.procedures + } + module.functions = [ + function + for function in module.functions + if (module_name, _native_symbol_name(function)) in selected or _native_symbol_name(function) in specifics + ] + declared = {_native_symbol_name(function) for function in module.functions} + module.functions.extend(deepcopy(procedure) for name, procedure in specifics.items() if name not in declared) + + +def _route_selected_callables(module, module_name, access_modules): + """Record the public native module and generic name used for each callable.""" + for declaration in (*module.functions, *module.overload_sets): + access = access_modules.get((module_name, _native_symbol_name(declaration))) + if access is None: + continue + procedures = declaration.procedures if isinstance(declaration, ProcedureOverloadSet) else (declaration,) + for procedure in procedures: + procedure.metadata[NATIVE_ACCESS_MODULE_METADATA] = access + if isinstance(declaration, ProcedureOverloadSet): + procedure.native_name = declaration.name + procedure.metadata[BIND_TARGET_METADATA] = declaration.name + + +def _retain_required_types(module, source_module, required_types): + """Publish only derived types required by selected values or signatures.""" + module_name = _native_module_name(module) + module.classes = [cls for cls in module.classes if (module_name, _native_symbol_name(cls)) in required_types] + for cls in module.classes: + cls.methods = [] + cls.overload_sets = [] + if cls.name not in module.exported_names: + module.exported_names.append(cls.name) + for reexport in source_module.reexports: + identity = (reexport.origin_module.casefold(), reexport.source_name.casefold()) + if reexport.entity_kind != "derived_type" or identity not in required_types: + continue + if any(item.local_name.casefold() == reexport.local_name.casefold() for item in module.reexports): + continue + dependency = deepcopy(reexport) + dependency.explicitly_public = True + dependency.python_exported = None + module.reexports.append(dependency) + module.exported_names.append(dependency.local_name) + + +def _required_type_identities(modules: list[SemanticModule], selected: set[tuple[str, str]]) -> set[tuple[str, str]]: + """Find derived declarations named by selected signatures and values.""" + required: set[tuple[str, str]] = set() + for module in modules: + module_name = _native_module_name(module) + owners = ( + *module.variables, + *module.functions, + *module.overload_sets, + ) + for owner in owners: + if (module_name, _native_symbol_name(owner)) not in selected: + continue + functions = owner.procedures if isinstance(owner, ProcedureOverloadSet) else (owner,) + for declaration in functions: + types = ( + (declaration.semantic_type,) + if hasattr(declaration, "semantic_type") + else ( + *(argument.semantic_type for argument in declaration.arguments), + declaration.return_type, + ) + ) + for semantic_type in types: + for item in _semantic_type_tree(semantic_type): + reference = item.metadata.get(EXTERNAL_TYPE_REF_METADATA) + origin = reference.get("origin_module") if isinstance(reference, dict) else module_name + name = reference.get("name") if isinstance(reference, dict) else item.name + required.add((str(origin).casefold(), str(name).casefold())) + return required + + def _validated_fortran_export_symbols(symbols: Iterable[str]) -> tuple[tuple[str, str], ...]: requested_text = tuple(str(symbol) for symbol in symbols) if not requested_text: - raise ValueError("Fortran export-symbol selection requires at least one module procedure identity") + raise ValueError("Fortran export-symbol selection requires at least one module symbol identity") requested = [] invalid = [] repeated = [] @@ -113,7 +237,7 @@ def _validated_fortran_export_symbols(symbols: Iterable[str]) -> tuple[tuple[str requested.append(identity) problems = [] if invalid: - problems.append("invalid procedure identities: " + ", ".join(invalid)) + problems.append("invalid symbol identities: " + ", ".join(invalid)) if repeated: problems.append("repeated identities: " + ", ".join(repeated)) if problems: @@ -122,38 +246,43 @@ def _validated_fortran_export_symbols(symbols: Iterable[str]) -> tuple[tuple[str def _fortran_export_candidates(modules: tuple[SemanticModule, ...]): - callables: dict[tuple[str, str], list[object]] = {} - non_callables: set[tuple[str, str]] = set() + selectable: dict[tuple[str, str], list[object]] = {} + non_selectable: set[tuple[str, str]] = set() for module in modules: module_name = _native_module_name(module) - for declaration in (*module.functions, *module.overload_sets): - callables.setdefault((module_name, _native_procedure_name(declaration)), []).append(declaration) - for declaration in (*module.variables, *module.classes, *module.prototypes): - non_callables.add((module_name, _native_procedure_name(declaration))) - return callables, non_callables + for declaration in (*module.functions, *module.overload_sets, *module.variables): + selectable.setdefault((module_name, _native_symbol_name(declaration)), []).append(declaration) + for reexport in module.reexports: + if reexport.entity_kind in {"procedure", "generic", "variable"}: + selectable.setdefault((module_name, reexport.local_name.casefold()), []).append(reexport) + for declaration in (*module.classes, *module.prototypes): + non_selectable.add((module_name, _native_symbol_name(declaration))) + return selectable, non_selectable -def _validate_fortran_export_resolution(requested, callables, non_callables, module_index) -> None: +def _validate_fortran_export_resolution(requested, selectable, non_selectable, module_index) -> None: problems = [] unknown_modules = [module for module, _name in requested if module not in module_index] - non_functions = [identity for identity in requested if identity in non_callables and identity not in callables] + non_symbols = [identity for identity in requested if identity in non_selectable and identity not in selectable] unknown = [ identity for identity in requested - if identity[0] in module_index and identity not in callables and identity not in non_callables + if identity[0] in module_index and identity not in selectable and identity not in non_selectable ] - ambiguous = [identity for identity in requested if len(callables.get(identity, ())) > 1] + ambiguous = [identity for identity in requested if len(selectable.get(identity, ())) > 1] inaccessible = [ identity for identity in requested - if any(getattr(declaration, "visibility", "public") == "private" for declaration in callables.get(identity, ())) + if any( + getattr(declaration, "visibility", "public") == "private" for declaration in selectable.get(identity, ()) + ) ] for label, identities in ( ("unknown modules", tuple(dict.fromkeys(unknown_modules))), - ("unknown procedures", unknown), - ("non-function declarations", non_functions), - ("ambiguous procedures", ambiguous), - ("private procedures", inaccessible), + ("unknown symbols", unknown), + ("unsupported declarations", non_symbols), + ("ambiguous symbols", ambiguous), + ("private symbols", inaccessible), ): if identities: formatted = [item if isinstance(item, str) else "::".join(item) for item in identities] @@ -166,7 +295,7 @@ def _native_module_name(module: SemanticModule) -> str: return str(module.origin.native_name or module.name).casefold() -def _native_procedure_name(declaration: object) -> str: +def _native_symbol_name(declaration: object) -> str: if isinstance(declaration, ProcedureOverloadSet): return str(declaration.name).casefold() if isinstance(declaration, SemanticFunction): diff --git a/prik/semantics/models.py b/prik/semantics/models.py index fd07317ba..b37f87596 100644 --- a/prik/semantics/models.py +++ b/prik/semantics/models.py @@ -28,6 +28,7 @@ PYTHON_VALUE_MUTABILITY_METADATA = "python_value_mutability" PYTHON_VALUE_IMMUTABLE = "immutable" NATIVE_BY_VALUE_METADATA = "native_by_value" +NATIVE_ACCESS_MODULE_METADATA = "native_access_module" RUNTIME_RELEASE_GIL_METADATA = "runtime_release_gil" RUNTIME_RETAIN_RESULT_OWNER_METADATA = "runtime_retain_result_owner" RUNTIME_STATUS_ERROR_METADATA = "runtime_status_error" @@ -423,6 +424,7 @@ class ProcedureOverloadSet: PYTHON_BOUND_POSITION_METADATA = "python_bound_position" PYTHON_METHOD_NAME_METADATA = "python_method_name" PYTHON_EXPORTS_METADATA = "python_exports" +NATIVE_STATED_EXPORT_NAMES_METADATA = "native_stated_export_names" CONTRACT_NAME_METADATA = "contract_name" CONTRACT_TARGET_NAME_METADATA = "contract_target_name" CONTRACT_BASE_NAMES_METADATA = "contract_base_names" diff --git a/prik/semantics/pyi2ir.py b/prik/semantics/pyi2ir.py index 231767c35..ff61db0e4 100644 --- a/prik/semantics/pyi2ir.py +++ b/prik/semantics/pyi2ir.py @@ -58,6 +58,7 @@ OVERLOAD_KIND_METADATA, OVERLOAD_TARGET_METADATA, NATIVE_BY_VALUE_METADATA, + NATIVE_ACCESS_MODULE_METADATA, PYTHON_BOUND_POSITION_METADATA, PYTHON_METHOD_NAME_METADATA, PYTHON_STATIC_METADATA, @@ -164,6 +165,7 @@ class _Decorators: overload_generic: str | None = None bind_target: str | None = None native_abi: str | None = None + native_module: str | None = None standalone: bool = False is_static: bool = False release_gil: bool = False @@ -896,6 +898,7 @@ def _apply_decorator(self, parsed: _Decorators, node: ast.expr, *, context: str) "overload": self._apply_overload_decorator, "bind": self._apply_bind_decorator, "native_abi": self._apply_native_abi_decorator, + "native_module": self._apply_native_module_decorator, "standalone": self._apply_standalone_decorator, "nogil": self._apply_nogil_decorator, "native_call": self._apply_native_call_decorator, @@ -1022,6 +1025,17 @@ def _apply_native_abi_decorator(self, parsed: _Decorators, node: ast.expr, conte raise ValueError('native_abi accepts only "c"') parsed.native_abi = "c" + def _apply_native_module_decorator(self, parsed: _Decorators, node: ast.expr, context: str) -> None: + """Record the Fortran module through which a native procedure is accessed.""" + if parsed.native_module is not None: + raise ValueError(f"Duplicate {context} native_module decorator") + if self.native_language != "fortran": + raise ValueError("native_module is only valid for Fortran semantic .pyi declarations") + value = self._required_string_decorator_argument(node, "native_module") + if re.fullmatch(r"[A-Za-z][A-Za-z0-9_]*", value) is None: + raise ValueError(f"native_module requires a Fortran module name: {value!r}") + parsed.native_module = value + @staticmethod def _apply_nogil_decorator(parsed: _Decorators, node: ast.expr, context: str) -> None: """Record a no-argument GIL-release request in decorator state.""" @@ -1213,6 +1227,8 @@ def _validated_overload_candidate( for key in (RUNTIME_RELEASE_GIL_METADATA, RUNTIME_STATUS_ERROR_METADATA): if key in declaration.metadata: candidate.metadata[key] = deepcopy(declaration.metadata[key]) + if NATIVE_ACCESS_MODULE_METADATA in declaration.metadata: + candidate.metadata[NATIVE_ACCESS_MODULE_METADATA] = declaration.metadata[NATIVE_ACCESS_MODULE_METADATA] if isinstance(owner, SemanticModule): if generic_name is not None: @@ -2525,18 +2541,7 @@ def _apply_metadata_name(self, semantic_type: SemanticType, name: str) -> bool: ordinary constraints while contradictions raise immediately. """ if name in {"ORDER_C", "ORDER_F", "ORDER_ANY"}: - array = self._require_array_storage(semantic_type) - if array.rank is None or array.rank <= 1: - raise ValueError(f"{name} requires a multidimensional array") - expected_order = self._flat_array_order(array.source_shape, array.rank) - if expected_order is not None and name != expected_order: - raise ValueError(f"{name} conflicts with {expected_order} implied by Flat placement") - default_order = self._array_order_for_dimensions(array.category, array.rank, array.source_shape) - if expected_order is None and name == default_order: - raise ValueError( - f"{name} is implicit for {self.native_language} semantic .pyi contracts; remove the annotation" - ) - array.order = name + self._apply_array_order_metadata(semantic_type, name) return True if name == "COPY_F": self._require_array_storage(semantic_type).copy_order = "ORDER_F" @@ -2564,6 +2569,9 @@ def _apply_metadata_name(self, semantic_type: SemanticType, name: str) -> bool: if name == "MaybeUnallocated": semantic_type.metadata[MAYBE_UNALLOCATED_METADATA] = True return True + if name == "NativeStorage": + semantic_type.metadata["native_storage"] = True + return True if name == "FortranAllocatable": semantic_type.metadata["fortran_allocatable"] = True return True @@ -2576,6 +2584,21 @@ def _apply_metadata_name(self, semantic_type: SemanticType, name: str) -> bool: return True return False + def _apply_array_order_metadata(self, semantic_type: SemanticType, name: str) -> None: + """Validate one explicit order against the declaration's shape and default.""" + array = self._require_array_storage(semantic_type) + if array.rank is None or array.rank <= 1: + raise ValueError(f"{name} requires a multidimensional array") + expected_order = self._flat_array_order(array.source_shape, array.rank) + if expected_order is not None and name != expected_order: + raise ValueError(f"{name} conflicts with {expected_order} implied by Flat placement") + default_order = self._array_order_for_dimensions(array.category, array.rank, array.source_shape) + if expected_order is None and name == default_order: + raise ValueError( + f"{name} is implicit for {self.native_language} semantic .pyi contracts; remove the annotation" + ) + array.order = name + @staticmethod def _validate_array_copy_metadata(semantic_type: SemanticType) -> None: """Fail closed on representation-copy forms outside the first dense lane.""" @@ -3667,6 +3690,8 @@ def _visit_FunctionDef(self, node: ast.FunctionDef) -> None: return if decorators.standalone: raise ValueError("standalone is not valid for a class method") + if decorators.native_module is not None: + raise ValueError("native_module is only valid for module procedures") if not node.decorator_list and self._is_generated_constructor(node): self.constructor_from_fields = True return @@ -3743,6 +3768,7 @@ def _visit_ClassDef(self, node: ast.ClassDef) -> None: or decorators.release_gil or decorators.error_status_policy is not None or decorators.standalone + or decorators.native_module is not None or decorators.abstract_method or decorators.destroy ): @@ -3877,6 +3903,8 @@ def _visit_FunctionDef(self, node: ast.FunctionDef) -> None: # The same fact a Fortran source records, which a specification # function in a declaration expression is required to carry. function.metadata["fortran_attributes"] = [*function.metadata.get("fortran_attributes", ()), "pure"] + if decorators.native_module is not None: + function.metadata[NATIVE_ACCESS_MODULE_METADATA] = decorators.native_module if decorators.overload_target is not None: self.parser._pending_overloads.append( _PendingOverload( diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/native/assumed_type_calls.f90 b/tests/fortran/assumed_types/end_to_end/fixtures/native/assumed_type_calls.f90 index dcc4689ed..bd9bc1463 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/native/assumed_type_calls.f90 +++ b/tests/fortran/assumed_types/end_to_end/fixtures/native/assumed_type_calls.f90 @@ -1,6 +1,11 @@ module assumed_type_calls use iso_c_binding + integer(c_int), bind(C), target :: native_value = 17 contains + logical(c_bool) function same_native_raw(x) bind(C) + type(*), dimension(*), target :: x + same_native_raw = c_associated(c_loc(x), c_loc(native_value)) + end function integer(c_int) function scalar_without_intent(x) bind(C) type(*) :: x scalar_without_intent = 11 diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py new file mode 100644 index 000000000..726f4aa33 --- /dev/null +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py @@ -0,0 +1,64 @@ +"""Executed by the Open MPI integration test under two or more ranks.""" + +import numpy as np + +from prik_openmpi_f08 import mpi_f08 as mpi + + +mpi.mpi_init() +assert isinstance(mpi.mpi_comm_world, mpi.Mpi_Comm) +assert isinstance(mpi.mpi_int, mpi.Mpi_Datatype) +assert isinstance(mpi.mpi_sum, mpi.Mpi_Op) +assert isinstance(mpi.mpi_status_ignore, mpi.Mpi_Status) +rank, rank_error = mpi.mpi_comm_rank(mpi.mpi_comm_world) +size, size_error = mpi.mpi_comm_size(mpi.mpi_comm_world) +rank, size = int(rank), int(size) +assert rank_error is None and size_error is None and size >= 2 +mpi.mpi_barrier(mpi.mpi_comm_world) + +if rank == 0: + sent = np.array([3, 5, 7, 11], dtype=np.int32) + mpi.mpi_send(sent, np.int32(sent.size), mpi.mpi_int, np.int32(1), np.int32(13), mpi.mpi_comm_world) +elif rank == 1: + received = np.empty(4, dtype=np.int32) + mpi.mpi_recv( + received, + np.int32(received.size), + mpi.mpi_int, + np.int32(0), + np.int32(13), + mpi.mpi_comm_world, + mpi.mpi_status_ignore, + ) + np.testing.assert_array_equal(received, [3, 5, 7, 11]) + +values = np.array([rank + 1, rank + 2], dtype=np.int32) +expected = np.array([size * (size + 1) // 2, size * (size + 3) // 2], dtype=np.int32) +reduced = np.empty_like(values) +mpi.mpi_allreduce(values, reduced, np.int32(values.size), mpi.mpi_int, mpi.mpi_sum, mpi.mpi_comm_world) +np.testing.assert_array_equal(reduced, expected) +floats = np.array([float(rank + 1)], dtype=np.float64) +float_reduced = np.empty_like(floats) +mpi.mpi_allreduce( + floats, + float_reduced, + np.int32(floats.size), + mpi.mpi_double_precision, + mpi.mpi_sum, + mpi.mpi_comm_world, +) +np.testing.assert_array_equal(float_reduced, [float(expected[0])]) + +in_place = values.copy() +assert isinstance(mpi.mpi_in_place, np.ndarray) and mpi.mpi_in_place.shape == () +mpi.mpi_allreduce( + mpi.mpi_in_place, + in_place, + np.int32(in_place.size), + mpi.mpi_int, + mpi.mpi_sum, + mpi.mpi_comm_world, +) +np.testing.assert_array_equal(in_place, expected) +mpi.mpi_finalize() +print(f"Open MPI rank {rank}: communication passed", flush=True) diff --git a/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py b/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py index 1c08d4f58..8ecb4ca45 100644 --- a/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py +++ b/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py @@ -117,12 +117,26 @@ def test_same_actual_uses_address_or_descriptor_from_dummy(calls): assert calls.scalar_without_intent(np.array(3, dtype=np.int64)) == 11 assert calls.adapted_rank(np.float64(3)) == 0 assert calls.assumed_size(np.arange(3, dtype=np.uint64)) == 20 + assert calls.assumed_size(np.array(3, dtype=np.int64)) == 20 with pytest.raises(TypeError, match="no supported descriptor dtype"): calls.assumed_rank(np.arange(3, dtype=np.uint64)) with pytest.raises(TypeError, match="cannot contain Python object references"): calls.assumed_size(np.array([object()], dtype=object)) +def test_native_scalar_module_storage_survives_generated_contract_replay(calls, native_build): + """A BIND(C) scalar view passes its own native address through TYPE(*).""" + generated = (native_build.output_dir / "contracts" / "assumed_type_calls.pyi").read_text() + assert "native_value: Annotated[Int32, Aliased, NativeStorage]" in generated + native = calls.native_value + assert isinstance(native, np.ndarray) and native.shape == () and native.dtype == np.dtype("int32") + assert bool(calls.same_native_raw(native)) + assert calls.assumed_rank(native) == 0 + native[()] = np.int32(29) + assert calls.native_value[()] == 29 + assert bool(calls.same_native_raw(calls.native_value)) + + def test_optional_absence_uses_dummy_specific_null_representation(calls): assert calls.optional_address() == 0 assert calls.optional_address(None) == 0 diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py new file mode 100644 index 000000000..10a5b1d29 --- /dev/null +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -0,0 +1,154 @@ +"""Real Open MPI source to generated contract to two-rank execution.""" + +from __future__ import annotations + +import os +import re +import shlex +import shutil +import subprocess +import sys +from pathlib import Path + +import pytest + +from prik.pipeline.build import NativeLinkItem, build_pyi_extension + + +pytestmark = pytest.mark.fortran_end_to_end +RUNTIME = Path(__file__).parent / "fixtures" / "runtime" / "openmpi_basic.py" +EXPORTS = ( + "MPI_Init", + "MPI_Finalize", + "MPI_Comm_rank", + "MPI_Comm_size", + "MPI_Barrier", + "MPI_Send", + "MPI_Recv", + "MPI_Allreduce", + "MPI_COMM_WORLD", + "MPI_INT", + "MPI_DOUBLE_PRECISION", + "MPI_SUM", + "MPI_IN_PLACE", + "MPI_STATUS_IGNORE", +) + + +def _configured_openmpi() -> tuple[Path, Path, str, str, str]: + """Find matching configured sources, wrapper compiler, and Open MPI launcher.""" + source_text = os.environ.get("PRIK_OPENMPI_SOURCE") + build_text = os.environ.get("PRIK_OPENMPI_BUILD") + if not source_text or not build_text: + pytest.skip("set PRIK_OPENMPI_SOURCE and PRIK_OPENMPI_BUILD to a matching configured Open MPI tree") + source, build = Path(source_text), Path(build_text) + mpifort = os.environ.get("PRIK_OPENMPI_MPIFORT") or shutil.which("mpifort") + launcher = os.environ.get("PRIK_OPENMPI_LAUNCHER") or shutil.which("orterun") or shutil.which("mpirun") + if not mpifort or not launcher: + pytest.skip("Open MPI Fortran compiler wrapper and launcher are required") + for path in ( + source / "ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-types.F90", + source / "ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-interfaces.F90", + source / "ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90", + build / "ompi/mpi/fortran/configure-fortran-output.h", + build / "ompi/mpi/fortran/use-mpi-f08/sizeof_f08.h", + ): + if not path.is_file(): + pytest.skip(f"configured Open MPI semantic input is unavailable: {path}") + version_file = (source / "VERSION").read_text(encoding="utf-8") + parts = [re.search(rf"^{part}=(\d+)$", version_file, flags=re.MULTILINE) for part in ("major", "minor", "release")] + if any(part is None for part in parts): + pytest.skip("Open MPI source version could not be read") + version = ".".join(part.group(1) for part in parts if part is not None) + compiler_version = subprocess.check_output([mpifort, "--showme:version"], text=True) + launcher_version = subprocess.check_output([launcher, "--version"], text=True) + if ( + f"Open MPI {version}" not in compiler_version + or version not in launcher_version + or not any(label in launcher_version for label in ("Open MPI", "OpenRTE")) + ): + pytest.skip("configured sources, mpifort, and launcher must belong to the same Open MPI version") + return source, build, mpifort, launcher, version + + +def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) -> None: + """The selected facade and native storage survive a real .pyi replay build.""" + source, build, mpifort, launcher, _version = _configured_openmpi() + contract = tmp_path / "contract" + exports = tmp_path / "exports.txt" + exports.write_text("".join(f"mpi_f08::{symbol}\n" for symbol in EXPORTS), encoding="utf-8") + includes = ( + build, + build / "ompi/mpi/fortran/use-mpi-f08", + source, + build / "ompi/include", + source / "ompi/include", + ) + subprocess.run( + [ + sys.executable, + "-m", + "prik", + "generate", + "--pyi", + str(source / "ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-types.F90"), + str(source / "ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-interfaces.F90"), + str(source / "ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90"), + "--export-symbols", + str(exports), + "--out", + str(contract), + "--compiler", + mpifort, + *(part for include in includes for part in ("-I", str(include))), + ], + check=True, + capture_output=True, + text=True, + timeout=180, + ) + facade = (contract / "mpi_f08.pyi").read_text(encoding="utf-8") + types = (contract / "mpi_f08_types.pyi").read_text(encoding="utf-8") + interfaces = (contract / "mpi_f08_interfaces.pyi").read_text(encoding="utf-8") + assert all(f'"{symbol.lower()}"' in facade for symbol in EXPORTS) + assert all(f'"Mpi_{name}"' in facade for name in ("Comm", "Datatype", "Op", "Status")) + assert "mpi_waitall" not in facade + assert "mpi_in_place: Annotated[Int32, NativeStorage]" in types + assert "mpi_comm_world: Final[Mpi_Comm]" in types + assert "mpi_sum: Final[Mpi_Op]" in types + assert "mpi_int: Final[Mpi_Datatype]" in types + assert "mpi_status_ignore: Mpi_Status" in types + assert all(f"class Mpi_{name}" in types for name in ("Comm", "Datatype", "Op", "Status")) + assert "AnyNative[" in interfaces and '@native_module("mpi_f08")' in interfaces + + def show(flag: str) -> list[str]: + return shlex.split(subprocess.check_output([mpifort, flag], text=True)) + + compile_flags, link_flags = show("--showme:compile"), show("--showme:link") + result = build_pyi_extension( + contract / "__init__.pyi", + input_compiler=show("--showme:command")[0], + native_include_dirs=[flag[2:] for flag in compile_flags if flag.startswith("-I")], + native_link_items=[NativeLinkItem("linker_argument", flag) for flag in link_flags], + native_linker_language="fortran", + output_name="prik_openmpi_f08", + output_dir=tmp_path / "extension", + jobs=2, + ) + assert result.native_build_plan is not None and not result.native_build_plan.compilation_units + bridge = next(path for path in result.generated_sources if path.suffix == ".f90").read_text(encoding="utf-8") + assert "use mpi_f08, only:" in bridge + assert "=> MPI_Allreduce" in bridge and "=> MPI_Send" in bridge + env = os.environ.copy() + env["PYTHONPATH"] = os.pathsep.join(filter(None, (str(result.output_dir), env.get("PYTHONPATH", "")))) + libdirs = [flag[2:] for flag in link_flags if flag.startswith("-L")] + env["LD_LIBRARY_PATH"] = os.pathsep.join((*libdirs, env.get("LD_LIBRARY_PATH", ""))) + completed = subprocess.run( + [launcher, "-n", "2", sys.executable, str(RUNTIME)], + env=env, + check=True, + capture_output=True, + text=True, + timeout=120, + ) + assert completed.stdout.count("communication passed") == 2 diff --git a/tests/fortran/functions/end_to_end/fixtures/native/export_selection_facade/facade.f90 b/tests/fortran/functions/end_to_end/fixtures/native/export_selection_facade/facade.f90 new file mode 100644 index 000000000..60517622d --- /dev/null +++ b/tests/fortran/functions/end_to_end/fixtures/native/export_selection_facade/facade.f90 @@ -0,0 +1,3 @@ +module facade + use owner, only: run, marker +end module diff --git a/tests/fortran/functions/end_to_end/fixtures/native/export_selection_facade/owner.f90 b/tests/fortran/functions/end_to_end/fixtures/native/export_selection_facade/owner.f90 new file mode 100644 index 000000000..d2d896b99 --- /dev/null +++ b/tests/fortran/functions/end_to_end/fixtures/native/export_selection_facade/owner.f90 @@ -0,0 +1,12 @@ +module owner + use iso_c_binding + integer(c_int), bind(C) :: marker = 7 + interface run + module procedure run_impl + end interface +contains + subroutine run_impl(value) + integer(c_int), intent(out) :: value + value = marker + end subroutine +end module diff --git a/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py b/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py index b9549ad79..39585c105 100644 --- a/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py +++ b/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py @@ -129,3 +129,56 @@ def callback(value, status, *, observed=seen): assert module.solve(np.int32(4), callback) == np.int32(5) assert seen == [(np.int32(5), None)] + + +@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") +def test_facade_selection_and_contract_replay_share_generic_and_native_variable(tmp_path: Path): + """A selected facade binds one generic and one live native scalar in both lanes.""" + sources = tuple((NATIVE.parent / "export_selection_facade" / name) for name in ("owner.f90", "facade.f90")) + exports = tmp_path / "exports.txt" + exports.write_text("facade::run\nfacade::marker\n", encoding="utf-8") + contract = tmp_path / "contract" + subprocess.run( + [ + sys.executable, + "-m", + "prik", + "generate", + "--pyi", + *map(str, sources), + "--export-symbols", + str(exports), + "--out", + str(contract), + ], + check=True, + capture_output=True, + text=True, + ) + facade_contract = (contract / "facade.pyi").read_text(encoding="utf-8") + owner_contract = (contract / "owner.pyi").read_text(encoding="utf-8") + assert "from .owner import" in facade_contract + assert '"run"' in facade_contract and '"marker"' in facade_contract + assert '@native_module("facade")' in owner_contract + assert "marker: Annotated[Int32, NativeStorage]" in owner_contract + + source = build_fortran_extension( + sources, + output_name="facade_source", + output_dir=tmp_path / "source", + export_symbols=("facade::run", "facade::marker"), + jobs=2, + ) + replay = build_pyi_extension( + contract / "__init__.pyi", + native_fortran_sources=sources, + output_name="facade_replay", + output_dir=tmp_path / "replay", + jobs=2, + ) + for result in (source, replay): + module = _import_from_build_dir(result.module_name, result.output_dir).facade + assert module.run() == np.int32(7) + assert isinstance(module.marker, np.ndarray) and module.marker.shape == () + module.marker[()] = np.int32(11) + assert module.run() == np.int32(11) diff --git a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py index 109920806..2b699e2a0 100644 --- a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py +++ b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py @@ -5,7 +5,8 @@ import pytest from prik.cli import _read_export_symbols -from prik.semantics.fortran_exports import select_fortran_export_functions +from prik.semantics.models import NATIVE_ACCESS_MODULE_METADATA +from prik.semantics.fortran_exports import select_fortran_export_symbols from prik.semantics.models import ( ProcedureOverloadSet, SemanticFunction, @@ -57,7 +58,7 @@ def test_selection_keeps_one_callable_root_and_its_declaration_module(): ], ) - result = select_fortran_export_functions([callbacks, selected], ["SOLVER_MOD::SOLVE"]) + result = select_fortran_export_symbols([callbacks, selected], ["SOLVER_MOD::SOLVE"]) assert [module.name for module in result.primary_modules] == ["solver_mod"] assert [function.name for function in result.primary_modules[0].functions] == ["solve"] @@ -66,6 +67,28 @@ def test_selection_keeps_one_callable_root_and_its_declaration_module(): assert result.context_modules[0].exported_names is None +def test_selection_publishes_a_variable_with_its_concrete_type(): + """A selected module object keeps its parsed type and native owner.""" + selected = _module( + "state_mod", + variables=[ + SemanticVariable( + name="sentinel", + semantic_type=SemanticType(name="Int32"), + origin=SemanticOrigin(source_language="fortran", native_name="sentinel", native_scope="state_mod"), + ), + SemanticVariable(name="other", semantic_type=SemanticType(name="Int32")), + ], + ) + + result = select_fortran_export_symbols([selected], ["STATE_MOD::SENTINEL"]) + + assert [variable.name for variable in result.primary_modules[0].variables] == ["sentinel"] + assert result.primary_modules[0].variables[0].semantic_type.name == "Int32" + assert result.primary_modules[0].variables[0].origin.native_scope == "state_mod" + assert result.primary_modules[0].exported_names == ["sentinel"] + + def test_fortran_export_file_accepts_comments_and_rejects_case_insensitive_duplicates(tmp_path: Path): export_file = tmp_path / "exports.txt" export_file.write_text("# reviewed\nsolver_mod::solve # public\n", encoding="utf-8") @@ -79,12 +102,11 @@ def test_fortran_export_file_accepts_comments_and_rejects_case_insensitive_dupli @pytest.mark.parametrize( ("symbols", "message"), [ - ([], "requires at least one module procedure identity"), - (["solve"], "invalid procedure identities: solve"), + ([], "requires at least one module symbol identity"), + (["solve"], "invalid symbol identities: solve"), (["solver_mod::solve", "SOLVER_MOD::SOLVE"], "repeated identities"), (["missing_mod::solve"], "unknown modules: missing_mod"), - (["solver_mod::missing"], "unknown procedures: solver_mod::missing"), - (["solver_mod::state"], "non-function declarations: solver_mod::state"), + (["solver_mod::missing"], "unknown symbols: solver_mod::missing"), ], ) def test_selection_rejects_invalid_or_unresolved_identities(symbols, message): @@ -95,14 +117,14 @@ def test_selection_rejects_invalid_or_unresolved_identities(symbols, message): ) with pytest.raises(ValueError, match=message): - select_fortran_export_functions([module], symbols) + select_fortran_export_symbols([module], symbols) def test_selection_rejects_private_module_procedure(): hidden = _function("solver_mod", "hidden") hidden.visibility = "private" - with pytest.raises(ValueError, match="private procedures: solver_mod::hidden"): - select_fortran_export_functions([_module("solver_mod", functions=[hidden])], ["solver_mod::hidden"]) + with pytest.raises(ValueError, match="private symbols: solver_mod::hidden"): + select_fortran_export_symbols([_module("solver_mod", functions=[hidden])], ["solver_mod::hidden"]) def test_selection_keeps_one_generic_with_its_specific_candidates(): @@ -115,17 +137,49 @@ def test_selection_keeps_one_generic_with_its_specific_candidates(): ) module.overload_sets = [generic] - selected = select_fortran_export_functions([module], ["SOLVER_MOD::SOLVE"]).primary_modules[0] + selected = select_fortran_export_symbols([module], ["SOLVER_MOD::SOLVE"]).primary_modules[0] - assert selected.functions == [] + assert [function.name for function in selected.functions] == ["solve_int", "solve_real"] assert [overload.name for overload in selected.overload_sets] == ["solve"] assert [candidate.name for candidate in selected.overload_sets[0].procedures] == ["solve_int", "solve_real"] assert selected.exported_names == ["solve"] +def test_facade_selection_retains_only_requested_native_owners_and_access_route(): + """A facade allowlist selects owner declarations without publishing siblings.""" + specific = _function("owner", "run_impl") + owner = _module( + "owner", + functions=[specific, _function("owner", "unrelated")], + variables=[ + SemanticVariable(name="marker", semantic_type=SemanticType(name="Int32")), + SemanticVariable(name="unrelated_state", semantic_type=SemanticType(name="Int32")), + ], + ) + owner.overload_sets = [ProcedureOverloadSet(name="run", procedures=[specific], native_scope="owner")] + facade = _module( + "facade", + reexports=[ + SemanticReexport("run", "owner", "run", "facade", entity_kind="generic"), + SemanticReexport("marker", "owner", "marker", "facade", entity_kind="variable"), + SemanticReexport("unrelated", "owner", "unrelated", "facade", entity_kind="procedure"), + ], + ) + + selected = select_fortran_export_symbols([owner, facade], ["facade::run", "facade::marker"]) + + owner_selected, facade_selected = selected.primary_modules + assert [function.name for function in owner_selected.functions] == ["run_impl"] + assert [variable.name for variable in owner_selected.variables] == ["marker"] + assert [item.local_name for item in facade_selected.reexports] == ["run", "marker"] + candidate = owner_selected.overload_sets[0].procedures[0] + assert candidate.native_name == "run" + assert candidate.metadata[NATIVE_ACCESS_MODULE_METADATA] == "facade" + + def test_external_root_cannot_satisfy_a_module_qualified_identity(): external = _module("foo", functions=[_function("foo", "external")]) external.origin.source_kind = "external_root" with pytest.raises(ValueError, match="unknown modules: foo"): - select_fortran_export_functions([external], ["foo::external"]) + select_fortran_export_symbols([external], ["foo::external"]) diff --git a/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/pipeline/test_declaring_namespace_publication.py b/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/pipeline/test_declaring_namespace_publication.py index 9920f4dfc..f9c36826c 100644 --- a/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/pipeline/test_declaring_namespace_publication.py +++ b/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/pipeline/test_declaring_namespace_publication.py @@ -3,8 +3,8 @@ A procedure or a derived type reaches Python as one object, so another namespace can bind it and PRIK re-exports it through an alias. A module variable likewise permits multiple publications, but all of them refer to the -one variable plan and its live native state. A generic remains a dispatch -surface rather than one object, so only its declaring namespace can publish it. +one variable plan and its live native state. A generic's Python dispatcher is +also one object that a facade can publish. """ from __future__ import annotations @@ -122,24 +122,18 @@ def test_a_facade_may_be_the_only_publication_of_a_declared_variable(tmp_path: P @pytest.mark.parametrize("home_exports", [ALL_NAMES, [item for item in ALL_NAMES if item != "area"]]) -def test_a_generic_cannot_be_published_from_a_facade(home_exports: list[str], tmp_path: Path): - """A generic is not one native entity that another namespace can bind. - - The restriction holds whether its declaring namespace also publishes it or - the facade is its only requested publication. - """ +def test_a_generic_can_be_published_from_a_facade(home_exports: list[str], tmp_path: Path): + """One generic dispatcher serves its declaring namespace and facade.""" entry = _package( tmp_path, home_exports=home_exports, facade='from .home import area\n\n__all__ = ["area"]\n', ) - with pytest.raises(ValueError) as error: - _plan(entry, tmp_path, "facade_area") + result = _plan(entry, tmp_path, "facade_area") - message = str(error.value) - assert "generic 'area' is declared in home and published in facade" in message - assert "publishable only by the namespace declaring it" in message + generated = (result.output_dir / "facade_area_wrapper.c").read_text(encoding="utf-8") + assert generated.count('"area"') >= 1 def test_a_procedure_still_reaches_python_through_a_facade(tmp_path: Path): diff --git a/tests/fortran/modules/codegen/test_module_array_view_lowering.py b/tests/fortran/modules/codegen/test_module_array_view_lowering.py index e137929a5..a4ed679e2 100644 --- a/tests/fortran/modules/codegen/test_module_array_view_lowering.py +++ b/tests/fortran/modules/codegen/test_module_array_view_lowering.py @@ -145,7 +145,7 @@ def _undecided_plan(): complete_semantic_policies(module) plan = WrapperPlanner().build(module) variable = plan.variables[0] - return plan, variable, replace(variable, array_address=None) + return plan, variable, replace(variable, storage_address=None) def test_module_array_view_plan_rejects_a_missing_address_mechanism(): diff --git a/tests/fortran/modules/parsing/test_project_scope_models.py b/tests/fortran/modules/parsing/test_project_scope_models.py index 229a962d3..1bd12f395 100644 --- a/tests/fortran/modules/parsing/test_project_scope_models.py +++ b/tests/fortran/modules/parsing/test_project_scope_models.py @@ -53,6 +53,24 @@ def test_declaration_level_private_attribute_overrides_public_module_default(): assert module.private_symbols == ["epsilon"] +def test_separate_parameter_statement_and_bind_c_module_storage_are_preserved(): + module = parse_fortran_file( + """ +module native_constants + integer :: limit + parameter (limit = 4) + integer, bind(c) :: addressable +end module native_constants +""" + ).modules[0] + + variables = {variable.name: variable for variable in module.variables} + assert variables["limit"].is_parameter + assert variables["limit"].value == "4" + assert not variables["addressable"].is_parameter + assert variables["addressable"]._fortran_bind_c + + def test_submodule_types_interfaces_and_project_dependencies_attach_to_public_models(): code = """ submodule (ancestor_mod:parent_mod) child_mod diff --git a/tests/fortran/modules/policy/test_module_variable_policy.py b/tests/fortran/modules/policy/test_module_variable_policy.py index 9ae5616f3..97a77f065 100644 --- a/tests/fortran/modules/policy/test_module_variable_policy.py +++ b/tests/fortran/modules/policy/test_module_variable_policy.py @@ -10,7 +10,7 @@ from prik.semantics.models import RESOLVED_MODULE_VARIABLE_POLICY_METADATA from prik.policy.ownership import AssignmentMode from prik.policy.models import ( - ModuleArrayAddressMechanism, + ModuleStorageAddressMechanism, ModuleGetterAction, ModuleVariablePolicy, NativeArrayDescriptorAttribute, @@ -172,8 +172,8 @@ def test_fixed_module_array_address_mechanism_follows_declared_addressability(): } assert [policy.supported for policy in policies.values()] == [True, True] assert policies["values"].getter_action is ModuleGetterAction.BORROWED_ARRAY_VIEW - assert policies["values"].array_address is ModuleArrayAddressMechanism.CAPTURED_ADDRESS - assert policies["addressable"].array_address is ModuleArrayAddressMechanism.TARGET_ADDRESS + assert policies["values"].storage_address is ModuleStorageAddressMechanism.CAPTURED_ADDRESS + assert policies["addressable"].storage_address is ModuleStorageAddressMechanism.TARGET_ADDRESS # Neither route hands Python the whole variable back to reassign. assert policies["values"].setter_action is SetterAction.REJECT_REPLACEMENT assert policies["addressable"].setter_action is SetterAction.REJECT_REPLACEMENT From 66260e2bc49e7eebcea94548a6bfa6f0ffc3ca6a Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 04:10:07 +0100 Subject: [PATCH 02/47] codex: express native module storage and facade binds in contracts --- CHANGELOG.md | 3 +- docs/user/guide/wrapping-modules.md | 5 +- docs/user/reference/pyi-format.md | 19 ++++---- prik/contracts/__init__.py | 4 -- prik/printers/pyi.py | 21 +++++++-- prik/semantics/pyi2ir.py | 47 ++++++++++--------- .../end_to_end/test_assumed_type_calls.py | 2 +- .../end_to_end/test_openmpi_f08.py | 4 +- .../test_fortran_export_symbol_workflow.py | 4 +- .../parsing/test_python_ast_contracts.py | 23 +++++++++ 10 files changed, 83 insertions(+), 49 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ab99f2f67..ce836f773 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,7 +20,8 @@ release tags add a leading `v` to the package version. - `--export-symbols` and `build_fortran_extension(export_symbols=...)` accept module-qualified Fortran procedures and variables, including symbols re-exported by a public facade. Generated contracts retain the selected - access module, required type declarations, and native scalar storage views. + access module through qualified `@bind`, required type declarations, and + native scalar storage views through `T[()]`. - The Open MPI `mpi_f08` tutorial and opt-in two-rank integration test build a wrapper from a restricted generated `.pyi` against a matching prebuilt Open MPI installation and exercise NumPy communication and in-place reduction. diff --git a/docs/user/guide/wrapping-modules.md b/docs/user/guide/wrapping-modules.md index 892acde75..564420a4d 100644 --- a/docs/user/guide/wrapping-modules.md +++ b/docs/user/guide/wrapping-modules.md @@ -137,8 +137,9 @@ Fixed-shape module arrays and derived-type array fields expose live views whether or not their declarations include `target`. Logical array dtypes follow the [data-type mapping](data-types.md#scalar-type-mapping). An addressable scalar module variable declared with `bind(C)` is also a live -rank-zero NumPy view. Use `value[()]` to read or update it; pass the view itself -when a `TYPE(*)` dummy needs its original native storage. +rank-zero NumPy view, written `T[()]` in its contract. Use `value[()]` to read +or update it; pass the view itself when a `TYPE(*)` dummy needs its original +native storage. - Allocatable module arrays use the `Allocatable[T[...]]` API. - Allocation, lifetime, NumPy views, and mutation rules are covered in the storage and objects section. diff --git a/docs/user/reference/pyi-format.md b/docs/user/reference/pyi-format.md index 106c3d40c..c3dbb0718 100644 --- a/docs/user/reference/pyi-format.md +++ b/docs/user/reference/pyi-format.md @@ -306,8 +306,8 @@ def update(value: Float64[()]) -> None: ... The leaf filename supplies the native module. No placement decorator is needed. When a public Fortran facade re-exports a procedure from a supporting module, -`@native_module("facade_name")` selects the module the bridge imports. Generated -contracts retain this access route, so building from the `.pyi` uses the facade. +`@bind("facade_name::procedure_name")` selects the module and procedure that the +bridge calls. Generated contracts retain this access route. ### Standalone Procedures @@ -406,9 +406,10 @@ scale: Float64 = 2.0 Fortran module variables can be buildable getters, setters, constants, wrapped objects, or descriptor handles according to their completed policy. A literal default on supported mutable scalar state is an import-time native initializer. -`Annotated[Int32, NativeStorage]` describes a scalar module variable whose -Python value is a live rank-zero NumPy view over its native storage. Use this -marker when rebuilding from a contract must preserve the variable's address. +`Int32[()]` describes a numeric scalar module variable whose Python value is a +live rank-zero NumPy view over its native storage. Pass that view to a native dummy +when the variable's original address matters. Plain `Int32` provides a scalar +value getter. C global declarations can be represented for inspection, but current C wrapper builds reject native global state. C functions remain the supported runtime @@ -558,9 +559,8 @@ Python declaration and native callable names differ. | Decorator | Valid target | Language and meaning | | --- | --- | --- | | `@private` | Function or method | Shared: declaration remains available to contract dependencies but is not exported. | -| `@bind("symbol")` | Function, method, constructor, prototype, or destructor | Shared: select a different native name. | +| `@bind("symbol")` | Function, method, constructor, prototype, or destructor | Shared: select a different native name. For module-level Fortran procedures, `"module::symbol"` also selects the native module used for the call. | | `@native_abi("c")` | Function, method, or prototype | Fortran only: original declaration is `bind(C)`. | -| `@native_module("module_name")` | Module-level function | Fortran only: import the procedure through this native module. | | `@standalone` | Module-level function | Fortran only: native procedure is outside a module. | | `@native_call([...], result=...)` | Function, method, or constructor | Shared: state the complete native argument order and optional native result mapping. | | `@overload("specific", generic=...)` | Function or method | Shared: add one exact candidate to a generated Python overload set. | @@ -810,7 +810,7 @@ stores or passes it: | Contract | Meaning | Languages | | --- | --- | --- | | `T` | Scalar Python value or wrapped object. | Shared. | -| `T[()]` | Caller-owned rank-zero NumPy storage. | Shared. | +| `T[()]` | Rank-zero NumPy storage; supported numeric module variables expose live native storage. | Shared. | | `T[n]` | Rank-one array with extent `n`. | Shared. | | `T[:]` | Rank-one array with runtime extent. | Shared. | | `T[:, :]` | Rank-two array with runtime extents. | Shared. | @@ -936,7 +936,6 @@ type. Metadata falls into four groups. | `PointerAssociation("runtime")` | Pointer association is runtime state. | Fortran pointer facts. | | `SourceName("native-name")` | Preserve a native name that the Python identifier cannot represent. | Shared. | | `FortranAllocatable` | Fortran allocatable source provenance. | Loaded source contracts. | -| `NativeStorage` | Scalar module variable uses its existing native address. | Prebuilt Fortran library state exposed as a live rank-zero NumPy view. | | `CAnonymousMember` | Field is the containing member for a nested anonymous C aggregate. | C inspection contracts. | ### Constraints @@ -1110,7 +1109,7 @@ valid and whether it is buildable. | Prototype direction | `In`, `Out`, `InOut` | | Native-call helpers | `Arg`, `Hidden`, `IsPresent`, `Len`, `Pass`, `Return`, `Value`, `Work` | | Exact C scalar helpers | `CBool`, `CChar`, `CSignedChar`, `CUnsignedChar`, `CShort`, `CUnsignedShort`, `CInt`, `CUnsignedInt`, `CLong`, `CUnsignedLong`, `CLongLong`, `CUnsignedLongLong`, `CFloat`, `CDouble`, `CLongDouble`, `CFloatComplex`, `CDoubleComplex`, `CLongDoubleComplex` | -| Decorators | `abstract`, `abstractmethod`, `bind`, `destroy`, `native_abi`, `native_call`, `native_module`, `nogil`, `overload`, `private`, `prototype`, `pure`, `raises`, `standalone` | +| Decorators | `abstract`, `abstractmethod`, `bind`, `destroy`, `native_abi`, `native_call`, `nogil`, `overload`, `private`, `prototype`, `pure`, `raises`, `standalone` | `staticmethod` is supported Python syntax for methods but is not exported by `prik.contracts`. diff --git a/prik/contracts/__init__.py b/prik/contracts/__init__.py index 02735c8e5..f22e8364c 100644 --- a/prik/contracts/__init__.py +++ b/prik/contracts/__init__.py @@ -236,7 +236,6 @@ def apply(target): FortranAllocatable = _ContractExpression() Immutable = _ContractExpression() MaybeUnallocated = _ContractExpression() -NativeStorage = _ContractExpression() ORDER_ANY = _ContractExpression() ORDER_C = _ContractExpression() ORDER_F = _ContractExpression() @@ -330,7 +329,6 @@ def destroy(target): bind = _decorator nogil = _decorator native_abi = _decorator -native_module = _decorator native_call = _decorator overload = _decorator prototype = _decorator @@ -396,7 +394,6 @@ def destroy(target): "Matrix", "AnyNative", "MaybeUnallocated", - "NativeStorage", "Opaque", "OpaqueHandle", "ORDER_ANY", @@ -433,7 +430,6 @@ def destroy(target): "destroy", "nogil", "native_abi", - "native_module", "native_call", "overload", "prototype", diff --git a/prik/printers/pyi.py b/prik/printers/pyi.py index 909a02841..0d46667d5 100644 --- a/prik/printers/pyi.py +++ b/prik/printers/pyi.py @@ -415,6 +415,7 @@ def _visit_ProcedureOverloadSet( if candidate.origin.native_symbol != candidate.origin.native_name else None ) + bind_target = self._module_bind_target(candidate, bind_target) bind = f"{indent}@{context.contract('bind')}({json.dumps(str(bind_target))})\n" if bind_target else "" native_abi = ( f'{indent}@{context.contract("native_abi")}("c")\n' @@ -1081,10 +1082,14 @@ def _emit_typed_name( ) -> str: """Emit typed name syntax.""" semantic_type = self._without_constant_constraint(arg.semantic_type) + if arg.semantic_type.metadata.get("native_storage"): + semantic_type = deepcopy(semantic_type) + semantic_type.storage = SemanticStorageContract( + kind="array", + array=SemanticArrayContract(rank=0, category=SCALAR_STORAGE_CATEGORY), + ) type_text = self._visit(semantic_type, context) annotation_metadata = [] - if arg.semantic_type.metadata.get("native_storage") and arg.origin.source_kind == "variable": - annotation_metadata.append(context.contract("NativeStorage")) if original_name is not None: annotation_metadata.append(f"{context.contract('SourceName')}({json.dumps(original_name)})") if annotation_metadata: @@ -2005,8 +2010,6 @@ def _identity_decorators( decorators.append(f"{indent}@staticmethod") if func.metadata.get(DEFERRED_BINDING_METADATA): decorators.append(f"{indent}@{context.contract('abstractmethod')}") - if native_module := func.metadata.get(NATIVE_ACCESS_MODULE_METADATA): - decorators.append(f"{indent}@{context.contract('native_module')}({json.dumps(str(native_module))})") is_native_c_abi = func.origin.source_language == "fortran" and func.origin.native_abi == "c" is_overload = bool(func.metadata.get(OVERLOAD_TARGET_METADATA)) if is_native_c_abi and not is_overload: @@ -2014,10 +2017,20 @@ def _identity_decorators( bind_target = self._bind_target( func, context=context, emitted_name=emitted_name, is_native_c_abi=is_native_c_abi ) + bind_target = self._module_bind_target(func, bind_target) if bind_target and not is_overload: decorators.append(f"{indent}@{context.contract('bind')}({json.dumps(str(bind_target))})") return decorators + @staticmethod + def _module_bind_target(func: SemanticFunction, bind_target: object | None) -> str | None: + """Render a Fortran module access route in the existing bind spelling.""" + module = func.metadata.get(NATIVE_ACCESS_MODULE_METADATA) + if module is None: + return str(bind_target) if bind_target is not None else None + symbol = bind_target or func.native_name or func.name + return f"{module}::{symbol}" + @staticmethod def _constructor_binds_its_own_type( overload_name: str, diff --git a/prik/semantics/pyi2ir.py b/prik/semantics/pyi2ir.py index ff61db0e4..524353a23 100644 --- a/prik/semantics/pyi2ir.py +++ b/prik/semantics/pyi2ir.py @@ -165,7 +165,7 @@ class _Decorators: overload_generic: str | None = None bind_target: str | None = None native_abi: str | None = None - native_module: str | None = None + bind_module: str | None = None standalone: bool = False is_static: bool = False release_gil: bool = False @@ -898,7 +898,6 @@ def _apply_decorator(self, parsed: _Decorators, node: ast.expr, *, context: str) "overload": self._apply_overload_decorator, "bind": self._apply_bind_decorator, "native_abi": self._apply_native_abi_decorator, - "native_module": self._apply_native_module_decorator, "standalone": self._apply_standalone_decorator, "nogil": self._apply_nogil_decorator, "native_call": self._apply_native_call_decorator, @@ -1012,7 +1011,15 @@ def _apply_bind_decorator(self, parsed: _Decorators, node: ast.expr, context: st """Store one native symbol binding in decorator state, rejecting duplicates.""" if parsed.bind_target is not None: raise ValueError(f"Duplicate {context} bind decorator") - parsed.bind_target = self._required_string_decorator_argument(node, "bind") + target = self._required_string_decorator_argument(node, "bind") + if "::" in target: + if self.native_language != "fortran" or context != ".pyi": + raise ValueError("qualified bind is only valid for Fortran module procedures") + parts = target.split("::") + if len(parts) != 2 or any(re.fullmatch(r"[A-Za-z][A-Za-z0-9_]*", part) is None for part in parts): + raise ValueError(f"bind requires a Fortran module::procedure name: {target!r}") + parsed.bind_module, target = parts + parsed.bind_target = target def _apply_native_abi_decorator(self, parsed: _Decorators, node: ast.expr, context: str) -> None: """Retain the C ABI declared by an original Fortran declaration.""" @@ -1025,17 +1032,6 @@ def _apply_native_abi_decorator(self, parsed: _Decorators, node: ast.expr, conte raise ValueError('native_abi accepts only "c"') parsed.native_abi = "c" - def _apply_native_module_decorator(self, parsed: _Decorators, node: ast.expr, context: str) -> None: - """Record the Fortran module through which a native procedure is accessed.""" - if parsed.native_module is not None: - raise ValueError(f"Duplicate {context} native_module decorator") - if self.native_language != "fortran": - raise ValueError("native_module is only valid for Fortran semantic .pyi declarations") - value = self._required_string_decorator_argument(node, "native_module") - if re.fullmatch(r"[A-Za-z][A-Za-z0-9_]*", value) is None: - raise ValueError(f"native_module requires a Fortran module name: {value!r}") - parsed.native_module = value - @staticmethod def _apply_nogil_decorator(parsed: _Decorators, node: ast.expr, context: str) -> None: """Record a no-argument GIL-release request in decorator state.""" @@ -2569,9 +2565,6 @@ def _apply_metadata_name(self, semantic_type: SemanticType, name: str) -> bool: if name == "MaybeUnallocated": semantic_type.metadata[MAYBE_UNALLOCATED_METADATA] = True return True - if name == "NativeStorage": - semantic_type.metadata["native_storage"] = True - return True if name == "FortranAllocatable": semantic_type.metadata["fortran_allocatable"] = True return True @@ -3690,8 +3683,6 @@ def _visit_FunctionDef(self, node: ast.FunctionDef) -> None: return if decorators.standalone: raise ValueError("standalone is not valid for a class method") - if decorators.native_module is not None: - raise ValueError("native_module is only valid for module procedures") if not node.decorator_list and self._is_generated_constructor(node): self.constructor_from_fields = True return @@ -3768,7 +3759,6 @@ def _visit_ClassDef(self, node: ast.ClassDef) -> None: or decorators.release_gil or decorators.error_status_policy is not None or decorators.standalone - or decorators.native_module is not None or decorators.abstract_method or decorators.destroy ): @@ -3825,7 +3815,14 @@ def _visit_ImportFrom(self, node: ast.ImportFrom) -> None: def _visit_AnnAssign(self, node: ast.AnnAssign) -> None: """Convert a module variable declaration.""" - self.parser.module.variables.append(self.parser.ann_assign(node)) + variable = self.parser.ann_assign(node) + storage = variable.semantic_type.storage + if storage is not None and storage.array is not None and storage.array.category == SCALAR_STORAGE_CATEGORY: + if self.parser.native_language != "fortran": + raise ValueError("rank-zero module storage is only supported for Fortran") + variable.semantic_type.storage = None + variable.semantic_type.metadata["native_storage"] = True + self.parser.module.variables.append(variable) def _visit_Assign(self, node: ast.Assign) -> None: """Record the list of names this contract states that it publishes.""" @@ -3876,6 +3873,8 @@ def _visit_FunctionDef(self, node: ast.FunctionDef) -> None: """Convert a function or overload declaration.""" decorators = self.parser.decorators(node.decorator_list, context=".pyi") if decorators.prototype: + if decorators.bind_module is not None: + raise ValueError("qualified bind requires a Fortran module procedure") self.parser.module.prototypes.append( self.parser.prototype_def( node, @@ -3903,8 +3902,10 @@ def _visit_FunctionDef(self, node: ast.FunctionDef) -> None: # The same fact a Fortran source records, which a specification # function in a declaration expression is required to carry. function.metadata["fortran_attributes"] = [*function.metadata.get("fortran_attributes", ()), "pure"] - if decorators.native_module is not None: - function.metadata[NATIVE_ACCESS_MODULE_METADATA] = decorators.native_module + if decorators.bind_module is not None: + if decorators.standalone or decorators.prototype: + raise ValueError("qualified bind requires a Fortran module procedure") + function.metadata[NATIVE_ACCESS_MODULE_METADATA] = decorators.bind_module if decorators.overload_target is not None: self.parser._pending_overloads.append( _PendingOverload( diff --git a/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py b/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py index 8ecb4ca45..626887cd1 100644 --- a/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py +++ b/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py @@ -127,7 +127,7 @@ def test_same_actual_uses_address_or_descriptor_from_dummy(calls): def test_native_scalar_module_storage_survives_generated_contract_replay(calls, native_build): """A BIND(C) scalar view passes its own native address through TYPE(*).""" generated = (native_build.output_dir / "contracts" / "assumed_type_calls.pyi").read_text() - assert "native_value: Annotated[Int32, Aliased, NativeStorage]" in generated + assert "native_value: Annotated[Int32[()], Aliased]" in generated native = calls.native_value assert isinstance(native, np.ndarray) and native.shape == () and native.dtype == np.dtype("int32") assert bool(calls.same_native_raw(native)) diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index 10a5b1d29..cf1760053 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -113,13 +113,13 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) assert all(f'"{symbol.lower()}"' in facade for symbol in EXPORTS) assert all(f'"Mpi_{name}"' in facade for name in ("Comm", "Datatype", "Op", "Status")) assert "mpi_waitall" not in facade - assert "mpi_in_place: Annotated[Int32, NativeStorage]" in types + assert "mpi_in_place: Int32[()]" in types assert "mpi_comm_world: Final[Mpi_Comm]" in types assert "mpi_sum: Final[Mpi_Op]" in types assert "mpi_int: Final[Mpi_Datatype]" in types assert "mpi_status_ignore: Mpi_Status" in types assert all(f"class Mpi_{name}" in types for name in ("Comm", "Datatype", "Op", "Status")) - assert "AnyNative[" in interfaces and '@native_module("mpi_f08")' in interfaces + assert "AnyNative[" in interfaces and '@bind("mpi_f08::MPI_Send")' in interfaces def show(flag: str) -> list[str]: return shlex.split(subprocess.check_output([mpifort, flag], text=True)) diff --git a/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py b/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py index 39585c105..1452835ff 100644 --- a/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py +++ b/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py @@ -159,8 +159,8 @@ def test_facade_selection_and_contract_replay_share_generic_and_native_variable( owner_contract = (contract / "owner.pyi").read_text(encoding="utf-8") assert "from .owner import" in facade_contract assert '"run"' in facade_contract and '"marker"' in facade_contract - assert '@native_module("facade")' in owner_contract - assert "marker: Annotated[Int32, NativeStorage]" in owner_contract + assert '@bind("facade::run")' in owner_contract + assert "marker: Int32[()]" in owner_contract source = build_fortran_extension( sources, diff --git a/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py b/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py index 2398a87eb..76a294b75 100644 --- a/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py +++ b/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py @@ -42,6 +42,29 @@ def test_convert_pyi_to_ir_accepts_parsed_pyi_ast_only(): convert_pyi_to_ir(source) +def test_module_scalar_storage_and_facade_bind_round_trip(): + """A contract retains native scalar storage and its public Fortran call route.""" + source = """from prik.contracts import Int32, bind + +live: Int32[()] +plain: Int32 + +@bind("facade::work") +def run() -> None: ... +""" + module = parse_pyi_text(source, module_name="owner") + emitted = emit_module(module) + + assert "live: Int32[()]" in emitted + assert "plain: Int32" in emitted + assert '@bind("facade::work")' in emitted + replay = parse_pyi_text(emitted, module_name="owner") + assert replay.variables[0].semantic_type.metadata["native_storage"] is True + assert "native_storage" not in replay.variables[1].semantic_type.metadata + assert replay.functions[0].native_name == "work" + assert replay.functions[0].metadata["native_access_module"] == "facade" + + def test_pyi_parser_reports_unsupported_lines_and_invalid_helpers(): with pytest.raises(ValueError, match=r"Unsupported .pyi node"): parse_pyi_text("bare_name\n", module_name="edited") From ca8fa9d872f9c3889c0d890ebce2082eae983c5b Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 09:18:06 +0100 Subject: [PATCH 03/47] codex: preserve live Fortran module scalar storage Expose mutable fixed module scalars as native rank-zero views and follow scalar allocatable and pointer storage through handles. Accept exact rank-zero NumPy actuals for bare scalar and fixed-character dummies while preserving reference and VALUE transport. Refresh contracts, documentation, and source/pyi runtime evidence, including real Open MPI replay. --- CHANGELOG.md | 7 + docs/developer/packages/codegen/c-binding.md | 8 +- docs/user/guide/allocatables.md | 12 +- docs/user/guide/data-types.md | 12 +- docs/user/guide/pointers.md | 11 +- docs/user/guide/strings.md | 10 +- docs/user/guide/wrapping-modules.md | 8 +- docs/user/reference/pyi-format.md | 30 +- prik/codegen/c/binding.py | 356 +++++++++++++++++- prik/codegen/fortran/bridge.py | 47 ++- prik/planning/entrypoints.py | 7 +- prik/planning/models.py | 5 + prik/planning/planner.py | 4 + prik/policy/completion.py | 2 + prik/policy/construction.py | 60 ++- prik/policy/models.py | 15 + prik/policy/ownership.py | 54 ++- prik/runtime/handles.py | 78 ++++ prik/semantics/fortran2ir.py | 24 +- .../end_to_end/test_scalar_allocatables.py | 14 +- .../test_rank_zero_scalar_storage.py | 19 +- .../end_to_end/test_value_and_bind_c.py | 5 + .../end_to_end/test_generic_interfaces.py | 2 + .../test_scalar_module_variable_lowering.py | 3 +- .../fchar_module_descriptors_f90/__init__.pyi | 3 + .../fchar_module_descriptors_f90.pyi | 23 ++ .../fchar_module_scalars_f90/__init__.pyi | 3 + .../fchar_module_scalars_f90.pyi | 13 + .../fmodule_vars_f90/fmodule_vars_f90.pyi | 6 +- .../modules_direct_bind_c_f90.pyi | 2 +- .../modules_mixed_bind_c_f90.pyi | 2 +- .../test_module_variable_reexport.py | 2 +- .../test_module_variables_and_state.py | 82 ++-- .../test_scalar_module_variable_plan.py | 28 +- .../policy/test_module_variable_policy.py | 4 +- .../end_to_end/test_optional_runtime.py | 1 + .../test_documented_string_journey.py | 8 + 37 files changed, 835 insertions(+), 135 deletions(-) create mode 100644 tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_descriptors_f90/__init__.pyi create mode 100644 tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_descriptors_f90/fchar_module_descriptors_f90.pyi create mode 100644 tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_scalars_f90/__init__.pyi create mode 100644 tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_scalars_f90/fchar_module_scalars_f90.pyi diff --git a/CHANGELOG.md b/CHANGELOG.md index ce836f773..f2d5fb5c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,13 @@ release tags add a leading `v` to the package version. ## Unreleased +- Mutable fixed-storage Fortran module scalars expose native-backed rank-zero + NumPy views, including fixed-length character bytes; primitive and fixed + character value dummies accept matching rank-zero storage as well as scalar + values, preserving the dummy's reference or `VALUE` ABI. +- Scalar allocatable and pointer module variables expose handles that follow + native allocation or association and provide current rank-zero NumPy views. + - Contributor test guidance focuses on supported behavior and meaningful validation boundaries after a feature is removed. diff --git a/docs/developer/packages/codegen/c-binding.md b/docs/developer/packages/codegen/c-binding.md index 86a614fea..4de281ab3 100644 --- a/docs/developer/packages/codegen/c-binding.md +++ b/docs/developer/packages/codegen/c-binding.md @@ -264,7 +264,13 @@ static PyObject * wrap_double_value(PyObject * self, PyObject * args, PyObject * double bound_value; double result; if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O", kwlist, &bound_value_obj)) return NULL; - if (prik_float64_unpack_exact(bound_value_obj, &bound_value) < 0) { if (!PyErr_Occurred()) { PyErr_Format(PyExc_TypeError, "Expected an argument of type numpy.float64 for argument value. Received ", Py_TYPE(bound_value_obj)->tp_name); } return NULL; }; + if (PyArray_Check(bound_value_obj)) { + if (PyArray_TYPE((PyArrayObject *)bound_value_obj) != NPY_FLOAT64 || PyArray_NDIM((PyArrayObject *)bound_value_obj) != 0) { PyErr_SetString(PyExc_TypeError, "Argument value requires exact rank-zero numpy.float64 storage"); return NULL; }; + if (!PyArray_ISNOTSWAPPED((PyArrayObject *)bound_value_obj) || !PyArray_ISALIGNED((PyArrayObject *)bound_value_obj)) { PyErr_SetString(PyExc_TypeError, "Argument value requires native byte order and aligned storage"); return NULL; }; + memcpy(&bound_value, PyArray_DATA((PyArrayObject *)bound_value_obj), sizeof(bound_value)); + } else { + if (prik_float64_unpack_exact(bound_value_obj, &bound_value) < 0) { if (!PyErr_Occurred()) { PyErr_Format(PyExc_TypeError, "Expected an argument of type numpy.float64 or rank-zero array for argument value. Received ", Py_TYPE(bound_value_obj)->tp_name); } return NULL; }; + } result = bind_c_double_value(bound_value); PyObject * result_obj = prik_float64_to_numpy(&result); if (result_obj == NULL) { diff --git a/docs/user/guide/allocatables.md b/docs/user/guide/allocatables.md index 12446f8fc..66bce7372 100644 --- a/docs/user/guide/allocatables.md +++ b/docs/user/guide/allocatables.md @@ -16,8 +16,9 @@ and a PRIK handle gives Python access to that descriptor. ## Key Concepts -- Scalar allocatables appear as `T | None`; array allocatables use - `Allocatable[T[...]]` handles. +- Scalar allocatable dummies and results appear as values or `None`. Scalar + module variables use handles; array allocatables use `Allocatable[T[...]]` + handles. - An array handle exposes allocation state and descriptor operations; it is not itself a NumPy array. - `allocated` reports whether storage exists; `to_numpy()` returns a live view @@ -82,8 +83,11 @@ assert values.allocated is True The annotation supplies the element dtype and rank. The handle creates its native storage when first passed to a matching writable argument. It stays the same Python object after the call. -`Allocatable[Float64]()` is not supported because scalar allocatables cross the -Python boundary as values rather than array handles. +`Allocatable[Float64]()` is not supported. A scalar module variable declared +`Allocatable[Float64]` returns a native scalar handle that follows allocation +changes. Its `allocated` property reports presence, `value` reads the current +NumPy scalar or `None`, and `to_numpy()` borrows a rank-zero view or returns +`None`. Acquire a fresh view after reallocation. A returned or attribute array handle remains present even when its descriptor is unallocated. Reading the Python attribute diff --git a/docs/user/guide/data-types.md b/docs/user/guide/data-types.md index ada7bcaf4..62f0218ad 100644 --- a/docs/user/guide/data-types.md +++ b/docs/user/guide/data-types.md @@ -274,8 +274,9 @@ their own default constructors, described in their later user-guide pages. ## Important Rules -- Use **exact NumPy scalar dtypes** (`np.float64`, `np.int32`, etc.) for - numeric scalar arguments and expect the matching NumPy scalar result. +- Use **exact NumPy scalar dtypes** (`np.float64`, `np.int32`, etc.) or matching + rank-zero NumPy arrays for numeric scalar arguments. Scalar results use the + matching NumPy scalar type. Boolean arguments accept `bool` or `np.bool_`, and Boolean scalar results are Python `bool` values. - Plain Python `float` and `int` values raise `TypeError` for numeric scalar @@ -293,10 +294,13 @@ A bare primitive type represents a Python-visible scalar: def double(value: Float64) -> Float64: ... ``` -The wrapper requires a `numpy.float64` input and returns a `numpy.float64`. +The wrapper accepts a `numpy.float64` scalar or a rank-zero `float64` array +and returns a `numpy.float64`. For a reference dummy, the array supplies its +own storage; a scalar uses call-local storage. A `VALUE` dummy receives a value +from either actual. Other primitive result types follow the mapping table above. -`T[()]` represents rank-zero NumPy storage: arguments accept a 0-D NumPy +`T[()]` represents rank-zero NumPy storage: arguments require a 0-D NumPy array, and results return a 0-D NumPy array. Raw integer addresses are an advanced boundary covered later in the guide. A bare numeric `T` result is the NumPy scalar listed in the mapping table; Boolean scalar results are Python diff --git a/docs/user/guide/pointers.md b/docs/user/guide/pointers.md index 5d573ede4..1a046eecf 100644 --- a/docs/user/guide/pointers.md +++ b/docs/user/guide/pointers.md @@ -18,8 +18,8 @@ shape, and strides. It does not by itself say who owns that target. - A pointer descriptor refers to target storage; it does not own that storage by default. -- Scalar pointers appear as `T | None`; array pointers use live - `Pointer[T[...]]` handles. +- Scalar pointer dummies and results appear as values or `None`. Scalar module + variables use handles; array pointers use live `Pointer[T[...]]` handles. - `associated` describes association, not ownership or target lifetime. - NumPy arrays returned by `to_numpy()` are live views, not copies. - Reassociation, resizing, or deallocation can invalidate existing views. @@ -75,8 +75,11 @@ assert target.associated is True The annotation supplies the element dtype and rank. The handle creates its native storage when first passed to a matching writable argument. It stays the same Python object after the call. -`Pointer[Float64]()` is not supported because scalar pointers cross the Python -boundary as values rather than array handles. +`Pointer[Float64]()` is not supported. A scalar module variable declared +`Pointer[Float64]` returns a native scalar handle that follows association +changes. Its `associated` property reports presence, `value` reads the current +NumPy scalar or `None`, and `to_numpy()` borrows a rank-zero view or returns +`None`. Acquire a fresh view after reassociation. | Member | Type | Behavior | | --- | --- | --- | diff --git a/docs/user/guide/strings.md b/docs/user/guide/strings.md index c78b1104b..b8408cc59 100644 --- a/docs/user/guide/strings.md +++ b/docs/user/guide/strings.md @@ -21,7 +21,7 @@ caller-owned storage. | Contract | Python value | Native mutation | | --- | --- | --- | | `String` | Variable-length `str` | Returned only when projected | -| `String[8]` | `str` encoded as exactly 8 bytes | Returned as a new `str` | +| `String[8]` | `str` or rank-zero NumPy `S8` array | `str` returns a new value; array storage changes in place | | `String[8][()]` | Rank-zero NumPy array with dtype `S8` | Visible in place | | `String[8][count]` | NumPy bytes array with dtype `S8` | Visible in place | | `Addr(String[8])` | Integer address | Visible through caller-owned memory | @@ -162,8 +162,9 @@ For the complete result-mapping rules, see ## Immutable Values -`String[8]` accepts a Python `str` whose encoded length is exactly eight bytes. -The wrapper copies it into native storage. +`String[8]` accepts a Python `str` whose encoded length is exactly eight bytes, +or a rank-zero NumPy array with dtype `S8`. A `str` uses call-local storage; +the array supplies its own bytes to a reference dummy. ```python import sys @@ -180,7 +181,8 @@ print(repr(make_text())) # 'ready ' ``` Python strings are immutable. `Returns[...]` copies the changed native buffer -into a new `str`. Without that projection, the mutation is discarded. +into a new `str`. A rank-zero array passed to `edit_text` also changes in place. +Without a projection, only array mutation remains visible. ## Mutable Scalar Storage diff --git a/docs/user/guide/wrapping-modules.md b/docs/user/guide/wrapping-modules.md index 564420a4d..8fc5bb3c7 100644 --- a/docs/user/guide/wrapping-modules.md +++ b/docs/user/guide/wrapping-modules.md @@ -136,10 +136,10 @@ Assign elements or slices, rather than rebinding the module variable. Fixed-shape module arrays and derived-type array fields expose live views whether or not their declarations include `target`. Logical array dtypes follow the [data-type mapping](data-types.md#scalar-type-mapping). -An addressable scalar module variable declared with `bind(C)` is also a live -rank-zero NumPy view, written `T[()]` in its contract. Use `value[()]` to read -or update it; pass the view itself when a `TYPE(*)` dummy needs its original -native storage. +Mutable fixed-storage scalar module variables expose live rank-zero NumPy +views. Numeric and logical scalars use `T[()]`; fixed-length character scalars +use `String[n][()]` with raw bytes storage. Pass the view itself when a native +dummy needs its original storage. `PARAMETER` declarations remain constants. - Allocatable module arrays use the `Allocatable[T[...]]` API. - Allocation, lifetime, NumPy views, and mutation rules are covered in the storage and objects section. diff --git a/docs/user/reference/pyi-format.md b/docs/user/reference/pyi-format.md index c3dbb0718..fc6d9881f 100644 --- a/docs/user/reference/pyi-format.md +++ b/docs/user/reference/pyi-format.md @@ -399,17 +399,17 @@ An annotated assignment declares native module state: ```python from prik.contracts import Float64, Int32 -counter: Int32 -scale: Float64 = 2.0 +counter: Int32[()] +scale: Float64[()] ``` -Fortran module variables can be buildable getters, setters, constants, wrapped -objects, or descriptor handles according to their completed policy. A literal -default on supported mutable scalar state is an import-time native initializer. -`Int32[()]` describes a numeric scalar module variable whose Python value is a -live rank-zero NumPy view over its native storage. Pass that view to a native dummy -when the variable's original address matters. Plain `Int32` provides a scalar -value getter. +Mutable Fortran module variables expose their native storage. Fixed-storage +numeric and logical scalars use live rank-zero `T[()]` NumPy views; fixed-length +character scalars use live rank-zero `String[n][()]` bytes views. `PARAMETER` +declarations use `Final[...]` values. Allocatable and pointer storage uses a +handle that follows allocation or association changes. An edited plain `T` +module declaration requests a scalar value getter. A literal default on +supported mutable scalar state is an import-time native initializer. C global declarations can be represented for inspection, but current C wrapper builds reject native global state. C functions remain the supported runtime @@ -865,9 +865,15 @@ PRIK does not silently pad or truncate a fixed-length public `str`. ### Python And Native Boundaries `T` describes the Python value; `@native_call` can refine how it reaches the -native procedure. A bare numeric scalar normally passes by value. -`Addr(Arg(i))` creates call-local scalar storage and passes its address. -`T[()]` and ranked arrays already expose storage and use `Arg(i)`. +native procedure. A bare numeric scalar argument accepts either an exact +NumPy scalar or a matching rank-zero NumPy array. For a reference dummy, the +scalar uses call-local storage and the array supplies its own address. For a +`VALUE` dummy, both supply a value. `T[()]` requires rank-zero array storage. +`String[n]` likewise accepts a Python `str` or matching rank-zero `S` +storage; `String[n][()]` requires the latter. The dummy declaration determines +the native ABI independently of the actual's Python representation. In a +`@native_call` map, `Addr(Arg(i))` transports a reference; `Arg(i)` or +`Value(Arg(i))` transports a value as required by the native type. Raw `Addr(T)` is different: the Python caller supplies the integer address itself. Wrapped class annotations pass generated wrapper instances and their diff --git a/prik/codegen/c/binding.py b/prik/codegen/c/binding.py index aee9efa35..da6054751 100644 --- a/prik/codegen/c/binding.py +++ b/prik/codegen/c/binding.py @@ -27,6 +27,7 @@ ) from prik.policy.models import ( ArgumentHandoffMode, + ScalarActualMode, ArrayEntrypointABI, ArrayPythonLayout, CallbackABIKind, @@ -219,6 +220,8 @@ class _COverloadDispatch: ModuleGetterAction.NATIVE_CONSTANT_ARRAY_VALUE: "Copies the parameter array into one read-only NumPy array.", ModuleGetterAction.DIRECT_VALUE: "Builds a Python scalar from the current native value.", ModuleGetterAction.NATIVE_SCALAR_VIEW: "Wraps live native scalar storage in a rank-zero NumPy view.", + ModuleGetterAction.NATIVE_CHARACTER_VIEW: "Wraps live native character bytes in a rank-zero NumPy view.", + ModuleGetterAction.NATIVE_SCALAR_HANDLE: "Returns a handle that queries current native scalar storage.", ModuleGetterAction.CHARACTER_VALUE: "Decodes the fixed-width native characters into a Python str.", ModuleGetterAction.NULLABLE_SNAPSHOT: "Returns a detached copy, or None when the native value holds nothing.", ModuleGetterAction.BORROWED_ARRAY_VIEW: "Wraps the native storage in a live NumPy array without copying.", @@ -4532,6 +4535,8 @@ def _module_array_owner_variables(self, plan: ModulePlan) -> tuple[ModuleVariabl ModuleGetterAction.BORROWED_ARRAY_VIEW, ModuleGetterAction.NATIVE_ARRAY_HANDLE, ModuleGetterAction.NATIVE_SCALAR_VIEW, + ModuleGetterAction.NATIVE_CHARACTER_VIEW, + ModuleGetterAction.NATIVE_SCALAR_HANDLE, } ) @@ -6129,6 +6134,10 @@ def _lower_module_getter(self, plan: ModuleVariablePlan) -> tuple[CFunction, ... return self._lower_module_getter_direct_value(plan) case ModuleGetterAction.NATIVE_SCALAR_VIEW: return self._lower_module_getter_native_scalar_view(plan) + case ModuleGetterAction.NATIVE_CHARACTER_VIEW: + return self._lower_module_getter_native_character_view(plan) + case ModuleGetterAction.NATIVE_SCALAR_HANDLE: + return self._lower_module_getter_native_scalar_handle(plan) case ModuleGetterAction.CHARACTER_VALUE: return self._lower_module_getter_character_value(plan) case ModuleGetterAction.NULLABLE_SNAPSHOT: @@ -6194,6 +6203,72 @@ def _lower_module_getter_native_scalar_view(self, plan: ModuleVariablePlan) -> t ), ) + def _lower_module_getter_native_character_view(self, plan: ModuleVariablePlan) -> tuple[CFunction, ...]: + """Expose fixed character storage as a live rank-zero NumPy bytes view.""" + length = self._module_character_length(plan) + owner = self._module_native_array_owner_name(plan) + return ( + CFunction( + self._module_getter_name(plan), + "PyObject *", + storage="static", + body=( + CDeclaration("data", "void *", CodeExpression(f"{self._module_bridge_getter_name(plan)}()")), + CDeclaration( + "result", + "PyObject *", + CodeExpression( + f"PyArray_New(&PyArray_Type, 0, NULL, NPY_STRING, NULL, data, {length}, " + "NPY_ARRAY_ALIGNED | NPY_ARRAY_WRITEABLE, NULL)" + ), + ), + *self._ordinary_array_field_owner_nodes("result", owner), + ), + ), + ) + + def _lower_module_getter_native_scalar_handle(self, plan: ModuleVariablePlan) -> tuple[CFunction, ...]: + """Create a scalar descriptor handle around one callable native inquiry.""" + owner = self._module_native_array_owner_name(plan) + descriptor = plan.entrypoint.descriptor_kind + if descriptor not in {"allocatable", "pointer"}: + raise ValueError(f"Scalar descriptor {plan.owner_path!r} has no completed descriptor kind") + return ( + CFunction( + self._module_getter_name(plan), + "PyObject *", + storage="static", + body=( + CDeclaration( + "runtime", "PyObject *", CodeExpression('PyImport_ImportModule("prik.runtime.handles")') + ), + CIf(CodeExpression("runtime == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CDeclaration( + "factory", + "PyObject *", + CodeExpression( + 'PyObject_GetAttrString(runtime, "_native_scalar_handle_from_generated_address")' + ), + ), + CExpressionStatement(CodeExpression("Py_DECREF(runtime)")), + CIf(CodeExpression("factory == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CDeclaration( + "result", + "PyObject *", + CodeExpression( + f'PyObject_CallFunction(factory, "KssOi", ' + f"(unsigned long long)(uintptr_t)&{self._module_bridge_getter_name(plan)}, " + f"{self._c_string_literal(plan.semantic_type_name)}, " + f"{self._c_string_literal(descriptor)}, {owner}, " + f"{1 if plan.datatype_family is DatatypeFamily.STRING else 0})" + ), + ), + CExpressionStatement(CodeExpression("Py_DECREF(factory)")), + CReturn(CodeExpression("result")), + ), + ), + ) + def _module_character_length(self, plan: ModuleVariablePlan) -> int: """Return the declared width one character module accessor copies.""" length = plan.character_length @@ -7460,6 +7535,11 @@ def _lower_argument_required_scalar_value( raise ValueError(f"Unsupported scalar input type {plan.semantic_type_name!r}") names = context.arguments[plan.owner_path] storage_type = plan.native_storage_c_type or scalar_type.c_spelling + if plan.binding.scalar_actual_mode in { + ScalarActualMode.NUMERIC_REFERENCE, + ScalarActualMode.NUMERIC_VALUE, + }: + return self._lower_argument_required_scalar_or_storage(plan, names, scalar_type, storage_type) if storage_type != scalar_type.c_spelling: converted_name = f"{names.value_name}_converted" return ( @@ -7495,6 +7575,78 @@ def _lower_argument_required_scalar_value( ), ) + def _lower_argument_required_scalar_or_storage( + self, + plan: ArgumentTransferPlan, + names: _CArgumentNames, + scalar_type, + storage_type: str, + ) -> tuple[CDeclaration | CExpressionStatement | CIf, ...]: + """Take a scalar value or borrow exact rank-zero native storage.""" + array = f"(PyArrayObject *){names.object_name}" + pointer = f"{names.value_name}_storage" + numpy_type, expected = self._numeric_array_dtype_selectors(plan) + reference = plan.binding.scalar_actual_mode is ScalarActualMode.NUMERIC_REFERENCE + array_nodes: list[CExpressionStatement] = [ + CExpressionStatement( + CodeExpression( + f"if (PyArray_TYPE({array}) != {numpy_type} || PyArray_NDIM({array}) != 0) {{ " + f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires exact ' + f'rank-zero {expected} storage"); return NULL; }}' + ) + ), + CExpressionStatement( + CodeExpression( + f"if (!PyArray_ISNOTSWAPPED({array}) || !PyArray_ISALIGNED({array})) {{ " + f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires native ' + 'byte order and aligned storage"); return NULL; }' + ) + ), + ] + if reference and plan.binding.scalar_storage_writable: + array_nodes.append( + CExpressionStatement( + CodeExpression( + f"if (!PyArray_ISWRITEABLE({array})) {{ " + f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires ' + 'writeable storage"); return NULL; }' + ) + ) + ) + array_nodes.append( + CExpressionStatement( + CodeExpression( + f"{pointer} = ({storage_type} *)PyArray_DATA({array})" + if reference + else f"memcpy(&{names.value_name}, PyArray_DATA({array}), sizeof({names.value_name}))" + ) + ) + ) + return ( + CDeclaration(names.object_name, "PyObject *"), + CDeclaration(names.value_name, storage_type), + *((CDeclaration(pointer, f"{storage_type} *", CodeExpression("NULL")),) if reference else ()), + CIf( + CodeExpression(f"PyArray_Check({names.object_name})"), + body=tuple(array_nodes), + else_body=( + self._scalar_exact_unpack_statement( + scalar_type, + names.object_name, + names.value_name, + ( + f'PyErr_Format(PyExc_TypeError, "Expected an argument of type ' + f"{scalar_type.python_type_name} or rank-zero array for argument " + f"{plan.binding.python_name}. Received \", " + f"Py_TYPE({names.object_name})->tp_name)" + ), + "NULL", + ), + *((CExpressionStatement(CodeExpression(f"{pointer} = &{names.value_name}")),) if reference else ()), + ), + ), + ) + # String argument lowering. def _lower_argument_required_string_value( self, @@ -7516,11 +7668,16 @@ def _lower_argument_required_string_input( ) -> tuple[CDeclaration | CExpressionStatement, ...]: """Validate and borrow one read-only UTF-8 payload for the call.""" names = context.arguments[plan.owner_path] + validation = ( + self._required_string_or_storage_nodes(plan, names, names.value_name) + if plan.binding.scalar_actual_mode is not None + else self._required_string_validation_nodes(plan, names, names.value_name) + ) return ( CDeclaration(names.object_name, "PyObject *"), CDeclaration(names.value_name, "const char *", CodeExpression("NULL")), CDeclaration(names.length_name, "Py_ssize_t", CodeExpression("0")), - *self._required_string_validation_nodes(plan, names, names.value_name), + *validation, ) def _lower_argument_required_string_replacement( @@ -7531,12 +7688,72 @@ def _lower_argument_required_string_replacement( """Validate one replacement string before call-local allocation.""" names = context.arguments[plan.owner_path] source_name = f"{names.value_name}_source" + validation = ( + self._required_string_or_storage_nodes(plan, names, source_name) + if plan.binding.scalar_actual_mode is not None + else self._required_string_validation_nodes(plan, names, source_name) + ) return ( CDeclaration(names.object_name, "PyObject *"), CDeclaration(source_name, "const char *", CodeExpression("NULL")), CDeclaration(names.value_name, "char *", CodeExpression("NULL")), CDeclaration(names.length_name, "Py_ssize_t", CodeExpression("0")), - *self._required_string_validation_nodes(plan, names, source_name), + *validation, + ) + + def _required_string_or_storage_nodes( + self, + plan: ArgumentTransferPlan, + names: _CArgumentNames, + payload_name: str, + ) -> tuple[CIf, ...]: + """Accept either encoded text or exact fixed-width rank-zero bytes.""" + length = plan.character_length + if length is None or length <= 0: + raise ValueError(f"Character storage {plan.owner_path!r} needs a fixed width") + array = f"(PyArrayObject *){names.object_name}" + checks = [ + CExpressionStatement( + CodeExpression( + f"if (PyArray_TYPE({array}) != NPY_STRING || PyArray_NDIM({array}) != 0 || " + f"PyArray_ITEMSIZE({array}) != {length}) {{ " + f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires ' + f'rank-zero S{length} storage"); return NULL; }}' + ) + ), + CExpressionStatement( + CodeExpression( + f"if (!PyArray_ISALIGNED({array})) {{ " + f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires ' + 'aligned storage"); return NULL; }' + ) + ), + ] + if ( + plan.binding.scalar_storage_writable + and plan.binding.scalar_actual_mode is ScalarActualMode.CHARACTER_REFERENCE + ): + checks.append( + CExpressionStatement( + CodeExpression( + f"if (!PyArray_ISWRITEABLE({array})) {{ " + f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires ' + 'writeable storage"); return NULL; }' + ) + ) + ) + checks.extend( + ( + CExpressionStatement(CodeExpression(f"{payload_name} = (const char *)PyArray_DATA({array})")), + CExpressionStatement(CodeExpression(f"{names.length_name} = {length}")), + ) + ) + return ( + CIf( + CodeExpression(f"PyArray_Check({names.object_name})"), + body=tuple(checks), + else_body=self._required_string_validation_nodes(plan, names, payload_name), + ), ) def _string_replacement_allocation_nodes( @@ -9585,6 +9802,32 @@ def _lower_argument_nullable_value( ) scalar_type = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) names = context.arguments[plan.owner_path] + if plan.binding.scalar_actual_mode in { + ScalarActualMode.NUMERIC_REFERENCE, + ScalarActualMode.NUMERIC_VALUE, + }: + required = self._lower_argument_required_scalar_or_storage( + plan, + names, + scalar_type, + plan.native_storage_c_type or scalar_type.c_spelling, + ) + declarations = tuple(node for node in required if isinstance(node, CDeclaration)) + body = tuple(node for node in required if not isinstance(node, CDeclaration)) + target = ( + f"{names.value_name}_storage" + if plan.binding.scalar_actual_mode is ScalarActualMode.NUMERIC_REFERENCE + else f"&{names.value_name}" + ) + return ( + CDeclaration(names.object_name, "PyObject *", CodeExpression("Py_None")), + *(node for node in declarations if node.name != names.object_name), + CDeclaration(names.nullable_name, "void *", CodeExpression("NULL")), + CIf( + CodeExpression(f"{names.object_name} != Py_None"), + body=(*body, CExpressionStatement(CodeExpression(f"{names.nullable_name} = {target}"))), + ), + ) return ( CDeclaration(names.object_name, "PyObject *", CodeExpression("Py_None")), CDeclaration(names.value_name, scalar_type.c_spelling), @@ -9654,7 +9897,11 @@ def _lower_argument_nullable_string_value( CDeclaration(names.value_name, "const char *", CodeExpression("NULL")), CIf( CodeExpression(f"{names.object_name} != Py_None"), - body=self._required_string_validation_nodes(plan, names, names.value_name), + body=( + self._required_string_or_storage_nodes(plan, names, names.value_name) + if plan.binding.scalar_actual_mode is not None + else self._required_string_validation_nodes(plan, names, names.value_name) + ), ), ) if action is CodegenAction.COPY_IN_OUT: @@ -9665,7 +9912,11 @@ def _lower_argument_nullable_string_value( CDeclaration(names.value_name, "char *", CodeExpression("NULL")), CIf( CodeExpression(f"{names.object_name} != Py_None"), - body=self._required_string_validation_nodes(plan, names, source_name), + body=( + self._required_string_or_storage_nodes(plan, names, source_name) + if plan.binding.scalar_actual_mode is not None + else self._required_string_validation_nodes(plan, names, source_name) + ), ), ) raise ValueError(f"Unsupported optional C string action for {plan.owner_path!r}: {action!r}") @@ -12811,6 +13062,36 @@ def _mixed_string_writeback_nodes( *self._output_failure_nodes(converted, failure_label), ), ) + if source.binding.scalar_actual_mode is ScalarActualMode.CHARACTER_REFERENCE: + array_conversion = CExpressionStatement( + CodeExpression( + f"{target} = PyUnicode_DecodeUTF8((const char *){names.value_name}, " + f'(Py_ssize_t){names.length_name}, "strict")' + ) + ) + converted_value = CIf( + CodeExpression(f"PyArray_Check({names.object_name})"), + body=(array_conversion,), + else_body=( + conversion, + CExpressionStatement(CodeExpression(f"free({names.value_name})")), + CExpressionStatement(CodeExpression(f"{names.value_name} = NULL")), + ), + ) + if source.binding.optional_mode is OptionalMode.REQUIRED: + return CDeclaration(target, "PyObject *", CodeExpression("NULL")), converted_value, failure + if source.binding.optional_mode is OptionalMode.NULLABLE_VALUE: + return ( + CDeclaration(target, "PyObject *", CodeExpression("NULL")), + CIf( + CodeExpression(f"{names.value_name} == NULL"), + body=( + CExpressionStatement(CodeExpression("Py_INCREF(Py_None)")), + CExpressionStatement(CodeExpression(f"{target} = Py_None")), + ), + else_body=(converted_value, failure), + ), + ) if source.binding.optional_mode is OptionalMode.REQUIRED: return ( CDeclaration(target, "PyObject *", CodeExpression("NULL")), @@ -13374,6 +13655,19 @@ def _scalar_writeback_contract_storage( ) -> tuple[str, tuple[CDeclaration, ...]]: """Convert an exact native scalar local back to public contract storage.""" storage_type = source.native_storage_c_type or scalar_type.c_spelling + if source.binding.scalar_actual_mode is ScalarActualMode.NUMERIC_REFERENCE: + pointer = f"{names.value_name}_storage" + if storage_type == scalar_type.c_spelling: + contract_name = f"{names.value_name}_contract" + return contract_name, ( + CDeclaration(contract_name, scalar_type.c_spelling, CodeExpression(f"*{pointer}")), + ) + contract_name = f"{names.value_name}_contract" + return contract_name, ( + CDeclaration( + contract_name, scalar_type.c_spelling, CodeExpression(f"({scalar_type.c_spelling})*{pointer}") + ), + ) if storage_type == scalar_type.c_spelling: return names.value_name, () contract_name = f"{names.value_name}_contract" @@ -13847,6 +14141,20 @@ def _string_replacement_setup_nodes( f"{names.value_name}_source", cleanup, ) + if argument.binding.scalar_actual_mode is ScalarActualMode.CHARACTER_REFERENCE: + allocation = ( + CIf( + CodeExpression(f"PyArray_Check({names.object_name})"), + body=( + CExpressionStatement( + CodeExpression( + f"{names.value_name} = (char *)PyArray_DATA((PyArrayObject *){names.object_name})" + ) + ), + ), + else_body=allocation, + ), + ) if argument.binding.optional_mode is OptionalMode.REQUIRED: nodes.extend(allocation) continue @@ -13878,7 +14186,13 @@ def _string_replacement_cleanup_nodes( return tuple( CExpressionStatement( CodeExpression( - f"if ({names.value_name} != NULL) {{ free({names.value_name}); {names.value_name} = NULL; }}" + f"if ({names.value_name} != NULL" + + ( + f" && !PyArray_Check({names.object_name})" + if argument.binding.scalar_actual_mode is ScalarActualMode.CHARACTER_REFERENCE + else "" + ) + + f") {{ free({names.value_name}); {names.value_name} = NULL; }}" ) ) for argument in reversed(self._string_replacement_arguments(plan)) @@ -14421,12 +14735,16 @@ def _scalar_entrypoint_argument_values( if passing is EntrypointPassingConvention.C_VALUE: return (names.value_name,) if passing is EntrypointPassingConvention.POINTER_REFERENCE: + if plan.binding.scalar_actual_mode is ScalarActualMode.NUMERIC_REFERENCE: + return (f"{names.value_name}_storage",) return (f"&{names.value_name}",) if passing is not plan.entrypoint.passing: raise ValueError(f"Unsupported projected scalar passing convention {passing.value!r}") if plan.entrypoint.handoff_mode is ArgumentHandoffMode.OPAQUE_ADDRESS: return (names.value_name,) if plan.entrypoint.handoff_mode is ArgumentHandoffMode.TYPED_REFERENCE: + if plan.binding.scalar_actual_mode is ScalarActualMode.NUMERIC_REFERENCE: + return (f"{names.value_name}_storage",) return (f"&{names.value_name}",) return (names.value_name,) @@ -15434,9 +15752,33 @@ def _overload_required_argument_condition( f"&& PyArray_TYPE((PyArrayObject *){value}) == {numpy_type}" ) if match.kind is OverloadMatchKind.STRING: - return f"PyUnicode_Check({value})" + predicate = f"PyUnicode_Check({value})" + if ( + match.scalar_actual_mode + in { + ScalarActualMode.CHARACTER_REFERENCE, + ScalarActualMode.CHARACTER_VALUE, + } + and match.character_length is not None + ): + array = f"(PyArrayObject *){value}" + predicate = ( + f"({predicate} || (PyArray_Check({value}) && PyArray_NDIM({array}) == 0 " + f"&& PyArray_TYPE({array}) == NPY_STRING && PyArray_ITEMSIZE({array}) == {match.character_length}))" + ) + return predicate if match.kind is OverloadMatchKind.NUMPY_SCALAR: predicate = f"PyArray_IsScalar({value}, {self._overload_numpy_scalar_kind(match.semantic_type_name)})" + if match.scalar_actual_mode in { + ScalarActualMode.NUMERIC_REFERENCE, + ScalarActualMode.NUMERIC_VALUE, + }: + array = f"(PyArrayObject *){value}" + numpy_type = PrimitiveScalarTypeRegistry.type_for(match.semantic_type_name).numpy_type_macro + predicate = ( + f"({predicate} || (PyArray_Check({value}) && PyArray_NDIM({array}) == 0 " + f"&& PyArray_TYPE({array}) == {numpy_type}))" + ) if match.builtin_scalar_family is not None: predicate = f"({predicate} || {self._overload_builtin_scalar_condition(match, value)})" return predicate @@ -15968,6 +16310,8 @@ def _module_native_array_owner_nodes( ModuleGetterAction.BORROWED_ARRAY_VIEW, ModuleGetterAction.NATIVE_ARRAY_HANDLE, ModuleGetterAction.NATIVE_SCALAR_VIEW, + ModuleGetterAction.NATIVE_CHARACTER_VIEW, + ModuleGetterAction.NATIVE_SCALAR_HANDLE, }: continue owner = self._module_native_array_owner_name(variable) diff --git a/prik/codegen/fortran/bridge.py b/prik/codegen/fortran/bridge.py index bde2e9708..6928d099e 100644 --- a/prik/codegen/fortran/bridge.py +++ b/prik/codegen/fortran/bridge.py @@ -142,6 +142,7 @@ ), ModuleGetterAction.DIRECT_VALUE: "Returns the variable's current value.", ModuleGetterAction.CHARACTER_VALUE: "Copies the characters into a fixed-width byte buffer.", + ModuleGetterAction.NATIVE_CHARACTER_VIEW: "Returns the address of fixed character storage.", ModuleGetterAction.NULLABLE_SNAPSHOT: ( "Copies the value into C-owned storage, or reports a null pointer when it holds nothing." ), @@ -2694,6 +2695,10 @@ def _lower_module_getter(self, plan: ModuleVariablePlan) -> tuple[FortranFunctio return self._lower_module_getter_direct_value(plan) case ModuleGetterAction.NATIVE_SCALAR_VIEW: return self._lower_module_getter_native_scalar_view(plan) + case ModuleGetterAction.NATIVE_CHARACTER_VIEW: + return self._lower_module_getter_native_scalar_view(plan) + case ModuleGetterAction.NATIVE_SCALAR_HANDLE: + return self._lower_module_getter_native_scalar_handle(plan) case ModuleGetterAction.CHARACTER_VALUE: return self._lower_module_getter_character_value(plan) case ModuleGetterAction.NULLABLE_SNAPSHOT: @@ -3452,6 +3457,41 @@ def _lower_module_getter_native_scalar_view(self, plan: ModuleVariablePlan) -> t ), ) + def _lower_module_getter_native_scalar_handle(self, plan: ModuleVariablePlan) -> tuple[FortranFunction, ...]: + """Query a scalar descriptor's current storage without retaining an old address.""" + native = self._native_variable_name(plan) + present = {"allocatable": "allocated", "pointer": "associated"}.get(plan.entrypoint.descriptor_kind) + if present is None: + raise ValueError(f"Scalar descriptor {plan.owner_path!r} has no descriptor kind") + character = plan.datatype_family is DatatypeFamily.STRING + return ( + FortranFunction( + name=self._module_bridge_getter_name(plan), + parameters=((FortranParameter("length", "integer(c_int64_t)", ("intent(out)",)),) if character else ()), + result_name="result", + result_type="type(c_ptr)", + bind_name=self._module_bridge_getter_name(plan), + body=( + FortranAssignment("result", CodeExpression("c_null_ptr")), + *((FortranAssignment("length", CodeExpression("0_c_int64_t")),) if character else ()), + FortranIf( + CodeExpression(f"{present}({native})"), + body=( + *( + (FortranAssignment("length", CodeExpression(f"len({native}, kind=c_int64_t)")),) + if character + else () + ), + FortranAssignment( + "result", + CodeExpression(f"{_MODULE_SCALAR_CAPTURE_NAME}({native})"), + ), + ), + ), + ), + ), + ) + def _module_character_length(self, plan: ModuleVariablePlan) -> int: """Return the declared width one character module accessor copies.""" length = plan.character_length @@ -3662,7 +3702,12 @@ def _module_array_capture_interfaces(self, plan: ModulePlan) -> tuple[FortranInt procedures = [] if any( variable.storage_address is ModuleStorageAddressMechanism.CAPTURED_ADDRESS - and variable.bridge.native_getter_action is ModuleGetterAction.NATIVE_SCALAR_VIEW + and variable.bridge.native_getter_action + in { + ModuleGetterAction.NATIVE_SCALAR_VIEW, + ModuleGetterAction.NATIVE_CHARACTER_VIEW, + ModuleGetterAction.NATIVE_SCALAR_HANDLE, + } for variable in self._variables(plan) ): procedures.append( diff --git a/prik/planning/entrypoints.py b/prik/planning/entrypoints.py index 3d25df4ca..0690b9377 100644 --- a/prik/planning/entrypoints.py +++ b/prik/planning/entrypoints.py @@ -1125,7 +1125,8 @@ def _primary_module_variable_operations(self, variable): ) result = self._opaque_result() elif ( - variable.bridge.native_getter_action is ModuleGetterAction.NULLABLE_SNAPSHOT + variable.bridge.native_getter_action + in {ModuleGetterAction.NULLABLE_SNAPSHOT, ModuleGetterAction.NATIVE_SCALAR_HANDLE} and variable.datatype_family is DatatypeFamily.STRING ): parameters = (self._int64_parameter("length", reference=True, intent="out"),) @@ -1134,6 +1135,8 @@ def _primary_module_variable_operations(self, variable): ModuleGetterAction.NULLABLE_SNAPSHOT, ModuleGetterAction.DERIVED_OBJECT, ModuleGetterAction.NATIVE_SCALAR_VIEW, + ModuleGetterAction.NATIVE_CHARACTER_VIEW, + ModuleGetterAction.NATIVE_SCALAR_HANDLE, }: parameters = () result = self._opaque_result() @@ -1163,7 +1166,7 @@ def _primary_module_variable_operations(self, variable): ) ) if variable.entrypoint.setter_role is not None: - if variable.bridge.native_getter_action is ModuleGetterAction.CHARACTER_VALUE: + if variable.binding.setter_converts_characters: value = self._value( "value", NativeEntrypointABIValueKind.CHARACTER, diff --git a/prik/planning/models.py b/prik/planning/models.py index f35f92642..511a4fd8c 100644 --- a/prik/planning/models.py +++ b/prik/planning/models.py @@ -33,6 +33,7 @@ from prik.policy.models import ( ArgumentConversionPhase, ArgumentHandoffMode, + ScalarActualMode, ArrayLogicalABI, ArrayEntrypointABI, ArrayPythonLayout, @@ -430,6 +431,8 @@ class OverloadArgumentMatchPlan(StageRecord): semantic_type_name: str rank: int derived_type_identity: tuple[str, str] | None + scalar_actual_mode: ScalarActualMode | None = None + character_length: int | None = None builtin_scalar_family: str | None = None @@ -952,6 +955,8 @@ class BindingArgumentPlan(StageRecord): nullable: bool writable: bool descriptor_boundary: bool + scalar_actual_mode: ScalarActualMode | None + scalar_storage_writable: bool native_array_element_c_type: str | None = None diff --git a/prik/planning/planner.py b/prik/planning/planner.py index c619af766..c8c02d135 100644 --- a/prik/planning/planner.py +++ b/prik/planning/planner.py @@ -1104,6 +1104,8 @@ def _overload_plan( semantic_type_name=argument.semantic_type_name, rank=argument.rank, derived_type_identity=argument.derived_type_identity, + scalar_actual_mode=argument.scalar_actual_mode, + character_length=argument.character_length, builtin_scalar_family=argument.builtin_scalar_family, ) for argument in candidate.arguments @@ -2129,6 +2131,8 @@ def _binding_argument_plan( nullable=policy.nullable, writable=policy.writable, descriptor_boundary=policy.descriptor_boundary, + scalar_actual_mode=policy.scalar_actual_mode, + scalar_storage_writable=policy.scalar_storage_writable, native_array_element_c_type=policy.native_array_element_c_type, ) diff --git a/prik/policy/completion.py b/prik/policy/completion.py index d0756441e..2b85bd301 100644 --- a/prik/policy/completion.py +++ b/prik/policy/completion.py @@ -992,6 +992,8 @@ def _overload_argument_match( semantic_type_name=argument.semantic_type_name, rank=argument.rank, derived_type_identity=derived_identity, + scalar_actual_mode=argument.scalar_actual_mode, + character_length=argument.character_length, builtin_scalar_family=_accepted_builtin_scalar_family( argument.semantic_type_name, match_kind=match_kind, diff --git a/prik/policy/construction.py b/prik/policy/construction.py index 78c26071f..cf64a6706 100644 --- a/prik/policy/construction.py +++ b/prik/policy/construction.py @@ -74,6 +74,7 @@ EntrypointProjectionAction, OptionalMode, ArgumentHandoffMode, + ScalarActualMode, ArgumentConversionPhase, BridgeDataAction, DirectResultABI, @@ -1273,7 +1274,12 @@ def _scalar_module_variable_policy( blockers=tuple(blockers), storage_address=( ModuleStorageAddressMechanism.CAPTURED_ADDRESS - if getter_action is ModuleGetterAction.NATIVE_SCALAR_VIEW + if getter_action + in { + ModuleGetterAction.NATIVE_SCALAR_VIEW, + ModuleGetterAction.NATIVE_CHARACTER_VIEW, + ModuleGetterAction.NATIVE_SCALAR_HANDLE, + } else None ), ) @@ -3204,6 +3210,8 @@ def _argument_policy( nullable=boundary.nullable, writable=boundary.writable, descriptor_boundary=boundary.descriptor_boundary, + scalar_actual_mode=_scalar_actual_mode(argument, boundary, entrypoint_passing), + scalar_storage_writable=decision.mutates_native, ownership=decision, codegen_action=boundary.codegen_action, python_barrier_action=boundary.python_barrier_action, @@ -3260,6 +3268,34 @@ def _argument_policy( ) +def _scalar_actual_mode( + argument: models.SemanticArgument, + boundary: _ArgumentBoundaryPolicy, + passing: EntrypointPassingConvention, +) -> ScalarActualMode | None: + """Complete dual scalar/value-or-storage input acceptance before planning.""" + if int(argument.semantic_type.rank or 0) != 0: + return None + if boundary.descriptor_boundary: + return None + if boundary.python_barrier_action is PythonBarrierAction.SCALAR_VALUE: + if passing is EntrypointPassingConvention.C_VALUE: + return ScalarActualMode.NUMERIC_VALUE + if passing is EntrypointPassingConvention.POINTER_REFERENCE: + return ScalarActualMode.NUMERIC_REFERENCE + if passing is EntrypointPassingConvention.NULLABLE_POINTER: + return ( + ScalarActualMode.NUMERIC_VALUE + if argument.origin.metadata.get("value") + else ScalarActualMode.NUMERIC_REFERENCE + ) + if boundary.python_barrier_action is PythonBarrierAction.STRING_VALUE and _character_length(argument.semantic_type): + if _native_by_value_argument(argument): + return ScalarActualMode.CHARACTER_VALUE + return ScalarActualMode.CHARACTER_REFERENCE + return None + + def _callback_handoff_policy(argument: models.SemanticArgument) -> CallbackHandoffPolicy | None: """Read the completed callback policy without interpreting callback syntax.""" policy = argument.semantic_type.metadata.get(models.RESOLVED_CALLBACK_POLICY_METADATA) @@ -6972,20 +7008,30 @@ def _scalar_module_getter_blockers( """Validate one completed scalar or literal-string getter.""" blockers = [] literal_string = _is_binding_literal_string(variable, getter_action) - character_value = getter_action is ModuleGetterAction.CHARACTER_VALUE + character_value = getter_action in {ModuleGetterAction.CHARACTER_VALUE, ModuleGetterAction.NATIVE_CHARACTER_VIEW} # A descriptor character module variable reaches Python through the same # nullable snapshot a descriptor scalar uses, carrying a runtime width. character_snapshot = ( - getter_action is ModuleGetterAction.NULLABLE_SNAPSHOT and variable.semantic_type.name == "String" + getter_action in {ModuleGetterAction.NULLABLE_SNAPSHOT, ModuleGetterAction.NATIVE_SCALAR_HANDLE} + and variable.semantic_type.name == "String" ) - string_getter = literal_string or character_value + string_getter = literal_string or character_value or character_snapshot if not (_is_first_lane_scalar_type(variable.semantic_type) or string_getter or character_snapshot): blockers.append("module variable is not a primitive rank-zero scalar") if character_value and _character_length(variable.semantic_type) is None: blockers.append("character module variable requires one declared length") expected_getter_kind = ObjectKind.STRING if string_getter else ObjectKind.SCALAR supported_getter_actions = ( - {CodegenAction.COPY_OUT} if string_getter else {CodegenAction.DIRECT_VALUE, CodegenAction.SNAPSHOT_COPY} + {CodegenAction.BORROWED_VIEW} + if getter_action + in { + ModuleGetterAction.NATIVE_SCALAR_VIEW, + ModuleGetterAction.NATIVE_CHARACTER_VIEW, + ModuleGetterAction.NATIVE_SCALAR_HANDLE, + } + else {CodegenAction.COPY_OUT} + if string_getter + else {CodegenAction.DIRECT_VALUE, CodegenAction.SNAPSHOT_COPY, CodegenAction.BORROWED_VIEW} ) if getter is None: blockers.append("module variable is missing completed getter policy") @@ -7087,8 +7133,12 @@ def _scalar_module_getter_action( if _source_parameter_needs_native_getter(variable): return ModuleGetterAction.NATIVE_CONSTANT_VALUE return ModuleGetterAction.CONSTANT_VALUE + if _scalar_module_descriptor_kind(variable) is not None: + return ModuleGetterAction.NATIVE_SCALAR_HANDLE if getter is not None and getter.codegen_action is CodegenAction.SNAPSHOT_COPY and getter.nullable: return ModuleGetterAction.NULLABLE_SNAPSHOT + if variable.semantic_type.metadata.get("native_storage") and _is_fixed_length_character_scalar(variable): + return ModuleGetterAction.NATIVE_CHARACTER_VIEW if _is_fixed_length_character_scalar(variable): # A character value cannot cross the C ABI by value, so it copies # through a fixed-width byte buffer the way a character field does. diff --git a/prik/policy/models.py b/prik/policy/models.py index 23e457f02..066347250 100644 --- a/prik/policy/models.py +++ b/prik/policy/models.py @@ -183,6 +183,15 @@ class ArgumentHandoffMode(str, Enum): NATIVE_DESCRIPTOR = "native_descriptor" +class ScalarActualMode(str, Enum): + """Accepted scalar actuals and their completed native transport.""" + + NUMERIC_REFERENCE = "numeric_reference" + NUMERIC_VALUE = "numeric_value" + CHARACTER_REFERENCE = "character_reference" + CHARACTER_VALUE = "character_value" + + class ArgumentConversionPhase(str, Enum): """Completed binding conversion schedule for one Python argument.""" @@ -330,6 +339,8 @@ class ModuleGetterAction(str, Enum): NATIVE_CONSTANT_ARRAY_VALUE = "native_constant_array_value" DIRECT_VALUE = "direct_value" NATIVE_SCALAR_VIEW = "native_scalar_view" + NATIVE_CHARACTER_VIEW = "native_character_view" + NATIVE_SCALAR_HANDLE = "native_scalar_handle" CHARACTER_VALUE = "character_value" NULLABLE_SNAPSHOT = "nullable_snapshot" BORROWED_ARRAY_VIEW = "borrowed_array_view" @@ -714,6 +725,8 @@ class OverloadArgumentPolicy: semantic_type_name: str rank: int derived_type_identity: tuple[str, str] | None + scalar_actual_mode: ScalarActualMode | None = None + character_length: int | None = None builtin_scalar_family: str | None = None @@ -1306,6 +1319,8 @@ class ArgumentPolicy: nullable: bool writable: bool descriptor_boundary: bool + scalar_actual_mode: ScalarActualMode | None + scalar_storage_writable: bool ownership: OwnershipDecision codegen_action: CodegenAction python_barrier_action: PythonBarrierAction diff --git a/prik/policy/ownership.py b/prik/policy/ownership.py index 6d8677740..7ee97db02 100644 --- a/prik/policy/ownership.py +++ b/prik/policy/ownership.py @@ -968,6 +968,13 @@ def decide_semantic_getter( storage = self.decide_semantic_variable(variable, context) if storage.is_blocked or storage.kind in {ObjectKind.NUMPY_ARRAY, ObjectKind.DERIVED_TYPE}: return storage + if context.is_module_variable and variable.semantic_type.metadata.get("native_storage"): + return storage + if context.is_module_variable and ( + variable.semantic_type.metadata.get("fortran_allocatable") + or variable.semantic_type.metadata.get("fortran_pointer") + ): + return storage if storage.kind is ObjectKind.SCALAR and storage.transfer is TransferMode.SNAPSHOT_COPY: return storage return self.decide_semantic_type(variable.semantic_type, OwnershipContext.result()) @@ -1000,7 +1007,11 @@ def decide_semantic_setter( return replace( incoming, assignment_mode=( - AssignmentMode.ALIAS if storage.storage_mode is StorageMode.ALIAS else AssignmentMode.VALUE_COPY + AssignmentMode.VALUE_COPY + if context.is_module_variable and variable.semantic_type.metadata.get("native_storage") + else AssignmentMode.ALIAS + if storage.storage_mode is StorageMode.ALIAS + else AssignmentMode.VALUE_COPY ), setter_action=self._setter_action(storage, incoming, context, variable), ) @@ -1019,9 +1030,23 @@ def _setter_action( ownership contract. """ if storage.kind is ObjectKind.SCALAR: + if context.is_module_variable and ( + variable.semantic_type.metadata.get("fortran_allocatable") + or variable.semantic_type.metadata.get("fortran_pointer") + ): + return SetterAction.REJECT_REPLACEMENT if storage.transfer is TransferMode.SNAPSHOT_COPY and storage.nullable: return SetterAction.REJECT_REPLACEMENT return SetterAction.WRITE_THROUGH + if ( + storage.kind is ObjectKind.STRING + and context.is_module_variable + and ( + variable.semantic_type.metadata.get("fortran_allocatable") + or variable.semantic_type.metadata.get("fortran_pointer") + ) + ): + return SetterAction.REJECT_REPLACEMENT if storage.kind is ObjectKind.STRING and context.is_field: return SetterAction.WRITE_THROUGH # A character module variable is written through the same fixed-width @@ -1857,10 +1882,18 @@ def _module_variable_decision(self, facts: _StorageFacts, context: OwnershipCont else "plain derived module storage uses live typed module access" ), ) - if facts.allocatable and facts.rank == 0: - return self._allocatable_scalar_decision(facts, context) - if facts.pointer and facts.rank == 0: - return self._pointer_scalar_decision(facts, context) + if (facts.allocatable or facts.pointer) and facts.rank == 0: + return OwnershipDecision( + self._kind(facts, OwnershipContext()), + OwnershipOwner.NATIVE, + TransferMode.BORROWED_VIEW, + DestructionPolicy.NATIVE_OWNER, + storage_mode=StorageMode.ALIAS, + boundary_storage_mode=StorageMode.ALIAS, + nullable=True, + borrowed=True, + reason="scalar module descriptor is queried through a persistent native handle", + ) if facts.rank > 0 or facts.is_ndarray: if facts.pointer: return self._pointer_array_decision(facts, context) @@ -1871,7 +1904,11 @@ def _module_variable_decision(self, facts: _StorageFacts, context: OwnershipCont OwnershipOwner.NATIVE, TransferMode.BORROWED_VIEW, DestructionPolicy.NATIVE_OWNER, - storage_mode=StorageMode.ALIAS if facts.rank > 0 else StorageMode.STACK, + storage_mode=( + StorageMode.ALIAS + if facts.rank > 0 or (facts.metadata or {}).get("native_storage") + else StorageMode.STACK + ), borrowed=True, reason="module variable storage is owned by native module state", ) @@ -2223,8 +2260,9 @@ def _pointer_container_blocker( "use PointerPolicy for extraction and descriptor operations" ) return None - if decision.transfer is not TransferMode.SNAPSHOT_COPY: - return "scalar pointer field and module accessors require snapshot_copy detached values" + required_transfer = TransferMode.BORROWED_VIEW if context.is_module_variable else TransferMode.SNAPSHOT_COPY + if decision.transfer is not required_transfer: + return f"scalar pointer {context.location} accessor requires {required_transfer.value} transfer" return None @staticmethod diff --git a/prik/runtime/handles.py b/prik/runtime/handles.py index 331de231d..23b8d6c53 100644 --- a/prik/runtime/handles.py +++ b/prik/runtime/handles.py @@ -41,6 +41,84 @@ def _retain_numpy_owner(value: np.ndarray, owner: Any) -> np.ndarray: return retained +class NativeScalarHandle: + """Follow the current storage of a native scalar allocatable or pointer.""" + + def __init__( + self, + address: int, + semantic_type_name: str, + descriptor_kind: str, + owner: Any, + character: bool, + ) -> None: + if descriptor_kind not in {"allocatable", "pointer"}: + raise ValueError("scalar handle descriptor kind must be allocatable or pointer") + self._owner = owner + self._descriptor_kind = descriptor_kind + self._character = character + self._dtype = ( + None + if character + else np.dtype("bool" if semantic_type_name in {"Bool", "Bool8"} else semantic_type_name.lower()) + ) + signature = ( + ctypes.CFUNCTYPE(ctypes.c_void_p, ctypes.POINTER(ctypes.c_int64)) + if character + else ctypes.CFUNCTYPE(ctypes.c_void_p) + ) + self._query = signature(address) + + def _current(self) -> tuple[int, np.dtype[Any] | None]: + if self._character: + width = ctypes.c_int64(0) + address = self._query(ctypes.byref(width)) + dtype = np.dtype(f"S{width.value}") if address and width.value >= 0 else None + else: + address = self._query() + dtype = self._dtype if address else None + return int(address or 0), dtype + + @property + def allocated(self) -> bool: + """Report whether the allocatable currently has storage.""" + if self._descriptor_kind != "allocatable": + raise AttributeError("allocated is only available for allocatable handles") + return bool(self._current()[0]) + + @property + def associated(self) -> bool: + """Report whether the pointer currently has a target.""" + if self._descriptor_kind != "pointer": + raise AttributeError("associated is only available for pointer handles") + return bool(self._current()[0]) + + def to_numpy(self) -> np.ndarray | None: + """Borrow a rank-zero view of the current native storage, if present.""" + address, dtype = self._current() + if dtype is None: + return None + buffer = (ctypes.c_char * max(dtype.itemsize, 1)).from_address(address) + return _retain_numpy_owner(np.ndarray((), dtype=dtype, buffer=buffer), self) + + @property + def value(self) -> np.generic | None: + """Read the scalar's current value without retaining an older address.""" + view = self.to_numpy() + return None if view is None else view[()] + + +def _native_scalar_handle_from_generated_address( + address: int, + semantic_type_name: str, + descriptor_kind: str, + owner: Any, + character: int, +) -> NativeScalarHandle: + """Bind a generated native inquiry while retaining its extension owner.""" + return NativeScalarHandle(address, semantic_type_name, descriptor_kind, owner, bool(character)) + + def _descriptor_facts(value: Any, rank: int) -> tuple[int, ...]: """Validate one flat descriptor-fact tuple reported for a handle.""" expected = _DESCRIPTOR_FACT_HEADER + _DESCRIPTOR_FACTS_PER_AXIS * int(rank) diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index 3880a71bc..38afe3dc3 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -506,16 +506,6 @@ def _visit_FortranVariable( declaration_arrays=declaration_arrays, ) - @staticmethod - def _has_native_scalar_storage(var: FortranVariable) -> bool: - """Identify concrete interoperable module storage with a stable address.""" - return ( - getattr(var, "_fortran_bind_c", False) - and var.rank == 0 - and not var.is_parameter - and var.base_type.casefold() in {"integer", "real", "complex", "logical"} - ) - def _convert_variable_type( self, var: FortranVariable, @@ -551,8 +541,6 @@ def _convert_variable_type( if getattr(var, "target", False): metadata["aliased"] = True metadata["fortran_target"] = True - if self._has_native_scalar_storage(var): - metadata["native_storage"] = True if getattr(var, "_fortran_protected", False): metadata["fortran_protected"] = True if getattr(var, "pointer", False): @@ -751,6 +739,18 @@ def _convert_data_member( derived_type_context=derived_type_context, declaration_arrays=declaration_arrays, ) + if ( + source_kind == "variable" + and var.rank == 0 + and not var.is_parameter + and not getattr(var, "allocatable", False) + and not getattr(var, "pointer", False) + and ( + var.base_type.casefold() in {"integer", "real", "complex", "logical"} + or (semantic_type.name == "String" and self._character_length(var).isdigit()) + ) + ): + semantic_type.metadata["native_storage"] = True if semantic_type.storage is not None and semantic_type.storage.array is not None: semantic_type.storage.array.allocatable = getattr(var, "allocatable", False) semantic_type.storage.array.pointer = getattr(var, "pointer", False) diff --git a/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py b/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py index 6b797c874..d86454367 100644 --- a/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py +++ b/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py @@ -30,18 +30,20 @@ def test_scalar_allocatables_project_values_and_unallocated_state( ) module.clear_module_value() - assert module.optional_scale is None + handle = module.optional_scale + assert not handle.allocated and handle.to_numpy() is None assert not hasattr(module, "get_optional_scale") assert not hasattr(module, "set_optional_scale") with pytest.raises(AttributeError): module.optional_scale = np.float64(9.0) module.set_module_value(np.float64(1.5)) - snapshot = module.optional_scale - assert snapshot == np.float64(1.5) + assert handle.allocated and handle.value == np.float64(1.5) + view = handle.to_numpy() + assert view is not None and view.shape == () and view.dtype == np.dtype("float64") module.bump_module_value() - assert snapshot == np.float64(1.5) - assert module.optional_scale == np.float64(11.5) + assert view[()] == np.float64(11.5) + assert handle.value == np.float64(11.5) assert module.echo_allocatable(np.float64(3.0)) == np.float64(4.0) assert module.echo_allocatable(None) == np.float64(-1.0) @@ -53,4 +55,4 @@ def test_scalar_allocatables_project_values_and_unallocated_state( assert module.maybe_allocatable(np.int32(0)) is None module.clear_module_value() - assert module.optional_scale is None + assert not handle.allocated and handle.to_numpy() is None diff --git a/tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py b/tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py index 6a53b0c37..1cbb1d358 100644 --- a/tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py +++ b/tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py @@ -21,7 +21,7 @@ def test_scalar_values_and_rank_zero_storage_cross_the_native_boundary(tmp_path: contract_text=""" from prik.contracts import Annotated, Final, Immutable, Int32, Float64, Return, Returns, native_call -counter: Int32 +counter: Int32[()] answer: Final[Int32] = 42 def value_input(value: Int32) -> Int32: ... @@ -47,14 +47,27 @@ def hidden_storage_result() -> Int32[()]: ... ) assert module.value_input(np.int32(5)) == np.int32(7) - assert module.counter == np.int32(3) + assert module.value_input(np.array(5, dtype=np.int32)) == np.int32(7) + with pytest.raises(TypeError): + module.value_input(np.array(5, dtype=np.int64)) + native_counter = module.counter + assert native_counter.shape == () + assert native_counter[()] == np.int32(3) + assert module.bump_value(native_counter) == np.int32(4) + assert module.counter[()] == np.int32(4) module.counter = np.int32(9) - assert module.counter == np.int32(9) + assert native_counter[()] == np.int32(9) assert module.answer == np.int32(42) original = np.int32(4) assert module.bump_value(original) == np.int32(5) assert original == np.int32(4) + borrowed = np.array(4, dtype=np.int32) + assert module.bump_value(borrowed) == np.int32(5) + assert borrowed[()] == np.int32(5) + borrowed.flags.writeable = False + with pytest.raises(TypeError, match="writeable"): + module.bump_value(borrowed) storage = np.array(6, dtype=np.int32) assert module.bump_storage(storage) is None diff --git a/tests/fortran/data_types/end_to_end/test_value_and_bind_c.py b/tests/fortran/data_types/end_to_end/test_value_and_bind_c.py index b280249c5..d6ddc2618 100644 --- a/tests/fortran/data_types/end_to_end/test_value_and_bind_c.py +++ b/tests/fortran/data_types/end_to_end/test_value_and_bind_c.py @@ -31,12 +31,17 @@ def test_value_and_existing_bind_c_renamed_symbol_use_correct_abi( ) assert module.plus_value(np.int32(5)) == np.int32(12) + value_storage = np.array(5, dtype=np.int32) + assert module.plus_value(value_storage) == np.int32(12) + assert value_storage[()] == np.int32(5) assert module.double_value(np.int32(6)) == np.int32(12) assert module.plus_reference(np.int32(5)) == np.int32(16) + assert module.plus_reference(np.array(5, dtype=np.int32)) == np.int32(16) assert module.scale_real(np.float64(4.0)) == np.float64(10.0) assert module.conjugate_value(np.complex128(2.0 + 3.0j)) == np.complex128(2.0 - 3.0j) assert bool(module.invert_flag(True)) is False assert module.char_code("A") == np.int32(65) + assert module.char_code(np.array("A", dtype="S1")) == np.int32(65) if pyi_parity_build_mode == "source": binding_source = (tmp_path / "source_build" / "fbind_value_f90_wrapper.c").read_text(encoding="utf-8") diff --git a/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py b/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py index 491746be8..e74fbb36f 100644 --- a/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py +++ b/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py @@ -58,7 +58,9 @@ def test_fortran_generic_interfaces_dispatch_in_generated_c_extension( assert "convert_complex" not in module.convert.__doc__ assert module.convert(np.int32(4)) == np.int32(14) + assert module.convert(np.array(4, dtype=np.int32)) == np.int32(14) assert module.convert(np.float64(4.0)) == np.float64(4.5) + assert module.convert(np.array(4.0, dtype=np.float64)) == np.float64(4.5) assert module.convert(value=np.int32(5)) == np.int32(15) assert module.convert(np.complex128(2.0 + 3.0j)) == np.complex128(3.0 + 2.0j) assert module.summarize(np.float64(2.5)) == np.float64(2.5) diff --git a/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py b/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py index 3445fe321..56a2315da 100644 --- a/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py +++ b/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py @@ -107,7 +107,7 @@ def test_module_variable_plan_contains_only_completed_dispatch_facts(): assert variables["counter"].bridge.native_assignment is AssignmentMode.VALUE_COPY assert variables["counter"].binding.initializer == 3 assert variables["target_scale"].bridge.native_assignment is AssignmentMode.VALUE_COPY - assert variables["optional_scale"].binding.getter_action is ModuleGetterAction.NULLABLE_SNAPSHOT + assert variables["optional_scale"].binding.getter_action is ModuleGetterAction.NATIVE_SCALAR_HANDLE assert variables["optional_scale"].entrypoint.descriptor_kind == "allocatable" assert variables["optional_scale"].binding.setter_action is SetterAction.REJECT_REPLACEMENT assert variables["optional_scale"].bridge.native_assignment is AssignmentMode.NONE @@ -260,7 +260,6 @@ def test_module_variable_generators_dispatch_get_set_and_rejection_from_plan(): assert "bind_c_set_counter(value)" in c_source assert "module variable optional_scale is read-only" in c_source assert "module variable selected_scale is read-only" in c_source - assert 'getenv("PRIK_WRAPPER_FAIL_ALLOC")' in c_source assert "result = native_counter" in fortran_source assert "native_counter = value" in fortran_source assert "allocated(native_optional_scale)" in fortran_source diff --git a/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_descriptors_f90/__init__.pyi b/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_descriptors_f90/__init__.pyi new file mode 100644 index 000000000..66c90d4d9 --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_descriptors_f90/__init__.pyi @@ -0,0 +1,3 @@ +from . import fchar_module_descriptors_f90 + +__all__ = ["fchar_module_descriptors_f90"] diff --git a/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_descriptors_f90/fchar_module_descriptors_f90.pyi b/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_descriptors_f90/fchar_module_descriptors_f90.pyi new file mode 100644 index 000000000..17bcd2644 --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_descriptors_f90/fchar_module_descriptors_f90.pyi @@ -0,0 +1,23 @@ +from prik.contracts import Aliased, Allocatable, Annotated, Final, Pointer, String + +deferred: Allocatable[String[:]] + +fixed: Allocatable[String[6]] + +link: Pointer[String[:]] + +store: Annotated[String[6][()], Aliased] + +pair: Final[String[2][2]] = ['ab', 'cd'] + +grid: Final[String[3][2, 2]] + +inferred: Final[String[...][3]] = ['alpha', 'beta ', 'gamma'] + +def setup() -> None: ... + +def grow() -> None: ... + +def clear() -> None: ... + +__all__ = ["deferred", "fixed", "link", "store", "pair", "grid", "inferred", "setup", "grow", "clear"] diff --git a/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_scalars_f90/__init__.pyi b/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_scalars_f90/__init__.pyi new file mode 100644 index 000000000..993919817 --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_scalars_f90/__init__.pyi @@ -0,0 +1,3 @@ +from . import fchar_module_scalars_f90 + +__all__ = ["fchar_module_scalars_f90"] diff --git a/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_scalars_f90/fchar_module_scalars_f90.pyi b/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_scalars_f90/fchar_module_scalars_f90.pyi new file mode 100644 index 000000000..7113fd48e --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/contracts/fchar_module_scalars_f90/fchar_module_scalars_f90.pyi @@ -0,0 +1,13 @@ +from prik.contracts import Final, String + +label: String[8][()] + +code: String[3][()] + +tag: Final[String] = 'fixed' + +def relabel() -> None: ... + +def read_label() -> String[8]: ... + +__all__ = ["label", "code", "tag", "relabel", "read_label"] diff --git a/tests/fortran/modules/end_to_end/fixtures/contracts/fmodule_vars_f90/fmodule_vars_f90.pyi b/tests/fortran/modules/end_to_end/fixtures/contracts/fmodule_vars_f90/fmodule_vars_f90.pyi index ecb394757..1bcc304a5 100644 --- a/tests/fortran/modules/end_to_end/fixtures/contracts/fmodule_vars_f90/fmodule_vars_f90.pyi +++ b/tests/fortran/modules/end_to_end/fixtures/contracts/fmodule_vars_f90/fmodule_vars_f90.pyi @@ -17,11 +17,11 @@ nmax: Final[Int32] = 12 black: Final[Rgb_Color] -counter: Int32 +counter: Int32[()] -scale: Float64 +scale: Float64[()] -saved_counter: Int32 +saved_counter: Int32[()] def summarize() -> Int32: ... diff --git a/tests/fortran/modules/end_to_end/fixtures/routing/contracts/modules_direct_bind_c_f90/modules_direct_bind_c_f90.pyi b/tests/fortran/modules/end_to_end/fixtures/routing/contracts/modules_direct_bind_c_f90/modules_direct_bind_c_f90.pyi index 9f76b9750..c3efe8353 100644 --- a/tests/fortran/modules/end_to_end/fixtures/routing/contracts/modules_direct_bind_c_f90/modules_direct_bind_c_f90.pyi +++ b/tests/fortran/modules/end_to_end/fixtures/routing/contracts/modules_direct_bind_c_f90/modules_direct_bind_c_f90.pyi @@ -2,7 +2,7 @@ from prik.contracts import Final, Int32, native_abi limit: Final[Int32] = 12 -counter: Int32 +counter: Int32[()] @native_abi("c") def direct_total( diff --git a/tests/fortran/modules/end_to_end/fixtures/routing/contracts/modules_mixed_bind_c_f90/modules_mixed_bind_c_f90.pyi b/tests/fortran/modules/end_to_end/fixtures/routing/contracts/modules_mixed_bind_c_f90/modules_mixed_bind_c_f90.pyi index 785699d9c..d3b7171d7 100644 --- a/tests/fortran/modules/end_to_end/fixtures/routing/contracts/modules_mixed_bind_c_f90/modules_mixed_bind_c_f90.pyi +++ b/tests/fortran/modules/end_to_end/fixtures/routing/contracts/modules_mixed_bind_c_f90/modules_mixed_bind_c_f90.pyi @@ -1,6 +1,6 @@ from prik.contracts import Addr, Arg, Int32, native_abi, native_call -counter: Int32 +counter: Int32[()] @native_abi("c") def direct_total( diff --git a/tests/fortran/modules/end_to_end/test_module_variable_reexport.py b/tests/fortran/modules/end_to_end/test_module_variable_reexport.py index 594e07ce4..cf33c5380 100644 --- a/tests/fortran/modules/end_to_end/test_module_variable_reexport.py +++ b/tests/fortran/modules/end_to_end/test_module_variable_reexport.py @@ -61,7 +61,7 @@ def test_a_multi_hop_publication_resolves_to_the_declaring_variable(built): def test_a_character_scalar_and_array_publish_one_storage(built): """String storage is shared the same way a scalar is.""" built.store_mod.label = "second " - assert built.facade_mod.label == "second " + assert built.facade_mod.label[()] == b"second " built.store_mod.tags[0] = b"zz " assert bytes(built.facade_mod.tags[0]) == b"zz " diff --git a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py index 1914d37dc..c39a1ac99 100644 --- a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py +++ b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py @@ -229,39 +229,40 @@ def test_fixed_shape_character_module_arrays_expose_one_live_bytes_view(tmp_path CHARACTER_MODULE_SCALAR_SOURCE = (NATIVE_FIXTURES / "fchar_module_scalars_f90.f90").read_text(encoding="utf-8") -def test_scalar_character_module_variables_read_and_write_through(tmp_path: Path): - """A character module variable is a `str` property, as a numeric one is a value. - - A character value has no by-value C ABI, so the accessors copy through a - fixed-width buffer; what has to hold is that the copy runs in both - directions and that a wrong width is refused rather than truncated. - """ - module = _build_text_and_import( - CHARACTER_MODULE_SCALAR_SOURCE, - "fchar_module_scalars_f90.f90", +def test_scalar_character_module_variables_read_and_write_through(pyi_parity_build_mode: str, tmp_path: Path): + """Fixed character storage keeps one native address across reads and writes.""" + module = _build_source_or_generated_pyi_and_import( + NATIVE_FIXTURES / "fchar_module_scalars_f90.f90", tmp_path, { "bind_c_fchar_module_scalars_f90_wrapper.f90", "fchar_module_scalars_f90_wrapper.c", "fchar_module_scalars_f90_wrapper.h", }, + CONTRACT_FIXTURES / "fchar_module_scalars_f90", + pyi_parity_build_mode, ) - assert module.label == "alpha " - assert module.code == "abc" + label = module.label + assert label.shape == () and label.dtype == np.dtype("S8") + assert label[()] == b"alpha " + assert module.code[()] == b"abc" assert module.tag == "fixed" # A native write is observed by the next read, not cached from import. module.relabel() - assert module.label == "ALPHA!!!" + assert label[()] == b"ALPHA!!!" # A Python write reaches the storage Fortran reads. module.label = "PYTHON!!" + assert label[()] == b"PYTHON!!" assert module.read_label() == "PYTHON!!" + label[()] = b"VIEW!!!!" + assert module.read_label() == "VIEW!!!!" # The declared length is a byte width, so a multi-byte encoding still fits exactly. module.label = "café!!!" - assert module.label == "café!!!" + assert label[()] == "café!!!".encode() assert module.read_label() == "café!!!" @@ -281,7 +282,7 @@ def test_scalar_character_module_variable_rejects_a_wrong_encoded_width(value: s with pytest.raises(TypeError, match="exactly 3 bytes"): module.code = value - assert module.code == "abc" + assert module.code[()] == b"abc" CHARACTER_MODULE_DESCRIPTOR_SOURCE = (NATIVE_FIXTURES / "fchar_module_descriptors_f90.f90").read_text(encoding="utf-8") @@ -300,38 +301,51 @@ def _character_descriptor_module(tmp_path: Path): ) -def test_descriptor_character_module_variables_snapshot_their_runtime_value(tmp_path: Path): - """An allocatable or pointer character module variable reads as a detached `str`. - - Its width is established at runtime, so the snapshot has to report the - length the descriptor currently holds rather than a width fixed at build - time, and re-reading after native code changes it must observe the change. - """ - module = _character_descriptor_module(tmp_path) +def test_descriptor_character_module_variables_follow_current_storage(pyi_parity_build_mode: str, tmp_path: Path): + """A retained handle queries the current association, address, and width.""" + module = _build_source_or_generated_pyi_and_import( + NATIVE_FIXTURES / "fchar_module_descriptors_f90.f90", + tmp_path, + { + "bind_c_fchar_module_descriptors_f90_wrapper.f90", + "fchar_module_descriptors_f90_wrapper.c", + "fchar_module_descriptors_f90_wrapper.h", + }, + CONTRACT_FIXTURES / "fchar_module_descriptors_f90", + pyi_parity_build_mode, + ) + deferred = module.deferred + fixed = module.fixed + link = module.link - assert module.deferred is None - assert module.fixed is None - assert module.link is None + assert deferred.to_numpy() is None and not deferred.allocated + assert fixed.to_numpy() is None and not fixed.allocated + assert link.to_numpy() is None and not link.associated module.setup() - assert module.deferred == "alpha" - assert module.fixed == "FIXEDV" - assert module.link == "STORED" + assert deferred.allocated and deferred.value == b"alpha" + assert fixed.allocated and fixed.value == b"FIXEDV" + assert link.associated and link.value == b"STORED" + view = link.to_numpy() + assert view is not None and view.shape == () and view.dtype == np.dtype("S6") + view[()] = b"PYTHON" + assert module.store[()] == b"PYTHON" - # A reallocation to a different width is observed by the next read. module.grow() - assert module.deferred == "alpha-more" + assert deferred.value == b"alpha-more" + assert deferred.to_numpy().dtype == np.dtype("S10") def test_descriptor_character_module_variables_report_absence_as_none(tmp_path: Path): """Deallocation and nullification are values Python observes, not stale reads.""" module = _character_descriptor_module(tmp_path) + deferred, fixed, link = module.deferred, module.fixed, module.link module.setup() module.clear() - assert module.deferred is None - assert module.fixed is None - assert module.link is None + assert deferred.to_numpy() is None and not deferred.allocated + assert fixed.to_numpy() is None and not fixed.allocated + assert link.to_numpy() is None and not link.associated def test_character_parameter_arrays_are_read_only_fixed_width_snapshots(tmp_path: Path): diff --git a/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py b/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py index fa52832a8..6d3c7b6ad 100644 --- a/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py +++ b/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py @@ -39,7 +39,6 @@ def _reload_native_module(build_dir: Path): def test_whole_scalar_module_variable_behavior_uses_canonical_plan( - monkeypatch: pytest.MonkeyPatch, tmp_path: Path, ): source = _write_whole_scalar_module(tmp_path / "fixture") @@ -57,8 +56,10 @@ def test_whole_scalar_module_variable_behavior_uses_canonical_plan( assert module.prefix == "D" assert module.counter == np.int32(3) assert module.target_scale == np.float64(1.5) - assert module.optional_scale is None - assert module.selected_scale is None + allocatable = module.optional_scale + pointer = module.selected_scale + assert not allocatable.allocated and allocatable.to_numpy() is None + assert not pointer.associated and pointer.to_numpy() is None values = module.values assert isinstance(values, AllocatableArray) assert values.allocated is False @@ -78,14 +79,15 @@ def test_whole_scalar_module_variable_behavior_uses_canonical_plan( module.selected_scale = np.float64(9.0) assert module.set_allocatable(np.float64(1.5)) == np.float64(1.5) - allocatable_snapshot = module.optional_scale assert module.point_to_target(np.float64(2.5)) == np.float64(2.5) - pointer_snapshot = module.selected_scale + allocatable_view = allocatable.to_numpy() + pointer_view = pointer.to_numpy() + assert allocatable_view is not None and pointer_view is not None assert module.bump_native() == np.float64(34.0) - assert allocatable_snapshot == np.float64(1.5) - assert pointer_snapshot == np.float64(2.5) - assert module.optional_scale == np.float64(11.5) - assert module.selected_scale == np.float64(22.5) + assert allocatable_view[()] == np.float64(11.5) + assert pointer_view[()] == np.float64(22.5) + assert allocatable.value == np.float64(11.5) + assert pointer.value == np.float64(22.5) module.allocate_values(np.int32(3)) assert module.values is values @@ -98,12 +100,8 @@ def test_whole_scalar_module_variable_behavior_uses_canonical_plan( assert values.allocated is False assert values.to_numpy() is None - monkeypatch.setenv("PRIK_WRAPPER_FAIL_ALLOC", "1") - assert module.optional_scale is None - assert module.selected_scale is None - monkeypatch.delenv("PRIK_WRAPPER_FAIL_ALLOC") - assert module.optional_scale == np.float64(11.5) - assert module.selected_scale == np.float64(22.5) + assert module.optional_scale.value == np.float64(11.5) + assert module.selected_scale.value == np.float64(22.5) module.nmax = np.int32(99) assert module.nmax == np.int32(99) diff --git a/tests/fortran/modules/policy/test_module_variable_policy.py b/tests/fortran/modules/policy/test_module_variable_policy.py index 97a77f065..510c3dff9 100644 --- a/tests/fortran/modules/policy/test_module_variable_policy.py +++ b/tests/fortran/modules/policy/test_module_variable_policy.py @@ -47,11 +47,11 @@ def test_scalar_module_variable_policy_completes_access_and_storage_before_plann assert policies["target_scale"].getter_action is ModuleGetterAction.DIRECT_VALUE assert policies["target_scale"].setter_action is SetterAction.WRITE_THROUGH assert policies["target_scale"].native_assignment is AssignmentMode.VALUE_COPY - assert policies["optional_scale"].getter_action is ModuleGetterAction.NULLABLE_SNAPSHOT + assert policies["optional_scale"].getter_action is ModuleGetterAction.NATIVE_SCALAR_HANDLE assert policies["optional_scale"].descriptor_kind == "allocatable" assert policies["optional_scale"].setter_action is SetterAction.REJECT_REPLACEMENT assert policies["optional_scale"].native_assignment is AssignmentMode.NONE - assert policies["selected_scale"].getter_action is ModuleGetterAction.NULLABLE_SNAPSHOT + assert policies["selected_scale"].getter_action is ModuleGetterAction.NATIVE_SCALAR_HANDLE assert policies["selected_scale"].descriptor_kind == "pointer" assert policies["selected_scale"].setter_action is SetterAction.REJECT_REPLACEMENT assert policies["selected_scale"].native_assignment is AssignmentMode.NONE diff --git a/tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py b/tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py index 45dfe08c6..a2a5be817 100644 --- a/tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py +++ b/tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py @@ -113,6 +113,7 @@ def test_optional_arguments_drive_fortran_present_behavior( assert module.summarize(np.int32(5)) == np.int32(5) assert module.summarize(np.int32(5), np.int32(4)) == np.int32(9) + assert module.summarize(np.array(5, dtype=np.int32), np.array(4, dtype=np.int32)) == np.int32(9) assert module.summarize(np.int32(5), None) == np.int32(5) assert module.summarize(np.int32(5), scale=None) == np.int32(5) assert module.summarize(np.int32(5), values=values) == np.int32(11) diff --git a/tests/fortran/strings/end_to_end/test_documented_string_journey.py b/tests/fortran/strings/end_to_end/test_documented_string_journey.py index 40acaaef8..9e03e944e 100644 --- a/tests/fortran/strings/end_to_end/test_documented_string_journey.py +++ b/tests/fortran/strings/end_to_end/test_documented_string_journey.py @@ -47,6 +47,14 @@ def edit_labels(count: Int32, labels: String[8][count]) -> None: ... original = "alpha " assert module.edit_text(original) == "Xlpha " assert original == "alpha " + borrowed_text = np.array("alpha ", dtype="S8") + assert module.edit_text(borrowed_text) == "Xlpha " + assert borrowed_text[()] == np.bytes_(b"Xlpha ") + with pytest.raises(TypeError): + module.edit_text(np.array("alpha ", dtype="S7")) + borrowed_text.flags.writeable = False + with pytest.raises(TypeError, match="writeable"): + module.edit_text(borrowed_text) assert module.make_text() == "ready " made_labels = module.make_labels() assert made_labels.dtype == np.dtype("S8") From 01cc077a3c07c3014dbdd84e509b8343f5292c3a Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 11:29:06 +0100 Subject: [PATCH 04/47] codex: return views for scalar module descriptors Read scalar allocatable and pointer module variables as nullable native rank-zero views. Remove the scalar-handle callback runtime, keep array descriptor handles, and verify source/pyi parity plus real Open MPI execution. --- CHANGELOG.md | 4 +- docs/user/guide/allocatables.md | 17 ++-- docs/user/guide/pointers.md | 14 ++-- docs/user/guide/wrapping-modules.md | 2 + docs/user/reference/pyi-format.md | 9 ++- prik/codegen/c/binding.py | 73 ++++++++++------- prik/codegen/fortran/bridge.py | 10 +-- prik/planning/entrypoints.py | 4 +- prik/policy/construction.py | 17 ++-- prik/policy/models.py | 2 +- prik/policy/ownership.py | 2 +- prik/runtime/handles.py | 78 ------------------- .../end_to_end/test_scalar_allocatables.py | 14 ++-- .../test_scalar_module_variable_lowering.py | 2 +- .../test_module_variables_and_state.py | 35 ++++----- .../test_scalar_module_variable_plan.py | 18 ++--- .../policy/test_module_variable_policy.py | 4 +- .../end_to_end/test_scalar_pointers.py | 12 +-- 18 files changed, 128 insertions(+), 189 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f2d5fb5c1..1385e3ee7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,8 +11,8 @@ release tags add a leading `v` to the package version. NumPy views, including fixed-length character bytes; primitive and fixed character value dummies accept matching rank-zero storage as well as scalar values, preserving the dummy's reference or `VALUE` ABI. -- Scalar allocatable and pointer module variables expose handles that follow - native allocation or association and provide current rank-zero NumPy views. +- Scalar allocatable and pointer module variables return live rank-zero NumPy + views or `None` when storage is absent. - Contributor test guidance focuses on supported behavior and meaningful validation boundaries after a feature is removed. diff --git a/docs/user/guide/allocatables.md b/docs/user/guide/allocatables.md index 66bce7372..e00b09d4f 100644 --- a/docs/user/guide/allocatables.md +++ b/docs/user/guide/allocatables.md @@ -12,13 +12,13 @@ publication: reviewed A Fortran allocatable descriptor records whether storage is allocated and, for arrays, its address, shape, and strides. The descriptor controls the allocation, -and a PRIK handle gives Python access to that descriptor. +and an array handle gives Python access to that descriptor. ## Key Concepts -- Scalar allocatable dummies and results appear as values or `None`. Scalar - module variables use handles; array allocatables use `Allocatable[T[...]]` - handles. +- Scalar allocatable dummies and results appear as values or `None`. Reading a + scalar allocatable module variable returns a live rank-zero NumPy view or + `None`; array allocatables use `Allocatable[T[...]]` handles. - An array handle exposes allocation state and descriptor operations; it is not itself a NumPy array. - `allocated` reports whether storage exists; `to_numpy()` returns a live view @@ -83,11 +83,10 @@ assert values.allocated is True The annotation supplies the element dtype and rank. The handle creates its native storage when first passed to a matching writable argument. It stays the same Python object after the call. -`Allocatable[Float64]()` is not supported. A scalar module variable declared -`Allocatable[Float64]` returns a native scalar handle that follows allocation -changes. Its `allocated` property reports presence, `value` reads the current -NumPy scalar or `None`, and `to_numpy()` borrows a rank-zero view or returns -`None`. Acquire a fresh view after reallocation. +`Allocatable[Float64]()` is not supported. Reading a scalar module variable +declared `Allocatable[Float64]` returns a live rank-zero `float64` array when +allocated, or `None` otherwise. Read the attribute again after reallocation; +an older view may refer to storage that is no longer valid. A returned or attribute array handle remains present even when its descriptor is unallocated. Reading the Python attribute diff --git a/docs/user/guide/pointers.md b/docs/user/guide/pointers.md index 1a046eecf..71b4c86ad 100644 --- a/docs/user/guide/pointers.md +++ b/docs/user/guide/pointers.md @@ -18,8 +18,9 @@ shape, and strides. It does not by itself say who owns that target. - A pointer descriptor refers to target storage; it does not own that storage by default. -- Scalar pointer dummies and results appear as values or `None`. Scalar module - variables use handles; array pointers use live `Pointer[T[...]]` handles. +- Scalar pointer dummies and results appear as values or `None`. Reading a + scalar pointer module variable returns a live rank-zero NumPy view or `None`; + array pointers use live `Pointer[T[...]]` handles. - `associated` describes association, not ownership or target lifetime. - NumPy arrays returned by `to_numpy()` are live views, not copies. - Reassociation, resizing, or deallocation can invalidate existing views. @@ -75,11 +76,10 @@ assert target.associated is True The annotation supplies the element dtype and rank. The handle creates its native storage when first passed to a matching writable argument. It stays the same Python object after the call. -`Pointer[Float64]()` is not supported. A scalar module variable declared -`Pointer[Float64]` returns a native scalar handle that follows association -changes. Its `associated` property reports presence, `value` reads the current -NumPy scalar or `None`, and `to_numpy()` borrows a rank-zero view or returns -`None`. Acquire a fresh view after reassociation. +`Pointer[Float64]()` is not supported. Reading a scalar module variable +declared `Pointer[Float64]` returns a live rank-zero `float64` array when +associated, or `None` otherwise. Read the attribute again after reassociation; +an older view may refer to storage that is no longer valid. | Member | Type | Behavior | | --- | --- | --- | diff --git a/docs/user/guide/wrapping-modules.md b/docs/user/guide/wrapping-modules.md index 8fc5bb3c7..e392b0e02 100644 --- a/docs/user/guide/wrapping-modules.md +++ b/docs/user/guide/wrapping-modules.md @@ -140,6 +140,8 @@ Mutable fixed-storage scalar module variables expose live rank-zero NumPy views. Numeric and logical scalars use `T[()]`; fixed-length character scalars use `String[n][()]` with raw bytes storage. Pass the view itself when a native dummy needs its original storage. `PARAMETER` declarations remain constants. +- Scalar allocatable and pointer module variables return a live rank-zero view + or `None` on each read. Read the attribute again after storage changes. - Allocatable module arrays use the `Allocatable[T[...]]` API. - Allocation, lifetime, NumPy views, and mutation rules are covered in the storage and objects section. diff --git a/docs/user/reference/pyi-format.md b/docs/user/reference/pyi-format.md index fc6d9881f..63133962d 100644 --- a/docs/user/reference/pyi-format.md +++ b/docs/user/reference/pyi-format.md @@ -406,10 +406,11 @@ scale: Float64[()] Mutable Fortran module variables expose their native storage. Fixed-storage numeric and logical scalars use live rank-zero `T[()]` NumPy views; fixed-length character scalars use live rank-zero `String[n][()]` bytes views. `PARAMETER` -declarations use `Final[...]` values. Allocatable and pointer storage uses a -handle that follows allocation or association changes. An edited plain `T` -module declaration requests a scalar value getter. A literal default on -supported mutable scalar state is an import-time native initializer. +declarations use `Final[...]` values. Scalar allocatable and pointer module +variables return a live rank-zero view or `None` on each read; array descriptors +use handles. An edited plain `T` module declaration requests a scalar value +getter. A literal default on supported mutable scalar state is an import-time +native initializer. C global declarations can be represented for inspection, but current C wrapper builds reject native global state. C functions remain the supported runtime diff --git a/prik/codegen/c/binding.py b/prik/codegen/c/binding.py index da6054751..26a79ed32 100644 --- a/prik/codegen/c/binding.py +++ b/prik/codegen/c/binding.py @@ -221,7 +221,7 @@ class _COverloadDispatch: ModuleGetterAction.DIRECT_VALUE: "Builds a Python scalar from the current native value.", ModuleGetterAction.NATIVE_SCALAR_VIEW: "Wraps live native scalar storage in a rank-zero NumPy view.", ModuleGetterAction.NATIVE_CHARACTER_VIEW: "Wraps live native character bytes in a rank-zero NumPy view.", - ModuleGetterAction.NATIVE_SCALAR_HANDLE: "Returns a handle that queries current native scalar storage.", + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW: "Borrows the current native scalar storage, or returns None.", ModuleGetterAction.CHARACTER_VALUE: "Decodes the fixed-width native characters into a Python str.", ModuleGetterAction.NULLABLE_SNAPSHOT: "Returns a detached copy, or None when the native value holds nothing.", ModuleGetterAction.BORROWED_ARRAY_VIEW: "Wraps the native storage in a live NumPy array without copying.", @@ -4536,7 +4536,7 @@ def _module_array_owner_variables(self, plan: ModulePlan) -> tuple[ModuleVariabl ModuleGetterAction.NATIVE_ARRAY_HANDLE, ModuleGetterAction.NATIVE_SCALAR_VIEW, ModuleGetterAction.NATIVE_CHARACTER_VIEW, - ModuleGetterAction.NATIVE_SCALAR_HANDLE, + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW, } ) @@ -6136,8 +6136,8 @@ def _lower_module_getter(self, plan: ModuleVariablePlan) -> tuple[CFunction, ... return self._lower_module_getter_native_scalar_view(plan) case ModuleGetterAction.NATIVE_CHARACTER_VIEW: return self._lower_module_getter_native_character_view(plan) - case ModuleGetterAction.NATIVE_SCALAR_HANDLE: - return self._lower_module_getter_native_scalar_handle(plan) + case ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW: + return self._lower_module_getter_native_nullable_scalar_view(plan) case ModuleGetterAction.CHARACTER_VALUE: return self._lower_module_getter_character_value(plan) case ModuleGetterAction.NULLABLE_SNAPSHOT: @@ -6227,44 +6227,61 @@ def _lower_module_getter_native_character_view(self, plan: ModuleVariablePlan) - ), ) - def _lower_module_getter_native_scalar_handle(self, plan: ModuleVariablePlan) -> tuple[CFunction, ...]: - """Create a scalar descriptor handle around one callable native inquiry.""" + def _lower_module_getter_native_nullable_scalar_view(self, plan: ModuleVariablePlan) -> tuple[CFunction, ...]: + """Borrow the currently present scalar storage as one rank-zero view.""" owner = self._module_native_array_owner_name(plan) - descriptor = plan.entrypoint.descriptor_kind - if descriptor not in {"allocatable", "pointer"}: - raise ValueError(f"Scalar descriptor {plan.owner_path!r} has no completed descriptor kind") + character = plan.datatype_family is DatatypeFamily.STRING + if character: + result = CodeExpression( + "PyArray_New(&PyArray_Type, 0, NULL, NPY_STRING, NULL, data, (int)length, " + "NPY_ARRAY_ALIGNED | NPY_ARRAY_WRITEABLE, NULL)" + ) + else: + scalar = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) + result = CodeExpression(f"PyArray_SimpleNewFromData(0, NULL, {scalar.array_numpy_type}, data)") return ( CFunction( self._module_getter_name(plan), "PyObject *", storage="static", body=( + *((CDeclaration("length", "int64_t", CodeExpression("0")),) if character else ()), CDeclaration( - "runtime", "PyObject *", CodeExpression('PyImport_ImportModule("prik.runtime.handles")') - ), - CIf(CodeExpression("runtime == NULL"), body=(CReturn(CodeExpression("NULL")),)), - CDeclaration( - "factory", - "PyObject *", + "data", + "void *", CodeExpression( - 'PyObject_GetAttrString(runtime, "_native_scalar_handle_from_generated_address")' + f"{self._module_bridge_getter_name(plan)}(&length)" + if character + else f"{self._module_bridge_getter_name(plan)}()" ), ), - CExpressionStatement(CodeExpression("Py_DECREF(runtime)")), - CIf(CodeExpression("factory == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CIf( + CodeExpression("data == NULL"), + body=(CExpressionStatement(CodeExpression("Py_RETURN_NONE")),), + ), + *( + ( + CIf( + CodeExpression("length < 0 || (int64_t)(int)length != length"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_SetString(PyExc_OverflowError, "Native character width exceeds NumPy itemsize")' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + ) + if character + else () + ), CDeclaration( "result", "PyObject *", - CodeExpression( - f'PyObject_CallFunction(factory, "KssOi", ' - f"(unsigned long long)(uintptr_t)&{self._module_bridge_getter_name(plan)}, " - f"{self._c_string_literal(plan.semantic_type_name)}, " - f"{self._c_string_literal(descriptor)}, {owner}, " - f"{1 if plan.datatype_family is DatatypeFamily.STRING else 0})" - ), + result, ), - CExpressionStatement(CodeExpression("Py_DECREF(factory)")), - CReturn(CodeExpression("result")), + *self._ordinary_array_field_owner_nodes("result", owner), ), ), ) @@ -16311,7 +16328,7 @@ def _module_native_array_owner_nodes( ModuleGetterAction.NATIVE_ARRAY_HANDLE, ModuleGetterAction.NATIVE_SCALAR_VIEW, ModuleGetterAction.NATIVE_CHARACTER_VIEW, - ModuleGetterAction.NATIVE_SCALAR_HANDLE, + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW, }: continue owner = self._module_native_array_owner_name(variable) diff --git a/prik/codegen/fortran/bridge.py b/prik/codegen/fortran/bridge.py index 6928d099e..80d53360b 100644 --- a/prik/codegen/fortran/bridge.py +++ b/prik/codegen/fortran/bridge.py @@ -2697,8 +2697,8 @@ def _lower_module_getter(self, plan: ModuleVariablePlan) -> tuple[FortranFunctio return self._lower_module_getter_native_scalar_view(plan) case ModuleGetterAction.NATIVE_CHARACTER_VIEW: return self._lower_module_getter_native_scalar_view(plan) - case ModuleGetterAction.NATIVE_SCALAR_HANDLE: - return self._lower_module_getter_native_scalar_handle(plan) + case ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW: + return self._lower_module_getter_native_nullable_scalar_view(plan) case ModuleGetterAction.CHARACTER_VALUE: return self._lower_module_getter_character_value(plan) case ModuleGetterAction.NULLABLE_SNAPSHOT: @@ -3457,8 +3457,8 @@ def _lower_module_getter_native_scalar_view(self, plan: ModuleVariablePlan) -> t ), ) - def _lower_module_getter_native_scalar_handle(self, plan: ModuleVariablePlan) -> tuple[FortranFunction, ...]: - """Query a scalar descriptor's current storage without retaining an old address.""" + def _lower_module_getter_native_nullable_scalar_view(self, plan: ModuleVariablePlan) -> tuple[FortranFunction, ...]: + """Query a scalar descriptor's current storage for one attribute read.""" native = self._native_variable_name(plan) present = {"allocatable": "allocated", "pointer": "associated"}.get(plan.entrypoint.descriptor_kind) if present is None: @@ -3706,7 +3706,7 @@ def _module_array_capture_interfaces(self, plan: ModulePlan) -> tuple[FortranInt in { ModuleGetterAction.NATIVE_SCALAR_VIEW, ModuleGetterAction.NATIVE_CHARACTER_VIEW, - ModuleGetterAction.NATIVE_SCALAR_HANDLE, + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW, } for variable in self._variables(plan) ): diff --git a/prik/planning/entrypoints.py b/prik/planning/entrypoints.py index 0690b9377..a92d51b32 100644 --- a/prik/planning/entrypoints.py +++ b/prik/planning/entrypoints.py @@ -1126,7 +1126,7 @@ def _primary_module_variable_operations(self, variable): result = self._opaque_result() elif ( variable.bridge.native_getter_action - in {ModuleGetterAction.NULLABLE_SNAPSHOT, ModuleGetterAction.NATIVE_SCALAR_HANDLE} + in {ModuleGetterAction.NULLABLE_SNAPSHOT, ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW} and variable.datatype_family is DatatypeFamily.STRING ): parameters = (self._int64_parameter("length", reference=True, intent="out"),) @@ -1136,7 +1136,7 @@ def _primary_module_variable_operations(self, variable): ModuleGetterAction.DERIVED_OBJECT, ModuleGetterAction.NATIVE_SCALAR_VIEW, ModuleGetterAction.NATIVE_CHARACTER_VIEW, - ModuleGetterAction.NATIVE_SCALAR_HANDLE, + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW, }: parameters = () result = self._opaque_result() diff --git a/prik/policy/construction.py b/prik/policy/construction.py index cf64a6706..c164d88e6 100644 --- a/prik/policy/construction.py +++ b/prik/policy/construction.py @@ -1278,7 +1278,7 @@ def _scalar_module_variable_policy( in { ModuleGetterAction.NATIVE_SCALAR_VIEW, ModuleGetterAction.NATIVE_CHARACTER_VIEW, - ModuleGetterAction.NATIVE_SCALAR_HANDLE, + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW, } else None ), @@ -7009,14 +7009,13 @@ def _scalar_module_getter_blockers( blockers = [] literal_string = _is_binding_literal_string(variable, getter_action) character_value = getter_action in {ModuleGetterAction.CHARACTER_VALUE, ModuleGetterAction.NATIVE_CHARACTER_VIEW} - # A descriptor character module variable reaches Python through the same - # nullable snapshot a descriptor scalar uses, carrying a runtime width. - character_snapshot = ( - getter_action in {ModuleGetterAction.NULLABLE_SNAPSHOT, ModuleGetterAction.NATIVE_SCALAR_HANDLE} + # A descriptor character getter reports its current width with the address. + character_descriptor = ( + getter_action in {ModuleGetterAction.NULLABLE_SNAPSHOT, ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW} and variable.semantic_type.name == "String" ) - string_getter = literal_string or character_value or character_snapshot - if not (_is_first_lane_scalar_type(variable.semantic_type) or string_getter or character_snapshot): + string_getter = literal_string or character_value or character_descriptor + if not (_is_first_lane_scalar_type(variable.semantic_type) or string_getter): blockers.append("module variable is not a primitive rank-zero scalar") if character_value and _character_length(variable.semantic_type) is None: blockers.append("character module variable requires one declared length") @@ -7027,7 +7026,7 @@ def _scalar_module_getter_blockers( in { ModuleGetterAction.NATIVE_SCALAR_VIEW, ModuleGetterAction.NATIVE_CHARACTER_VIEW, - ModuleGetterAction.NATIVE_SCALAR_HANDLE, + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW, } else {CodegenAction.COPY_OUT} if string_getter @@ -7134,7 +7133,7 @@ def _scalar_module_getter_action( return ModuleGetterAction.NATIVE_CONSTANT_VALUE return ModuleGetterAction.CONSTANT_VALUE if _scalar_module_descriptor_kind(variable) is not None: - return ModuleGetterAction.NATIVE_SCALAR_HANDLE + return ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW if getter is not None and getter.codegen_action is CodegenAction.SNAPSHOT_COPY and getter.nullable: return ModuleGetterAction.NULLABLE_SNAPSHOT if variable.semantic_type.metadata.get("native_storage") and _is_fixed_length_character_scalar(variable): diff --git a/prik/policy/models.py b/prik/policy/models.py index 066347250..33ccafdf3 100644 --- a/prik/policy/models.py +++ b/prik/policy/models.py @@ -340,7 +340,7 @@ class ModuleGetterAction(str, Enum): DIRECT_VALUE = "direct_value" NATIVE_SCALAR_VIEW = "native_scalar_view" NATIVE_CHARACTER_VIEW = "native_character_view" - NATIVE_SCALAR_HANDLE = "native_scalar_handle" + NATIVE_NULLABLE_SCALAR_VIEW = "native_nullable_scalar_view" CHARACTER_VALUE = "character_value" NULLABLE_SNAPSHOT = "nullable_snapshot" BORROWED_ARRAY_VIEW = "borrowed_array_view" diff --git a/prik/policy/ownership.py b/prik/policy/ownership.py index 7ee97db02..969b66f07 100644 --- a/prik/policy/ownership.py +++ b/prik/policy/ownership.py @@ -1892,7 +1892,7 @@ def _module_variable_decision(self, facts: _StorageFacts, context: OwnershipCont boundary_storage_mode=StorageMode.ALIAS, nullable=True, borrowed=True, - reason="scalar module descriptor is queried through a persistent native handle", + reason="scalar module descriptor supplies a current native view on each read", ) if facts.rank > 0 or facts.is_ndarray: if facts.pointer: diff --git a/prik/runtime/handles.py b/prik/runtime/handles.py index 23b8d6c53..331de231d 100644 --- a/prik/runtime/handles.py +++ b/prik/runtime/handles.py @@ -41,84 +41,6 @@ def _retain_numpy_owner(value: np.ndarray, owner: Any) -> np.ndarray: return retained -class NativeScalarHandle: - """Follow the current storage of a native scalar allocatable or pointer.""" - - def __init__( - self, - address: int, - semantic_type_name: str, - descriptor_kind: str, - owner: Any, - character: bool, - ) -> None: - if descriptor_kind not in {"allocatable", "pointer"}: - raise ValueError("scalar handle descriptor kind must be allocatable or pointer") - self._owner = owner - self._descriptor_kind = descriptor_kind - self._character = character - self._dtype = ( - None - if character - else np.dtype("bool" if semantic_type_name in {"Bool", "Bool8"} else semantic_type_name.lower()) - ) - signature = ( - ctypes.CFUNCTYPE(ctypes.c_void_p, ctypes.POINTER(ctypes.c_int64)) - if character - else ctypes.CFUNCTYPE(ctypes.c_void_p) - ) - self._query = signature(address) - - def _current(self) -> tuple[int, np.dtype[Any] | None]: - if self._character: - width = ctypes.c_int64(0) - address = self._query(ctypes.byref(width)) - dtype = np.dtype(f"S{width.value}") if address and width.value >= 0 else None - else: - address = self._query() - dtype = self._dtype if address else None - return int(address or 0), dtype - - @property - def allocated(self) -> bool: - """Report whether the allocatable currently has storage.""" - if self._descriptor_kind != "allocatable": - raise AttributeError("allocated is only available for allocatable handles") - return bool(self._current()[0]) - - @property - def associated(self) -> bool: - """Report whether the pointer currently has a target.""" - if self._descriptor_kind != "pointer": - raise AttributeError("associated is only available for pointer handles") - return bool(self._current()[0]) - - def to_numpy(self) -> np.ndarray | None: - """Borrow a rank-zero view of the current native storage, if present.""" - address, dtype = self._current() - if dtype is None: - return None - buffer = (ctypes.c_char * max(dtype.itemsize, 1)).from_address(address) - return _retain_numpy_owner(np.ndarray((), dtype=dtype, buffer=buffer), self) - - @property - def value(self) -> np.generic | None: - """Read the scalar's current value without retaining an older address.""" - view = self.to_numpy() - return None if view is None else view[()] - - -def _native_scalar_handle_from_generated_address( - address: int, - semantic_type_name: str, - descriptor_kind: str, - owner: Any, - character: int, -) -> NativeScalarHandle: - """Bind a generated native inquiry while retaining its extension owner.""" - return NativeScalarHandle(address, semantic_type_name, descriptor_kind, owner, bool(character)) - - def _descriptor_facts(value: Any, rank: int) -> tuple[int, ...]: """Validate one flat descriptor-fact tuple reported for a handle.""" expected = _DESCRIPTOR_FACT_HEADER + _DESCRIPTOR_FACTS_PER_AXIS * int(rank) diff --git a/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py b/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py index d86454367..20bacd982 100644 --- a/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py +++ b/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py @@ -30,20 +30,20 @@ def test_scalar_allocatables_project_values_and_unallocated_state( ) module.clear_module_value() - handle = module.optional_scale - assert not handle.allocated and handle.to_numpy() is None + assert module.optional_scale is None assert not hasattr(module, "get_optional_scale") assert not hasattr(module, "set_optional_scale") with pytest.raises(AttributeError): module.optional_scale = np.float64(9.0) module.set_module_value(np.float64(1.5)) - assert handle.allocated and handle.value == np.float64(1.5) - view = handle.to_numpy() + view = module.optional_scale assert view is not None and view.shape == () and view.dtype == np.dtype("float64") + assert view[()] == np.float64(1.5) + view[()] = np.float64(2.5) module.bump_module_value() - assert view[()] == np.float64(11.5) - assert handle.value == np.float64(11.5) + assert view[()] == np.float64(12.5) + assert module.optional_scale[()] == np.float64(12.5) assert module.echo_allocatable(np.float64(3.0)) == np.float64(4.0) assert module.echo_allocatable(None) == np.float64(-1.0) @@ -55,4 +55,4 @@ def test_scalar_allocatables_project_values_and_unallocated_state( assert module.maybe_allocatable(np.int32(0)) is None module.clear_module_value() - assert not handle.allocated and handle.to_numpy() is None + assert module.optional_scale is None diff --git a/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py b/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py index 56a2315da..04379e91f 100644 --- a/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py +++ b/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py @@ -107,7 +107,7 @@ def test_module_variable_plan_contains_only_completed_dispatch_facts(): assert variables["counter"].bridge.native_assignment is AssignmentMode.VALUE_COPY assert variables["counter"].binding.initializer == 3 assert variables["target_scale"].bridge.native_assignment is AssignmentMode.VALUE_COPY - assert variables["optional_scale"].binding.getter_action is ModuleGetterAction.NATIVE_SCALAR_HANDLE + assert variables["optional_scale"].binding.getter_action is ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW assert variables["optional_scale"].entrypoint.descriptor_kind == "allocatable" assert variables["optional_scale"].binding.setter_action is SetterAction.REJECT_REPLACEMENT assert variables["optional_scale"].bridge.native_assignment is AssignmentMode.NONE diff --git a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py index c39a1ac99..b8c232c88 100644 --- a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py +++ b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py @@ -302,7 +302,7 @@ def _character_descriptor_module(tmp_path: Path): def test_descriptor_character_module_variables_follow_current_storage(pyi_parity_build_mode: str, tmp_path: Path): - """A retained handle queries the current association, address, and width.""" + """Each attribute read borrows the current address and character width.""" module = _build_source_or_generated_pyi_and_import( NATIVE_FIXTURES / "fchar_module_descriptors_f90.f90", tmp_path, @@ -314,38 +314,37 @@ def test_descriptor_character_module_variables_follow_current_storage(pyi_parity CONTRACT_FIXTURES / "fchar_module_descriptors_f90", pyi_parity_build_mode, ) - deferred = module.deferred - fixed = module.fixed - link = module.link - - assert deferred.to_numpy() is None and not deferred.allocated - assert fixed.to_numpy() is None and not fixed.allocated - assert link.to_numpy() is None and not link.associated + assert module.deferred is None + assert module.fixed is None + assert module.link is None module.setup() - assert deferred.allocated and deferred.value == b"alpha" - assert fixed.allocated and fixed.value == b"FIXEDV" - assert link.associated and link.value == b"STORED" - view = link.to_numpy() + deferred = module.deferred + fixed = module.fixed + view = module.link + assert deferred is not None and deferred.shape == () and deferred.dtype == np.dtype("S5") + assert deferred[()] == b"alpha" + assert fixed is not None and fixed[()] == b"FIXEDV" assert view is not None and view.shape == () and view.dtype == np.dtype("S6") + assert view[()] == b"STORED" view[()] = b"PYTHON" assert module.store[()] == b"PYTHON" module.grow() - assert deferred.value == b"alpha-more" - assert deferred.to_numpy().dtype == np.dtype("S10") + grown = module.deferred + assert grown is not None and grown.shape == () and grown.dtype == np.dtype("S10") + assert grown[()] == b"alpha-more" def test_descriptor_character_module_variables_report_absence_as_none(tmp_path: Path): """Deallocation and nullification are values Python observes, not stale reads.""" module = _character_descriptor_module(tmp_path) - deferred, fixed, link = module.deferred, module.fixed, module.link module.setup() module.clear() - assert deferred.to_numpy() is None and not deferred.allocated - assert fixed.to_numpy() is None and not fixed.allocated - assert link.to_numpy() is None and not link.associated + assert module.deferred is None + assert module.fixed is None + assert module.link is None def test_character_parameter_arrays_are_read_only_fixed_width_snapshots(tmp_path: Path): diff --git a/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py b/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py index 6d3c7b6ad..52e1f21be 100644 --- a/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py +++ b/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py @@ -56,10 +56,8 @@ def test_whole_scalar_module_variable_behavior_uses_canonical_plan( assert module.prefix == "D" assert module.counter == np.int32(3) assert module.target_scale == np.float64(1.5) - allocatable = module.optional_scale - pointer = module.selected_scale - assert not allocatable.allocated and allocatable.to_numpy() is None - assert not pointer.associated and pointer.to_numpy() is None + assert module.optional_scale is None + assert module.selected_scale is None values = module.values assert isinstance(values, AllocatableArray) assert values.allocated is False @@ -80,14 +78,14 @@ def test_whole_scalar_module_variable_behavior_uses_canonical_plan( assert module.set_allocatable(np.float64(1.5)) == np.float64(1.5) assert module.point_to_target(np.float64(2.5)) == np.float64(2.5) - allocatable_view = allocatable.to_numpy() - pointer_view = pointer.to_numpy() + allocatable_view = module.optional_scale + pointer_view = module.selected_scale assert allocatable_view is not None and pointer_view is not None assert module.bump_native() == np.float64(34.0) assert allocatable_view[()] == np.float64(11.5) assert pointer_view[()] == np.float64(22.5) - assert allocatable.value == np.float64(11.5) - assert pointer.value == np.float64(22.5) + assert module.optional_scale[()] == np.float64(11.5) + assert module.selected_scale[()] == np.float64(22.5) module.allocate_values(np.int32(3)) assert module.values is values @@ -100,8 +98,8 @@ def test_whole_scalar_module_variable_behavior_uses_canonical_plan( assert values.allocated is False assert values.to_numpy() is None - assert module.optional_scale.value == np.float64(11.5) - assert module.selected_scale.value == np.float64(22.5) + assert module.optional_scale[()] == np.float64(11.5) + assert module.selected_scale[()] == np.float64(22.5) module.nmax = np.int32(99) assert module.nmax == np.int32(99) diff --git a/tests/fortran/modules/policy/test_module_variable_policy.py b/tests/fortran/modules/policy/test_module_variable_policy.py index 510c3dff9..2b0ab3cc8 100644 --- a/tests/fortran/modules/policy/test_module_variable_policy.py +++ b/tests/fortran/modules/policy/test_module_variable_policy.py @@ -47,11 +47,11 @@ def test_scalar_module_variable_policy_completes_access_and_storage_before_plann assert policies["target_scale"].getter_action is ModuleGetterAction.DIRECT_VALUE assert policies["target_scale"].setter_action is SetterAction.WRITE_THROUGH assert policies["target_scale"].native_assignment is AssignmentMode.VALUE_COPY - assert policies["optional_scale"].getter_action is ModuleGetterAction.NATIVE_SCALAR_HANDLE + assert policies["optional_scale"].getter_action is ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW assert policies["optional_scale"].descriptor_kind == "allocatable" assert policies["optional_scale"].setter_action is SetterAction.REJECT_REPLACEMENT assert policies["optional_scale"].native_assignment is AssignmentMode.NONE - assert policies["selected_scale"].getter_action is ModuleGetterAction.NATIVE_SCALAR_HANDLE + assert policies["selected_scale"].getter_action is ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW assert policies["selected_scale"].descriptor_kind == "pointer" assert policies["selected_scale"].setter_action is SetterAction.REJECT_REPLACEMENT assert policies["selected_scale"].native_assignment is AssignmentMode.NONE diff --git a/tests/fortran/pointers/end_to_end/test_scalar_pointers.py b/tests/fortran/pointers/end_to_end/test_scalar_pointers.py index fe02345d9..382d6e5da 100644 --- a/tests/fortran/pointers/end_to_end/test_scalar_pointers.py +++ b/tests/fortran/pointers/end_to_end/test_scalar_pointers.py @@ -60,7 +60,7 @@ def _scalar_pointer_module(build_mode: str, tmp_path: Path): return _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) -def test_scalar_pointers_project_nullable_copied_values( +def test_scalar_pointers_project_nullable_native_views( pyi_parity_build_mode: str, tmp_path: Path, ): @@ -72,11 +72,13 @@ def test_scalar_pointers_project_nullable_copied_values( module.selected_scale = np.float64(9.0) module.point_to_target(np.float64(2.5)) - snapshot = module.selected_scale - assert snapshot == np.float64(2.5) + view = module.selected_scale + assert view is not None and view.shape == () and view.dtype == np.dtype("float64") + assert view[()] == np.float64(2.5) + view[()] = np.float64(3.5) module.bump_native() - assert snapshot == np.float64(2.5) - assert module.selected_scale == np.float64(22.5) + assert view[()] == np.float64(23.5) + assert module.selected_scale[()] == np.float64(23.5) assert module.echo_pointer(np.float64(3.0)) == np.float64(5.0) assert module.echo_pointer(None) == np.float64(-2.0) From 026f26f5015871eb9180a363e743c5bcffdb57e5 Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 14:52:56 +0100 Subject: [PATCH 05/47] Fix review findings in native storage, logicals, and export selection - Scalar allocatable and pointer module variables lend read-only views; assignment allocates an allocatable or writes a pointer's current target. - Rank-zero actual transport follows the dummy's VALUE attribute on direct and adapted routes; Immutable arguments copy rank-zero storage in. - Logical scalar dummies wider than one byte borrow integer storage of their own width, as logical arrays do, so intent(inout) updates reach Python. - An omitted optional intent(inout) scalar returns None instead of reading a NULL storage pointer; storage-typed scalars unpack then convert. - Empty deferred-length character module variables read as b"". - Module-scope PARAMETER statements honor IMPLICIT rules and implicit none. - Export selection keeps component and parent types of selected types, and source builds publish the same surface as generated contracts. - Update tests and golden fixtures that pinned the replaced behavior. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 20 +- docs/user/guide/allocatables.md | 15 +- docs/user/guide/data-types.md | 8 +- docs/user/guide/pointers.md | 12 +- docs/user/guide/wrapping-modules.md | 5 +- .../pyi-contracts/calls-and-results.md | 4 +- docs/user/reference/python-api.md | 4 +- prik/codegen/c/binding.py | 214 ++++++++++++++---- prik/codegen/docstrings.py | 16 +- prik/codegen/fortran/bridge.py | 97 +++++++- prik/parsers/fortran/parser.py | 130 ++++++++++- prik/pipeline/build.py | 37 ++- prik/pipeline/wrapper.py | 11 +- prik/planning/entrypoints.py | 39 ++-- prik/planning/models.py | 3 +- prik/planning/planner.py | 4 +- prik/policy/construction.py | 105 ++++++--- prik/policy/models.py | 9 +- prik/policy/ownership.py | 51 +++-- prik/semantics/fortran_exports.py | 59 ++++- .../test_exact_native_scalar_lowering.py | 5 +- .../end_to_end/test_direct_c_runtime.py | 2 +- .../end_to_end/test_scalar_allocatables.py | 15 +- .../fixtures/native/flogical_storage_f90.f90 | 29 +++ .../end_to_end/test_logical_scalar_storage.py | 55 +++++ .../test_rank_zero_scalar_storage.py | 8 +- .../codegen/test_status_error_lowering.py | 4 +- .../codegen/test_multiple_function_results.py | 2 +- .../leaf_types.f90 | 6 + .../nested_types.f90 | 29 +++ .../test_fortran_export_symbol_workflow.py | 68 ++++++ .../policy/test_native_entrypoint_policy.py | 82 +++++++ .../test_fortran_export_symbol_selection.py | 36 +++ .../test_ordinary_fortran_codegen_baseline.py | 4 +- .../pipeline/test_wrapper_generator.py | 2 +- .../policy/test_policy_completion.py | 43 ++-- .../general/expected/modern_pyi_example.json | 8 +- .../scope_name_reuse_combinations.json | 19 +- .../test_projected_entrypoint_routes.py | 8 +- .../contracts/fnaming_f90/fnaming_f90.pyi | 2 +- .../pipeline/fixtures/modern_math_physics.pyi | 2 +- .../test_pyi_printer_imports_and_packages.py | 4 +- .../test_scalar_module_variable_lowering.py | 18 +- .../test_module_variables_and_state.py | 19 +- .../test_scalar_module_variable_plan.py | 6 +- .../parsing/test_project_scope_models.py | 41 ++++ .../policy/test_module_variable_policy.py | 8 +- .../codegen/test_optional_lowering.py | 2 +- .../end_to_end/test_scalar_pointers.py | 9 +- .../codegen/test_hidden_scalar_outputs.py | 2 +- 50 files changed, 1152 insertions(+), 229 deletions(-) create mode 100644 tests/fortran/data_types/end_to_end/fixtures/native/flogical_storage_f90.f90 create mode 100644 tests/fortran/data_types/end_to_end/test_logical_scalar_storage.py create mode 100644 tests/fortran/functions/end_to_end/fixtures/native/export_selection_dependencies/leaf_types.f90 create mode 100644 tests/fortran/functions/end_to_end/fixtures/native/export_selection_dependencies/nested_types.f90 diff --git a/CHANGELOG.md b/CHANGELOG.md index 1385e3ee7..9e815ceed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,9 +10,17 @@ release tags add a leading `v` to the package version. - Mutable fixed-storage Fortran module scalars expose native-backed rank-zero NumPy views, including fixed-length character bytes; primitive and fixed character value dummies accept matching rank-zero storage as well as scalar - values, preserving the dummy's reference or `VALUE` ABI. -- Scalar allocatable and pointer module variables return live rank-zero NumPy - views or `None` when storage is absent. + values, preserving the dummy's reference or `VALUE` ABI; an `Immutable` + argument copies the array instead of updating it. Logical scalar + dummies wider than one byte use integer storage of their own width, as + logical arrays do, so default-logical `intent(inout)` updates reach Python. +- Omitting an optional `intent(inout)` scalar argument returns `None` for it. +- Scalar allocatable and pointer module variables return live read-only + rank-zero NumPy views, or `None` when storage is absent. Assigning to the + attribute allocates an allocatable (resizing a deferred-length character) or + writes a pointer's current target. +- A separate module-level `PARAMETER` statement types an undeclared name by the + module's `IMPLICIT` rules and is rejected under `implicit none`. - Contributor test guidance focuses on supported behavior and meaningful validation boundaries after a feature is removed. @@ -27,8 +35,10 @@ release tags add a leading `v` to the package version. - `--export-symbols` and `build_fortran_extension(export_symbols=...)` accept module-qualified Fortran procedures and variables, including symbols re-exported by a public facade. Generated contracts retain the selected - access module through qualified `@bind`, required type declarations, and - native scalar storage views through `T[()]`. + access module through qualified `@bind`, required type declarations + (including the component and parent types they declare), and native scalar + storage views through `T[()]`. Source builds and generated contracts publish + the same selected surface. - The Open MPI `mpi_f08` tutorial and opt-in two-rank integration test build a wrapper from a restricted generated `.pyi` against a matching prebuilt Open MPI installation and exercise NumPy communication and in-place reduction. diff --git a/docs/user/guide/allocatables.md b/docs/user/guide/allocatables.md index e00b09d4f..a75331be3 100644 --- a/docs/user/guide/allocatables.md +++ b/docs/user/guide/allocatables.md @@ -17,8 +17,9 @@ and an array handle gives Python access to that descriptor. ## Key Concepts - Scalar allocatable dummies and results appear as values or `None`. Reading a - scalar allocatable module variable returns a live rank-zero NumPy view or - `None`; array allocatables use `Allocatable[T[...]]` handles. + scalar allocatable module variable returns a live read-only rank-zero NumPy + view or `None`; assigning to it allocates when needed. Array allocatables use + `Allocatable[T[...]]` handles. - An array handle exposes allocation state and descriptor operations; it is not itself a NumPy array. - `allocated` reports whether storage exists; `to_numpy()` returns a live view @@ -84,9 +85,13 @@ The annotation supplies the element dtype and rank. The handle creates its native storage when first passed to a matching writable argument. It stays the same Python object after the call. `Allocatable[Float64]()` is not supported. Reading a scalar module variable -declared `Allocatable[Float64]` returns a live rank-zero `float64` array when -allocated, or `None` otherwise. Read the attribute again after reallocation; -an older view may refer to storage that is no longer valid. +declared `Allocatable[Float64]` returns a live read-only rank-zero `float64` +array when allocated, or `None` otherwise. Assign to the attribute to change +the value: `module.scale = np.float64(2.0)` allocates the variable when it is +unallocated, and a deferred-length character takes the width of the assigned +`str`. Read the attribute again after reallocation; an older view may refer to +storage that is no longer valid. An allocated empty deferred-length character +reads as `b""`. A returned or attribute array handle remains present even when its descriptor is unallocated. Reading the Python attribute diff --git a/docs/user/guide/data-types.md b/docs/user/guide/data-types.md index 62f0218ad..dd86fc174 100644 --- a/docs/user/guide/data-types.md +++ b/docs/user/guide/data-types.md @@ -278,7 +278,9 @@ their own default constructors, described in their later user-guide pages. rank-zero NumPy arrays for numeric scalar arguments. Scalar results use the matching NumPy scalar type. Boolean arguments accept `bool` or `np.bool_`, and Boolean scalar results - are Python `bool` values. + are Python `bool` values. A logical dummy also accepts rank-zero storage with + the dtype its arrays use: `bool` for `logical(c_bool)` and 1-byte kinds, and + `int16`, `int32`, or `int64` for wider kinds. - Plain Python `float` and `int` values raise `TypeError` for numeric scalar arguments. - PRIK resolves kinds using the selected compiler (`gfortran` by default). @@ -296,8 +298,8 @@ def double(value: Float64) -> Float64: ... The wrapper accepts a `numpy.float64` scalar or a rank-zero `float64` array and returns a `numpy.float64`. For a reference dummy, the array supplies its -own storage; a scalar uses call-local storage. A `VALUE` dummy receives a value -from either actual. +own storage; a scalar uses call-local storage. A `VALUE` dummy, or an argument +annotated `Immutable`, receives a copy of the value from either actual. Other primitive result types follow the mapping table above. `T[()]` represents rank-zero NumPy storage: arguments require a 0-D NumPy diff --git a/docs/user/guide/pointers.md b/docs/user/guide/pointers.md index 71b4c86ad..58fe297bf 100644 --- a/docs/user/guide/pointers.md +++ b/docs/user/guide/pointers.md @@ -19,8 +19,9 @@ shape, and strides. It does not by itself say who owns that target. - A pointer descriptor refers to target storage; it does not own that storage by default. - Scalar pointer dummies and results appear as values or `None`. Reading a - scalar pointer module variable returns a live rank-zero NumPy view or `None`; - array pointers use live `Pointer[T[...]]` handles. + scalar pointer module variable returns a live read-only rank-zero NumPy view + or `None`; assigning to it writes the current target. Array pointers use live + `Pointer[T[...]]` handles. - `associated` describes association, not ownership or target lifetime. - NumPy arrays returned by `to_numpy()` are live views, not copies. - Reassociation, resizing, or deallocation can invalidate existing views. @@ -77,8 +78,11 @@ The annotation supplies the element dtype and rank. The handle creates its native storage when first passed to a matching writable argument. It stays the same Python object after the call. `Pointer[Float64]()` is not supported. Reading a scalar module variable -declared `Pointer[Float64]` returns a live rank-zero `float64` array when -associated, or `None` otherwise. Read the attribute again after reassociation; +declared `Pointer[Float64]` returns a live read-only rank-zero `float64` array +when associated, or `None` otherwise. Assigning to the attribute copies the +value into the current target; it raises `ValueError` when the pointer is not +associated, and a character value must encode to the target's width. Assignment +never reassociates the pointer. Read the attribute again after reassociation; an older view may refer to storage that is no longer valid. | Member | Type | Behavior | diff --git a/docs/user/guide/wrapping-modules.md b/docs/user/guide/wrapping-modules.md index e392b0e02..abf6578cc 100644 --- a/docs/user/guide/wrapping-modules.md +++ b/docs/user/guide/wrapping-modules.md @@ -140,8 +140,9 @@ Mutable fixed-storage scalar module variables expose live rank-zero NumPy views. Numeric and logical scalars use `T[()]`; fixed-length character scalars use `String[n][()]` with raw bytes storage. Pass the view itself when a native dummy needs its original storage. `PARAMETER` declarations remain constants. -- Scalar allocatable and pointer module variables return a live rank-zero view - or `None` on each read. Read the attribute again after storage changes. +- Scalar allocatable and pointer module variables return a live read-only + rank-zero view or `None` on each read. Assign to the attribute to change the + value, and read it again after storage changes. - Allocatable module arrays use the `Allocatable[T[...]]` API. - Allocation, lifetime, NumPy views, and mutation rules are covered in the storage and objects section. diff --git a/docs/user/reference/pyi-contracts/calls-and-results.md b/docs/user/reference/pyi-contracts/calls-and-results.md index f585d2655..9068e541f 100644 --- a/docs/user/reference/pyi-contracts/calls-and-results.md +++ b/docs/user/reference/pyi-contracts/calls-and-results.md @@ -196,7 +196,9 @@ def scale( ``` PRIK calls the native procedure with separate writable storage and returns the -replacement. The original array remains unchanged. +replacement. The original array remains unchanged. The same holds for a scalar: +a rank-zero array passed to `Annotated[Int32, Immutable]` is copied in, and only +the returned replacement carries the native update. Do not combine replacement-only mutation with a writable borrowed view. Those requests contradict each other and are rejected. diff --git a/docs/user/reference/python-api.md b/docs/user/reference/python-api.md index 835d91371..0ad914b12 100644 --- a/docs/user/reference/python-api.md +++ b/docs/user/reference/python-api.md @@ -101,7 +101,9 @@ Once you author or generate a semantic `.pyi` contract, that contract's own `build_fortran_extension` accepts the same option with module-qualified native procedure and variable identities. PRIK retains signature and type -dependencies while publishing only the selected symbols: +dependencies while publishing only the selected symbols. A derived type a +selected signature names is published with the component and parent types it +needs, so its values can be constructed from Python: ```python from prik import build_fortran_extension diff --git a/prik/codegen/c/binding.py b/prik/codegen/c/binding.py index 26a79ed32..f16f14a34 100644 --- a/prik/codegen/c/binding.py +++ b/prik/codegen/c/binding.py @@ -20,6 +20,7 @@ render_declaration_extent, ) from prik.policy.ownership import ( + AssignmentMode, CodegenAction, ObjectKind, PythonBarrierAction, @@ -221,7 +222,7 @@ class _COverloadDispatch: ModuleGetterAction.DIRECT_VALUE: "Builds a Python scalar from the current native value.", ModuleGetterAction.NATIVE_SCALAR_VIEW: "Wraps live native scalar storage in a rank-zero NumPy view.", ModuleGetterAction.NATIVE_CHARACTER_VIEW: "Wraps live native character bytes in a rank-zero NumPy view.", - ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW: "Borrows the current native scalar storage, or returns None.", + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW: "Lends the current native scalar storage read-only, or returns None.", ModuleGetterAction.CHARACTER_VALUE: "Decodes the fixed-width native characters into a Python str.", ModuleGetterAction.NULLABLE_SNAPSHOT: "Returns a detached copy, or None when the native value holds nothing.", ModuleGetterAction.BORROWED_ARRAY_VIEW: "Wraps the native storage in a live NumPy array without copying.", @@ -6228,17 +6229,18 @@ def _lower_module_getter_native_character_view(self, plan: ModuleVariablePlan) - ) def _lower_module_getter_native_nullable_scalar_view(self, plan: ModuleVariablePlan) -> tuple[CFunction, ...]: - """Borrow the currently present scalar storage as one rank-zero view.""" + """Lend the currently present scalar storage as one read-only rank-zero view. + + The descriptor may be reallocated or reassociated after this read, so + Python writes through the setter rather than through the view. + """ owner = self._module_native_array_owner_name(plan) character = plan.datatype_family is DatatypeFamily.STRING + getter = self._module_bridge_getter_name(plan) if character: - result = CodeExpression( - "PyArray_New(&PyArray_Type, 0, NULL, NPY_STRING, NULL, data, (int)length, " - "NPY_ARRAY_ALIGNED | NPY_ARRAY_WRITEABLE, NULL)" - ) + numpy_type, width = "NPY_STRING", "(int)length" else: - scalar = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) - result = CodeExpression(f"PyArray_SimpleNewFromData(0, NULL, {scalar.array_numpy_type}, data)") + numpy_type, width = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_numpy_type, "0" return ( CFunction( self._module_getter_name(plan), @@ -6247,45 +6249,63 @@ def _lower_module_getter_native_nullable_scalar_view(self, plan: ModuleVariableP body=( *((CDeclaration("length", "int64_t", CodeExpression("0")),) if character else ()), CDeclaration( - "data", - "void *", - CodeExpression( - f"{self._module_bridge_getter_name(plan)}(&length)" - if character - else f"{self._module_bridge_getter_name(plan)}()" - ), + "data", "void *", CodeExpression(f"{getter}(&length)" if character else f"{getter}()") ), CIf( CodeExpression("data == NULL"), body=(CExpressionStatement(CodeExpression("Py_RETURN_NONE")),), ), - *( - ( - CIf( - CodeExpression("length < 0 || (int64_t)(int)length != length"), - body=( - CExpressionStatement( - CodeExpression( - 'PyErr_SetString(PyExc_OverflowError, "Native character width exceeds NumPy itemsize")' - ) - ), - CReturn(CodeExpression("NULL")), - ), - ), - ) - if character - else () - ), + *(self._nullable_character_view_width_nodes() if character else ()), CDeclaration( "result", "PyObject *", - result, + CodeExpression( + f"PyArray_New(&PyArray_Type, 0, NULL, {numpy_type}, NULL, data, {width}, " + "NPY_ARRAY_ALIGNED, NULL)" + ), ), *self._ordinary_array_field_owner_nodes("result", owner), ), ), ) + @staticmethod + def _nullable_character_view_width_nodes() -> tuple[CIf, ...]: + """Reject an unrepresentable width and return empty text as a detached value. + + NumPy has no zero-width bytes dtype, so an allocated empty character + reads as a read-only ``S1`` value holding ``b""``. + """ + return ( + CIf( + CodeExpression("length < 0 || (int64_t)(int)length != length"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_SetString(PyExc_OverflowError, "Native character width exceeds NumPy itemsize")' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + CIf( + CodeExpression("length == 0"), + body=( + CExpressionStatement( + CodeExpression( + "PyObject *empty = PyArray_New(&PyArray_Type, 0, NULL, NPY_STRING, NULL, NULL, 1, 0, NULL)" + ) + ), + CIf(CodeExpression("empty == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CExpressionStatement(CodeExpression("((char *)PyArray_DATA((PyArrayObject *)empty))[0] = '\\0'")), + CExpressionStatement( + CodeExpression("PyArray_CLEARFLAGS((PyArrayObject *)empty, NPY_ARRAY_WRITEABLE)") + ), + CReturn(CodeExpression("empty")), + ), + ), + ) + def _module_character_length(self, plan: ModuleVariablePlan) -> int: """Return the declared width one character module accessor copies.""" length = plan.character_length @@ -6815,8 +6835,10 @@ def _lower_module_setter(self, plan: ModuleVariablePlan) -> tuple[CFunction, ... def _lower_module_setter_write_through(self, plan: ModuleVariablePlan) -> tuple[CFunction, ...]: """Return a Python-to-native scalar write-through helper.""" - if plan.binding.setter_converts_characters: + if plan.binding.native_assignment is AssignmentMode.CHARACTER_COPY: return self._lower_module_setter_character_value(plan) + if plan.binding.native_assignment in {AssignmentMode.ALLOCATING_COPY, AssignmentMode.TARGET_COPY}: + return self._lower_module_setter_descriptor(plan) scalar_type = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) return ( CFunction( @@ -6836,6 +6858,101 @@ def _lower_module_setter_write_through(self, plan: ModuleVariablePlan) -> tuple[ ), ) + def _lower_module_setter_descriptor(self, plan: ModuleVariablePlan) -> tuple[CFunction, ...]: + """Validate one value and assign it through a scalar allocatable or pointer. + + The bridge reports a disassociated pointer or a character width that + differs from the pointer target's, which become Python exceptions here. + """ + name = plan.owner_path.rsplit(".", 1)[-1] + setter = self._module_bridge_setter_name(plan) + if plan.datatype_family is DatatypeFamily.STRING: + conversion = self._module_setter_text_nodes(plan, name) + call = f"{setter}(value, (int64_t)value_length)" + else: + scalar_type = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) + conversion = ( + CDeclaration("value", scalar_type.c_spelling), + self._module_setter_unpack_statement(plan, scalar_type), + ) + call = f"{setter}(value)" + failures = ( + ( + CIf( + CodeExpression("status == 1"), + body=( + CExpressionStatement( + CodeExpression( + f'PyErr_SetString(PyExc_ValueError, "Module variable {name} has no pointer target")' + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CIf( + CodeExpression("status == 2"), + body=( + CExpressionStatement( + CodeExpression( + f'PyErr_SetString(PyExc_TypeError, "Module variable {name} must encode to ' + "the pointer target's width\")" + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + ) + if plan.binding.native_assignment is AssignmentMode.TARGET_COPY + else () + ) + return ( + CFunction( + self._module_setter_name(plan), + "int", + parameters=(CParameter("value_obj", "PyObject *"),), + storage="static", + body=( + *conversion, + CDeclaration("status", "int", CodeExpression(call)), + *failures, + CReturn(CodeExpression("0")), + ), + ), + ) + + @staticmethod + def _module_setter_text_nodes(plan: ModuleVariablePlan, name: str) -> tuple: + """Encode one Python string, requiring the declared width when there is one.""" + width = plan.character_length + width_check = f"value_length != {width} || " if width is not None else "" + width_text = f"exactly {width} bytes" if width is not None else "bytes" + return ( + CIf( + CodeExpression("!PyUnicode_Check(value_obj)"), + body=( + CExpressionStatement( + CodeExpression(f'PyErr_SetString(PyExc_TypeError, "Expected str for module variable {name}")') + ), + CReturn(CodeExpression("-1")), + ), + ), + CDeclaration("value_length", "Py_ssize_t", CodeExpression("0")), + CDeclaration("value", "const char *", CodeExpression("PyUnicode_AsUTF8AndSize(value_obj, &value_length)")), + CIf(CodeExpression("value == NULL"), body=(CReturn(CodeExpression("-1")),)), + CIf( + CodeExpression(f"{width_check}(Py_ssize_t)strlen(value) != value_length"), + body=( + CExpressionStatement( + CodeExpression( + f'PyErr_SetString(PyExc_TypeError, "Module variable {name} must encode to ' + f'{width_text} without embedded NUL")' + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + ) + def _lower_module_setter_reject_replacement(self, _plan: ModuleVariablePlan) -> tuple[CFunction, ...]: """Read-only descriptor rejection is emitted by module attribute routing.""" return () @@ -7639,9 +7756,14 @@ def _lower_argument_required_scalar_or_storage( ) ) ) + # A Python value unpacks as its canonical C type, then converts to the + # storage the native dummy declares when that spelling differs. + converted = storage_type != scalar_type.c_spelling + unpacked = f"{names.value_name}_converted" if converted else names.value_name return ( CDeclaration(names.object_name, "PyObject *"), CDeclaration(names.value_name, storage_type), + *((CDeclaration(unpacked, scalar_type.c_spelling),) if converted else ()), *((CDeclaration(pointer, f"{storage_type} *", CodeExpression("NULL")),) if reference else ()), CIf( CodeExpression(f"PyArray_Check({names.object_name})"), @@ -7650,7 +7772,7 @@ def _lower_argument_required_scalar_or_storage( self._scalar_exact_unpack_statement( scalar_type, names.object_name, - names.value_name, + unpacked, ( f'PyErr_Format(PyExc_TypeError, "Expected an argument of type ' f"{scalar_type.python_type_name} or rank-zero array for argument " @@ -7659,6 +7781,11 @@ def _lower_argument_required_scalar_or_storage( ), "NULL", ), + *( + (CExpressionStatement(CodeExpression(f"{names.value_name} = ({storage_type}){unpacked}")),) + if converted + else () + ), *((CExpressionStatement(CodeExpression(f"{pointer} = &{names.value_name}")),) if reference else ()), ), ), @@ -13645,7 +13772,15 @@ def _scalar_writeback_value_nodes( CodeExpression(f"{target} == NULL"), body=self._output_failure_nodes(converted, failure_label), ) - if source.entrypoint.descriptor_output_presence_role is None: + if source.entrypoint.descriptor_output_presence_role is not None: + absent = f"!{self._descriptor_output_present_name(names)}" + elif ( + source.binding.scalar_actual_mode is ScalarActualMode.NUMERIC_REFERENCE + and source.binding.optional_mode is not OptionalMode.REQUIRED + ): + # An omitted optional actual lends no storage for the native call to write. + absent = f"{names.value_name}_storage == NULL" + else: return ( CDeclaration(target, "PyObject *", CodeExpression("NULL")), *contract_conversion, @@ -13655,7 +13790,7 @@ def _scalar_writeback_value_nodes( return ( CDeclaration(target, "PyObject *", CodeExpression("NULL")), CIf( - CodeExpression(f"!{self._descriptor_output_present_name(names)}"), + CodeExpression(absent), body=( CExpressionStatement(CodeExpression("Py_INCREF(Py_None)")), CExpressionStatement(CodeExpression(f"{target} = Py_None")), @@ -15061,7 +15196,10 @@ def _scalar_entrypoint_argument_type( passing: EntrypointPassingConvention, ) -> str: """Return the C ABI type for one scalar entrypoint input.""" - scalar_type = PrimitiveScalarTypeRegistry.type_for(argument.semantic_type_name).c_spelling + scalar_type = ( + argument.native_storage_c_type + or PrimitiveScalarTypeRegistry.type_for(argument.semantic_type_name).c_spelling + ) if passing is EntrypointPassingConvention.C_VALUE: return scalar_type if passing in { diff --git a/prik/codegen/docstrings.py b/prik/codegen/docstrings.py index 029af2b48..79c84d355 100644 --- a/prik/codegen/docstrings.py +++ b/prik/codegen/docstrings.py @@ -13,6 +13,7 @@ from prik.policy.ownership import OwnershipOwner, PythonBarrierAction, SetterAction, TransferMode from prik.policy.models import ( ArrayPythonLayout, + ScalarActualMode, ClassConstructorKind, EntrypointOptionalityAction, ModuleGetterAction, @@ -815,14 +816,25 @@ def _output_lines( lines.append(f" {state} state remains inside the returned handle.") if isinstance(output, ArgumentTransferPlan): lines.extend(self._ownership_lines(output.ownership_owner)) - if output.transfer_mode is TransferMode.COPY_RETURN: - lines.append(" Detached replacement; the original Python value is unchanged.") + lines.extend(self._replacement_lines(output)) elif output.datatype_family is DatatypeFamily.DERIVED or output.array is not None: lines.extend(self._ownership_lines(output.ownership_owner)) if nullable and output.native_array_handle is None: lines.append(" May be None.") return tuple(lines) + @staticmethod + def _replacement_lines(output: ArgumentTransferPlan) -> tuple[str, ...]: + """Say whether a returned replacement also updated the caller's storage.""" + if output.transfer_mode is not TransferMode.COPY_RETURN: + return () + if output.binding.scalar_actual_mode in { + ScalarActualMode.NUMERIC_REFERENCE, + ScalarActualMode.CHARACTER_REFERENCE, + }: + return (" Replacement value; a rank-zero array argument is also updated in place.",) + return (" Detached replacement; the original Python value is unchanged.",) + @staticmethod def _optional_lines(argument: ArgumentTransferPlan) -> tuple[str, ...]: """Describe a transfer's completed optional or nullable input contract. diff --git a/prik/codegen/fortran/bridge.py b/prik/codegen/fortran/bridge.py index 80d53360b..d9b60a98d 100644 --- a/prik/codegen/fortran/bridge.py +++ b/prik/codegen/fortran/bridge.py @@ -119,6 +119,8 @@ # address. The binding defines it; the bridge declares and calls it. _MODULE_ARRAY_CAPTURE_NAME = "prik_capture_address" _MODULE_SCALAR_CAPTURE_NAME = "prik_capture_scalar_address" +# A descriptor setter's result: 0 assigned, 1 no pointer target, 2 width differs. +_MODULE_SETTER_STATUS = "prik_setter_status" # The binding answers these from the live descriptor the handle's entry point # supplies, so the bridge emits no procedure of its own for them. @@ -154,6 +156,8 @@ AssignmentMode.NONE: "No native assignment is generated.", AssignmentMode.VALUE_COPY: "Copies the incoming value into the variable.", AssignmentMode.ALIAS: "Points the variable at the incoming storage.", + AssignmentMode.ALLOCATING_COPY: "Assigns the incoming value, allocating the variable when needed.", + AssignmentMode.TARGET_COPY: "Copies the incoming value into the current pointer target.", } @@ -3866,8 +3870,85 @@ def _lower_module_setter(self, plan: ModuleVariablePlan) -> tuple[FortranFunctio return self._lower_module_setter_value_copy(plan) case AssignmentMode.CHARACTER_COPY: return self._lower_module_setter_character_value(plan) + case AssignmentMode.ALLOCATING_COPY: + return self._lower_module_setter_allocating_copy(plan) + case AssignmentMode.TARGET_COPY: + return self._lower_module_setter_target_copy(plan) raise ValueError(f"Unsupported Fortran module setter assignment for {plan.owner_path!r}: {action!r}") + def _lower_module_setter_allocating_copy(self, plan: ModuleVariablePlan) -> tuple[FortranFunction, ...]: + """Assign into a scalar allocatable; intrinsic assignment allocates it when needed.""" + native = self._native_variable_name(plan) + return self._module_descriptor_setter(plan, (FortranAssignment(native, self._module_setter_value(plan)),)) + + def _lower_module_setter_target_copy(self, plan: ModuleVariablePlan) -> tuple[FortranFunction, ...]: + """Copy into a scalar pointer's current target, reporting an absent or narrower target.""" + native = self._native_variable_name(plan) + assignment: FortranAssignment | FortranIf = FortranAssignment(native, self._module_setter_value(plan)) + if plan.datatype_family is DatatypeFamily.STRING: + assignment = FortranIf( + CodeExpression(f"len({native}, kind=c_int64_t) /= length"), + body=(FortranAssignment(_MODULE_SETTER_STATUS, CodeExpression("2_c_int")),), + else_body=(assignment,), + ) + return self._module_descriptor_setter( + plan, + ( + FortranIf( + CodeExpression(f"associated({native})"), + body=(assignment,), + else_body=(FortranAssignment(_MODULE_SETTER_STATUS, CodeExpression("1_c_int")),), + ), + ), + ) + + def _module_descriptor_setter( + self, + plan: ModuleVariablePlan, + body: tuple[FortranAssignment | FortranIf, ...], + ) -> tuple[FortranFunction, ...]: + """Wrap one descriptor assignment in a setter that reports its status. + + A character arrives as an address and a width, since a descriptor + character's width is only known when Python supplies the value. + """ + name = self._module_bridge_setter_name(plan) + declarations: tuple[FortranDeclaration, ...] = () + prologue: tuple[FortranCall, ...] = () + if plan.datatype_family is DatatypeFamily.STRING: + parameters = ( + FortranParameter("value", "type(c_ptr)", ("value",)), + FortranParameter("length", "integer(c_int64_t)", ("value",)), + ) + declarations = (FortranDeclaration("bytes", "character(kind=c_char)", ("pointer", "dimension(:)")),) + prologue = ( + FortranCall( + "c_f_pointer", + (CodeExpression("value"), CodeExpression("bytes"), CodeExpression("[length]")), + ), + ) + else: + scalar_type = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) + parameters = (FortranParameter("value", scalar_type.fortran_spelling, ("value",)),) + return ( + FortranFunction( + name=name, + parameters=parameters, + result_name=_MODULE_SETTER_STATUS, + result_type="integer(c_int)", + bind_name=name, + declarations=declarations, + body=(FortranAssignment(_MODULE_SETTER_STATUS, CodeExpression("0_c_int")), *prologue, *body), + ), + ) + + @staticmethod + def _module_setter_value(plan: ModuleVariablePlan) -> CodeExpression: + """Return the incoming value as the variable's native type.""" + if plan.datatype_family is DatatypeFamily.STRING: + return CodeExpression("transfer(bytes, repeat(' ', int(length)))") + return CodeExpression("value") + def _lower_module_setter_none(self, _plan: ModuleVariablePlan) -> tuple[FortranFunction, ...]: """Return no native setter when the bridge assignment is omitted.""" return () @@ -4310,8 +4391,16 @@ def _lower_argument_descriptor(self, plan: ArgumentTransferPlan) -> tuple[Fortra def _parameter(self, plan: ArgumentTransferPlan, attributes: tuple[str, ...]) -> FortranParameter: """Return one entrypoint ABI parameter from its completed transfer plan.""" - scalar_type = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) - return FortranParameter(plan.entrypoint.parameter_name, scalar_type.fortran_spelling, attributes) + return FortranParameter(plan.entrypoint.parameter_name, self._scalar_argument_type(plan), attributes) + + @staticmethod + def _scalar_argument_type(plan: ArgumentTransferPlan) -> str: + """Spell a scalar dummy, using the native logical kind when policy lends its storage.""" + if plan.scalar_logical_abi is ScalarLogicalABI.NATIVE_KIND_STORAGE: + if not plan.scalar_native_type: + raise ValueError(f"Logical argument {plan.owner_path!r} has no native type spelling") + return plan.scalar_native_type + return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).fortran_spelling def _function_body( self, @@ -5225,7 +5314,7 @@ def _optional_argument_declarations( return self._derived_argument_declarations(argument) scalar_type = PrimitiveScalarTypeRegistry.type_for(argument.semantic_type_name) if mode is OptionalMode.NULLABLE_VALUE: - return (FortranDeclaration(name, scalar_type.fortran_spelling, ("pointer",)),) + return (FortranDeclaration(name, self._scalar_argument_type(argument), ("pointer",)),) declarations = [FortranDeclaration(f"{name}_input", scalar_type.fortran_spelling, ("pointer",))] descriptor_attribute = "pointer" if argument.projected_call_slot.value_kind == "pointer" else "allocatable" declarations.append( @@ -5331,7 +5420,7 @@ def _opaque_argument_type(self, argument: ArgumentTransferPlan) -> str: """Return the typed scalar or derived pointee selected by policy.""" if argument.object_kind is ObjectKind.DERIVED_TYPE: return f"type({self._derived_native_alias(argument.derived.backend_symbol)})" - return PrimitiveScalarTypeRegistry.type_for(argument.semantic_type_name).fortran_spelling + return self._scalar_argument_type(argument) def _opaque_address_initializers( self, diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index 433a12592..fd590ce60 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -415,6 +415,53 @@ class _ProcedureState: explicit_result: bool = False +_IMPLICIT_LETTER_SPEC = re.compile(r"^(?P.+?)\s*\((?P[^()]*)\)$") + + +@dataclass +class _ImplicitTyping: + """Record a module-like scope's IMPLICIT statements as type-spec text by letter.""" + + none: bool = False + type_specs: dict[str, str] = dataclass_field(default_factory=dict) + + def record(self, body: str) -> bool: + """Apply one IMPLICIT statement body, or return False when it is not understood.""" + if re.match(r"^none\b", body, flags=re.IGNORECASE): + self.none = True + return True + mappings = [] + for item in split_csv(body): + match = _IMPLICIT_LETTER_SPEC.match(item.strip()) + letters = self._letters(match.group("letters")) if match is not None else None + if letters is None: + return False + mappings.append((match.group("type").strip(), letters)) + for type_spec, letters in mappings: + self.type_specs.update(dict.fromkeys(letters, type_spec)) + return True + + @staticmethod + def _letters(letter_specs: str) -> list[str] | None: + """Expand ``a-h, o`` style letter specifications, or return None.""" + letters = [] + for letter_spec in split_csv(letter_specs): + bounds = [part.strip().casefold() for part in letter_spec.split("-")] + if len(bounds) not in {1, 2} or not all(len(bound) == 1 and bound.isalpha() for bound in bounds): + return None + letters.extend(chr(code) for code in range(ord(bounds[0]), ord(bounds[-1]) + 1)) + return letters + + def type_spec_for(self, name: str) -> str | None: + """Return the type-spec text implied for ``name``, or None under ``implicit none``.""" + first = name.strip()[:1].casefold() + if first in self.type_specs: + return self.type_specs[first] + if self.none: + return None + return "integer" if "i" <= first <= "n" else "real" + + @dataclass class _ParserScope: """Carry explicit ownership and mutable state while visiting one unit. @@ -422,7 +469,8 @@ class _ParserScope: ``model`` receives parsed declarations, ``parent`` preserves lexical ownership, and procedure visitors attach their temporary :class:`_ProcedureState`. Helpers receive this record explicitly rather - than relying on parser-global scope. + than relying on parser-global scope. ``implicit`` holds a module-like + scope's IMPLICIT statements. """ kind: str @@ -431,6 +479,7 @@ class _ParserScope: parent: _ParserScope | None = None module_owner: str | None = None state: _ProcedureState | None = None + implicit: _ImplicitTyping = dataclass_field(default_factory=_ImplicitTyping) @dataclass(frozen=True) @@ -3501,9 +3550,19 @@ def _parse_module_like_spec_line( target.uses.append(parsed_use) return + implicit = re.match(r"^implicit\b\s*(?P.*)$", stripped, flags=re.IGNORECASE) + if implicit and scope.implicit.record(implicit.group("body")): + return + legacy_parameter = _REGEX["legacy_parameter"].match(stripped) if legacy_parameter: - self._record_scope_legacy_parameters(target, legacy_parameter.group("body")) + self._record_scope_legacy_parameters( + scope, + legacy_parameter.group("body"), + filename=filename, + lineno=lineno, + source_line=source_line, + ) return if _REGEX["derived_type"].match(stripped): @@ -3536,24 +3595,73 @@ def _parse_module_like_spec_line( return self._raise_unsupported_module_like_declaration(target, stripped, filename, lineno, source_line) - @staticmethod - def _record_scope_legacy_parameters(target, assignments: str) -> None: - """Apply a separate PARAMETER statement to its module-like declarations.""" - variables = {variable.name.casefold(): variable for variable in target.variables} + def _record_scope_legacy_parameters( + self, + scope: _ParserScope, + assignments: str, + *, + filename: str | None, + lineno: int | None, + source_line: str | None, + ) -> None: + """Apply a separate PARAMETER statement to its module-like declarations. + + An undeclared name is declared from the scope's IMPLICIT rules through + the ordinary declaration backend, so an implied type-spec keeps its kind. + """ + location = {"filename": filename, "lineno": lineno, "source_line": source_line} for assignment in split_csv(assignments): if "=" not in assignment: continue name, expression = (part.strip() for part in assignment.split("=", 1)) - variable = variables.get(name.casefold()) - if variable is None: - variable = FortranArgument(name=name, base_type=FortranParser._infer_implicit_base_type(name)) - target.variables.append(variable) - variables[name.casefold()] = variable + variable = self._scope_variable(scope.model, name) or self._declare_implicit_parameter( + scope, name, **location + ) variable.is_parameter = True variable.value = FortranParser._normalize_parameter_value(expression) variable.symbolic_value = expression variable.value_type = "expression" + def _declare_implicit_parameter( + self, + scope: _ParserScope, + name: str, + *, + filename: str | None, + lineno: int | None, + source_line: str | None, + ): + """Declare an undeclared PARAMETER name from its scope's implicit type-spec.""" + type_spec = scope.implicit.type_spec_for(name) + owner_kind, owner_name = self._variable_scope_label(scope.model) + owner = f"{owner_kind} '{owner_name or ''}'" + if type_spec is None: + problem = f"Unknown datatype for PARAMETER symbol '{name}' in {owner} (implicit none is active)." + elif self._helper_parse_declaration_line( + f"{type_spec} :: {name}", + scope, + role=self._source_unit_scanner.grammar(scope.kind).declaration_role or "module_variable", + filename=filename, + lineno=lineno, + source_line=source_line, + ): + return self._scope_variable(scope.model, name) + else: + problem = f"Unsupported implicit type '{type_spec}' for PARAMETER symbol '{name}' in {owner}." + raise FortranParseError( + problem, + filename=filename, + line_number=lineno, + source_line=source_line, + code="PARSE_UNKNOWN_PARAMETER_TYPE", + ) + + @staticmethod + def _scope_variable(target, name: str): + """Return the module-like variable declared under ``name``, if any.""" + wanted = name.casefold() + return next((variable for variable in target.variables if variable.name.casefold() == wanted), None) + def _raise_unsupported_openmp_declaration(self, target, line, filename, lineno, source_line) -> None: """Raise the stable diagnostic for an unsupported OpenMP declaration. diff --git a/prik/pipeline/build.py b/prik/pipeline/build.py index 0f095e494..ec5ef2f0a 100644 --- a/prik/pipeline/build.py +++ b/prik/pipeline/build.py @@ -2318,29 +2318,48 @@ def _apply_source_python_exports(modules: list[SemanticModule]) -> None: ), ) - variables_by_identity = { - (module.name.casefold(), str(variable.origin.native_name or variable.name).casefold()): variable + _apply_source_reexport_publications(modules) + + +def _apply_source_reexport_publications(modules: list[SemanticModule]) -> None: + """Publish re-exported variables, and unpublished re-exported types, where they are re-exported. + + A variable gains one publication per re-export, all naming its one native + storage. A type its declaring module leaves unpublished, as export + selection does for context modules, is published where it is re-exported; + otherwise each re-export aliases the declaring publication. + """ + declarations = { + (module.name.casefold(), str(declaration.origin.native_name or declaration.name).casefold()): declaration for module in modules - for variable in module.variables + for declaration in module.variables } + declarations.update( + { + (module.name.casefold(), str(semantic_class.native_name or semantic_class.name).casefold()): semantic_class + for module in modules + for semantic_class in module.classes + if not _declaration_exports(semantic_class) + } + ) for module in modules: for reexport in module.reexports: - if not reexport.publishes_to_python(): - continue - if reexport.entity_kind != "variable": + if not reexport.publishes_to_python() or reexport.entity_kind not in {"variable", "derived_type"}: continue - variable = variables_by_identity.get( + declaration = declarations.get( (str(reexport.origin_module).casefold(), str(reexport.source_name).casefold()) ) - if variable is None: + if declaration is None and reexport.entity_kind == "variable": raise ValueError( f"Cannot resolve re-exported module variable {reexport.origin_module}.{reexport.source_name}" ) + if declaration is None: + continue export = { "namespace": tuple(part.casefold() for part in str(reexport.module).split(".") if part), "name": str(reexport.local_name), } - exports = _declaration_exports(variable) + exports = _declaration_exports(declaration) if export not in exports: exports.append(export) diff --git a/prik/pipeline/wrapper.py b/prik/pipeline/wrapper.py index 453719023..3db24e3d9 100644 --- a/prik/pipeline/wrapper.py +++ b/prik/pipeline/wrapper.py @@ -1408,9 +1408,14 @@ def _module_write_through_setter_diagnostics( ) -> tuple[WrapperPlanDiagnostic, ...]: """Validate one scalar module write-through setter.""" diagnostics = [] - # A character write copies a byte buffer rather than a value, but it is - # the same write-through contract; every other mechanism is rejected. - if plan.bridge.native_assignment not in {AssignmentMode.VALUE_COPY, AssignmentMode.CHARACTER_COPY}: + # A character or descriptor write uses its own native mechanism, but it + # is the same write-through contract; an alias or no assignment is rejected. + if plan.bridge.native_assignment not in { + AssignmentMode.VALUE_COPY, + AssignmentMode.CHARACTER_COPY, + AssignmentMode.ALLOCATING_COPY, + AssignmentMode.TARGET_COPY, + }: diagnostics.append( self._diagnostic(plan.owner_path, "invalid-module-native-assignment", plan.bridge.native_assignment) ) diff --git a/prik/planning/entrypoints.py b/prik/planning/entrypoints.py index a92d51b32..c1e355c2f 100644 --- a/prik/planning/entrypoints.py +++ b/prik/planning/entrypoints.py @@ -28,7 +28,7 @@ NativeArrayOperation, NativeDescriptorHandoffABI, ) -from prik.policy.ownership import ObjectKind, SetterAction +from prik.policy.ownership import AssignmentMode, ObjectKind, SetterAction from .models import ( ArgumentTransferPlan, @@ -1166,27 +1166,40 @@ def _primary_module_variable_operations(self, variable): ) ) if variable.entrypoint.setter_role is not None: - if variable.binding.setter_converts_characters: - value = self._value( - "value", - NativeEntrypointABIValueKind.CHARACTER, - pointer_depth=1, - const=True, - character_length=variable.character_length, - intent="in", - ) - else: - value = self._scalar_parameter(variable.semantic_type_name) operations.append( self._operation( variable.owner_path, "module:set", f"bind_c_set_{variable.symbol_name}", - (value,), + *self._module_setter_signature(variable), ) ) return tuple(operations) + def _module_setter_signature(self, variable): + """Return the setter parameters and result its native assignment requires. + + A descriptor assignment can fail at run time, so it reports a status, + and a descriptor character carries its incoming width beside the bytes. + """ + assignment = variable.binding.native_assignment + if assignment is AssignmentMode.CHARACTER_COPY: + value = self._value( + "value", + NativeEntrypointABIValueKind.CHARACTER, + pointer_depth=1, + const=True, + character_length=variable.character_length, + intent="in", + ) + return (value,), None + if assignment not in {AssignmentMode.ALLOCATING_COPY, AssignmentMode.TARGET_COPY}: + return (self._scalar_parameter(variable.semantic_type_name),), None + if variable.datatype_family is not DatatypeFamily.STRING: + return (self._scalar_parameter(variable.semantic_type_name),), self._int_result() + value = self._value("value", NativeEntrypointABIValueKind.OPAQUE, pointer_depth=1, const=True) + return (value, self._int64_parameter("length")), self._int_result() + def _module_native_array_operations(self, variable): handle = variable.native_array_handle if handle is None or handle.array.rank is None: diff --git a/prik/planning/models.py b/prik/planning/models.py index 511a4fd8c..6700e54cc 100644 --- a/prik/planning/models.py +++ b/prik/planning/models.py @@ -775,7 +775,8 @@ class BindingModuleVariablePlan(StageRecord): setter_action: SetterAction initializer: Any constant_value: Any - setter_converts_characters: bool = False + # The completed native assignment selects the setter's value transport. + native_assignment: AssignmentMode = AssignmentMode.NONE @dataclass diff --git a/prik/planning/planner.py b/prik/planning/planner.py index c8c02d135..02daa82f6 100644 --- a/prik/planning/planner.py +++ b/prik/planning/planner.py @@ -73,7 +73,7 @@ ) from prik.naming.generated_files import bridge_source_name from prik.policy.exports import PythonExportPolicy -from prik.policy.ownership import AssignmentMode, NativeBarrierAction, SetterAction +from prik.policy.ownership import NativeBarrierAction, SetterAction from prik.planning.models import ( ArrayHandoffPlan, ArgumentTransferPlan, @@ -1386,7 +1386,7 @@ def _module_variable_plan( setter_action=policy.setter_action, initializer=policy.initializer, constant_value=policy.constant_value, - setter_converts_characters=policy.native_assignment is AssignmentMode.CHARACTER_COPY, + native_assignment=policy.native_assignment, ), entrypoint=NativeEntrypointModuleVariablePlan( descriptor_kind=policy.descriptor_kind, diff --git a/prik/policy/construction.py b/prik/policy/construction.py index c164d88e6..086d78776 100644 --- a/prik/policy/construction.py +++ b/prik/policy/construction.py @@ -2833,7 +2833,7 @@ def _direct_argument_ineligibility(argument: ArgumentPolicy) -> tuple[str, ...]: reasons.append(f"argument {argument.name!r} requires a specialized native handoff") if argument.transformations: reasons.append(f"argument {argument.name!r} requires representation transformation") - if argument.scalar_logical_abi is ScalarLogicalABI.NATIVE_KIND_COPY: + if argument.scalar_logical_abi in {ScalarLogicalABI.NATIVE_KIND_COPY, ScalarLogicalABI.NATIVE_KIND_STORAGE}: reasons.append(f"argument {argument.name!r} uses non-C Boolean storage") if argument.entrypoint_passing is EntrypointPassingConvention.BLOCKED: reasons.append(f"argument {argument.name!r} has no completed C passing convention") @@ -3199,6 +3199,7 @@ def _argument_policy( rank=int(argument.semantic_type.rank or 0), scalar_logical_abi=scalar_logical_abi, scalar_native_type=scalar_native_type, + native_storage_c_type=_logical_storage_c_type(scalar_logical_abi, argument), array_logical_abi=array_logical_abi, array_native_type=array_native_type, optional=argument.optional, @@ -3210,7 +3211,13 @@ def _argument_policy( nullable=boundary.nullable, writable=boundary.writable, descriptor_boundary=boundary.descriptor_boundary, - scalar_actual_mode=_scalar_actual_mode(argument, boundary, entrypoint_passing), + scalar_actual_mode=_scalar_actual_mode( + argument, + boundary, + entrypoint_passing, + native_slot, + scalar_logical_abi, + ), scalar_storage_writable=decision.mutates_native, ownership=decision, codegen_action=boundary.codegen_action, @@ -3272,27 +3279,34 @@ def _scalar_actual_mode( argument: models.SemanticArgument, boundary: _ArgumentBoundaryPolicy, passing: EntrypointPassingConvention, + slot: NativeCallSlotPolicy | None, + logical_abi: ScalarLogicalABI, ) -> ScalarActualMode | None: - """Complete dual scalar/value-or-storage input acceptance before planning.""" - if int(argument.semantic_type.rank or 0) != 0: - return None - if boundary.descriptor_boundary: + """Complete dual scalar/value-or-storage input acceptance before planning. + + The dummy's declared ``VALUE`` attribute selects the transport, as it does + for a ``bind(C)`` procedure called directly: a bridge that receives the + value by reference still hands the native procedure a copy, so the caller's + storage is never borrowed for it. An ``Immutable`` value is copied the same + way, so the native update reaches Python only as the replacement result. A + logical copied through ``c_bool`` has no storage of its native width to + lend, so it accepts values only. + """ + if int(argument.semantic_type.rank or 0) != 0 or boundary.descriptor_boundary: return None + by_value = _argument_passes_by_value(argument, slot) or ( + argument.semantic_type.metadata.get(models.PYTHON_VALUE_MUTABILITY_METADATA) == models.PYTHON_VALUE_IMMUTABLE + ) if boundary.python_barrier_action is PythonBarrierAction.SCALAR_VALUE: - if passing is EntrypointPassingConvention.C_VALUE: + if logical_abi is ScalarLogicalABI.NATIVE_KIND_COPY: + return None + if by_value or passing is EntrypointPassingConvention.C_VALUE: return ScalarActualMode.NUMERIC_VALUE - if passing is EntrypointPassingConvention.POINTER_REFERENCE: + if passing in {EntrypointPassingConvention.POINTER_REFERENCE, EntrypointPassingConvention.NULLABLE_POINTER}: return ScalarActualMode.NUMERIC_REFERENCE - if passing is EntrypointPassingConvention.NULLABLE_POINTER: - return ( - ScalarActualMode.NUMERIC_VALUE - if argument.origin.metadata.get("value") - else ScalarActualMode.NUMERIC_REFERENCE - ) + return None if boundary.python_barrier_action is PythonBarrierAction.STRING_VALUE and _character_length(argument.semantic_type): - if _native_by_value_argument(argument): - return ScalarActualMode.CHARACTER_VALUE - return ScalarActualMode.CHARACTER_REFERENCE + return ScalarActualMode.CHARACTER_VALUE if by_value else ScalarActualMode.CHARACTER_REFERENCE return None @@ -3760,6 +3774,7 @@ def _hidden_result_candidate( argument, bridge_data_action, bridge_copy_reason, + hidden_result=True, ) if bridge_data_action is BridgeDataAction.BLOCKED and decision.kind is not ObjectKind.SCALAR: blockers = (*blockers, f"{label} has no completed bridge data action") @@ -4229,8 +4244,9 @@ def _hidden_result_native_call_slot_policy( argument, bridge_data_action, bridge_copy_reason, + hidden_result=True, ) - scalar_logical_abi, scalar_native_type = _scalar_logical_argument_abi(argument) + scalar_logical_abi, scalar_native_type = _scalar_logical_argument_abi(argument, hidden_result=True) array_logical_abi, array_native_type = _array_logical_argument_abi(argument) blockers = ( (f"native-call result slot {native_position} has no completed bridge data action",) @@ -7096,6 +7112,12 @@ def _scalar_module_initializer_blockers( return tuple(blockers) +_SCALAR_MODULE_DESCRIPTOR_ASSIGNMENTS = { + "allocatable": {AssignmentMode.ALLOCATING_COPY}, + "pointer": {AssignmentMode.TARGET_COPY}, +} + + def _scalar_module_setter_blockers( setter: OwnershipDecision, descriptor_kind: str | None, @@ -7107,8 +7129,12 @@ def _scalar_module_setter_blockers( return ("scalar constant must omit native setter assignment",) return () if setter.setter_action is SetterAction.WRITE_THROUGH: - if setter.assignment_mode not in {AssignmentMode.VALUE_COPY, AssignmentMode.CHARACTER_COPY}: - return ("write-through scalar setter requires value-copy native assignment",) + expected_assignments = _SCALAR_MODULE_DESCRIPTOR_ASSIGNMENTS.get( + descriptor_kind, + {AssignmentMode.VALUE_COPY, AssignmentMode.CHARACTER_COPY}, + ) + if setter.assignment_mode not in expected_assignments: + return (f"write-through scalar setter cannot use {setter.assignment_mode.value!r} native assignment",) expected_python_action = ( PythonBarrierAction.STRING_VALUE if setter.kind is ObjectKind.STRING else PythonBarrierAction.SCALAR_VALUE ) @@ -7426,10 +7452,21 @@ def _fortran_logical_native_type(argument: models.SemanticArgument) -> str | Non return source_type +# Integer storage of each wider logical's own width, as its NumPy arrays use. +_LOGICAL_STORAGE_C_TYPES = {"Bool16": "int16_t", "Bool32": "int32_t", "Bool64": "int64_t"} + + def _scalar_logical_argument_abi( argument: models.SemanticArgument, + *, + hidden_result: bool = False, ) -> tuple[ScalarLogicalABI, str | None]: - """Complete exact native-kind storage for one Fortran logical scalar.""" + """Complete exact native-kind storage for one Fortran logical scalar. + + A visible dummy wider than ``c_bool`` receives integer storage of its own + width, like a logical array element, so nothing is copied. A hidden + result converts through ``c_bool`` instead. + """ semantic_type = argument.semantic_type if not is_boolean_semantic_type_name(semantic_type.name) or int(semantic_type.rank or 0) != 0: return ScalarLogicalABI.NOT_APPLICABLE, None @@ -7438,15 +7475,21 @@ def _scalar_logical_argument_abi( if semantic_type.name in {"Bool", "Bool8"}: return ScalarLogicalABI.C_BOOL, "logical(c_bool)" native_kind = {"Bool16": 2, "Bool32": 4, "Bool64": 8}.get(semantic_type.name) - return ( - (ScalarLogicalABI.NATIVE_KIND_COPY, f"logical(kind={native_kind})") - if native_kind is not None - else (ScalarLogicalABI.NATIVE_KIND_COPY, None) - ) - compact = "".join(source_type.casefold().split()) - if compact == "logical(kind=c_bool)": + source_type = f"logical(kind={native_kind})" if native_kind is not None else None + elif "".join(source_type.casefold().split()) == "logical(kind=c_bool)": return ScalarLogicalABI.C_BOOL, "logical(c_bool)" - return ScalarLogicalABI.NATIVE_KIND_COPY, source_type + borrows_storage = source_type is not None and semantic_type.name in _LOGICAL_STORAGE_C_TYPES and not hidden_result + return ( + ScalarLogicalABI.NATIVE_KIND_STORAGE if borrows_storage else ScalarLogicalABI.NATIVE_KIND_COPY, + source_type, + ) + + +def _logical_storage_c_type(abi: ScalarLogicalABI, argument: models.SemanticArgument) -> str | None: + """Return the integer storage a native-kind logical dummy borrows, if any.""" + if abi is not ScalarLogicalABI.NATIVE_KIND_STORAGE: + return None + return _LOGICAL_STORAGE_C_TYPES[argument.semantic_type.name] def _array_logical_argument_abi( @@ -7469,9 +7512,11 @@ def _logical_argument_bridge_action( argument: models.SemanticArgument, action: BridgeDataAction, reason: str | None, + *, + hidden_result: bool = False, ) -> tuple[BridgeDataAction, str | None]: """Select explicit representation copying for a non-C logical argument.""" - abi, _native_type = _scalar_logical_argument_abi(argument) + abi, _native_type = _scalar_logical_argument_abi(argument, hidden_result=hidden_result) if abi is ScalarLogicalABI.NATIVE_KIND_COPY: return BridgeDataAction.COPY_REPRESENTATION, LOGICAL_SCALAR_KIND_COPY_REASON return action, reason diff --git a/prik/policy/models.py b/prik/policy/models.py index 33ccafdf3..0bde49d44 100644 --- a/prik/policy/models.py +++ b/prik/policy/models.py @@ -217,10 +217,17 @@ class DirectResultABI(str, Enum): class ScalarLogicalABI(str, Enum): - """Completed scalar logical adaptation between the C and native dummies.""" + """Completed scalar logical adaptation between the C and native dummies. + + ``NATIVE_KIND_STORAGE`` passes integer storage of the logical's own width + straight to the dummy, as a logical array does, so nothing is copied. + ``NATIVE_KIND_COPY`` converts through ``c_bool``, for a hidden result or a + logical whose width no compiler probe established. + """ NOT_APPLICABLE = "not_applicable" C_BOOL = "c_bool" + NATIVE_KIND_STORAGE = "native_kind_storage" NATIVE_KIND_COPY = "native_kind_copy" diff --git a/prik/policy/ownership.py b/prik/policy/ownership.py index 969b66f07..01075767b 100644 --- a/prik/policy/ownership.py +++ b/prik/policy/ownership.py @@ -269,12 +269,19 @@ class AssignmentMode(str, Enum): incoming fixed-width byte buffer into existing native character storage, which has no by-value C ABI. ``ALIAS`` associates the destination with existing storage rather than copying it. + ``ALLOCATING_COPY`` assigns into a scalar allocatable, allocating it + when unallocated and giving a deferred-length character the incoming + width. ``TARGET_COPY`` copies into a scalar pointer's current target + and fails when the pointer is disassociated or, for a character, when + the incoming width differs from the target's. """ NONE = "none" VALUE_COPY = "value_copy" CHARACTER_COPY = "character_copy" ALIAS = "alias" + ALLOCATING_COPY = "allocating_copy" + TARGET_COPY = "target_copy" class SetterAction(str, Enum): @@ -1004,6 +1011,13 @@ def decide_semantic_setter( setter_action=SetterAction.OMIT, ) incoming = self.decide_semantic_type(variable.semantic_type, OwnershipContext.argument()) + descriptor_assignment = self._module_scalar_descriptor_assignment(storage, context, variable) + if descriptor_assignment is not None: + return replace( + incoming, + assignment_mode=descriptor_assignment, + setter_action=SetterAction.WRITE_THROUGH, + ) return replace( incoming, assignment_mode=( @@ -1016,6 +1030,27 @@ def decide_semantic_setter( setter_action=self._setter_action(storage, incoming, context, variable), ) + @staticmethod + def _module_scalar_descriptor_assignment( + storage: OwnershipDecision, + context: OwnershipContext, + variable: Any, + ) -> AssignmentMode | None: + """Select how a scalar allocatable or pointer module variable is assigned. + + Its getter lends a read-only view of the current storage, so Python + writes only through the setter: an allocatable takes intrinsic + assignment, while a pointer's current target receives the value. + """ + if not context.is_module_variable or storage.kind not in {ObjectKind.SCALAR, ObjectKind.STRING}: + return None + metadata = variable.semantic_type.metadata + if metadata.get("fortran_allocatable"): + return AssignmentMode.ALLOCATING_COPY + if metadata.get("fortran_pointer"): + return AssignmentMode.TARGET_COPY + return None + @staticmethod def _setter_action( storage: OwnershipDecision, @@ -1030,23 +1065,9 @@ def _setter_action( ownership contract. """ if storage.kind is ObjectKind.SCALAR: - if context.is_module_variable and ( - variable.semantic_type.metadata.get("fortran_allocatable") - or variable.semantic_type.metadata.get("fortran_pointer") - ): - return SetterAction.REJECT_REPLACEMENT if storage.transfer is TransferMode.SNAPSHOT_COPY and storage.nullable: return SetterAction.REJECT_REPLACEMENT return SetterAction.WRITE_THROUGH - if ( - storage.kind is ObjectKind.STRING - and context.is_module_variable - and ( - variable.semantic_type.metadata.get("fortran_allocatable") - or variable.semantic_type.metadata.get("fortran_pointer") - ) - ): - return SetterAction.REJECT_REPLACEMENT if storage.kind is ObjectKind.STRING and context.is_field: return SetterAction.WRITE_THROUGH # A character module variable is written through the same fixed-width @@ -1892,7 +1913,7 @@ def _module_variable_decision(self, facts: _StorageFacts, context: OwnershipCont boundary_storage_mode=StorageMode.ALIAS, nullable=True, borrowed=True, - reason="scalar module descriptor supplies a current native view on each read", + reason="scalar module descriptor lends a read-only view of its current storage on each read", ) if facts.rank > 0 or facts.is_ndarray: if facts.pointer: diff --git a/prik/semantics/fortran_exports.py b/prik/semantics/fortran_exports.py index 9e13b5cb2..c6569b928 100644 --- a/prik/semantics/fortran_exports.py +++ b/prik/semantics/fortran_exports.py @@ -77,7 +77,7 @@ def select_fortran_export_symbols( primary_sources.append(module) primary_modules.append(selected_module) - required_types = _required_type_identities(primary_modules, selected) + required_types = _with_dependent_types(_required_type_identities(primary_modules, selected), module_index) for module in primary_modules: _retain_required_types(module, module_index[_native_module_name(module)], required_types) @@ -209,14 +209,61 @@ def _required_type_identities(modules: list[SemanticModule], selected: set[tuple ) ) for semantic_type in types: - for item in _semantic_type_tree(semantic_type): - reference = item.metadata.get(EXTERNAL_TYPE_REF_METADATA) - origin = reference.get("origin_module") if isinstance(reference, dict) else module_name - name = reference.get("name") if isinstance(reference, dict) else item.name - required.add((str(origin).casefold(), str(name).casefold())) + required.update(_type_identities(semantic_type, module_name)) return required +def _type_identities(semantic_type, module_name: str) -> set[tuple[str, str]]: + """Return the declaring identities one semantic type names, including callback types.""" + identities = set() + for item in _semantic_type_tree(semantic_type): + reference = item.metadata.get(EXTERNAL_TYPE_REF_METADATA) + origin = reference.get("origin_module") if isinstance(reference, dict) else module_name + name = reference.get("name") if isinstance(reference, dict) else item.name + identities.add((str(origin).casefold(), str(name).casefold())) + return identities + + +def _with_dependent_types(required: set[tuple[str, str]], module_index) -> set[tuple[str, str]]: + """Close required derived types over the component and parent types they declare. + + A published type is usable only with the types its components and parent + name, so each of those is retained wherever it is declared. + """ + closed: set[tuple[str, str]] = set() + pending = list(required) + while pending: + identity = pending.pop() + if identity in closed: + continue + closed.add(identity) + module = module_index.get(identity[0]) + if module is None: + continue + declaration = next((cls for cls in module.classes if _native_symbol_name(cls) == identity[1]), None) + if declaration is None: + pending.extend( + (reexport.origin_module.casefold(), reexport.source_name.casefold()) + for reexport in module.reexports + if reexport.entity_kind == "derived_type" and reexport.local_name.casefold() == identity[1] + ) + continue + for component in declaration.fields: + pending.extend(_type_identities(component.semantic_type, identity[0])) + pending.extend(_named_type_identity(module, base) for base in declaration.base_classes) + return closed + + +def _named_type_identity(module: SemanticModule, name: str) -> tuple[str, str]: + """Resolve a type name written in ``module`` to its local or use-associated declaration.""" + wanted = name.casefold() + for semantic_import in module.imports: + for item in semantic_import.items: + if (item.target or item.source).casefold() == wanted: + return semantic_import.module.casefold(), item.source.casefold() + return _native_module_name(module), wanted + + def _validated_fortran_export_symbols(symbols: Iterable[str]) -> tuple[tuple[str, str], ...]: requested_text = tuple(str(symbol) for symbol in symbols) if not requested_text: diff --git a/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py b/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py index 022fba66b..704c77531 100644 --- a/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py +++ b/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py @@ -50,8 +50,9 @@ def update(value: Int64) -> Returns["value", Int64]: ... assert "void update(long long * value);" in binding assert "long long bound_value;" in binding assert "bound_value = (long long)bound_value_converted;" in binding - assert "update(&bound_value);" in binding - assert "int64_t bound_value_contract = (int64_t)bound_value;" in binding + assert "bound_value_storage = &bound_value;" in binding + assert "update(bound_value_storage);" in binding + assert "int64_t bound_value_contract = (int64_t)*bound_value_storage;" in binding assert "prik_int64_to_numpy(&bound_value_contract)" in binding diff --git a/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py b/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py index 28f31a0b3..0e8891d8d 100644 --- a/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py +++ b/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py @@ -166,7 +166,7 @@ def combine(scale: Float64, count: Int32) -> Float64: ... assert module.combine(np.float64(2.5), np.int32(4)) == np.float64(10.0) assert "double combine(int32_t count, double scale);" in binding - with pytest.raises(TypeError, match=r"numpy\.float64 for argument scale"): + with pytest.raises(TypeError, match=r"numpy\.float64 or rank-zero array for argument scale"): module.combine(np.int32(4), np.int32(4)) diff --git a/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py b/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py index 20bacd982..d7e1e9bc0 100644 --- a/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py +++ b/tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py @@ -33,17 +33,20 @@ def test_scalar_allocatables_project_values_and_unallocated_state( assert module.optional_scale is None assert not hasattr(module, "get_optional_scale") assert not hasattr(module, "set_optional_scale") - with pytest.raises(AttributeError): - module.optional_scale = np.float64(9.0) - module.set_module_value(np.float64(1.5)) + # Assignment allocates the unallocated variable; the view it reads is live but read-only. + module.optional_scale = np.float64(2.5) view = module.optional_scale assert view is not None and view.shape == () and view.dtype == np.dtype("float64") - assert view[()] == np.float64(1.5) - view[()] = np.float64(2.5) + assert view[()] == np.float64(2.5) + with pytest.raises(ValueError, match="read-only"): + view[()] = np.float64(3.5) module.bump_module_value() assert view[()] == np.float64(12.5) - assert module.optional_scale[()] == np.float64(12.5) + + # Native reallocation replaces the storage, so the attribute is read again. + module.set_module_value(np.float64(1.5)) + assert module.optional_scale[()] == np.float64(1.5) assert module.echo_allocatable(np.float64(3.0)) == np.float64(4.0) assert module.echo_allocatable(None) == np.float64(-1.0) diff --git a/tests/fortran/data_types/end_to_end/fixtures/native/flogical_storage_f90.f90 b/tests/fortran/data_types/end_to_end/fixtures/native/flogical_storage_f90.f90 new file mode 100644 index 000000000..dfde4fd7e --- /dev/null +++ b/tests/fortran/data_types/end_to_end/fixtures/native/flogical_storage_f90.f90 @@ -0,0 +1,29 @@ +module flogical_storage_f90 + use iso_c_binding, only: c_bool + implicit none +contains + subroutine flip(flag) + logical, intent(inout) :: flag + flag = .not. flag + end subroutine flip + + subroutine flip_wide(flag) + logical(8), intent(inout) :: flag + flag = .not. flag + end subroutine flip_wide + + subroutine flip_c_bool(flag) + logical(c_bool), intent(inout) :: flag + flag = .not. flag + end subroutine flip_c_bool + + subroutine maybe_flip(flag) + logical, intent(inout), optional :: flag + if (present(flag)) flag = .not. flag + end subroutine maybe_flip + + integer function count_value(flag) + logical, value, intent(in) :: flag + count_value = merge(1, 0, flag) + end function count_value +end module flogical_storage_f90 diff --git a/tests/fortran/data_types/end_to_end/test_logical_scalar_storage.py b/tests/fortran/data_types/end_to_end/test_logical_scalar_storage.py new file mode 100644 index 000000000..68d7bbdf5 --- /dev/null +++ b/tests/fortran/data_types/end_to_end/test_logical_scalar_storage.py @@ -0,0 +1,55 @@ +"""Logical scalar dummies share the storage width their logical arrays use.""" + +from pathlib import Path + +import numpy as np +import pytest + +from tests.fortran._support.wrapper_build import _build_source_or_generated_pyi_and_import + +SOURCE = Path(__file__).parent / "fixtures" / "native" / "flogical_storage_f90.f90" +pytestmark = pytest.mark.fortran_end_to_end + + +def test_logical_reference_dummies_borrow_native_width_storage(pyi_parity_build_mode: str, tmp_path: Path): + """A wider logical crosses as integer storage of its own width, with no copy.""" + module = _build_source_or_generated_pyi_and_import( + SOURCE, + tmp_path, + { + "bind_c_flogical_storage_f90_wrapper.f90", + "flogical_storage_f90_wrapper.c", + "flogical_storage_f90_wrapper.h", + }, + None, + pyi_parity_build_mode, + ) + + # A Python bool uses call-local storage and returns the native update. + assert module.flip(True) is False + assert module.flip(np.bool_(False)) is True + + # Rank-zero storage of the logical's own width is updated in place. + flag = np.array(0, dtype=np.int32) + assert module.flip(flag) is True + assert flag[()] == 1 + wide = np.array(1, dtype=np.int64) + assert module.flip_wide(wide) is False + assert wide[()] == 0 + with pytest.raises(TypeError, match="int32"): + module.flip(np.array(True)) + + # A c_bool logical borrows bool storage. + c_flag = np.array(True) + assert module.flip_c_bool(c_flag) is False + assert not c_flag[()] + + # An omitted optional lends no storage, so nothing is returned. + assert module.maybe_flip() is None + assert module.maybe_flip(False) is True + optional_flag = np.array(1, dtype=np.int32) + assert module.maybe_flip(optional_flag) is False + assert optional_flag[()] == 0 + + assert module.count_value(True) == np.int32(1) + assert module.count_value(np.array(1, dtype=np.int32)) == np.int32(1) diff --git a/tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py b/tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py index 1cbb1d358..41e9a8e8d 100644 --- a/tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py +++ b/tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py @@ -54,7 +54,7 @@ def hidden_storage_result() -> Int32[()]: ... assert native_counter.shape == () assert native_counter[()] == np.int32(3) assert module.bump_value(native_counter) == np.int32(4) - assert module.counter[()] == np.int32(4) + assert module.counter[()] == np.int32(3) module.counter = np.int32(9) assert native_counter[()] == np.int32(9) assert module.answer == np.int32(42) @@ -62,12 +62,12 @@ def hidden_storage_result() -> Int32[()]: ... original = np.int32(4) assert module.bump_value(original) == np.int32(5) assert original == np.int32(4) + # An Immutable value copies rank-zero storage in; only the replacement changes. borrowed = np.array(4, dtype=np.int32) assert module.bump_value(borrowed) == np.int32(5) - assert borrowed[()] == np.int32(5) + assert borrowed[()] == np.int32(4) borrowed.flags.writeable = False - with pytest.raises(TypeError, match="writeable"): - module.bump_value(borrowed) + assert module.bump_value(borrowed) == np.int32(5) storage = np.array(6, dtype=np.int32) assert module.bump_storage(storage) is None diff --git a/tests/fortran/error_handling/codegen/test_status_error_lowering.py b/tests/fortran/error_handling/codegen/test_status_error_lowering.py index 44e65703f..ded967939 100644 --- a/tests/fortran/error_handling/codegen/test_status_error_lowering.py +++ b/tests/fortran/error_handling/codegen/test_status_error_lowering.py @@ -99,8 +99,8 @@ def test_direct_binding_lowering_places_only_opted_in_native_call_outside_the_gi assert released.index("bind_c_pause_for_one_second()") < released.index("Py_END_ALLOW_THREADS") assert "Py_BEGIN_ALLOW_THREADS" not in held assert "Py_END_ALLOW_THREADS" not in held - assert solve.index("Py_BEGIN_ALLOW_THREADS") < solve.index("bind_c_solve(&bound_value, &status, &message)") - assert solve.index("bind_c_solve(&bound_value, &status, &message)") < solve.index("Py_END_ALLOW_THREADS") + assert solve.index("Py_BEGIN_ALLOW_THREADS") < solve.index("bind_c_solve(bound_value_storage, &status, &message)") + assert solve.index("bind_c_solve(bound_value_storage, &status, &message)") < solve.index("Py_END_ALLOW_THREADS") assert solve.index("Py_END_ALLOW_THREADS") < solve.index("prik_status_message_text") assert solve.index("prik_status_message_text") < solve.index("status != 0") assert "PyErr_SetObject(PyExc_RuntimeError, message_obj)" in solve diff --git a/tests/fortran/functions/codegen/test_multiple_function_results.py b/tests/fortran/functions/codegen/test_multiple_function_results.py index 10c67a2fd..d0ac0cb41 100644 --- a/tests/fortran/functions/codegen/test_multiple_function_results.py +++ b/tests/fortran/functions/codegen/test_multiple_function_results.py @@ -64,7 +64,7 @@ def test_multiple_scalar_results_lower_to_binding_tuple_and_one_bridge_function_ bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") assert "int32_t bind_c_with_scalar(int32_t * n, int32_t * status);" in c_source - assert "result = bind_c_with_scalar(&bound_n, &status);" in c_source + assert "result = bind_c_with_scalar(bound_n_storage, &status);" in c_source assert "PyObject * result_0_obj = prik_int32_to_numpy(&result);" in c_source assert "PyObject * result_1_obj = prik_int32_to_numpy(&status);" in c_source assert "PyObject * result_obj = PyTuple_New(2);" in c_source diff --git a/tests/fortran/functions/end_to_end/fixtures/native/export_selection_dependencies/leaf_types.f90 b/tests/fortran/functions/end_to_end/fixtures/native/export_selection_dependencies/leaf_types.f90 new file mode 100644 index 000000000..88ef5daf0 --- /dev/null +++ b/tests/fortran/functions/end_to_end/fixtures/native/export_selection_dependencies/leaf_types.f90 @@ -0,0 +1,6 @@ +module leaf_types + implicit none + type :: leaf_t + integer :: v = 7 + end type leaf_t +end module leaf_types diff --git a/tests/fortran/functions/end_to_end/fixtures/native/export_selection_dependencies/nested_types.f90 b/tests/fortran/functions/end_to_end/fixtures/native/export_selection_dependencies/nested_types.f90 new file mode 100644 index 000000000..f129630f8 --- /dev/null +++ b/tests/fortran/functions/end_to_end/fixtures/native/export_selection_dependencies/nested_types.f90 @@ -0,0 +1,29 @@ +module nested_types + use leaf_types, only: leaf_t + implicit none + type :: base_t + integer :: b = 1 + end type base_t + + type :: inner_t + type(leaf_t) :: leaf + integer :: a = 2 + end type inner_t + + type, extends(base_t) :: outer_t + type(inner_t) :: inner + end type outer_t + + type :: unrelated_t + integer :: u = 0 + end type unrelated_t +contains + subroutine use_outer(x, total) + type(outer_t), intent(in) :: x + integer, intent(out) :: total + total = x%b + x%inner%a + x%inner%leaf%v + end subroutine use_outer + + subroutine unrelated() + end subroutine unrelated +end module nested_types diff --git a/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py b/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py index 1452835ff..e3cbd23f4 100644 --- a/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py +++ b/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py @@ -182,3 +182,71 @@ def test_facade_selection_and_contract_replay_share_generic_and_native_variable( assert isinstance(module.marker, np.ndarray) and module.marker.shape == () module.marker[()] = np.int32(11) assert module.run() == np.int32(11) + + +@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") +def test_selection_keeps_the_component_and_parent_types_a_selected_signature_needs(tmp_path: Path): + """A selected type is usable in both lanes: its parent and component types come with it.""" + sources = tuple( + NATIVE.parent / "export_selection_dependencies" / name for name in ("leaf_types.f90", "nested_types.f90") + ) + exports = tmp_path / "exports.txt" + exports.write_text("nested_types::use_outer\n", encoding="utf-8") + contract = tmp_path / "contract" + subprocess.run( + [ + sys.executable, + "-m", + "prik", + "generate", + "--pyi", + *map(str, sources), + "--export-symbols", + str(exports), + "--out", + str(contract), + ], + check=True, + capture_output=True, + text=True, + ) + nested_contract = (contract / "nested_types.pyi").read_text(encoding="utf-8") + assert "class Outer_T(Base_T):" in nested_contract + assert "inner: Inner_T" in nested_contract + assert "leaf: Leaf_T" in nested_contract + assert "Unrelated" not in nested_contract and "def unrelated(" not in nested_contract + + source = build_fortran_extension( + sources, + output_name="dependencies_source", + output_dir=tmp_path / "source", + export_symbols=("nested_types::use_outer",), + jobs=2, + ) + replay = build_pyi_extension( + contract / "__init__.pyi", + native_fortran_sources=sources, + output_name="dependencies_replay", + output_dir=tmp_path / "replay", + jobs=2, + ) + published = [] + for result in (source, replay): + root = _import_from_build_dir(result.module_name, result.output_dir) + module = root.nested_types + published.append( + ( + {name for name in dir(root) if not name.startswith("_")}, + {name for name in dir(module) if not name.startswith("_")}, + ) + ) + assert module.use_outer(module.Outer_T()) == np.int32(10) + assert module.Outer_T().inner.leaf.v == np.int32(7) + assert ( + published[0] + == published[1] + == ( + {"nested_types"}, + {"Base_T", "Inner_T", "Leaf_T", "Outer_T", "use_outer"}, + ) + ) diff --git a/tests/fortran/functions/policy/test_native_entrypoint_policy.py b/tests/fortran/functions/policy/test_native_entrypoint_policy.py index f65e88914..bd50be64c 100644 --- a/tests/fortran/functions/policy/test_native_entrypoint_policy.py +++ b/tests/fortran/functions/policy/test_native_entrypoint_policy.py @@ -1,3 +1,5 @@ +import ast + from prik.parsers.fortran import parse_fortran_file from prik.policy import complete_semantic_policies from prik.policy.construction import completed_function_wrapper_policy @@ -6,9 +8,11 @@ EntrypointPassingConvention, EntrypointProjectionAction, NativeEntrypointAction, + ScalarActualMode, ScalarLogicalABI, ) from prik.semantics.fortran2ir import fortran_module_to_semantic_module +from prik.semantics.pyi2ir import convert_pyi_to_ir def _completed_policies(source: str): @@ -110,3 +114,81 @@ def test_entrypoint_policy_distinguishes_c_bool_from_ordinary_logical_storage(): ordinary = policies["ordinary_bool"] assert ordinary.entrypoint_action is NativeEntrypointAction.GENERATED_FORTRAN_ADAPTER assert ordinary.arguments[0].scalar_logical_abi is ScalarLogicalABI.NATIVE_KIND_COPY + + +def test_rank_zero_actual_transport_follows_the_dummy_on_direct_and_adapted_routes(): + """A VALUE dummy copies a rank-zero actual whether or not a bridge adapts it.""" + policies = _completed_policies( + """ +module actual_routes + use iso_c_binding +contains + subroutine direct_value(n) bind(C) + integer(c_int), value :: n + end subroutine direct_value + + subroutine adapted_value(n) + integer, value :: n + end subroutine adapted_value + + subroutine adapted_reference(n) + integer, intent(inout) :: n + end subroutine adapted_reference + + subroutine c_bool_logical(flag) + logical(c_bool), intent(inout) :: flag + end subroutine c_bool_logical +end module actual_routes +""" + ) + + assert {name: policy.arguments[0].scalar_actual_mode for name, policy in policies.items()} == { + "direct_value": ScalarActualMode.NUMERIC_VALUE, + "adapted_value": ScalarActualMode.NUMERIC_VALUE, + "adapted_reference": ScalarActualMode.NUMERIC_REFERENCE, + "c_bool_logical": ScalarActualMode.NUMERIC_REFERENCE, + } + + +def test_wider_logical_dummies_borrow_integer_storage_of_their_own_width(): + """A logical wider than c_bool crosses as its own integer width, as its arrays do.""" + source = """ +from prik.contracts import Bool, Bool16, Bool32, Bool64 + +def flags(narrow: Bool, short: Bool16, default: Bool32, wide: Bool64) -> None: ... +""" + module = convert_pyi_to_ir(ast.parse(source), module_name="logical_storage", source=source) + complete_semantic_policies(module) + arguments = completed_function_wrapper_policy(module.functions[0]).arguments + + assert [(argument.scalar_logical_abi, argument.native_storage_c_type) for argument in arguments] == [ + (ScalarLogicalABI.C_BOOL, None), + (ScalarLogicalABI.NATIVE_KIND_STORAGE, "int16_t"), + (ScalarLogicalABI.NATIVE_KIND_STORAGE, "int32_t"), + (ScalarLogicalABI.NATIVE_KIND_STORAGE, "int64_t"), + ] + assert all(argument.scalar_actual_mode is not None for argument in arguments) + + +def test_immutable_values_copy_rank_zero_actuals_instead_of_lending_storage(): + """Immutable storage is only read; a plain update lends the actual's storage.""" + source = """ +from prik.contracts import Annotated, Immutable, Int32, Returns, String + +def bump(value: Annotated[Int32, Immutable]) -> Returns["value", Int32]: ... +def bump_in_place(value: Int32) -> Returns["value", Int32]: ... +def label(text: Annotated[String[8], Immutable]) -> Returns["text", String[8]]: ... +def label_in_place(text: String[8]) -> Returns["text", String[8]]: ... +""" + module = convert_pyi_to_ir(ast.parse(source), module_name="immutable_actuals", source=source) + complete_semantic_policies(module) + + assert { + function.name: completed_function_wrapper_policy(function).arguments[0].scalar_actual_mode + for function in module.functions + } == { + "bump": ScalarActualMode.NUMERIC_VALUE, + "bump_in_place": ScalarActualMode.NUMERIC_REFERENCE, + "label": ScalarActualMode.CHARACTER_VALUE, + "label_in_place": ScalarActualMode.CHARACTER_REFERENCE, + } diff --git a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py index 2b699e2a0..0b3c23a77 100644 --- a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py +++ b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py @@ -5,6 +5,8 @@ import pytest from prik.cli import _read_export_symbols +from prik.parsers.fortran import parse_fortran_file +from prik.semantics.fortran2ir import fortran_module_to_semantic_module from prik.semantics.models import NATIVE_ACCESS_MODULE_METADATA from prik.semantics.fortran_exports import select_fortran_export_symbols from prik.semantics.models import ( @@ -183,3 +185,37 @@ def test_external_root_cannot_satisfy_a_module_qualified_identity(): with pytest.raises(ValueError, match="unknown modules: foo"): select_fortran_export_symbols([external], ["foo::external"]) + + +def test_selection_retains_component_and_parent_types_of_a_selected_signature(): + """A selected type brings the types its components and parent declare, and no others.""" + module = fortran_module_to_semantic_module( + parse_fortran_file( + """ +module shapes + implicit none + type :: base_t + integer :: b = 1 + end type base_t + type :: inner_t + integer :: a = 2 + end type inner_t + type, extends(base_t) :: outer_t + type(inner_t) :: inner + end type outer_t + type :: unrelated_t + integer :: u = 0 + end type unrelated_t +contains + subroutine use_outer(x) + type(outer_t), intent(in) :: x + end subroutine use_outer +end module shapes +""" + ).modules[0] + ) + + selected = select_fortran_export_symbols([module], ["shapes::use_outer"]).primary_modules[0] + + assert sorted(cls.name for cls in selected.classes) == ["base_t", "inner_t", "outer_t"] + assert set(selected.exported_names) == {"use_outer", "base_t", "inner_t", "outer_t"} diff --git a/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py b/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py index 31e5bcd21..feba1457f 100644 --- a/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py +++ b/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py @@ -25,8 +25,8 @@ def test_ordinary_fortran_wrapper_preserves_exact_generated_bytes(): "01c092ac9eaa0d90b58f0289a49ba0c71c967510e60a384602fe2e6e1e9b035f", ), "ordinary_entrypoint_baseline_wrapper.c": ( - 1941, - "9b944e6ebb8f5b1eef87407e046117b5d2b350286cc32917bb2f8182ab3bbb30", + 2613, + "19180196b0e7a25848c88e8985e94da0bda5ffecb78d68bf38909893cae58c3a", ), "ordinary_entrypoint_baseline_wrapper.h": ( 248, diff --git a/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py b/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py index 8d5c2c970..458477873 100644 --- a/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py +++ b/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py @@ -104,7 +104,7 @@ def swap_args(x: Float64, y: Float64) -> Float64: ... assert 'static char * kwlist[] = {"x", "y", NULL};' in c_source assert 'PyArg_ParseTupleAndKeywords(args, kwargs, "OO", kwlist, &bound_x_obj, &bound_y_obj)' in c_source assert "prik_float64_unpack_exact(bound_x_obj, &bound_x)" in c_source - assert "result = bind_c_swap_args(&bound_y, &bound_x);" in c_source + assert "result = bind_c_swap_args(bound_y_storage, bound_x_storage);" in c_source assert "PyObject * result_obj = prik_float64_to_numpy(&result);" in c_source assert "PyMODINIT_FUNC PyInit_render_demo(void)" in c_source assert "static PyObject * wrap_swap_args" in c_header diff --git a/tests/fortran/infrastructure/policy/test_policy_completion.py b/tests/fortran/infrastructure/policy/test_policy_completion.py index 799b91a3f..9bbf2d30b 100644 --- a/tests/fortran/infrastructure/policy/test_policy_completion.py +++ b/tests/fortran/infrastructure/policy/test_policy_completion.py @@ -114,7 +114,12 @@ def test_scalar_accessor_policies_are_complete_before_ir_lowering(): assert setter.setter_action is SetterAction.WRITE_THROUGH -def test_scalar_descriptor_accessor_policies_are_nullable_snapshots(): +def test_scalar_descriptor_fields_snapshot_while_module_variables_lend_current_storage(): + """A descriptor field copies its value; a module descriptor lends its current storage. + + A module variable's setter assigns through the descriptor instead: an + allocatable is allocated when needed and a pointer writes its target. + """ module = parse_pyi_text( """ alloc_value: Allocatable[Float64] @@ -129,30 +134,34 @@ class point: complete_semantic_policies(module) - variables = [ - module.variables[0], - module.variables[1], - module.classes[0].fields[0], - module.classes[0].fields[1], - ] - for variable in variables: - storage = variable.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA] - getter = variable.metadata[RESOLVED_GETTER_OWNERSHIP_POLICY_METADATA] - setter = variable.metadata[RESOLVED_SETTER_OWNERSHIP_POLICY_METADATA] + alloc_module, ptr_module = module.variables + alloc_field, ptr_field = module.classes[0].fields + for field in (alloc_field, ptr_field): + storage = field.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA] + getter = field.metadata[RESOLVED_GETTER_OWNERSHIP_POLICY_METADATA] assert storage.transfer is TransferMode.SNAPSHOT_COPY assert storage.nullable is True assert storage.codegen_action is CodegenAction.SNAPSHOT_COPY assert getter.transfer is TransferMode.SNAPSHOT_COPY assert getter.nullable is True - assert getter.codegen_action is CodegenAction.SNAPSHOT_COPY - assert setter.setter_action is SetterAction.REJECT_REPLACEMENT - - alloc_module, ptr_module, alloc_field, ptr_field = variables - assert alloc_module.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA].storage_mode is StorageMode.HEAP + assert ( + field.metadata[RESOLVED_SETTER_OWNERSHIP_POLICY_METADATA].setter_action is SetterAction.REJECT_REPLACEMENT + ) assert alloc_field.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA].storage_mode is StorageMode.HEAP - assert ptr_module.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA].storage_mode is StorageMode.ALIAS assert ptr_field.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA].storage_mode is StorageMode.ALIAS + for variable, assignment in ( + (alloc_module, AssignmentMode.ALLOCATING_COPY), + (ptr_module, AssignmentMode.TARGET_COPY), + ): + getter = variable.metadata[RESOLVED_GETTER_OWNERSHIP_POLICY_METADATA] + setter = variable.metadata[RESOLVED_SETTER_OWNERSHIP_POLICY_METADATA] + assert getter.transfer is TransferMode.BORROWED_VIEW + assert getter.nullable is True + assert getter.storage_mode is StorageMode.ALIAS + assert setter.setter_action is SetterAction.WRITE_THROUGH + assert setter.assignment_mode is assignment + def test_scalar_descriptor_function_boundaries_use_normal_scalar_values(): module = parse_pyi_text( diff --git a/tests/fortran/infrastructure/semantic_ir/semantics/fixtures/general/expected/modern_pyi_example.json b/tests/fortran/infrastructure/semantic_ir/semantics/fixtures/general/expected/modern_pyi_example.json index 0520ccd5e..5e3569e96 100644 --- a/tests/fortran/infrastructure/semantic_ir/semantics/fixtures/general/expected/modern_pyi_example.json +++ b/tests/fortran/infrastructure/semantic_ir/semantics/fixtures/general/expected/modern_pyi_example.json @@ -2468,7 +2468,9 @@ "coercions": [], "constraints": [], "dtype": "Int32", - "metadata": {}, + "metadata": { + "native_storage": true + }, "name": "Int32", "origin": { "metadata": { @@ -2533,7 +2535,9 @@ "coercions": [], "constraints": [], "dtype": "Float64", - "metadata": {}, + "metadata": { + "native_storage": true + }, "name": "Float64", "origin": { "metadata": { diff --git a/tests/fortran/infrastructure/semantic_ir/semantics/fixtures/general/expected/scope_name_reuse_combinations.json b/tests/fortran/infrastructure/semantic_ir/semantics/fixtures/general/expected/scope_name_reuse_combinations.json index 367296a8a..e415e62a6 100644 --- a/tests/fortran/infrastructure/semantic_ir/semantics/fixtures/general/expected/scope_name_reuse_combinations.json +++ b/tests/fortran/infrastructure/semantic_ir/semantics/fixtures/general/expected/scope_name_reuse_combinations.json @@ -1513,7 +1513,9 @@ "coercions": [], "constraints": [], "dtype": "Int32", - "metadata": {}, + "metadata": { + "native_storage": true + }, "name": "Int32", "origin": { "metadata": { @@ -1578,7 +1580,9 @@ "coercions": [], "constraints": [], "dtype": "Float32", - "metadata": {}, + "metadata": { + "native_storage": true + }, "name": "Float32", "origin": { "metadata": { @@ -1643,7 +1647,9 @@ "coercions": [], "constraints": [], "dtype": "Bool", - "metadata": {}, + "metadata": { + "native_storage": true + }, "name": "Bool", "origin": { "metadata": { @@ -1708,7 +1714,9 @@ "coercions": [], "constraints": [], "dtype": "Complex64", - "metadata": {}, + "metadata": { + "native_storage": true + }, "name": "Complex64", "origin": { "metadata": { @@ -1774,7 +1782,8 @@ "constraints": [], "dtype": "String", "metadata": { - "fortran_character_length": "8" + "fortran_character_length": "8", + "native_storage": true }, "name": "String", "origin": { diff --git a/tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_projected_entrypoint_routes.py b/tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_projected_entrypoint_routes.py index a896bd6f3..23a5f2759 100644 --- a/tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_projected_entrypoint_routes.py +++ b/tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_projected_entrypoint_routes.py @@ -68,11 +68,11 @@ def test_direct_projection_reorders_value_and_address_actuals_and_materializes_l } binding = (result.output_dir / "direct_projection_runtime_wrapper.c").read_text(encoding="utf-8") assert "int32_t projected_native(int32_t right, int32_t * left, int32_t literal_2);" in binding - assert "result = projected_native(bound_right, &bound_left, 5);" in binding + assert "result = projected_native(bound_right, bound_left_storage, 5);" in binding assert ( "void projected_output_native(int32_t right, int32_t * left, int32_t literal_2, int32_t * output);" in binding ) - assert "projected_output_native(bound_right, &bound_left, 5, &output);" in binding + assert "projected_output_native(bound_right, bound_left_storage, 5, &output);" in binding def test_adapted_projection_uses_the_same_binding_owned_actual_sequence(tmp_path: Path): @@ -92,8 +92,8 @@ def test_adapted_projection_uses_the_same_binding_owned_actual_sequence(tmp_path } binding = (result.output_dir / "adapted_projection_runtime_wrapper.c").read_text(encoding="utf-8") bridge = (result.output_dir / "bind_c_adapted_projection_runtime_wrapper.f90").read_text(encoding="utf-8") - assert "bind_c_projected(bound_right, &bound_left, 5)" in binding - assert "bind_c_projected_output(bound_right, &bound_left, 5, &output)" in binding + assert "bind_c_projected(bound_right, bound_left_storage, 5)" in binding + assert "bind_c_projected_output(bound_right, bound_left_storage, 5, &output)" in binding assert "function bind_c_projected(right, left, literal_2)" in bridge assert "native_projected(right, left, literal_2)" in bridge assert "subroutine bind_c_projected_output(right, left, literal_2, output)" in bridge diff --git a/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/fixtures/visibility/contracts/fnaming_f90/fnaming_f90.pyi b/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/fixtures/visibility/contracts/fnaming_f90/fnaming_f90.pyi index 4116a049a..81d4f5298 100644 --- a/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/fixtures/visibility/contracts/fnaming_f90/fnaming_f90.pyi +++ b/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/fixtures/visibility/contracts/fnaming_f90/fnaming_f90.pyi @@ -14,7 +14,7 @@ class Visible_T: @bind("Visible_T.from") def from_(self) -> Int32: ... -value: Int32 +value: Int32[()] @bind("lambda") @native_call([Addr(Arg(0))]) diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/fixtures/modern_math_physics.pyi b/tests/fortran/infrastructure/semantic_pyi/pipeline/fixtures/modern_math_physics.pyi index 5ff22953b..5e081762e 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/fixtures/modern_math_physics.pyi +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/fixtures/modern_math_physics.pyi @@ -17,7 +17,7 @@ class vector3: values: Float64[3] -counter: Int32 +counter: Int32[()] @native_call([Arg(0), Addr(Arg(1)), Addr(Arg(2)), Addr(Arg(3)), Addr(Arg(4)), Addr(Arg(5))]) def init_particle( diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py index 29477554a..1615d81ab 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py @@ -685,8 +685,8 @@ def test_fortran_contract_records_a_source_name_python_cannot_spell(): complete_python_export_policy(module) code = emit_module(module, normalize_public_names=True) - assert 'lambda_: Annotated[Int32, SourceName("lambda")]' in code - assert 'lambda__2: Annotated[Int32, SourceName("LAMBDA_")]' in code + assert 'lambda_: Annotated[Int32[()], SourceName("lambda")]' in code + assert 'lambda__2: Annotated[Int32[()], SourceName("LAMBDA_")]' in code assert '@bind("ASSERT")\n@native_call([Addr(Arg(0))])\ndef assert_(' in code diff --git a/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py b/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py index 04379e91f..ef6e6ee5f 100644 --- a/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py +++ b/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py @@ -109,10 +109,12 @@ def test_module_variable_plan_contains_only_completed_dispatch_facts(): assert variables["target_scale"].bridge.native_assignment is AssignmentMode.VALUE_COPY assert variables["optional_scale"].binding.getter_action is ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW assert variables["optional_scale"].entrypoint.descriptor_kind == "allocatable" - assert variables["optional_scale"].binding.setter_action is SetterAction.REJECT_REPLACEMENT - assert variables["optional_scale"].bridge.native_assignment is AssignmentMode.NONE + assert variables["optional_scale"].binding.setter_action is SetterAction.WRITE_THROUGH + assert variables["optional_scale"].binding.native_assignment is AssignmentMode.ALLOCATING_COPY + assert variables["optional_scale"].bridge.native_assignment is AssignmentMode.ALLOCATING_COPY assert variables["selected_scale"].entrypoint.descriptor_kind == "pointer" - assert variables["selected_scale"].bridge.native_assignment is AssignmentMode.NONE + assert variables["selected_scale"].binding.native_assignment is AssignmentMode.TARGET_COPY + assert variables["selected_scale"].bridge.native_assignment is AssignmentMode.TARGET_COPY def test_symbolic_source_parameter_reuses_scalar_bridge_getter_for_module_initialization(): @@ -205,7 +207,7 @@ def test_fortran_module_setter_rejects_unsupported_bridge_assignment(): [ ("counter", AssignmentMode.NONE), ("counter", AssignmentMode.ALIAS), - ("optional_scale", AssignmentMode.VALUE_COPY), + ("optional_scale", AssignmentMode.NONE), ("optional_scale", AssignmentMode.ALIAS), ("limit", AssignmentMode.VALUE_COPY), ], @@ -258,14 +260,14 @@ def test_module_variable_generators_dispatch_get_set_and_rejection_from_plan(): assert "return bind_c_get_counter();" not in c_source assert "bind_c_get_counter()" in c_source assert "bind_c_set_counter(value)" in c_source - assert "module variable optional_scale is read-only" in c_source - assert "module variable selected_scale is read-only" in c_source + assert "bind_c_set_optional_scale(value)" in c_source + assert "Module variable selected_scale has no pointer target" in c_source assert "result = native_counter" in fortran_source assert "native_counter = value" in fortran_source assert "allocated(native_optional_scale)" in fortran_source assert "associated(native_selected_scale)" in fortran_source - assert "optional_scale = value" not in fortran_source - assert "selected_scale = value" not in fortran_source + assert "native_optional_scale = value" in fortran_source + assert "native_selected_scale = value" in fortran_source def test_generated_support_procedure_symbol_is_shared_by_both_boundary_lowerers(): diff --git a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py index b8c232c88..8f508b634 100644 --- a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py +++ b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py @@ -302,7 +302,7 @@ def _character_descriptor_module(tmp_path: Path): def test_descriptor_character_module_variables_follow_current_storage(pyi_parity_build_mode: str, tmp_path: Path): - """Each attribute read borrows the current address and character width.""" + """Each read lends the current storage read-only; assignment writes through the descriptor.""" module = _build_source_or_generated_pyi_and_import( NATIVE_FIXTURES / "fchar_module_descriptors_f90.f90", tmp_path, @@ -327,14 +327,29 @@ def test_descriptor_character_module_variables_follow_current_storage(pyi_parity assert fixed is not None and fixed[()] == b"FIXEDV" assert view is not None and view.shape == () and view.dtype == np.dtype("S6") assert view[()] == b"STORED" - view[()] = b"PYTHON" + with pytest.raises(ValueError, match="read-only"): + view[()] = b"PYTHON" + module.link = "PYTHON" + assert view[()] == b"PYTHON" assert module.store[()] == b"PYTHON" + with pytest.raises(TypeError, match="pointer target's width"): + module.link = "SHORT" module.grow() grown = module.deferred assert grown is not None and grown.shape == () and grown.dtype == np.dtype("S10") assert grown[()] == b"alpha-more" + # A deferred-length assignment reallocates to the encoded width, including zero. + module.deferred = "omega" + assert module.deferred[()] == b"omega" + module.deferred = "" + assert module.deferred is not None and module.deferred[()] == b"" + with pytest.raises(TypeError, match="exactly 6 bytes"): + module.fixed = "WIDE!!!" + module.fixed = "NARROW" + assert module.fixed[()] == b"NARROW" + def test_descriptor_character_module_variables_report_absence_as_none(tmp_path: Path): """Deallocation and nullification are values Python observes, not stale reads.""" diff --git a/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py b/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py index 52e1f21be..788c9c5f0 100644 --- a/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py +++ b/tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py @@ -71,9 +71,9 @@ def test_whole_scalar_module_variable_behavior_uses_canonical_plan( module.counter = np.float64(4.0) with pytest.raises(AttributeError): del module.counter - with pytest.raises(AttributeError): - module.optional_scale = np.float64(9.0) - with pytest.raises(AttributeError): + module.optional_scale = np.float64(9.0) + assert module.optional_scale[()] == np.float64(9.0) + with pytest.raises(ValueError, match="no pointer target"): module.selected_scale = np.float64(9.0) assert module.set_allocatable(np.float64(1.5)) == np.float64(1.5) diff --git a/tests/fortran/modules/parsing/test_project_scope_models.py b/tests/fortran/modules/parsing/test_project_scope_models.py index 1bd12f395..f4e852bbe 100644 --- a/tests/fortran/modules/parsing/test_project_scope_models.py +++ b/tests/fortran/modules/parsing/test_project_scope_models.py @@ -5,6 +5,7 @@ from prik.parsers.fortran import FortranParseError, parse_fortran_file, parse_fortran_project from prik.parsers.fortran.scope import ScopeUses from prik.parsers.fortran.parser import FortranParser +from prik.semantics.fortran2ir import fortran_module_to_semantic_module def test_module_visibility_public_and_private_spec_lines_are_applied(): @@ -71,6 +72,46 @@ def test_separate_parameter_statement_and_bind_c_module_storage_are_preserved(): assert variables["addressable"]._fortran_bind_c +@pytest.mark.parametrize( + ("implicit", "expected"), + [ + pytest.param("", {"pi": "Float32", "n": "Int32"}, id="default-letter-rules"), + pytest.param( + "implicit double precision (a-h,o-z), integer(kind=8) (n)", + {"pi": "Float64", "n": "Int64"}, + id="implicit-statement-mapping", + ), + ], +) +def test_separate_parameter_statement_types_an_undeclared_name_by_module_implicit_rules(implicit, expected): + module = parse_fortran_file( + f""" +module legacy_constants + {implicit} + parameter (pi = 3.14159265358979d0, n = 4) +end module legacy_constants +""" + ).modules[0] + + semantic = fortran_module_to_semantic_module(module) + assert {variable.name: variable.semantic_type.name for variable in semantic.variables} == expected + assert all(variable.is_parameter for variable in module.variables) + + +def test_separate_parameter_statement_under_implicit_none_requires_a_declaration(): + with pytest.raises(FortranParseError, match="implicit none is active") as error: + parse_fortran_file( + """ +module strict_constants + implicit none + parameter (undeclared = 3) +end module strict_constants +""" + ) + + assert error.value.code == "PARSE_UNKNOWN_PARAMETER_TYPE" + + def test_submodule_types_interfaces_and_project_dependencies_attach_to_public_models(): code = """ submodule (ancestor_mod:parent_mod) child_mod diff --git a/tests/fortran/modules/policy/test_module_variable_policy.py b/tests/fortran/modules/policy/test_module_variable_policy.py index 2b0ab3cc8..dff1676cb 100644 --- a/tests/fortran/modules/policy/test_module_variable_policy.py +++ b/tests/fortran/modules/policy/test_module_variable_policy.py @@ -49,12 +49,12 @@ def test_scalar_module_variable_policy_completes_access_and_storage_before_plann assert policies["target_scale"].native_assignment is AssignmentMode.VALUE_COPY assert policies["optional_scale"].getter_action is ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW assert policies["optional_scale"].descriptor_kind == "allocatable" - assert policies["optional_scale"].setter_action is SetterAction.REJECT_REPLACEMENT - assert policies["optional_scale"].native_assignment is AssignmentMode.NONE + assert policies["optional_scale"].setter_action is SetterAction.WRITE_THROUGH + assert policies["optional_scale"].native_assignment is AssignmentMode.ALLOCATING_COPY assert policies["selected_scale"].getter_action is ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW assert policies["selected_scale"].descriptor_kind == "pointer" - assert policies["selected_scale"].setter_action is SetterAction.REJECT_REPLACEMENT - assert policies["selected_scale"].native_assignment is AssignmentMode.NONE + assert policies["selected_scale"].setter_action is SetterAction.WRITE_THROUGH + assert policies["selected_scale"].native_assignment is AssignmentMode.TARGET_COPY def test_fixed_character_handles_publish_only_the_descriptor_attribute_their_callback_can_supply(): diff --git a/tests/fortran/optional_arguments/codegen/test_optional_lowering.py b/tests/fortran/optional_arguments/codegen/test_optional_lowering.py index 9eb21411c..37841cfa4 100644 --- a/tests/fortran/optional_arguments/codegen/test_optional_lowering.py +++ b/tests/fortran/optional_arguments/codegen/test_optional_lowering.py @@ -52,7 +52,7 @@ def test_optional_scalar_lowering_distinguishes_absent_or_none_from_value(): assert 'PyArg_ParseTupleAndKeywords(args, kwargs, "O|O"' in c_source assert "PyObject * bound_factor_obj = Py_None;" in c_source assert "if (bound_factor_obj != Py_None)" in c_source - assert "bound_factor_nullable = &bound_factor;" in c_source + assert "bound_factor_nullable = bound_factor_storage;" in c_source assert "bind_c_optional_scale(base, bound_factor)" in fortran_source assert "if (c_associated(bound_factor)) then" in fortran_source assert "result = optional_scale(base=base, factor=prik_optional_factor)" in fortran_source diff --git a/tests/fortran/pointers/end_to_end/test_scalar_pointers.py b/tests/fortran/pointers/end_to_end/test_scalar_pointers.py index 382d6e5da..64139838f 100644 --- a/tests/fortran/pointers/end_to_end/test_scalar_pointers.py +++ b/tests/fortran/pointers/end_to_end/test_scalar_pointers.py @@ -68,14 +68,17 @@ def test_scalar_pointers_project_nullable_native_views( module.clear_pointer() assert module.selected_scale is None - with pytest.raises(AttributeError): + with pytest.raises(ValueError, match="no pointer target"): module.selected_scale = np.float64(9.0) + # Assignment writes the current target; the view it reads is live but read-only. module.point_to_target(np.float64(2.5)) + module.selected_scale = np.float64(3.5) view = module.selected_scale assert view is not None and view.shape == () and view.dtype == np.dtype("float64") - assert view[()] == np.float64(2.5) - view[()] = np.float64(3.5) + assert view[()] == np.float64(3.5) + with pytest.raises(ValueError, match="read-only"): + view[()] = np.float64(4.5) module.bump_native() assert view[()] == np.float64(23.5) assert module.selected_scale[()] == np.float64(23.5) diff --git a/tests/fortran/subroutines/codegen/test_hidden_scalar_outputs.py b/tests/fortran/subroutines/codegen/test_hidden_scalar_outputs.py index 16477e2b1..877594f08 100644 --- a/tests/fortran/subroutines/codegen/test_hidden_scalar_outputs.py +++ b/tests/fortran/subroutines/codegen/test_hidden_scalar_outputs.py @@ -30,7 +30,7 @@ def scale(x: Float64) -> Float64: ... fortran_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") assert "void bind_c_scale(double * x, double * result);" in c_source - assert "bind_c_scale(&bound_x, &result);" in c_source + assert "bind_c_scale(bound_x_storage, &result);" in c_source assert "PyObject * result_obj = prik_float64_to_numpy(&result);" in c_source assert 'subroutine bind_c_scale(x, result) bind(c, name="bind_c_scale")' in fortran_source assert "external :: SCALE_OUT" in fortran_source From bac50fb7b088ff35cd5fe97968da1543bf2e86ac Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 15:06:56 +0100 Subject: [PATCH 06/47] fix mkdocs --- mkdocs.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mkdocs.yml b/mkdocs.yml index 626fb8131..a33bf9977 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -52,8 +52,7 @@ nav: - Strings: user/guide/strings.md - Wrapping Functions: user/guide/wrapping-functions.md - Wrapping Subroutines: user/guide/wrapping-subroutines.md - - Wrapping Modules: user/guide/wrapping-modules.md - - Open MPI mpi_f08: user/tutorials/openmpi-f08.md + - Wrapping Modules: user/guide/wrapping-modules.md - Optional Arguments: user/guide/optional-arguments.md - Generic Interfaces (Overloading): user/guide/generic-interfaces.md - Wrapping Derived Types: user/guide/wrapping-derived-types.md @@ -77,6 +76,7 @@ nav: - Tutorials: - Run PRIK in a Notebook: user/tutorials/notebook-quickstart.md - Design a Pythonic BLAS API: user/tutorials/pythonic-blas.md + - Wrap Open MPI mpi_f08: user/tutorials/openmpi-f08.md - Examples: - Overview: user/examples/index.md - Fortran: From 99fe05c6619edff6f1c12b4bacd14198cf393af3 Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 15:48:19 +0100 Subject: [PATCH 07/47] Remove module-qualified @bind facade routes The bridge calls each selected procedure through the module that declares it, so a declaring module's contract no longer names a facade that imports it. Export selection still resolves facade names to their declarations, and the facade contract still re-exports them. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 10 +++--- docs/user/reference/pyi-format.md | 3 -- prik/policy/construction.py | 6 +--- prik/printers/pyi.py | 12 ------- prik/semantics/fortran_exports.py | 34 ++++--------------- prik/semantics/models.py | 1 - prik/semantics/pyi2ir.py | 20 +---------- .../end_to_end/test_openmpi_f08.py | 4 +-- .../test_fortran_export_symbol_workflow.py | 1 - .../test_fortran_export_symbol_selection.py | 7 ++-- .../parsing/test_python_ast_contracts.py | 12 ++----- 11 files changed, 21 insertions(+), 89 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9e815ceed..2153b28ac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,11 +34,11 @@ release tags add a leading `v` to the package version. - `--export-symbols` and `build_fortran_extension(export_symbols=...)` accept module-qualified Fortran procedures and variables, including symbols - re-exported by a public facade. Generated contracts retain the selected - access module through qualified `@bind`, required type declarations - (including the component and parent types they declare), and native scalar - storage views through `T[()]`. Source builds and generated contracts publish - the same selected surface. + re-exported by a public facade; the bridge calls each procedure through the + module that declares it. Generated contracts retain required type + declarations (including the component and parent types they declare) and + native scalar storage views through `T[()]`. Source builds and generated + contracts publish the same selected surface. - The Open MPI `mpi_f08` tutorial and opt-in two-rank integration test build a wrapper from a restricted generated `.pyi` against a matching prebuilt Open MPI installation and exercise NumPy communication and in-place reduction. diff --git a/docs/user/reference/pyi-format.md b/docs/user/reference/pyi-format.md index 63133962d..767298298 100644 --- a/docs/user/reference/pyi-format.md +++ b/docs/user/reference/pyi-format.md @@ -305,9 +305,6 @@ def update(value: Float64[()]) -> None: ... ``` The leaf filename supplies the native module. No placement decorator is needed. -When a public Fortran facade re-exports a procedure from a supporting module, -`@bind("facade_name::procedure_name")` selects the module and procedure that the -bridge calls. Generated contracts retain this access route. ### Standalone Procedures diff --git a/prik/policy/construction.py b/prik/policy/construction.py index 086d78776..4063f5aaf 100644 --- a/prik/policy/construction.py +++ b/prik/policy/construction.py @@ -8332,11 +8332,7 @@ def _native_module(function: models.SemanticFunction, owner_path: str) -> str | """Return the completed native module scope for non-standalone procedures.""" if _is_standalone(function): return None - return str( - function.metadata.get(models.NATIVE_ACCESS_MODULE_METADATA) - or function.origin.native_scope - or owner_path.split(".", maxsplit=1)[0] - ) + return str(function.origin.native_scope or owner_path.split(".", maxsplit=1)[0]) def _native_is_subroutine(function: models.SemanticFunction) -> bool: diff --git a/prik/printers/pyi.py b/prik/printers/pyi.py index 0d46667d5..b5a040e86 100644 --- a/prik/printers/pyi.py +++ b/prik/printers/pyi.py @@ -46,7 +46,6 @@ OVERLOAD_KIND_METADATA, OVERLOAD_TARGET_METADATA, NATIVE_BY_VALUE_METADATA, - NATIVE_ACCESS_MODULE_METADATA, PYTHON_BOUND_POSITION_METADATA, PYTHON_METHOD_NAME_METADATA, PYTHON_STATIC_METADATA, @@ -415,7 +414,6 @@ def _visit_ProcedureOverloadSet( if candidate.origin.native_symbol != candidate.origin.native_name else None ) - bind_target = self._module_bind_target(candidate, bind_target) bind = f"{indent}@{context.contract('bind')}({json.dumps(str(bind_target))})\n" if bind_target else "" native_abi = ( f'{indent}@{context.contract("native_abi")}("c")\n' @@ -2017,20 +2015,10 @@ def _identity_decorators( bind_target = self._bind_target( func, context=context, emitted_name=emitted_name, is_native_c_abi=is_native_c_abi ) - bind_target = self._module_bind_target(func, bind_target) if bind_target and not is_overload: decorators.append(f"{indent}@{context.contract('bind')}({json.dumps(str(bind_target))})") return decorators - @staticmethod - def _module_bind_target(func: SemanticFunction, bind_target: object | None) -> str | None: - """Render a Fortran module access route in the existing bind spelling.""" - module = func.metadata.get(NATIVE_ACCESS_MODULE_METADATA) - if module is None: - return str(bind_target) if bind_target is not None else None - symbol = bind_target or func.native_name or func.name - return f"{module}::{symbol}" - @staticmethod def _constructor_binds_its_own_type( overload_name: str, diff --git a/prik/semantics/fortran_exports.py b/prik/semantics/fortran_exports.py index c6569b928..5d6def196 100644 --- a/prik/semantics/fortran_exports.py +++ b/prik/semantics/fortran_exports.py @@ -7,10 +7,8 @@ from dataclasses import dataclass import re -from prik.semantics.metadata import BIND_TARGET_METADATA from prik.semantics.models import ( EXTERNAL_TYPE_REF_METADATA, - NATIVE_ACCESS_MODULE_METADATA, ProcedureOverloadSet, SemanticFunction, SemanticModule, @@ -65,7 +63,7 @@ def select_fortran_export_symbols( selectable, non_selectable = _fortran_export_candidates(source_modules) _validate_fortran_export_resolution(requested, selectable, non_selectable, module_index) - selected, access_modules = _selection_routes(requested, module_index) + selected = _selected_identities(requested, module_index) primary_names = {module_name for module_name, _symbol_name in selected} primary_sources = [] primary_modules = [] @@ -73,7 +71,7 @@ def select_fortran_export_symbols( module_name = _native_module_name(module) if module_name not in primary_names: continue - selected_module = _select_module_surface(module, selected, set(requested), access_modules) + selected_module = _select_module_surface(module, selected, set(requested)) primary_sources.append(module) primary_modules.append(selected_module) @@ -91,20 +89,17 @@ def select_fortran_export_symbols( return FortranExportSelection(tuple(primary_sources), tuple(primary_modules), context_modules) -def _selection_routes(requested, module_index): - """Resolve selected facade names to their declaring identities and access routes.""" +def _selected_identities(requested, module_index): + """Add the declaring identity of each name selected through a facade re-export.""" selected = set(requested) - access_modules: dict[tuple[str, str], str] = {} for module_name, symbol_name in requested: for reexport in module_index[module_name].reexports: if reexport.local_name.casefold() == symbol_name: - identity = (reexport.origin_module.casefold(), reexport.source_name.casefold()) - selected.add(identity) - access_modules[identity] = module_name - return selected, access_modules + selected.add((reexport.origin_module.casefold(), reexport.source_name.casefold())) + return selected -def _select_module_surface(module, selected, requested, access_modules): +def _select_module_surface(module, selected, requested): """Retain selected declarations while keeping generic specifics private to them.""" selected_module = deepcopy(module) module_name = _native_module_name(module) @@ -114,7 +109,6 @@ def _select_module_surface(module, selected, requested, access_modules): if (module_name, _native_symbol_name(overload)) in selected ] _retain_selected_procedures(selected_module, module_name, selected) - _route_selected_callables(selected_module, module_name, access_modules) selected_module.variables = [ variable for variable in selected_module.variables if (module_name, _native_symbol_name(variable)) in selected ] @@ -149,20 +143,6 @@ def _retain_selected_procedures(module, module_name, selected): module.functions.extend(deepcopy(procedure) for name, procedure in specifics.items() if name not in declared) -def _route_selected_callables(module, module_name, access_modules): - """Record the public native module and generic name used for each callable.""" - for declaration in (*module.functions, *module.overload_sets): - access = access_modules.get((module_name, _native_symbol_name(declaration))) - if access is None: - continue - procedures = declaration.procedures if isinstance(declaration, ProcedureOverloadSet) else (declaration,) - for procedure in procedures: - procedure.metadata[NATIVE_ACCESS_MODULE_METADATA] = access - if isinstance(declaration, ProcedureOverloadSet): - procedure.native_name = declaration.name - procedure.metadata[BIND_TARGET_METADATA] = declaration.name - - def _retain_required_types(module, source_module, required_types): """Publish only derived types required by selected values or signatures.""" module_name = _native_module_name(module) diff --git a/prik/semantics/models.py b/prik/semantics/models.py index b37f87596..b9a0ddcdd 100644 --- a/prik/semantics/models.py +++ b/prik/semantics/models.py @@ -28,7 +28,6 @@ PYTHON_VALUE_MUTABILITY_METADATA = "python_value_mutability" PYTHON_VALUE_IMMUTABLE = "immutable" NATIVE_BY_VALUE_METADATA = "native_by_value" -NATIVE_ACCESS_MODULE_METADATA = "native_access_module" RUNTIME_RELEASE_GIL_METADATA = "runtime_release_gil" RUNTIME_RETAIN_RESULT_OWNER_METADATA = "runtime_retain_result_owner" RUNTIME_STATUS_ERROR_METADATA = "runtime_status_error" diff --git a/prik/semantics/pyi2ir.py b/prik/semantics/pyi2ir.py index 524353a23..019d01890 100644 --- a/prik/semantics/pyi2ir.py +++ b/prik/semantics/pyi2ir.py @@ -58,7 +58,6 @@ OVERLOAD_KIND_METADATA, OVERLOAD_TARGET_METADATA, NATIVE_BY_VALUE_METADATA, - NATIVE_ACCESS_MODULE_METADATA, PYTHON_BOUND_POSITION_METADATA, PYTHON_METHOD_NAME_METADATA, PYTHON_STATIC_METADATA, @@ -165,7 +164,6 @@ class _Decorators: overload_generic: str | None = None bind_target: str | None = None native_abi: str | None = None - bind_module: str | None = None standalone: bool = False is_static: bool = False release_gil: bool = False @@ -1011,15 +1009,7 @@ def _apply_bind_decorator(self, parsed: _Decorators, node: ast.expr, context: st """Store one native symbol binding in decorator state, rejecting duplicates.""" if parsed.bind_target is not None: raise ValueError(f"Duplicate {context} bind decorator") - target = self._required_string_decorator_argument(node, "bind") - if "::" in target: - if self.native_language != "fortran" or context != ".pyi": - raise ValueError("qualified bind is only valid for Fortran module procedures") - parts = target.split("::") - if len(parts) != 2 or any(re.fullmatch(r"[A-Za-z][A-Za-z0-9_]*", part) is None for part in parts): - raise ValueError(f"bind requires a Fortran module::procedure name: {target!r}") - parsed.bind_module, target = parts - parsed.bind_target = target + parsed.bind_target = self._required_string_decorator_argument(node, "bind") def _apply_native_abi_decorator(self, parsed: _Decorators, node: ast.expr, context: str) -> None: """Retain the C ABI declared by an original Fortran declaration.""" @@ -1223,8 +1213,6 @@ def _validated_overload_candidate( for key in (RUNTIME_RELEASE_GIL_METADATA, RUNTIME_STATUS_ERROR_METADATA): if key in declaration.metadata: candidate.metadata[key] = deepcopy(declaration.metadata[key]) - if NATIVE_ACCESS_MODULE_METADATA in declaration.metadata: - candidate.metadata[NATIVE_ACCESS_MODULE_METADATA] = declaration.metadata[NATIVE_ACCESS_MODULE_METADATA] if isinstance(owner, SemanticModule): if generic_name is not None: @@ -3873,8 +3861,6 @@ def _visit_FunctionDef(self, node: ast.FunctionDef) -> None: """Convert a function or overload declaration.""" decorators = self.parser.decorators(node.decorator_list, context=".pyi") if decorators.prototype: - if decorators.bind_module is not None: - raise ValueError("qualified bind requires a Fortran module procedure") self.parser.module.prototypes.append( self.parser.prototype_def( node, @@ -3902,10 +3888,6 @@ def _visit_FunctionDef(self, node: ast.FunctionDef) -> None: # The same fact a Fortran source records, which a specification # function in a declaration expression is required to carry. function.metadata["fortran_attributes"] = [*function.metadata.get("fortran_attributes", ()), "pure"] - if decorators.bind_module is not None: - if decorators.standalone or decorators.prototype: - raise ValueError("qualified bind requires a Fortran module procedure") - function.metadata[NATIVE_ACCESS_MODULE_METADATA] = decorators.bind_module if decorators.overload_target is not None: self.parser._pending_overloads.append( _PendingOverload( diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index cf1760053..71e22a511 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -119,7 +119,7 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) assert "mpi_int: Final[Mpi_Datatype]" in types assert "mpi_status_ignore: Mpi_Status" in types assert all(f"class Mpi_{name}" in types for name in ("Comm", "Datatype", "Op", "Status")) - assert "AnyNative[" in interfaces and '@bind("mpi_f08::MPI_Send")' in interfaces + assert "AnyNative[" in interfaces and '@overload("mpi_send_f08")\ndef mpi_send(' in interfaces def show(flag: str) -> list[str]: return shlex.split(subprocess.check_output([mpifort, flag], text=True)) @@ -137,7 +137,7 @@ def show(flag: str) -> list[str]: ) assert result.native_build_plan is not None and not result.native_build_plan.compilation_units bridge = next(path for path in result.generated_sources if path.suffix == ".f90").read_text(encoding="utf-8") - assert "use mpi_f08, only:" in bridge + assert "use mpi_f08_interfaces, only:" in bridge assert "=> MPI_Allreduce" in bridge and "=> MPI_Send" in bridge env = os.environ.copy() env["PYTHONPATH"] = os.pathsep.join(filter(None, (str(result.output_dir), env.get("PYTHONPATH", "")))) diff --git a/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py b/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py index e3cbd23f4..2937ab568 100644 --- a/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py +++ b/tests/fortran/functions/end_to_end/test_fortran_export_symbol_workflow.py @@ -159,7 +159,6 @@ def test_facade_selection_and_contract_replay_share_generic_and_native_variable( owner_contract = (contract / "owner.pyi").read_text(encoding="utf-8") assert "from .owner import" in facade_contract assert '"run"' in facade_contract and '"marker"' in facade_contract - assert '@bind("facade::run")' in owner_contract assert "marker: Int32[()]" in owner_contract source = build_fortran_extension( diff --git a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py index 0b3c23a77..7cf0e28ac 100644 --- a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py +++ b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py @@ -7,7 +7,6 @@ from prik.cli import _read_export_symbols from prik.parsers.fortran import parse_fortran_file from prik.semantics.fortran2ir import fortran_module_to_semantic_module -from prik.semantics.models import NATIVE_ACCESS_MODULE_METADATA from prik.semantics.fortran_exports import select_fortran_export_symbols from prik.semantics.models import ( ProcedureOverloadSet, @@ -147,7 +146,7 @@ def test_selection_keeps_one_generic_with_its_specific_candidates(): assert selected.exported_names == ["solve"] -def test_facade_selection_retains_only_requested_native_owners_and_access_route(): +def test_facade_selection_retains_only_requested_native_owners(): """A facade allowlist selects owner declarations without publishing siblings.""" specific = _function("owner", "run_impl") owner = _module( @@ -174,9 +173,7 @@ def test_facade_selection_retains_only_requested_native_owners_and_access_route( assert [function.name for function in owner_selected.functions] == ["run_impl"] assert [variable.name for variable in owner_selected.variables] == ["marker"] assert [item.local_name for item in facade_selected.reexports] == ["run", "marker"] - candidate = owner_selected.overload_sets[0].procedures[0] - assert candidate.native_name == "run" - assert candidate.metadata[NATIVE_ACCESS_MODULE_METADATA] == "facade" + assert owner_selected.overload_sets[0].procedures[0].native_name == "run_impl" def test_external_root_cannot_satisfy_a_module_qualified_identity(): diff --git a/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py b/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py index 76a294b75..9752812ff 100644 --- a/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py +++ b/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py @@ -42,27 +42,21 @@ def test_convert_pyi_to_ir_accepts_parsed_pyi_ast_only(): convert_pyi_to_ir(source) -def test_module_scalar_storage_and_facade_bind_round_trip(): - """A contract retains native scalar storage and its public Fortran call route.""" - source = """from prik.contracts import Int32, bind +def test_module_scalar_storage_round_trips(): + """A contract retains which module scalars expose native storage.""" + source = """from prik.contracts import Int32 live: Int32[()] plain: Int32 - -@bind("facade::work") -def run() -> None: ... """ module = parse_pyi_text(source, module_name="owner") emitted = emit_module(module) assert "live: Int32[()]" in emitted assert "plain: Int32" in emitted - assert '@bind("facade::work")' in emitted replay = parse_pyi_text(emitted, module_name="owner") assert replay.variables[0].semantic_type.metadata["native_storage"] is True assert "native_storage" not in replay.variables[1].semantic_type.metadata - assert replay.functions[0].native_name == "work" - assert replay.functions[0].metadata["native_access_module"] == "facade" def test_pyi_parser_reports_unsupported_lines_and_invalid_helpers(): From 7ab48e9ea96cf15eee44aa0f9d1f9dcdd794c0a9 Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 18:03:52 +0100 Subject: [PATCH 08/47] Keep non-C logicals behind bind(C), parse IMPLICIT NONE specifiers, use all mpifort flags - Native-width logical scalars cross the generated bind(C) boundary as an address or a same-width interoperable integer; the logical kind is formed behind the boundary with c_f_pointer or transfer, so the bridge compiles under -std=f2018. - IMPLICIT NONE disables implicit typing only when plain or naming TYPE, in module and procedure scopes; NONE(EXTERNAL) keeps it. - The Open MPI test and tutorial take the compiler command, include directories, remaining compile flags, link arguments, and library directories from mpifort --showme instead of keeping only -I flags. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 2 + docs/user/tutorials/openmpi-f08.md | 11 ++-- prik/codegen/fortran/bridge.py | 66 +++++++++++++++---- prik/parsers/fortran/parser.py | 27 ++++++-- .../end_to_end/test_openmpi_f08.py | 15 +++-- .../codegen/test_logical_storage_boundary.py | 32 +++++++++ .../test_declaration_and_scope_regressions.py | 26 +++++++- .../parsing/test_project_scope_models.py | 20 +++++- 8 files changed, 168 insertions(+), 31 deletions(-) create mode 100644 tests/fortran/data_types/codegen/test_logical_storage_boundary.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 2153b28ac..cb6f7cb2a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,8 @@ release tags add a leading `v` to the package version. writes a pointer's current target. - A separate module-level `PARAMETER` statement types an undeclared name by the module's `IMPLICIT` rules and is rejected under `implicit none`. +- `IMPLICIT NONE (EXTERNAL)` no longer disables implicit typing; only plain + `IMPLICIT NONE` or a specifier list naming `TYPE` does. - Contributor test guidance focuses on supported behavior and meaningful validation boundaries after a feature is removed. diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 4d76959ba..2d36d664f 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -53,7 +53,8 @@ python3 -m prik generate --pyi \ ``` Build from the generated `contract/__init__.pyi`. Query the installed wrapper -compiler for its module paths and link arguments: +compiler for its compiler command, module and include directories, remaining +compile flags, and ordered link arguments: ```python import shlex @@ -66,11 +67,13 @@ def show(option): return shlex.split(subprocess.check_output(["mpifort", option], text=True)) -compile_flags = show("--showme:compile") +command, compile_flags = show("--showme:command"), show("--showme:compile") +include_dirs = [*show("--showme:incdirs"), *(flag[2:] for flag in compile_flags if flag.startswith("-I"))] build_pyi_extension( "contract/__init__.pyi", - input_compiler=show("--showme:command")[0], - native_include_dirs=[flag[2:] for flag in compile_flags if flag.startswith("-I")], + input_compiler=command[0], + native_include_dirs=list(dict.fromkeys(include_dirs)), + wrapper_fortran_flags=[*command[1:], *(flag for flag in compile_flags if not flag.startswith("-I"))], native_link_items=[NativeLinkItem("linker_argument", flag) for flag in show("--showme:link")], native_linker_language="fortran", output_name="prik_openmpi_f08", diff --git a/prik/codegen/fortran/bridge.py b/prik/codegen/fortran/bridge.py index d9b60a98d..cab3b33bd 100644 --- a/prik/codegen/fortran/bridge.py +++ b/prik/codegen/fortran/bridge.py @@ -4390,8 +4390,25 @@ def _lower_argument_descriptor(self, plan: ArgumentTransferPlan) -> tuple[Fortra ) def _parameter(self, plan: ArgumentTransferPlan, attributes: tuple[str, ...]) -> FortranParameter: - """Return one entrypoint ABI parameter from its completed transfer plan.""" - return FortranParameter(plan.entrypoint.parameter_name, self._scalar_argument_type(plan), attributes) + """Return one entrypoint ABI parameter from its completed transfer plan. + + A logical wider than ``c_bool`` is not interoperable, so its native-width + storage crosses as an address or a same-width integer and takes its + logical type only behind the ``bind(C)`` boundary. + """ + name = plan.entrypoint.parameter_name + if plan.scalar_logical_abi is ScalarLogicalABI.NATIVE_KIND_STORAGE: + if plan.entrypoint.handoff_mode is ArgumentHandoffMode.VALUE: + return FortranParameter(f"bound_{name}", self._logical_storage_integer(plan), ("value",)) + return FortranParameter(f"bound_{name}", "type(c_ptr)", ("value",)) + return FortranParameter(name, self._scalar_argument_type(plan), attributes) + + @staticmethod + def _logical_storage_integer(plan: ArgumentTransferPlan) -> str: + """Spell the interoperable integer that carries a logical's native-width storage.""" + if not plan.native_storage_c_type: + raise ValueError(f"Logical argument {plan.owner_path!r} has no native storage type") + return f"integer(c_{plan.native_storage_c_type})" @staticmethod def _scalar_argument_type(plan: ArgumentTransferPlan) -> str: @@ -5331,6 +5348,12 @@ def _logical_scalar_argument_declarations( """Declare exact-kind native locals selected by scalar logical policy.""" declarations = [] for argument in plan.arguments: + if self._required_logical_storage(argument): + attributes = () if argument.entrypoint.handoff_mode is ArgumentHandoffMode.VALUE else ("pointer",) + declarations.append( + FortranDeclaration(argument.entrypoint.parameter_name, argument.scalar_native_type, attributes) + ) + continue if argument.scalar_logical_abi is not ScalarLogicalABI.NATIVE_KIND_COPY: continue if not argument.scalar_native_type: @@ -5343,20 +5366,37 @@ def _logical_scalar_argument_declarations( ) return tuple(declarations) + @staticmethod + def _required_logical_storage(argument: ArgumentTransferPlan) -> bool: + """Return whether a required logical takes its native type behind the boundary.""" + return ( + argument.scalar_logical_abi is ScalarLogicalABI.NATIVE_KIND_STORAGE + and argument.entrypoint.optional_mode is OptionalMode.REQUIRED + ) + def _logical_scalar_argument_initializers( self, plan: FunctionPlan, - ) -> tuple[FortranAssignment, ...]: - """Copy required C Boolean values into their exact native kinds.""" - return tuple( - FortranAssignment( - f"{argument.entrypoint.parameter_name}_native", - CodeExpression(argument.entrypoint.parameter_name), - ) - for argument in plan.arguments - if argument.scalar_logical_abi is ScalarLogicalABI.NATIVE_KIND_COPY - and argument.entrypoint.optional_mode is OptionalMode.REQUIRED - ) + ) -> tuple[FortranAssignment | FortranCall, ...]: + """Give required logicals their exact native kinds behind the C boundary. + + A copied C Boolean is assigned to its native local; native-width + storage is associated with, or reinterpreted as, the native logical. + """ + nodes: list[FortranAssignment | FortranCall] = [] + for argument in plan.arguments: + name = argument.entrypoint.parameter_name + if self._required_logical_storage(argument): + if argument.entrypoint.handoff_mode is ArgumentHandoffMode.VALUE: + nodes.append(FortranAssignment(name, CodeExpression(f"transfer(bound_{name}, {name})"))) + else: + nodes.append(FortranCall("c_f_pointer", (CodeExpression(f"bound_{name}"), CodeExpression(name)))) + elif ( + argument.scalar_logical_abi is ScalarLogicalABI.NATIVE_KIND_COPY + and argument.entrypoint.optional_mode is OptionalMode.REQUIRED + ): + nodes.append(FortranAssignment(f"{name}_native", CodeExpression(name))) + return tuple(nodes) def _logical_scalar_argument_finalizers( self, diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index fd590ce60..e63e8ada6 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -416,6 +416,23 @@ class _ProcedureState: _IMPLICIT_LETTER_SPEC = re.compile(r"^(?P.+?)\s*\((?P[^()]*)\)$") +_IMPLICIT_NONE = re.compile(r"^none\s*(?:\((?P[^()]*)\))?\s*$", re.IGNORECASE) + + +def _implicit_none_forbids_typing(body: str) -> bool | None: + """Return whether an ``IMPLICIT NONE`` body disables implicit typing, or None for another body. + + Plain ``NONE``, an empty specifier list, or one naming ``TYPE`` disables + it; ``NONE(EXTERNAL)`` alone only requires external procedures to be + declared. + """ + match = _IMPLICIT_NONE.match(body.strip()) + if match is None: + return None + specs = match.group("specs") + if specs is None or not specs.strip(): + return True + return "type" in {spec.strip().casefold() for spec in specs.split(",")} @dataclass @@ -427,8 +444,9 @@ class _ImplicitTyping: def record(self, body: str) -> bool: """Apply one IMPLICIT statement body, or return False when it is not understood.""" - if re.match(r"^none\b", body, flags=re.IGNORECASE): - self.none = True + forbids_typing = _implicit_none_forbids_typing(body) + if forbids_typing is not None: + self.none = self.none or forbids_typing return True mappings = [] for item in split_csv(body): @@ -4665,9 +4683,10 @@ def _handle_proc_implicit_line( `_finalize_proc` can require every argument to have an explicit declaration. """ - if not re.match(r"^implicit\b", line, flags=re.IGNORECASE): + implicit = re.match(r"^implicit\b\s*(?P.*)$", line, flags=re.IGNORECASE) + if not implicit: return False - if re.match(r"^implicit\s+none\b", line, flags=re.IGNORECASE): + if _implicit_none_forbids_typing(implicit.group("body")): proc_state.implicit_none = True return True diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index 71e22a511..83d51b813 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -124,12 +124,16 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) def show(flag: str) -> list[str]: return shlex.split(subprocess.check_output([mpifort, flag], text=True)) - compile_flags, link_flags = show("--showme:compile"), show("--showme:link") + # The wrapper compiler's command may carry its own flags, and its compile + # flags are more than include directories; keep every one of them. + command, compile_flags = show("--showme:command"), show("--showme:compile") + include_dirs = [*show("--showme:incdirs"), *(flag[2:] for flag in compile_flags if flag.startswith("-I"))] result = build_pyi_extension( contract / "__init__.pyi", - input_compiler=show("--showme:command")[0], - native_include_dirs=[flag[2:] for flag in compile_flags if flag.startswith("-I")], - native_link_items=[NativeLinkItem("linker_argument", flag) for flag in link_flags], + input_compiler=command[0], + native_include_dirs=list(dict.fromkeys(include_dirs)), + wrapper_fortran_flags=[*command[1:], *(flag for flag in compile_flags if not flag.startswith("-I"))], + native_link_items=[NativeLinkItem("linker_argument", flag) for flag in show("--showme:link")], native_linker_language="fortran", output_name="prik_openmpi_f08", output_dir=tmp_path / "extension", @@ -141,8 +145,7 @@ def show(flag: str) -> list[str]: assert "=> MPI_Allreduce" in bridge and "=> MPI_Send" in bridge env = os.environ.copy() env["PYTHONPATH"] = os.pathsep.join(filter(None, (str(result.output_dir), env.get("PYTHONPATH", "")))) - libdirs = [flag[2:] for flag in link_flags if flag.startswith("-L")] - env["LD_LIBRARY_PATH"] = os.pathsep.join((*libdirs, env.get("LD_LIBRARY_PATH", ""))) + env["LD_LIBRARY_PATH"] = os.pathsep.join((*show("--showme:libdirs"), env.get("LD_LIBRARY_PATH", ""))) completed = subprocess.run( [launcher, "-n", "2", sys.executable, str(RUNTIME)], env=env, diff --git a/tests/fortran/data_types/codegen/test_logical_storage_boundary.py b/tests/fortran/data_types/codegen/test_logical_storage_boundary.py new file mode 100644 index 000000000..508f425c9 --- /dev/null +++ b/tests/fortran/data_types/codegen/test_logical_storage_boundary.py @@ -0,0 +1,32 @@ +"""Native-width logical storage stays interoperable at the generated bind(C) boundary.""" + +from tests.fortran._support.ownership_policy import parse_pyi_text +from prik.pipeline.wrapper import WrapperGenerator +from prik.planning import WrapperPlanner +from prik.policy.completion import complete_semantic_policies + + +def test_wider_logical_crosses_bind_c_as_an_address_or_same_width_integer(): + """The logical kind exists only behind the boundary, for reference and value transport alike.""" + module = parse_pyi_text( + """from prik.contracts import Bool32, Int32, Returns + +def flip(flag: Bool32) -> Returns["flag", Bool32]: ... + +def count_true(flag: Bool32) -> Int32: ... +""", + module_name="logical_boundary", + ) + complete_semantic_policies(module) + + bridge = next( + source.text + for source in WrapperGenerator().generate(WrapperPlanner().build(module)).sources + if source.path.suffix == ".f90" + ) + + assert "type(c_ptr), value :: bound_flag" in bridge + assert "call c_f_pointer(bound_flag, flag)" in bridge + assert "integer(c_int32_t), value :: bound_flag" in bridge + assert "flag = transfer(bound_flag, flag)" in bridge + assert "logical(kind=4), pointer :: flag" in bridge diff --git a/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py b/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py index 0c4873584..227ccd788 100644 --- a/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py +++ b/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py @@ -2,7 +2,7 @@ import pytest from pathlib import Path -from prik.parsers.fortran import FortranParseError +from prik.parsers.fortran import FortranParseError, parse_fortran_file from prik.parsers.fortran.models import ( FortranArgument, FortranDerivedType, @@ -735,3 +735,27 @@ def test_derived_type_collection_retains_sibling_and_nested_scope_contexts(): ("file", None), ("module", "owner_mod"), ] + + +@pytest.mark.parametrize( + ("statement", "forbids_typing"), + [ + pytest.param("implicit none", True, id="plain"), + pytest.param("implicit none (type)", True, id="type"), + pytest.param("implicit none (type, external)", True, id="type-and-external"), + pytest.param("implicit none (external)", False, id="external-only"), + ], +) +def test_implicit_none_specifiers_decide_whether_undeclared_dummies_are_typed(statement: str, forbids_typing: bool): + """Only NONE or NONE(TYPE) forbids implicit typing; NONE(EXTERNAL) keeps it.""" + source = f""" +subroutine scale(n) + {statement} +end subroutine scale +""" + if forbids_typing: + with pytest.raises(FortranParseError, match="implicit none is active"): + parse_fortran_file(source) + return + (argument,) = parse_fortran_file(source).procedures[0].arguments + assert argument.base_type == "integer" diff --git a/tests/fortran/modules/parsing/test_project_scope_models.py b/tests/fortran/modules/parsing/test_project_scope_models.py index f4e852bbe..b4ccf8343 100644 --- a/tests/fortran/modules/parsing/test_project_scope_models.py +++ b/tests/fortran/modules/parsing/test_project_scope_models.py @@ -98,12 +98,13 @@ def test_separate_parameter_statement_types_an_undeclared_name_by_module_implici assert all(variable.is_parameter for variable in module.variables) -def test_separate_parameter_statement_under_implicit_none_requires_a_declaration(): +@pytest.mark.parametrize("statement", ["implicit none", "implicit none (type)", "implicit none (type, external)"]) +def test_separate_parameter_statement_under_implicit_none_requires_a_declaration(statement: str): with pytest.raises(FortranParseError, match="implicit none is active") as error: parse_fortran_file( - """ + f""" module strict_constants - implicit none + {statement} parameter (undeclared = 3) end module strict_constants """ @@ -112,6 +113,19 @@ def test_separate_parameter_statement_under_implicit_none_requires_a_declaration assert error.value.code == "PARSE_UNKNOWN_PARAMETER_TYPE" +def test_implicit_none_external_keeps_implicit_typing_for_a_separate_parameter(): + module = parse_fortran_file( + """ +module external_only + implicit none (external) + parameter (n = 4) +end module external_only +""" + ).modules[0] + + assert fortran_module_to_semantic_module(module).variables[0].semantic_type.name == "Int32" + + def test_submodule_types_interfaces_and_project_dependencies_attach_to_public_models(): code = """ submodule (ancestor_mod:parent_mod) child_mod From 8709d04bb887fb2107546515b5eeb29b1b38a885 Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 18:39:17 +0100 Subject: [PATCH 09/47] Discover Fortran module sources from entry files --module-source-dir (and build_fortran_extension(module_source_dirs=...)) lets a multi-module Fortran library be supplied by its entry source: PRIK follows each use, after preprocessing and as the parser reads it, to the one source under the given directories that defines the module, transitively, and reads it dependencies first. The directories are only scanned for module lines to locate candidates. A used module with no defining source, or with several, is an error instead of an undefined type. The Open MPI test and tutorial now name only mpi-f08.F90, so the module layout of each Open MPI version is found rather than listed. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 7 + docs/user/reference/cli-commands.md | 12 ++ docs/user/reference/python-api.md | 4 + docs/user/tutorials/openmpi-f08.md | 9 +- prik/cli.py | 49 +++++- prik/parsers/fortran/module_sources.py | 145 ++++++++++++++++++ prik/pipeline/build.py | 14 ++ .../end_to_end/test_openmpi_f08.py | 8 +- .../native/module_discovery/api/api_entry.f90 | 9 ++ .../interfaces/types-layer.f90 | 4 + .../shared/base-definitions.f90 | 7 + .../test_module_source_discovery.py | 52 +++++++ .../parsing/test_module_source_resolution.py | 58 +++++++ 13 files changed, 367 insertions(+), 11 deletions(-) create mode 100644 prik/parsers/fortran/module_sources.py create mode 100644 tests/fortran/modules/end_to_end/fixtures/native/module_discovery/api/api_entry.f90 create mode 100644 tests/fortran/modules/end_to_end/fixtures/native/module_discovery/interfaces/types-layer.f90 create mode 100644 tests/fortran/modules/end_to_end/fixtures/native/module_discovery/shared/base-definitions.f90 create mode 100644 tests/fortran/modules/end_to_end/test_module_source_discovery.py create mode 100644 tests/fortran/modules/parsing/test_module_source_resolution.py diff --git a/CHANGELOG.md b/CHANGELOG.md index cb6f7cb2a..c812001ce 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,13 @@ release tags add a leading `v` to the package version. ## Unreleased +- Fortran source commands accept `--module-source-dir DIR`, and + `build_fortran_extension` accepts `module_source_dirs`: from the given entry + sources, PRIK follows each `use` to the source under those directories that + defines the module and reads it too, so a multi-module library such as Open + MPI's `mpi_f08` is supplied by its entry file. A used module with no source, + or with several, is an error. + - Mutable fixed-storage Fortran module scalars expose native-backed rank-zero NumPy views, including fixed-length character bytes; primitive and fixed character value dummies accept matching rank-zero storage as well as scalar diff --git a/docs/user/reference/cli-commands.md b/docs/user/reference/cli-commands.md index cc702d0a2..a64255b9c 100644 --- a/docs/user/reference/cli-commands.md +++ b/docs/user/reference/cli-commands.md @@ -64,6 +64,7 @@ The default build accepts either one or more Fortran or supported C source | `--version` | Prints the installed PRIK version and exits. | | `--language {fortran,c}` | Selects the source or source-free contract language explicitly. C source and C-native contracts require `c`. | | `--build-manifest PATH` | Replays a saved `prik-build.json`. It does not generate one. | +| `--module-source-dir DIR` | Fortran only. Finds the sources of modules the inputs `use` under DIR, recursively, and reads them too. Repeat to search several directories. | | `--jobs N` | Limits concurrent compiler processes. The default uses available CPUs. | Compiled wrapper builds support Fortran and the documented C subset — @@ -79,6 +80,17 @@ source files can usually be inferred from their suffix; lists the accepted ones. C files, directories, and unknown suffixes require `--language c`. +A Fortran `use` names a module, not a file. Give only the entry source and +point `--module-source-dir` at the tree that holds the rest: PRIK follows each +`use`, after preprocessing, to the one source under those directories that +defines the module, transitively, and reads those sources as if they were +listed, dependencies first. Intrinsic modules need no source. A used module +that no source defines, or that several sources define, fails the command. + +```bash +python3 -m prik generate --pyi src/api/api.f90 --module-source-dir src --out contract +``` + ## Wrapper builds A positional Fortran or C source is both a semantic input and a native diff --git a/docs/user/reference/python-api.md b/docs/user/reference/python-api.md index 0ad914b12..65be18fef 100644 --- a/docs/user/reference/python-api.md +++ b/docs/user/reference/python-api.md @@ -115,6 +115,10 @@ build = build_fortran_extension( ) ``` +Pass `module_source_dirs=[...]` to read the sources of the modules the given +sources `use` from those directories, as `--module-source-dir` does, so a +multi-module library can be supplied by its entry source. + For an authored C semantic contract, use `build_pyi_extension` with `native_language="c"` and `native_c_sources=[...]`. [C Pointers, Arrays, and Strings](../guide/c/pointers-arrays-and-strings.md#author-a-contract-for-pointers-and-arrays) diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 2d36d664f..4b61c6c92 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -11,8 +11,10 @@ publication: reviewed # Wrap Open MPI `mpi_f08` Use a configured Open MPI source tree and the corresponding installed Open MPI -toolchain. PRIK reads the interface and type sources to generate a contract; -the extension compiles against the installed modules and libraries. +toolchain. PRIK reads `mpi-f08.F90` and finds the sources of the modules it +uses in the source tree, whatever their names and layout in your Open MPI +version, to generate a contract; the extension compiles against the installed +modules and libraries. Set `PRIK_OPENMPI_SOURCE` to the matching Open MPI source root and `PRIK_OPENMPI_BUILD` to its configured build root. The build must contain its @@ -41,9 +43,8 @@ mpi_f08::MPI_STATUS_IGNORE EOF python3 -m prik generate --pyi \ - "$PRIK_OPENMPI_SOURCE/ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-types.F90" \ - "$PRIK_OPENMPI_SOURCE/ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-interfaces.F90" \ "$PRIK_OPENMPI_SOURCE/ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90" \ + --module-source-dir "$PRIK_OPENMPI_SOURCE" \ --export-symbols exports.txt --out contract --compiler mpifort \ -I "$PRIK_OPENMPI_BUILD" \ -I "$PRIK_OPENMPI_BUILD/ompi/mpi/fortran/use-mpi-f08" \ diff --git a/prik/cli.py b/prik/cli.py index 6f1461e19..aed47ddee 100644 --- a/prik/cli.py +++ b/prik/cli.py @@ -460,6 +460,7 @@ class _SemanticPipelineContext: refresh_fortran_type_probe: bool = False assume_intent_in_scalars: bool = False export_symbols: tuple[str, ...] | None = None + module_source_dirs: tuple[Path, ...] = () @dataclass(frozen=True) @@ -501,6 +502,7 @@ def _converted_semantic_files( refresh_fortran_type_probe: bool = False, assume_intent_in_scalars: bool = False, export_symbols: tuple[str, ...] | None = None, + module_source_dirs: tuple[Path, ...] = (), ) -> _ConvertedSemanticSources: context = _SemanticPipelineContext( paths=paths, @@ -516,6 +518,7 @@ def _converted_semantic_files( refresh_fortran_type_probe=refresh_fortran_type_probe, assume_intent_in_scalars=assume_intent_in_scalars, export_symbols=export_symbols, + module_source_dirs=module_source_dirs, ) pipeline = _SOURCE_SEMANTIC_PIPELINES[language] parsed = pipeline.parser(context) @@ -551,6 +554,7 @@ def _semantic_report( refresh_fortran_type_probe: bool = False, assume_intent_in_scalars: bool = False, export_symbols: tuple[str, ...] | None = None, + module_source_dirs: tuple[Path, ...] = (), ) -> dict[str, dict]: preprocessing = preprocessing or PreprocessingConfig() converted = _converted_semantic_files( @@ -564,6 +568,7 @@ def _semantic_report( refresh_fortran_type_probe=refresh_fortran_type_probe, assume_intent_in_scalars=assume_intent_in_scalars, export_symbols=export_symbols, + module_source_dirs=module_source_dirs, ) return _semantic_payload_for_converted_files( converted.files, @@ -623,9 +628,18 @@ def _convert_c_semantic_sources( def _parse_fortran_semantic_sources(context: _SemanticPipelineContext) -> _ParsedSemanticSources: if not context.source_paths: return _ParsedSemanticSources(context.source_paths, []) + source_paths = context.source_paths + if context.module_source_dirs: + from prik.parsers.fortran.module_sources import resolve_fortran_module_sources + + source_paths = resolve_fortran_module_sources( + source_paths, + context.module_source_dirs, + lambda path: _fortran_source_for_path(path, context.preprocessing)[0], + ) return _ParsedSemanticSources( - context.source_paths, - _parse_fortran_source_files(list(context.source_paths), context.preprocessing), + source_paths, + _parse_fortran_source_files(list(source_paths), context.preprocessing), ) @@ -1078,6 +1092,11 @@ def _validate_pyi_wrapper_options(args: argparse.Namespace, parser: argparse.Arg "--export-symbols selects the public surface while reading native source; a semantic .pyi " "contract already states its public surface in __all__" ) + if getattr(args, "module_source_dirs", None): + parser.error( + "--module-source-dir finds Fortran module sources while reading native source; a semantic .pyi " + "contract already states the modules it declares" + ) if not getattr(args, "external_native_implementation", False) and not ( getattr(args, "native_fortran_sources", None) or getattr(args, "native_c_sources", None) @@ -1116,6 +1135,7 @@ def _validate_manifest_wrapper_options(args: argparse.Namespace, parser: argpars getattr(args, "strict_wrapper_names", False) or getattr(args, "assume_intent_in_scalars", False) or getattr(args, "export_symbols", None) + or getattr(args, "module_source_dirs", None) or _wrapper_compile_options_used(args) ): parser.error("--build-manifest replays saved wrapper behavior and compiler flags") @@ -1322,9 +1342,20 @@ def _validate_main_options(args: argparse.Namespace, parser: argparse.ArgumentPa _validate_output_options(args, parser) _complete_export_symbol_options(args, parser) + _validate_module_source_dirs(args, parser) return args.print_limit +def _validate_module_source_dirs(args: argparse.Namespace, parser: argparse.ArgumentParser) -> None: + """Require existing directories and a Fortran source language for module discovery.""" + directories = getattr(args, "module_source_dirs", None) or () + if directories and args.language != "fortran": + parser.error("--module-source-dir finds Fortran module sources; it does not apply to C") + missing = [directory for directory in directories if not Path(directory).is_dir()] + if missing: + parser.error(f"--module-source-dir is not a directory: {', '.join(missing)}") + + def _c_type_facts_for_stages(args: argparse.Namespace, preprocessing: PreprocessingConfig): if args.language != "c" or not _has_semantic_stage(args): return None @@ -1343,6 +1374,8 @@ def _semantic_stage_options( options["assume_intent_in_scalars"] = True if getattr(args, "_resolved_export_symbols", None) is not None: options["export_symbols"] = args._resolved_export_symbols + if getattr(args, "module_source_dirs", None): + options["module_source_dirs"] = tuple(Path(path) for path in args.module_source_dirs) return options @@ -1729,6 +1762,7 @@ def record_total_build_time(elapsed: float) -> None: positional_only=getattr(args, "positional_only", False), assume_intent_in_scalars=getattr(args, "assume_intent_in_scalars", False), export_symbols=getattr(args, "_resolved_export_symbols", None), + module_source_dirs=getattr(args, "module_source_dirs", None), compile_input_sources=not getattr(args, "no_compile_input_sources", False), standard_logicals=getattr(args, "standard_logicals", True), native_fortran_sources=getattr(args, "native_fortran_sources", None), @@ -2379,6 +2413,16 @@ def _add_semantic_interpretation_options( "conservative intent(inout) default, so its value is not returned; a declared intent always wins" ), ) + group.add_argument( + "--module-source-dir", + dest="module_source_dirs", + metavar="DIR", + action="append", + help=( + "Search DIR recursively for the Fortran sources of modules the given sources use, and read " + "them too, transitively; repeat to add directories" + ), + ) group.add_argument( "--export-symbols", metavar="FILE", @@ -2630,6 +2674,7 @@ def _add_diagnostic_controls(group: argparse._ArgumentGroup, *, allow_verbose: b "public_includes": None, "private_includes": None, "export_symbols": None, + "module_source_dirs": None, } diff --git a/prik/parsers/fortran/module_sources.py b/prik/parsers/fortran/module_sources.py new file mode 100644 index 000000000..9203c1799 --- /dev/null +++ b/prik/parsers/fortran/module_sources.py @@ -0,0 +1,145 @@ +"""Discover the Fortran sources that define the modules a project uses. + +A Fortran ``use`` names a module, not a file, and no rule ties the two: a +module may live in any file under any directory. Given the sources a caller +names and the directories to search, this resolver follows each ``use`` to the +one file that defines that module, transitively, so a project can be supplied +by its entry file alone. + +Which modules a source defines and uses are parser facts, read here exactly as +compile ordering reads them. The directory index only locates candidate files +by their ``module`` lines, since parsing every file under a search root to +find one module would be needlessly slow. +""" + +from __future__ import annotations + +from collections.abc import Callable, Iterable, Sequence +from pathlib import Path +import re + +from prik.parsers.fortran.models import FortranFile, FortranParseError +from prik.parsers.fortran.parser import FortranParser +from prik.parsers.fortran.scope import used_module_names + + +# Suffixes a Fortran compiler accepts as free- or fixed-form source. +_FORTRAN_SOURCE_SUFFIXES = frozenset({".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08", ".fpp"}) +_MODULE_LINE = re.compile( + r"^[ \t]*module[ \t]+(?!(?:procedure|function|subroutine|pure|impure|elemental|recursive)\b)" + r"(?P[a-z][a-z0-9_]*)[ \t]*(?:!.*)?$", + re.IGNORECASE | re.MULTILINE, +) +# Modules the processor supplies without a source file. +_INTRINSIC_MODULES = frozenset( + { + "iso_c_binding", + "iso_fortran_env", + "ieee_arithmetic", + "ieee_exceptions", + "ieee_features", + "omp_lib", + "omp_lib_kinds", + "openacc", + } +) + + +def resolve_fortran_module_sources( + entries: Sequence[Path], + search_dirs: Iterable[Path], + read_source: Callable[[Path], str], +) -> tuple[Path, ...]: + """Return ``entries`` with the sources of every module they use, dependencies first. + + ``read_source`` returns a file's preprocessed text, so a ``use`` inside an + inactive conditional block is not followed. A used module that no source + read so far defines must be defined by exactly one Fortran source under + ``search_dirs``; otherwise a :class:`FortranParseError` names the module + and the source that uses it. + """ + candidates = _module_candidates(search_dirs) + parser = FortranParser() + facts: dict[Path, tuple[set[str], set[str]]] = {} + owners: dict[str, Path] = {} + + def read(path: Path) -> tuple[set[str], set[str]]: + if path not in facts: + parsed = parser.parse_file(read_source(path), filename=str(path)) + facts[path] = _defined_and_used_modules(parsed) + for name in facts[path][0]: + owners.setdefault(name, path) + return facts[path] + + for entry in entries: + read(entry.resolve()) + ordered: list[Path] = [] + visiting: set[Path] = set() + + def visit(path: Path) -> None: + if path in ordered or path in visiting: + return + visiting.add(path) + defined, used = read(path) + for name in sorted(used - defined - _INTRINSIC_MODULES): + dependency = owners.get(name) or _defining_source(name, candidates, path, read) + visit(dependency) + visiting.discard(path) + ordered.append(path) + + for entry in entries: + visit(entry.resolve()) + originals = {entry.resolve(): entry for entry in entries} + return tuple(originals.get(path, path) for path in ordered) + + +def _defining_source( + name: str, + candidates: dict[str, list[Path]], + user: Path, + read: Callable[[Path], tuple[set[str], set[str]]], +) -> Path: + """Return the one searched source whose parsed modules define ``name``, or raise.""" + defining = [path for path in candidates.get(name, ()) if name in read(path)[0]] + if len(defining) == 1: + return defining[0] + if not defining: + raise FortranParseError( + f"No Fortran source defines module '{name}' used by {user}; " + "add the directory that contains it as a module source directory.", + filename=str(user), + code="PARSE_MODULE_SOURCE_NOT_FOUND", + ) + listed = ", ".join(str(path) for path in defining) + raise FortranParseError( + f"Module '{name}' used by {user} is defined by several sources ({listed}); " + "narrow the module source directories to one of them.", + filename=str(user), + code="PARSE_AMBIGUOUS_MODULE_SOURCE", + ) + + +def _module_candidates(search_dirs: Iterable[Path]) -> dict[str, list[Path]]: + """Map each module name to the Fortran sources under ``search_dirs`` with a matching ``module`` line.""" + candidates: dict[str, list[Path]] = {} + for directory in search_dirs: + for path in sorted(Path(directory).rglob("*")): + if path.suffix.casefold() not in _FORTRAN_SOURCE_SUFFIXES or not path.is_file(): + continue + text = path.read_text(encoding="utf-8", errors="replace") + for match in _MODULE_LINE.finditer(text): + paths = candidates.setdefault(match.group("name").casefold(), []) + if path.resolve() not in paths: + paths.append(path.resolve()) + return candidates + + +def _defined_and_used_modules(parsed: FortranFile) -> tuple[set[str], set[str]]: + """Return the modules one parsed source defines and every module it uses.""" + defined = {str(module.name).casefold() for module in parsed.modules} + used: set[str] = set() + for owner in (*parsed.modules, *parsed.submodules, *parsed.programs, *parsed.procedures): + used.update(used_module_names(owner)) + # A submodule extends the module it names first, which must be available. + used.update(str(submodule.ancestor or submodule.parent).casefold() for submodule in parsed.submodules) + return defined, used diff --git a/prik/pipeline/build.py b/prik/pipeline/build.py index ec5ef2f0a..a9d2da756 100644 --- a/prik/pipeline/build.py +++ b/prik/pipeline/build.py @@ -39,6 +39,7 @@ from prik.parsers.c import parse_c_file from prik.parsers.c.cli import attach_preprocessing_recipe from prik.parsers.fortran.parser import parse_fortran_project +from prik.parsers.fortran.module_sources import resolve_fortran_module_sources from prik.parsers.fortran.scope import used_module_names from prik.preprocessing.probes.fortran_types import ( evaluate_fortran_type_facts, @@ -3662,6 +3663,7 @@ def build_fortran_extension( positional_only: bool = False, assume_intent_in_scalars: bool = False, export_symbols: Iterable[str] | None = None, + module_source_dirs: Iterable[str | Path] | None = None, fortran_type_report=None, fortran_type_probe_runner: list[str] | None = None, fortran_type_probe_cache_dir: str | Path | None = None, @@ -3734,6 +3736,12 @@ def build_fortran_extension( from the source universe. Signature dependencies remain available but are not added to the callable surface. A generated semantic contract records the corresponding Python surface in ``__all__``. + module_source_dirs + Directories searched recursively for the sources of modules that the + given sources ``use``. Each found source is read, and compiled with + ``compile_input_sources``, as if it had been passed, transitively and + before the sources that use it; a used module found in no directory, + or in several sources, is an error. fortran_type_report, fortran_type_probe_runner, fortran_type_probe_cache_dir, refresh_fortran_type_probe Optional controls for compiler-probed Fortran type facts used while @@ -3800,6 +3808,12 @@ def build_fortran_extension( output_path, shared_library_output_path = _wrapper_output_paths(output_dir) output_path.mkdir(parents=True, exist_ok=True) preprocessing = preprocessing or _default_preprocessing_config() + if module_source_dirs: + source_paths = resolve_fortran_module_sources( + source_paths, + tuple(Path(directory) for directory in module_source_dirs), + lambda path: _fortran_source_and_dependencies(path, preprocessing)[0], + ) supplemental_source_paths = tuple(Path(path) for path in (native_fortran_sources or ())) input_implementation_paths = source_paths if compile_input_sources else () implementation_source_paths = (*input_implementation_paths, *supplemental_source_paths) diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index 83d51b813..ff0f380c0 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -46,12 +46,10 @@ def _configured_openmpi() -> tuple[Path, Path, str, str, str]: launcher = os.environ.get("PRIK_OPENMPI_LAUNCHER") or shutil.which("orterun") or shutil.which("mpirun") if not mpifort or not launcher: pytest.skip("Open MPI Fortran compiler wrapper and launcher are required") + # Only the entry source is named; the modules it uses are discovered. for path in ( - source / "ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-types.F90", - source / "ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-interfaces.F90", source / "ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90", build / "ompi/mpi/fortran/configure-fortran-output.h", - build / "ompi/mpi/fortran/use-mpi-f08/sizeof_f08.h", ): if not path.is_file(): pytest.skip(f"configured Open MPI semantic input is unavailable: {path}") @@ -91,9 +89,9 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) "prik", "generate", "--pyi", - str(source / "ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-types.F90"), - str(source / "ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-interfaces.F90"), str(source / "ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90"), + "--module-source-dir", + str(source), "--export-symbols", str(exports), "--out", diff --git a/tests/fortran/modules/end_to_end/fixtures/native/module_discovery/api/api_entry.f90 b/tests/fortran/modules/end_to_end/fixtures/native/module_discovery/api/api_entry.f90 new file mode 100644 index 000000000..b5072b346 --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/native/module_discovery/api/api_entry.f90 @@ -0,0 +1,9 @@ +module api + use api_types + implicit none +contains + subroutine touch(h) + type(handle_t), intent(inout) :: h + h%val = h%val + 1 + end subroutine touch +end module api diff --git a/tests/fortran/modules/end_to_end/fixtures/native/module_discovery/interfaces/types-layer.f90 b/tests/fortran/modules/end_to_end/fixtures/native/module_discovery/interfaces/types-layer.f90 new file mode 100644 index 000000000..39896203d --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/native/module_discovery/interfaces/types-layer.f90 @@ -0,0 +1,4 @@ +module api_types + use base_types + implicit none +end module api_types diff --git a/tests/fortran/modules/end_to_end/fixtures/native/module_discovery/shared/base-definitions.f90 b/tests/fortran/modules/end_to_end/fixtures/native/module_discovery/shared/base-definitions.f90 new file mode 100644 index 000000000..046c33bba --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/native/module_discovery/shared/base-definitions.f90 @@ -0,0 +1,7 @@ +module base_types + use, intrinsic :: iso_c_binding, only: c_int + implicit none + type :: handle_t + integer(c_int) :: val = 0 + end type handle_t +end module base_types diff --git a/tests/fortran/modules/end_to_end/test_module_source_discovery.py b/tests/fortran/modules/end_to_end/test_module_source_discovery.py new file mode 100644 index 000000000..dfd17b23e --- /dev/null +++ b/tests/fortran/modules/end_to_end/test_module_source_discovery.py @@ -0,0 +1,52 @@ +"""An entry source is enough: the modules it uses are found under search directories.""" + +from __future__ import annotations + +import subprocess +import sys +from pathlib import Path + +import numpy as np +import pytest + +from prik import build_fortran_extension +from tests.fortran._support.wrapper_build import _import_from_build_dir + +pytestmark = pytest.mark.fortran_end_to_end + +PROJECT = Path(__file__).parent / "fixtures" / "native" / "module_discovery" +ENTRY = PROJECT / "api" / "api_entry.f90" + + +def test_entry_source_discovers_used_modules_for_contract_and_build(tmp_path: Path): + """Modules named by `use`, in any file and directory, reach both the contract and the build.""" + contract = tmp_path / "contract" + subprocess.run( + [ + sys.executable, + "-m", + "prik", + "generate", + "--pyi", + str(ENTRY), + "--module-source-dir", + str(PROJECT), + "--out", + str(contract), + ], + check=True, + capture_output=True, + text=True, + ) + assert "class Handle_T:" in (contract / "base_types.pyi").read_text(encoding="utf-8") + + result = build_fortran_extension( + ENTRY, + module_source_dirs=[PROJECT], + output_name="discovered_api", + output_dir=tmp_path / "build", + ) + root = _import_from_build_dir(result.module_name, result.output_dir) + handle = root.base_types.Handle_T() + root.api.touch(handle) + assert handle.val == np.int32(1) diff --git a/tests/fortran/modules/parsing/test_module_source_resolution.py b/tests/fortran/modules/parsing/test_module_source_resolution.py new file mode 100644 index 000000000..f22316533 --- /dev/null +++ b/tests/fortran/modules/parsing/test_module_source_resolution.py @@ -0,0 +1,58 @@ +"""Module source resolution follows `use` from entry sources to the files that define each module.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from prik.parsers.fortran import FortranParseError +from prik.parsers.fortran.module_sources import resolve_fortran_module_sources + + +def _write(root: Path, relative: str, text: str) -> Path: + path = root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + return path + + +def _resolve(entries, search): + return resolve_fortran_module_sources(entries, search, lambda path: path.read_text(encoding="utf-8")) + + +def test_used_modules_resolve_transitively_dependencies_first(tmp_path: Path): + """File names and directories are irrelevant; intrinsic modules need no source.""" + base = _write(tmp_path, "lib/deep/b-defs.f90", "module base\nuse, intrinsic :: iso_c_binding\nend module base\n") + middle = _write(tmp_path, "lib/middle.F90", "module middle\nuse base\nend module middle\n") + entry = _write(tmp_path, "app/entry.f90", "module app\nuse middle\nuse iso_fortran_env\nend module app\n") + + assert _resolve([entry], [tmp_path / "lib"]) == (base.resolve(), middle.resolve(), entry) + + +def test_a_module_an_entry_defines_is_not_searched_for(tmp_path: Path): + """A second definition under a search directory does not compete with an entry's own.""" + _write(tmp_path, "search/copy.f90", "module shared\nend module shared\n") + shared = _write(tmp_path, "shared.f90", "module shared\nend module shared\n") + user = _write(tmp_path, "user.f90", "module user\nuse shared\nend module user\n") + + assert _resolve([user, shared], [tmp_path / "search"]) == (shared, user) + + +@pytest.mark.parametrize( + ("definitions", "code"), + [ + pytest.param((), "PARSE_MODULE_SOURCE_NOT_FOUND", id="missing"), + pytest.param(("one.f90", "two.f90"), "PARSE_AMBIGUOUS_MODULE_SOURCE", id="ambiguous"), + ], +) +def test_a_used_module_needs_exactly_one_defining_source(tmp_path: Path, definitions, code: str): + for name in definitions: + _write(tmp_path, f"search/{name}", "module needed\nend module needed\n") + (tmp_path / "search").mkdir(exist_ok=True) + entry = _write(tmp_path, "entry.f90", "module entry\nuse needed\nend module entry\n") + + with pytest.raises(FortranParseError, match="needed") as error: + _resolve([entry], [tmp_path / "search"]) + + assert error.value.code == code From f14efa9d0095240806120d5dab5863be3ecf053a Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 19:23:13 +0100 Subject: [PATCH 10/47] Support Open MPI 5 mpi_f08: follow re-exported types and keep dotted operators - A derived type a module reaches through another module's re-export now resolves to the module declaring it, using the same chain-following resolution contract imports use; before, the contract wrote an undefined type name (Open MPI 5 declares handles in mpi_types, re-exported by mpi_f08_types). - Contracts keep a dotted comparison spelling such as operator(.EQ.) in @overload(..., generic=...), and replay accepts it, so the bridge imports the operator under the spelling its module declares. - The Open MPI test and tutorial search both the source and configured build trees and include the generated mod/ headers; handle classes are checked wherever the version declares them. Verified with Open MPI 4.1.2 and 5.0.11. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 9 +- docs/user/tutorials/openmpi-f08.md | 20 +++-- prik/printers/pyi.py | 8 ++ prik/semantics/fortran2ir.py | 85 ++++++++++++++++--- prik/semantics/pyi2ir.py | 13 +-- .../end_to_end/test_openmpi_f08.py | 7 +- .../test_imported_derived_semantics.py | 43 ++++++++-- .../pipeline/test_classes_and_methods.py | 38 +++++++++ 8 files changed, 186 insertions(+), 37 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c812001ce..4ab1a6bd1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,6 +28,9 @@ release tags add a leading `v` to the package version. writes a pointer's current target. - A separate module-level `PARAMETER` statement types an undeclared name by the module's `IMPLICIT` rules and is rejected under `implicit none`. +- A derived type a module reaches through another module's re-export is + resolved to the module that declares it, so generated contracts name it + instead of writing an undefined type. - `IMPLICIT NONE (EXTERNAL)` no longer disables implicit typing; only plain `IMPLICIT NONE` or a specifier list naming `TYPE` does. @@ -50,7 +53,11 @@ release tags add a leading `v` to the package version. contracts publish the same selected surface. - The Open MPI `mpi_f08` tutorial and opt-in two-rank integration test build a wrapper from a restricted generated `.pyi` against a matching prebuilt - Open MPI installation and exercise NumPy communication and in-place reduction. + Open MPI installation and exercise NumPy communication and in-place reduction; + it is verified with Open MPI 4.1.2 and 5.0.11. +- A contract keeps a dotted comparison spelling such as `operator(.EQ.)` in + `@overload(..., generic=...)`, so a replayed bridge imports the operator + under the name its module declares. - The PRIMA example links five derivative-free solvers against one statically compiled `libprimaf` archive through a generated semantic contract and runs in the real-library portability matrix. Its guide includes a reproducible diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 4b61c6c92..1ee482bce 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -12,15 +12,17 @@ publication: reviewed Use a configured Open MPI source tree and the corresponding installed Open MPI toolchain. PRIK reads `mpi-f08.F90` and finds the sources of the modules it -uses in the source tree, whatever their names and layout in your Open MPI -version, to generate a contract; the extension compiles against the installed -modules and libraries. +uses in the source and build trees, whatever their names and layout in your +Open MPI version, to generate a contract; the extension compiles against the +installed modules and libraries. Set `PRIK_OPENMPI_SOURCE` to the matching Open MPI source root and -`PRIK_OPENMPI_BUILD` to its configured build root. The build must contain its -generated Fortran includes, including `configure-fortran-output.h` and -`sizeof_f08.h`. Confirm that `mpifort --showme:version` reports the same -Open MPI version as the source tree. +`PRIK_OPENMPI_BUILD` to its build root. Use a tree that has been built with +`make`, which generates the Fortran includes the sources read, such as +`configure-fortran-output.h` and `sizeof_f08.h`; `configure` also generates some +module sources into the build tree, so search both trees. Confirm that +`mpifort --showme:version` reports the same Open MPI version as the source +tree. Select the public facade's small initial API: @@ -45,9 +47,11 @@ EOF python3 -m prik generate --pyi \ "$PRIK_OPENMPI_SOURCE/ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90" \ --module-source-dir "$PRIK_OPENMPI_SOURCE" \ + --module-source-dir "$PRIK_OPENMPI_BUILD" \ --export-symbols exports.txt --out contract --compiler mpifort \ -I "$PRIK_OPENMPI_BUILD" \ -I "$PRIK_OPENMPI_BUILD/ompi/mpi/fortran/use-mpi-f08" \ + -I "$PRIK_OPENMPI_BUILD/ompi/mpi/fortran/use-mpi-f08/mod" \ -I "$PRIK_OPENMPI_SOURCE" \ -I "$PRIK_OPENMPI_BUILD/ompi/include" \ -I "$PRIK_OPENMPI_SOURCE/ompi/include" @@ -85,7 +89,7 @@ build_pyi_extension( Run a Python program under the matching Open MPI launcher. The selected functions live in `prik_openmpi_f08.mpi_f08`; Fortran `Int32` arguments such as counts and ranks use `numpy.int32` values. NumPy arrays provide the storage for -choice buffers. In the tested Open MPI 4.1.2 configuration, `mpi_in_place` is +choice buffers. In the tested Open MPI 4.1.2 and 5.0.11 configurations, `mpi_in_place` is a concrete integer module object exposed as a live rank-zero NumPy view, so pass that view directly to `mpi_allreduce`. A two-rank example lives at diff --git a/prik/printers/pyi.py b/prik/printers/pyi.py index b5a040e86..a9b93f580 100644 --- a/prik/printers/pyi.py +++ b/prik/printers/pyi.py @@ -1309,7 +1309,15 @@ def _overload_generic_argument(procedure: SemanticFunction, public_name: str) -> return "" if generic_name == public_name else f', generic="{generic_name}"' if procedure.metadata.get(OVERLOAD_KIND_METADATA) not in {"operator", "comparison"}: return "" + # A dotted spelling names the same operator as its symbol, but a + # compiler matches `use, only:` against the spelling the module wrote. if re.sub(r"\s+", "", generic_name).casefold() not in { + "operator(.eq.)", + "operator(.ne.)", + "operator(.lt.)", + "operator(.le.)", + "operator(.gt.)", + "operator(.ge.)", "operator(.eqv.)", "operator(.neqv.)", }: diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index 38afe3dc3..3caba5361 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -15,7 +15,7 @@ from __future__ import annotations -from collections.abc import Iterable, Iterator +from collections.abc import Iterable, Iterator, Mapping from typing import NamedTuple from copy import deepcopy from dataclasses import dataclass, replace @@ -239,6 +239,9 @@ class _DerivedTypeContext: uses: list[FortranUseStatement] | None = None procedure_uses: list[FortranUseStatement] | None = None local_types: frozenset[str] = frozenset() + # The parsed modules a ``use`` can reach, so a type another module only + # re-exports resolves to the module that declares it. + module_index: Mapping[str, FortranModule] | None = None @dataclass(frozen=True) @@ -1006,7 +1009,10 @@ def _callback_semantic_type( # it, so its dummies resolve there rather than in the module that # imported the interface -- which need not import the types it names. declaring_context = ( - self._module_derived_type_context(resolved.module) + self._module_derived_type_context( + resolved.module, + derived_type_context.module_index if derived_type_context is not None else None, + ) if resolved is not None and resolved.module is not None else derived_type_context ) @@ -1520,9 +1526,9 @@ def _visit_FortranModule( constants, imports, and visibility. It deliberately records facts only; later policy completion owns wrapper behavior decisions. """ - context = self._module_derived_type_context(module) - self._record_abstract_type_names(module) index = module_index if module_index is not None else self._callback_module_index([module]) + context = self._module_derived_type_context(module, index) + self._record_abstract_type_names(module) callback_interfaces = self._module_callback_interfaces(index, module) source_procedures = [ *module.procedures, @@ -2406,12 +2412,16 @@ def _abstract_types_from_project(project: FortranProject) -> set[tuple[str, str] } @staticmethod - def _module_derived_type_context(module: FortranModule) -> _DerivedTypeContext: + def _module_derived_type_context( + module: FortranModule, + module_index: Mapping[str, FortranModule] | None = None, + ) -> _DerivedTypeContext: """Create the lexical type lookup context owned by ``module``.""" return _DerivedTypeContext( module=module.name, uses=module.uses, local_types=frozenset(dtype.name.lower() for dtype in module.derived_types), + module_index=module_index, ) @staticmethod @@ -2431,6 +2441,7 @@ def _procedure_derived_type_context( uses=uses, procedure_uses=FortranToIRConverter._procedure_local_uses(proc), local_types=parent.local_types if parent is not None else frozenset(), + module_index=parent.module_index if parent is not None else None, ) @staticmethod @@ -2497,10 +2508,12 @@ def _resolve_derived_type_origin( if lname in context.local_types: return _ResolvedDerivedTypeOrigin(context.module, local_name) - resolved = self._resolve_derived_type_origin_from_uses(local_name, context.uses) + resolved = self._resolve_derived_type_origin_from_uses(local_name, context.uses, context.module_index) if resolved.module is None: return resolved - procedure_resolved = self._resolve_derived_type_origin_from_uses(local_name, context.procedure_uses) + procedure_resolved = self._resolve_derived_type_origin_from_uses( + local_name, context.procedure_uses, context.module_index + ) if (procedure_resolved.module, procedure_resolved.name) == (resolved.module, resolved.name): return _ResolvedDerivedTypeOrigin(resolved.module, resolved.name, import_scope="procedure") return resolved @@ -2509,19 +2522,24 @@ def _resolve_derived_type_origin_from_uses( self, local_name: str, uses: list[FortranUseStatement] | None, + module_index: Mapping[str, FortranModule] | None = None, ) -> _ResolvedDerivedTypeOrigin: """Resolve one derived-type spelling from explicit or wildcard ``use`` maps. - Only an unambiguous match is returned. Ambiguous explicit or wildcard - imports intentionally remain unresolved so this conversion stage does - not invent a native identity. + A module offers the types it declares and those it publicly re-exports; + a match is reported where the type is declared, however many modules it + passed through. Only an unambiguous match is returned. Ambiguous + explicit or wildcard imports intentionally remain unresolved so this + conversion stage does not invent a native identity. """ scope = ScopeUses(uses or ()) - offered = self._wrapped_type_names() + index = module_index or {} + offered = self._offered_type_names(index) routes = scope.routes_for(local_name, offered) - identities = {route.key for route in routes} + identities = {self._declared_type_identity(index, route.module, route.source_name) for route in routes} if len(identities) == 1: - return _ResolvedDerivedTypeOrigin(routes[0].module, routes[0].source_name) + module, name = identities.pop() + return _ResolvedDerivedTypeOrigin(module, name) if identities: return _ResolvedDerivedTypeOrigin(None, local_name) unresolved = scope.unresolved_routes_for(local_name, offered) @@ -2529,6 +2547,45 @@ def _resolve_derived_type_origin_from_uses( return _ResolvedDerivedTypeOrigin(unresolved[0].module, unresolved[0].source_name) return _ResolvedDerivedTypeOrigin(None, local_name) + def _offered_type_names(self, index: Mapping[str, FortranModule]): + """Return the type names each module declares or publicly re-exports, or ``None``.""" + declared = self._wrapped_type_names() + + def offered(module_name: str): + names = declared(module_name) + module = index.get(module_name.casefold()) + if names is None or module is None: + return names + return names | self._reexported_type_names(module, index) + + return offered + + def _reexported_type_names(self, module: FortranModule, index: Mapping[str, FortranModule]) -> set[str]: + """Return the derived-type names ``module`` makes public through ``use``.""" + cache = self.__dict__.setdefault("_reexported_type_cache", {}) + key = (id(module), id(index)) + if key not in cache: + is_public = self._effective_accessibility(module) + cache[key] = { + local_name.casefold() + for local_name, routes, (kind, _origin_module, _origin_name) in self._use_associations(module, index) + if kind == "derived_type" and is_public(local_name, routes) + } + return cache[key] + + @classmethod + def _declared_type_identity( + cls, + index: Mapping[str, FortranModule], + module_name: str, + source_name: str, + ) -> tuple[str, str]: + """Return the module and name declaring a type reached through ``module_name``.""" + kind, origin_module, origin_name = cls._resolve_reexport_origin(index, module_name, source_name) + if kind == "derived_type": + return origin_module, origin_name + return module_name, source_name + def _wrapped_type_names(self): """Return the wrapped type names each module declares, or ``None``.""" by_module: dict[str, set[str]] = {} @@ -3704,7 +3761,7 @@ def _inherited_generic_specifics( lookup: dict[tuple[str, str], SemanticFunction] = {} for source_module, source_generic in self._imported_generic_interfaces(module, generic_name, modules): signatures = {procedure.name.casefold(): procedure for procedure in source_module.procedures} - source_context = self._module_derived_type_context(source_module) + source_context = self._module_derived_type_context(source_module, modules) names = source_generic.specific_procedures or [item.name for item in source_generic.procedures] for name in names: target = _SpecificProcedure(source_module.name, name) diff --git a/prik/semantics/pyi2ir.py b/prik/semantics/pyi2ir.py index 019d01890..13fe6c825 100644 --- a/prik/semantics/pyi2ir.py +++ b/prik/semantics/pyi2ir.py @@ -1563,11 +1563,14 @@ def _validated_generic_override( if generic_name is None: return identity compact = re.sub(r"\s+", "", generic_name).casefold() - allowed_overrides = { - "__eq__": {"operator(==)", "operator(.eq.)", "operator(.eqv.)"}, - "__ne__": {"operator(/=)", "operator(.ne.)", "operator(.neqv.)"}, - } - if compact not in allowed_overrides.get(method_name, {identity[1].casefold()}): + canonical = identity[1].casefold() + # A comparison may be spelled with its dotted keyword; the logical + # equivalence operators also back equality on Boolean operands. + dotted = {"==": ".eq.", "/=": ".ne.", "<": ".lt.", "<=": ".le.", ">": ".gt.", ">=": ".ge."} + token = canonical.removeprefix("operator(").removesuffix(")") + allowed = {canonical, *((f"operator({dotted[token]})",) if token in dotted else ())} + allowed |= {"__eq__": {"operator(.eqv.)"}, "__ne__": {"operator(.neqv.)"}}.get(method_name, set()) + if compact not in allowed: raise ValueError(f"overload generic {generic_name!r} is incompatible with method {method_name!r}") return identity[0], generic_name diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index ff0f380c0..210b2efaa 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -78,6 +78,7 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) includes = ( build, build / "ompi/mpi/fortran/use-mpi-f08", + build / "ompi/mpi/fortran/use-mpi-f08/mod", source, build / "ompi/include", source / "ompi/include", @@ -92,6 +93,8 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) str(source / "ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90"), "--module-source-dir", str(source), + "--module-source-dir", + str(build), "--export-symbols", str(exports), "--out", @@ -116,7 +119,9 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) assert "mpi_sum: Final[Mpi_Op]" in types assert "mpi_int: Final[Mpi_Datatype]" in types assert "mpi_status_ignore: Mpi_Status" in types - assert all(f"class Mpi_{name}" in types for name in ("Comm", "Datatype", "Op", "Status")) + # Where the handle types are declared depends on the Open MPI version. + declarations = "".join(path.read_text(encoding="utf-8") for path in contract.glob("*.pyi")) + assert all(f"class Mpi_{name}" in declarations for name in ("Comm", "Datatype", "Op", "Status")) assert "AnyNative[" in interfaces and '@overload("mpi_send_f08")\ndef mpi_send(' in interfaces def show(flag: str) -> list[str]: diff --git a/tests/fortran/derived_types/semantics/test_imported_derived_semantics.py b/tests/fortran/derived_types/semantics/test_imported_derived_semantics.py index 23d50d4f7..263b4f378 100644 --- a/tests/fortran/derived_types/semantics/test_imported_derived_semantics.py +++ b/tests/fortran/derived_types/semantics/test_imported_derived_semantics.py @@ -20,6 +20,7 @@ fortran_project_to_semantic_modules, ) from prik.semantics.models import ( + EXTERNAL_TYPE_REF_METADATA, SemanticField, SemanticVariable, ) @@ -72,19 +73,25 @@ def test_converter_preserves_imported_derived_contexts_through_dispatch_paths(): assert converter.visit(imported_type, derived_type_context=context).metadata["external_type_ref"] == external_ref assert ( - converter.visit(imported_argument, derived_type_context=context).semantic_type.metadata["external_type_ref"] + converter.visit(imported_argument, derived_type_context=context).semantic_type.metadata[ + EXTERNAL_TYPE_REF_METADATA + ] == external_ref ) assert converter.visit(proc, derived_type_context=context).arguments[0].semantic_type.metadata[ "external_type_ref" ] == (external_ref) assert ( - converter.visit(parsed_file)[0].classes[0].fields[0].semantic_type.metadata["external_type_ref"] == external_ref + converter.visit(parsed_file)[0].classes[0].fields[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] + == external_ref + ) + assert ( + converter.visit(project)[0].classes[0].fields[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] + == external_ref ) - assert converter.visit(project)[0].classes[0].fields[0].semantic_type.metadata["external_type_ref"] == external_ref - assert semantic_module.classes[0].fields[0].semantic_type.metadata["external_type_ref"] == external_ref + assert semantic_module.classes[0].fields[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] == external_ref assert "external_type_ref" not in semantic_module.classes[0].fields[1].semantic_type.metadata - assert semantic_class.fields[0].semantic_type.metadata["external_type_ref"] == external_ref + assert semantic_class.fields[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] == external_ref assert isinstance(semantic_class.fields[0], SemanticField) assert semantic_class.visibility == "private" assert semantic_class.origin.source_language == "fortran" @@ -94,8 +101,8 @@ def test_converter_preserves_imported_derived_contexts_through_dispatch_paths(): semantic_proc = semantic_module.functions[0] assert semantic_proc.native_name == "step" assert semantic_proc.locals == [] - assert semantic_proc.arguments[0].semantic_type.metadata["external_type_ref"] == external_ref - assert semantic_module.variables[0].semantic_type.metadata["external_type_ref"] == external_ref + assert semantic_proc.arguments[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] == external_ref + assert semantic_module.variables[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] == external_ref assert isinstance(semantic_module.variables[0], SemanticVariable) assert [method.name for method in semantic_module.classes[0].methods] == ["step"] assert semantic_module.classes[0].methods[0].projection == semantic_proc.projection @@ -234,7 +241,8 @@ def test_imported_derived_type_is_an_opaque_external_reference_by_default(): wrapped_derived_types={("types_mod", "particle")}, ) assert ( - get_function(wrapped_module, "move").arguments[0].semantic_type.metadata["external_type_ref"]["wrapped"] is True + get_function(wrapped_module, "move").arguments[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA]["wrapped"] + is True ) @@ -273,3 +281,22 @@ def test_explicit_project_target_resolves_imported_derived_type_without_reexport "wrapped": True, "representation": "wrapped", } + + +def test_type_reached_through_a_reexporting_module_resolves_to_its_declaration(): + """A module that only re-exports a type still leads to the module declaring it.""" + project = parse_fortran_project( + { + "base.f90": "module base\n type :: handle_t\n integer :: val\n end type handle_t\nend module base\n", + "middle.f90": "module middle\n use base\nend module middle\n", + "api.f90": ( + "module api\n use middle\ncontains\n subroutine touch(h)\n" + " type(handle_t), intent(inout) :: h\n end subroutine touch\nend module api\n" + ), + } + ) + + api = next(module for module in fortran_project_to_semantic_modules(project) if module.name == "api") + reference = api.functions[0].arguments[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] + + assert (reference["origin_module"], reference["name"]) == ("base", "handle_t") diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_classes_and_methods.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_classes_and_methods.py index 455a594bf..a7ab6efb7 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_classes_and_methods.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_classes_and_methods.py @@ -482,3 +482,41 @@ def test_generic_specifics_with_projected_outputs_round_trip(): overloads = [item for item in module.overload_sets if item.name == "ink"] assert len(overloads) == 1 assert [procedure.name for procedure in overloads[0].procedures] == ["ink_default", "ink_extended"] + + +def test_dotted_comparison_spelling_survives_contract_replay_into_the_bridge_import(): + """A compiler matches `use, only:` by spelling, so `.EQ.` stays `.EQ.` through the contract.""" + source = """ +module handles + implicit none + private + public :: handle_t, operator(.EQ.), operator(.LT.) + type :: handle_t + integer :: val = 0 + end type handle_t + interface operator (.EQ.) + module procedure handle_eq + end interface operator (.EQ.) + interface operator (.LT.) + module procedure handle_lt + end interface operator (.LT.) +contains + logical function handle_eq(a, b) + type(handle_t), intent(in) :: a, b + handle_eq = a%val == b%val + end function handle_eq + logical function handle_lt(a, b) + type(handle_t), intent(in) :: a, b + handle_lt = a%val < b%val + end function handle_lt +end module handles +""" + semantic_module = fortran_module_to_semantic_module(parse_fortran_source(source, filename="handles.f90")) + pyi = emit_module(semantic_module) + loaded = parse_pyi_text(pyi, module_name=semantic_module.name) + + assert '@overload("handle_eq", generic="operator (.EQ.)")' in pyi + assert '@overload("handle_lt", generic="operator (.LT.)")' in pyi + assert emit_module(loaded) == pyi + bridge = rendered_source(generate_wrapper(loaded), ".f90") + assert "operator (.EQ.)" in bridge and "operator (.LT.)" in bridge From af72105ff0d037bb1b786f1c953a898305001c5c Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 20:09:51 +0100 Subject: [PATCH 11/47] Run Open MPI in CI, match configured trees to installs, select same-named generics - An Open MPI Integration lane builds Open MPI 4.1.8 and 5.0.11 from source, caches the source, configured build, and installation, and runs the mpi_f08 workflow test with PRIK_OPENMPI_REQUIRED=1 so missing setup fails instead of skipping. Pull-request validation requires it. - The Open MPI test also requires the configured tree and the installation to share their Fortran compiler and the installation to provide mpi_f08. - Export selection treats a generic that shares a specific's name as that one generic name instead of reporting it ambiguous; a two-level facade selection test covers the Open MPI 5 re-export shape. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/merge-validation.yml | 8 ++ .github/workflows/openmpi-integration.yml | 93 +++++++++++++++++++ CHANGELOG.md | 7 +- docs/developer/workflows/ci.md | 1 + docs/user/tutorials/openmpi-f08.md | 3 +- prik/semantics/fortran_exports.py | 14 ++- .../end_to_end/test_openmpi_f08.py | 46 ++++++++- .../test_fortran_export_symbol_selection.py | 62 ++++++++++++- 8 files changed, 224 insertions(+), 10 deletions(-) create mode 100644 .github/workflows/openmpi-integration.yml diff --git a/.github/workflows/merge-validation.yml b/.github/workflows/merge-validation.yml index 3e5df9ad6..7e3aa4f72 100644 --- a/.github/workflows/merge-validation.yml +++ b/.github/workflows/merge-validation.yml @@ -465,6 +465,11 @@ jobs: python tools/print_pytest_failures.py "$report" done + openmpi-integration: + name: Open MPI Integration + needs: [unit-tests] + uses: ./.github/workflows/openmpi-integration.yml + real-libraries-portability: name: Real Libraries Portability needs: [unit-tests, unit-tests-macos] @@ -621,6 +626,7 @@ jobs: - compiler-smoke-macos - unit-tests - unit-tests-macos + - openmpi-integration - real-libraries-portability - documentation-benchmark - documentation-build @@ -632,6 +638,7 @@ jobs: COMPILER_SMOKE_MACOS_RESULT: ${{ needs.compiler-smoke-macos.result }} UNIT_TESTS_RESULT: ${{ needs.unit-tests.result }} UNIT_TESTS_MACOS_RESULT: ${{ needs.unit-tests-macos.result }} + OPENMPI_INTEGRATION_RESULT: ${{ needs.openmpi-integration.result }} REAL_LIBRARIES_PORTABILITY_RESULT: ${{ needs.real-libraries-portability.result }} DOCUMENTATION_BENCHMARK_RESULT: ${{ needs.documentation-benchmark.result }} DOCUMENTATION_BUILD_RESULT: ${{ needs.documentation-build.result }} @@ -647,6 +654,7 @@ jobs: "compiler-smoke-macos=$COMPILER_SMOKE_MACOS_RESULT" \ "unit-tests=$UNIT_TESTS_RESULT" \ "unit-tests-macos=$UNIT_TESTS_MACOS_RESULT" \ + "openmpi-integration=$OPENMPI_INTEGRATION_RESULT" \ "real-libraries-portability=$REAL_LIBRARIES_PORTABILITY_RESULT" \ "documentation-benchmark=$DOCUMENTATION_BENCHMARK_RESULT" \ "documentation-build=$DOCUMENTATION_BUILD_RESULT" diff --git a/.github/workflows/openmpi-integration.yml b/.github/workflows/openmpi-integration.yml new file mode 100644 index 000000000..b7fd0565b --- /dev/null +++ b/.github/workflows/openmpi-integration.yml @@ -0,0 +1,93 @@ +name: Open MPI Integration + +on: + workflow_call: + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: openmpi-integration-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + openmpi: + name: Open MPI mpi_f08 · ${{ matrix.version }} · Ubuntu 24.04 + runs-on: ubuntu-24.04 + timeout-minutes: 90 + strategy: + fail-fast: false + matrix: + include: + # One release per module layout: 4.1 declares the handle types in + # mpi_f08_types, 5.0 re-exports them from a configured mpi_types. + - version: "4.1.8" + series: "v4.1" + - version: "5.0.11" + series: "v5.0" + env: + OPENMPI_VERSION: ${{ matrix.version }} + OPENMPI_SERIES: ${{ matrix.series }} + PRIK_GFORTRAN_BINARY: gfortran-13 + PRIK_GFORTRAN_PACKAGE: gfortran-13 + PYTHONPATH: . + steps: + - name: Checkout repository + uses: actions/checkout@v4 + - name: Set up Python 3.12 + uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: Install test dependencies + run: | + python -m pip install --upgrade pip + python -m pip install -e ".[qa]" + - name: Install pinned GFortran + shell: bash + run: | + if ! command -v "$PRIK_GFORTRAN_BINARY" >/dev/null 2>&1; then + sudo apt-get update + sudo apt-get install --yes "$PRIK_GFORTRAN_PACKAGE" + fi + compiler_dir="$RUNNER_TEMP/prik-gfortran" + mkdir -p "$compiler_dir" + ln -sf "$(command -v "$PRIK_GFORTRAN_BINARY")" "$compiler_dir/gfortran" + echo "$compiler_dir" >> "$GITHUB_PATH" + "$compiler_dir/gfortran" --version + - name: Restore the Open MPI source, configured build, and installation + id: openmpi-cache + uses: actions/cache@v4 + with: + path: ~/prik-openmpi/${{ matrix.version }} + key: openmpi-${{ matrix.version }}-ubuntu-24.04-gfortran-13-v1 + - name: Build and install Open MPI + if: steps.openmpi-cache.outputs.cache-hit != 'true' + shell: bash + run: | + root="$HOME/prik-openmpi/$OPENMPI_VERSION" + mkdir -p "$root/build" + curl -fsSL "https://download.open-mpi.org/release/open-mpi/$OPENMPI_SERIES/openmpi-$OPENMPI_VERSION.tar.bz2" \ + | tar -xj -C "$root" + mv "$root/openmpi-$OPENMPI_VERSION" "$root/source" + cd "$root/build" + ../source/configure --prefix="$root/install" --enable-mpi-fortran=usempif08 FC=gfortran + make -j"$(nproc)" + make install + # The test reads sources and generated headers from these trees and + # links the installation, so build objects are not cached. + find . \( -name '*.o' -o -name '*.lo' -o -name '*.a' -o -name '*.la' \) -delete + find . -type d -name .libs -prune -exec rm -rf {} + + - name: Run the Open MPI mpi_f08 workflow test + shell: bash + env: + PRIK_OPENMPI_REQUIRED: "1" + run: | + root="$HOME/prik-openmpi/$OPENMPI_VERSION" + export PATH="$root/install/bin:$PATH" + export LD_LIBRARY_PATH="$root/install/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" + export PRIK_OPENMPI_SOURCE="$root/source" + export PRIK_OPENMPI_BUILD="$root/build" + export PRIK_OPENMPI_MPIFORT="$root/install/bin/mpifort" + export PRIK_OPENMPI_LAUNCHER="$root/install/bin/mpirun" + python -m pytest -q -rs tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 4ab1a6bd1..6669374cc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,7 +54,12 @@ release tags add a leading `v` to the package version. - The Open MPI `mpi_f08` tutorial and opt-in two-rank integration test build a wrapper from a restricted generated `.pyi` against a matching prebuilt Open MPI installation and exercise NumPy communication and in-place reduction; - it is verified with Open MPI 4.1.2 and 5.0.11. + it is verified with Open MPI 4.1.2 and 5.0.11. Pull-request validation runs + it in an Open MPI Integration lane that builds Open MPI 4.1.8 and 5.0.11 from + source, and the test requires the configured tree and the installation to + share their version and Fortran compiler. +- Export selection accepts a generic that shares its name with one of its + specifics; the name selects the generic instead of being reported ambiguous. - A contract keeps a dotted comparison spelling such as `operator(.EQ.)` in `@overload(..., generic=...)`, so a replayed bridge imports the operator under the name its module declares. diff --git a/docs/developer/workflows/ci.md b/docs/developer/workflows/ci.md index 9a668dfa2..77926e7fa 100644 --- a/docs/developer/workflows/ci.md +++ b/docs/developer/workflows/ci.md @@ -17,6 +17,7 @@ contributors need to administer. | --- | --- | | Static analysis | Linting, formatting, security, dead code, and changed-code complexity policy. | | Compiler and platform tests | Supported Python versions, Linux and macOS, GNU Fortran, IFX, and Flang. | +| Open MPI Integration | The Open MPI `mpi_f08` workflow on Ubuntu for one Open MPI 4.1 and one 5.0 release: each is built from source, a restricted contract is generated from `mpi-f08.F90` with module discovery, and a two-rank program runs against the built wrapper. | | Real Libraries Portability | Maintained real-library examples across the hosted Linux and macOS architecture/compiler matrix, with deep BLAS and LAPACK audits on Linux x86-64. | | Documentation and benchmarks | Required performance benchmark and generated snapshot, documentation tests, and a strict site build. | diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 1ee482bce..c03472779 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -22,7 +22,8 @@ Set `PRIK_OPENMPI_SOURCE` to the matching Open MPI source root and `configure-fortran-output.h` and `sizeof_f08.h`; `configure` also generates some module sources into the build tree, so search both trees. Confirm that `mpifort --showme:version` reports the same Open MPI version as the source -tree. +tree, and configure the tree with the Fortran compiler the installation was +built with, which `ompi_info --parsable` reports as `compiler:fortran:absolute`. Select the public facade's small initial API: diff --git a/prik/semantics/fortran_exports.py b/prik/semantics/fortran_exports.py index 5d6def196..e8a94ff3a 100644 --- a/prik/semantics/fortran_exports.py +++ b/prik/semantics/fortran_exports.py @@ -124,6 +124,8 @@ def _select_module_surface(module, selected, requested): if (module_name, _native_symbol_name(declaration)) in selected ] selected_module.exported_names.extend(reexport.local_name for reexport in selected_module.reexports) + # A generic and a specific sharing its name publish that one name. + selected_module.exported_names = list(dict.fromkeys(selected_module.exported_names)) return selected_module @@ -277,8 +279,18 @@ def _fortran_export_candidates(modules: tuple[SemanticModule, ...]): non_selectable: set[tuple[str, str]] = set() for module in modules: module_name = _native_module_name(module) + # A generic may share its name with one of its specifics; the name then + # denotes the generic, which keeps that specific when it is selected. + generic_specifics = { + (_native_symbol_name(overload), _native_symbol_name(procedure)) + for overload in module.overload_sets + for procedure in overload.procedures + } for declaration in (*module.functions, *module.overload_sets, *module.variables): - selectable.setdefault((module_name, _native_symbol_name(declaration)), []).append(declaration) + name = _native_symbol_name(declaration) + if isinstance(declaration, SemanticFunction) and (name, name) in generic_specifics: + continue + selectable.setdefault((module_name, name), []).append(declaration) for reexport in module.reexports: if reexport.entity_kind in {"procedure", "generic", "variable"}: selectable.setdefault((module_name, reexport.local_name.casefold()), []).append(reexport) diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index 210b2efaa..d61f342db 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -9,6 +9,7 @@ import subprocess import sys from pathlib import Path +from typing import NoReturn import pytest @@ -35,28 +36,35 @@ ) +def _unavailable(reason: str) -> NoReturn: + """Skip locally, but fail where ``PRIK_OPENMPI_REQUIRED`` says Open MPI is provisioned.""" + if os.environ.get("PRIK_OPENMPI_REQUIRED") == "1": + pytest.fail(reason) + pytest.skip(reason) + + def _configured_openmpi() -> tuple[Path, Path, str, str, str]: """Find matching configured sources, wrapper compiler, and Open MPI launcher.""" source_text = os.environ.get("PRIK_OPENMPI_SOURCE") build_text = os.environ.get("PRIK_OPENMPI_BUILD") if not source_text or not build_text: - pytest.skip("set PRIK_OPENMPI_SOURCE and PRIK_OPENMPI_BUILD to a matching configured Open MPI tree") + _unavailable("set PRIK_OPENMPI_SOURCE and PRIK_OPENMPI_BUILD to a matching configured Open MPI tree") source, build = Path(source_text), Path(build_text) mpifort = os.environ.get("PRIK_OPENMPI_MPIFORT") or shutil.which("mpifort") launcher = os.environ.get("PRIK_OPENMPI_LAUNCHER") or shutil.which("orterun") or shutil.which("mpirun") if not mpifort or not launcher: - pytest.skip("Open MPI Fortran compiler wrapper and launcher are required") + _unavailable("Open MPI Fortran compiler wrapper and launcher are required") # Only the entry source is named; the modules it uses are discovered. for path in ( source / "ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90", build / "ompi/mpi/fortran/configure-fortran-output.h", ): if not path.is_file(): - pytest.skip(f"configured Open MPI semantic input is unavailable: {path}") + _unavailable(f"configured Open MPI semantic input is unavailable: {path}") version_file = (source / "VERSION").read_text(encoding="utf-8") parts = [re.search(rf"^{part}=(\d+)$", version_file, flags=re.MULTILINE) for part in ("major", "minor", "release")] if any(part is None for part in parts): - pytest.skip("Open MPI source version could not be read") + _unavailable("Open MPI source version could not be read") version = ".".join(part.group(1) for part in parts if part is not None) compiler_version = subprocess.check_output([mpifort, "--showme:version"], text=True) launcher_version = subprocess.check_output([launcher, "--version"], text=True) @@ -65,10 +73,38 @@ def _configured_openmpi() -> tuple[Path, Path, str, str, str]: or version not in launcher_version or not any(label in launcher_version for label in ("Open MPI", "OpenRTE")) ): - pytest.skip("configured sources, mpifort, and launcher must belong to the same Open MPI version") + _unavailable("configured sources, mpifort, and launcher must belong to the same Open MPI version") + _require_same_fortran_configuration(build, mpifort) return source, build, mpifort, launcher, version +def _require_same_fortran_configuration(build: Path, mpifort: str) -> None: + """Require the configured tree and the installation to share their Fortran build. + + One version configured twice can generate different Fortran sources and + headers, so the tree the contract is read from must have been configured + with the Fortran compiler the installation was built with, and the + installation must provide the ``mpi_f08`` module. + """ + ompi_info = Path(mpifort).with_name("ompi_info") + info = subprocess.check_output([str(ompi_info) if ompi_info.is_file() else "ompi_info", "--parsable"], text=True) + installed = re.search(r"^compiler:fortran:absolute:(.+)$", info, flags=re.MULTILINE) + configured_header = build / "opal/include/opal_config.h" + configured = ( + re.search(r'^#define OMPI_FC_ABSOLUTE "([^"]+)"', configured_header.read_text(encoding="utf-8"), re.MULTILINE) + if configured_header.is_file() + else None + ) + if re.search(r"^bindings:use_mpi_f08:yes", info, flags=re.MULTILINE) is None: + _unavailable("the installed Open MPI does not provide the mpi_f08 module") + if ( + installed is None + or configured is None + or Path(installed.group(1).strip()).resolve() != Path(configured.group(1)).resolve() + ): + _unavailable("the configured Open MPI tree and the installation were built with different Fortran compilers") + + def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) -> None: """The selected facade and native storage survive a real .pyi replay build.""" source, build, mpifort, launcher, _version = _configured_openmpi() diff --git a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py index 7cf0e28ac..e647495a9 100644 --- a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py +++ b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py @@ -5,8 +5,8 @@ import pytest from prik.cli import _read_export_symbols -from prik.parsers.fortran import parse_fortran_file -from prik.semantics.fortran2ir import fortran_module_to_semantic_module +from prik.parsers.fortran import parse_fortran_file, parse_fortran_project +from prik.semantics.fortran2ir import fortran_module_to_semantic_module, fortran_project_to_semantic_modules from prik.semantics.fortran_exports import select_fortran_export_symbols from prik.semantics.models import ( ProcedureOverloadSet, @@ -216,3 +216,61 @@ def test_selection_retains_component_and_parent_types_of_a_selected_signature(): assert sorted(cls.name for cls in selected.classes) == ["base_t", "inner_t", "outer_t"] assert set(selected.exported_names) == {"use_outer", "base_t", "inner_t", "outer_t"} + + +def test_selection_through_a_two_level_facade_reaches_each_declaring_module(): + """A generic, a procedure, and a variable re-exported twice resolve to where they are declared.""" + project = parse_fortran_project( + { + "base.f90": ( + "module base\n integer, parameter :: sentinel = 42\n interface area\n" + " module procedure area_real\n end interface area\ncontains\n" + " real function area_real(x)\n real, intent(in) :: x\n area_real = x\n" + " end function area_real\n subroutine unrelated()\n end subroutine unrelated\nend module base\n" + ), + "middle.f90": "module middle\n use base\nend module middle\n", + "tools.f90": "module tools\ncontains\n subroutine touch()\n end subroutine touch\nend module tools\n", + "facade.f90": "module facade\n use middle\n use tools\nend module facade\n", + } + ) + modules = fortran_project_to_semantic_modules(project) + + selected = select_fortran_export_symbols(modules, ["facade::area", "facade::sentinel", "facade::touch"]) + + owners = {module.name: module for module in selected.primary_modules} + assert [overload.name for overload in owners["base"].overload_sets] == ["area"] + assert [variable.name for variable in owners["base"].variables] == ["sentinel"] + assert "unrelated" not in {function.name for function in owners["base"].functions} + assert [function.name for function in owners["tools"].functions] == ["touch"] + assert sorted(owners["facade"].exported_names) == ["area", "sentinel", "touch"] + assert "middle" not in owners + + +def test_a_generic_sharing_a_specific_name_is_one_selectable_name(): + """Fortran lets a generic share a specific's name; the name then selects the generic.""" + module = fortran_module_to_semantic_module( + parse_fortran_file( + """ +module shapes + interface area + module procedure area, area_int + end interface area +contains + real function area(x) + real, intent(in) :: x + area = x + end function area + integer function area_int(i) + integer, intent(in) :: i + area_int = i + end function area_int +end module shapes +""" + ).modules[0] + ) + + selected = select_fortran_export_symbols([module], ["shapes::area"]).primary_modules[0] + + assert [overload.name for overload in selected.overload_sets] == ["area"] + assert sorted(function.name for function in selected.functions) == ["area", "area_int"] + assert selected.exported_names == ["area"] From 8ba34123f449a035edae0845b1d9312dbd6e38f0 Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 20:09:51 +0100 Subject: [PATCH 12/47] Keep native build artifacts out of the repository root The test session refuses to start while .mod, object, or library files sit in the repository root, where a stale module silently shadows later builds, and fails when a test leaves one there. AGENTS.md asks for ad-hoc compiler commands to run outside the root. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 3 +++ conftest.py | 43 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index d855d507d..4c5bfa4d2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -239,6 +239,9 @@ compilation should use the focused owners under `tests/fortran/infrastructure/building/compiling/` as applicable. Include the relevant end-to-end feature tests whenever a generated or compiled mechanism changes; run a broader suite when behavior spans multiple stages. +Run ad-hoc compiler and build commands outside the repository root, for example in a temporary +directory, so no `.mod`, object, or library file lands there; the test session refuses to start +while native build artifacts sit in the root, since a stale one silently shadows a later build. Run pytest with at most `-n 2`. Never `-n 4`, `-n 8`, or `-n auto`. The development machine has 12 cores but only about 7 GB of RAM, and every xdist worker loads NumPy while the Fortran end-to-end tests fork gfortran and cc per test on top of `pytest-monitor` profiling each one. Higher parallelism exhausts memory and thrashes swap, which has hard-frozen the machine and forced a reboot. Prefer the narrowest owning test path over a full suite run, and commit verified work promptly rather than batching it behind a long run. Do not run LAPACK wrapper tests locally unless the user explicitly asks for them. Local verification may run everything else, including BLAS-only real-library tests; leave LAPACK coverage to GitHub Actions by default. Do not run the full coverage workflow for routine changes. Run focused tests plus the required static-analysis suite. Reserve the complete CI-style coverage workflow for explicit pre-merge or pull-request verification, or when the user specifically requests it. diff --git a/conftest.py b/conftest.py index f13c1d896..a6ce81100 100644 --- a/conftest.py +++ b/conftest.py @@ -33,6 +33,49 @@ def pytest_addoption(parser: pytest.Parser) -> None: ) +# Native builds keep their outputs in their own directories. A compiler run from +# the repository root leaves modules or objects here instead, and a stale +# ``m.mod`` then silently shadows another source's module in later builds. +_ROOT_ARTIFACT_SUFFIXES = frozenset( + {".mod", ".smod", ".o", ".obj", ".so", ".a", ".dylib", ".dll", ".pyd", ".lib", ".exe"} +) + + +def _root_build_artifacts() -> list[str]: + """Return native build artifacts sitting directly in the repository root.""" + return sorted( + path.name + for path in ROOT.iterdir() + if path.is_file() and (path.suffix.casefold() in _ROOT_ARTIFACT_SUFFIXES or path.name == "a.out") + ) + + +def pytest_sessionstart(session: pytest.Session) -> None: + """Refuse to run over stale root build artifacts that later builds could pick up.""" + if hasattr(session.config, "workerinput"): + return + stale = _root_build_artifacts() + if stale: + raise pytest.UsageError( + f"Remove native build artifacts from the repository root before testing: {', '.join(stale)}" + ) + + +def pytest_sessionfinish(session: pytest.Session, exitstatus: int) -> None: + """Fail a session whose tests left native build artifacts in the repository root.""" + if hasattr(session.config, "workerinput"): + return + leaked = _root_build_artifacts() + if leaked: + reporter = session.config.pluginmanager.get_plugin("terminalreporter") + if reporter is not None: + reporter.write_line( + f"ERROR: tests left native build artifacts in the repository root: {', '.join(leaked)}", + red=True, + ) + session.exitstatus = pytest.ExitCode.TESTS_FAILED + + try: from hypothesis import HealthCheck, settings except ImportError: # pragma: no cover - base test installs can omit QA extras. From 4db396d0899686418c3046646ebd4e215a916a77 Mon Sep 17 00:00:00 2001 From: said Date: Thu, 24 Sep 2026 20:38:14 +0100 Subject: [PATCH 13/47] Report parse diagnostics at main-source lines through compiler line markers Compiler preprocessing prefixes its output with line markers and renumbers around #include and #if blocks, so parse errors reported preprocessor output lines (line 7 of a 4-line file). The parser now follows the markers: a line keeps its number in the main source, and a line an included file contributes is reported at the #include line. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 3 ++ prik/parsers/fortran/parser.py | 53 ++++++++++++++++++- ...source_form_and_diagnostics_regressions.py | 27 ++++++++++ 3 files changed, 82 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6669374cc..69046c70d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,9 @@ release tags add a leading `v` to the package version. - A derived type a module reaches through another module's re-export is resolved to the module that declares it, so generated contracts name it instead of writing an undefined type. +- Fortran parse diagnostics on compiler-preprocessed sources report the line + in the source file, or the `#include` line for text an included file + contributes, instead of a line in the preprocessor output. - `IMPLICIT NONE (EXTERNAL)` no longer disables implicit typing; only plain `IMPLICIT NONE` or a specifier list naming `TYPE` does. diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index e63e8ada6..f152db07c 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -191,6 +191,52 @@ def _binding_visibility(attributes: list[str], default_visibility: str) -> str: r'^\s*#\s*(?:line\s+)?\d+(?:\s+(?:"(?:[^"\\]|\\.)*"|\S+))?(?:\s+\d+)*\s*$', re.IGNORECASE, ) +_FORTRAN_LINEMARKER_PARTS_RE = re.compile( + r'^\s*#\s*(?:line\s+)?(?P\d+)(?:\s+"(?P(?:[^"\\]|\\.)*)")?', + re.IGNORECASE, +) + + +def _main_source_line_numbers(source: str) -> dict[int, int]: + """Map each preprocessed line to its line in the main source, following compiler line markers. + + A marker ``# N "file"`` states that the next line is line ``N`` of that + file. The first real file named is the main source. A line an included + file contributes is reported at the ``#include`` line, the one before the + line the main source resumes at. Text before any marker keeps its own + line number. + """ + origins: dict[int, int] = {} + main_file: str | None = None + current_file: str | None = None + next_line: int | None = None + included: list[int] = [] + for index, text in enumerate(source.splitlines(), start=1): + marker = _FORTRAN_LINEMARKER_RE.match(text.strip()) and _FORTRAN_LINEMARKER_PARTS_RE.match(text.strip()) + if marker: + named = marker.group("file") + if named is not None: + if main_file is None and not named.startswith("<"): + main_file = named + current_file = named + next_line = int(marker.group("line")) + if current_file == main_file: + origins.update(dict.fromkeys(included, max(next_line - 1, 1))) + included.clear() + continue + if next_line is None: + origins[index] = index + elif current_file == main_file: + origins[index] = next_line + else: + included.append(index) + if next_line is not None: + next_line += 1 + # An include still open at the end has no resume marker to place it. + origins.update(dict.fromkeys(included, next_line or 1)) + return origins + + _INTRINSIC_COMPILE_TIME_MODULES = frozenset({"iso_c_binding", "iso_fortran_env"}) _FORTRAN_SOURCE_SUFFIXES = (".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08") @@ -2482,7 +2528,12 @@ def _preprocessed_lines(self, source: _SourceOrLines, filename: str | None) -> _ re.sub(r"[^\r\n]", "", line) if _FORTRAN_LINEMARKER_RE.match(line.strip()) else line for line in source.splitlines(keepends=True) ) - lines = preprocess_lines(source_without_linemarkers, filename) + # Compiler output numbers its own lines; report the main source's. + origins = _main_source_line_numbers(source) + lines = [ + (line, origins.get(lineno, lineno), source_line) + for line, lineno, source_line in preprocess_lines(source_without_linemarkers, filename) + ] for line, lineno, source_line in lines: self._raise_for_raw_cpp_directive(line, filename, lineno, source_line) return lines diff --git a/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py b/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py index db6f3e357..b002dc991 100644 --- a/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py +++ b/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py @@ -648,3 +648,30 @@ def test_unit_models_preserve_filename_propagation(): assert parsed.submodules[0].filename == "unit_models.f90" assert parsed.programs[0].filename == "unit_models.f90" assert parsed.block_data_units[0].filename == "unit_models.f90" + + +@pytest.mark.parametrize( + ("preprocessed", "line_number"), + [ + pytest.param( + '# 1 "bad.F90"\n# 1 ""\n# 1 ""\n# 1 "bad.F90"\n' + 'module bad\n implicit none\n\n# 1 "consts.h" 1\n integer :: from_header\n' + '# 4 "bad.F90" 2\n\n\n\n x = 3\nend module bad\n', + 7, + id="main-source-after-include-and-conditional", + ), + pytest.param( + '# 1 "bad.F90"\n# 1 ""\n# 1 ""\n# 1 "bad.F90"\n' + 'module bad\n implicit none\n\n# 1 "bad_inc.h" 1\n x = 3\n# 4 "bad.F90" 2\nend module bad\n', + 3, + id="included-line-reports-the-include", + ), + ], +) +def test_diagnostics_on_compiler_output_report_main_source_lines(preprocessed: str, line_number: int): + """Compiler line markers renumber diagnostics to the source the user wrote.""" + with pytest.raises(FortranParseError) as error: + parse_fortran_file(preprocessed, filename="bad.F90") + + assert error.value.code == "PARSE_EXECUTABLE_IN_SPECIFICATION" + assert error.value.line_number == line_number From 18cc376b5995123e31128bd5f3a372dafe6097d0 Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 07:11:19 +0100 Subject: [PATCH 14/47] Honor use natures and nested submodules in module discovery; reject multi-token mpifort commands - The parser records whether a use statement states INTRINSIC or NON_INTRINSIC, and the shared scope helper exposes the statements, so discovery reads the same facts compile ordering does. An intrinsic module is never searched; a non_intrinsic one must be found; one with no stated nature prefers a searched source and otherwise falls back to a known processor module. - Submodule definitions are indexed by ancestor:name, and a submodule depends on its direct parent, so a nested submodule chain resolves in parent-first order. - The Open MPI test and tutorial require mpifort --showme:command to be one compiler executable instead of treating extra tokens as flags. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 5 +- docs/user/tutorials/openmpi-f08.md | 7 +- prik/parsers/fortran/models.py | 3 + prik/parsers/fortran/module_sources.py | 113 ++++++++++++------ prik/parsers/fortran/parser.py | 7 +- prik/parsers/fortran/scope.py | 15 ++- .../end_to_end/test_openmpi_f08.py | 8 +- .../fixtures/general/module_vars_use.json | 6 +- .../parsing/test_module_source_resolution.py | 48 ++++++++ 9 files changed, 158 insertions(+), 54 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 69046c70d..00441270e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,8 +11,9 @@ release tags add a leading `v` to the package version. `build_fortran_extension` accepts `module_source_dirs`: from the given entry sources, PRIK follows each `use` to the source under those directories that defines the module and reads it too, so a multi-module library such as Open - MPI's `mpi_f08` is supplied by its entry file. A used module with no source, - or with several, is an error. + MPI's `mpi_f08` is supplied by its entry file. Discovery honors + `use, intrinsic` and `use, non_intrinsic`, and follows each submodule to its + direct parent. A needed module with no source, or with several, is an error. - Mutable fixed-storage Fortran module scalars expose native-backed rank-zero NumPy views, including fixed-length character bytes; primitive and fixed diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index c03472779..4bbea88e2 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -60,7 +60,8 @@ python3 -m prik generate --pyi \ Build from the generated `contract/__init__.pyi`. Query the installed wrapper compiler for its compiler command, module and include directories, remaining -compile flags, and ordered link arguments: +compile flags, and ordered link arguments. The compiler command must be a single +executable; a command such as `ccache gfortran` is not a compiler plus flags: ```python import shlex @@ -74,12 +75,14 @@ def show(option): command, compile_flags = show("--showme:command"), show("--showme:compile") +if len(command) != 1: + raise SystemExit(f"mpifort wraps the multi-token command {command}; set one compiler executable") include_dirs = [*show("--showme:incdirs"), *(flag[2:] for flag in compile_flags if flag.startswith("-I"))] build_pyi_extension( "contract/__init__.pyi", input_compiler=command[0], native_include_dirs=list(dict.fromkeys(include_dirs)), - wrapper_fortran_flags=[*command[1:], *(flag for flag in compile_flags if not flag.startswith("-I"))], + wrapper_fortran_flags=[flag for flag in compile_flags if not flag.startswith("-I")], native_link_items=[NativeLinkItem("linker_argument", flag) for flag in show("--showme:link")], native_linker_language="fortran", output_name="prik_openmpi_f08", diff --git a/prik/parsers/fortran/models.py b/prik/parsers/fortran/models.py index 7e9c0e1d4..17d4f3cc0 100644 --- a/prik/parsers/fortran/models.py +++ b/prik/parsers/fortran/models.py @@ -449,6 +449,9 @@ class FortranUseStatement: module: str only: bool = False mappings: tuple[FortranUseMapping, ...] = () + # ``"intrinsic"`` or ``"non_intrinsic"`` when the statement names the + # module's nature, which decides whether a processor module is meant. + nature: str | None = None @dataclass diff --git a/prik/parsers/fortran/module_sources.py b/prik/parsers/fortran/module_sources.py index 9203c1799..87c470ff6 100644 --- a/prik/parsers/fortran/module_sources.py +++ b/prik/parsers/fortran/module_sources.py @@ -2,9 +2,9 @@ A Fortran ``use`` names a module, not a file, and no rule ties the two: a module may live in any file under any directory. Given the sources a caller -names and the directories to search, this resolver follows each ``use`` to the -one file that defines that module, transitively, so a project can be supplied -by its entry file alone. +names and the directories to search, this resolver follows each ``use`` and +each submodule's parent to the one file that defines it, transitively, so a +project can be supplied by its entry file alone. Which modules a source defines and uses are parser facts, read here exactly as compile ordering reads them. The directory index only locates candidate files @@ -20,17 +20,23 @@ from prik.parsers.fortran.models import FortranFile, FortranParseError from prik.parsers.fortran.parser import FortranParser -from prik.parsers.fortran.scope import used_module_names +from prik.parsers.fortran.scope import used_module_statements # Suffixes a Fortran compiler accepts as free- or fixed-form source. _FORTRAN_SOURCE_SUFFIXES = frozenset({".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08", ".fpp"}) +_SUBMODULE_LINE = re.compile( + r"^[ \t]*submodule[ \t]*\([ \t]*(?P[a-z][a-z0-9_]*)[ \t]*(?::[ \t]*[a-z][a-z0-9_]*[ \t]*)?\)" + r"[ \t]*(?P[a-z][a-z0-9_]*)", + re.IGNORECASE | re.MULTILINE, +) _MODULE_LINE = re.compile( r"^[ \t]*module[ \t]+(?!(?:procedure|function|subroutine|pure|impure|elemental|recursive)\b)" r"(?P[a-z][a-z0-9_]*)[ \t]*(?:!.*)?$", re.IGNORECASE | re.MULTILINE, ) -# Modules the processor supplies without a source file. +# Modules a processor supplies. A ``use`` stating no nature falls back to one of +# these only when no searched source defines a module of that name. _INTRINSIC_MODULES = frozenset( { "iso_c_binding", @@ -53,22 +59,23 @@ def resolve_fortran_module_sources( """Return ``entries`` with the sources of every module they use, dependencies first. ``read_source`` returns a file's preprocessed text, so a ``use`` inside an - inactive conditional block is not followed. A used module that no source - read so far defines must be defined by exactly one Fortran source under - ``search_dirs``; otherwise a :class:`FortranParseError` names the module - and the source that uses it. + inactive conditional block is not followed. An ``intrinsic`` module is + never searched. Any other used module, and a submodule's parent, that no + source read so far defines must be defined by exactly one Fortran source + under ``search_dirs``; otherwise a :class:`FortranParseError` names it and + the source that needs it. """ - candidates = _module_candidates(search_dirs) + candidates = _unit_candidates(search_dirs) parser = FortranParser() - facts: dict[Path, tuple[set[str], set[str]]] = {} + facts: dict[Path, tuple[set[str], dict[str, str | None]]] = {} owners: dict[str, Path] = {} - def read(path: Path) -> tuple[set[str], set[str]]: + def read(path: Path) -> tuple[set[str], dict[str, str | None]]: if path not in facts: parsed = parser.parse_file(read_source(path), filename=str(path)) - facts[path] = _defined_and_used_modules(parsed) - for name in facts[path][0]: - owners.setdefault(name, path) + facts[path] = _defined_and_required_units(parsed) + for unit in facts[path][0]: + owners.setdefault(unit, path) return facts[path] for entry in entries: @@ -80,10 +87,13 @@ def visit(path: Path) -> None: if path in ordered or path in visiting: return visiting.add(path) - defined, used = read(path) - for name in sorted(used - defined - _INTRINSIC_MODULES): - dependency = owners.get(name) or _defining_source(name, candidates, path, read) - visit(dependency) + defined, required = read(path) + for unit, nature in sorted(required.items()): + if unit in defined or nature == "intrinsic": + continue + dependency = owners.get(unit) or _defining_source(unit, nature, candidates, path, read) + if dependency is not None: + visit(dependency) visiting.discard(path) ordered.append(path) @@ -94,52 +104,81 @@ def visit(path: Path) -> None: def _defining_source( - name: str, + unit: str, + nature: str | None, candidates: dict[str, list[Path]], user: Path, - read: Callable[[Path], tuple[set[str], set[str]]], -) -> Path: - """Return the one searched source whose parsed modules define ``name``, or raise.""" - defining = [path for path in candidates.get(name, ()) if name in read(path)[0]] + read: Callable[[Path], tuple[set[str], dict[str, str | None]]], +) -> Path | None: + """Return the one searched source whose parsed units define ``unit``, or raise. + + A ``use`` that states no nature names an intrinsic module only when no + other module of that name is accessible, so a known intrinsic name is + satisfied by the processor when no source defines it. + """ + defining = [path for path in candidates.get(unit, ()) if unit in read(path)[0]] if len(defining) == 1: return defining[0] + if not defining and nature is None and unit in _INTRINSIC_MODULES: + return None + kind = "submodule" if ":" in unit else "module" if not defining: raise FortranParseError( - f"No Fortran source defines module '{name}' used by {user}; " + f"No Fortran source defines {kind} '{unit}' used by {user}; " "add the directory that contains it as a module source directory.", filename=str(user), code="PARSE_MODULE_SOURCE_NOT_FOUND", ) listed = ", ".join(str(path) for path in defining) raise FortranParseError( - f"Module '{name}' used by {user} is defined by several sources ({listed}); " + f"{kind.capitalize()} '{unit}' used by {user} is defined by several sources ({listed}); " "narrow the module source directories to one of them.", filename=str(user), code="PARSE_AMBIGUOUS_MODULE_SOURCE", ) -def _module_candidates(search_dirs: Iterable[Path]) -> dict[str, list[Path]]: - """Map each module name to the Fortran sources under ``search_dirs`` with a matching ``module`` line.""" +def _unit_candidates(search_dirs: Iterable[Path]) -> dict[str, list[Path]]: + """Map each module or submodule to the Fortran sources under ``search_dirs`` that open it.""" candidates: dict[str, list[Path]] = {} for directory in search_dirs: for path in sorted(Path(directory).rglob("*")): if path.suffix.casefold() not in _FORTRAN_SOURCE_SUFFIXES or not path.is_file(): continue text = path.read_text(encoding="utf-8", errors="replace") - for match in _MODULE_LINE.finditer(text): - paths = candidates.setdefault(match.group("name").casefold(), []) + units = [match.group("name").casefold() for match in _MODULE_LINE.finditer(text)] + units.extend( + f"{match.group('ancestor')}:{match.group('name')}".casefold() + for match in _SUBMODULE_LINE.finditer(text) + ) + for unit in units: + paths = candidates.setdefault(unit, []) if path.resolve() not in paths: paths.append(path.resolve()) return candidates -def _defined_and_used_modules(parsed: FortranFile) -> tuple[set[str], set[str]]: - """Return the modules one parsed source defines and every module it uses.""" +def _defined_and_required_units(parsed: FortranFile) -> tuple[set[str], dict[str, str | None]]: + """Return the units one parsed source defines and the units it requires, with each ``use`` nature. + + A module is named by itself and a submodule by ``ancestor:name``. A + submodule requires its direct parent: the submodule it names after its + ancestor, or the ancestor module itself. + """ defined = {str(module.name).casefold() for module in parsed.modules} - used: set[str] = set() + defined.update( + f"{submodule.ancestor or submodule.parent}:{submodule.name}".casefold() for submodule in parsed.submodules + ) + required: dict[str, str | None] = {} for owner in (*parsed.modules, *parsed.submodules, *parsed.programs, *parsed.procedures): - used.update(used_module_names(owner)) - # A submodule extends the module it names first, which must be available. - used.update(str(submodule.ancestor or submodule.parent).casefold() for submodule in parsed.submodules) - return defined, used + for statement in used_module_statements(owner): + name = statement.module.casefold() + # An explicit nature is kept over a statement that states none. + if required.get(name) is None: + required[name] = statement.nature + for submodule in parsed.submodules: + parent = ( + f"{submodule.ancestor}:{submodule.parent}" if submodule.ancestor else str(submodule.parent) + ).casefold() + required[parent] = "non_intrinsic" + return defined, required diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index f152db07c..6f0c1293d 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -148,7 +148,7 @@ def _binding_visibility(attributes: list[str], default_visibility: str) -> str: re.IGNORECASE, ), "use": re.compile( - r"^use\s*(?:,\s*(?:intrinsic|non_intrinsic)\s*)?(?:::)?\s*(?P\w+)\s*(?P,\s*.*)?$", + r"^use\s*(?:,\s*(?Pintrinsic|non_intrinsic)\s*)?(?:::)?\s*(?P\w+)\s*(?P,\s*.*)?$", re.IGNORECASE, ), "include": re.compile(r"^(?:#\s*)?include\s*(?P['\"][^'\"]+['\"])", re.IGNORECASE), @@ -6106,8 +6106,9 @@ def _parse_use_statement(line: str) -> FortranUseStatement | None: if not match: return None rest = (match.group("rest") or "").strip() + nature = match.group("nature").casefold() if match.group("nature") else None if not rest: - return FortranUseStatement(match.group("module")) + return FortranUseStatement(match.group("module"), nature=nature) payload = rest.lstrip(",").strip() only_match = re.match(r"^only\s*:\s*(?P.*)$", payload, re.IGNORECASE) if only_match: @@ -6123,7 +6124,7 @@ def _parse_use_statement(line: str) -> FortranUseStatement | None: source = token target = None mappings.append(FortranUseMapping(source=source, target=target)) - return FortranUseStatement(match.group("module"), only_match is not None, tuple(mappings)) + return FortranUseStatement(match.group("module"), only_match is not None, tuple(mappings), nature=nature) # ----------------------------------------------------------------------------- diff --git a/prik/parsers/fortran/scope.py b/prik/parsers/fortran/scope.py index b64d23027..9564d8a05 100644 --- a/prik/parsers/fortran/scope.py +++ b/prik/parsers/fortran/scope.py @@ -128,13 +128,13 @@ def _renamed_away(self, module: str) -> frozenset[str]: return frozenset(item.source.casefold() for item in self.mappings(module) if item.target) -def used_module_names(owner: object) -> set[str]: - """Return every module one scope names, lowercased. +def used_module_statements(owner: object) -> list[FortranUseStatement]: + """Return every ``use`` statement one scope writes, including nested ones. A ``use`` written inside a contained procedure or an interface body is a dependency of the scope holding it just as much as one written at its top, - so the whole tree is read. Compile ordering and project dependencies both - ask this, and they have to get the same answer. + so the whole tree is read. Compile ordering, project dependencies, and + module source discovery all ask this, and they have to get the same answer. """ statements: list[FortranUseStatement] = list(getattr(owner, "uses", ())) for procedure in getattr(owner, "procedures", ()): @@ -142,4 +142,9 @@ def used_module_names(owner: object) -> set[str]: for interface in getattr(owner, "interfaces", ()): for procedure in getattr(interface, "procedures", ()): statements.extend(getattr(procedure, "uses", ())) - return {statement.module.lower() for statement in statements} + return statements + + +def used_module_names(owner: object) -> set[str]: + """Return every module one scope names, lowercased.""" + return {statement.module.lower() for statement in used_module_statements(owner)} diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index d61f342db..be067cba6 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -163,15 +163,17 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) def show(flag: str) -> list[str]: return shlex.split(subprocess.check_output([mpifort, flag], text=True)) - # The wrapper compiler's command may carry its own flags, and its compile - # flags are more than include directories; keep every one of them. + # The underlying compiler command may take more than one token, such as a + # launcher before the compiler, which is not a compiler plus flags. command, compile_flags = show("--showme:command"), show("--showme:compile") + if len(command) != 1: + _unavailable(f"mpifort --showme:command is a multi-token command {command}; pass one compiler executable") include_dirs = [*show("--showme:incdirs"), *(flag[2:] for flag in compile_flags if flag.startswith("-I"))] result = build_pyi_extension( contract / "__init__.pyi", input_compiler=command[0], native_include_dirs=list(dict.fromkeys(include_dirs)), - wrapper_fortran_flags=[*command[1:], *(flag for flag in compile_flags if not flag.startswith("-I"))], + wrapper_fortran_flags=[flag for flag in compile_flags if not flag.startswith("-I")], native_link_items=[NativeLinkItem("linker_argument", flag) for flag in show("--showme:link")], native_linker_language="fortran", output_name="prik_openmpi_f08", diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/module_vars_use.json b/tests/fortran/infrastructure/parsing/fixtures/general/module_vars_use.json index 9cc188849..25322bb60 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/module_vars_use.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/module_vars_use.json @@ -20,7 +20,8 @@ "source": "c_double", "target": null } - ] + ], + "nature": null } ], "variables": [ @@ -109,7 +110,8 @@ "source": "c_double", "target": null } - ] + ], + "nature": null } ], "variables": [ diff --git a/tests/fortran/modules/parsing/test_module_source_resolution.py b/tests/fortran/modules/parsing/test_module_source_resolution.py index f22316533..620501cf7 100644 --- a/tests/fortran/modules/parsing/test_module_source_resolution.py +++ b/tests/fortran/modules/parsing/test_module_source_resolution.py @@ -56,3 +56,51 @@ def test_a_used_module_needs_exactly_one_defining_source(tmp_path: Path, definit _resolve([entry], [tmp_path / "search"]) assert error.value.code == code + + +@pytest.mark.parametrize( + ("statement", "user_source", "found"), + [ + pytest.param("use, intrinsic :: iso_fortran_env", True, False, id="intrinsic-never-searched"), + pytest.param("use, intrinsic :: vendor_runtime", False, False, id="unlisted-intrinsic-not-searched"), + pytest.param("use, non_intrinsic :: iso_fortran_env", True, True, id="non-intrinsic-uses-the-source"), + pytest.param("use iso_fortran_env", True, True, id="unstated-prefers-a-source"), + pytest.param("use iso_fortran_env", False, False, id="unstated-falls-back-to-the-processor"), + ], +) +def test_use_nature_decides_whether_a_module_source_is_needed(tmp_path: Path, statement, user_source, found): + user_module = "module iso_fortran_env\nend module iso_fortran_env\n" + definition = _write(tmp_path, "search/iso_fortran_env.f90", user_module) if user_source else None + (tmp_path / "search").mkdir(exist_ok=True) + entry = _write(tmp_path, "entry.f90", f"module entry\n {statement}\nend module entry\n") + + resolved = _resolve([entry], [tmp_path / "search"]) + + assert resolved == ((definition.resolve(), entry) if found else (entry,)) + + +def test_explicit_non_intrinsic_module_without_a_source_is_not_found(tmp_path: Path): + (tmp_path / "search").mkdir() + entry = _write(tmp_path, "entry.f90", "module entry\n use, non_intrinsic :: iso_fortran_env\nend module entry\n") + + with pytest.raises(FortranParseError) as error: + _resolve([entry], [tmp_path / "search"]) + + assert error.value.code == "PARSE_MODULE_SOURCE_NOT_FOUND" + + +def test_nested_submodule_resolves_its_direct_parent_before_the_ancestor_module(tmp_path: Path): + """``submodule (base:middle) leaf`` needs the ``middle`` submodule, which needs ``base``.""" + base = _write( + tmp_path, + "src/base.f90", + "module base\n interface\n module subroutine run()\n end subroutine run\n end interface\nend module base\n", + ) + middle = _write(tmp_path, "src/impl/middle.f90", "submodule (base) middle\nend submodule middle\n") + leaf = _write( + tmp_path, + "leaf.f90", + "submodule (base:middle) leaf\ncontains\n module subroutine run()\n end subroutine run\nend submodule leaf\n", + ) + + assert _resolve([leaf], [tmp_path / "src"]) == (base.resolve(), middle.resolve(), leaf) From 0edb411c2e4ef9e862988edc68f95ee75c519994 Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 11:16:10 +0100 Subject: [PATCH 15/47] Resolve imports and re-exports by the use nature, not the module name Use routes now carry the INTRINSIC or NON_INTRINSIC nature their use statement states, and re-export resolution reads it instead of a global set of intrinsic names: an intrinsic use names the processor module, a non_intrinsic use always names the parsed module, and a use stating no nature names a parsed module of that name when one exists. A module used intrinsic is never enumerated through a same-named user module. A facade re-exporting from a user iso_fortran_env now imports and re-exports it in the generated contract. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 3 ++ prik/parsers/fortran/scope.py | 32 +++++++++++---- prik/semantics/fortran2ir.py | 32 +++++++++++---- .../test_generated_module_contracts.py | 27 +++++++++++++ .../semantics/test_reexport_accessibility.py | 39 +++++++++++++++++++ 5 files changed, 119 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 00441270e..be4fb1ac5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,9 @@ release tags add a leading `v` to the package version. MPI's `mpi_f08` is supplied by its entry file. Discovery honors `use, intrinsic` and `use, non_intrinsic`, and follows each submodule to its direct parent. A needed module with no source, or with several, is an error. + Import and re-export resolution follows the same rule, so a user module + named like an intrinsic one, such as `iso_fortran_env`, is imported and + re-exported when a `use` selects it. - Mutable fixed-storage Fortran module scalars expose native-backed rank-zero NumPy views, including fixed-length character bytes; primitive and fixed diff --git a/prik/parsers/fortran/scope.py b/prik/parsers/fortran/scope.py index 9564d8a05..327205c63 100644 --- a/prik/parsers/fortran/scope.py +++ b/prik/parsers/fortran/scope.py @@ -26,10 +26,15 @@ @dataclass(frozen=True) class UseRoute: - """One way a scope reaches a name: the module used, and the name there.""" + """One way a scope reaches a name: the module used, and the name there. + + ``nature`` is the ``intrinsic`` or ``non_intrinsic`` the ``use`` stated, if + any; it decides whether the processor module or a same-named one is meant. + """ module: str source_name: str + nature: str | None = None @property def key(self) -> tuple[str, str]: @@ -53,6 +58,17 @@ def modules(self) -> tuple[str, ...]: """Return each used module once, spelled as its first statement wrote it.""" return tuple(statements[0].module for statements in self._by_module.values()) + def nature(self, module: str) -> str | None: + """Return the nature the statements for ``module`` state, or ``None``.""" + return next( + (statement.nature for statement in self._by_module.get(module.casefold(), ()) if statement.nature), + None, + ) + + def _offered(self, module: str, offered: OfferedNames) -> Collection[str] | None: + """Return the names ``module`` offers; an intrinsic module's cannot be enumerated.""" + return None if self.nature(module) == "intrinsic" else offered(module) + def imports_all(self, module: str) -> bool: """Return whether any statement for ``module`` omitted ``only``.""" return any(not statement.only for statement in self._by_module.get(module.casefold(), ())) @@ -78,14 +94,14 @@ def routes_for(self, local_name: str, offered: OfferedNames) -> tuple[UseRoute, for module in self.modules(): for mapping in self.mappings(module): if mapping.local_name.casefold() == folded: - route = UseRoute(module, mapping.source) + route = UseRoute(module, mapping.source, self.nature(module)) routes.setdefault(route.key, route) for module in self.modules(): if not self.imports_all(module) or folded in self._renamed_away(module): continue - names = offered(module) + names = self._offered(module, offered) if names is not None and folded in names: - route = UseRoute(module, local_name) + route = UseRoute(module, local_name, self.nature(module)) routes.setdefault(route.key, route) return tuple(routes.values()) @@ -103,7 +119,7 @@ def accessible_names(self, offered: OfferedNames) -> tuple[str, ...]: if not self.imports_all(module): continue renamed_away = self._renamed_away(module) - for name in sorted(offered(module) or ()): + for name in sorted(self._offered(module, offered) or ()): if name not in renamed_away: names.setdefault(name.casefold(), name) return tuple(names.values()) @@ -118,9 +134,11 @@ def unresolved_routes_for(self, local_name: str, offered: OfferedNames) -> tuple """ folded = local_name.casefold() return tuple( - UseRoute(module, local_name) + UseRoute(module, local_name, self.nature(module)) for module in self.modules() - if self.imports_all(module) and offered(module) is None and folded not in self._renamed_away(module) + if self.imports_all(module) + and self._offered(module, offered) is None + and folded not in self._renamed_away(module) ) def _renamed_away(self, module: str) -> frozenset[str]: diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index 3caba5361..d3ab8bf8a 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -2005,7 +2005,10 @@ def _use_associations( if local_name.casefold() in declared or not routes: continue origin = cls._reconcile_routes( - [cls._resolve_reexport_origin(index, route.module, route.source_name) for route in routes] + [ + cls._resolve_reexport_origin(index, route.module, route.source_name, nature=route.nature) + for route in routes + ] ) if origin is not None: yield local_name, tuple(dict.fromkeys(route.module for route in routes)), origin @@ -2075,6 +2078,7 @@ def _resolve_reexport_origin( module_name: str, source_name: str, seen: frozenset[tuple[str, str]] = frozenset(), + nature: str | None = None, ) -> tuple[str, str, str]: """Return where a published name is declared, following every hop. @@ -2091,10 +2095,18 @@ def _resolve_reexport_origin( only while every route through that module names one entity. Two routes naming different declarations leave the origin genuinely ambiguous there, exactly as they would in the importing module. + + ``nature`` is what the ``use`` reaching this module stated. An + ``intrinsic`` use names the processor's module, which has no declaration + to name and no contract a name could be read from. A ``use`` stating no + nature does too for a known intrinsic name, but only when no parsed + module of that name exists; ``non_intrinsic`` always names that module. """ - if module_name.casefold() in _INTRINSIC_FORTRAN_MODULES: - # The compiler supplies it: there is no declaration to name, and no - # contract a name could be read from. + if nature == "intrinsic" or ( + nature is None + and module_name.casefold() in _INTRINSIC_FORTRAN_MODULES + and module_name.casefold() not in index + ): return "intrinsic", module_name, source_name key = (module_name.casefold(), source_name.casefold()) declaring = index.get(module_name.casefold()) @@ -2109,7 +2121,10 @@ def _resolve_reexport_origin( if not routes or not cls._effective_accessibility(declaring)(source_name, route_names): return "unknown", module_name, source_name origin = cls._reconcile_routes( - [cls._resolve_reexport_origin(index, route.module, route.source_name, seen) for route in routes] + [ + cls._resolve_reexport_origin(index, route.module, route.source_name, seen, nature=route.nature) + for route in routes + ] ) return origin if origin is not None else ("unknown", module_name, source_name) @@ -2536,7 +2551,9 @@ def _resolve_derived_type_origin_from_uses( index = module_index or {} offered = self._offered_type_names(index) routes = scope.routes_for(local_name, offered) - identities = {self._declared_type_identity(index, route.module, route.source_name) for route in routes} + identities = { + self._declared_type_identity(index, route.module, route.source_name, route.nature) for route in routes + } if len(identities) == 1: module, name = identities.pop() return _ResolvedDerivedTypeOrigin(module, name) @@ -2579,9 +2596,10 @@ def _declared_type_identity( index: Mapping[str, FortranModule], module_name: str, source_name: str, + nature: str | None = None, ) -> tuple[str, str]: """Return the module and name declaring a type reached through ``module_name``.""" - kind, origin_module, origin_name = cls._resolve_reexport_origin(index, module_name, source_name) + kind, origin_module, origin_name = cls._resolve_reexport_origin(index, module_name, source_name, nature=nature) if kind == "derived_type": return origin_module, origin_name return module_name, source_name diff --git a/tests/fortran/modules/pipeline/test_generated_module_contracts.py b/tests/fortran/modules/pipeline/test_generated_module_contracts.py index 7e78a4e77..d46fb06f6 100644 --- a/tests/fortran/modules/pipeline/test_generated_module_contracts.py +++ b/tests/fortran/modules/pipeline/test_generated_module_contracts.py @@ -3,6 +3,8 @@ from __future__ import annotations from pathlib import Path +import subprocess +import sys import pytest @@ -39,3 +41,28 @@ def test_generated_module_contract_matches_fixture( tmp_path: Path, ): assert_generated_contract_matches_fixture(case, tmp_path) + + +def test_non_intrinsic_module_with_an_intrinsic_name_is_imported_and_reexported(tmp_path: Path): + """A facade re-export from a user ``iso_fortran_env`` reaches the contract as an import.""" + user_module = tmp_path / "user_env.f90" + user_module.write_text( + "module iso_fortran_env\n implicit none\n integer :: my_value = 7\nend module iso_fortran_env\n", + encoding="utf-8", + ) + facade = tmp_path / "facade.f90" + facade.write_text( + "module facade\n use, non_intrinsic :: iso_fortran_env, only: my_value\n implicit none\nend module facade\n", + encoding="utf-8", + ) + contract = tmp_path / "contract" + subprocess.run( + [sys.executable, "-m", "prik", "generate", "--pyi", str(user_module), str(facade), "--out", str(contract)], + check=True, + capture_output=True, + text=True, + ) + + text = (contract / "facade.pyi").read_text(encoding="utf-8") + assert "from .iso_fortran_env import my_value" in text + assert '"my_value"' in text diff --git a/tests/fortran/modules/semantics/test_reexport_accessibility.py b/tests/fortran/modules/semantics/test_reexport_accessibility.py index f8204111a..0839d39e8 100644 --- a/tests/fortran/modules/semantics/test_reexport_accessibility.py +++ b/tests/fortran/modules/semantics/test_reexport_accessibility.py @@ -947,3 +947,42 @@ def test_a_non_only_rename_still_carries_imported_compile_time_symbols(tmp_path: ) assert (declared.kind, declared.shape) == ("8", ["4"]) + + +USER_ISO_FORTRAN_ENV = """\ +module iso_fortran_env + implicit none + integer :: my_value = 7 +end module iso_fortran_env +""" + + +@pytest.mark.parametrize( + ("statement", "expected"), + [ + pytest.param( + "use, non_intrinsic :: iso_fortran_env, only: my_value", + ("my_value", "variable", "iso_fortran_env"), + id="non-intrinsic-names-the-user-module", + ), + pytest.param( + "use iso_fortran_env, only: my_value", + ("my_value", "variable", "iso_fortran_env"), + id="unstated-prefers-the-parsed-module", + ), + pytest.param( + "use, intrinsic :: iso_fortran_env, only: int32", + ("int32", "intrinsic", "iso_fortran_env"), + id="intrinsic-names-the-processor-module", + ), + ], +) +def test_use_nature_decides_whether_an_intrinsic_name_is_the_users_module(tmp_path: Path, statement, expected): + """A user module may share an intrinsic module's name; the ``use`` nature decides which is meant.""" + source = tmp_path / "project.f90" + source.write_text(f"{USER_ISO_FORTRAN_ENV}\nmodule facade\n {statement}\nend module facade\n", encoding="utf-8") + + modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) + facade = next(module for module in modules if module.name == "facade") + + assert [(item.local_name, item.entity_kind, item.origin_module) for item in facade.reexports] == [expected] From e444a8b20a935280da6860407972b85b58ac80bc Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 12:11:26 +0100 Subject: [PATCH 16/47] Never read a same-named user module through use, intrinsic UseRoute.names_parsed_module states the rule once: an intrinsic use selects the processor module, so nothing it reaches is read from a parsed module of that name. Callback interface lookup, specification-expression procedure scoping, imported generic specifics, and the parser's compile-time constant import all consult it, and the intrinsic-constant alias fallback no longer applies under use, non_intrinsic. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 7 +- prik/parsers/fortran/parser.py | 4 +- prik/parsers/fortran/scope.py | 9 +++ prik/semantics/fortran2ir.py | 9 ++- .../parsing/test_project_scope_models.py | 22 +++++++ .../semantics/test_reexport_accessibility.py | 65 +++++++++++++++++++ 6 files changed, 109 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index be4fb1ac5..35966faf7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,9 +14,10 @@ release tags add a leading `v` to the package version. MPI's `mpi_f08` is supplied by its entry file. Discovery honors `use, intrinsic` and `use, non_intrinsic`, and follows each submodule to its direct parent. A needed module with no source, or with several, is an error. - Import and re-export resolution follows the same rule, so a user module - named like an intrinsic one, such as `iso_fortran_env`, is imported and - re-exported when a `use` selects it. + Import, re-export, callback, generic, specification-expression, and constant + resolution follow the same rule, so a user module named like an intrinsic + one, such as `iso_fortran_env`, is read when a `use` selects it and never + when `use, intrinsic` selects the processor module. - Mutable fixed-storage Fortran module scalars expose native-backed rank-zero NumPy views, including fixed-length character bytes; primitive and fixed diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index 6f0c1293d..4701a0d3c 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -5522,7 +5522,7 @@ def _imported_compile_time_symbols( expressions = { offered[route.module][route.source_name.casefold()] for route in scope.routes_for(name, lambda module: offered[module]) - if route.source_name.casefold() in offered[route.module] + if route.names_parsed_module and route.source_name.casefold() in offered[route.module] } # Routes that disagree leave the name meaning more than one value, # which is not something to choose between. @@ -5533,7 +5533,7 @@ def _imported_compile_time_symbols( # An intrinsic module has no parsed symbols, so a name imported from # one stands for its own target-dependent spelling. for module in scope.modules(): - if module.casefold() not in _INTRINSIC_COMPILE_TIME_MODULES: + if module.casefold() not in _INTRINSIC_COMPILE_TIME_MODULES or scope.nature(module) == "non_intrinsic": continue for mapping in scope.mappings(module): imported.setdefault(mapping.local_name.casefold(), mapping.source) diff --git a/prik/parsers/fortran/scope.py b/prik/parsers/fortran/scope.py index 327205c63..22b51b0d6 100644 --- a/prik/parsers/fortran/scope.py +++ b/prik/parsers/fortran/scope.py @@ -41,6 +41,15 @@ def key(self) -> tuple[str, str]: """Return the case-folded identity two spellings of one route share.""" return self.module.casefold(), self.source_name.casefold() + @property + def names_parsed_module(self) -> bool: + """Return whether this route can name a parsed module rather than the processor's. + + An ``intrinsic`` use selects the processor module even when a parsed + module shares its name, so nothing it reaches is read from that module. + """ + return self.nature != "intrinsic" + class ScopeUses: """One scope's ``use`` statements, grouped by the module each names. diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index d3ab8bf8a..894559bbd 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -944,7 +944,9 @@ def _reached_callback_interface( route: UseRoute, exported: dict[str, dict[str, _CallbackInterface]], ) -> _CallbackInterface | None: - """Return the interface one route reaches, or ``None`` for an unread module.""" + """Return the interface one route reaches, or ``None`` for an unread or processor module.""" + if not route.names_parsed_module: + return None lookup = exported.get(route.module) return None if lookup is None else lookup.get(route.source_name.casefold()) @@ -2245,7 +2247,8 @@ def _resolve_declaration_callable( scope = ScopeUses(context.uses) offered = self._known_procedure_names() routes = scope.routes_for(name, offered) - if len({route.key for route in routes}) != 1: + # A processor module's procedure has no parsed declaration to name. + if len({route.key for route in routes}) != 1 or not routes[0].names_parsed_module: return None return SemanticExpressionCallable( name=name, @@ -3827,6 +3830,8 @@ def _imported_generic_interfaces( """ contributors: list[tuple[FortranModule, FortranInterface]] = [] for route in cls._name_routes(module, modules, generic_name): + if not route.names_parsed_module: + continue source_module = modules.get(route.module.casefold()) if source_module is None: continue diff --git a/tests/fortran/modules/parsing/test_project_scope_models.py b/tests/fortran/modules/parsing/test_project_scope_models.py index b4ccf8343..ccf9e156c 100644 --- a/tests/fortran/modules/parsing/test_project_scope_models.py +++ b/tests/fortran/modules/parsing/test_project_scope_models.py @@ -645,3 +645,25 @@ def test_project_resolution_uses_file_level_use_only_and_local_parameters(tmp_pa "rk", "n", ] + + +@pytest.mark.parametrize( + ("nature", "kind"), + [ + pytest.param("non_intrinsic", "3", id="user-module-value"), + pytest.param("intrinsic", "real64", id="processor-spelling"), + ], +) +def test_an_imported_kind_constant_follows_the_use_nature(nature: str, kind: str): + """A kind named through ``use, intrinsic`` is the processor's, even beside a same-named user module.""" + project = parse_fortran_project( + { + "user.f90": "module iso_fortran_env\n integer, parameter :: real64 = 3\nend module iso_fortran_env\n", + "consumer.f90": ( + f"module consumer\n use, {nature} :: iso_fortran_env, only: wp => real64\n" + " real(kind=wp) :: v\nend module consumer\n" + ), + } + ) + + assert project.modules["consumer"].variables[0].kind == kind diff --git a/tests/fortran/modules/semantics/test_reexport_accessibility.py b/tests/fortran/modules/semantics/test_reexport_accessibility.py index 0839d39e8..6a2ed6135 100644 --- a/tests/fortran/modules/semantics/test_reexport_accessibility.py +++ b/tests/fortran/modules/semantics/test_reexport_accessibility.py @@ -986,3 +986,68 @@ def test_use_nature_decides_whether_an_intrinsic_name_is_the_users_module(tmp_pa facade = next(module for module in modules if module.name == "facade") assert [(item.local_name, item.entity_kind, item.origin_module) for item in facade.reexports] == [expected] + + +USER_IEEE_ARITHMETIC = """\ +module ieee_arithmetic + implicit none + abstract interface + subroutine ieee_cb(x) + real, intent(inout) :: x + end subroutine ieee_cb + end interface + interface ieee_scale + module procedure scale_real + end interface ieee_scale +contains + subroutine scale_real(x) + real, intent(inout) :: x + end subroutine scale_real + pure integer function ieee_size(n) + integer, intent(in) :: n + ieee_size = n + end function ieee_size +end module ieee_arithmetic +""" + + +@pytest.mark.parametrize( + ("nature", "callback_storage", "bound_scope", "specifics"), + [ + pytest.param("non_intrinsic", "callback", "ieee_arithmetic", ["scale_real", "scale_int"], id="user-module"), + pytest.param("intrinsic", "reference", None, ["scale_int"], id="processor-module"), + ], +) +def test_an_intrinsic_use_reads_nothing_from_a_same_named_user_module( + tmp_path: Path, nature, callback_storage, bound_scope, specifics +): + """Callbacks, specification-expression calls, and generics all follow the ``use`` nature.""" + consumer = f"""\ +module consumer + use, {nature} :: ieee_arithmetic, only: ieee_cb, ieee_scale, ieee_size + implicit none + interface ieee_scale + module procedure scale_int + end interface ieee_scale +contains + subroutine scale_int(i) + integer, intent(inout) :: i + end subroutine scale_int + subroutine apply(cb, n, values) + procedure(ieee_cb) :: cb + integer, intent(in) :: n + real, intent(inout) :: values(ieee_size(n)) + end subroutine apply +end module consumer +""" + source = tmp_path / "project.f90" + source.write_text(f"{USER_IEEE_ARITHMETIC}\n{consumer}", encoding="utf-8") + + modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) + module = next(item for item in modules if item.name == "consumer") + callback, _count, values = next(function for function in module.functions if function.name == "apply").arguments + (bound_call,) = values.semantic_type.storage.array.expression_callables[0] + + assert callback.semantic_type.storage.kind == callback_storage + assert (bound_call.name, bound_call.native_scope) == ("ieee_size", bound_scope) + assert [procedure.name for procedure in module.overload_sets[0].procedures] == specifics From 1fd93304e14296a270190c5e7c5749f3c2d63052 Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 12:40:44 +0100 Subject: [PATCH 17/47] Own merged imported generics, share the intrinsic inventory, and find macro-named modules - A module reaching two generics of one name through separate use statements owns the merged generic as its own overload set instead of re-exporting the first contributor, so a pure facade dispatches over every specific and --export-symbols facade_mod::convert selects the whole generic. - Discovery and semantic resolution read one intrinsic-module inventory, and a derived type reached from a processor module (ieee_arithmetic included) is left to the processor rather than wrapped from a same-named parse. - --module-source-dir falls back to preprocessing and parsing the searched sources when the raw scan names no module, so a module named by a macro or an include is found. - The Open MPI test and tutorial pass mpifort --showme:incdirs and --showme:compile through verbatim. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 14 +- docs/user/tutorials/openmpi-f08.md | 5 +- prik/parsers/fortran/intrinsic_modules.py | 24 ++++ prik/parsers/fortran/module_sources.py | 67 +++++---- prik/policy/exports.py | 2 +- prik/semantics/fortran2ir.py | 127 ++++++++++++++---- prik/semantics/fortran_exports.py | 2 + .../end_to_end/test_openmpi_f08.py | 5 +- .../fixtures/native/pure_facade_generic.f90 | 32 +++++ .../end_to_end/test_merged_generic_replay.py | 39 +++++- .../test_generic_contributor_merging.py | 35 ++++- .../parsing/test_module_source_resolution.py | 24 ++++ .../semantics/test_reexport_accessibility.py | 35 +++++ 13 files changed, 350 insertions(+), 61 deletions(-) create mode 100644 prik/parsers/fortran/intrinsic_modules.py create mode 100644 tests/fortran/generic_interfaces/end_to_end/fixtures/native/pure_facade_generic.f90 diff --git a/CHANGELOG.md b/CHANGELOG.md index 35966faf7..02c80803f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,11 +13,21 @@ release tags add a leading `v` to the package version. defines the module and reads it too, so a multi-module library such as Open MPI's `mpi_f08` is supplied by its entry file. Discovery honors `use, intrinsic` and `use, non_intrinsic`, and follows each submodule to its - direct parent. A needed module with no source, or with several, is an error. + direct parent. A module whose name a macro or an `#include` supplies is + found by preprocessing and parsing the searched sources when a plain scan + of them does not name it. A needed module with no source, or with several, + is an error. Import, re-export, callback, generic, specification-expression, and constant resolution follow the same rule, so a user module named like an intrinsic one, such as `iso_fortran_env`, is read when a `use` selects it and never - when `use, intrinsic` selects the processor module. + when `use, intrinsic` selects the processor module. A derived type reached + from a processor module, such as `ieee_arithmetic`'s, is left to the + processor rather than read from a parsed module of that name. +- A module that reaches two generics of one name through separate `use` + statements, without declaring the generic itself, owns the merged generic: + it dispatches over every contributor's specifics in source and contract + builds, and `--export-symbols facade_mod::convert` selects the merged + generic. It previously re-exported the first contributor only. - Mutable fixed-storage Fortran module scalars expose native-backed rank-zero NumPy views, including fixed-length character bytes; primitive and fixed diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 4bbea88e2..e5c9a89f3 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -77,12 +77,11 @@ def show(option): command, compile_flags = show("--showme:command"), show("--showme:compile") if len(command) != 1: raise SystemExit(f"mpifort wraps the multi-token command {command}; set one compiler executable") -include_dirs = [*show("--showme:incdirs"), *(flag[2:] for flag in compile_flags if flag.startswith("-I"))] build_pyi_extension( "contract/__init__.pyi", input_compiler=command[0], - native_include_dirs=list(dict.fromkeys(include_dirs)), - wrapper_fortran_flags=[flag for flag in compile_flags if not flag.startswith("-I")], + native_include_dirs=show("--showme:incdirs"), + wrapper_fortran_flags=compile_flags, native_link_items=[NativeLinkItem("linker_argument", flag) for flag in show("--showme:link")], native_linker_language="fortran", output_name="prik_openmpi_f08", diff --git a/prik/parsers/fortran/intrinsic_modules.py b/prik/parsers/fortran/intrinsic_modules.py new file mode 100644 index 000000000..ec4f6c53f --- /dev/null +++ b/prik/parsers/fortran/intrinsic_modules.py @@ -0,0 +1,24 @@ +"""Modules a Fortran processor supplies without a source file. + +Source discovery and semantic resolution both ask whether a ``use`` stating +no nature names a processor module, and they must give the same answer, so +the inventory lives here once. +""" + +from __future__ import annotations + +#: The standard intrinsic modules, and processor modules the common Fortran +#: toolchains supply for OpenMP and OpenACC. A ``use`` stating no nature names +#: one of these only when no parsed module of that name is accessible. +INTRINSIC_FORTRAN_MODULES = frozenset( + { + "iso_c_binding", + "iso_fortran_env", + "ieee_arithmetic", + "ieee_exceptions", + "ieee_features", + "omp_lib", + "omp_lib_kinds", + "openacc", + } +) diff --git a/prik/parsers/fortran/module_sources.py b/prik/parsers/fortran/module_sources.py index 87c470ff6..fbe15aef1 100644 --- a/prik/parsers/fortran/module_sources.py +++ b/prik/parsers/fortran/module_sources.py @@ -18,6 +18,7 @@ from pathlib import Path import re +from prik.parsers.fortran.intrinsic_modules import INTRINSIC_FORTRAN_MODULES from prik.parsers.fortran.models import FortranFile, FortranParseError from prik.parsers.fortran.parser import FortranParser from prik.parsers.fortran.scope import used_module_statements @@ -35,20 +36,6 @@ r"(?P[a-z][a-z0-9_]*)[ \t]*(?:!.*)?$", re.IGNORECASE | re.MULTILINE, ) -# Modules a processor supplies. A ``use`` stating no nature falls back to one of -# these only when no searched source defines a module of that name. -_INTRINSIC_MODULES = frozenset( - { - "iso_c_binding", - "iso_fortran_env", - "ieee_arithmetic", - "ieee_exceptions", - "ieee_features", - "omp_lib", - "omp_lib_kinds", - "openacc", - } -) def resolve_fortran_module_sources( @@ -65,7 +52,7 @@ def resolve_fortran_module_sources( under ``search_dirs``; otherwise a :class:`FortranParseError` names it and the source that needs it. """ - candidates = _unit_candidates(search_dirs) + candidates, searched_files = _unit_candidates(search_dirs) parser = FortranParser() facts: dict[Path, tuple[set[str], dict[str, str | None]]] = {} owners: dict[str, Path] = {} @@ -78,6 +65,26 @@ def read(path: Path) -> tuple[set[str], dict[str, str | None]]: owners.setdefault(unit, path) return facts[path] + scanned: set[Path] = set() + + def parsed_definers(unit: str) -> list[Path]: + """Parse every searched source not yet read and return those defining ``unit``. + + A module named through a macro or an included line has no ``module`` + line the fast index can see, so the preprocessed sources are read + once, when a needed unit is otherwise missing. A source that cannot + be preprocessed or parsed cannot define it. + """ + for path in searched_files: + if path in scanned: + continue + scanned.add(path) + try: + read(path) + except Exception: # an unreadable candidate defines nothing + continue + return [path for path in searched_files if path in facts and unit in facts[path][0]] + for entry in entries: read(entry.resolve()) ordered: list[Path] = [] @@ -91,7 +98,7 @@ def visit(path: Path) -> None: for unit, nature in sorted(required.items()): if unit in defined or nature == "intrinsic": continue - dependency = owners.get(unit) or _defining_source(unit, nature, candidates, path, read) + dependency = owners.get(unit) or _defining_source(unit, nature, candidates, path, read, parsed_definers) if dependency is not None: visit(dependency) visiting.discard(path) @@ -109,18 +116,24 @@ def _defining_source( candidates: dict[str, list[Path]], user: Path, read: Callable[[Path], tuple[set[str], dict[str, str | None]]], + parsed_definers: Callable[[str], list[Path]], ) -> Path | None: """Return the one searched source whose parsed units define ``unit``, or raise. A ``use`` that states no nature names an intrinsic module only when no other module of that name is accessible, so a known intrinsic name is - satisfied by the processor when no source defines it. + satisfied by the processor when no source defines it. Only a unit that is + still missing reads every searched source in full. """ defining = [path for path in candidates.get(unit, ()) if unit in read(path)[0]] if len(defining) == 1: return defining[0] - if not defining and nature is None and unit in _INTRINSIC_MODULES: + if not defining and nature is None and unit in INTRINSIC_FORTRAN_MODULES: return None + if not defining: + defining = parsed_definers(unit) + if len(defining) == 1: + return defining[0] kind = "submodule" if ":" in unit else "module" if not defining: raise FortranParseError( @@ -138,13 +151,21 @@ def _defining_source( ) -def _unit_candidates(search_dirs: Iterable[Path]) -> dict[str, list[Path]]: - """Map each module or submodule to the Fortran sources under ``search_dirs`` that open it.""" +def _unit_candidates(search_dirs: Iterable[Path]) -> tuple[dict[str, list[Path]], list[Path]]: + """Map each module or submodule to the sources under ``search_dirs`` that open it, and list every source. + + The map reads raw ``module`` and ``submodule`` lines, the fast path; the + list lets a unit those lines cannot show be found by parsing. + """ candidates: dict[str, list[Path]] = {} + searched: list[Path] = [] for directory in search_dirs: for path in sorted(Path(directory).rglob("*")): if path.suffix.casefold() not in _FORTRAN_SOURCE_SUFFIXES or not path.is_file(): continue + resolved = path.resolve() + if resolved not in searched: + searched.append(resolved) text = path.read_text(encoding="utf-8", errors="replace") units = [match.group("name").casefold() for match in _MODULE_LINE.finditer(text)] units.extend( @@ -153,9 +174,9 @@ def _unit_candidates(search_dirs: Iterable[Path]) -> dict[str, list[Path]]: ) for unit in units: paths = candidates.setdefault(unit, []) - if path.resolve() not in paths: - paths.append(path.resolve()) - return candidates + if resolved not in paths: + paths.append(resolved) + return candidates, searched def _defined_and_required_units(parsed: FortranFile) -> tuple[set[str], dict[str, str | None]]: diff --git a/prik/policy/exports.py b/prik/policy/exports.py index 1972d1123..e02320592 100644 --- a/prik/policy/exports.py +++ b/prik/policy/exports.py @@ -677,7 +677,7 @@ def imported_type_reference(semantic_type: models.SemanticType) -> ImportedTypeR a module the contract imports itself. """ ref = semantic_type.metadata.get(models.EXTERNAL_TYPE_REF_METADATA) - if not isinstance(ref, dict): + if not isinstance(ref, dict) or ref.get("processor"): return None module, name = ref.get("origin_module"), ref.get("name") local = ref.get("local_name") or name diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index 894559bbd..b4420f4a7 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -22,6 +22,7 @@ import re from pathlib import Path +from prik.parsers.fortran.intrinsic_modules import INTRINSIC_FORTRAN_MODULES from prik.parsers.fortran.scope import ScopeUses, UseRoute from prik.parsers.fortran.models import ( FortranArgument, @@ -250,11 +251,14 @@ class _ResolvedDerivedTypeOrigin: ``import_scope`` records when a procedure-local ``use`` made the selected name visible, which determines whether the public semantic name is scoped. + ``processor`` records that the type belongs to a processor module, so no + parsed module of the same name declares or wraps it. """ module: str | None name: str import_scope: str | None = None + processor: bool = False @dataclass(frozen=True) @@ -324,7 +328,6 @@ def substitute(fragment: str) -> str: # Language-owned modules are contract vocabulary, not sibling contract leaves. -_INTRINSIC_FORTRAN_MODULES = frozenset({"iso_c_binding", "iso_fortran_env"}) class FortranToIRConverter(ClassVisitor): @@ -1945,6 +1948,51 @@ def _reconcile_routes(origins: list[tuple[str, str, str]]) -> tuple[str, str, st return distinct[0] return None + @classmethod + def _merges_generics(cls, name: str, origins: list[tuple[str, str, str]]) -> bool: + """Return whether a name's routes reach several generics, which merge into one. + + Only a procedure generic is assembled into a module's overload set; a + defined operator attaches to the types it operates on instead. + """ + distinct = set(origins) + return ( + len(distinct) > 1 + and all(kind == "generic" for kind, _module, _name in distinct) + and cls._is_procedure_generic_name(name) + ) + + @classmethod + def _route_origins( + cls, + module: FortranModule, + index: dict[str, FortranModule], + routes: tuple[UseRoute, ...], + ) -> list[tuple[str, str, str]]: + """Return the declaration each of one module's routes to a name reaches.""" + return [ + cls._resolve_reexport_origin(index, route.module, route.source_name, nature=route.nature) + for route in routes + ] + + @classmethod + def _merged_imported_generics(cls, module: FortranModule, index: dict[str, FortranModule]) -> tuple[str, ...]: + """Return the generics this module assembles from several imported ones. + + Two accessible generics spelled alike are one generic in the module + reaching both, and neither contributor declares that whole generic, so + the module where they meet owns it: it becomes that module's overload + set rather than a re-export naming one contributor. A module that + declares the name itself already owns it through its own block. + """ + declared = cls._module_declared_names(module) + return tuple( + name + for name in cls._use_associated_names(module, index) + if name.casefold() not in declared + and cls._merges_generics(name, cls._route_origins(module, index, cls._name_routes(module, index, name))) + ) + @classmethod def _offered_names(cls, index: dict[str, FortranModule], seen: frozenset[str] = frozenset()): """Return what each used module publishes, or ``None`` when unparsed.""" @@ -2006,12 +2054,11 @@ def _use_associations( routes = cls._name_routes(module, index, local_name) if local_name.casefold() in declared or not routes: continue - origin = cls._reconcile_routes( - [ - cls._resolve_reexport_origin(index, route.module, route.source_name, nature=route.nature) - for route in routes - ] - ) + origins = cls._route_origins(module, index, routes) + if cls._merges_generics(local_name, origins): + # The generic these routes merge into is this module's own. + continue + origin = cls._reconcile_routes(origins) if origin is not None: yield local_name, tuple(dict.fromkeys(route.module for route in routes)), origin @@ -2106,7 +2153,7 @@ def _resolve_reexport_origin( """ if nature == "intrinsic" or ( nature is None - and module_name.casefold() in _INTRINSIC_FORTRAN_MODULES + and module_name.casefold() in INTRINSIC_FORTRAN_MODULES and module_name.casefold() not in index ): return "intrinsic", module_name, source_name @@ -2122,12 +2169,14 @@ def _resolve_reexport_origin( route_names = tuple(dict.fromkeys(route.module for route in routes)) if not routes or not cls._effective_accessibility(declaring)(source_name, route_names): return "unknown", module_name, source_name - origin = cls._reconcile_routes( - [ - cls._resolve_reexport_origin(index, route.module, route.source_name, seen, nature=route.nature) - for route in routes - ] - ) + origins = [ + cls._resolve_reexport_origin(index, route.module, route.source_name, seen, nature=route.nature) + for route in routes + ] + if cls._merges_generics(source_name, origins): + # Generics merged here form a generic this module owns. + return "generic", declaring.name, source_name + origin = cls._reconcile_routes(origins) return origin if origin is not None else ("unknown", module_name, source_name) @staticmethod @@ -2494,7 +2543,7 @@ def _derived_type_ref( local_type = bool(context is not None and context.module and local_name.lower() in context.local_types) if local_type or origin.module is None: return None - wrapped = bool((origin.module.lower(), origin.name.lower()) in self.wrapped_derived_types) + wrapped = not origin.processor and (origin.module.lower(), origin.name.lower()) in self.wrapped_derived_types public_name = local_name if origin.import_scope == "procedure": public_name = f"{origin.module}.{origin.name}" @@ -2507,6 +2556,9 @@ def _derived_type_ref( } if origin.import_scope is not None: metadata["import_scope"] = origin.import_scope + if origin.processor: + # No parsed module declares it, so no contract imports it. + metadata["processor"] = True return public_name, metadata def _resolve_derived_type_origin( @@ -2533,7 +2585,9 @@ def _resolve_derived_type_origin( local_name, context.procedure_uses, context.module_index ) if (procedure_resolved.module, procedure_resolved.name) == (resolved.module, resolved.name): - return _ResolvedDerivedTypeOrigin(resolved.module, resolved.name, import_scope="procedure") + return _ResolvedDerivedTypeOrigin( + resolved.module, resolved.name, import_scope="procedure", processor=resolved.processor + ) return resolved def _resolve_derived_type_origin_from_uses( @@ -2558,15 +2612,33 @@ def _resolve_derived_type_origin_from_uses( self._declared_type_identity(index, route.module, route.source_name, route.nature) for route in routes } if len(identities) == 1: - module, name = identities.pop() - return _ResolvedDerivedTypeOrigin(module, name) + module, name, processor = identities.pop() + return _ResolvedDerivedTypeOrigin(module, name, processor=processor) if identities: return _ResolvedDerivedTypeOrigin(None, local_name) unresolved = scope.unresolved_routes_for(local_name, offered) if len({route.key for route in unresolved}) == 1: - return _ResolvedDerivedTypeOrigin(unresolved[0].module, unresolved[0].source_name) + route = unresolved[0] + return _ResolvedDerivedTypeOrigin( + route.module, route.source_name, processor=self._names_processor_module(index, route) + ) return _ResolvedDerivedTypeOrigin(None, local_name) + @staticmethod + def _names_processor_module(index: Mapping[str, FortranModule], route: UseRoute) -> bool: + """Return whether a route names a processor module rather than a parsed one. + + ``use, intrinsic`` always does. A ``use`` stating no nature does for a + known intrinsic name that no parsed module provides. + """ + if not route.names_parsed_module: + return True + return ( + route.nature is None + and route.module.casefold() in INTRINSIC_FORTRAN_MODULES + and route.module.casefold() not in index + ) + def _offered_type_names(self, index: Mapping[str, FortranModule]): """Return the type names each module declares or publicly re-exports, or ``None``.""" declared = self._wrapped_type_names() @@ -2600,12 +2672,12 @@ def _declared_type_identity( module_name: str, source_name: str, nature: str | None = None, - ) -> tuple[str, str]: - """Return the module and name declaring a type reached through ``module_name``.""" + ) -> tuple[str, str, bool]: + """Return the module and name declaring a type reached through ``module_name``, and whether the processor does.""" kind, origin_module, origin_name = cls._resolve_reexport_origin(index, module_name, source_name, nature=nature) if kind == "derived_type": - return origin_module, origin_name - return module_name, source_name + return origin_module, origin_name, False + return module_name, source_name, kind == "intrinsic" def _wrapped_type_names(self): """Return the wrapped type names each module declares, or ``None``.""" @@ -3253,7 +3325,14 @@ def _module_overload_sets( overload_sets: list[ProcedureOverloadSet] = [] inherited_functions: list[SemanticFunction] = [] class_map = {semantic_class.name.casefold(): semantic_class for semantic_class in semantic_classes} - for interface in module.interfaces: + # A generic merged from several imported ones is declared by no block + # here, yet this module owns it, so it is assembled as a block that + # adds no specifics of its own to what it inherits. + merged = [ + FortranInterface(name=name, module=module.name) + for name in self._merged_imported_generics(module, module_index or {}) + ] + for interface in [*module.interfaces, *merged]: if not interface.name or interface.abstract: continue if interface.declaring_scope_kind == "procedure": diff --git a/prik/semantics/fortran_exports.py b/prik/semantics/fortran_exports.py index e8a94ff3a..5d4349b4c 100644 --- a/prik/semantics/fortran_exports.py +++ b/prik/semantics/fortran_exports.py @@ -200,6 +200,8 @@ def _type_identities(semantic_type, module_name: str) -> set[tuple[str, str]]: identities = set() for item in _semantic_type_tree(semantic_type): reference = item.metadata.get(EXTERNAL_TYPE_REF_METADATA) + if isinstance(reference, dict) and reference.get("processor"): + continue origin = reference.get("origin_module") if isinstance(reference, dict) else module_name name = reference.get("name") if isinstance(reference, dict) else item.name identities.add((str(origin).casefold(), str(name).casefold())) diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index be067cba6..dc940c2a8 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -168,12 +168,11 @@ def show(flag: str) -> list[str]: command, compile_flags = show("--showme:command"), show("--showme:compile") if len(command) != 1: _unavailable(f"mpifort --showme:command is a multi-token command {command}; pass one compiler executable") - include_dirs = [*show("--showme:incdirs"), *(flag[2:] for flag in compile_flags if flag.startswith("-I"))] result = build_pyi_extension( contract / "__init__.pyi", input_compiler=command[0], - native_include_dirs=list(dict.fromkeys(include_dirs)), - wrapper_fortran_flags=[flag for flag in compile_flags if not flag.startswith("-I")], + native_include_dirs=show("--showme:incdirs"), + wrapper_fortran_flags=compile_flags, native_link_items=[NativeLinkItem("linker_argument", flag) for flag in show("--showme:link")], native_linker_language="fortran", output_name="prik_openmpi_f08", diff --git a/tests/fortran/generic_interfaces/end_to_end/fixtures/native/pure_facade_generic.f90 b/tests/fortran/generic_interfaces/end_to_end/fixtures/native/pure_facade_generic.f90 new file mode 100644 index 000000000..06abe856d --- /dev/null +++ b/tests/fortran/generic_interfaces/end_to_end/fixtures/native/pure_facade_generic.f90 @@ -0,0 +1,32 @@ +module ints_mod + implicit none + private + public :: convert + interface convert + module procedure convert_int + end interface convert +contains + integer function convert_int(i) + integer, intent(in) :: i + convert_int = i + 1 + end function convert_int +end module ints_mod + +module reals_mod + implicit none + private + public :: convert + interface convert + module procedure convert_real + end interface convert +contains + real(8) function convert_real(x) + real(8), intent(in) :: x + convert_real = x * 2 + end function convert_real +end module reals_mod + +module facade_mod + use ints_mod, only : convert + use reals_mod, only : convert +end module facade_mod diff --git a/tests/fortran/generic_interfaces/end_to_end/test_merged_generic_replay.py b/tests/fortran/generic_interfaces/end_to_end/test_merged_generic_replay.py index 6c23aef8d..1f23a1b4e 100644 --- a/tests/fortran/generic_interfaces/end_to_end/test_merged_generic_replay.py +++ b/tests/fortran/generic_interfaces/end_to_end/test_merged_generic_replay.py @@ -1,11 +1,13 @@ """A generic merged from several modules builds the same from source and from its contracts.""" +import subprocess +import sys from pathlib import Path import numpy as np import pytest -from prik import build_fortran_extension +from prik import build_fortran_extension, build_pyi_extension from tests.fortran._support.wrapper_build import ( _build_generated_pyi_and_import, _import_from_build_dir, @@ -37,3 +39,38 @@ def test_merged_generic_dispatches_every_contributor(tmp_path: Path, lane: str): assert facade.convert(np.int32(3)) == np.int32(3) assert facade.convert(np.float32(2.5)) == np.float32(2.5) assert facade.convert(np.bool_(True)) + + +PURE_FACADE = NATIVE_FIXTURES / "pure_facade_generic.f90" + + +@pytest.mark.parametrize("selected", [False, True], ids=["whole-project", "selected"]) +def test_a_pure_facade_dispatches_every_generic_it_merges(tmp_path: Path, selected: bool): + """A facade declaring nothing still owns the generic its two imports merge into. + + Both contributors keep their specifics private, so neither is reachable by + name. Built from source and replayed from its generated contracts, the + facade dispatches over both, and selecting ``facade_mod::convert`` selects + the merged generic rather than one contributor. + """ + exports = ("facade_mod::convert",) if selected else None + source = build_fortran_extension( + PURE_FACADE, output_dir=tmp_path / "source", output_name="pure_facade_source", export_symbols=exports + ) + command = [sys.executable, "-m", "prik", "generate", "--pyi", str(PURE_FACADE), "--out", str(tmp_path / "pyi")] + if exports: + symbols = tmp_path / "exports.txt" + symbols.write_text("\n".join(exports) + "\n", encoding="utf-8") + command += ["--export-symbols", str(symbols)] + subprocess.run(command, check=True, capture_output=True, text=True) + replay = build_pyi_extension( + tmp_path / "pyi" / "__init__.pyi", + native_fortran_sources=(PURE_FACADE,), + output_dir=tmp_path / "replay", + output_name="pure_facade_replay", + ) + + for result in (source, replay): + facade = _import_from_build_dir(result.module_name, result.output_dir).facade_mod + assert facade.convert(np.int32(3)) == np.int32(4) + assert facade.convert(np.float64(1.5)) == np.float64(3.0) diff --git a/tests/fortran/generic_interfaces/semantics/test_generic_contributor_merging.py b/tests/fortran/generic_interfaces/semantics/test_generic_contributor_merging.py index a9b77ba17..39c821978 100644 --- a/tests/fortran/generic_interfaces/semantics/test_generic_contributor_merging.py +++ b/tests/fortran/generic_interfaces/semantics/test_generic_contributor_merging.py @@ -133,8 +133,12 @@ def test_generic_contributors_survive_a_transitive_chain(tmp_path: Path): ] -def test_two_imported_generics_remain_one_accessible_name(tmp_path: Path): - """Generic routes are contributors, so they do not cancel each other out.""" +def test_a_facade_merging_two_imported_generics_owns_the_merged_generic(tmp_path: Path): + """Neither contributor declares the whole generic, so the module where they meet owns it. + + Re-exporting the first contributor published a generic that dispatched + over only that contributor's specifics. + """ modules = _modules( tmp_path, CONTRIBUTORS, @@ -147,8 +151,31 @@ def test_two_imported_generics_remain_one_accessible_name(tmp_path: Path): """, ) - reexports = {item.local_name: item for item in modules["facade_mod"].reexports} - assert reexports["convert"].entity_kind == "generic" + assert _specifics(modules["facade_mod"], "convert") == ["convert_i", "convert_r"] + assert [item.local_name for item in modules["facade_mod"].reexports] == [] + + +def test_a_module_reached_through_a_merging_facade_reexports_the_facade(tmp_path: Path): + """The merged generic is the facade's own, so a later ``use`` reaches it there.""" + modules = _modules( + tmp_path, + CONTRIBUTORS, + """\ +module facade_mod + use ints_mod, only : convert + use reals_mod, only : convert + implicit none +end module facade_mod + +module client_mod + use facade_mod + implicit none +end module client_mod +""", + ) + + reexports = {item.local_name: item for item in modules["client_mod"].reexports} + assert (reexports["convert"].entity_kind, reexports["convert"].origin_module) == ("generic", "facade_mod") def test_a_generic_and_a_variable_of_one_name_are_not_merged(tmp_path: Path): diff --git a/tests/fortran/modules/parsing/test_module_source_resolution.py b/tests/fortran/modules/parsing/test_module_source_resolution.py index 620501cf7..33a566e92 100644 --- a/tests/fortran/modules/parsing/test_module_source_resolution.py +++ b/tests/fortran/modules/parsing/test_module_source_resolution.py @@ -2,12 +2,14 @@ from __future__ import annotations +import shutil from pathlib import Path import pytest from prik.parsers.fortran import FortranParseError from prik.parsers.fortran.module_sources import resolve_fortran_module_sources +from prik.preprocessing import PreprocessingConfig, preprocess_source def _write(root: Path, relative: str, text: str) -> Path: @@ -66,6 +68,7 @@ def test_a_used_module_needs_exactly_one_defining_source(tmp_path: Path, definit pytest.param("use, non_intrinsic :: iso_fortran_env", True, True, id="non-intrinsic-uses-the-source"), pytest.param("use iso_fortran_env", True, True, id="unstated-prefers-a-source"), pytest.param("use iso_fortran_env", False, False, id="unstated-falls-back-to-the-processor"), + pytest.param("use ieee_arithmetic", False, False, id="unstated-ieee-module-falls-back-to-the-processor"), ], ) def test_use_nature_decides_whether_a_module_source_is_needed(tmp_path: Path, statement, user_source, found): @@ -104,3 +107,24 @@ def test_nested_submodule_resolves_its_direct_parent_before_the_ancestor_module( ) assert _resolve([leaf], [tmp_path / "src"]) == (base.resolve(), middle.resolve(), leaf) + + +@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran preprocessing") +def test_a_module_named_through_a_macro_is_found_by_parsing_the_searched_sources(tmp_path: Path): + """A ``module`` line the raw index cannot read is found once the preprocessed sources are parsed.""" + config = PreprocessingConfig(mode="compiler", compiler="gfortran") + generated = _write( + tmp_path, + "lib/gen.F90", + "#define MODNAME generated_mod\nmodule MODNAME\n integer, parameter :: answer = 42\nend module MODNAME\n", + ) + _write(tmp_path, "lib/broken.F90", '#include "missing_header.h"\nmodule broken\nend module broken\n') + entry = _write(tmp_path, "app.f90", "module app\n use generated_mod, only: answer\nend module app\n") + + resolved = resolve_fortran_module_sources( + [entry], + [tmp_path / "lib"], + lambda path: preprocess_source(path, language="fortran", config=config).source, + ) + + assert resolved == (generated.resolve(), entry) diff --git a/tests/fortran/modules/semantics/test_reexport_accessibility.py b/tests/fortran/modules/semantics/test_reexport_accessibility.py index 6a2ed6135..86cd48137 100644 --- a/tests/fortran/modules/semantics/test_reexport_accessibility.py +++ b/tests/fortran/modules/semantics/test_reexport_accessibility.py @@ -1051,3 +1051,38 @@ def test_an_intrinsic_use_reads_nothing_from_a_same_named_user_module( assert callback.semantic_type.storage.kind == callback_storage assert (bound_call.name, bound_call.native_scope) == ("ieee_size", bound_scope) assert [procedure.name for procedure in module.overload_sets[0].procedures] == specifics + + +@pytest.mark.parametrize( + ("nature", "processor", "wrapped"), + [ + pytest.param("intrinsic", True, False, id="processor-type"), + pytest.param("non_intrinsic", False, True, id="user-type"), + ], +) +def test_a_wildcard_use_resolves_a_derived_type_by_its_nature(tmp_path: Path, nature, processor, wrapped): + """A type reached through ``use, intrinsic`` is the processor's even beside a same-named user module.""" + user = "module ieee_arithmetic\n type :: ieee_class_type\n integer :: v\n end type ieee_class_type\nend module ieee_arithmetic\n" + consumer = ( + f"module consumer\n use, {nature} :: ieee_arithmetic\ncontains\n subroutine inspect(value)\n" + " type(ieee_class_type), intent(in) :: value\n end subroutine inspect\nend module consumer\n" + ) + source = tmp_path / "project.f90" + source.write_text(f"{user}\n{consumer}", encoding="utf-8") + + modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) + argument = next(module for module in modules if module.name == "consumer").functions[0].arguments[0] + reference = argument.semantic_type.metadata["external_type_ref"] + + assert reference["origin_module"] == "ieee_arithmetic" + assert (bool(reference.get("processor")), reference["wrapped"]) == (processor, wrapped) + + +def test_a_plain_use_of_an_ieee_module_names_the_processor_module(tmp_path: Path): + """Semantic resolution and source discovery share one inventory of processor modules.""" + source = tmp_path / "facade.f90" + source.write_text("module facade\n use ieee_arithmetic, only: ieee_is_nan\nend module facade\n", encoding="utf-8") + + (facade,) = fortran_project_to_semantic_modules(parse_fortran_project([source])) + + assert [(item.local_name, item.entity_kind) for item in facade.reexports] == [("ieee_is_nan", "intrinsic")] From d3fef1680b6fb256b59574437ce9da1f414b761b Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 14:00:17 +0100 Subject: [PATCH 18/47] Resolve module sources from preprocessed facts and identify submodules by ancestor - --module-source-dir reads what each searched source defines from its preprocessed text (a source without directives is read as written), parses the located candidates, and selects a source only once it is a unit's one definition. Reading a candidate no longer makes it a project owner, a raw definition no longer hides a macro-named duplicate, and an unstated use of an intrinsic module's name reads a defining source before the processor's. - Use natures reduce per module with non_intrinsic over unstated over intrinsic, in one helper shared by discovery, project ordering, and compile scheduling; an intrinsic use no longer orders after a same-named project source. Uses in internal procedures and BLOCK constructs are recorded as nested dependencies. - Submodules are identified as ancestor:name in project registries, dependencies, file ordering, compile batches, interface ownership, owner scopes, and compile-time symbol tables, so a:impl and b:impl coexist. - Separate module procedures declared by module function/subroutine interface bodies are wrapped under ordinary accessibility, and discovery brings in the submodules descending from each selected module. - Module docstrings describe scalar views and their nullability; the bridge documents both scalar view getters. - --export-symbols wording names procedures, generics, and module variables; the feature matrix describes live scalar views. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 35 +- docs/user/faq/index.md | 3 +- docs/user/language-support/feature-matrix.md | 2 +- docs/user/reference/cli-commands.md | 9 +- docs/user/reference/python-api.md | 4 +- prik/cli.py | 7 +- prik/codegen/docstrings.py | 36 +- prik/codegen/fortran/bridge.py | 4 + prik/parsers/fortran/models.py | 34 ++ prik/parsers/fortran/module_sources.py | 334 +++++++++++------- prik/parsers/fortran/parser.py | 125 +++++-- prik/parsers/fortran/scope.py | 75 +++- prik/pipeline/build.py | 48 +-- prik/preprocessing/source.py | 17 + prik/semantics/fortran2ir.py | 24 +- prik/semantics/fortran_exports.py | 4 +- .../end_to_end/test_multi_source_builds.py | 38 ++ .../pipeline/test_parallel_compilation.py | 32 ++ .../assumed_shape_and_derived_args.json | 18 +- .../fixtures/general/basic_subroutine.json | 6 +- .../general/compile_time_all_exprs.json | 6 +- .../general/compile_time_shape_exprs.json | 6 +- .../fixtures/general/derived_type.json | 6 +- .../fixtures/general/f77_subroutine.json | 6 +- .../fixtures/general/modern_pyi_example.json | 42 ++- .../general/procedures_and_functions.json | 12 +- .../scope_name_reuse_combinations.json | 48 ++- .../test_declaration_and_scope_regressions.py | 27 +- ...ortran_parser_procedures_and_interfaces.py | 5 +- .../parsing/test_public_entrypoints.py | 3 +- .../semantics/test_compile_time_values.py | 12 +- .../test_module_source_discovery.py | 70 +++- .../test_module_variables_and_state.py | 10 +- .../parsing/test_module_source_resolution.py | 146 +++++++- .../parsing/test_project_scope_models.py | 67 +++- 35 files changed, 979 insertions(+), 342 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 02c80803f..45cc8ffc3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,18 +11,38 @@ release tags add a leading `v` to the package version. `build_fortran_extension` accepts `module_source_dirs`: from the given entry sources, PRIK follows each `use` to the source under those directories that defines the module and reads it too, so a multi-module library such as Open - MPI's `mpi_f08` is supplied by its entry file. Discovery honors - `use, intrinsic` and `use, non_intrinsic`, and follows each submodule to its - direct parent. A module whose name a macro or an `#include` supplies is - found by preprocessing and parsing the searched sources when a plain scan - of them does not name it. A needed module with no source, or with several, - is an error. + MPI's `mpi_f08` is supplied by its entry file. What a source defines is + read from its preprocessed text, so a module a macro or an `#include` + names is found, and a second definition only preprocessing reveals makes + the module ambiguous. A needed module with no source, or with several, is + an error. Discovery honors `use, intrinsic` and `use, non_intrinsic` for + each scope separately, and an unstated `use` of an intrinsic module's name + reads a source defining it before falling back to the processor. It + follows `use` statements in internal procedures and `BLOCK` constructs, + follows each submodule to its direct parent, and brings in every submodule + descending from a used module, which implements its separate module + procedures. Import, re-export, callback, generic, specification-expression, and constant resolution follow the same rule, so a user module named like an intrinsic one, such as `iso_fortran_env`, is read when a `use` selects it and never when `use, intrinsic` selects the processor module. A derived type reached from a processor module, such as `ieee_arithmetic`'s, is left to the processor rather than read from a parsed module of that name. +- A separate module procedure declared by a `module function` or + `module subroutine` interface body is wrapped as its module's own + procedure, under ordinary accessibility; it previously needed an explicit + `public` statement naming it. +- Fortran submodules are identified by `ancestor:name` throughout parsing, + project ordering, compile scheduling, and kind resolution, so two modules + may each have a submodule of the same name. `FortranProject.submodules` and + its dependency keys use that identity, and an entity a submodule declares + records it as its owner. +- Compile ordering follows `use` natures: a scope using the processor's + module through `use, intrinsic` no longer waits on a project source of the + same name, and `use` statements in internal procedures and `BLOCK` + constructs order compilation too. +- Generated module docstrings describe mutable module scalars as live + rank-zero views and include `None` for allocatable and pointer scalars. - A module that reaches two generics of one name through separate `use` statements, without declaring the generic itself, owns the merged generic: it dispatches over every contributor's specifics in source and contract @@ -63,7 +83,8 @@ release tags add a leading `v` to the package version. retain buffers for nonblocking operations. - `--export-symbols` and `build_fortran_extension(export_symbols=...)` accept - module-qualified Fortran procedures and variables, including symbols + module-qualified public Fortran symbols -- procedures, generics, and module + variables -- including symbols re-exported by a public facade; the bridge calls each procedure through the module that declares it. Generated contracts retain required type declarations (including the component and parent types they declare) and diff --git a/docs/user/faq/index.md b/docs/user/faq/index.md index 6a6ade42f..506680a6e 100644 --- a/docs/user/faq/index.md +++ b/docs/user/faq/index.md @@ -133,7 +133,8 @@ controls what the contract publishes and `--export-symbols` is no longer used. Adding a name to `__all__` publishes a declaration the contract already reaches; it cannot conjure one the C sources never declared. -For Fortran, list module-qualified procedures instead: +For Fortran, list module-qualified public symbols (procedures, generics, and +module variables) instead: ```text solver_mod::solve diff --git a/docs/user/language-support/feature-matrix.md b/docs/user/language-support/feature-matrix.md index 312b62714..12bab1cbc 100644 --- a/docs/user/language-support/feature-matrix.md +++ b/docs/user/language-support/feature-matrix.md @@ -67,7 +67,7 @@ where they apply. | Derived-type scalar boundaries and methods | Supported | [Derived types](../guide/wrapping-derived-types.md) | [Derived boundary tests](../../../tests/fortran/derived_types/end_to_end/test_derived_boundaries.py), [method tests](../../../tests/fortran/derived_types/end_to_end/test_type_bound_methods.py) | Derived-type arrays and some polymorphic forms are not included. | | Default and keyword constructors with finalizers | Supported | [Constructors and finalizers](../guide/wrapping-derived-types.md#key-concepts) | [Constructor/finalizer tests](../../../tests/fortran/derived_types/end_to_end/test_default_constructors_and_finalizers.py), [borrowed finalizer tests](../../../tests/fortran/derived_types/end_to_end/test_borrowed_components.py) | Construction commits ownership only after initialization; borrowed wrappers never run an owning finalizer. | | Generic constructor interfaces and overloaded runtime initialization | Supported | [Constructors](../guide/wrapping-derived-types.md#custom-constructor) | [Edited class surface tests](../../../tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/end_to_end/test_edited_class_surfaces.py), [class policy tests](../../../tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/policy/test_class_surface_policy.py) | Candidates require distinguishable completed Python signatures; incomplete or ambiguous sets are blocked before emission. | -| Module variables, constants, saved state, and common-block procedure state | Supported | [Wrapping modules](../guide/wrapping-modules.md) | [Module state tests](../../../tests/fortran/modules/end_to_end/test_module_variables_and_state.py), [scalar-derived matrix tests](../../../tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py), [common-block tests](../../../tests/fortran/modules/end_to_end/test_common_blocks.py) | Common-block storage is not exported as Python variables. Rank-zero derived module objects use direct, scoped, allocation-transaction, or pointer-transaction handoff selected before lowering. `character` module state is supported in every form: a declared-length scalar reads and writes as `str` at exactly its declared byte width, an `allocatable` or `pointer` scalar reads as a detached `str` or `None`, and arrays reach Python as fixed-width bytes. Only declared-length non-descriptor scalars are writable by assignment; descriptor scalars are read-only snapshots for numeric and `character` state alike, and arrays are mutated in place through their view or handle rather than rebound. | +| Module variables, constants, saved state, and common-block procedure state | Supported | [Wrapping modules](../guide/wrapping-modules.md) | [Module state tests](../../../tests/fortran/modules/end_to_end/test_module_variables_and_state.py), [scalar-derived matrix tests](../../../tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py), [common-block tests](../../../tests/fortran/modules/end_to_end/test_common_blocks.py) | Common-block storage is not exported as Python variables. Rank-zero derived module objects use direct, scoped, allocation-transaction, or pointer-transaction handoff selected before lowering. Mutable fixed-storage scalars are live rank-zero NumPy views, written through the view or by assigning the attribute; a declared-length `character` scalar is a fixed-width bytes view (`String[n][()]`) that assignment fills at exactly its declared width. `allocatable` and `pointer` scalars, numeric and `character` alike, read as live read-only rank-zero views or `None`, and assigning the attribute allocates an allocatable (resizing a deferred-length `character`) or writes a pointer's current target. `character` arrays reach Python as fixed-width bytes, and arrays are mutated in place through their view or handle rather than rebound. | | Fortran enum constants | Supported | [Enumerations](../guide/enumerations.md) | [Enum runtime tests](../../../tests/fortran/enumerations/end_to_end/test_enum_runtime.py), [enum semantic tests](../../../tests/fortran/enumerations/semantics/test_enum_semantics.py), [enum diagnostics](../../../tests/fortran/enumerations/parsing/test_enum_diagnostics.py) | No Python `Enum` or `IntEnum` classes are generated. | | Scalar character arguments, results, and fields | Supported | [Strings](../guide/strings.md) | [Character argument tests](../../../tests/fortran/strings/end_to_end/test_character_boundaries.py), [edge-case tests](../../../tests/fortran/strings/end_to_end/test_character_edge_cases.py) | Character arrays use fixed-width NumPy bytes dtype. Scalar `character` `allocatable` and `pointer` values are supported for `intent(in)`, `intent(out)`, `intent(inout)`, and function results, at deferred (`len=:`) and declared (`len=n`) length; a mutable dummy returns the value the procedure left behind, or `None`. PRIK frees the target it allocated for the call while it can still prove that identity, but never a target the procedure reassociated or the library owns; a procedure that returns a fresh allocation each call leaks unless it frees its own. | | Character arrays and caller-supplied deferred-length character storage | Supported | [Strings](../guide/strings.md) | [Character edge tests](../../../tests/fortran/strings/end_to_end/test_character_edge_cases.py) | Character arrays use fixed-width NumPy bytes dtype, whose width each accessor reports from the Fortran declaration; Unicode/object arrays are unsupported. Scalar `character` `allocatable` and `pointer` values work for every intent and as function results. A mutable `pointer` dummy that the native procedure reassociates without deallocating orphans the target the adapter allocated for that call. | diff --git a/docs/user/reference/cli-commands.md b/docs/user/reference/cli-commands.md index a64255b9c..8188b7097 100644 --- a/docs/user/reference/cli-commands.md +++ b/docs/user/reference/cli-commands.md @@ -406,19 +406,22 @@ vendor_open vendor_close ``` -Fortran module procedures and module variables use a case-insensitive, -module-qualified identity: +Module-qualified public Fortran symbols -- procedures, generics, and module +variables -- use a case-insensitive, module-qualified identity: ```text bobyqa_mod::bobyqa cobyla_mod::cobyla state_mod::counter +facade_mod::convert ``` Qualification keeps symbols with the same spelling in different modules distinct. The module side must name a declared Fortran `module`, not a file-level external-procedure group. Every listed identity must resolve to -one public procedure or variable. Empty files, invalid or repeated identities, +one public procedure, generic, or variable. A generic a module merges from +several imported generics of one name is that module's own, so selecting it +selects every specific it dispatches over. Empty files, invalid or repeated identities, and unknown or private declarations fail the command. Fortran extraction retains declarations needed to express selected signatures diff --git a/docs/user/reference/python-api.md b/docs/user/reference/python-api.md index 65be18fef..a7988fa01 100644 --- a/docs/user/reference/python-api.md +++ b/docs/user/reference/python-api.md @@ -99,8 +99,8 @@ Unknown names fail the build rather than silently producing a smaller module. Once you author or generate a semantic `.pyi` contract, that contract's own `__all__` states the public surface and `export_symbols` no longer applies. -`build_fortran_extension` accepts the same option with module-qualified native -procedure and variable identities. PRIK retains signature and type +`build_fortran_extension` accepts the same option with module-qualified public +Fortran symbols: procedures, generics, and module variables. PRIK retains signature and type dependencies while publishing only the selected symbols. A derived type a selected signature names is published with the component and parent types it needs, so its values can be constructed from Python: diff --git a/prik/cli.py b/prik/cli.py index aed47ddee..6dd764b1e 100644 --- a/prik/cli.py +++ b/prik/cli.py @@ -636,6 +636,7 @@ def _parse_fortran_semantic_sources(context: _SemanticPipelineContext) -> _Parse source_paths, context.module_source_dirs, lambda path: _fortran_source_for_path(path, context.preprocessing)[0], + command_line_macros=context.preprocessing.defines_command_line_macros, ) return _ParsedSemanticSources( source_paths, @@ -2427,9 +2428,9 @@ def _add_semantic_interpretation_options( "--export-symbols", metavar="FILE", help=( - "Select exact reachable C functions or module-qualified Fortran procedures from a UTF-8 " - "name file as the source-side public surface; generate --pyi records the corresponding " - "Python names in __all__" + "Select exact reachable C functions, or module-qualified public Fortran symbols " + "(procedures, generics, and module variables), from a UTF-8 name file as the " + "source-side public surface; generate --pyi records the corresponding Python names in __all__" ), ) diff --git a/prik/codegen/docstrings.py b/prik/codegen/docstrings.py index 79c84d355..6cf5b15a2 100644 --- a/prik/codegen/docstrings.py +++ b/prik/codegen/docstrings.py @@ -73,6 +73,22 @@ _UNKNOWN_EXTENTS = frozenset({"", ":", "*", ".."}) +# A module getter that may find no storage reports ``None`` for it. +_NULLABLE_MODULE_GETTERS = frozenset( + {ModuleGetterAction.NULLABLE_SNAPSHOT, ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW} +) + +# A scalar view reads and writes the module's own storage rather than a copy. +_MODULE_SCALAR_VIEW_NOTES = { + ModuleGetterAction.NATIVE_SCALAR_VIEW: "Live view of the module's storage; writing through it updates the module.", + ModuleGetterAction.NATIVE_CHARACTER_VIEW: ( + "Live view of the module's fixed-width character bytes; writing through it updates the module." + ), + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW: ( + "Live read-only view of the current storage, or None when it holds none." + ), +} + class WrapperDocstringBuilder: """Build compact, public NumPy-style documentation from completed plans. @@ -489,8 +505,24 @@ def module_variable(self, variable: ModuleVariablePlan) -> str: comes directly from the completed variable plan. """ name = variable.owner_path.rsplit(".", 1)[-1] - nullable = variable.binding.getter_action is ModuleGetterAction.NULLABLE_SNAPSHOT - lines = [f"{name} : {self._type(variable, nullable=nullable, signature=False)}"] + action = variable.binding.getter_action + nullable = action in _NULLABLE_MODULE_GETTERS + view_note = _MODULE_SCALAR_VIEW_NOTES.get(action) + if view_note is not None: + # A scalar view is a rank-zero array over the module's storage. + type_name = variable.semantic_type_name + element = ( + "bytes" if type_name == "String" else _ARRAY_ELEMENT_TYPES.get(type_name, self._base_type(variable)) + ) + lines = [ + f"{name} : ndarray[{element}]" + (" or None" if nullable else ""), + " Rank: 0", + f" {view_note}", + ] + if type_name in _ARRAY_ELEMENT_TYPES: + lines.append(f" {_LOGICAL_ARRAY_NOTE}") + else: + lines = [f"{name} : {self._type(variable, nullable=nullable, signature=False)}"] lines.extend(self._array_lines(variable.array)) lines.extend(self._logical_array_lines(variable)) if variable.binding.getter_action in { diff --git a/prik/codegen/fortran/bridge.py b/prik/codegen/fortran/bridge.py index cab3b33bd..b4c7d92ad 100644 --- a/prik/codegen/fortran/bridge.py +++ b/prik/codegen/fortran/bridge.py @@ -143,6 +143,10 @@ "Copies the parameter array into persistent storage and reports its width and extents." ), ModuleGetterAction.DIRECT_VALUE: "Returns the variable's current value.", + ModuleGetterAction.NATIVE_SCALAR_VIEW: "Returns the address of the scalar's storage, without copying.", + ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW: ( + "Returns the address of the scalar's current storage, or a null pointer when it holds nothing." + ), ModuleGetterAction.CHARACTER_VALUE: "Copies the characters into a fixed-width byte buffer.", ModuleGetterAction.NATIVE_CHARACTER_VIEW: "Returns the address of fixed character storage.", ModuleGetterAction.NULLABLE_SNAPSHOT: ( diff --git a/prik/parsers/fortran/models.py b/prik/parsers/fortran/models.py index 17d4f3cc0..6f8205fb4 100644 --- a/prik/parsers/fortran/models.py +++ b/prik/parsers/fortran/models.py @@ -380,6 +380,12 @@ class FortranProcedureSignature: in_interface: bool = False variables: dict[str, FortranVariable] = field(default_factory=dict) common_variables: list[str] = field(default_factory=list) + nested_uses: list[FortranUseStatement] = field(default_factory=list) + """``use`` statements of the internal procedures and ``BLOCK`` constructs inside. + + They name modules this procedure depends on, so discovery and compile + ordering read them, but they make nothing visible in the procedure itself. + """ @dataclass @@ -472,6 +478,14 @@ class FortranModule: @dataclass class FortranSubmodule: + """One submodule, named relative to the module it descends from. + + A submodule name is local to its ancestor module, so ``submodule (a) impl`` + and ``submodule (b) impl`` are two units. ``parent`` is the direct parent + as written: the ancestor module itself, or with ``ancestor`` set, another + submodule of that ancestor. + """ + name: str parent: str ancestor: str | None = None @@ -484,6 +498,21 @@ class FortranSubmodule: enums: list[FortranEnum] = field(default_factory=list) common_variables: list[str] = field(default_factory=list) + @property + def ancestor_module(self) -> str: + """Return the module this submodule descends from.""" + return self.ancestor or self.parent + + @property + def identity(self) -> str: + """Return ``ancestor:name``, the name that identifies this submodule.""" + return f"{self.ancestor_module}:{self.name}" + + @property + def parent_identity(self) -> str: + """Return the identity of the direct parent: a module, or ``ancestor:parent``.""" + return f"{self.ancestor}:{self.parent}" if self.ancestor else self.parent + @dataclass class FortranProgram: @@ -494,6 +523,11 @@ class FortranProgram: procedures: list[FortranProcedureSignature] = field(default_factory=list) enums: list[FortranEnum] = field(default_factory=list) common_variables: list[str] = field(default_factory=list) + nested_uses: list[FortranUseStatement] = field(default_factory=list) + """``use`` statements of the internal procedures and ``BLOCK`` constructs inside. + + They are dependencies of the program, but make nothing visible in it. + """ @dataclass diff --git a/prik/parsers/fortran/module_sources.py b/prik/parsers/fortran/module_sources.py index fbe15aef1..ccd96fa32 100644 --- a/prik/parsers/fortran/module_sources.py +++ b/prik/parsers/fortran/module_sources.py @@ -6,87 +6,83 @@ each submodule's parent to the one file that defines it, transitively, so a project can be supplied by its entry file alone. -Which modules a source defines and uses are parser facts, read here exactly as -compile ordering reads them. The directory index only locates candidate files -by their ``module`` lines, since parsing every file under a search root to -find one module would be needlessly slow. +What a source defines is decided by its preprocessed text, never by its raw +text: a macro or an ``#include`` can name a module, and a conditional block can +remove one. Every searched source is therefore located by the ``module`` and +``submodule`` statements its preprocessed text holds, and a source whose raw +text has nothing a preprocessor could change is read as it stands, which spares +the preprocessor for it. The sources located for a unit are then parsed, and +their parsed units are the definition. A source is selected into the project +only once it is the unit's one definition, so reading a candidate never makes +it part of the project. """ from __future__ import annotations from collections.abc import Callable, Iterable, Sequence +from concurrent.futures import ThreadPoolExecutor +from dataclasses import dataclass from pathlib import Path import re from prik.parsers.fortran.intrinsic_modules import INTRINSIC_FORTRAN_MODULES -from prik.parsers.fortran.models import FortranFile, FortranParseError +from prik.parsers.fortran.models import FortranParseError from prik.parsers.fortran.parser import FortranParser -from prik.parsers.fortran.scope import used_module_statements - +from prik.parsers.fortran.scope import file_defined_units, file_unit_requirements # Suffixes a Fortran compiler accepts as free- or fixed-form source. _FORTRAN_SOURCE_SUFFIXES = frozenset({".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08", ".fpp"}) -_SUBMODULE_LINE = re.compile( - r"^[ \t]*submodule[ \t]*\([ \t]*(?P[a-z][a-z0-9_]*)[ \t]*(?::[ \t]*[a-z][a-z0-9_]*[ \t]*)?\)" - r"[ \t]*(?P[a-z][a-z0-9_]*)", +# A statement may follow another on one line after ``;``, and a name may +# follow its keyword on a continuation line, so both are accepted. Locating a +# source that turns out not to define a unit costs a parse, never a result. +_STATEMENT_START = r"(?:^|;)[ \t]*" +_CONTINUATION = r"(?:[ \t]*&[ \t]*(?:!.*)?\n[ \t]*&?)?" +_MODULE_STATEMENT = re.compile( + _STATEMENT_START + r"module" + r"(?:[ \t]+|" + _CONTINUATION + r"[ \t]*)" + r"(?!(?:procedure|function|subroutine|pure|impure|elemental|recursive|non_recursive)\b)" + r"(?P[a-z][a-z0-9_]*)\b(?![ \t]*[(=%])", re.IGNORECASE | re.MULTILINE, ) -_MODULE_LINE = re.compile( - r"^[ \t]*module[ \t]+(?!(?:procedure|function|subroutine|pure|impure|elemental|recursive)\b)" - r"(?P[a-z][a-z0-9_]*)[ \t]*(?:!.*)?$", +_SUBMODULE_STATEMENT = re.compile( + _STATEMENT_START + r"submodule[ \t]*" + _CONTINUATION + r"\([ \t]*(?P[a-z][a-z0-9_]*)[ \t]*" + r"(?::[ \t]*[a-z][a-z0-9_]*[ \t]*)?\)[ \t]*" + _CONTINUATION + r"[ \t]*(?P[a-z][a-z0-9_]*)", re.IGNORECASE | re.MULTILINE, ) +# Raw text a preprocessor can change: a directive, or a Fortran ``include``, +# which PRIK's preprocessing expands as well. +_PREPROCESSED_TEXT = re.compile(r"^[ \t]*(?:#|include[ \t]*['\"])", re.IGNORECASE | re.MULTILINE) +# Sources are preprocessed by separate compiler processes, which is I/O-bound +# work for this process, so a few run at once. +_LOCATE_WORKERS = 4 def resolve_fortran_module_sources( entries: Sequence[Path], search_dirs: Iterable[Path], read_source: Callable[[Path], str], + *, + command_line_macros: bool = True, ) -> tuple[Path, ...]: """Return ``entries`` with the sources of every module they use, dependencies first. - ``read_source`` returns a file's preprocessed text, so a ``use`` inside an - inactive conditional block is not followed. An ``intrinsic`` module is - never searched. Any other used module, and a submodule's parent, that no - source read so far defines must be defined by exactly one Fortran source - under ``search_dirs``; otherwise a :class:`FortranParseError` names it and + ``read_source`` returns a file's preprocessed text. ``command_line_macros`` + states whether the preprocessing defines macros of its own, such as ``-D`` + flags; then every searched source is preprocessed, since any of its names + may be one, and otherwise a source without directives is read as written. + + Every submodule descending from a selected module is selected as well, + since it implements that module's separate module procedures. A unit + every scope uses as ``intrinsic`` is the processor's and is never + searched. A ``use`` stating no nature reads a module's source when one is + defined, and the processor's module of that name only when none is. Any + other needed module, and a submodule's parent, must be defined by exactly + one searched source; otherwise a :class:`FortranParseError` names it and the source that needs it. """ - candidates, searched_files = _unit_candidates(search_dirs) - parser = FortranParser() - facts: dict[Path, tuple[set[str], dict[str, str | None]]] = {} - owners: dict[str, Path] = {} - - def read(path: Path) -> tuple[set[str], dict[str, str | None]]: - if path not in facts: - parsed = parser.parse_file(read_source(path), filename=str(path)) - facts[path] = _defined_and_required_units(parsed) - for unit in facts[path][0]: - owners.setdefault(unit, path) - return facts[path] - - scanned: set[Path] = set() - - def parsed_definers(unit: str) -> list[Path]: - """Parse every searched source not yet read and return those defining ``unit``. - - A module named through a macro or an included line has no ``module`` - line the fast index can see, so the preprocessed sources are read - once, when a needed unit is otherwise missing. A source that cannot - be preprocessed or parsed cannot define it. - """ - for path in searched_files: - if path in scanned: - continue - scanned.add(path) - try: - read(path) - except Exception: # an unreadable candidate defines nothing - continue - return [path for path in searched_files if path in facts and unit in facts[path][0]] - + searched = _SearchedSources(tuple(search_dirs), read_source, command_line_macros=command_line_macros) + project = _SelectedSources(searched) for entry in entries: - read(entry.resolve()) + project.select(entry.resolve(), requested_by=None) ordered: list[Path] = [] visiting: set[Path] = set() @@ -94,11 +90,11 @@ def visit(path: Path) -> None: if path in ordered or path in visiting: return visiting.add(path) - defined, required = read(path) - for unit, nature in sorted(required.items()): - if unit in defined or nature == "intrinsic": + facts = searched.facts(path) + for unit, nature in sorted(facts.required.items(), key=lambda item: item[0]): + if unit in facts.defined or nature == "intrinsic": continue - dependency = owners.get(unit) or _defining_source(unit, nature, candidates, path, read, parsed_definers) + dependency = project.provider(unit, nature, user=path) if dependency is not None: visit(dependency) visiting.discard(path) @@ -106,43 +102,150 @@ def visit(path: Path) -> None: for entry in entries: visit(entry.resolve()) + # A separate module procedure is implemented in a submodule, which no + # ``use`` names, so every submodule descending from a selected module is + # part of the project too, however deeply nested. + added = True + while added: + added = False + for unit in searched.located_units(): + ancestor = unit.partition(":")[0] + if ":" not in unit or project.owner(unit) is not None or project.owner(ancestor) is None: + continue + if searched.definers(unit): + descendant = project.provider(unit, "non_intrinsic", user=project.owner(ancestor)) + if descendant is not None: + visit(descendant) + added = True originals = {entry.resolve(): entry for entry in entries} return tuple(originals.get(path, path) for path in ordered) -def _defining_source( - unit: str, - nature: str | None, - candidates: dict[str, list[Path]], - user: Path, - read: Callable[[Path], tuple[set[str], dict[str, str | None]]], - parsed_definers: Callable[[str], list[Path]], -) -> Path | None: - """Return the one searched source whose parsed units define ``unit``, or raise. - - A ``use`` that states no nature names an intrinsic module only when no - other module of that name is accessible, so a known intrinsic name is - satisfied by the processor when no source defines it. Only a unit that is - still missing reads every searched source in full. - """ - defining = [path for path in candidates.get(unit, ()) if unit in read(path)[0]] - if len(defining) == 1: - return defining[0] - if not defining and nature is None and unit in INTRINSIC_FORTRAN_MODULES: - return None - if not defining: - defining = parsed_definers(unit) - if len(defining) == 1: - return defining[0] - kind = "submodule" if ":" in unit else "module" - if not defining: +@dataclass(frozen=True) +class _UnitFacts: + """The units one parsed source defines, and those it requires with their natures.""" + + defined: frozenset[str] + required: dict[str, str | None] + + +class _SearchedSources: + """What every searched source defines, read once and never selected here.""" + + def __init__( + self, + search_dirs: tuple[Path, ...], + read_source: Callable[[Path], str], + *, + command_line_macros: bool, + ) -> None: + self._read_source = read_source + self._command_line_macros = command_line_macros + self.files = _searched_files(search_dirs) + self._located: dict[Path, frozenset[str]] | None = None + self._texts: dict[Path, str] = {} + self._facts: dict[Path, _UnitFacts] = {} + self.unreadable: dict[Path, Exception] = {} + self._parser = FortranParser() + + def facts(self, path: Path) -> _UnitFacts: + """Parse one source once and return its units.""" + if path not in self._facts: + text = self._texts.pop(path, None) + parsed = self._parser.parse_file(self._read_source(path) if text is None else text, filename=str(path)) + self._facts[path] = _UnitFacts(frozenset(file_defined_units(parsed)), file_unit_requirements(parsed)) + return self._facts[path] + + def definers(self, unit: str) -> list[Path]: + """Return every searched source whose parsed units define ``unit``, in search order.""" + located = self._locate() + return [path for path in self.files if unit in located.get(path, ()) and unit in self.facts(path).defined] + + def located_units(self) -> list[str]: + """Return every unit some searched source's text states, sorted.""" + return sorted({unit for units in self._locate().values() for unit in units}) + + def raw_definers(self, unit: str) -> list[Path]: + """Return the unreadable sources whose raw text shows ``unit``, to explain a missing one.""" + return [path for path in self.unreadable if unit in _statement_units(path.read_text(errors="replace"))] + + def _locate(self) -> dict[Path, frozenset[str]]: + """Return the units each searched source's preprocessed text states, computed once.""" + if self._located is None: + located: dict[Path, frozenset[str]] = {} + opaque: list[Path] = [] + for path in self.files: + raw = path.read_text(encoding="utf-8", errors="replace") + if self._command_line_macros or _PREPROCESSED_TEXT.search(raw): + opaque.append(path) + else: + located[path] = _statement_units(raw) + with ThreadPoolExecutor(max_workers=_LOCATE_WORKERS) as pool: + for path, text in zip(opaque, pool.map(self._preprocessed_text, opaque), strict=True): + if isinstance(text, Exception): + # A source that cannot be preprocessed here defines + # nothing; it is named if a needed unit stays missing. + self.unreadable[path] = text + continue + located[path] = _statement_units(text) + if located[path]: + self._texts[path] = text + self._located = located + return self._located + + def _preprocessed_text(self, path: Path) -> str | Exception: + try: + return self._read_source(path) + except Exception as error: # reported through ``unreadable`` + return error + + +class _SelectedSources: + """The sources chosen into the project, and the units each one provides.""" + + def __init__(self, searched: _SearchedSources) -> None: + self._searched = searched + self._owners: dict[str, Path] = {} + + def select(self, path: Path, *, requested_by: Path | None) -> None: + """Add one source to the project; a unit another selected source defines is ambiguous.""" + for unit in sorted(self._searched.facts(path).defined): + owner = self._owners.setdefault(unit, path) + if owner != path: + _raise_ambiguous(unit, requested_by or path, [owner, path]) + + def owner(self, unit: str) -> Path | None: + """Return the selected source defining ``unit``, if one is selected.""" + return self._owners.get(unit) + + def provider(self, unit: str, nature: str | None, *, user: Path) -> Path | None: + """Return the one source defining ``unit``, or ``None`` for the processor's module.""" + if unit in self._owners: + return self._owners[unit] + definers = self._searched.definers(unit) + if len(definers) > 1: + _raise_ambiguous(unit, user, definers) + if definers: + self.select(definers[0], requested_by=user) + return definers[0] + if nature is None and unit in INTRINSIC_FORTRAN_MODULES: + return None + kind = "submodule" if ":" in unit else "module" + unreadable = self._searched.raw_definers(unit) + detail = "".join( + f" {path} names it but could not be preprocessed: {self._searched.unreadable[path]}" for path in unreadable + ) raise FortranParseError( f"No Fortran source defines {kind} '{unit}' used by {user}; " - "add the directory that contains it as a module source directory.", + f"add the directory that contains it as a module source directory.{detail}", filename=str(user), code="PARSE_MODULE_SOURCE_NOT_FOUND", ) - listed = ", ".join(str(path) for path in defining) + + +def _raise_ambiguous(unit: str, user: Path, definers: Sequence[Path]) -> None: + kind = "submodule" if ":" in unit else "module" + listed = ", ".join(str(path) for path in definers) raise FortranParseError( f"{kind.capitalize()} '{unit}' used by {user} is defined by several sources ({listed}); " "narrow the module source directories to one of them.", @@ -151,55 +254,20 @@ def _defining_source( ) -def _unit_candidates(search_dirs: Iterable[Path]) -> tuple[dict[str, list[Path]], list[Path]]: - """Map each module or submodule to the sources under ``search_dirs`` that open it, and list every source. - - The map reads raw ``module`` and ``submodule`` lines, the fast path; the - list lets a unit those lines cannot show be found by parsing. - """ - candidates: dict[str, list[Path]] = {} - searched: list[Path] = [] +def _searched_files(search_dirs: Iterable[Path]) -> tuple[Path, ...]: + """Return every Fortran source under ``search_dirs`` once, in a stable order.""" + files: dict[Path, None] = {} for directory in search_dirs: for path in sorted(Path(directory).rglob("*")): - if path.suffix.casefold() not in _FORTRAN_SOURCE_SUFFIXES or not path.is_file(): - continue - resolved = path.resolve() - if resolved not in searched: - searched.append(resolved) - text = path.read_text(encoding="utf-8", errors="replace") - units = [match.group("name").casefold() for match in _MODULE_LINE.finditer(text)] - units.extend( - f"{match.group('ancestor')}:{match.group('name')}".casefold() - for match in _SUBMODULE_LINE.finditer(text) - ) - for unit in units: - paths = candidates.setdefault(unit, []) - if resolved not in paths: - paths.append(resolved) - return candidates, searched - - -def _defined_and_required_units(parsed: FortranFile) -> tuple[set[str], dict[str, str | None]]: - """Return the units one parsed source defines and the units it requires, with each ``use`` nature. - - A module is named by itself and a submodule by ``ancestor:name``. A - submodule requires its direct parent: the submodule it names after its - ancestor, or the ancestor module itself. - """ - defined = {str(module.name).casefold() for module in parsed.modules} - defined.update( - f"{submodule.ancestor or submodule.parent}:{submodule.name}".casefold() for submodule in parsed.submodules + if path.suffix.casefold() in _FORTRAN_SOURCE_SUFFIXES and path.is_file(): + files.setdefault(path.resolve(), None) + return tuple(files) + + +def _statement_units(text: str) -> frozenset[str]: + """Return the modules and ``ancestor:name`` submodules the statements of ``text`` open.""" + units = {match.group("name").casefold() for match in _MODULE_STATEMENT.finditer(text)} + units.update( + f"{match.group('ancestor')}:{match.group('name')}".casefold() for match in _SUBMODULE_STATEMENT.finditer(text) ) - required: dict[str, str | None] = {} - for owner in (*parsed.modules, *parsed.submodules, *parsed.programs, *parsed.procedures): - for statement in used_module_statements(owner): - name = statement.module.casefold() - # An explicit nature is kept over a statement that states none. - if required.get(name) is None: - required[name] = statement.nature - for submodule in parsed.submodules: - parent = ( - f"{submodule.ancestor}:{submodule.parent}" if submodule.ancestor else str(submodule.parent) - ).casefold() - required[parent] = "non_intrinsic" - return defined, required + return frozenset(units) diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index 4701a0d3c..b812e6e9f 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -28,7 +28,12 @@ from prik.utilities.visitor import ClassVisitor from prik.parsers.fortran.lexer import preprocess_lines -from prik.parsers.fortran.scope import ScopeUses, used_module_names +from prik.parsers.fortran.scope import ( + ScopeUses, + file_defined_units, + file_unit_requirements, + source_module_dependencies, +) from prik.parsers.fortran.models import ( FortranArgument, FortranBlockData, @@ -1752,7 +1757,8 @@ def _belongs_to_module_like(item, target, *, exclude_interface: bool = False) -> ``exclude_interface`` keeps interface procedure signatures attached to their interface instead of duplicating them in ``target.procedures``. """ - belongs = bool(item.module and item.module.lower() == target.name.lower()) + owner = target.identity if isinstance(target, FortranSubmodule) else target.name + belongs = bool(item.module and item.module.lower() == owner.lower()) return belongs and not (exclude_interface and item.in_interface) def _populate_module_like_children(self, target, child_units, *, scope, filename) -> None: @@ -1805,6 +1811,7 @@ def _visit_ProgramUnit( program.enums.extend( self._visit(child, parent_scope=scope, filename=filename) for child in child_units if child.kind == "enum" ) + program.nested_uses = self._nested_use_statements(unit) self._validate_variable_declarations( program.variables, owner_kind="program", @@ -1813,6 +1820,25 @@ def _visit_ProgramUnit( ) return program + @classmethod + def _nested_use_statements(cls, unit: SourceUnit) -> list[FortranUseStatement]: + """Return the ``use`` statements written inside one unit but outside its own specification. + + An internal procedure and a ``BLOCK`` construct each have a scope of + their own, so their imports are dependencies of the unit without being + visible in it. An interface body inside the unit is modeled with its + own ``use`` statements, so its lines are left to that model. + """ + own = {(line[1], line[0]) for line in unit.specification} + modeled = {(line[1], line[0]) for child in unit.children if child.kind == "interface" for line in child.lines} + return [ + statement + for line in unit.lines + if (line[1], line[0]) not in own + and (line[1], line[0]) not in modeled + and (statement := cls._parse_use_statement(line[0].strip())) is not None + ] + def _visit_BlockDataUnit( self, unit: BlockDataUnit, @@ -2001,6 +2027,7 @@ def _visit_ProcedureUnit( filename=filename, ) self._helper_apply_local_interface_declarations(proc_state, unit, scope, filename=filename) + proc_state.signature.nested_uses = self._nested_use_statements(unit) return self._finalize_proc(proc_state) # ------------------------------------------------------------------ @@ -2079,7 +2106,7 @@ def _helper_attach_file_interfaces( ] for submodule in units.submodules: submodule.interfaces = [ - iface for iface in interfaces if iface.module and iface.module.lower() == submodule.name.lower() + iface for iface in interfaces if iface.module and iface.module.lower() == submodule.identity.lower() ] return [iface for iface in interfaces if iface.module is None] @@ -2208,7 +2235,7 @@ def _helper_build_fortran_file( for model in [*units.modules, *units.submodules, *units.procedures]: self._insert_unique_scope_symbol( parsed_file.symbols, - model.name.lower(), + (model.identity if isinstance(model, FortranSubmodule) else model.name).lower(), model, label="file scope", filename=filename, @@ -2261,22 +2288,13 @@ def _parse_named_project_sources( @staticmethod def _project_file_requirements(parsed_file: FortranFile) -> set[str]: - """Return module or submodule names required by one parsed file. + """Return the modules and submodules one parsed file needs from project sources. - Requirements come from module and submodule ``use`` statements plus a - submodule's parent and optional ancestor. For example, a child - submodule with parent ``api`` and ``use kinds`` returns at least - ``{"api", "kinds"}``. + Every scope's ``use`` counts, with a submodule's direct parent, but an + ``intrinsic`` use names the processor's module and never a project + file that shares its name. A submodule is named ``ancestor:name``. """ - requirements: set[str] = set() - for module in parsed_file.modules: - requirements.update(used_module_names(module)) - for submodule in parsed_file.submodules: - requirements.update(used_module_names(submodule)) - requirements.add(submodule.parent.lower()) - if submodule.ancestor: - requirements.add(submodule.ancestor.lower()) - return requirements + return {name for name, nature in file_unit_requirements(parsed_file).items() if nature != "intrinsic"} def _order_project_files(self, parsed_files: list[FortranFile]) -> list[FortranFile]: """Return existing file models in dependency-first order. @@ -2294,8 +2312,7 @@ def _order_project_files(self, parsed_files: list[FortranFile]) -> list[FortranF if filename is None: raise ValueError("Dependency ordering requires every parsed project file to have a filename.") files_by_name[filename] = parsed_file - unit_to_file.update((module.name.lower(), filename) for module in parsed_file.modules) - unit_to_file.update((submodule.name.lower(), filename) for submodule in parsed_file.submodules) + unit_to_file.update((unit, filename) for unit in file_defined_units(parsed_file)) file_dependencies: dict[str, set[str]] = {} for filename, parsed_file in files_by_name.items(): @@ -2421,22 +2438,24 @@ def _helper_index_project_module(self, project: FortranProject, module: FortranM """Index one module and its owned public models.""" module_key = module.name.lower() self._insert_unique_scope_symbol(project.modules, module_key, module, label="project module scope") - project.dependencies[module_key] = used_module_names(module) + project.dependencies[module_key] = source_module_dependencies([module]) self._helper_index_project_owner_members(project, module, module_key) def _helper_index_project_submodule(self, project: FortranProject, submodule: FortranSubmodule) -> None: """Index one submodule, its dependencies, and its public models.""" - submodule_key = submodule.name.lower() + # A submodule name is local to its ancestor, so ``a:impl`` and + # ``b:impl`` are two submodules, and each depends on its direct parent. + submodule_key = submodule.identity.lower() self._insert_unique_scope_symbol( project.submodules, submodule_key, submodule, label="project submodule scope", ) - dependencies = {submodule.parent.lower(), *used_module_names(submodule)} - if submodule.ancestor: - dependencies.add(submodule.ancestor.lower()) - project.dependencies[submodule_key] = dependencies + project.dependencies[submodule_key] = { + submodule.parent_identity.lower(), + *source_module_dependencies([submodule]), + } self._helper_index_project_owner_members(project, submodule, submodule_key) def _helper_index_project_owner_members( @@ -2491,7 +2510,7 @@ def _helper_index_project_program(self, project: FortranProject, program: Fortra return program_key = program.name.lower() self._insert_unique_scope_symbol(project.programs, program_key, program, label="project program scope") - project.dependencies[program_key] = used_module_names(program) + project.dependencies[program_key] = source_module_dependencies([program]) def _helper_index_project_interface( self, @@ -2604,11 +2623,12 @@ def collect(scope: _ParserScope, child_units: Sequence[SourceUnit]) -> None: interfaces.append((child, scope)) continue if child.kind in {"module", "submodule"}: + owner = self._module_like_unit_owner(child) child_scope = _ParserScope( kind=child.kind, - name=child.name, + name=owner, parent=scope, - module_owner=child.name, + module_owner=owner, ) collect(child_scope, child.children) continue @@ -2640,11 +2660,12 @@ def collect(scope: _ParserScope, child_units: Sequence[SourceUnit]) -> None: types.append((child, scope)) continue if child.kind in {"module", "submodule", "program"}: + owner = self._module_like_unit_owner(child) if child.kind != "program" else child.name child_scope = _ParserScope( kind=child.kind, - name=child.name, + name=owner, parent=scope, - module_owner=child.name if child.kind in {"module", "submodule"} else scope.module_owner, + module_owner=owner if child.kind != "program" else scope.module_owner, ) collect(child_scope, child.children) continue @@ -3345,8 +3366,12 @@ def _helper_scope_for_model( """ name = getattr(model, "name", None) inherited_owner = module_owner if module_owner is not None else (parent.module_owner if parent else None) - if kind in {"module", "submodule"}: + if kind == "module": inherited_owner = name + elif kind == "submodule": + # A submodule name is local to its ancestor, so what it owns is + # identified by ``ancestor:name``. + inherited_owner = model.identity return _ParserScope( kind=kind, name=name, @@ -3356,6 +3381,13 @@ def _helper_scope_for_model( state=state, ) + def _module_like_unit_owner(self, unit: SourceUnit) -> str | None: + """Return the owner name of a module or submodule unit: a submodule's is ``ancestor:name``.""" + if unit.kind != "submodule": + return unit.name + submodule = self._parse_submodule_header(unit.header[0].strip(), None) + return submodule.identity if submodule is not None else unit.name + @staticmethod def _scope_key(name: str) -> str: """Normalize a case-insensitive Fortran scope key.""" @@ -5428,9 +5460,16 @@ def _module_parameter_expressions( expression = variable.symbolic_value if variable.symbolic_value is not None else variable.value if expression is not None: owner_expressions[variable.name.casefold()] = expression - expressions[owner.name.casefold()] = owner_expressions + expressions[FortranParser._module_like_key(owner)] = owner_expressions return expressions + @staticmethod + def _module_like_key(owner: object) -> str: + """Return the symbol-table key of a module-like owner: a submodule's is ``ancestor:name``.""" + if isinstance(owner, FortranSubmodule): + return owner.identity.casefold() + return str(getattr(owner, "name", "") or "").casefold() + @staticmethod def _resolve_compile_time_symbols( module_expressions: Mapping[str, Mapping[str, str]], @@ -5468,7 +5507,7 @@ def _build_compile_time_symbols( The returned table is immutable and already transitively resolved. """ owners: dict[str, FortranModule | FortranSubmodule] = { - owner.name.casefold(): owner for owner in (*modules, *submodules) + self._module_like_key(owner): owner for owner in (*modules, *submodules) } raw_expressions = self._module_parameter_expressions([*modules, *submodules]) initial = self._resolve_compile_time_symbols(raw_expressions) @@ -5480,9 +5519,11 @@ def _build_compile_time_symbols( active = _CompileTimeSymbols(resolved) owner_symbols = dict(active.in_module(owner_name)) if isinstance(owner, FortranSubmodule): + # Host association runs through the ancestor module and, + # for a nested submodule, its direct parent submodule. + owner_symbols.update(active.in_module(owner.ancestor_module)) if owner.ancestor: - owner_symbols.update(active.in_module(owner.ancestor)) - owner_symbols.update(active.in_module(owner.parent)) + owner_symbols.update(active.in_module(owner.parent_identity)) owner_symbols.update( self._imported_compile_time_symbols( owner.uses, @@ -5531,9 +5572,15 @@ def _imported_compile_time_symbols( if not include_intrinsic_aliases: return imported # An intrinsic module has no parsed symbols, so a name imported from - # one stands for its own target-dependent spelling. + # one stands for its own target-dependent spelling. A ``use`` stating + # no nature reads a parsed module of that name instead when one exists. for module in scope.modules(): - if module.casefold() not in _INTRINSIC_COMPILE_TIME_MODULES or scope.nature(module) == "non_intrinsic": + nature = scope.nature(module) + if ( + module.casefold() not in _INTRINSIC_COMPILE_TIME_MODULES + or nature == "non_intrinsic" + or (nature is None and module.casefold() in symbols.modules) + ): continue for mapping in scope.mappings(module): imported.setdefault(mapping.local_name.casefold(), mapping.source) @@ -5680,7 +5727,7 @@ def _resolve_module_like_compile_time_facts( the method returns nothing. """ visible = FortranParser._compile_time_symbols_for_scope( - getattr(owner, "name", None), + FortranParser._module_like_key(owner) or None, getattr(owner, "uses", {}), symbols, ) diff --git a/prik/parsers/fortran/scope.py b/prik/parsers/fortran/scope.py index 22b51b0d6..44fe05956 100644 --- a/prik/parsers/fortran/scope.py +++ b/prik/parsers/fortran/scope.py @@ -158,20 +158,79 @@ def _renamed_away(self, module: str) -> frozenset[str]: def used_module_statements(owner: object) -> list[FortranUseStatement]: """Return every ``use`` statement one scope writes, including nested ones. - A ``use`` written inside a contained procedure or an interface body is a - dependency of the scope holding it just as much as one written at its top, - so the whole tree is read. Compile ordering, project dependencies, and - module source discovery all ask this, and they have to get the same answer. + A ``use`` written inside a contained procedure, an interface body, an + internal procedure or a ``BLOCK`` construct is a dependency of the scope + holding it just as much as one written at its top, so the whole tree is + read. Compile ordering, project dependencies, and module source discovery + all ask this, and they have to get the same answer. """ - statements: list[FortranUseStatement] = list(getattr(owner, "uses", ())) + statements: list[FortranUseStatement] = [*getattr(owner, "uses", ()), *getattr(owner, "nested_uses", ())] for procedure in getattr(owner, "procedures", ()): statements.extend(getattr(procedure, "uses", ())) + statements.extend(getattr(procedure, "nested_uses", ())) for interface in getattr(owner, "interfaces", ()): for procedure in getattr(interface, "procedures", ()): statements.extend(getattr(procedure, "uses", ())) return statements -def used_module_names(owner: object) -> set[str]: - """Return every module one scope names, lowercased.""" - return {statement.module.lower() for statement in used_module_statements(owner)} +# How strongly a requirement asks for a module's source: ``non_intrinsic`` +# requires one, a ``use`` stating no nature prefers one over the processor's +# module of that name, and ``intrinsic`` names the processor's module only. +_NATURE_STRENGTH = {"intrinsic": 0, None: 1, "non_intrinsic": 2} + + +def used_module_natures(owners: Iterable[object]) -> dict[str, str | None]: + """Return each module the scopes use, with the nature that decides where it is read from. + + Scopes naming one module differently are read together: one needing the + module's source outweighs one that would accept the processor's module, + so ``non_intrinsic`` wins over no stated nature, which wins over + ``intrinsic``. Only a module every scope uses as ``intrinsic`` is the + processor's alone. Names are case-folded. + """ + natures: dict[str, str | None] = {} + for owner in owners: + for statement in used_module_statements(owner): + name = statement.module.casefold() + if name not in natures or _NATURE_STRENGTH[statement.nature] > _NATURE_STRENGTH[natures[name]]: + natures[name] = statement.nature + return natures + + +def source_module_dependencies(owners: Iterable[object]) -> set[str]: + """Return the modules the scopes use that a parsed source may provide. + + An ``intrinsic`` use names the processor's module, so a project source + that happens to share its name is not a dependency. + """ + return {name for name, nature in used_module_natures(owners).items() if nature != "intrinsic"} + + +def file_unit_requirements(parsed_file: object) -> dict[str, str | None]: + """Return every unit one parsed file needs, with the nature each is needed by. + + A module is named by itself and a submodule by ``ancestor:name``. Every + scope in the file is read, including file-level interface bodies, and a + submodule needs its direct parent's source. This is the one reading that + module source discovery, project file ordering, and compile scheduling + share. + """ + owners = ( + *getattr(parsed_file, "modules", ()), + *getattr(parsed_file, "submodules", ()), + *getattr(parsed_file, "programs", ()), + *getattr(parsed_file, "procedures", ()), + *getattr(parsed_file, "interfaces", ()), + ) + requirements = used_module_natures(owners) + for submodule in getattr(parsed_file, "submodules", ()): + requirements[submodule.parent_identity.casefold()] = "non_intrinsic" + return requirements + + +def file_defined_units(parsed_file: object) -> set[str]: + """Return the units one parsed file defines: modules by name, submodules by ``ancestor:name``.""" + defined = {str(module.name).casefold() for module in getattr(parsed_file, "modules", ())} + defined.update(submodule.identity.casefold() for submodule in getattr(parsed_file, "submodules", ())) + return defined diff --git a/prik/pipeline/build.py b/prik/pipeline/build.py index a9d2da756..3502dbd60 100644 --- a/prik/pipeline/build.py +++ b/prik/pipeline/build.py @@ -40,7 +40,7 @@ from prik.parsers.c.cli import attach_preprocessing_recipe from prik.parsers.fortran.parser import parse_fortran_project from prik.parsers.fortran.module_sources import resolve_fortran_module_sources -from prik.parsers.fortran.scope import used_module_names +from prik.parsers.fortran.scope import file_defined_units, file_unit_requirements from prik.preprocessing.probes.fortran_types import ( evaluate_fortran_type_facts, evaluate_fortran_type_requirements, @@ -1640,31 +1640,6 @@ def _serial_compile_batches(object_files: Iterable[ObjectFile]) -> tuple[tuple[O return tuple((object_file,) for object_file in object_files) -def _fortran_file_used_modules(parsed_file: object) -> set[str]: - """Return lowercased module dependencies declared by one parsed file. - - Scans top-level parsed owners, interfaces, and submodule parent/ancestor - relationships. The returned names let the scheduler order object files; - the parsed file remains unmodified. - """ - owners = ( - *getattr(parsed_file, "modules", ()), - *getattr(parsed_file, "submodules", ()), - *getattr(parsed_file, "programs", ()), - *getattr(parsed_file, "procedures", ()), - ) - used = set() - for owner in owners: - used.update(used_module_names(owner)) - for interface in getattr(parsed_file, "interfaces", ()): - used.update(used_module_names(interface)) - for submodule in getattr(parsed_file, "submodules", ()): - used.add(str(submodule.parent).lower()) - if submodule.ancestor: - used.add(str(submodule.ancestor).lower()) - return used - - def _dependency_compile_batches( object_files: tuple[ObjectFile, ...], dependencies: dict[Path, set[Path]], @@ -1706,20 +1681,21 @@ def _project_compile_batches( if set(parsed_by_source) != object_sources: return _serial_compile_batches(object_files) - # Map providers before resolving each file's module dependencies. + # Map providers before resolving each file's module dependencies. A + # submodule is provided as ``ancestor:name``, and an ``intrinsic`` use + # never waits on a project source sharing the processor module's name. module_sources: dict[str, Path] = {} for source, parsed_file in parsed_by_source.items(): - for module in getattr(parsed_file, "modules", ()): - module_sources[str(module.name).lower()] = source - for submodule in getattr(parsed_file, "submodules", ()): - module_sources[str(submodule.name).lower()] = source + module_sources.update(dict.fromkeys(file_defined_units(parsed_file), source)) dependencies: dict[Path, set[Path]] = {} for source, parsed_file in parsed_by_source.items(): dependencies[source] = { dependency_source - for name in _fortran_file_used_modules(parsed_file) - if (dependency_source := module_sources.get(name)) is not None and dependency_source != source + for name, nature in file_unit_requirements(parsed_file).items() + if nature != "intrinsic" + and (dependency_source := module_sources.get(name)) is not None + and dependency_source != source } return _dependency_compile_batches(object_files, dependencies) @@ -3732,8 +3708,9 @@ def build_fortran_extension( always honored, and arrays, derived-type objects, and allocatable or pointer scalars are unaffected. export_symbols - Exact case-insensitive ``module::procedure`` identities to publish - from the source universe. Signature dependencies remain available but + Exact case-insensitive ``module::name`` identities of module-qualified + public Fortran symbols -- procedures, generics, and module variables -- + to publish from the source universe. Signature dependencies remain available but are not added to the callable surface. A generated semantic contract records the corresponding Python surface in ``__all__``. module_source_dirs @@ -3813,6 +3790,7 @@ def build_fortran_extension( source_paths, tuple(Path(directory) for directory in module_source_dirs), lambda path: _fortran_source_and_dependencies(path, preprocessing)[0], + command_line_macros=preprocessing.defines_command_line_macros, ) supplemental_source_paths = tuple(Path(path) for path in (native_fortran_sources or ())) input_implementation_paths = source_paths if compile_input_sources else () diff --git a/prik/preprocessing/source.py b/prik/preprocessing/source.py index e0525ec84..85bcbe83f 100644 --- a/prik/preprocessing/source.py +++ b/prik/preprocessing/source.py @@ -375,6 +375,23 @@ def uses_compiler(self) -> bool: """Whether this configuration authorizes compiler-backed preprocessing.""" return self.mode == "compiler" + @property + def defines_command_line_macros(self) -> bool: + """Whether compiler preprocessing may define macros that no source states. + + ``-D`` flags, a compile database, or a command template can each + define a macro, and then any name in any source may expand, so a + source without directives is not known to read as it is written. + """ + if not self.uses_compiler: + return False + return bool( + self.defines + or self.compile_commands + or self.command_template + or any(str(arg).startswith(("-D", "-include", "-imacros")) for arg in self.compiler_args) + ) + def fortran_internal_recipe(self, path: Path) -> dict[str, object] | None: """Return parser-test macro metadata when compiler invocation is absent. diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index b4420f4a7..f5a96c0a1 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -2442,11 +2442,15 @@ def _known_procedures_from_project(project: FortranProject) -> set[tuple[str, st def _module_explicit_interface_procedures( module: FortranModule, ) -> list[FortranProcedureSignature]: - """Return explicitly public procedures declared by unnamed interfaces. - - An explicit public list makes the module declaration the authoritative - wrapper contract. Other unnamed interface declarations remain - interface-only facts even when a matching implementation is parsed. + """Return the module procedures unnamed interface blocks declare. + + A ``module function`` or ``module subroutine`` body declares a separate + module procedure: it is this module's own procedure, implemented in a + submodule, so it is one exactly as a contained procedure is and follows + the same accessibility. Any other body describes an external procedure, + which becomes part of the wrapper contract only when an explicit public + list names it; otherwise it stays an interface-only fact even when a + matching implementation is parsed. """ public_names = {name.casefold() for name in module.public_symbols} declared_names = {procedure.name.casefold() for procedure in module.procedures} @@ -2458,7 +2462,7 @@ def _module_explicit_interface_procedures( name = procedure.name.casefold() if name in declared_names: continue - if name not in public_names: + if name not in public_names and "module" not in procedure.attributes: continue declared_names.add(name) procedures.append(procedure) @@ -4469,8 +4473,12 @@ def _module_variable_contexts( *, unit_kind: str, ): - """Return variable, procedure, and type contexts owned by a module-like node.""" - owner = node.name + """Return variable, procedure, and type contexts owned by a module-like node. + + A submodule is labelled ``ancestor:name``, since its name alone is local + to its ancestor and two submodules may share it. + """ + owner = node.identity if isinstance(node, FortranSubmodule) else node.name contexts = [ _variable_context(variable, unit_kind=unit_kind, unit=owner, module=owner, role="variable") for variable in node.variables diff --git a/prik/semantics/fortran_exports.py b/prik/semantics/fortran_exports.py index 5d4349b4c..ed9788c3d 100644 --- a/prik/semantics/fortran_exports.py +++ b/prik/semantics/fortran_exports.py @@ -45,10 +45,10 @@ def select_fortran_export_symbols( modules: Iterable[SemanticModule], symbols: Iterable[str], ) -> FortranExportSelection: - """Select exact module procedures and variables with semantic source context. + """Select exact module procedures, generics, and variables with semantic source context. Selection is expressed in native identities before policy names anything. - Primary module copies contain only requested procedures and variables; + Primary module copies contain only requested procedures, generics, and variables; classes and prototypes remain available as signature facts. Other source modules remain available as context. Contract-import policy decides which of them the generated contract needs to emit. diff --git a/tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py b/tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py index b8e62d38f..3659eea92 100644 --- a/tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py +++ b/tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py @@ -483,3 +483,41 @@ def test_reexport_is_owned_by_its_declaring_contract_whatever_the_entry_lists_fi assert generated.count("static PyObject * wrap_scale_twice") == 1 assert 'prik_bind_namespace_alias(namespace_facade_mod, "scale_twice", namespace_owner_mod' in generated assert 'prik_bind_namespace_alias(namespace_renaming_mod, "doubled", namespace_owner_mod' in generated + + +def test_same_named_submodules_of_two_modules_build_and_dispatch_separately(tmp_path: Path): + """``a:impl`` and ``b:impl`` are two submodules, each compiled after its own parent. + + After the first module, the sources arrive children first, so each nested + ``leaf`` compiles only because scheduling follows its ``ancestor:impl`` + parent rather than a bare ``impl`` that two files define. + """ + + def module(name: str) -> tuple[str, str]: + return ( + f"{name}.f90", + f"module {name}\n implicit none\n interface\n module function answer() result(value)\n" + f" integer :: value\n end function answer\n end interface\nend module {name}\n", + ) + + sources = [module("a")] + for ancestor, value in (("a", 1), ("b", 2)): + sources.append( + ( + f"{ancestor}_leaf.f90", + f"submodule ({ancestor}:impl) leaf\ncontains\n module procedure answer\n" + f" value = offset + {value}\n end procedure answer\nend submodule leaf\n", + ) + ) + sources.append( + ( + f"{ancestor}_impl.f90", + f"submodule ({ancestor}) impl\n integer, parameter :: offset = {10 * value}\nend submodule impl\n", + ) + ) + sources.append(module("b")) + + package, _payload = _build_sources_and_import(sources, tmp_path) + + assert package.a.answer() == 11 + assert package.b.answer() == 22 diff --git a/tests/fortran/infrastructure/building/pipeline/test_parallel_compilation.py b/tests/fortran/infrastructure/building/pipeline/test_parallel_compilation.py index 9312d74bd..bf1e85622 100644 --- a/tests/fortran/infrastructure/building/pipeline/test_parallel_compilation.py +++ b/tests/fortran/infrastructure/building/pipeline/test_parallel_compilation.py @@ -62,6 +62,38 @@ def test_project_compile_batches_respect_module_dependencies_and_group_ready_sou ] +def test_project_compile_batches_follow_use_natures_and_submodule_identities(tmp_path: Path) -> None: + """An intrinsic use waits on no project source, and a submodule waits on its own parent. + + ``a:impl`` and ``b:impl`` share a name; each nested ``leaf`` compiles after + its own ``impl``, and a scope using the processor's ``ieee_arithmetic`` + does not wait on a user source of that name. + """ + paths = { + name: tmp_path / f"{name}.f90" + for name in ("ieee", "processor", "a", "b", "a_impl", "b_impl", "a_leaf", "b_leaf") + } + sources = { + str(paths["ieee"]): "module ieee_arithmetic\nend module ieee_arithmetic\n", + str(paths["processor"]): "module processor\nuse, intrinsic :: ieee_arithmetic\nend module processor\n", + str(paths["a"]): "module a\nend module a\n", + str(paths["b"]): "module b\nend module b\n", + str(paths["a_impl"]): "submodule (a) impl\nend submodule impl\n", + str(paths["b_impl"]): "submodule (b) impl\nend submodule impl\n", + str(paths["a_leaf"]): "submodule (a:impl) leaf\nend submodule leaf\n", + str(paths["b_leaf"]): "submodule (b:impl) leaf\nend submodule leaf\n", + } + project = parse_fortran_project(sources) + objects = tuple(_object(Path(source), tmp_path / "build") for source in sources) + + batches = _project_compile_batches(project, objects) + + batch_of = {item.source: index for index, batch in enumerate(batches) for item in batch} + assert batch_of[paths["processor"]] == batch_of[paths["ieee"]] == 0 + for ancestor in ("a", "b"): + assert batch_of[paths[ancestor]] < batch_of[paths[f"{ancestor}_impl"]] < batch_of[paths[f"{ancestor}_leaf"]] + + def test_project_compile_batches_fall_back_to_input_order_for_unparsed_native_sources(tmp_path: Path) -> None: wrapped = tmp_path / "wrapped.f90" supplemental = tmp_path / "supplemental.f90" diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/assumed_shape_and_derived_args.json b/tests/fortran/infrastructure/parsing/fixtures/general/assumed_shape_and_derived_args.json index e099a08b1..54232eb2c 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/assumed_shape_and_derived_args.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/assumed_shape_and_derived_args.json @@ -50,7 +50,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "update_plane", @@ -94,7 +95,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "step", @@ -129,7 +131,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "interfaces": [], @@ -180,7 +183,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, "update_plane": { "name": "update_plane", @@ -224,7 +228,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, "step": { "name": "step", @@ -259,7 +264,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } } } diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/basic_subroutine.json b/tests/fortran/infrastructure/parsing/fixtures/general/basic_subroutine.json index da3d89d3e..ffc4b1fc9 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/basic_subroutine.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/basic_subroutine.json @@ -70,7 +70,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [], @@ -158,7 +159,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [], diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_all_exprs.json b/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_all_exprs.json index ad78fded7..62bbe02f1 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_all_exprs.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_all_exprs.json @@ -455,7 +455,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [], @@ -928,7 +929,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [], diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_shape_exprs.json b/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_shape_exprs.json index affa73cba..90ef0c153 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_shape_exprs.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_shape_exprs.json @@ -119,7 +119,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [], @@ -256,7 +257,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [], diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/derived_type.json b/tests/fortran/infrastructure/parsing/fixtures/general/derived_type.json index 086f2efd9..24262a6e3 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/derived_type.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/derived_type.json @@ -43,7 +43,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [ @@ -181,7 +182,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [ diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/f77_subroutine.json b/tests/fortran/infrastructure/parsing/fixtures/general/f77_subroutine.json index b9299654e..98aeca83b 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/f77_subroutine.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/f77_subroutine.json @@ -116,7 +116,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "interfaces": [], @@ -233,7 +234,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } } } diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/modern_pyi_example.json b/tests/fortran/infrastructure/parsing/fixtures/general/modern_pyi_example.json index f6c902f0b..5e6ca98e4 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/modern_pyi_example.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/modern_pyi_example.json @@ -191,7 +191,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "kinetic_energy", @@ -309,7 +310,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "scale_vector", @@ -371,7 +373,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "dot3", @@ -459,7 +462,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "fill_identity3", @@ -503,7 +507,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "normalize_particle", @@ -538,7 +543,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "hidden_proc", @@ -573,7 +579,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [ @@ -952,7 +959,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "kinetic_energy", @@ -1070,7 +1078,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "scale_vector", @@ -1132,7 +1141,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "dot3", @@ -1220,7 +1230,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "fill_identity3", @@ -1264,7 +1275,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "normalize_particle", @@ -1299,7 +1311,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "hidden_proc", @@ -1334,7 +1347,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [ diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/procedures_and_functions.json b/tests/fortran/infrastructure/parsing/fixtures/general/procedures_and_functions.json index 48cec8b72..498e3c573 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/procedures_and_functions.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/procedures_and_functions.json @@ -69,7 +69,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "scale", @@ -131,7 +132,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [], @@ -218,7 +220,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "scale", @@ -280,7 +283,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [], diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/scope_name_reuse_combinations.json b/tests/fortran/infrastructure/parsing/fixtures/general/scope_name_reuse_combinations.json index 34be05db6..041f58329 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/scope_name_reuse_combinations.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/scope_name_reuse_combinations.json @@ -149,7 +149,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "do_work_r", @@ -184,7 +185,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "do_work_l", @@ -219,7 +221,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "host_one", @@ -254,7 +257,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "host_two", @@ -289,7 +293,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "convert_to_complex", @@ -344,7 +349,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "convert_to_char", @@ -399,7 +405,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "convert_to_logical", @@ -454,7 +461,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [ @@ -673,7 +681,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "do_work_r", @@ -708,7 +717,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "do_work_l", @@ -743,7 +753,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "host_one", @@ -778,7 +789,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "host_two", @@ -813,7 +825,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "convert_to_complex", @@ -868,7 +881,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "convert_to_char", @@ -923,7 +937,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] }, { "name": "convert_to_logical", @@ -978,7 +993,8 @@ "uses": [], "in_interface": false, "variables": {}, - "common_variables": [] + "common_variables": [], + "nested_uses": [] } ], "derived_types": [ diff --git a/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py b/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py index 227ccd788..b19e37626 100644 --- a/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py +++ b/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py @@ -309,13 +309,13 @@ def test_directory_project_parses_once_and_assembles_dependency_ordered_models(t "parent.f90": "module Parent_Mod\n use Ancestor_Mod\n type :: Parent_State\n end type Parent_State\nend module Parent_Mod\n", "helper.f90": "module Helper_Mod\nend module Helper_Mod\n", "child.f90": ( - "submodule (Ancestor_Mod:Parent_Mod) Child_Mod\n" + "submodule (Parent_Mod) Child_Mod\n" " use Helper_Mod\n" " type :: Child_State\n" " end type Child_State\n" "end submodule Child_Mod\n" ), - "grandchild.f90": "submodule (Child_Mod) Grandchild_Mod\nend submodule Grandchild_Mod\n", + "grandchild.f90": "submodule (Parent_Mod:Child_Mod) Grandchild_Mod\nend submodule Grandchild_Mod\n", "units.f90": ( "type :: File_State\n" "end type File_State\n" @@ -386,9 +386,10 @@ def read_text(path, *args, **kwargs): "Child_Mod": child, "Grandchild_Mod": grandchild, } + # A submodule is keyed ``ancestor:name`` and depends on its direct parent. assert project.dependencies["parent_mod"] == {"ancestor_mod"} - assert project.dependencies["child_mod"] == {"ancestor_mod", "parent_mod", "helper_mod"} - assert project.dependencies["grandchild_mod"] == {"child_mod"} + assert project.dependencies["parent_mod:child_mod"] == {"parent_mod", "helper_mod"} + assert project.dependencies["parent_mod:grandchild_mod"] == {"parent_mod:child_mod"} def test_project_registries_preserve_qualified_aliases_values_and_dependencies(): @@ -441,37 +442,37 @@ def test_project_registries_preserve_qualified_aliases_values_and_dependencies() ) module = project.modules["api_mod"] - submodule = project.submodules["child_mod"] + submodule = project.submodules["api_mod:child_mod"] assert set(project.modules) == {"api_mod"} - assert set(project.submodules) == {"child_mod"} + assert set(project.submodules) == {"api_mod:child_mod"} assert set(project.programs) == {"driver"} assert project.dependencies == { "api_mod": set(), - "child_mod": {"api_mod"}, + "api_mod:child_mod": {"api_mod"}, "driver": {"api_mod"}, } - assert set(project.procedures) == {"api_mod.step", "step", "child_mod.reset", "reset"} + assert set(project.procedures) == {"api_mod.step", "step", "api_mod:child_mod.reset", "reset"} assert project.procedures["api_mod.step"] is project.procedures["step"] is module.procedures[0] - assert project.procedures["child_mod.reset"] is project.procedures["reset"] is submodule.procedures[0] + assert project.procedures["api_mod:child_mod.reset"] is project.procedures["reset"] is submodule.procedures[0] assert set(project.derived_types) == { "api_mod.state_t", "state_t", - "child_mod.child_state_t", + "api_mod:child_mod.child_state_t", "child_state_t", "global_state_t", } assert project.derived_types["api_mod.state_t"] is project.derived_types["state_t"] - assert project.derived_types["child_mod.child_state_t"] is project.derived_types["child_state_t"] + assert project.derived_types["api_mod:child_mod.child_state_t"] is project.derived_types["child_state_t"] assert set(project.interfaces) == { "api_mod.callback", "callback", - "child_mod.child_callback", + "api_mod:child_mod.child_callback", "child_callback", "global_callback", } assert project.interfaces["api_mod.callback"] is project.interfaces["callback"] - assert project.interfaces["child_mod.child_callback"] is project.interfaces["child_callback"] + assert project.interfaces["api_mod:child_mod.child_callback"] is project.interfaces["child_callback"] @pytest.mark.parametrize("standalone_first", [False, True]) diff --git a/tests/fortran/infrastructure/parsing/test_fortran_parser_procedures_and_interfaces.py b/tests/fortran/infrastructure/parsing/test_fortran_parser_procedures_and_interfaces.py index 8ac4428b4..403769005 100644 --- a/tests/fortran/infrastructure/parsing/test_fortran_parser_procedures_and_interfaces.py +++ b/tests/fortran/infrastructure/parsing/test_fortran_parser_procedures_and_interfaces.py @@ -416,12 +416,13 @@ def test_submodule_procedures_and_namespace_dependencies(tmp_path): namespace = parse_fortran_project({str(p.name): p.read_text(encoding="utf-8") for p in tmp_path.glob("*.f90")}) assert len(namespace.files) == 2 assert len(namespace.submodules) == 1 - submodule = namespace.submodules["child_impl"] + submodule = namespace.submodules["parent_mod:child_impl"] assert submodule.name == "child_impl" assert submodule.parent == "parent_mod" assert submodule.ancestor is None assert [p.name for p in submodule.procedures] == ["scale"] - assert submodule.procedures[0].module == "child_impl" + # A submodule name is local to its ancestor, so its procedures are owned by ``ancestor:name``. + assert submodule.procedures[0].module == "parent_mod:child_impl" def test_submodule_module_procedure_stub_and_additional_program_units(): diff --git a/tests/fortran/infrastructure/parsing/test_public_entrypoints.py b/tests/fortran/infrastructure/parsing/test_public_entrypoints.py index d485c3d75..1fc1fad8d 100644 --- a/tests/fortran/infrastructure/parsing/test_public_entrypoints.py +++ b/tests/fortran/infrastructure/parsing/test_public_entrypoints.py @@ -141,7 +141,8 @@ def test_public_instance_visitor_entrypoints_use_source_strings(): " type :: file_state\n" " end type file_state\n" "end submodule owner_submod\n", - "owner_submod", + # A submodule name is local to its ancestor, so it owns as ``ancestor:name``. + "parent_mod:owner_submod", ), ("program driver\n type :: file_state\n end type file_state\nend program driver\n", None), ( diff --git a/tests/fortran/infrastructure/semantic_ir/semantics/test_compile_time_values.py b/tests/fortran/infrastructure/semantic_ir/semantics/test_compile_time_values.py index 113ad4469..167b6824e 100644 --- a/tests/fortran/infrastructure/semantic_ir/semantics/test_compile_time_values.py +++ b/tests/fortran/infrastructure/semantic_ir/semantics/test_compile_time_values.py @@ -203,23 +203,23 @@ def test_semantic_compile_time_requirements_cover_all_parser_contexts(): }, "child_scale": { "unit_kind": "submodule", - "unit": "solver_child", - "module": "solver_child", + "unit": "solver_mod:solver_child", + "module": "solver_mod:solver_child", "symbol": "child_scale", "role": "variable", }, "y": { "unit_kind": "procedure", - "unit": "solver_child.child_step", - "module": "solver_child", + "unit": "solver_mod:solver_child.child_step", + "module": "solver_mod:solver_child", "procedure": "child_step", "symbol": "y", "role": "argument", }, "value": { "unit_kind": "derived_type", - "unit": "solver_child.child_t", - "module": "solver_child", + "unit": "solver_mod:solver_child.child_t", + "module": "solver_mod:solver_child", "type_owner": "child_t", "symbol": "value", "role": "field", diff --git a/tests/fortran/modules/end_to_end/test_module_source_discovery.py b/tests/fortran/modules/end_to_end/test_module_source_discovery.py index dfd17b23e..d02846d33 100644 --- a/tests/fortran/modules/end_to_end/test_module_source_discovery.py +++ b/tests/fortran/modules/end_to_end/test_module_source_discovery.py @@ -9,7 +9,7 @@ import numpy as np import pytest -from prik import build_fortran_extension +from prik import build_fortran_extension, build_pyi_extension from tests.fortran._support.wrapper_build import _import_from_build_dir pytestmark = pytest.mark.fortran_end_to_end @@ -50,3 +50,71 @@ def test_entry_source_discovers_used_modules_for_contract_and_build(tmp_path: Pa handle = root.base_types.Handle_T() root.api.touch(handle) assert handle.val == np.int32(1) + + +def test_a_separate_module_procedure_is_wrapped_and_its_submodule_discovered(tmp_path: Path): + """A ``module function`` declared in an interface block is the module's own procedure. + + Its body lives in a submodule that no ``use`` names; discovery still + brings that submodule in, so the contract publishes the function and both + the source build and a replay of the contract link its implementation. + """ + library = tmp_path / "lib" + library.mkdir() + (library / "shapes.f90").write_text( + """module shapes + implicit none + interface + module function area(side) result(value) + real(8), intent(in) :: side + real(8) :: value + end function area + end interface +end module shapes +""", + encoding="utf-8", + ) + (library / "shapes_impl.f90").write_text( + """submodule (shapes) shapes_impl +contains + module procedure area + value = side * side + end procedure area +end submodule shapes_impl +""", + encoding="utf-8", + ) + entry = tmp_path / "app.f90" + entry.write_text("module app\n use shapes, only: area\n implicit none\nend module app\n", encoding="utf-8") + contract = tmp_path / "contract" + subprocess.run( + [ + sys.executable, + "-m", + "prik", + "generate", + "--pyi", + str(entry), + "--module-source-dir", + str(library), + "--out", + str(contract), + ], + check=True, + capture_output=True, + text=True, + ) + assert "def area(" in (contract / "shapes.pyi").read_text(encoding="utf-8") + + source = build_fortran_extension( + entry, module_source_dirs=[library], output_name="separate_source", output_dir=tmp_path / "source" + ) + replay = build_pyi_extension( + contract / "__init__.pyi", + native_fortran_sources=(library / "shapes.f90", library / "shapes_impl.f90"), + output_name="separate_replay", + output_dir=tmp_path / "replay", + ) + for result in (source, replay): + root = _import_from_build_dir(result.module_name, result.output_dir) + assert root.shapes.area(np.float64(3.0)) == np.float64(9.0) diff --git a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py index 8f508b634..2896a2dc2 100644 --- a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py +++ b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py @@ -53,9 +53,10 @@ def test_scalar_module_variables_use_attributes_and_parameters_have_no_native_se assert module_docstring.index("Module Attributes") < module_docstring.index("Functions") assert module_docstring.index("Functions") < module_docstring.index("Classes") assert "nmax : int32\n Read-only constant." in module_docstring - assert "counter : int32" in module_docstring - assert "scale : float64" in module_docstring - assert "saved_counter : int32" in module_docstring + # A mutable module scalar is a live rank-zero view, and the docstring says so. + assert "counter : ndarray[int32]\n Rank: 0\n Live view" in module_docstring + assert "scale : ndarray[float64]\n Rank: 0\n Live view" in module_docstring + assert "saved_counter : ndarray[int32]" in module_docstring assert "Assignment writes through to native storage." not in module_docstring assert module.nmax == np.int32(12) @@ -360,6 +361,9 @@ def test_descriptor_character_module_variables_report_absence_as_none(tmp_path: assert module.deferred is None assert module.fixed is None assert module.link is None + # The documented type admits the ``None`` those reads return. + for name in ("deferred", "fixed", "link"): + assert f"{name} : ndarray[bytes] or None" in module.__doc__ def test_character_parameter_arrays_are_read_only_fixed_width_snapshots(tmp_path: Path): diff --git a/tests/fortran/modules/parsing/test_module_source_resolution.py b/tests/fortran/modules/parsing/test_module_source_resolution.py index 33a566e92..65871e335 100644 --- a/tests/fortran/modules/parsing/test_module_source_resolution.py +++ b/tests/fortran/modules/parsing/test_module_source_resolution.py @@ -109,22 +109,144 @@ def test_nested_submodule_resolves_its_direct_parent_before_the_ancestor_module( assert _resolve([leaf], [tmp_path / "src"]) == (base.resolve(), middle.resolve(), leaf) -@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran preprocessing") -def test_a_module_named_through_a_macro_is_found_by_parsing_the_searched_sources(tmp_path: Path): - """A ``module`` line the raw index cannot read is found once the preprocessed sources are parsed.""" +requires_gfortran = pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran preprocessing") + + +def _preprocessed_resolve(entries, search): config = PreprocessingConfig(mode="compiler", compiler="gfortran") - generated = _write( - tmp_path, - "lib/gen.F90", - "#define MODNAME generated_mod\nmodule MODNAME\n integer, parameter :: answer = 42\nend module MODNAME\n", + return resolve_fortran_module_sources( + entries, + search, + lambda path: preprocess_source(path, language="fortran", config=config).source, + command_line_macros=False, ) + + +MACRO_NAMED = "#define MODNAME {name}\nmodule MODNAME\n integer, parameter :: answer = 42\nend module MODNAME\n" + + +@requires_gfortran +def test_a_module_named_through_a_macro_is_found_by_its_preprocessed_text(tmp_path: Path): + """A module a macro names is located, and a source that cannot be preprocessed defines nothing.""" + generated = _write(tmp_path, "lib/gen.F90", MACRO_NAMED.format(name="generated_mod")) _write(tmp_path, "lib/broken.F90", '#include "missing_header.h"\nmodule broken\nend module broken\n') entry = _write(tmp_path, "app.f90", "module app\n use generated_mod, only: answer\nend module app\n") - resolved = resolve_fortran_module_sources( - [entry], - [tmp_path / "lib"], - lambda path: preprocess_source(path, language="fortran", config=config).source, + assert _preprocessed_resolve([entry], [tmp_path / "lib"]) == (generated.resolve(), entry) + + +@requires_gfortran +def test_a_macro_named_user_module_shadows_the_processor_module_it_is_named_after(tmp_path: Path): + """A ``use`` stating no nature falls back to the processor only when no source defines the module.""" + user_module = _write(tmp_path, "lib/ieee.F90", MACRO_NAMED.format(name="ieee_arithmetic")) + entry = _write(tmp_path, "app.f90", "module app\n use ieee_arithmetic, only: answer\nend module app\n") + + assert _preprocessed_resolve([entry], [tmp_path / "lib"]) == (user_module.resolve(), entry) + + +@requires_gfortran +def test_a_macro_named_second_definition_makes_a_module_ambiguous(tmp_path: Path): + """One definition visible in the raw text does not hide another that only preprocessing reveals.""" + _write(tmp_path, "lib/plain.f90", "module shared_mod\nend module shared_mod\n") + _write(tmp_path, "lib/generated.F90", MACRO_NAMED.format(name="shared_mod")) + entry = _write(tmp_path, "app.f90", "module app\n use shared_mod\nend module app\n") + + with pytest.raises(FortranParseError, match="shared_mod") as error: + _preprocessed_resolve([entry], [tmp_path / "lib"]) + + assert error.value.code == "PARSE_AMBIGUOUS_MODULE_SOURCE" + + +@requires_gfortran +def test_reading_every_source_for_one_module_selects_none_of_the_others(tmp_path: Path): + """Locating a macro-named module reads every source, but a later module is still checked for uniqueness.""" + _write(tmp_path, "lib/gen.F90", MACRO_NAMED.format(name="aa_generated")) + _write(tmp_path, "lib/one.f90", "module zz_dup\nend module zz_dup\n") + _write(tmp_path, "lib/two.f90", "module zz_dup\nend module zz_dup\n") + entry = _write(tmp_path, "app.f90", "module app\n use aa_generated\n use zz_dup\nend module app\n") + + with pytest.raises(FortranParseError, match="zz_dup") as error: + _preprocessed_resolve([entry], [tmp_path / "lib"]) + + assert error.value.code == "PARSE_AMBIGUOUS_MODULE_SOURCE" + + +@pytest.mark.parametrize("non_intrinsic_first", [False, True]) +def test_scopes_in_one_file_that_name_a_module_differently_are_each_honored(tmp_path: Path, non_intrinsic_first: bool): + """One scope's ``intrinsic`` use never cancels another scope's need for the module's source.""" + user_module = _write( + tmp_path, "lib/ieee.f90", "module ieee_arithmetic\n integer :: mine\nend module ieee_arithmetic\n" + ) + processor = "module a\n use, intrinsic :: ieee_arithmetic\nend module a\n" + source = "module b\n use, non_intrinsic :: ieee_arithmetic, only: mine\nend module b\n" + entry = _write(tmp_path, "app.f90", source + processor if non_intrinsic_first else processor + source) + + assert _resolve([entry], [tmp_path / "lib"]) == (user_module.resolve(), entry) + + +def test_uses_inside_internal_procedures_and_block_constructs_are_followed(tmp_path: Path): + """A nested scope's ``use`` is a dependency of the file even though its host cannot see it.""" + inner = _write(tmp_path, "lib/inner.f90", "module inner_mod\nend module inner_mod\n") + block = _write(tmp_path, "lib/block.f90", "module block_mod\nend module block_mod\n") + entry = _write( + tmp_path, + "app.f90", + """module app +contains + subroutine run() + block + use block_mod + end block + contains + subroutine helper() + use inner_mod + end subroutine helper + end subroutine run +end module app +""", + ) + + assert set(_resolve([entry], [tmp_path / "lib"])) == {inner.resolve(), block.resolve(), entry} + + +def test_same_named_submodules_of_different_ancestors_are_separate_units(tmp_path: Path): + """``a:impl`` and ``b:impl`` are two submodules, and each nested child finds its own parent.""" + units = {} + for ancestor in ("a", "b"): + units[ancestor] = _write(tmp_path, f"lib/{ancestor}.f90", f"module {ancestor}\nend module {ancestor}\n") + units[f"{ancestor}:impl"] = _write( + tmp_path, f"lib/{ancestor}_impl.f90", f"submodule ({ancestor}) impl\nend submodule impl\n" + ) + leaves = [ + _write(tmp_path, f"{ancestor}_leaf.f90", f"submodule ({ancestor}:impl) leaf\nend submodule leaf\n") + for ancestor in ("a", "b") + ] + + resolved = _resolve(leaves, [tmp_path / "lib"]) + + assert set(resolved) == {path.resolve() for path in units.values()} | set(leaves) + for ancestor, leaf in zip(("a", "b"), leaves, strict=True): + order = resolved.index + assert order(units[ancestor].resolve()) < order(units[f"{ancestor}:impl"].resolve()) < order(leaf) + + +def test_submodules_implementing_a_used_module_are_selected_with_it(tmp_path: Path): + """No ``use`` names a submodule, yet the module's separate procedures are implemented there.""" + api = _write( + tmp_path, + "lib/api.f90", + "module api\n interface\n module subroutine run()\n end subroutine run\n end interface\nend module api\n", ) + impl = _write(tmp_path, "lib/impl/api_impl.f90", "submodule (api) impl\nend submodule impl\n") + leaf = _write( + tmp_path, + "lib/impl/api_leaf.f90", + "submodule (api:impl) leaf\ncontains\n module subroutine run()\n end subroutine run\nend submodule leaf\n", + ) + _write(tmp_path, "lib/other.f90", "module other\nend module other\nsubmodule (other) impl\nend submodule impl\n") + entry = _write(tmp_path, "app.f90", "module app\n use api\nend module app\n") + + resolved = _resolve([entry], [tmp_path / "lib"]) - assert resolved == (generated.resolve(), entry) + assert set(resolved) == {api.resolve(), impl.resolve(), leaf.resolve(), entry} + assert resolved.index(api.resolve()) < resolved.index(impl.resolve()) < resolved.index(leaf.resolve()) diff --git a/tests/fortran/modules/parsing/test_project_scope_models.py b/tests/fortran/modules/parsing/test_project_scope_models.py index ccf9e156c..16136c8b8 100644 --- a/tests/fortran/modules/parsing/test_project_scope_models.py +++ b/tests/fortran/modules/parsing/test_project_scope_models.py @@ -1,5 +1,7 @@ """Project-level registries, dependencies, and scope model behavior.""" +from pathlib import Path + import pytest from prik.parsers.fortran import FortranParseError, parse_fortran_file, parse_fortran_project @@ -153,9 +155,10 @@ def test_submodule_types_interfaces_and_project_dependencies_attach_to_public_mo assert [iface.name for iface in submodule.interfaces] == ["callbacks"] assert [proc.name for proc in submodule.procedures] == ["reset"] + # A nested submodule depends on its direct parent, identified through its ancestor. project = parse_fortran_project({"child.f90": code}) - assert project.dependencies["child_mod"] == {"ancestor_mod", "parent_mod"} - assert "child_mod.reset" in project.procedures + assert project.dependencies["ancestor_mod:child_mod"] == {"ancestor_mod:parent_mod"} + assert "ancestor_mod:child_mod.reset" in project.procedures def test_project_registry_includes_module_types_interfaces_and_program_dependencies(): @@ -214,9 +217,8 @@ def test_project_directory_orders_ancestor_submodule_dependencies(tmp_path): ) (tmp_path / "parent.f90").write_text( """ -module parent_mod - use ancestor_mod -end module parent_mod +submodule (ancestor_mod) parent_mod +end submodule parent_mod """, encoding="utf-8", ) @@ -242,9 +244,10 @@ def test_project_directory_orders_ancestor_submodule_dependencies(tmp_path): project = parse_fortran_project(tmp_path) assert "ancestor_mod" in project.modules - assert "parent_mod" in project.modules - assert "child_mod" in project.submodules - assert project.dependencies["child_mod"] == {"ancestor_mod", "parent_mod", "helper_mod"} + assert {"ancestor_mod:parent_mod", "ancestor_mod:child_mod"} <= set(project.submodules) + assert project.dependencies["ancestor_mod:child_mod"] == {"ancestor_mod:parent_mod", "helper_mod"} + ordered = [Path(parsed.filename).name for parsed in project.files] + assert ordered.index("ancestor.f90") < ordered.index("parent.f90") < ordered.index("child.f90") def test_program_contains_and_unnamed_block_data_public_models(): @@ -517,7 +520,7 @@ def test_project_resolves_submodule_host_associated_kind(): } ) - procedure = project.submodules["transform_impl"].procedures[0] + procedure = project.submodules["transform_api:transform_impl"].procedures[0] assert procedure.arguments[0].kind == "real64" assert procedure.result.kind == "real64" prototype = project.modules["transform_api"].interfaces[0].procedures[0] @@ -547,7 +550,7 @@ def test_directory_project_records_missing_and_parent_only_submodule_dependencie project = parse_fortran_project(tmp_path) - assert project.dependencies["child_mod"] == {"parent_mod", "missing_mod"} + assert project.dependencies["parent_mod:child_mod"] == {"parent_mod", "missing_mod"} def test_program_and_block_data_scope_errors_use_public_parse_paths(): @@ -667,3 +670,47 @@ def test_an_imported_kind_constant_follows_the_use_nature(nature: str, kind: str ) assert project.modules["consumer"].variables[0].kind == kind + + +def test_same_named_submodules_of_different_ancestors_are_separate_project_scopes(): + """A submodule name is local to its ancestor, so ``a:impl`` and ``b:impl`` coexist. + + Each is keyed by its identity, depends on its own parent, and resolves + kinds through its own ancestor's parameters. A nested child of each does + the same through its direct parent. + """ + sources = {} + for ancestor, kind in (("a", 4), ("b", 8)): + sources[f"{ancestor}.f90"] = f""" +module {ancestor} + integer, parameter :: wp = {kind} + interface + module subroutine run(x) + real(wp), intent(inout) :: x + end subroutine run + end interface +end module {ancestor} +""" + sources[f"{ancestor}_impl.f90"] = f""" +submodule ({ancestor}) impl +contains + module subroutine run(x) + real(wp), intent(inout) :: x + end subroutine run +end submodule impl +""" + sources[f"{ancestor}_leaf.f90"] = f""" +submodule ({ancestor}:impl) leaf + real(wp) :: scale +end submodule leaf +""" + + project = parse_fortran_project(sources) + + assert set(project.submodules) == {"a:impl", "b:impl", "a:leaf", "b:leaf"} + assert project.dependencies["a:leaf"] == {"a:impl"} + assert project.dependencies["b:impl"] == {"b"} + assert project.submodules["a:impl"].procedures[0].arguments[0].kind == "4" + assert project.submodules["b:impl"].procedures[0].arguments[0].kind == "8" + assert project.submodules["a:leaf"].variables[0].kind == "4" + assert project.submodules["b:leaf"].variables[0].kind == "8" From 858c1bcf636d409abfd41e607986c58c56cdf244 Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 15:55:18 +0100 Subject: [PATCH 19/47] Take scalar and character storage arguments through shared runtime helpers Accepting rank-zero storage inlined NumPy type, rank, order, alignment, and writeability checks, with their error strings, into every scalar and fixed-width character argument of every wrapped routine. For the full reference BLAS that grew the generated C wrapper from 1.48 MB to 2.38 MB and its single-threaded -O3 compile from 6.4 s to 10.9 s, which set the whole optimized build time. Each such argument is now one call into the bundled runtime: prik__or_storage for numeric scalars (one per common type, with the generic prik_rank_zero_storage for converted storage), and prik_character_input / prik_character_result for fixed-width characters. The checks and messages are unchanged. The helpers are out of line and marked possibly unused, so they neither re-expand at each call nor warn in bindings that do not call them. The optimized full-BLAS build is back to main's time (about 13.5 s here) and the wrapper is 1.46 MB. Co-Authored-By: Claude Opus 5.5 --- docs/developer/packages/codegen/c-binding.md | 8 +- prik/codegen/c/binding.py | 138 +++++------- prik/runtime/native_support/prik_binding.h | 199 ++++++++++++++++++ .../codegen/test_positional_only_lowering.py | 3 +- .../test_dense_array_shape_lowering.py | 2 +- .../test_primitive_scalar_input_lowering.py | 12 +- .../test_ordinary_fortran_codegen_baseline.py | 4 +- .../pipeline/test_wrapper_generator.py | 4 +- .../runtime/test_native_support.py | 2 +- .../codegen/test_fixed_string_writeback.py | 17 +- .../codegen/test_string_input_lowering.py | 6 +- 11 files changed, 284 insertions(+), 111 deletions(-) diff --git a/docs/developer/packages/codegen/c-binding.md b/docs/developer/packages/codegen/c-binding.md index 4de281ab3..e0735c27d 100644 --- a/docs/developer/packages/codegen/c-binding.md +++ b/docs/developer/packages/codegen/c-binding.md @@ -264,13 +264,7 @@ static PyObject * wrap_double_value(PyObject * self, PyObject * args, PyObject * double bound_value; double result; if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O", kwlist, &bound_value_obj)) return NULL; - if (PyArray_Check(bound_value_obj)) { - if (PyArray_TYPE((PyArrayObject *)bound_value_obj) != NPY_FLOAT64 || PyArray_NDIM((PyArrayObject *)bound_value_obj) != 0) { PyErr_SetString(PyExc_TypeError, "Argument value requires exact rank-zero numpy.float64 storage"); return NULL; }; - if (!PyArray_ISNOTSWAPPED((PyArrayObject *)bound_value_obj) || !PyArray_ISALIGNED((PyArrayObject *)bound_value_obj)) { PyErr_SetString(PyExc_TypeError, "Argument value requires native byte order and aligned storage"); return NULL; }; - memcpy(&bound_value, PyArray_DATA((PyArrayObject *)bound_value_obj), sizeof(bound_value)); - } else { - if (prik_float64_unpack_exact(bound_value_obj, &bound_value) < 0) { if (!PyErr_Occurred()) { PyErr_Format(PyExc_TypeError, "Expected an argument of type numpy.float64 or rank-zero array for argument value. Received ", Py_TYPE(bound_value_obj)->tp_name); } return NULL; }; - } + if (prik_float64_or_storage(bound_value_obj, NPY_FLOAT64, 0, "numpy.float64", "numpy.float64", "value", &bound_value, NULL) < 0) { return NULL; }; result = bind_c_double_value(bound_value); PyObject * result_obj = prik_float64_to_numpy(&result); if (result_obj == NULL) { diff --git a/prik/codegen/c/binding.py b/prik/codegen/c/binding.py index f16f14a34..7591e8884 100644 --- a/prik/codegen/c/binding.py +++ b/prik/codegen/c/binding.py @@ -236,6 +236,13 @@ class _COverloadDispatch: } +# Scalar types whose bundled runtime takes a scalar-or-rank-zero-storage +# argument in one call (``prik__or_storage``). +_SCALAR_OR_STORAGE_HELPERS = frozenset( + {"bool", "int8", "int16", "int32", "int64", "float32", "float64", "complex64", "complex128"} +) + + class CBindingGenerator(ClassVisitor): """Build the CPython C half of a wrapper from validated binding-plan views. @@ -7716,58 +7723,61 @@ def _lower_argument_required_scalar_or_storage( scalar_type, storage_type: str, ) -> tuple[CDeclaration | CExpressionStatement | CIf, ...]: - """Take a scalar value or borrow exact rank-zero native storage.""" - array = f"(PyArrayObject *){names.object_name}" + """Take a scalar value or borrow exact rank-zero native storage. + + The rank-zero checks are one call into the bundled runtime, so a module + wrapping many routines with many scalar arguments does not repeat them + in every conversion it generates. + """ pointer = f"{names.value_name}_storage" + data = f"{names.value_name}_data" numpy_type, expected = self._numeric_array_dtype_selectors(plan) reference = plan.binding.scalar_actual_mode is ScalarActualMode.NUMERIC_REFERENCE - array_nodes: list[CExpressionStatement] = [ + writeable = int(reference and plan.binding.scalar_storage_writable) + array_nodes = ( CExpressionStatement( CodeExpression( - f"if (PyArray_TYPE({array}) != {numpy_type} || PyArray_NDIM({array}) != 0) {{ " - f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires exact ' - f'rank-zero {expected} storage"); return NULL; }}' + f"if (prik_rank_zero_storage({names.object_name}, {numpy_type}, {writeable}, " + f'"{expected}", "{plan.binding.python_name}", &{data}) < 0) {{ return NULL; }}' ) ), CExpressionStatement( CodeExpression( - f"if (!PyArray_ISNOTSWAPPED({array}) || !PyArray_ISALIGNED({array})) {{ " - f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires native ' - 'byte order and aligned storage"); return NULL; }' - ) - ), - ] - if reference and plan.binding.scalar_storage_writable: - array_nodes.append( - CExpressionStatement( - CodeExpression( - f"if (!PyArray_ISWRITEABLE({array})) {{ " - f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires ' - 'writeable storage"); return NULL; }' - ) - ) - ) - array_nodes.append( - CExpressionStatement( - CodeExpression( - f"{pointer} = ({storage_type} *)PyArray_DATA({array})" + f"{pointer} = ({storage_type} *){data}" if reference - else f"memcpy(&{names.value_name}, PyArray_DATA({array}), sizeof({names.value_name}))" + else f"memcpy(&{names.value_name}, {data}, sizeof({names.value_name}))" ) - ) + ), ) # A Python value unpacks as its canonical C type, then converts to the # storage the native dummy declares when that spelling differs. converted = storage_type != scalar_type.c_spelling + suffix = self._scalar_helper_suffix(scalar_type) + if not converted and suffix in _SCALAR_OR_STORAGE_HELPERS: + # The whole transfer is one call into the bundled runtime. + target = f"&{pointer}" if reference else "NULL" + return ( + CDeclaration(names.object_name, "PyObject *"), + CDeclaration(names.value_name, storage_type), + *((CDeclaration(pointer, f"{storage_type} *", CodeExpression("NULL")),) if reference else ()), + CExpressionStatement( + CodeExpression( + f"if (prik_{suffix}_or_storage({names.object_name}, {numpy_type}, {writeable}, " + f'"{scalar_type.python_type_name}", "{expected}", "{plan.binding.python_name}", ' + f"&{names.value_name}, {target}) < 0) {{ return NULL; }}" + ) + ), + ) unpacked = f"{names.value_name}_converted" if converted else names.value_name return ( CDeclaration(names.object_name, "PyObject *"), CDeclaration(names.value_name, storage_type), *((CDeclaration(unpacked, scalar_type.c_spelling),) if converted else ()), *((CDeclaration(pointer, f"{storage_type} *", CodeExpression("NULL")),) if reference else ()), + CDeclaration(data, "void *", CodeExpression("NULL")), CIf( CodeExpression(f"PyArray_Check({names.object_name})"), - body=tuple(array_nodes), + body=array_nodes, else_body=( self._scalar_exact_unpack_statement( scalar_type, @@ -7850,53 +7860,26 @@ def _required_string_or_storage_nodes( plan: ArgumentTransferPlan, names: _CArgumentNames, payload_name: str, - ) -> tuple[CIf, ...]: - """Accept either encoded text or exact fixed-width rank-zero bytes.""" + ) -> tuple[CExpressionStatement, ...]: + """Accept either encoded text or exact fixed-width rank-zero bytes. + + Both routes are one call into the bundled runtime, so each character + argument of every wrapped routine does not repeat them. + """ length = plan.character_length if length is None or length <= 0: raise ValueError(f"Character storage {plan.owner_path!r} needs a fixed width") - array = f"(PyArrayObject *){names.object_name}" - checks = [ - CExpressionStatement( - CodeExpression( - f"if (PyArray_TYPE({array}) != NPY_STRING || PyArray_NDIM({array}) != 0 || " - f"PyArray_ITEMSIZE({array}) != {length}) {{ " - f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires ' - f'rank-zero S{length} storage"); return NULL; }}' - ) - ), - CExpressionStatement( - CodeExpression( - f"if (!PyArray_ISALIGNED({array})) {{ " - f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires ' - 'aligned storage"); return NULL; }' - ) - ), - ] - if ( + writeable = int( plan.binding.scalar_storage_writable and plan.binding.scalar_actual_mode is ScalarActualMode.CHARACTER_REFERENCE - ): - checks.append( - CExpressionStatement( - CodeExpression( - f"if (!PyArray_ISWRITEABLE({array})) {{ " - f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} requires ' - 'writeable storage"); return NULL; }' - ) - ) - ) - checks.extend( - ( - CExpressionStatement(CodeExpression(f"{payload_name} = (const char *)PyArray_DATA({array})")), - CExpressionStatement(CodeExpression(f"{names.length_name} = {length}")), - ) ) return ( - CIf( - CodeExpression(f"PyArray_Check({names.object_name})"), - body=tuple(checks), - else_body=self._required_string_validation_nodes(plan, names, payload_name), + CExpressionStatement( + CodeExpression( + f"if (prik_character_input({names.object_name}, {length}, " + f"{int(bool(plan.character_allows_embedded_nul))}, {writeable}, " + f'"{plan.binding.python_name}", &{payload_name}, &{names.length_name}) < 0) {{ return NULL; }}' + ) ), ) @@ -13207,21 +13190,14 @@ def _mixed_string_writeback_nodes( ), ) if source.binding.scalar_actual_mode is ScalarActualMode.CHARACTER_REFERENCE: - array_conversion = CExpressionStatement( + # Decoding updated storage or converting and releasing the copy is + # one runtime call. + converted_value = CExpressionStatement( CodeExpression( - f"{target} = PyUnicode_DecodeUTF8((const char *){names.value_name}, " - f'(Py_ssize_t){names.length_name}, "strict")' + f"{target} = prik_character_result({names.object_name}, &{names.value_name}, " + f"(Py_ssize_t){names.length_name})" ) ) - converted_value = CIf( - CodeExpression(f"PyArray_Check({names.object_name})"), - body=(array_conversion,), - else_body=( - conversion, - CExpressionStatement(CodeExpression(f"free({names.value_name})")), - CExpressionStatement(CodeExpression(f"{names.value_name} = NULL")), - ), - ) if source.binding.optional_mode is OptionalMode.REQUIRED: return CDeclaration(target, "PyObject *", CodeExpression("NULL")), converted_value, failure if source.binding.optional_mode is OptionalMode.NULLABLE_VALUE: diff --git a/prik/runtime/native_support/prik_binding.h b/prik/runtime/native_support/prik_binding.h index 49b75f539..8371beae7 100644 --- a/prik/runtime/native_support/prik_binding.h +++ b/prik/runtime/native_support/prik_binding.h @@ -191,6 +191,13 @@ static inline const char *prik_derived_type_info_capsule_name(void) #define PRIK_NO_INLINE #endif +/* A shared helper a given binding may never call; it compiles without warning. */ +#if defined(__GNUC__) || defined(__clang__) +#define PRIK_MAYBE_UNUSED __attribute__((unused)) +#else +#define PRIK_MAYBE_UNUSED +#endif + #ifdef PRIK_BINDING_CAPTURE_ADDRESS /* * Report the address a caller already passed by reference. @@ -1052,6 +1059,136 @@ static inline int prik_array_validate( argument_name); } +/* + * Borrow the exact rank-zero NumPy storage an ndarray passes for one numeric + * scalar argument. + * + * value must be an ndarray. Returns 0 with *data set to its element, or -1 + * with a TypeError set when it is not rank-zero storage of numpy_type in + * native order. A scalar argument may appear in every routine a module wraps, + * so the checks live here once rather than in each generated conversion. + */ +PRIK_NO_INLINE PRIK_MAYBE_UNUSED static int prik_rank_zero_storage( + PyObject *value, + int numpy_type, + int require_writeable, + const char *python_type, + const char *argument_name, + void **data) +{ + PyArrayObject *array = (PyArrayObject *)value; + if (PyArray_TYPE(array) != numpy_type || PyArray_NDIM(array) != 0) { + PyErr_Format(PyExc_TypeError, "Argument %s requires exact rank-zero %s storage", argument_name, python_type); + return -1; + } + if (!PyArray_ISNOTSWAPPED(array) || !PyArray_ISALIGNED(array)) { + PyErr_Format(PyExc_TypeError, "Argument %s requires native byte order and aligned storage", argument_name); + return -1; + } + if (require_writeable && !PyArray_ISWRITEABLE(array)) { + PyErr_Format(PyExc_TypeError, "Argument %s requires writeable storage", argument_name); + return -1; + } + *data = PyArray_DATA(array); + return 0; +} + +/* + * Borrow the rank-zero fixed-width bytes an ndarray passes for one character + * argument. + * + * value must be an ndarray. Returns 0 with *data set to its bytes, or -1 with + * a TypeError set when it is not rank-zero S storage. + */ +PRIK_NO_INLINE PRIK_MAYBE_UNUSED static int prik_rank_zero_bytes( + PyObject *value, + Py_ssize_t width, + int require_writeable, + const char *argument_name, + const char **data) +{ + PyArrayObject *array = (PyArrayObject *)value; + if (PyArray_TYPE(array) != NPY_STRING || PyArray_NDIM(array) != 0 || PyArray_ITEMSIZE(array) != width) { + PyErr_Format(PyExc_TypeError, "Argument %s requires rank-zero S%zd storage", argument_name, width); + return -1; + } + if (!PyArray_ISALIGNED(array)) { + PyErr_Format(PyExc_TypeError, "Argument %s requires aligned storage", argument_name); + return -1; + } + if (require_writeable && !PyArray_ISWRITEABLE(array)) { + PyErr_Format(PyExc_TypeError, "Argument %s requires writeable storage", argument_name); + return -1; + } + *data = (const char *)PyArray_DATA(array); + return 0; +} + +/* + * Take one fixed-width character argument that also accepts rank-zero bytes. + * + * An ndarray must be rank-zero S storage, and *source then points at + * its bytes. Any other value must be a str whose UTF-8 encoding is exactly + * width bytes, without embedded NUL unless allow_embedded_nul is set. + * *length receives the byte count. Returns 0, or -1 with an exception set. + */ +PRIK_NO_INLINE PRIK_MAYBE_UNUSED static int prik_character_input( + PyObject *value, + Py_ssize_t width, + int allow_embedded_nul, + int require_writeable, + const char *argument_name, + const char **source, + Py_ssize_t *length) +{ + if (PyArray_Check(value)) { + if (prik_rank_zero_bytes(value, width, require_writeable, argument_name, source) < 0) { + return -1; + } + *length = width; + return 0; + } + if (!PyUnicode_Check(value)) { + PyErr_Format( + PyExc_TypeError, + "Expected an argument of type str for argument %s. Received ", + argument_name, + Py_TYPE(value)->tp_name); + return -1; + } + *source = PyUnicode_AsUTF8AndSize(value, length); + if (*source == NULL) { + return -1; + } + if (!allow_embedded_nul && (Py_ssize_t)strlen(*source) != *length) { + PyErr_Format(PyExc_TypeError, "Argument %s cannot contain embedded NUL", argument_name); + return -1; + } + if (*length != width) { + PyErr_Format(PyExc_TypeError, "Argument %s must encode to exactly %zd bytes", argument_name, width); + return -1; + } + return 0; +} + +/* + * Return the Python value of one fixed-width character argument after the call. + * + * Rank-zero bytes storage was updated in place, so its bytes are decoded. A + * str was copied into *buffer, which is converted and then released. + */ +PRIK_NO_INLINE PRIK_MAYBE_UNUSED static PyObject *prik_character_result(PyObject *value, char **buffer, Py_ssize_t length) +{ + PyObject *result; + if (PyArray_Check(value)) { + return PyUnicode_DecodeUTF8((const char *)*buffer, length, "strict"); + } + result = Py_BuildValue("s", (const char *)*buffer); + free(*buffer); + *buffer = NULL; + return result; +} + #ifdef PRIK_BINDING_ASSUMED_TYPE /* One call-local native actual. Its CFI metadata describes the payload; the * type tag also distinguishes PRIK derived types with identical storage sizes. */ @@ -1508,6 +1645,68 @@ static inline int prik_complex128_unpack_exact(PyObject *value, double complex * return 0; } +/* + * Take one numeric scalar argument that also accepts rank-zero storage. + * + * An ndarray must be exact rank-zero storage of numpy_type; storage, when not + * NULL, then receives its element's address, and otherwise the element is + * copied into local. Any other value must be an exact NumPy scalar, unpacked + * into local, which storage then points at. Returns 0, or -1 with a TypeError + * set. One out-of-line helper per type keeps every scalar argument of every + * wrapped routine a single call in the generated binding. + */ +#define PRIK_DEFINE_SCALAR_OR_STORAGE(suffix, ctype) \ + PRIK_NO_INLINE PRIK_MAYBE_UNUSED static int prik_##suffix##_or_storage( \ + PyObject *value, \ + int numpy_type, \ + int require_writeable, \ + const char *scalar_type, \ + const char *array_type, \ + const char *argument_name, \ + ctype *local, \ + ctype **storage) \ + { \ + void *data = NULL; \ + if (PyArray_Check(value)) { \ + if (prik_rank_zero_storage( \ + value, numpy_type, require_writeable, array_type, argument_name, &data) < 0) { \ + return -1; \ + } \ + if (storage != NULL) { \ + *storage = (ctype *)data; \ + } else { \ + memcpy(local, data, sizeof(ctype)); \ + } \ + return 0; \ + } \ + if (prik_##suffix##_unpack_exact(value, local) < 0) { \ + if (!PyErr_Occurred()) { \ + PyErr_Format( \ + PyExc_TypeError, \ + "Expected an argument of type %s or rank-zero array for argument %s. " \ + "Received ", \ + scalar_type, \ + argument_name, \ + Py_TYPE(value)->tp_name); \ + } \ + return -1; \ + } \ + if (storage != NULL) { \ + *storage = local; \ + } \ + return 0; \ + } + +PRIK_DEFINE_SCALAR_OR_STORAGE(bool, bool) +PRIK_DEFINE_SCALAR_OR_STORAGE(int8, int8_t) +PRIK_DEFINE_SCALAR_OR_STORAGE(int16, int16_t) +PRIK_DEFINE_SCALAR_OR_STORAGE(int32, int32_t) +PRIK_DEFINE_SCALAR_OR_STORAGE(int64, int64_t) +PRIK_DEFINE_SCALAR_OR_STORAGE(float32, float) +PRIK_DEFINE_SCALAR_OR_STORAGE(float64, double) +PRIK_DEFINE_SCALAR_OR_STORAGE(complex64, float complex) +PRIK_DEFINE_SCALAR_OR_STORAGE(complex128, double complex) + /* Convert Python truth value into native bool storage, propagating conversion errors. */ static inline int prik_bool_unpack(PyObject *value, bool *destination) { diff --git a/tests/c/functions/codegen/test_positional_only_lowering.py b/tests/c/functions/codegen/test_positional_only_lowering.py index 14e6f3aa9..05e0bcea5 100644 --- a/tests/c/functions/codegen/test_positional_only_lowering.py +++ b/tests/c/functions/codegen/test_positional_only_lowering.py @@ -28,7 +28,8 @@ def test_a_positional_only_binding_takes_no_keyword_dictionary(): # The native declaration keeps the header's spelling; the Python surface does not. assert "double blend(double __x, double __y);" in binding assert "blend(arg0, arg1) -> float64" in binding - assert "for argument arg0." in binding + # Errors name the argument the way Python does. + assert '"arg0", &bound_arg0' in binding assert "__x" not in binding.split("static PyObject * wrap_blend")[1] diff --git a/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py b/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py index c2adab2b7..6dadfc820 100644 --- a/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py +++ b/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py @@ -216,7 +216,7 @@ def test_external_interface_declares_late_extent_before_dependent_array(): "late_extent_external.late_extent.n", "late_extent_external.late_extent.values", ) - assert c_source.index("prik_int32_unpack_exact(bound_n_obj, &bound_n)") < c_source.index( + assert c_source.index("prik_int32_or_storage(bound_n_obj, ") < c_source.index( "bound_values_bind_fixed[0] = (long long)(bound_n);" ) signature = "subroutine late_extent(values, n)" diff --git a/tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py b/tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py index f8929aea7..6ddc49e61 100644 --- a/tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py +++ b/tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py @@ -33,10 +33,11 @@ def test_scalar_input_registry_lowers_completed_type_into_the_native_support_api c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") assert f"{c_type} bound_x;" in c_source - assert ( - f"if (prik_{helper_suffix.casefold().removeprefix('npy_')}_unpack_exact(bound_x_obj, &bound_x) < 0)" in c_source - ) - assert "if (!PyErr_Occurred())" in c_source + # The value, or rank-zero storage of the same dtype, crosses in one call + # to the type's native-support helper. + suffix = helper_suffix.casefold().removeprefix("npy_") + assert f"if (prik_{suffix}_or_storage(bound_x_obj, {helper_suffix}, " in c_source + assert "&bound_x, " in c_source def test_binding_locals_are_isolated_from_identifiers_imported_by_c_headers(): @@ -51,5 +52,6 @@ def test_binding_locals_are_isolated_from_identifiers_imported_by_c_headers(): assert '#include "binding_support/prik_binding.h"' in c_source assert "double bound_complex;" in c_source - assert "prik_float64_unpack_exact(bound_complex_obj, &bound_complex)" in c_source + assert "prik_float64_or_storage(bound_complex_obj, NPY_FLOAT64, " in c_source + assert "&bound_complex, " in c_source assert "double complex;" not in c_source diff --git a/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py b/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py index feba1457f..37c16437e 100644 --- a/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py +++ b/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py @@ -25,8 +25,8 @@ def test_ordinary_fortran_wrapper_preserves_exact_generated_bytes(): "01c092ac9eaa0d90b58f0289a49ba0c71c967510e60a384602fe2e6e1e9b035f", ), "ordinary_entrypoint_baseline_wrapper.c": ( - 2613, - "19180196b0e7a25848c88e8985e94da0bda5ffecb78d68bf38909893cae58c3a", + 1823, + "ccefd674a96a84969abe54dbc9f599f9f004c2deefd1c3edc37117e34b9a1151", ), "ordinary_entrypoint_baseline_wrapper.h": ( 248, diff --git a/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py b/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py index 458477873..8d4b1a3fe 100644 --- a/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py +++ b/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py @@ -103,7 +103,7 @@ def swap_args(x: Float64, y: Float64) -> Float64: ... assert "double bind_c_swap_args(double * y, double * x);" in c_source assert 'static char * kwlist[] = {"x", "y", NULL};' in c_source assert 'PyArg_ParseTupleAndKeywords(args, kwargs, "OO", kwlist, &bound_x_obj, &bound_y_obj)' in c_source - assert "prik_float64_unpack_exact(bound_x_obj, &bound_x)" in c_source + assert "prik_float64_or_storage(bound_x_obj, NPY_FLOAT64, " in c_source assert "result = bind_c_swap_args(bound_y_storage, bound_x_storage);" in c_source assert "PyObject * result_obj = prik_float64_to_numpy(&result);" in c_source assert "PyMODINIT_FUNC PyInit_render_demo(void)" in c_source @@ -533,7 +533,7 @@ def test_scalar_copy_in_out_reuses_one_binding_local_without_bridge_copy(): bridge_source = next(source.text for source in generated_wrapper.sources if source.path.suffix == ".f90") assert c_source.count("int32_t bound_value;") == 1 - assert "prik_int32_unpack_exact(bound_value_obj, &bound_value)" in c_source + assert "prik_int32_or_storage(bound_value_obj, NPY_INT32, " in c_source assert "bind_c_bump(&bound_value);" in c_source assert "PyObject * result_obj = NULL;" in c_source assert "result_obj = prik_int32_to_numpy(&bound_value);" in c_source diff --git a/tests/fortran/infrastructure/runtime/test_native_support.py b/tests/fortran/infrastructure/runtime/test_native_support.py index 9d0f0a405..722199502 100644 --- a/tests/fortran/infrastructure/runtime/test_native_support.py +++ b/tests/fortran/infrastructure/runtime/test_native_support.py @@ -16,7 +16,6 @@ def test_native_binding_support_is_header_only_and_exposes_the_small_prik_api(): assert "static inline int prik_array_validate(" in header assert "static inline int prik_array_validate_ndarray(" in header assert "PyArrayObject *array," in header - assert header.count("PyArray_Check(value)") == 1 assert "PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F" in header assert "prik_array_actual" in header assert "prik_release_owned_memory" in header @@ -35,6 +34,7 @@ def test_native_binding_support_is_header_only_and_exposes_the_small_prik_api(): ) for suffix in scalar_suffixes: assert f"prik_{suffix}_unpack_exact" in header + assert f"PRIK_DEFINE_SCALAR_OR_STORAGE({suffix}, " in header assert f"prik_{suffix}_unpack" in header assert f"prik_{suffix}_to_python" in header assert f"prik_{suffix}_to_numpy" in header diff --git a/tests/fortran/strings/codegen/test_fixed_string_writeback.py b/tests/fortran/strings/codegen/test_fixed_string_writeback.py index a9db89381..27db3d440 100644 --- a/tests/fortran/strings/codegen/test_fixed_string_writeback.py +++ b/tests/fortran/strings/codegen/test_fixed_string_writeback.py @@ -93,11 +93,10 @@ def test_fixed_string_writeback_dispatches_to_named_binding_and_bridge_lowering( assert "memcpy(bound_name, bound_name_source, (size_t)bound_name_length);" in c_source assert "bound_name[bound_name_length] = '\\0';" in c_source assert "bind_c_replace_name(bound_name, (int64_t)bound_name_length);" in c_source - assert 'Py_BuildValue("s", (const char *)bound_name)' in c_source - conversion = c_source.index('Py_BuildValue("s", (const char *)bound_name)') - release = c_source.index("free(bound_name);", conversion) - assert conversion < release - assert release < c_source.index("if (result_obj == NULL)", conversion) + # Converting the updated value and releasing the copied buffer is one + # runtime call, made before the result is checked. + conversion = c_source.index("prik_character_result(bound_name_obj, &bound_name, ") + assert conversion < c_source.index("if (result_obj == NULL)", conversion) assert "void bind_c_discard_name(const char * name, int64_t name_length);" in c_source assert "bind_c_discard_name(bound_name, (int64_t)bound_name_length);" in c_source @@ -127,18 +126,18 @@ def replace_names( first_allocation = "bound_first = (char *)prik_malloc((size_t)bound_first_length + 1);" second_allocation = "bound_second = (char *)prik_malloc((size_t)bound_second_length + 1);" - assert c_source.index("prik_int32_unpack_exact(bound_count_obj, &bound_count)") < c_source.index(first_allocation) - assert c_source.index("bound_second_source = PyUnicode_AsUTF8AndSize") < c_source.index(first_allocation) + assert c_source.index("prik_int32_or_storage(bound_count_obj, ") < c_source.index(first_allocation) + assert c_source.index("prik_character_input(bound_second_obj, ") < c_source.index(first_allocation) assert c_source.index(first_allocation) < c_source.index(second_allocation) second_failure = c_source[c_source.index("if (bound_second == NULL)") : c_source.index(second_allocation) + 900] assert "free(bound_first); bound_first = NULL;" in second_failure assert "free(bound_second); bound_second = NULL;" in second_failure - first_conversion = 'result_0_obj = Py_BuildValue("s", (const char *)bound_first);' + # Converting the first string also releases its buffer, before the scalar result is read. + first_conversion = "result_0_obj = prik_character_result(bound_first_obj, &bound_first, " scalar_conversion = "prik_int32_to_numpy(&__return_0)" assert c_source.index(first_conversion) < c_source.index(scalar_conversion) - assert c_source.index("free(bound_first);", c_source.index(first_conversion)) < c_source.index(scalar_conversion) def test_string_writeback_conversion_failure_releases_unpublished_native_results(): diff --git a/tests/fortran/strings/codegen/test_string_input_lowering.py b/tests/fortran/strings/codegen/test_string_input_lowering.py index 1ef73c6b6..362533d43 100644 --- a/tests/fortran/strings/codegen/test_string_input_lowering.py +++ b/tests/fortran/strings/codegen/test_string_input_lowering.py @@ -70,10 +70,12 @@ def test_required_string_values_dispatch_to_named_binding_and_bridge_lowering(): assert "#include " in c_source assert "const char * bound_text = NULL;" in c_source + # A fixed width also accepts rank-zero bytes; both routes and their checks + # are one runtime call with the width and the NUL rule. + assert 'prik_character_input(bound_text_obj, 8, 0, 0, "text", &bound_text, &bound_text_length)' in c_source + # An assumed length has no storage route, so it validates encoded text inline. assert "bound_text = PyUnicode_AsUTF8AndSize(bound_text_obj, &bound_text_length);" in c_source assert "strlen(bound_text) != bound_text_length" in c_source - assert "bound_text_length != 8" in c_source - assert "must encode to exactly 8 bytes" in c_source assert "bind_c_fixed(bound_text, (int64_t)bound_text_length)" in c_source assert "bind_c_assumed(bound_text, (int64_t)bound_text_length)" in c_source From 965c8b92b3522dae21e460871bc2e7c974f1fcaf Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 16:17:54 +0100 Subject: [PATCH 20/47] Read separate module procedures as module procedures everywhere; locate units with the parser's scanner - One semantic inventory, _module_procedures, holds a module's contained procedures, its separate module procedures, and publicly listed external interfaces. Wrapping, the names a plain use offers declaration expressions, inherited generic specifics, and declaration call names all read it, so a module extending a generic built from separate specifics dispatches over them, and a separate specification function a plain use reaches resolves to its module. - FortranModule.separate_procedures names them in the parser model; FortranProject.procedures indexes them under their module, and single-file parsing resolves their kinds through the module's parameters, as project parsing already did. - --module-source-dir locates each searched source's modules and submodules with FortranParser.defined_units, the unit scanner parse_file runs first, instead of its own statement regexes, so fixed-form column-6 continuation and free-form keyword splits are read as the parser reads them. A source the scanner rejects is recorded as unreadable. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 15 ++- prik/parsers/fortran/models.py | 24 +++++ prik/parsers/fortran/module_sources.py | 50 ++++------ prik/parsers/fortran/parser.py | 36 +++++++- prik/semantics/fortran2ir.py | 56 ++++++++---- .../native/separate_generic_specific.f90 | 45 +++++++++ .../test_separate_generic_specifics.py | 47 ++++++++++ .../parsing/test_module_source_resolution.py | 37 ++++++++ .../test_separate_module_procedures.py | 91 +++++++++++++++++++ 9 files changed, 342 insertions(+), 59 deletions(-) create mode 100644 tests/fortran/modules/end_to_end/fixtures/native/separate_generic_specific.f90 create mode 100644 tests/fortran/modules/end_to_end/test_separate_generic_specifics.py create mode 100644 tests/fortran/modules/semantics/test_separate_module_procedures.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 45cc8ffc3..1f8c9982b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,8 +12,9 @@ release tags add a leading `v` to the package version. sources, PRIK follows each `use` to the source under those directories that defines the module and reads it too, so a multi-module library such as Open MPI's `mpi_f08` is supplied by its entry file. What a source defines is - read from its preprocessed text, so a module a macro or an `#include` - names is found, and a second definition only preprocessing reveals makes + read from its preprocessed text by the parser's own unit scanner, so a + module a macro or an `#include` names is found, a `module` or `submodule` + statement continued across lines in either source form is found, and a second definition only preprocessing reveals makes the module ambiguous. A needed module with no source, or with several, is an error. Discovery honors `use, intrinsic` and `use, non_intrinsic` for each scope separately, and an unstated `use` of an intrinsic module's name @@ -29,9 +30,13 @@ release tags add a leading `v` to the package version. from a processor module, such as `ieee_arithmetic`'s, is left to the processor rather than read from a parsed module of that name. - A separate module procedure declared by a `module function` or - `module subroutine` interface body is wrapped as its module's own - procedure, under ordinary accessibility; it previously needed an explicit - `public` statement naming it. + `module subroutine` interface body is its module's own procedure + everywhere: it is wrapped under ordinary accessibility (it previously + needed an explicit `public` statement naming it), a module extending a + generic inherits it as a specific, a declaration another module writes + resolves it as a specification function, it is indexed in + `FortranProject.procedures`, and its kinds resolve through its module's + parameters when a single file is parsed. - Fortran submodules are identified by `ancestor:name` throughout parsing, project ordering, compile scheduling, and kind resolution, so two modules may each have a submodule of the same name. `FortranProject.submodules` and diff --git a/prik/parsers/fortran/models.py b/prik/parsers/fortran/models.py index 6f8205fb4..ffda24248 100644 --- a/prik/parsers/fortran/models.py +++ b/prik/parsers/fortran/models.py @@ -475,6 +475,24 @@ class FortranModule: private_symbols: list[str] = field(default_factory=list) common_variables: list[str] = field(default_factory=list) + @property + def separate_procedures(self) -> list[FortranProcedureSignature]: + """Return the separate module procedures this module declares. + + A ``module function`` or ``module subroutine`` body in one of the + module's own interface blocks declares a procedure of this module that + a submodule implements. It is as much the module's procedure as one it + contains, but its declaration lives with the interface block rather + than in ``procedures``, which holds the bodies the module contains. + """ + return [ + signature + for interface in self.interfaces + if interface.name is None and not interface.abstract and interface.declaring_scope_kind == "module" + for signature in interface.procedures + if "module" in signature.attributes + ] + @dataclass class FortranSubmodule: @@ -614,6 +632,12 @@ class FortranProject: programs: dict[str, FortranProgram] = field(default_factory=dict) procedures: dict[str, FortranProcedureSignature] = field(default_factory=dict) + """Procedures by ``owner.name`` and, first seen, by bare name. + + A module's entries include its separate module procedures, keyed by the + module, as well as those it contains; a submodule's entries are the bodies + it contains, keyed by its ``ancestor:name``. + """ derived_types: dict[str, FortranDerivedType] = field(default_factory=dict) diff --git a/prik/parsers/fortran/module_sources.py b/prik/parsers/fortran/module_sources.py index ccd96fa32..0f7ceb5f5 100644 --- a/prik/parsers/fortran/module_sources.py +++ b/prik/parsers/fortran/module_sources.py @@ -8,11 +8,12 @@ What a source defines is decided by its preprocessed text, never by its raw text: a macro or an ``#include`` can name a module, and a conditional block can -remove one. Every searched source is therefore located by the ``module`` and -``submodule`` statements its preprocessed text holds, and a source whose raw -text has nothing a preprocessor could change is read as it stands, which spares -the preprocessor for it. The sources located for a unit are then parsed, and -their parsed units are the definition. A source is selected into the project +remove one. Every searched source is therefore located by the program units +the parser's own unit scanner finds in its preprocessed text -- the same first +step parsing takes, with the same logical lines in either source form -- and a +source whose raw text has nothing a preprocessor could change is read as it +stands, which spares the preprocessor for it. The sources located for a unit +are then parsed, and their parsed units are the definition. A source is selected into the project only once it is the unit's one definition, so reading a candidate never makes it part of the project. """ @@ -32,22 +33,6 @@ # Suffixes a Fortran compiler accepts as free- or fixed-form source. _FORTRAN_SOURCE_SUFFIXES = frozenset({".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08", ".fpp"}) -# A statement may follow another on one line after ``;``, and a name may -# follow its keyword on a continuation line, so both are accepted. Locating a -# source that turns out not to define a unit costs a parse, never a result. -_STATEMENT_START = r"(?:^|;)[ \t]*" -_CONTINUATION = r"(?:[ \t]*&[ \t]*(?:!.*)?\n[ \t]*&?)?" -_MODULE_STATEMENT = re.compile( - _STATEMENT_START + r"module" + r"(?:[ \t]+|" + _CONTINUATION + r"[ \t]*)" - r"(?!(?:procedure|function|subroutine|pure|impure|elemental|recursive|non_recursive)\b)" - r"(?P[a-z][a-z0-9_]*)\b(?![ \t]*[(=%])", - re.IGNORECASE | re.MULTILINE, -) -_SUBMODULE_STATEMENT = re.compile( - _STATEMENT_START + r"submodule[ \t]*" + _CONTINUATION + r"\([ \t]*(?P[a-z][a-z0-9_]*)[ \t]*" - r"(?::[ \t]*[a-z][a-z0-9_]*[ \t]*)?\)[ \t]*" + _CONTINUATION + r"[ \t]*(?P[a-z][a-z0-9_]*)", - re.IGNORECASE | re.MULTILINE, -) # Raw text a preprocessor can change: a directive, or a Fortran ``include``, # which PRIK's preprocessing expands as well. _PREPROCESSED_TEXT = re.compile(r"^[ \t]*(?:#|include[ \t]*['\"])", re.IGNORECASE | re.MULTILINE) @@ -167,7 +152,7 @@ def located_units(self) -> list[str]: def raw_definers(self, unit: str) -> list[Path]: """Return the unreadable sources whose raw text shows ``unit``, to explain a missing one.""" - return [path for path in self.unreadable if unit in _statement_units(path.read_text(errors="replace"))] + return [path for path in self.unreadable if unit in self._scan(path, path.read_text(errors="replace"))] def _locate(self) -> dict[Path, frozenset[str]]: """Return the units each searched source's preprocessed text states, computed once.""" @@ -179,7 +164,7 @@ def _locate(self) -> dict[Path, frozenset[str]]: if self._command_line_macros or _PREPROCESSED_TEXT.search(raw): opaque.append(path) else: - located[path] = _statement_units(raw) + located[path] = self._scan(path, raw) with ThreadPoolExecutor(max_workers=_LOCATE_WORKERS) as pool: for path, text in zip(opaque, pool.map(self._preprocessed_text, opaque), strict=True): if isinstance(text, Exception): @@ -187,12 +172,20 @@ def _locate(self) -> dict[Path, frozenset[str]]: # nothing; it is named if a needed unit stays missing. self.unreadable[path] = text continue - located[path] = _statement_units(text) + located[path] = self._scan(path, text) if located[path]: self._texts[path] = text self._located = located return self._located + def _scan(self, path: Path, text: str) -> frozenset[str]: + """Return the units one source's text opens; text the parser rejects opens none.""" + try: + return self._parser.defined_units(text, str(path)) + except FortranParseError as error: + self.unreadable.setdefault(path, error) + return frozenset() + def _preprocessed_text(self, path: Path) -> str | Exception: try: return self._read_source(path) @@ -262,12 +255,3 @@ def _searched_files(search_dirs: Iterable[Path]) -> tuple[Path, ...]: if path.suffix.casefold() in _FORTRAN_SOURCE_SUFFIXES and path.is_file(): files.setdefault(path.resolve(), None) return tuple(files) - - -def _statement_units(text: str) -> frozenset[str]: - """Return the modules and ``ancestor:name`` submodules the statements of ``text`` open.""" - units = {match.group("name").casefold() for match in _MODULE_STATEMENT.finditer(text)} - units.update( - f"{match.group('ancestor')}:{match.group('name')}".casefold() for match in _SUBMODULE_STATEMENT.finditer(text) - ) - return frozenset(units) diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index b812e6e9f..f28a0802b 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -1510,6 +1510,24 @@ def __init__(self) -> None: """Create the stateless structural scanner used by parser entrypoints.""" self._source_unit_scanner = _SourceUnitScanner() + def defined_units(self, code: str, filename: str | None = None) -> frozenset[str]: + """Return the modules and submodules one source opens, without visiting them. + + This is the first step of :meth:`parse_file`: the source is read by + the parser's own logical-line rules, including both source forms' + continuation, and sliced into its program units. A module is named by + itself and a submodule by ``ancestor:name``, case-folded. A source the + slicer rejects raises :class:`FortranParseError`, as parsing it would. + """ + _lines, _root, units = self._helper_prepare_source_units(code, filename) + return frozenset( + str(owner).casefold() + for unit in units + if unit.kind in {"module", "submodule"} + for owner in (self._module_like_unit_owner(unit),) + if owner + ) + def parse_file( self, source_or_path: str | Path, @@ -2202,10 +2220,16 @@ def _resolve_file_compile_time_facts(self, units: _ParsedFileUnits) -> None: @staticmethod def _helper_file_procedures(units: _ParsedFileUnits) -> tuple[FortranProcedureSignature, ...]: - """Return file procedures in their established resolution order.""" + """Return file procedures in their established resolution order. + + A separate module procedure's interface body is host associated with + its module, as a contained procedure is, so its kinds resolve against + the module's parameters too. + """ procedures = list(units.procedures) for module in units.modules: procedures.extend(module.procedures) + procedures.extend(module.separate_procedures) for submodule in units.submodules: procedures.extend(submodule.procedures) return tuple(procedures) @@ -2440,6 +2464,16 @@ def _helper_index_project_module(self, project: FortranProject, module: FortranM self._insert_unique_scope_symbol(project.modules, module_key, module, label="project module scope") project.dependencies[module_key] = source_module_dependencies([module]) self._helper_index_project_owner_members(project, module, module_key) + # A separate module procedure is the module's procedure though an + # interface body declares it, so it is indexed as the module's own. + for procedure in module.separate_procedures: + self._insert_unique_scope_symbol( + project.procedures, + f"{module_key}.{procedure.name.lower()}", + procedure, + label="project procedure scope", + ) + project.procedures.setdefault(procedure.name.lower(), procedure) def _helper_index_project_submodule(self, project: FortranProject, submodule: FortranSubmodule) -> None: """Index one submodule, its dependencies, and its public models.""" diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index f5a96c0a1..2adc07f7c 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -1278,15 +1278,15 @@ def _module_prototypes( ) return prototypes - @staticmethod - def _module_declaration_call_names(module: FortranModule) -> set[str]: + @classmethod + def _module_declaration_call_names(cls, module: FortranModule) -> set[str]: """Collect bare call names appearing in module-owned declaration shapes.""" variables = [ *getattr(module, "variables", ()), *(field for derived in module.derived_types for field in derived.fields), *( variable - for procedure in module.procedures + for procedure in cls._module_procedures(module) for variable in (*procedure.arguments, procedure.result) if variable is not None ), @@ -1535,10 +1535,7 @@ def _visit_FortranModule( context = self._module_derived_type_context(module, index) self._record_abstract_type_names(module) callback_interfaces = self._module_callback_interfaces(index, module) - source_procedures = [ - *module.procedures, - *self._module_explicit_interface_procedures(module), - ] + source_procedures = list(self._module_procedures(module)) semantic_functions = [ self.visit( proc, @@ -2428,20 +2425,38 @@ def _abstract_types_from_file(parsed_file: FortranFile) -> set[tuple[str, str]]: if any(str(attribute).casefold() == "abstract" for attribute in dtype.attributes) } - @staticmethod - def _known_procedures_from_file(parsed_file: FortranFile) -> set[tuple[str, str]]: + @classmethod + def _known_procedures_from_file(cls, parsed_file: FortranFile) -> set[tuple[str, str]]: """Collect module-qualified procedures declared by one parsed file.""" - return {(module.name, procedure.name) for module in parsed_file.modules for procedure in module.procedures} + return { + (module.name, procedure.name) + for module in parsed_file.modules + for procedure in cls._module_procedures(module) + } - @staticmethod - def _known_procedures_from_project(project: FortranProject) -> set[tuple[str, str]]: + @classmethod + def _known_procedures_from_project(cls, project: FortranProject) -> set[tuple[str, str]]: """Collect module-qualified procedures known to one parsed project.""" - return {(module.name, procedure.name) for module in project.modules.values() for procedure in module.procedures} + return { + (module.name, procedure.name) + for module in project.modules.values() + for procedure in cls._module_procedures(module) + } - @staticmethod - def _module_explicit_interface_procedures( - module: FortranModule, - ) -> list[FortranProcedureSignature]: + @classmethod + def _module_procedures(cls, module: FortranModule) -> tuple[FortranProcedureSignature, ...]: + """Return every procedure one module declares, which is what a ``use`` of it can call. + + That is each procedure the module contains, each separate module + procedure an interface body declares, and each external procedure an + explicit public list publishes from an unnamed interface block. Every + reading of "this module's procedures" -- wrapping, generic specifics a + later module inherits, and callables declarations name -- is this one. + """ + return (*module.procedures, *cls._module_interface_procedures(module)) + + @classmethod + def _module_interface_procedures(cls, module: FortranModule) -> list[FortranProcedureSignature]: """Return the module procedures unnamed interface blocks declare. A ``module function`` or ``module subroutine`` body declares a separate @@ -2453,16 +2468,17 @@ def _module_explicit_interface_procedures( matching implementation is parsed. """ public_names = {name.casefold() for name in module.public_symbols} + separate = {id(signature) for signature in module.separate_procedures} declared_names = {procedure.name.casefold() for procedure in module.procedures} procedures: list[FortranProcedureSignature] = [] - for interface in module.interfaces: + for interface in cls._module_interfaces(module): if interface.name is not None or interface.abstract: continue for procedure in interface.procedures: name = procedure.name.casefold() if name in declared_names: continue - if name not in public_names and "module" not in procedure.attributes: + if name not in public_names and id(procedure) not in separate: continue declared_names.add(name) procedures.append(procedure) @@ -3864,7 +3880,7 @@ def _inherited_generic_specifics( inherited: list[_SpecificProcedure] = [] lookup: dict[tuple[str, str], SemanticFunction] = {} for source_module, source_generic in self._imported_generic_interfaces(module, generic_name, modules): - signatures = {procedure.name.casefold(): procedure for procedure in source_module.procedures} + signatures = {procedure.name.casefold(): procedure for procedure in self._module_procedures(source_module)} source_context = self._module_derived_type_context(source_module, modules) names = source_generic.specific_procedures or [item.name for item in source_generic.procedures] for name in names: diff --git a/tests/fortran/modules/end_to_end/fixtures/native/separate_generic_specific.f90 b/tests/fortran/modules/end_to_end/fixtures/native/separate_generic_specific.f90 new file mode 100644 index 000000000..116dda071 --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/native/separate_generic_specific.f90 @@ -0,0 +1,45 @@ +module operations + implicit none + private + public :: convert + + interface + module function convert_i(x) result(y) + integer, intent(in) :: x + integer :: y + end function convert_i + module function convert_r(x) result(y) + real(8), intent(in) :: x + real(8) :: y + end function convert_r + end interface + + interface convert + module procedure convert_i, convert_r + end interface convert +end module operations + +submodule (operations) operations_impl + implicit none +contains + module procedure convert_i + y = x + 1 + end procedure convert_i + + module procedure convert_r + y = 2 * x + end procedure convert_r +end submodule operations_impl + +module facade + use operations, only : convert + implicit none + interface convert + module procedure convert_k + end interface convert +contains + integer(8) function convert_k(x) + integer(8), intent(in) :: x + convert_k = 10 * x + end function convert_k +end module facade diff --git a/tests/fortran/modules/end_to_end/test_separate_generic_specifics.py b/tests/fortran/modules/end_to_end/test_separate_generic_specifics.py new file mode 100644 index 000000000..dc2ca5295 --- /dev/null +++ b/tests/fortran/modules/end_to_end/test_separate_generic_specifics.py @@ -0,0 +1,47 @@ +"""A separate module procedure is a generic specific a later module inherits. + +``module function`` and ``module subroutine`` interface bodies declare +procedures the module owns and a submodule implements, so a module extending +a generic built from them dispatches over them too. +""" + +from __future__ import annotations + +from pathlib import Path + +import numpy as np +import pytest + +from prik import build_fortran_extension +from tests.fortran._support.wrapper_build import ( + _build_generated_pyi_and_import, + _import_from_build_dir, + _sole_native_module, +) + +pytestmark = pytest.mark.fortran_end_to_end + +NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" + + +def _package(source: Path, tmp_path: Path, lane: str): + """Build one fixture from source or through its generated contract and import it.""" + if lane == "source": + result = build_fortran_extension(source, output_dir=tmp_path / "source", output_name=source.stem) + return _import_from_build_dir(result.module_name, result.output_dir) + return _build_generated_pyi_and_import(source, tmp_path / "pyi") + + +def _namespace(package, name: str): + return getattr(package, name) if hasattr(package, name) else getattr(_sole_native_module(package), name) + + +@pytest.mark.parametrize("lane", ["source", "generated_pyi"]) +def test_a_generic_of_separate_specifics_dispatches_through_an_importing_module(tmp_path: Path, lane: str): + """``facade`` extends ``convert``, so it inherits the specifics only interface bodies declare.""" + package = _package(NATIVE_FIXTURES / "separate_generic_specific.f90", tmp_path, lane) + facade = _namespace(package, "facade") + + assert facade.convert(np.int32(3)) == np.int32(4) + assert facade.convert(np.float64(1.5)) == np.float64(3.0) + assert facade.convert(np.int64(2)) == np.int64(20) diff --git a/tests/fortran/modules/parsing/test_module_source_resolution.py b/tests/fortran/modules/parsing/test_module_source_resolution.py index 65871e335..2ffba5316 100644 --- a/tests/fortran/modules/parsing/test_module_source_resolution.py +++ b/tests/fortran/modules/parsing/test_module_source_resolution.py @@ -250,3 +250,40 @@ def test_submodules_implementing_a_used_module_are_selected_with_it(tmp_path: Pa assert set(resolved) == {api.resolve(), impl.resolve(), leaf.resolve(), entry} assert resolved.index(api.resolve()) < resolved.index(impl.resolve()) < resolved.index(leaf.resolve()) + + +@pytest.mark.parametrize( + ("relative", "text", "unit"), + [ + pytest.param( + "lib/dep.f", + " module\n &fixed_dep\n integer x\n end module fixed_dep\n", + "fixed_dep", + id="fixed-form-column-six-continuation", + ), + pytest.param( + "lib/dep.f90", + "modu&\n&le split_dep\n integer :: x\nend module split_dep\n", + "split_dep", + id="free-form-keyword-split-across-lines", + ), + ], +) +def test_a_module_statement_continued_across_lines_is_located(tmp_path: Path, relative: str, text: str, unit: str): + """Sources are located by the parser's own logical lines, so a continued ``module`` statement counts.""" + definition = _write(tmp_path, relative, text) + entry = _write(tmp_path, "app.f90", f"module app\n use {unit}\nend module app\n") + + assert _resolve([entry], [tmp_path / "lib"]) == (definition.resolve(), entry) + + +@requires_gfortran +def test_a_preprocessed_fixed_form_submodule_header_continued_across_lines_is_located(tmp_path: Path): + """A ``.F`` source is preprocessed and still read in fixed form, continuation included.""" + base = _write(tmp_path, "lib/base.F", "#define UNUSED 1\n module base\n end module base\n") + child = _write( + tmp_path, "lib/child.F", "#define UNUSED 1\n submodule\n &(base) child\n end submodule child\n" + ) + leaf = _write(tmp_path, "leaf.f90", "submodule (base:child) leaf\nend submodule leaf\n") + + assert _preprocessed_resolve([leaf], [tmp_path / "lib"]) == (base.resolve(), child.resolve(), leaf) diff --git a/tests/fortran/modules/semantics/test_separate_module_procedures.py b/tests/fortran/modules/semantics/test_separate_module_procedures.py new file mode 100644 index 000000000..8ca807e1d --- /dev/null +++ b/tests/fortran/modules/semantics/test_separate_module_procedures.py @@ -0,0 +1,91 @@ +"""A separate module procedure is one of its module's procedures wherever those are read. + +An interface body with the ``module`` prefix declares a procedure the module +owns and a submodule implements. Name resolution, the project registry, and +generic inheritance read it exactly as they read a procedure the module +contains. +""" + +from __future__ import annotations + +import pytest + +from prik.parsers.fortran import parse_fortran_file, parse_fortran_project +from prik.semantics.fortran2ir import fortran_project_to_semantic_modules + +SIZES = """ +module sizes + implicit none + interface + pure module function extent(n) result(m) + integer, intent(in) :: n + integer :: m + end function extent + end interface +end module sizes + +submodule (sizes) sizes_impl + implicit none +contains + module procedure extent + m = 2 * n + end procedure extent +end submodule sizes_impl + +module user + use sizes + implicit none +contains + subroutine fill(n, x) + integer, intent(in) :: n + real(8), intent(out) :: x(extent(n)) + x = real(n, 8) + end subroutine fill +end module user +""" + + +def test_a_separate_function_a_plain_use_reaches_sizes_a_declaration(): + """``x(extent(n))`` calls ``sizes.extent``, which only an interface body declares. + + A plain ``use`` offers the names its module declares, so leaving separate + procedures out made the call an unresolved reference with no native scope. + """ + project = parse_fortran_project({"sizes.f90": SIZES}) + modules = {module.name: module for module in fortran_project_to_semantic_modules(project)} + x = modules["user"].functions[0].arguments[1] + + callables = [ + (item.name, item.native_scope, item.placement) + for axis in x.semantic_type.storage.array.expression_callables + for item in axis + ] + + assert callables == [("extent", "sizes", "module")] + assert project.procedures["sizes.extent"].name == "extent" + + +HOSTED_KIND = """ +module m + implicit none + integer, parameter :: wp = 8 + interface + module function twice(x) result(y) + real(wp), intent(in) :: x + real(wp) :: y + end function twice + end interface +end module m +""" + + +@pytest.mark.parametrize("route", ["file", "project"]) +def test_a_separate_interface_body_resolves_kinds_through_its_module(route: str): + """The body is host associated with its module, so ``real(wp)`` is ``real(8)`` however it is parsed.""" + if route == "file": + module = parse_fortran_file(HOSTED_KIND, filename="m.f90").modules[0] + else: + module = parse_fortran_project({"m.f90": HOSTED_KIND}).modules["m"] + signature = module.separate_procedures[0] + + assert (signature.arguments[0].kind, signature.result.kind) == ("8", "8") From 4d4c617cebf28e686b733a2fd0ca98398b7437d4 Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 17:04:28 +0100 Subject: [PATCH 21/47] Resolve one file and a project through one path; add the one-path rule - parse_file resolves kinds, values, and shapes with the project resolution pass on its one file, and the single-file pass and its procedure list are gone, so an interface body such as a callback prototype resolves the same way whether the file is parsed alone or within a project. - Fortran file and project conversion share one context builder over the modules a conversion reaches and one per-file converter; a file converted with sibling modules now knows their procedures and types. The six per-file/per-project context helpers are gone. - prik generate parses into one assembled project and converts it through fortran_project_to_semantic_files, instead of its own per-file route and wrapped-type helper; both parse reports assemble their inputs and share one per-file report, and the parser CLI's semantic report parses once. - AGENTS.md and CLAUDE.md state the rule: one question, one path; merge duplicates into one owner and test both entry points against each other. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 15 ++ CHANGELOG.md | 9 + CLAUDE.md | 5 + prik/cli.py | 102 ++++------ prik/parsers/fortran/cli.py | 65 +++--- prik/parsers/fortran/parser.py | 65 ++---- prik/semantics/__init__.py | 2 + prik/semantics/fortran2ir.py | 185 +++++++++--------- .../cli/pipeline/test_stage_dispatch.py | 27 ++- .../test_file_and_project_parse_agree.py | 63 ++++++ .../test_separate_module_procedures.py | 26 ++- 11 files changed, 320 insertions(+), 244 deletions(-) create mode 100644 tests/fortran/infrastructure/parsing/test_file_and_project_parse_agree.py diff --git a/AGENTS.md b/AGENTS.md index 4c5bfa4d2..df019dac9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,6 +36,21 @@ Ignore: - *.json Do not spend context window or analysis on those files unless explicitly requested. +Keep one path for one question. When two entry points answer the same +question -- one file and a project, a library route and its CLI wrapper, source +discovery and compile ordering, a source build and a contract replay -- they +must call the same owner and differ only in the inputs they pass, such as which +files or modules are in scope. Do not write a second loop, list, inventory, +regex, lexer, or conversion route that re-derives what an existing owner +decides, even as a fast path: a fast path may narrow what the owner reads, but +the owner's answer stays the only answer. Before adding a helper that +enumerates or classifies something -- a module's procedures, a file's program +units, a `use` nature, the intrinsic modules, Fortran source suffixes, a +submodule's identity -- find the existing owner and extend it. When two copies +are found, merge them into one owner instead of fixing only the copy that +failed, and prove the merge with a test that runs both entry points on one +input and compares their results. + When asked to change or move an API, import path, command, feature, or behavior, do not add or keep compatibility layers, aliases, shims, fallback paths, or legacy entrypoints unless explicitly requested. A requested change means the old behavior should be removed. When updating tests, remove obsolete tests that only assert removed/old implementation behavior does not exist. Do not preserve rejection or absence checks for API/features that were intentionally removed unless explicitly requested. Do not add tests whose purpose is only to prove that removed or nonexistent features are rejected. Test supported behavior and meaningful validation boundaries instead. For example, if `ArrayCategory` is removed, delete its tests; do not add a test asserting that `ArrayCategory` now fails. diff --git a/CHANGELOG.md b/CHANGELOG.md index 1f8c9982b..a3de9a5d0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -42,6 +42,15 @@ release tags add a leading `v` to the package version. may each have a submodule of the same name. `FortranProject.submodules` and its dependency keys use that identity, and an entity a submodule declares records it as its owner. +- A Fortran file parsed alone and within a project resolves everything it + declares the same way: `parse_fortran_file` runs the project's resolution + pass on its one file, so a kind in an interface body, such as a callback + prototype's `real(wp)`, resolves there too. A file converted with its + sibling modules resolves their procedures and types as project conversion + does. `prik generate` converts its inputs through the project route, and + `fortran_project_to_semantic_files` returns that conversion grouped by + file. Both parse reports assemble their inputs first, so a kind one input + file declares for another is resolved in the report. - Compile ordering follows `use` natures: a scope using the processor's module through `use, intrinsic` no longer waits on a project source of the same name, and `use` statements in internal procedures and `BLOCK` diff --git a/CLAUDE.md b/CLAUDE.md index 0d4b0fb15..f45100ba1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -169,6 +169,11 @@ table and markers (`fortran_end_to_end`, `real_library`, `toolchain_smoke`, `pro - Update `CHANGELOG.md` under **Unreleased** for any user/maintainer-visible change (public APIs, features, examples, CI/build workflow, benchmark methodology, documented limitations). Skip it for invisible internal cleanup. +- One question, one path: when two entry points (single file vs. project, + library vs. CLI, discovery vs. compile ordering) answer the same question, + they call the same owner and differ only in inputs. Never add a second + list, regex, or route that re-derives an existing owner's answer; merge + duplicates into one owner and test both entry points against each other. - When asked to move/change an API, import path, command, or behavior, remove the old path — do not add compatibility shims, aliases, or fallbacks unless explicitly asked to keep them. diff --git a/prik/cli.py b/prik/cli.py index 6dd764b1e..199bcae36 100644 --- a/prik/cli.py +++ b/prik/cli.py @@ -15,11 +15,11 @@ from prik.parsers.c.cli import attach_preprocessing_recipe, expand_c_paths, format_c_report, parse_c_report from prik.parsers.c.models import CParseError from prik.parsers.c.parser import CParser -from prik.parsers.fortran.cli import _format_report, _limit_items -from prik.parsers.fortran.models import FortranParseError +from prik.parsers.fortran.cli import _format_report, _limit_items, parsed_file_report +from prik.parsers.fortran.models import FortranParseError, FortranProject from prik.parsers.fortran.parser import FortranParser from prik.semantics.c2ir import c_project_to_semantic_modules, select_c_export_functions -from prik.semantics.fortran2ir import fortran_file_to_semantic_modules +from prik.semantics.fortran2ir import fortran_project_to_semantic_files from prik.preprocessing.probes.c_types import ( CStandardTypeProbeError, probe_c_standard_types_cached, @@ -381,23 +381,24 @@ def _parse_c_project( def _parse_report(paths: list[str], preprocessing: PreprocessingConfig | None = None) -> dict[str, dict]: + """Parse the named sources as one project and report each file. + + The sources are assembled together, as a build assembles them, so a kind + one file declares for another is resolved in the report too. + """ preprocessing = preprocessing or PreprocessingConfig() - out: dict[str, dict] = {} parser = FortranParser() + parsed_files = [] + recipes = {} for p in _expand_paths(paths): - code, preprocessing_recipe = _fortran_source_for_path(p, preprocessing) - parsed = parser.parse_file(code, filename=str(p)) - payload = { - "signatures": [_to_dict_no_parent(s) for s in parsed.procedures], - "types": [_to_dict_no_parent(t) for t in parsed.derived_types], - "modules": [_to_dict_no_parent(m) for m in parsed.modules], - "submodules": [_to_dict_no_parent(m) for m in parsed.submodules], - "programs": [_to_dict_no_parent(m) for m in parsed.programs], - "block_data": [_to_dict_no_parent(m) for m in parsed.block_data_units], - } - if preprocessing_recipe is not None: - payload["preprocessing_recipe"] = preprocessing_recipe - out[str(p)] = payload + code, recipes[str(p)] = _fortran_source_for_path(p, preprocessing) + parsed_files.append(parser.parse_file(code, filename=str(p))) + out: dict[str, dict] = {} + for parsed in parser._assemble_project(parsed_files).files: + payload = parsed_file_report(parsed) + if recipes[str(parsed.filename)] is not None: + payload["preprocessing_recipe"] = recipes[str(parsed.filename)] + out[str(parsed.filename)] = payload return out @@ -579,24 +580,19 @@ def _semantic_report( def _parse_fortran_source_files( paths: list[Path], preprocessing: PreprocessingConfig, -): - """Parse Fortran sources and apply the parser's shared project resolution. - - Each path is preprocessed and parsed once while retaining its path/model - pair. The completed models are then passed together to the parser's project - compile-time coordinator. For example, a kind parameter from the first file - can resolve a procedure or derived field in the second file without the CLI - owning a second resolution algorithm. The ordered ``(path, file)`` pairs - are returned for stage reporting. +) -> FortranProject: + """Parse Fortran sources in the given order into one assembled project. + + Each path is preprocessed and parsed once, and the parser's own project + assembly resolves names across them and indexes the result, so the CLI + owns no second resolution or conversion route. For example, a kind + parameter from the first file resolves a procedure in the second. """ parser = FortranParser() - parsed_files = [] - for path in paths: - code, _preprocessing_recipe = _fortran_source_for_path(path, preprocessing) - parsed_files.append((path, parser.parse_file(code, filename=str(path)))) - - parser._resolve_project_compile_time_facts([parsed for _path, parsed in parsed_files]) - return parsed_files + parsed_files = [ + parser.parse_file(_fortran_source_for_path(path, preprocessing)[0], filename=str(path)) for path in paths + ] + return parser._assemble_project(parsed_files) def _parse_c_semantic_sources(context: _SemanticPipelineContext) -> _ParsedSemanticSources: @@ -627,7 +623,7 @@ def _convert_c_semantic_sources( def _parse_fortran_semantic_sources(context: _SemanticPipelineContext) -> _ParsedSemanticSources: if not context.source_paths: - return _ParsedSemanticSources(context.source_paths, []) + return _ParsedSemanticSources(context.source_paths, FortranProject()) source_paths = context.source_paths if context.module_source_dirs: from prik.parsers.fortran.module_sources import resolve_fortran_module_sources @@ -648,17 +644,15 @@ def _convert_fortran_semantic_sources( parsed_sources: _ParsedSemanticSources, context: _SemanticPipelineContext, ) -> list[tuple[Path, list[object]]]: - parsed_files = list(parsed_sources.parsed) - if not parsed_files: + project = parsed_sources.parsed + if not project.files: return [] - wrapped_derived_types = _fortran_wrapped_derived_types(fobj for _p, fobj in parsed_files) probe_options = _fortran_probe_options( report=context.fortran_type_report, runner=context.fortran_type_probe_runner, cache_dir=context.fortran_type_probe_cache_dir, refresh=context.refresh_fortran_type_probe, ) - project = FortranParser()._assemble_project([fobj for _path, fobj in parsed_files]) compile_time_values = _fortran_compile_time_values(project, context.preprocessing, **probe_options) type_facts = _fortran_type_facts( project, @@ -666,24 +660,18 @@ def _convert_fortran_semantic_sources( compile_time_values=compile_time_values, **probe_options, ) - converted_files = [] - # A module that imports an abstract interface from another supplied file - # must resolve it here, exactly as a multi-file wrapper build does. - modules_by_file = {id(fobj): list(fobj.modules) for _p, fobj in parsed_files} - for p, fobj in parsed_files: - modules = fortran_file_to_semantic_modules( - fobj, - standalone_module_name=p.stem, + # Every file is converted as part of the project, so a name one file + # imports from another resolves exactly as in a wrapper build. + paths = {str(path): path for path in parsed_sources.source_paths} + return [ + (paths[str(parsed_file.filename)], modules) + for parsed_file, modules in fortran_project_to_semantic_files( + project, compile_time_values=compile_time_values, - wrapped_derived_types=wrapped_derived_types, assume_intent_in_scalars=context.assume_intent_in_scalars, - sibling_modules=[ - module for key, modules in modules_by_file.items() if key != id(fobj) for module in modules - ], **({"type_facts": type_facts} if type_facts is not None else {}), ) - converted_files.append((p, modules)) - return converted_files + ] _SOURCE_SEMANTIC_PIPELINES = { @@ -904,16 +892,6 @@ def _write_pyi_dependencies( path.write_text(text + "\n", encoding="utf-8") -def _fortran_wrapped_derived_types(parsed_files) -> set[tuple[str, str]]: - return { - (dtype.module.lower(), dtype.name.lower()) - for parsed in parsed_files - for module in parsed.modules - for dtype in module.derived_types - if dtype.module - } - - def _fortran_compile_time_values( parsed, preprocessing: PreprocessingConfig, diff --git a/prik/parsers/fortran/cli.py b/prik/parsers/fortran/cli.py index cff43eabc..a2f70f0d2 100644 --- a/prik/parsers/fortran/cli.py +++ b/prik/parsers/fortran/cli.py @@ -17,7 +17,7 @@ from dataclasses import asdict, fields, is_dataclass from pathlib import Path -from prik.parsers.fortran.models import FortranParseError +from prik.parsers.fortran.models import FortranFile, FortranParseError, FortranProject from prik.parsers.fortran.parser import FortranParser @@ -62,10 +62,8 @@ def _collect_extensions(path: Path) -> list[Path]: return sorted(p for p in path.rglob("*") if p.suffix.lower() in exts) -def _parse_paths(paths: list[str]) -> dict[str, dict]: - """Parse one or more files/directories into a per-file report structure.""" - out: dict[str, dict] = {} - parser = FortranParser() +def _source_paths(paths: list[str]) -> list[Path]: + """Expand files and directories into the sorted Fortran sources they name.""" expanded: list[Path] = [] for raw in paths: p = Path(raw) @@ -73,19 +71,36 @@ def _parse_paths(paths: list[str]) -> dict[str, dict]: expanded.extend(_collect_extensions(p)) else: expanded.append(p) + return sorted(set(expanded)) + + +def _parsed_project(paths: list[str]) -> FortranProject: + """Parse every named source once and assemble them as one project. + + Assembly resolves what one file declares for the others, so a report + shows the kinds and values a build of these files would use. + """ + parser = FortranParser() + return parser._assemble_project( + [parser.parse_file(p.read_text(encoding="utf-8"), filename=str(p)) for p in _source_paths(paths)] + ) - for p in sorted(set(expanded)): - code = p.read_text(encoding="utf-8") - parsed = parser.parse_file(code, filename=str(p)) - out[str(p)] = { - "signatures": [_to_dict_no_parent(s) for s in parsed.procedures], - "types": [_to_dict_no_parent(t) for t in parsed.derived_types], - "modules": [_to_dict_no_parent(m) for m in parsed.modules], - "submodules": [_to_dict_no_parent(m) for m in parsed.submodules], - "programs": [_to_dict_no_parent(m) for m in parsed.programs], - "block_data": [_to_dict_no_parent(m) for m in parsed.block_data_units], - } - return out + +def parsed_file_report(parsed: FortranFile) -> dict[str, list[dict]]: + """Return the per-file parse report every parse command prints.""" + return { + "signatures": [_to_dict_no_parent(s) for s in parsed.procedures], + "types": [_to_dict_no_parent(t) for t in parsed.derived_types], + "modules": [_to_dict_no_parent(m) for m in parsed.modules], + "submodules": [_to_dict_no_parent(m) for m in parsed.submodules], + "programs": [_to_dict_no_parent(m) for m in parsed.programs], + "block_data": [_to_dict_no_parent(m) for m in parsed.block_data_units], + } + + +def _parse_paths(paths: list[str]) -> dict[str, dict]: + """Parse one or more files/directories into a per-file report structure.""" + return {str(parsed.filename): parsed_file_report(parsed) for parsed in _parsed_project(paths).files} def _semantic_report(paths: list[str]) -> dict[str, dict]: @@ -95,22 +110,12 @@ def _semantic_report(paths: list[str]) -> dict[str, dict]: ``prik generate --pyi`` emits it, so an import names what the module it reads from declares and the report shows the contract a build would use. """ - from prik.parsers.fortran.models import FortranProject from prik.pipeline.pyi import emit_module_stubs - from prik.semantics.fortran2ir import fortran_project_to_semantic_modules + from prik.semantics.fortran2ir import fortran_project_to_semantic_files - parser = FortranParser() - files = { - fname: parser.parse_file(Path(fname).read_text(encoding="utf-8"), filename=fname) - for fname in _parse_paths(paths) - } - converted = { - module.name.casefold(): module - for module in fortran_project_to_semantic_modules(FortranProject(files=list(files.values()))) - } modules_by_file = { - fname: [converted[module.name.casefold()] for module in parsed.modules if module.name.casefold() in converted] - for fname, parsed in files.items() + str(parsed.filename): file_modules + for parsed, file_modules in fortran_project_to_semantic_files(_parsed_project(paths)) } modules = [module for file_modules in modules_by_file.values() for module in file_modules] stubs = emit_module_stubs(modules, normalize_public_names=True) if modules else {} diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index f28a0802b..e3de7cb12 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -1552,10 +1552,15 @@ def parse_file( units = self._helper_parse_file_units(top_units, root_scope, filename) self._helper_resolve_file_types(units) interfaces = self._helper_attach_file_interfaces(lines, filename, units) - self._resolve_file_compile_time_facts(units) # Stage 3: assemble the stable file model and its source metadata. - return self._helper_build_fortran_file(code, filename, encoding, units, interfaces) + parsed_file = self._helper_build_fortran_file(code, filename, encoding, units, interfaces) + + # Stage 4: resolve kinds, values, and shapes exactly as a project of + # this one file does, so one file parsed alone and within a project + # agree on everything the file itself declares. + self._resolve_project_compile_time_facts([parsed_file]) + return parsed_file def parse_project( self, @@ -2185,55 +2190,6 @@ def _merged_generic_interfaces( existing.specific_procedures.extend(interface.specific_procedures) return result - def _resolve_file_compile_time_facts(self, units: _ParsedFileUnits) -> None: - """Apply source-visible compile-time symbols within one parsed file. - - ``units`` receives the models already constructed from one source file. - Their parameter variables build a resolved symbol table; that table is - then applied to procedure kinds, module-like values/shapes, and derived - fields. For example, module parameters ``word = 4`` and - ``rk = word * 2`` resolve ``real(rk)`` to kind ``8`` without rescanning - the source text. The method mutates the supplied parser models and - returns nothing. - """ - variable_units = [*units.modules, *units.submodules, *units.programs, *units.block_data_units] - symbols = self._build_compile_time_symbols(units.modules, units.submodules) - if any( - var.kind or var.value is not None or var.symbolic_value is not None - for unit in variable_units - for var in getattr(unit, "variables", []) - ): - for unit in variable_units: - self._resolve_module_like_compile_time_facts(unit, symbols) - for procedure in self._helper_file_procedures(units): - self._resolve_procedure_compile_time_facts( - procedure, - symbols, - resolve_shapes=False, - ) - derived_types = [ - *units.derived_types, - *(derived_type for module in (*units.modules, *units.submodules) for derived_type in module.derived_types), - ] - for derived_type in derived_types: - self._resolve_derived_type_compile_time_facts(derived_type, symbols) - - @staticmethod - def _helper_file_procedures(units: _ParsedFileUnits) -> tuple[FortranProcedureSignature, ...]: - """Return file procedures in their established resolution order. - - A separate module procedure's interface body is host associated with - its module, as a contained procedure is, so its kinds resolve against - the module's parameters too. - """ - procedures = list(units.procedures) - for module in units.modules: - procedures.extend(module.procedures) - procedures.extend(module.separate_procedures) - for submodule in units.submodules: - procedures.extend(submodule.procedures) - return tuple(procedures) - def _helper_build_fortran_file( self, code: str, @@ -2365,7 +2321,12 @@ def _assemble_project(self, parsed_files: list[FortranFile]) -> FortranProject: return project def _resolve_project_compile_time_facts(self, parsed_files: list[FortranFile]) -> None: - """Apply one resolved source-symbol table across parsed project files. + """Apply one resolved source-symbol table across parsed files. + + This is the only resolution pass: :meth:`parse_file` runs it on its + one file, and project assembly runs it again across every file, so a + name declared anywhere in the project resolves while the declarations + visited are always the same ones. ``parsed_files`` contains models that were already parsed separately. The method combines their module and submodule parameters, imports, diff --git a/prik/semantics/__init__.py b/prik/semantics/__init__.py index f9959d159..4c22b2abb 100644 --- a/prik/semantics/__init__.py +++ b/prik/semantics/__init__.py @@ -11,6 +11,7 @@ collect_semantic_compile_time_requirements, fortran_file_to_semantic_modules, fortran_module_to_semantic_module, + fortran_project_to_semantic_files, fortran_project_to_semantic_modules, resolve_semantic_compile_time_values, ) @@ -42,6 +43,7 @@ "convert_pyi_to_ir", "fortran_file_to_semantic_modules", "fortran_module_to_semantic_module", + "fortran_project_to_semantic_files", "fortran_project_to_semantic_modules", "resolve_semantic_compile_time_values", "select_c_export_functions", diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index 2adc07f7c..ab6080989 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -419,59 +419,86 @@ def _visit_FortranFile( ) -> list[SemanticModule]: """Convert every module and standalone procedure group in one file. - The method first expands the wrapped-derived-type lookup from the file, - then preserves parser module order. Standalone procedures are emitted - last as the requested synthetic module when present. ``sibling_modules`` - supplies modules parsed from other files so that an abstract interface - imported across files resolves the same way it does for a project. + The file is converted exactly as a project file is, with the modules it + can reach as context: its own and ``sibling_modules``, the modules + parsed from other files alongside it. Parser module order is kept, and + standalone procedures come last as the requested synthetic module. """ - siblings = tuple(sibling_modules) - converter = self._with_additional_known_modules(module.name for module in (*siblings, *parsed_file.modules)) - converter = converter._with_additional_wrapped_types(self._wrapped_types_from_file(parsed_file)) - converter = converter._with_additional_known_procedures(self._known_procedures_from_file(parsed_file)) - converter = converter._with_additional_abstract_types(self._abstract_types_from_file(parsed_file)) - index = self._callback_module_index(siblings, parsed_file.modules) - modules = [converter.visit(module, module_index=index) for module in parsed_file.modules] - if parsed_file.procedures: - modules.append( - converter.procedures_to_semantic_module( - parsed_file.procedures, - name=standalone_module_name or self._standalone_module_name(parsed_file), - callback_interfaces=self._declared_callback_interfaces(parsed_file), - module_index=index, - ) - ) - return modules + index = self._callback_module_index(tuple(sibling_modules), parsed_file.modules) + return self._with_reachable_modules(index)._convert_parsed_file( + parsed_file, + index, + standalone_module_name=standalone_module_name, + ) def _visit_FortranProject(self, project: FortranProject) -> list[SemanticModule]: - """Convert project files in order with project-wide type and callback context. + """Convert project files in order, every file with the whole project as context.""" + return [ + module for _parsed_file, modules in self.project_files_to_semantic_modules(project) for module in modules + ] - Each file receives the known project type set plus its own declarations, - while imported callback interfaces are resolved against the project. - The returned module ordering matches the input file and parser order. + def project_files_to_semantic_modules( + self, + project: FortranProject, + ) -> list[tuple[FortranFile, list[SemanticModule]]]: + """Convert project files in order and report each file's semantic modules. + + Every file sees every project module as context, so a derived type, + procedure, or interface one file imports from another resolves. A + caller that reports per source file reads the pairing; the modules in + file order are the project's conversion. """ index = self._callback_module_index( project.modules.values(), (module for parsed_file in project.files for module in parsed_file.modules), ) - converter = self._with_additional_known_modules(module.name for module in index.values()) - converter = converter._with_additional_wrapped_types(self._wrapped_types_from_project(project)) - converter = converter._with_additional_known_procedures(self._known_procedures_from_project(project)) - converter = converter._with_additional_abstract_types(self._abstract_types_from_project(project)) - semantic_modules = [] - for parsed_file in project.files: - file_converter = converter._with_additional_wrapped_types(converter._wrapped_types_from_file(parsed_file)) - semantic_modules.extend(file_converter.visit(module, module_index=index) for module in parsed_file.modules) - if parsed_file.procedures: - semantic_modules.append( - file_converter.procedures_to_semantic_module( - parsed_file.procedures, - name=self._standalone_module_name(parsed_file), - callback_interfaces=self._declared_callback_interfaces(parsed_file), - module_index=index, - ) + converter = self._with_reachable_modules(index) + return [(parsed_file, converter._convert_parsed_file(parsed_file, index)) for parsed_file in project.files] + + def _with_reachable_modules(self, index: Mapping[str, FortranModule]) -> FortranToIRConverter: + """Return a converter that knows what every reachable module declares. + + The modules are named, and their derived types, abstract types, and + procedures become known, so a name any of them declares resolves the + same way whether one file or a whole project is being converted. + """ + modules = tuple(index.values()) + converter = self._with_additional_known_modules(module.name for module in modules) + converter = converter._with_additional_wrapped_types( + (str(dtype.module).lower(), dtype.name.lower()) + for module in modules + for dtype in module.derived_types + if dtype.module + ) + converter = converter._with_additional_known_procedures( + (module.name, procedure.name) for module in modules for procedure in self._module_procedures(module) + ) + return converter._with_additional_abstract_types( + (module.name.casefold(), dtype.name.casefold()) + for module in modules + for dtype in module.derived_types + if any(str(attribute).casefold() == "abstract" for attribute in dtype.attributes) + ) + + def _convert_parsed_file( + self, + parsed_file: FortranFile, + index: Mapping[str, FortranModule], + *, + standalone_module_name: str | None = None, + ) -> list[SemanticModule]: + """Convert one file's modules, then its standalone procedures as one synthetic module.""" + modules = [self.visit(module, module_index=index) for module in parsed_file.modules] + if parsed_file.procedures: + modules.append( + self.procedures_to_semantic_module( + parsed_file.procedures, + name=standalone_module_name or self._standalone_module_name(parsed_file), + callback_interfaces=self._declared_callback_interfaces(parsed_file), + module_index=index, ) - return semantic_modules + ) + return modules # Variable and argument visitors @@ -2405,44 +2432,6 @@ def _with_additional_abstract_types( converter._known_modules = self._known_modules | {module for module, _name in merged} return converter - @staticmethod - def _wrapped_types_from_file(parsed_file: FortranFile) -> set[tuple[str, str]]: - """Collect module-qualified derived types declared by one parsed file.""" - return { - (dtype.module.lower(), dtype.name.lower()) - for module in parsed_file.modules - for dtype in module.derived_types - if dtype.module - } - - @staticmethod - def _abstract_types_from_file(parsed_file: FortranFile) -> set[tuple[str, str]]: - """Collect module-qualified abstract types declared by one parsed file.""" - return { - (module.name.casefold(), dtype.name.casefold()) - for module in parsed_file.modules - for dtype in module.derived_types - if any(str(attribute).casefold() == "abstract" for attribute in dtype.attributes) - } - - @classmethod - def _known_procedures_from_file(cls, parsed_file: FortranFile) -> set[tuple[str, str]]: - """Collect module-qualified procedures declared by one parsed file.""" - return { - (module.name, procedure.name) - for module in parsed_file.modules - for procedure in cls._module_procedures(module) - } - - @classmethod - def _known_procedures_from_project(cls, project: FortranProject) -> set[tuple[str, str]]: - """Collect module-qualified procedures known to one parsed project.""" - return { - (module.name, procedure.name) - for module in project.modules.values() - for procedure in cls._module_procedures(module) - } - @classmethod def _module_procedures(cls, module: FortranModule) -> tuple[FortranProcedureSignature, ...]: """Return every procedure one module declares, which is what a ``use`` of it can call. @@ -2484,20 +2473,6 @@ def _module_interface_procedures(cls, module: FortranModule) -> list[FortranProc procedures.append(procedure) return procedures - @staticmethod - def _wrapped_types_from_project(project: FortranProject) -> set[tuple[str, str]]: - """Collect project-known module-qualified derived types for import resolution.""" - return {(dtype.module.lower(), dtype.name.lower()) for dtype in project.derived_types.values() if dtype.module} - - @staticmethod - def _abstract_types_from_project(project: FortranProject) -> set[tuple[str, str]]: - """Collect project-known module-qualified abstract derived types.""" - return { - (dtype.module.casefold(), dtype.name.casefold()) - for dtype in project.derived_types.values() - if dtype.module and any(str(attribute).casefold() == "abstract" for attribute in dtype.attributes) - } - @staticmethod def _module_derived_type_context( module: FortranModule, @@ -4955,6 +4930,26 @@ def fortran_project_to_semantic_modules( ).visit(project) +def fortran_project_to_semantic_files( + project: FortranProject, + *, + compile_time_values: dict[str, int | str] | None = None, + type_facts: dict[tuple[str, str | None], dict[str, object]] | None = None, + assume_intent_in_scalars: bool = False, +) -> list[tuple[FortranFile, list[SemanticModule]]]: + """Convert a parsed Fortran project and pair each file with its semantic modules. + + This is :func:`fortran_project_to_semantic_modules` for a caller that + reports per source file, such as a generated contract for each input: the + conversion, and every file's view of the rest of the project, is the same. + """ + return _converter_for( + compile_time_values, + type_facts=type_facts, + assume_intent_in_scalars=assume_intent_in_scalars, + ).project_files_to_semantic_modules(project) + + if __name__ == "__main__": from prik.parsers.fortran.models import FortranArgument, FortranModule, FortranProcedureSignature diff --git a/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py b/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py index 148848a09..589b13fa4 100644 --- a/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py +++ b/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py @@ -232,8 +232,8 @@ def test_single_file_cli_resolves_direct_intrinsic_kind_rename_before_probing(tm encoding="utf-8", ) - parsed_files = prik_cli._parse_fortran_source_files([source], PreprocessingConfig()) - parsed = parsed_files[0][1] + project = prik_cli._parse_fortran_source_files([source], PreprocessingConfig()) + parsed = project.files[0] module = parsed.modules[0] assert module.variables[0].kind == "real64" @@ -265,11 +265,11 @@ def test_cli_cross_file_resolution_reaches_imported_derived_field_kinds(tmp_path encoding="utf-8", ) - parsed_files = prik_cli._parse_fortran_source_files( + project = prik_cli._parse_fortran_source_files( [precision, records], PreprocessingConfig(), ) - record_file = next(parsed for path, parsed in parsed_files if path == records) + record_file = next(parsed for parsed in project.files if parsed.filename == str(records)) assert record_file.modules[0].derived_types[0].fields[0].kind == "8" @@ -838,3 +838,22 @@ def test_doctor_cmake_reports_the_discovery_facts_a_build_would_use(): assert report[label] for group in ("cmake.root", "cmake.module"): assert report[f"entry point {group}"] + + +def test_parse_reports_resolve_kinds_one_input_file_declares_for_another(tmp_path: Path): + """Both parse commands assemble their inputs as a build does, so a kind from another file resolves.""" + kinds = tmp_path / "kinds.f90" + kinds.write_text("module kinds\n integer, parameter :: wp = 8\nend module kinds\n", encoding="utf-8") + user = tmp_path / "user.f90" + user.write_text( + "module user\n use kinds, only: wp\ncontains\n subroutine run(x)\n real(wp) :: x\n" + " end subroutine run\nend module user\n", + encoding="utf-8", + ) + + for report in ( + prik_cli._parse_report([str(kinds), str(user)]), + fortran_parser_cli._parse_paths([str(kinds), str(user)]), + ): + procedure = report[str(user)]["modules"][0]["procedures"][0] + assert procedure["arguments"][0]["kind"] == "8" diff --git a/tests/fortran/infrastructure/parsing/test_file_and_project_parse_agree.py b/tests/fortran/infrastructure/parsing/test_file_and_project_parse_agree.py new file mode 100644 index 000000000..ce5c51b9a --- /dev/null +++ b/tests/fortran/infrastructure/parsing/test_file_and_project_parse_agree.py @@ -0,0 +1,63 @@ +"""One file parsed alone and within a project resolves what it declares the same way. + +``parse_file`` resolves kinds, values, and shapes with the same pass project +assembly runs, visiting the same declarations, so parsing a self-contained +source alone never leaves a name unresolved that a project parse resolves. +""" + +from __future__ import annotations + +from prik.parsers.fortran import parse_fortran_file, parse_fortran_project + +SELF_CONTAINED = """ +module m + implicit none + integer, parameter :: wp = 8 + interface + subroutine callback(x) + import :: wp + real(wp) :: x + end subroutine callback + module function twice(x) result(y) + real(wp), intent(in) :: x + real(wp) :: y + end function twice + end interface +contains + subroutine run(y) + real(wp) :: y + end subroutine run +end module m + +submodule (m) impl + implicit none + interface + subroutine hook(z) + import :: wp + real(wp) :: z + end subroutine hook + end interface +end submodule impl +""" + + +def _kinds(parsed_file) -> dict[str, str | None]: + """Return every argument kind the file declares, keyed by owner and name.""" + owners = (*parsed_file.modules, *parsed_file.submodules) + signatures = [ + *(procedure for owner in owners for procedure in owner.procedures), + *(procedure for owner in owners for interface in owner.interfaces for procedure in interface.procedures), + ] + return { + f"{signature.module}.{signature.name}.{argument.name}": argument.kind + for signature in signatures + for argument in signature.arguments + } + + +def test_a_file_resolves_every_kind_it_declares_as_a_project_of_it_does(): + alone = _kinds(parse_fortran_file(SELF_CONTAINED, filename="m.f90")) + in_project = _kinds(parse_fortran_project({"m.f90": SELF_CONTAINED}).files[0]) + + assert alone == in_project + assert set(alone.values()) == {"8"} diff --git a/tests/fortran/modules/semantics/test_separate_module_procedures.py b/tests/fortran/modules/semantics/test_separate_module_procedures.py index 8ca807e1d..1c1faab65 100644 --- a/tests/fortran/modules/semantics/test_separate_module_procedures.py +++ b/tests/fortran/modules/semantics/test_separate_module_procedures.py @@ -11,7 +11,7 @@ import pytest from prik.parsers.fortran import parse_fortran_file, parse_fortran_project -from prik.semantics.fortran2ir import fortran_project_to_semantic_modules +from prik.semantics.fortran2ir import fortran_file_to_semantic_modules, fortran_project_to_semantic_modules SIZES = """ module sizes @@ -89,3 +89,27 @@ def test_a_separate_interface_body_resolves_kinds_through_its_module(route: str) signature = module.separate_procedures[0] assert (signature.arguments[0].kind, signature.result.kind) == ("8", "8") + + +def test_a_file_converted_with_its_siblings_resolves_their_procedures_as_a_project_does(): + """The context sibling modules supply is the same one project conversion builds. + + A single file converted with its siblings knew their module names but not + what they declared, so a sibling's procedure a plain ``use`` reaches stayed + unresolved there while the project route resolved it. + """ + sizes_source, _header, user_body = SIZES.partition("\nmodule user\n") + project = parse_fortran_project({"sizes.f90": sizes_source, "user.f90": "module user\n" + user_body}) + sizes_file, user_file = project.files + alone = fortran_file_to_semantic_modules(user_file, sibling_modules=sizes_file.modules) + within = [module for module in fortran_project_to_semantic_modules(project) if module.name == "user"] + + def callables(modules): + x = modules[0].functions[0].arguments[1] + return [ + (item.native_scope, item.placement) + for axis in x.semantic_type.storage.array.expression_callables + for item in axis + ] + + assert callables(alone) == callables(within) == [("sizes", "module")] From ce5c5b1b695218161bf2031ee52962cde64050b0 Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 17:50:28 +0100 Subject: [PATCH 22/47] Give every duplicated source question one owner - prik.pipeline.sources is the one route from Fortran sources to semantic IR: builds and prik generate both read through read_fortran_source, parse one dependency-ordered project, measure compile-time values and type storage with FortranTypeProbe, convert per file, and select exports there. The CLI's own parse, probe, and conversion copies and the build's reader and probe wrappers are gone. - prik.parsers.c.sources.parse_c_source is the one C parse for builds, prik generate, and parse reports; the build keeps its fail-closed check on top. parse_c_report takes a PreprocessingConfig. - prik.preprocessing.languages owns source suffixes, a Fortran file's source form, and input expansion and validation, replacing six suffix sets, two source-form decisions, and five path expanders. .fpp is fixed-form Fortran everywhere, FortranFile.format reports "fixed" or "free", and commands keep inputs in the order named. - c_project_to_semantic_module and its helpers are removed; the C .pyi fixtures record each project's implementation file converted per file. - A parity test runs prik generate and a build on the same sources and compares their modules. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 17 + docs/developer/packages/parsers.md | 9 +- docs/developer/packages/pipeline.md | 2 + docs/developer/packages/preprocessing.md | 4 +- prik/cli.py | 388 ++++-------------- prik/parsers/c/cli.py | 92 +---- prik/parsers/c/sources.py | 80 ++++ prik/parsers/fortran/cli.py | 24 +- prik/parsers/fortran/lexer.py | 8 +- prik/parsers/fortran/module_sources.py | 10 +- prik/parsers/fortran/parser.py | 32 +- prik/parsers/fortran/utils.py | 32 +- prik/pipeline/build.py | 291 ++----------- prik/pipeline/sources.py | 201 +++++++++ prik/preprocessing/__init__.py | 4 + prik/preprocessing/languages.py | 132 ++++++ prik/preprocessing/source.py | 37 +- prik/semantics/__init__.py | 2 - prik/semantics/c2ir.py | 106 ----- tests/c/_support/fixture_outputs.py | 18 +- .../semantics/test_types_and_constants.py | 4 - .../semantics/test_c_enum_semantics.py | 3 +- .../pyi/general/modern_math_physics.pyi | 2 +- .../pipeline/test_c_cli_output_contract.py | 10 - .../cli/pipeline/test_c_cli_skeleton.py | 23 +- .../cli/pipeline/test_c_cli_stage_dispatch.py | 61 +-- .../test_projects_and_diagnostics.py | 41 -- tests/fortran/_support/wrapper_build.py | 7 +- .../policy/test_allocatable_result_policy.py | 6 +- .../callbacks/policy/test_callback_policy.py | 6 +- .../policy/test_function_result_policy.py | 6 +- .../policy/test_generic_policy.py | 6 +- .../cli/pipeline/test_output_contract.py | 29 +- .../cli/pipeline/test_stage_dispatch.py | 35 +- .../assumed_shape_and_derived_args.json | 2 +- .../fixtures/general/basic_subroutine.json | 2 +- .../general/compile_time_all_exprs.json | 2 +- .../general/compile_time_shape_exprs.json | 2 +- .../fixtures/general/derived_type.json | 2 +- .../general/derived_types_and_methods.json | 2 +- .../fixtures/general/f77_subroutine.json | 2 +- .../fixtures/general/modern_pyi_example.json | 2 +- .../fixtures/general/module_vars_use.json | 2 +- .../general/procedures_and_functions.json | 2 +- .../scope_name_reuse_combinations.json | 2 +- .../parsing/test_error_handling.py | 2 +- .../parsing/test_public_entrypoints.py | 5 +- ...source_form_and_diagnostics_regressions.py | 3 - .../pipeline/test_source_route_parity.py | 72 ++++ .../policy/test_wrapper_policy.py | 6 +- .../preprocessing/test_source_languages.py | 43 ++ .../policy/test_optional_policy.py | 6 +- .../policy/test_raw_address_policy.py | 6 +- .../codegen/test_string_input_lowering.py | 5 +- .../policy/test_string_wrapper_policy.py | 8 +- .../policy/test_subroutine_output_policy.py | 6 +- 56 files changed, 845 insertions(+), 1067 deletions(-) create mode 100644 prik/parsers/c/sources.py create mode 100644 prik/pipeline/sources.py create mode 100644 prik/preprocessing/languages.py create mode 100644 tests/fortran/infrastructure/pipeline/test_source_route_parity.py create mode 100644 tests/fortran/infrastructure/preprocessing/test_source_languages.py diff --git a/CHANGELOG.md b/CHANGELOG.md index a3de9a5d0..7795615b6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -51,6 +51,23 @@ release tags add a leading `v` to the package version. `fortran_project_to_semantic_files` returns that conversion grouped by file. Both parse reports assemble their inputs first, so a kind one input file declares for another is resolved in the report. +- Wrapper builds and `prik generate` turn Fortran sources into semantic IR + through one route (`prik.pipeline.sources`): both read sources through + `prik.preprocessing.read_fortran_source`, parse them as one + dependency-ordered project, measure compile-time values and type storage, + and apply `--export-symbols` the same way, so a generated contract + describes what a build of the same sources wraps. C builds, `prik generate`, + and C parse reports parse each input through + `prik.parsers.c.sources.parse_c_source`; `parse_c_report` takes a + `PreprocessingConfig`. +- Source suffixes, a Fortran file's source form, and input expansion have one + owner, `prik.preprocessing.languages`. `.fpp` is a fixed-form Fortran source + everywhere, `FortranFile.format` reports the form the lexer read (`"fixed"` + or `"free"`), and every command keeps its inputs in the order named, with + each directory's sources in sorted order. +- `c_project_to_semantic_module` is removed; convert C projects per file with + `c_project_to_semantic_modules`. The C `.pyi` fixtures record each fixture + project's implementation file. - Compile ordering follows `use` natures: a scope using the processor's module through `use, intrinsic` no longer waits on a project source of the same name, and `use` statements in internal procedures and `BLOCK` diff --git a/docs/developer/packages/parsers.md b/docs/developer/packages/parsers.md index a55f7afe1..88dd9c3d4 100644 --- a/docs/developer/packages/parsers.md +++ b/docs/developer/packages/parsers.md @@ -73,6 +73,7 @@ prik/parsers/ ├── lexer.py ├── models.py ├── parser.py + ├── sources.py └── type_resolver.py ``` @@ -83,14 +84,14 @@ prik/parsers/ | [`prik/parsers/__init__.py`](../../../prik/parsers/__init__.py) | Names the language frontend namespaces; it does not flatten their APIs. | The parser-frontend layout changes. | | [`prik/parsers/fortran/__init__.py`](../../../prik/parsers/fortran/__init__.py) | Re-exports `FortranParser`, `parse_fortran_file()`, `parse_fortran_project()`, parser models, and `FortranParseError`. | The supported Fortran-parser import surface changes. | | [`prik/parsers/fortran/__main__.py`](../../../prik/parsers/fortran/__main__.py) | Runs the Fortran parser CLI for `python3 -m prik.parsers.fortran`. | Module-launch behavior changes. | -| [`prik/parsers/fortran/utils.py`](../../../prik/parsers/fortran/utils.py) | `detect_source_form()` chooses fixed or free form; `split_csv()` separates only top-level Fortran comma lists. | Source-form detection or grammar-neutral list splitting changes. | +| [`prik/parsers/fortran/utils.py`](../../../prik/parsers/fortran/utils.py) | `split_csv()` separates only top-level Fortran comma lists. | Grammar-neutral list splitting changes. | | [`prik/parsers/fortran/lexer.py`](../../../prik/parsers/fortran/lexer.py) | `preprocess_lines()` produces logical lines with original coordinates; `strip_comment()` preserves string literals and OpenMP directives. | Comment handling, continuation folding, or location preservation changes. | | [`prik/parsers/fortran/models.py`](../../../prik/parsers/fortran/models.py) | Passive source-fact records: `FortranFile`, `FortranProject`, units, declarations, shapes, and `FortranParseError`. | A parser result, source fact, or diagnostic representation changes. | | [`prik/parsers/fortran/scope.py`](../../../prik/parsers/fortran/scope.py) | `ScopeUses` aggregates a scope's `use` statements and is the authority for rename semantics, accessible local names, and candidate routes. Semantic consumers decide only what those routes mean for their entity category. | `use` association or scope dependency interpretation changes. | | [`prik/parsers/fortran/type_resolver.py`](../../../prik/parsers/fortran/type_resolver.py) | `extract_kind_from_type_spec()` preserves intrinsic kind and character syntax after declaration parsing. | Parser-level type-spec spelling extraction changes. | | [`prik/parsers/fortran/parser.py`](../../../prik/parsers/fortran/parser.py) | `FortranParser`, `parse_fortran_file()`, and `parse_fortran_project()` build file and project models. | Grammar, source-unit structure, declarations, parser diagnostics, or project assembly changes. | | [`prik/parsers/fortran/cli.py`](../../../prik/parsers/fortran/cli.py) | `main()` formats parser reports and diagnostics. Its `--semantics` and `--pyi` options explicitly invoke later stages; `--pyi` emits every inspected module through `emit_module_stubs()`, so it shows the contract `prik generate --pyi` writes. | Parser CLI arguments, report layout, or diagnostic presentation changes. | -| [`prik/parsers/c/`](../../../prik/parsers/c/README.md) | `parse_c_file()` and `parse_c_project()` build `CFile`/`CProject` records; the local lexer, models, resolver, and CLI preserve C declarations, project facts, diagnostics, and report output. | C tokenization, declarations, type resolution, project assembly, or parser reports change. | +| [`prik/parsers/c/`](../../../prik/parsers/c/README.md) | `parse_c_file()` and `parse_c_project()` build `CFile`/`CProject` records; `sources.parse_c_source()` is the one route that preprocesses and parses a C path for builds, `prik generate`, and parse reports; the local lexer, models, resolver, and CLI preserve C declarations, project facts, diagnostics, and report output. | C tokenization, declarations, type resolution, project assembly, or parser reports change. | | [`prik/parsers/pyi/__init__.py`](../../../prik/parsers/pyi/__init__.py) | Re-exports `parse_pyi_text()` and `parse_pyi_file()`. | The supported raw-`.pyi` parser import surface changes. | | [`prik/parsers/pyi/parser.py`](../../../prik/parsers/pyi/parser.py) | Parses text or a file into `ast.Module` with no contract interpretation. | Raw Python syntax input, file reading, or parse diagnostics change. | @@ -129,8 +130,8 @@ the visitor for the unit you are changing before its private helper group. ### `lexer.py`, `utils.py`, and `type_resolver.py`: syntax preservation -`detect_source_form()` uses a known filename suffix first, then a small -fixed-form continuation-column heuristic. `preprocess_lines()` removes comments +`preprocess_lines()` reads the source form from +`prik.preprocessing.languages.fortran_source_form()`, removes comments without touching quoted strings, folds fixed- and free-form continuations, and returns `(logical_line, original_line_number, original_source_line)` tuples. Those tuples are the location contract used by parser diagnostics. diff --git a/docs/developer/packages/pipeline.md b/docs/developer/packages/pipeline.md index 7dd813ef3..12ad36d9c 100644 --- a/docs/developer/packages/pipeline.md +++ b/docs/developer/packages/pipeline.md @@ -48,6 +48,7 @@ datatype facts without creating a wrapper. ```text prik/pipeline/ ├── pyi.py +├── sources.py ├── type_mapping_report.py ├── wrapper.py └── build.py @@ -58,6 +59,7 @@ prik/pipeline/ | Module | Main entrypoints and contents | Change it when | | --- | --- | --- | | [`prik/pipeline/pyi.py`](../../../prik/pipeline/pyi.py) | `pyi_*_to_semantic_module()` loads text, files, or path sets into semantic modules. `emit_module_stubs()` completes copied modules and renders `.pyi` stubs. | Contract loading, external-type reconciliation, per-operation cache behavior, or stub output. | +| [`prik/pipeline/sources.py`](../../../prik/pipeline/sources.py) | `fortran_sources_to_semantic_modules()` reads, parses, measures, converts, and selects Fortran sources for both a build and `prik generate`; `discover_fortran_sources()` runs module source discovery. | How Fortran sources become semantic modules, for every route at once. | | [`prik/pipeline/type_mapping_report.py`](../../../prik/pipeline/type_mapping_report.py) | Converts compiler probe facts through semantic conversion and backend dtype projection into a measured report record, then renders it as Markdown. | Datatype-report content or its cross-stage evidence. | | [`prik/pipeline/wrapper.py`](../../../prik/pipeline/wrapper.py) | `WrapperGenerator.generate()` freezes and validates a `ModulePlan`, delegates backend generation and printing, and returns an in-memory `GeneratedWrapper`. | Plan-to-rendered-wrapper orchestration. | | [`prik/pipeline/build.py`](../../../prik/pipeline/build.py) | `build_fortran_extension()`, `build_c_extension()`, `build_pyi_extension()`, and `build_pyi_extension_from_manifest()` write artifacts, prepare native inputs, compile/link, and return `WrapperBuildResult`. `NativeBuildPlan` records those native inputs. | Public build behavior, artifact layout, build modes, manifests, scheduling, linking, or extension import. | diff --git a/docs/developer/packages/preprocessing.md b/docs/developer/packages/preprocessing.md index 88f60de75..80488377a 100644 --- a/docs/developer/packages/preprocessing.md +++ b/docs/developer/packages/preprocessing.md @@ -52,6 +52,7 @@ plus its recipe; it is not a stable semantic scalar or NumPy dtype. ```text prik/preprocessing/ ├── __init__.py +├── languages.py ├── source.py ├── fortran.py ├── c.py C source preparation @@ -65,7 +66,8 @@ prik/preprocessing/ | Module | Public boundary and result | Change it when | | --- | --- | --- | | [`prik/preprocessing/__init__.py`](../../../prik/preprocessing/__init__.py) | Re-exports the supported shared source-preparation records, adapters, and entrypoints. | The shared preprocessing import surface changes. | -| [`prik/preprocessing/source.py`](../../../prik/preprocessing/source.py) | `preprocess_source()` is the compiler-backed route. `PreprocessingConfig` selects its command; `PreprocessResult` returns expanded text, provenance, and diagnostics. | Compiler adapters, invocations, recipes, mappings, dependencies, macros, or diagnostics change. | +| [`prik/preprocessing/languages.py`](../../../prik/preprocessing/languages.py) | The one owner of source suffixes, a Fortran file's fixed or free form, and the sources a list of files and directories names (`expand_source_paths()`, and `validated_source_paths()` for a build). | A supported suffix, source-form rule, or input expansion changes. | +| [`prik/preprocessing/source.py`](../../../prik/preprocessing/source.py) | `preprocess_source()` is the compiler-backed route, and `read_fortran_source()` is how every route reads a Fortran path for parsing. `PreprocessingConfig` selects its command; `PreprocessResult` returns expanded text, provenance, and diagnostics. | Compiler adapters, invocations, recipes, mappings, dependencies, macros, or diagnostics change. | | [`prik/preprocessing/fortran.py`](../../../prik/preprocessing/fortran.py) | `expand_native_fortran_includes()` turns remaining textual `INCLUDE` statements into parser input while retaining mappings and diagnostics. | Native Fortran include discovery or expansion changes. | | [`prik/preprocessing/c.py`](../../../prik/preprocessing/c.py) | Collects C directive and include metadata and normalizes prepared C source without interpreting declarations. | C directive provenance, include metadata, or parser preparation changes. | | [`prik/preprocessing/probes/fortran_types.py`](../../../prik/preprocessing/probes/fortran_types.py) | `evaluate_fortran_type_requirements()` and `evaluate_fortran_type_facts()` turn semantic requirements into cached compiler measurements; `FortranTypeProbeReport` retains values and recipe. | Fortran fact generation, validation, cache identity, or semantic-facing probe results change. | diff --git a/prik/cli.py b/prik/cli.py index 199bcae36..e22ded863 100644 --- a/prik/cli.py +++ b/prik/cli.py @@ -12,14 +12,27 @@ from pathlib import Path from prik import __version__ -from prik.parsers.c.cli import attach_preprocessing_recipe, expand_c_paths, format_c_report, parse_c_report +from prik.parsers.c.cli import expand_c_paths, format_c_report, parse_c_report +from prik.parsers.c.sources import parse_c_source from prik.parsers.c.models import CParseError from prik.parsers.c.parser import CParser from prik.parsers.fortran.cli import _format_report, _limit_items, parsed_file_report -from prik.parsers.fortran.models import FortranParseError, FortranProject +from prik.parsers.fortran.models import FortranParseError from prik.parsers.fortran.parser import FortranParser from prik.semantics.c2ir import c_project_to_semantic_modules, select_c_export_functions -from prik.semantics.fortran2ir import fortran_project_to_semantic_files +from prik.pipeline.sources import FortranTypeProbe, discover_fortran_sources, fortran_sources_to_semantic_modules +from prik.preprocessing import read_fortran_source +from prik.preprocessing.languages import ( + C_IMPLEMENTATION_SUFFIXES, + C_SOURCE_SUFFIXES, + FORTRAN_SOURCE_SUFFIXES, + MissingSourceError, + SourceInputError, + expand_source_paths, + is_c_source, + is_fortran_source, + validated_source_paths, +) from prik.preprocessing.probes.c_types import ( CStandardTypeProbeError, probe_c_standard_types_cached, @@ -37,16 +50,13 @@ from prik.preprocessing import ( PreprocessingConfig, PreprocessingError, - run_compiler_preprocessor_with_recipe, validate_macro_name, ) _TRUE_VALUES = {"1", "true", "yes", "on"} -_FORTRAN_SOURCE_SUFFIXES = {".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08"} -_C_SOURCE_SUFFIXES = {".c", ".h", ".i"} _SOURCE_SUFFIXES_BY_LANGUAGE = { - "fortran": _FORTRAN_SOURCE_SUFFIXES, - "c": _C_SOURCE_SUFFIXES, + "fortran": FORTRAN_SOURCE_SUFFIXES, + "c": C_SOURCE_SUFFIXES, } _HELP_DIVIDER = "------------------------------ EXAMPLES ------------------------------" _TOP_LEVEL_USAGE = ( @@ -236,34 +246,13 @@ def _to_dict_no_parent(obj): return obj -def _collect_extensions(path: Path) -> list[Path]: - return sorted(p for p in path.rglob("*") if p.suffix.lower() in _FORTRAN_SOURCE_SUFFIXES) - - -def _collect_pyi_extensions(path: Path) -> list[Path]: - return sorted(p for p in path.rglob("*.pyi") if p.is_file()) - - def _expand_paths(paths: list[str]) -> list[Path]: - expanded: list[Path] = [] - for raw in paths: - p = Path(raw) - if p.is_dir(): - expanded.extend(_collect_extensions(p)) - else: - expanded.append(p) - return list(dict.fromkeys(expanded)) + return list(expand_source_paths(paths, FORTRAN_SOURCE_SUFFIXES)) def _expand_pyi_paths(paths: list[str]) -> list[Path]: - expanded: list[Path] = [] - for raw in paths: - p = Path(raw) - if p.is_dir(): - expanded.extend(_collect_pyi_extensions(p)) - elif p.suffix.lower() == ".pyi": - expanded.append(p) - return sorted(set(expanded)) + """Return the semantic contracts the named files and directories hold.""" + return [path for path in expand_source_paths(paths, (".pyi",)) if path.suffix.casefold() == ".pyi"] def _resolve_language( @@ -299,10 +288,9 @@ def language_for_suffix(suffix: str) -> str | None: "pass --language fortran or --language c. Use --help for examples." ) - suffix = path.suffix.lower() - if suffix in _C_SOURCE_SUFFIXES: + if is_c_source(path): parser.error(f"C input {path} requires explicit --language c. Use --help for examples.") - if suffix not in _FORTRAN_SOURCE_SUFFIXES and suffix != ".pyi": + if not is_fortran_source(path) and path.suffix.casefold() != ".pyi": parser.error( f"Cannot determine the input language for {path}; " "pass --language fortran or --language c. Use --help for examples." @@ -310,73 +298,12 @@ def language_for_suffix(suffix: str) -> str | None: return "fortran" -def _fortran_source_for_path( - path: Path, - preprocessing: PreprocessingConfig, -) -> tuple[str, dict[str, object] | None]: - if preprocessing.uses_compiler: - source, recipe = run_compiler_preprocessor_with_recipe( - path, - language="fortran", - config=preprocessing, - ) - return source, recipe.to_dict() - return ( - path.read_text(encoding="utf-8"), - preprocessing.fortran_internal_recipe(path), - ) - - -def _c_source_loader(preprocessing: PreprocessingConfig): - if not preprocessing.uses_compiler: - return None - - def load(path: Path) -> tuple[str, dict[str, object]]: - source, recipe = run_compiler_preprocessor_with_recipe( - path, - language="c", - config=preprocessing, - ) - return source, recipe.to_dict() - - return load - - -def _c_parser_preprocessing_mode(preprocessing: PreprocessingConfig) -> str: - return "compiler" if preprocessing.uses_compiler else "raw" - - -def _parse_c_path( - parser: CParser, - path: Path, - preprocessing: PreprocessingConfig, -): - source_loader = _c_source_loader(preprocessing) - if source_loader is None: - return parser.parse_file( - path, - filename=str(path), - include_dirs=preprocessing.include_dirs, - preprocessing=_c_parser_preprocessing_mode(preprocessing), - ) - - source, preprocessing_recipe = source_loader(path) - parsed = parser.parse_file( - source, - filename=str(path), - include_dirs=preprocessing.include_dirs, - preprocessing=_c_parser_preprocessing_mode(preprocessing), - ) - attach_preprocessing_recipe(parsed, preprocessing_recipe) - return parsed - - def _parse_c_project( paths: list[str], preprocessing: PreprocessingConfig, ): parser = CParser() - parsed_files = {str(path): _parse_c_path(parser, path, preprocessing) for path in expand_c_paths(paths)} + parsed_files = {str(path): parse_c_source(path, preprocessing, parser=parser) for path in expand_c_paths(paths)} return parser._assemble_project(parsed_files) @@ -391,8 +318,9 @@ def _parse_report(paths: list[str], preprocessing: PreprocessingConfig | None = parsed_files = [] recipes = {} for p in _expand_paths(paths): - code, recipes[str(p)] = _fortran_source_for_path(p, preprocessing) - parsed_files.append(parser.parse_file(code, filename=str(p))) + text = read_fortran_source(p, preprocessing) + recipes[str(p)] = text.recipe + parsed_files.append(parser.parse_file(text.source, filename=str(p))) out: dict[str, dict] = {} for parsed in parser._assemble_project(parsed_files).files: payload = parsed_file_report(parsed) @@ -430,25 +358,6 @@ def _c_standard_type_report( return probe_c_standard_types_cached(preprocessing).to_dict() -def _fortran_probe_options( - *, - report: FortranTypeProbeReport | None, - runner: list[str] | None, - cache_dir: str | None, - refresh: bool, -) -> dict[str, object]: - options: dict[str, object] = {} - if report is not None: - options["report"] = report - if runner is not None: - options["runner"] = runner - if cache_dir is not None: - options["cache_dir"] = cache_dir - if refresh: - options["refresh"] = True - return options - - @dataclass(frozen=True) class _SemanticPipelineContext: paths: list[str] @@ -476,12 +385,6 @@ class _ConvertedSemanticSources: available_modules: tuple[object, ...] -@dataclass(frozen=True) -class _SourceSemanticPipeline: - parser: Callable[[_SemanticPipelineContext], _ParsedSemanticSources] - converter_to_ir: Callable[[_ParsedSemanticSources, _SemanticPipelineContext], list[tuple[Path, list[object]]]] - - def _source_paths_for_semantic_pipeline( paths: list[str], *, @@ -521,26 +424,32 @@ def _converted_semantic_files( export_symbols=export_symbols, module_source_dirs=module_source_dirs, ) - pipeline = _SOURCE_SEMANTIC_PIPELINES[language] - parsed = pipeline.parser(context) - converted_files = pipeline.converter_to_ir(parsed, context) - available_modules = tuple(module for _path, modules in converted_files for module in modules) - if language != "fortran" or export_symbols is None: - return _ConvertedSemanticSources(tuple(converted_files), available_modules) - - from prik.semantics.fortran_exports import select_fortran_export_symbols - - selection = select_fortran_export_symbols(available_modules, export_symbols) - selected_by_source = { - id(source): selected - for source, selected in zip(selection.primary_sources, selection.primary_modules, strict=True) - } - selected_files = [] - for path, modules in converted_files: - selected_modules = [selected_by_source[id(module)] for module in modules if id(module) in selected_by_source] - if selected_modules: - selected_files.append((path, selected_modules)) - return _ConvertedSemanticSources(tuple(selected_files), selection.available_modules) + if language == "fortran": + source_paths = context.source_paths + if module_source_dirs: + source_paths = discover_fortran_sources(source_paths, module_source_dirs, preprocessing) + sources = fortran_sources_to_semantic_modules( + source_paths, + preprocessing, + probe=FortranTypeProbe( + preprocessing, + report=fortran_type_report, + runner=fortran_type_probe_runner, + cache_dir=fortran_type_probe_cache_dir, + refresh=refresh_fortran_type_probe, + ), + assume_intent_in_scalars=assume_intent_in_scalars, + export_symbols=export_symbols, + ) + return _ConvertedSemanticSources( + tuple((path, list(modules)) for path, modules in sources.files), + sources.modules, + ) + converted_files = _convert_c_semantic_sources(_parse_c_semantic_sources(context), context) + return _ConvertedSemanticSources( + tuple(converted_files), + tuple(module for _path, modules in converted_files for module in modules), + ) def _semantic_report( @@ -577,24 +486,6 @@ def _semantic_report( ) -def _parse_fortran_source_files( - paths: list[Path], - preprocessing: PreprocessingConfig, -) -> FortranProject: - """Parse Fortran sources in the given order into one assembled project. - - Each path is preprocessed and parsed once, and the parser's own project - assembly resolves names across them and indexes the result, so the CLI - owns no second resolution or conversion route. For example, a kind - parameter from the first file resolves a procedure in the second. - """ - parser = FortranParser() - parsed_files = [ - parser.parse_file(_fortran_source_for_path(path, preprocessing)[0], filename=str(path)) for path in paths - ] - return parser._assemble_project(parsed_files) - - def _parse_c_semantic_sources(context: _SemanticPipelineContext) -> _ParsedSemanticSources: if not context.source_paths: return _ParsedSemanticSources(context.source_paths, None) @@ -621,71 +512,6 @@ def _convert_c_semantic_sources( return [(path, modules_by_source[str(path)]) for path in parsed_sources.source_paths] -def _parse_fortran_semantic_sources(context: _SemanticPipelineContext) -> _ParsedSemanticSources: - if not context.source_paths: - return _ParsedSemanticSources(context.source_paths, FortranProject()) - source_paths = context.source_paths - if context.module_source_dirs: - from prik.parsers.fortran.module_sources import resolve_fortran_module_sources - - source_paths = resolve_fortran_module_sources( - source_paths, - context.module_source_dirs, - lambda path: _fortran_source_for_path(path, context.preprocessing)[0], - command_line_macros=context.preprocessing.defines_command_line_macros, - ) - return _ParsedSemanticSources( - source_paths, - _parse_fortran_source_files(list(source_paths), context.preprocessing), - ) - - -def _convert_fortran_semantic_sources( - parsed_sources: _ParsedSemanticSources, - context: _SemanticPipelineContext, -) -> list[tuple[Path, list[object]]]: - project = parsed_sources.parsed - if not project.files: - return [] - probe_options = _fortran_probe_options( - report=context.fortran_type_report, - runner=context.fortran_type_probe_runner, - cache_dir=context.fortran_type_probe_cache_dir, - refresh=context.refresh_fortran_type_probe, - ) - compile_time_values = _fortran_compile_time_values(project, context.preprocessing, **probe_options) - type_facts = _fortran_type_facts( - project, - context.preprocessing, - compile_time_values=compile_time_values, - **probe_options, - ) - # Every file is converted as part of the project, so a name one file - # imports from another resolves exactly as in a wrapper build. - paths = {str(path): path for path in parsed_sources.source_paths} - return [ - (paths[str(parsed_file.filename)], modules) - for parsed_file, modules in fortran_project_to_semantic_files( - project, - compile_time_values=compile_time_values, - assume_intent_in_scalars=context.assume_intent_in_scalars, - **({"type_facts": type_facts} if type_facts is not None else {}), - ) - ] - - -_SOURCE_SEMANTIC_PIPELINES = { - "c": _SourceSemanticPipeline( - parser=_parse_c_semantic_sources, - converter_to_ir=_convert_c_semantic_sources, - ), - "fortran": _SourceSemanticPipeline( - parser=_parse_fortran_semantic_sources, - converter_to_ir=_convert_fortran_semantic_sources, - ), -} - - def _semantic_payload_for_converted_files( converted_files, *, @@ -892,61 +718,6 @@ def _write_pyi_dependencies( path.write_text(text + "\n", encoding="utf-8") -def _fortran_compile_time_values( - parsed, - preprocessing: PreprocessingConfig, - *, - report: FortranTypeProbeReport | None = None, - runner: list[str] | None = None, - cache_dir: str | None = None, - refresh: bool = False, -) -> dict[str, int] | None: - """Evaluate compiler-dependent Fortran values when a compiler is configured.""" - if report is None and ( - not isinstance(preprocessing, PreprocessingConfig) - or not preprocessing.uses_compiler - or not preprocessing.compiler - ): - return None - - from prik.semantics.fortran2ir import collect_semantic_compile_time_requirements - from prik.preprocessing.probes.fortran_types import evaluate_fortran_type_requirements - - requirements = collect_semantic_compile_time_requirements(parsed) - if not requirements: - return None - probe_options = _fortran_probe_options(report=report, runner=runner, cache_dir=cache_dir, refresh=refresh) - return evaluate_fortran_type_requirements(preprocessing, requirements, **probe_options) - - -def _fortran_type_facts( - parsed, - preprocessing: PreprocessingConfig, - *, - compile_time_values: dict[str, int] | None = None, - report: FortranTypeProbeReport | None = None, - runner: list[str] | None = None, - cache_dir: str | None = None, - refresh: bool = False, -) -> dict[tuple[str, str | None], dict[str, object]] | None: - """Measure compiler-dependent storage for intrinsic types used by one source.""" - if report is None and ( - not isinstance(preprocessing, PreprocessingConfig) - or not preprocessing.uses_compiler - or not preprocessing.compiler - ): - return None - - from prik.semantics.fortran2ir import collect_fortran_type_storage_requirements - from prik.preprocessing.probes.fortran_types import evaluate_fortran_type_facts - - requirements = collect_fortran_type_storage_requirements(parsed, compile_time_values=compile_time_values) - if not requirements: - return None - probe_options = _fortran_probe_options(report=report, runner=runner, cache_dir=cache_dir, refresh=refresh) - return evaluate_fortran_type_facts(preprocessing, requirements, **probe_options) - - def _build_preprocessing_config(args: argparse.Namespace, parser: argparse.ArgumentParser) -> PreprocessingConfig: """Build and validate the shared preprocessing CLI configuration.""" defines = list(args.defines or []) @@ -986,14 +757,6 @@ def _build_preprocessing_config(args: argparse.Namespace, parser: argparse.Argum return config -def _path_is_fortran_source(path: str) -> bool: - return Path(path).suffix.lower() in _FORTRAN_SOURCE_SUFFIXES - - -def _path_is_c_source(path: str) -> bool: - return Path(path).suffix.lower() == ".c" - - def _path_is_pyi_contract(path: str) -> bool: return Path(path).suffix.lower() == ".pyi" @@ -1121,23 +884,27 @@ def _validate_manifest_wrapper_options(args: argparse.Namespace, parser: argpars def _validate_source_wrapper_options(args: argparse.Namespace, parser: argparse.ArgumentParser) -> None: - language = args.language - label = "C" if language == "c" else "Fortran" - source_check = _path_is_c_source if language == "c" else _path_is_fortran_source - if not args.paths: - parser.error( - f"A wrapper build expects at least one {label} source, source directory, or semantic .pyi contract" - ) - unsupported = [path for path in args.paths if not Path(path).is_dir() and not source_check(path)] - if unsupported: - parser.error( - f"A wrapper build expects recognized {label} source suffixes or one semantic .pyi contract; " - f"unsupported input: {unsupported[0]}" - ) - collect = (lambda path: sorted(path.rglob("*.c"))) if language == "c" else _collect_extensions - empty_directories = [path for path in args.paths if Path(path).is_dir() and not collect(Path(path))] - if empty_directories: - parser.error(f"A wrapper build found no recognized {label} sources under: {empty_directories[0]}") + label = "C" if args.language == "c" else "Fortran" + suffixes = C_IMPLEMENTATION_SUFFIXES if args.language == "c" else FORTRAN_SOURCE_SUFFIXES + # The build validates its inputs the same way; this reports misuse early. + # A missing file is left to the build, which reports it with its path. + try: + validated_source_paths(args.paths, suffixes, label=label) + except MissingSourceError: + pass + except SourceInputError as error: + if error.reason == "empty": + parser.error( + f"A wrapper build expects at least one {label} source, source directory, or semantic .pyi contract" + ) + if error.reason == "no_sources": + parser.error(f"A wrapper build found no recognized {label} sources under: {error.path}") + if error.reason == "unsupported": + parser.error( + f"A wrapper build expects recognized {label} source suffixes or one semantic .pyi contract; " + f"unsupported input: {error.path}" + ) + parser.error(str(error)) if not getattr(args, "no_compile_input_sources", False): return if not getattr(args, "external_native_implementation", False) and not ( @@ -1532,12 +1299,7 @@ def _parse_stage_report(args: argparse.Namespace, preprocessing: PreprocessingCo if not args.parse: return None if args.language == "c": - return parse_c_report( - args.paths, - include_dirs=preprocessing.include_dirs, - preprocessing=_c_parser_preprocessing_mode(preprocessing), - source_loader=_c_source_loader(preprocessing), - ) + return parse_c_report(args.paths, preprocessing) return _parse_report(args.paths, preprocessing) diff --git a/prik/parsers/c/cli.py b/prik/parsers/c/cli.py index 99d90aac8..0aa60d19a 100644 --- a/prik/parsers/c/cli.py +++ b/prik/parsers/c/cli.py @@ -4,15 +4,15 @@ import json import os import sys -from collections.abc import Callable, Sequence from pathlib import Path -from typing import Any -from prik.parsers.c.models import CFile, CMacro, CParseError, CSourceLocation, c_model_to_dict +from prik.preprocessing.languages import C_SOURCE_SUFFIXES, expand_source_paths +from prik.parsers.c.models import CFile, CParseError, c_model_to_dict from prik.parsers.c.parser import CParser +from prik.parsers.c.sources import parse_c_source +from prik.preprocessing import PreprocessingConfig -_C_SOURCE_SUFFIXES = {".c", ".h", ".i"} _TRUE_VALUES = {"1", "true", "yes", "on"} @@ -24,89 +24,21 @@ def _diagnostic_color_enabled(*, disabled: bool) -> bool: return not disabled and "NO_COLOR" not in os.environ -def _collect_c_extensions(path: Path) -> list[Path]: - return sorted(p for p in path.rglob("*") if p.is_file() and p.suffix.lower() in _C_SOURCE_SUFFIXES) - - def expand_c_paths(paths: list[str]) -> list[Path]: - expanded: list[Path] = [] - for raw in paths: - p = Path(raw) - if p.is_dir(): - expanded.extend(_collect_c_extensions(p)) - else: - expanded.append(p) - return sorted(set(expanded)) - - -def attach_preprocessing_recipe(parsed: CFile, preprocessing_recipe: dict[str, Any] | None) -> None: - """Attach compiler recipe side-channel facts to a parsed C file.""" - - parsed.preprocessing_recipe = preprocessing_recipe - if not preprocessing_recipe: - return - existing = { - ( - macro.name, - macro.source_location.filename if macro.source_location else None, - macro.source_location.line if macro.source_location else None, - ) - for macro in parsed.macros - } - for item in preprocessing_recipe.get("macros") or []: - if not isinstance(item, dict): - continue - name = item.get("name") - if not isinstance(name, str) or not name: - continue - location = CSourceLocation( - filename=item.get("path") if isinstance(item.get("path"), str) else None, - line=item.get("line") if isinstance(item.get("line"), int) else None, - column=1, - ) - key = (name, location.filename, location.line) - if key in existing: - continue - parsed.macros.append( - CMacro( - name=name, - value=item.get("value") if isinstance(item.get("value"), str) else None, - function_like=bool(item.get("function_like")), - source_location=location, - ) - ) - existing.add(key) + """Return the C inputs the named files and directories hold, in the caller's order.""" + return list(expand_source_paths(paths, C_SOURCE_SUFFIXES)) def parse_c_report( paths: list[str], - *, - include_dirs: Sequence[str | Path] | None = None, - preprocessing: str = "raw", - source_loader: Callable[[Path], str | tuple[str, dict[str, Any] | None]] | None = None, + preprocessing: PreprocessingConfig | None = None, ) -> dict[str, dict]: - out: dict[str, dict] = {} + """Parse each named C input and return its report, keyed by path.""" + preprocessing = preprocessing or PreprocessingConfig() parser = CParser() - for p in expand_c_paths(paths): - if source_loader is None: - parsed = parser.parse_file( - p, - filename=str(p), - include_dirs=include_dirs, - preprocessing=preprocessing, - ) - else: - loaded = source_loader(p) - source, preprocessing_recipe = loaded if isinstance(loaded, tuple) else (loaded, None) - parsed = parser.parse_file( - source, - filename=str(p), - include_dirs=include_dirs, - preprocessing=preprocessing, - ) - attach_preprocessing_recipe(parsed, preprocessing_recipe) - out[str(p)] = c_model_to_dict(parsed) - return out + return { + str(path): c_model_to_dict(parse_c_source(path, preprocessing, parser=parser)) for path in expand_c_paths(paths) + } def _label_items(items: list[object], *, keys: tuple[str, ...], fallback: str) -> list[str]: diff --git a/prik/parsers/c/sources.py b/prik/parsers/c/sources.py new file mode 100644 index 000000000..f3256da03 --- /dev/null +++ b/prik/parsers/c/sources.py @@ -0,0 +1,80 @@ +"""Read one C input the way every C route parses it. + +A build, ``prik generate``, and a parse report all parse a C path here, so the +same preprocessing, include directories, and recipe provenance apply to each. +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from prik.parsers.c.models import CFile, CMacro, CSourceLocation +from prik.parsers.c.parser import CParser +from prik.preprocessing import PreprocessingConfig, run_compiler_preprocessor_with_recipe + + +def parse_c_source(path: Path | str, preprocessing: PreprocessingConfig, *, parser: CParser | None = None) -> CFile: + """Parse one C input under ``preprocessing``. + + Compiler preprocessing expands the input first and attaches its recipe, + which include exposure reads to keep an included header's declarations + out of the input's own. Otherwise the text is parsed as written. + """ + parser = parser or CParser() + source_path = Path(path) + if not preprocessing.uses_compiler: + return parser.parse_file( + source_path, + filename=str(source_path), + include_dirs=preprocessing.include_dirs, + preprocessing="raw", + ) + source, recipe = run_compiler_preprocessor_with_recipe(source_path, language="c", config=preprocessing) + parsed = parser.parse_file( + source, + filename=str(source_path), + include_dirs=preprocessing.include_dirs, + preprocessing="compiler", + ) + attach_preprocessing_recipe(parsed, recipe.to_dict()) + return parsed + + +def attach_preprocessing_recipe(parsed: CFile, preprocessing_recipe: dict[str, Any] | None) -> None: + """Attach compiler recipe side-channel facts to a parsed C file.""" + + parsed.preprocessing_recipe = preprocessing_recipe + if not preprocessing_recipe: + return + existing = { + ( + macro.name, + macro.source_location.filename if macro.source_location else None, + macro.source_location.line if macro.source_location else None, + ) + for macro in parsed.macros + } + for item in preprocessing_recipe.get("macros") or []: + if not isinstance(item, dict): + continue + name = item.get("name") + if not isinstance(name, str) or not name: + continue + location = CSourceLocation( + filename=item.get("path") if isinstance(item.get("path"), str) else None, + line=item.get("line") if isinstance(item.get("line"), int) else None, + column=1, + ) + key = (name, location.filename, location.line) + if key in existing: + continue + parsed.macros.append( + CMacro( + name=name, + value=item.get("value") if isinstance(item.get("value"), str) else None, + function_like=bool(item.get("function_like")), + source_location=location, + ) + ) + existing.add(key) diff --git a/prik/parsers/fortran/cli.py b/prik/parsers/fortran/cli.py index a2f70f0d2..4a1184738 100644 --- a/prik/parsers/fortran/cli.py +++ b/prik/parsers/fortran/cli.py @@ -19,6 +19,7 @@ from prik.parsers.fortran.models import FortranFile, FortranParseError, FortranProject from prik.parsers.fortran.parser import FortranParser +from prik.preprocessing.languages import FORTRAN_SOURCE_SUFFIXES, expand_source_paths _TRUE_VALUES = {"1", "true", "yes", "on"} @@ -56,24 +57,6 @@ def _to_dict_no_parent(obj): return obj -def _collect_extensions(path: Path) -> list[Path]: - """Recursively collect Fortran source files under a directory.""" - exts = {".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08"} - return sorted(p for p in path.rglob("*") if p.suffix.lower() in exts) - - -def _source_paths(paths: list[str]) -> list[Path]: - """Expand files and directories into the sorted Fortran sources they name.""" - expanded: list[Path] = [] - for raw in paths: - p = Path(raw) - if p.is_dir(): - expanded.extend(_collect_extensions(p)) - else: - expanded.append(p) - return sorted(set(expanded)) - - def _parsed_project(paths: list[str]) -> FortranProject: """Parse every named source once and assemble them as one project. @@ -82,7 +65,10 @@ def _parsed_project(paths: list[str]) -> FortranProject: """ parser = FortranParser() return parser._assemble_project( - [parser.parse_file(p.read_text(encoding="utf-8"), filename=str(p)) for p in _source_paths(paths)] + [ + parser.parse_file(p.read_text(encoding="utf-8"), filename=str(p)) + for p in expand_source_paths(paths, FORTRAN_SOURCE_SUFFIXES) + ] ) diff --git a/prik/parsers/fortran/lexer.py b/prik/parsers/fortran/lexer.py index eefd3c7e0..798164e29 100644 --- a/prik/parsers/fortran/lexer.py +++ b/prik/parsers/fortran/lexer.py @@ -1,6 +1,6 @@ from __future__ import annotations -from prik.parsers.fortran.utils import detect_source_form +from prik.preprocessing.languages import fortran_source_form def strip_comment(line: str, form: str) -> str: @@ -46,7 +46,7 @@ def preprocess_lines(code: str, filename: str | None = None) -> list[tuple[str, This is the lexer/preprocessing stage used by all parsing entrypoints. It performs: - - **Source-form detection** (fixed vs free) via `detect_source_form`. + - **Source-form detection** (fixed vs free) via `fortran_source_form`. - **Comment stripping** (including fixed-form comment-line rules). - **Continuation folding**: - fixed-form: continuation in column 6 (1-based) @@ -56,7 +56,7 @@ def preprocess_lines(code: str, filename: str | None = None) -> list[tuple[str, tuples so downstream parsers can raise `FortranParseError` with accurate location context even after folding. """ - form = detect_source_form(code, filename) + form = fortran_source_form(code, filename) raw_lines = code.splitlines() raw = [ (strip_comment(raw_line.rstrip("\n"), form), i + 1, raw_line.rstrip("\n")) @@ -130,6 +130,6 @@ def preprocess_lines(code: str, filename: str | None = None) -> list[tuple[str, real, intent(in) :: offset end subroutine shift """ - print("Detected source form:", detect_source_form(example_source, "shift.f90")) + print("Detected source form:", fortran_source_form(example_source, "shift.f90")) for logical_line, source_line, _original_line in preprocess_lines(example_source, "shift.f90"): print(f"line {source_line}: {logical_line}") diff --git a/prik/parsers/fortran/module_sources.py b/prik/parsers/fortran/module_sources.py index 0f7ceb5f5..68d34c070 100644 --- a/prik/parsers/fortran/module_sources.py +++ b/prik/parsers/fortran/module_sources.py @@ -30,9 +30,8 @@ from prik.parsers.fortran.models import FortranParseError from prik.parsers.fortran.parser import FortranParser from prik.parsers.fortran.scope import file_defined_units, file_unit_requirements +from prik.preprocessing.languages import FORTRAN_SOURCE_SUFFIXES, expand_source_paths -# Suffixes a Fortran compiler accepts as free- or fixed-form source. -_FORTRAN_SOURCE_SUFFIXES = frozenset({".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08", ".fpp"}) # Raw text a preprocessor can change: a directive, or a Fortran ``include``, # which PRIK's preprocessing expands as well. _PREPROCESSED_TEXT = re.compile(r"^[ \t]*(?:#|include[ \t]*['\"])", re.IGNORECASE | re.MULTILINE) @@ -249,9 +248,4 @@ def _raise_ambiguous(unit: str, user: Path, definers: Sequence[Path]) -> None: def _searched_files(search_dirs: Iterable[Path]) -> tuple[Path, ...]: """Return every Fortran source under ``search_dirs`` once, in a stable order.""" - files: dict[Path, None] = {} - for directory in search_dirs: - for path in sorted(Path(directory).rglob("*")): - if path.suffix.casefold() in _FORTRAN_SOURCE_SUFFIXES and path.is_file(): - files.setdefault(path.resolve(), None) - return tuple(files) + return tuple(dict.fromkeys(path.resolve() for path in expand_source_paths(search_dirs, FORTRAN_SOURCE_SUFFIXES))) diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index e3de7cb12..a1e85a7e1 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -54,6 +54,7 @@ ) from prik.parsers.fortran.type_resolver import extract_character_selector, extract_kind_from_type_spec from prik.parsers.fortran.utils import split_csv +from prik.preprocessing.languages import FORTRAN_SOURCE_SUFFIXES, expand_source_paths, fortran_source_form _PARSER_ARCHITECTURE_GUIDE = """ Parser architecture quick guide @@ -243,7 +244,6 @@ def _main_source_line_numbers(source: str) -> dict[int, int]: _INTRINSIC_COMPILE_TIME_MODULES = frozenset({"iso_c_binding", "iso_fortran_env"}) -_FORTRAN_SOURCE_SUFFIXES = (".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08") _PreprocessedLine = tuple[str, int | None, str | None] @@ -1581,7 +1581,7 @@ def parse_project( if isinstance(files, dict): parsed_files = self._parse_named_project_sources(files, encoding=encoding) elif isinstance(files, str | Path): - paths = self._discover_project_paths(Path(files)) + paths = list(expand_source_paths([files], FORTRAN_SOURCE_SUFFIXES)) parsed_files = self._parse_project_files(paths, encoding=encoding) parsed_files = self._order_project_files(parsed_files) else: @@ -2203,7 +2203,7 @@ def _helper_build_fortran_file( filename=filename, source=code, encoding=encoding, - format=self._source_form(filename), + format=fortran_source_form(code, filename), modules=units.modules, submodules=units.submodules, programs=units.programs, @@ -2222,20 +2222,6 @@ def _helper_build_fortran_file( ) return parsed_file - @staticmethod - def _discover_project_paths( - root: Path, - extensions: tuple[str, ...] = _FORTRAN_SOURCE_SUFFIXES, - ) -> list[Path]: - """Return supported Fortran paths below one project directory. - - Discovery only identifies files; it does not read or parse them. The - paths are sorted so unrelated files have deterministic order before - dependency analysis. For example, a directory containing ``b.f90``, - ``a.f90``, and ``notes.txt`` produces ``[a.f90, b.f90]``. - """ - return sorted(path for path in root.rglob("*") if path.suffix.lower() in extensions) - def _parse_project_files( self, paths: Sequence[str | Path], @@ -6036,18 +6022,6 @@ def _expect_single_parse_result( code="PARSE_AMBIGUOUS_ENTRYPOINT", ) - @staticmethod - def _source_form(filename: str | None) -> str: - """Infer fixed, modern, or unknown source form from a filename suffix.""" - if not filename: - return "unknown" - ext = Path(filename).suffix.lower() - if ext in {".f", ".for", ".ftn", ".f77"}: - return "f77" - if ext in {".f90", ".f95", ".f03", ".f08"}: - return "modern" - return "unknown" - @staticmethod def _infer_implicit_base_type(symbol_name: str) -> str: """Apply the default Fortran I-N integer implicit typing rule.""" diff --git a/prik/parsers/fortran/utils.py b/prik/parsers/fortran/utils.py index be64ba266..7c52a6498 100644 --- a/prik/parsers/fortran/utils.py +++ b/prik/parsers/fortran/utils.py @@ -1,9 +1,9 @@ """Grammar-neutral lexical helpers for the Fortran parser. -``detect_source_form`` selects fixed or free form before lexical preparation. ``split_csv`` separates only top-level comma lists after a caller has chosen -the relevant Fortran construct. Neither function builds parser models or -interprets declarations; ``lexer.py`` and ``parser.py`` own those next steps. +the relevant Fortran construct. It builds no parser models and interprets no +declarations; ``lexer.py`` and ``parser.py`` own those next steps. Which +source form a file uses is ``prik.preprocessing.languages.fortran_source_form``. """ from __future__ import annotations @@ -11,32 +11,6 @@ from prik.utilities.declaration_expressions import split_top_level_expression -def detect_source_form(code: str, filename: str | None = None) -> str: - """Detect whether a source looks like fixed-form or free-form Fortran. - - Resolution order: - - If ``filename`` is provided, decide based on its suffix (most reliable). - - Otherwise, use a small heuristic on the first ~20 lines to detect the - fixed-form continuation column (column 6). - - Returns - ------- - str - ``"fixed"`` or ``"free"``. - """ - if filename: - lowered = filename.lower() - if lowered.endswith((".f", ".for", ".ftn", ".f77")): - return "fixed" - if lowered.endswith((".f90", ".f95", ".f03", ".f08")): - return "free" - - for line in code.splitlines()[:20]: - if len(line) >= 6 and line[:5].strip() == "" and line[5:6].strip(): - return "fixed" - return "free" - - def split_csv(text: str | None) -> list[str]: """Split a comma-separated list while respecting nested expression syntax. diff --git a/prik/pipeline/build.py b/prik/pipeline/build.py index 3502dbd60..8f890e3a6 100644 --- a/prik/pipeline/build.py +++ b/prik/pipeline/build.py @@ -36,26 +36,27 @@ from prik.compiler.compilers import Compiler, get_condaless_search_path from prik.compiler.native_support import BINDING_SUPPORT_IMPORT, install_native_support from prik.naming.generated_files import stub_identifier -from prik.parsers.c import parse_c_file -from prik.parsers.c.cli import attach_preprocessing_recipe -from prik.parsers.fortran.parser import parse_fortran_project -from prik.parsers.fortran.module_sources import resolve_fortran_module_sources +from prik.parsers.c.sources import parse_c_source from prik.parsers.fortran.scope import file_defined_units, file_unit_requirements from prik.preprocessing.probes.fortran_types import ( - evaluate_fortran_type_facts, - evaluate_fortran_type_requirements, resolve_fortran_logical_storage_types, ) -from prik.preprocessing import PreprocessingConfig, preprocess_source -from prik.preprocessing.source import run_compiler_preprocessor_with_recipe +from prik.preprocessing import PreprocessingConfig from prik.preprocessing.probes.c_types import probe_c_standard_types +from prik.pipeline.sources import ( + FortranTypeProbe, + discover_fortran_sources, + fortran_sources_to_semantic_modules, + semantic_dependency_paths, +) +from prik.preprocessing.languages import ( + C_IMPLEMENTATION_SUFFIXES, + FORTRAN_SOURCE_SUFFIXES, + is_fortran_source, + validated_source_paths, +) from prik.pipeline.pyi import emit_module_stubs from prik.pipeline.wrapper import GeneratedSource, GeneratedWrapper, WrapperGenerator -from prik.semantics.fortran2ir import ( - collect_fortran_type_storage_requirements, - collect_semantic_compile_time_requirements, - fortran_project_to_semantic_modules, -) from prik.semantics.c2ir import CToIRConverter, c_file_to_semantic_modules, select_c_export_functions from prik.semantics.metadata import EXPLICIT_C_EXPORT_METADATA from prik.semantics.models import ( @@ -87,20 +88,11 @@ _DEFAULT_BUILD_DIR_NAME = "__prik__" _BUILD_MANIFEST_NAME = "prik-build.json" _BUILD_MANIFEST_SCHEMA_VERSION = 5 -_FORTRAN_SOURCE_SUFFIXES = {".f", ".f03", ".f08", ".f77", ".f90", ".f95", ".for", ".ftn"} -_C_SOURCE_SUFFIXES = {".c"} _NATIVE_PATH_LINK_KINDS = frozenset({"object", "archive", "shared_library"}) _NATIVE_LINK_KINDS = frozenset({*_NATIVE_PATH_LINK_KINDS, "named_library", "linker_argument"}) _GENERATED_WRAPPER_SOURCE_LANGUAGES = { - ".c": "c", - ".f": "fortran", - ".f03": "fortran", - ".f08": "fortran", - ".f77": "fortran", - ".f90": "fortran", - ".f95": "fortran", - ".for": "fortran", - ".ftn": "fortran", + **dict.fromkeys(C_IMPLEMENTATION_SUFFIXES, "c"), + **dict.fromkeys(FORTRAN_SOURCE_SUFFIXES, "fortran"), } _GENERATED_WRAPPER_NATIVE_SUPPORT_IMPORTS = { "binding_support": ("binding_support/prik_binding",), @@ -598,48 +590,17 @@ def _parse_c_wrapper_source(path: Path, preprocessing: PreprocessingConfig): not model is raised here instead of silently disappearing from the public API of a build that promises to fail closed. """ - if preprocessing.uses_compiler: - source, recipe = run_compiler_preprocessor_with_recipe(path, language="c", config=preprocessing) - parsed = parse_c_file( - source, - filename=str(path), - include_dirs=preprocessing.include_dirs, - preprocessing="compiler", - ) - # Include exposure needs the recipe: without it every declaration - # expanded from a system header would be published as public API. - attach_preprocessing_recipe(parsed, recipe.to_dict()) - else: - parsed = parse_c_file(path, filename=str(path), include_dirs=preprocessing.include_dirs) + parsed = parse_c_source(path, preprocessing) _reject_unmodeled_c_declarations(parsed, path) return parsed -def _semantic_dependency_paths( - root: Path, - included_files: Iterable[object], -) -> tuple[Path, ...]: - """Return existing root and transitive preprocessing inputs in stable order.""" - dependencies = [root.resolve(strict=False)] - for item in included_files: - raw_path = item.get("path") if isinstance(item, Mapping) else getattr(item, "path", None) - if not isinstance(raw_path, str | Path) or str(raw_path).startswith("<"): - continue - path = Path(raw_path) - if not path.is_absolute(): - path = root.parent / path - path = path.resolve(strict=False) - if path.is_file(): - dependencies.append(path) - return _unique_paths(dependencies) - - def _c_wrapper_semantic_dependencies(parsed_sources, source_paths: tuple[Path, ...]) -> tuple[Path, ...]: """Collect source and included-header dependencies recorded by C preprocessing.""" dependencies = [] for parsed, source_path in zip(parsed_sources, source_paths, strict=True): recipe = parsed.preprocessing_recipe or {} - dependencies.extend(_semantic_dependency_paths(source_path, recipe.get("included_files") or ())) + dependencies.extend(semantic_dependency_paths(source_path, recipe.get("included_files") or ())) return _unique_paths(dependencies) @@ -668,29 +629,6 @@ def _reject_unmodeled_c_declarations(parsed, path: Path) -> None: ) -def _fortran_source_for_pipeline(path: Path, preprocessing: PreprocessingConfig) -> str: - """Read one source path in the form required by the Fortran parser. - - Compiler-backed preprocessing produces the expanded source text; other - modes read UTF-8 text directly. The helper reads ``path`` but does not - change the source file or preprocessing configuration. - """ - if preprocessing.uses_compiler: - return preprocess_source(path, language="fortran", config=preprocessing).source - return path.read_text(encoding="utf-8") - - -def _fortran_source_and_dependencies( - path: Path, - preprocessing: PreprocessingConfig, -) -> tuple[str, tuple[Path, ...]]: - """Preprocess one wrapper source and retain every interface dependency.""" - if preprocessing.uses_compiler: - result = preprocess_source(path, language="fortran", config=preprocessing) - return result.source, _semantic_dependency_paths(path, result.included_files) - return path.read_text(encoding="utf-8"), _semantic_dependency_paths(path, ()) - - def _compiler_flags(flags: Iterable[str] | None) -> tuple[str, ...]: """Normalize optional caller compiler flags into an immutable tuple. @@ -1703,59 +1641,6 @@ def _project_compile_batches( # Source and semantic-contract inputs -def _source_paths(sources: str | Path | Iterable[str | Path]) -> tuple[Path, ...]: - """Validate and expand wrapper source inputs into a unique ordered tuple. - - A file must have a supported Fortran suffix; a directory is recursively - expanded in sorted order. The result preserves the caller's input order - while removing repeated paths. Missing files, empty directories, and - unsupported suffixes raise clear input errors before parsing begins. - """ - inputs = (Path(sources),) if isinstance(sources, str | Path) else tuple(Path(source) for source in sources) - if not inputs: - raise ValueError("wrapper build requires at least one Fortran source file or directory") - - paths: list[Path] = [] - for path in inputs: - if path.is_dir(): - discovered = sorted( - candidate - for candidate in path.rglob("*") - if candidate.is_file() and candidate.suffix.lower() in _FORTRAN_SOURCE_SUFFIXES - ) - if not discovered: - raise ValueError(f"No recognized Fortran sources found under: {path}") - paths.extend(discovered) - continue - if not path.is_file(): - raise FileNotFoundError(f"Fortran source not found: {path}") - if path.suffix.lower() not in _FORTRAN_SOURCE_SUFFIXES: - raise ValueError(f"Unrecognized Fortran source suffix: {path}") - paths.append(path) - return tuple(dict.fromkeys(paths)) - - -def _c_source_paths(sources: str | Path | Iterable[str | Path]) -> tuple[Path, ...]: - """Validate and expand explicit C implementation sources in stable order.""" - inputs = (Path(sources),) if isinstance(sources, str | Path) else tuple(Path(source) for source in sources) - if not inputs: - raise ValueError("wrapper build requires at least one C source file or directory") - paths: list[Path] = [] - for path in inputs: - if path.is_dir(): - discovered = sorted(candidate for candidate in path.rglob("*.c") if candidate.is_file()) - if not discovered: - raise ValueError(f"No recognized C sources found under: {path}") - paths.extend(discovered) - continue - if not path.is_file(): - raise FileNotFoundError(f"C source not found: {path}") - if path.suffix.lower() not in _C_SOURCE_SUFFIXES: - raise ValueError(f"Unrecognized C source suffix: {path}") - paths.append(path) - return tuple(dict.fromkeys(paths)) - - def _wrapper_output_paths(output_dir: str | Path | None) -> tuple[Path, Path]: """Return build and extension directories owned by one wrapper invocation.""" if output_dir is not None: @@ -3206,7 +3091,7 @@ def _command_output(command: tuple[str, ...]) -> str | None: def _command_source(command: tuple[str, ...]) -> str | None: """Return the first recognized native source argument in a compiler command.""" for part in command: - if Path(part).suffix.lower() in _FORTRAN_SOURCE_SUFFIXES | _C_SOURCE_SUFFIXES: + if is_fortran_source(part) or Path(part).suffix.casefold() in C_IMPLEMENTATION_SUFFIXES: return part return None @@ -3227,7 +3112,7 @@ def _command_language( return None if source_languages is not None and (language := source_languages.get(str(Path(source)))) is not None: return language - return "fortran" if Path(source).suffix.lower() in _FORTRAN_SOURCE_SUFFIXES else "c" + return "fortran" if is_fortran_source(source) else "c" def _absolute_command_path(path: str | Path, working_directory: Path) -> Path: @@ -3385,11 +3270,6 @@ def _write_build_makefile( # Fortran type probing -def _can_probe_fortran_types(preprocessing: PreprocessingConfig) -> bool: - """Return whether the preprocessing configuration can invoke a compiler probe.""" - return preprocessing.uses_compiler and bool(preprocessing.compiler) - - def _type_probe_preprocessing( preprocessing: PreprocessingConfig, native_fortran_flags: Iterable[str], @@ -3404,67 +3284,6 @@ def _type_probe_preprocessing( ) -def _wrap_compile_time_values( - parsed, - preprocessing: PreprocessingConfig, - *, - report=None, - runner: list[str] | None = None, - cache_dir: str | Path | None = None, - refresh: bool = False, -) -> dict[str, int] | None: - """Measure only the compile-time values required by a parsed source project. - - Returns ``None`` when no report/probe is possible or no values are needed. - Otherwise it delegates the parsed requirements and optional probe controls - to the type evaluator, which may read or refresh its cache. - """ - if report is None and not _can_probe_fortran_types(preprocessing): - return None - requirements = collect_semantic_compile_time_requirements(parsed) - if not requirements: - return None - return evaluate_fortran_type_requirements( - preprocessing, - requirements, - report=report, - runner=runner, - cache_dir=cache_dir, - refresh=refresh, - ) - - -def _wrap_type_facts( - parsed, - preprocessing: PreprocessingConfig, - *, - compile_time_values: dict[str, int] | None, - report=None, - runner: list[str] | None = None, - cache_dir: str | Path | None = None, - refresh: bool = False, -) -> dict[tuple[str, str | None], dict[str, object]] | None: - """Measure native type-storage facts required by a parsed source project. - - Uses prior ``compile_time_values`` to derive requirements. Returns - ``None`` when probing is unavailable or unnecessary; otherwise delegates to - the type-fact evaluator, which may execute or reuse a compiler probe. - """ - if report is None and not _can_probe_fortran_types(preprocessing): - return None - requirements = collect_fortran_type_storage_requirements(parsed, compile_time_values=compile_time_values) - if not requirements: - return None - return evaluate_fortran_type_facts( - preprocessing, - requirements, - report=report, - runner=runner, - cache_dir=cache_dir, - refresh=refresh, - ) - - def _bundle_output_name(bundle: _PyiContractBundle) -> str: """Derive a default extension name from a file or package-entry contract. @@ -3493,49 +3312,26 @@ def _fortran_wrapper_module( assume_intent_in_scalars: bool = False, export_symbols: Iterable[str] | None = None, ) -> tuple[object, SemanticModule, tuple[SemanticModule, ...], tuple[Path, ...]]: - """Parse Fortran sources, resolve type facts, and form one wrapper module.""" - # Preprocess and parse the complete source project. - preprocessed_sources = {} - semantic_dependencies = [] - for source_path in source_paths: - source, dependencies = _fortran_source_and_dependencies(source_path, preprocessing) - preprocessed_sources[str(source_path)] = source - semantic_dependencies.extend(dependencies) - parsed = parse_fortran_project(preprocessed_sources) - - # Measure compiler-dependent values before building semantic IR. - compile_time_values = _wrap_compile_time_values( - parsed, - type_probe_preprocessing, - report=fortran_type_report, - runner=fortran_type_probe_runner, - cache_dir=fortran_type_probe_cache_dir, - refresh=refresh_fortran_type_probe, - ) - type_facts = _wrap_type_facts( - parsed, - type_probe_preprocessing, - compile_time_values=compile_time_values, - report=fortran_type_report, - runner=fortran_type_probe_runner, - cache_dir=fortran_type_probe_cache_dir, - refresh=refresh_fortran_type_probe, - ) - - # Preserve source export paths while flattening the wrapper-facing module. - modules = fortran_project_to_semantic_modules( - parsed, - compile_time_values=compile_time_values, - type_facts=type_facts, + """Turn Fortran sources into one wrapper module through the shared source route.""" + sources = fortran_sources_to_semantic_modules( + source_paths, + preprocessing, + probe=FortranTypeProbe( + type_probe_preprocessing, + report=fortran_type_report, + runner=fortran_type_probe_runner, + cache_dir=fortran_type_probe_cache_dir, + refresh=refresh_fortran_type_probe, + ), assume_intent_in_scalars=assume_intent_in_scalars, + export_symbols=export_symbols, ) - if export_symbols is not None: - from prik.semantics.fortran_exports import select_fortran_export_symbols - - selection = select_fortran_export_symbols(modules, export_symbols) - for context_module in selection.context_modules: - context_module.exported_names = [] - modules = list(selection.available_modules) + # Context a selection keeps is compiled into the wrapper but not published. + for context_module in sources.context_modules: + context_module.exported_names = [] + modules = list(sources.modules) + parsed = sources.project + semantic_dependencies = sources.dependencies _apply_source_python_exports(modules) module_name = _validated_wrapper_module_name(output_name, source_paths[0].stem) return ( @@ -3781,17 +3577,12 @@ def build_fortran_extension( build_started = time.perf_counter() # 1. Collect the source and native implementation inputs. - source_paths = _source_paths(sources) + source_paths = validated_source_paths(sources, FORTRAN_SOURCE_SUFFIXES, label="Fortran") output_path, shared_library_output_path = _wrapper_output_paths(output_dir) output_path.mkdir(parents=True, exist_ok=True) preprocessing = preprocessing or _default_preprocessing_config() if module_source_dirs: - source_paths = resolve_fortran_module_sources( - source_paths, - tuple(Path(directory) for directory in module_source_dirs), - lambda path: _fortran_source_and_dependencies(path, preprocessing)[0], - command_line_macros=preprocessing.defines_command_line_macros, - ) + source_paths = discover_fortran_sources(source_paths, module_source_dirs, preprocessing) supplemental_source_paths = tuple(Path(path) for path in (native_fortran_sources or ())) input_implementation_paths = source_paths if compile_input_sources else () implementation_source_paths = (*input_implementation_paths, *supplemental_source_paths) @@ -3968,7 +3759,7 @@ def build_c_extension( raise ValueError("An external native implementation is valid only for source generation or planning") build_started = time.perf_counter() selected_exports = None if export_symbols is None else tuple(export_symbols) - source_paths = _c_source_paths(sources) + source_paths = validated_source_paths(sources, C_IMPLEMENTATION_SUFFIXES, label="C") output_path, shared_library_output_path = _wrapper_output_paths(output_dir) supplemental_c_paths = tuple(Path(path) for path in (native_c_sources or ())) native_inputs = _native_build_inputs( diff --git a/prik/pipeline/sources.py b/prik/pipeline/sources.py new file mode 100644 index 000000000..caff2b71b --- /dev/null +++ b/prik/pipeline/sources.py @@ -0,0 +1,201 @@ +"""Native sources to semantic IR: the one route builds and generated contracts share. + +A wrapper build and ``prik generate`` both turn Fortran sources into semantic +modules, and they must turn the same sources into the same modules, or a +contract generated from a project would describe something other than what a +build of that project wraps. Both therefore read, parse, measure, convert, +and select exports here, and differ only in what they do with the result. +""" + +from __future__ import annotations + +from collections.abc import Iterable, Mapping, Sequence +from dataclasses import dataclass, field +from pathlib import Path + +from prik.parsers.fortran.models import FortranProject +from prik.parsers.fortran.module_sources import resolve_fortran_module_sources +from prik.parsers.fortran.parser import parse_fortran_project +from prik.preprocessing import PreprocessingConfig, read_fortran_source +from prik.preprocessing.probes.fortran_types import ( + evaluate_fortran_type_facts, + evaluate_fortran_type_requirements, +) +from prik.semantics.models import SemanticModule +from prik.semantics.fortran2ir import ( + collect_fortran_type_storage_requirements, + collect_semantic_compile_time_requirements, + fortran_project_to_semantic_files, +) + + +@dataclass(frozen=True) +class FortranTypeProbe: + """How compiler-dependent Fortran values and type storage are measured. + + ``preprocessing`` selects the compiler and target flags the probe runs + with; without a compiler, and without a ``report`` to read instead, no + probe runs and those facts stay symbolic. + """ + + preprocessing: PreprocessingConfig + report: object | None = None + runner: list[str] | None = None + cache_dir: str | Path | None = None + refresh: bool = False + + @property + def available(self) -> bool: + """Return whether a report or a compiler can answer the probe.""" + return self.report is not None or (self.preprocessing.uses_compiler and bool(self.preprocessing.compiler)) + + def options(self) -> dict[str, object]: + """Return the evaluator options this probe states.""" + options: dict[str, object] = {} + if self.report is not None: + options["report"] = self.report + if self.runner is not None: + options["runner"] = self.runner + if self.cache_dir is not None: + options["cache_dir"] = self.cache_dir + if self.refresh: + options["refresh"] = True + return options + + +@dataclass(frozen=True) +class FortranSemanticSources: + """Fortran sources read, parsed, measured, converted, and selected. + + ``files`` pairs each source, in the order named, with its semantic + modules, which are the selected ones when symbols were selected. + ``modules`` is every module a consumer needs, in project order, and + ``context_modules`` those among them only reached as selection context. + """ + + project: FortranProject + files: tuple[tuple[Path, tuple[SemanticModule, ...]], ...] + modules: tuple[SemanticModule, ...] + context_modules: tuple[SemanticModule, ...] = () + recipes: Mapping[Path, dict[str, object] | None] = field(default_factory=dict) + dependencies: tuple[Path, ...] = () + + +def discover_fortran_sources( + entries: Sequence[Path], + module_source_dirs: Iterable[Path], + preprocessing: PreprocessingConfig, +) -> tuple[Path, ...]: + """Return ``entries`` with every source of a module they use, dependencies first.""" + return resolve_fortran_module_sources( + entries, + tuple(Path(directory) for directory in module_source_dirs), + lambda path: read_fortran_source(path, preprocessing).source, + command_line_macros=preprocessing.defines_command_line_macros, + ) + + +def fortran_sources_to_semantic_modules( + source_paths: Sequence[Path], + preprocessing: PreprocessingConfig, + *, + probe: FortranTypeProbe | None = None, + assume_intent_in_scalars: bool = False, + export_symbols: Iterable[str] | None = None, +) -> FortranSemanticSources: + """Turn Fortran sources into the semantic modules a build or a contract uses. + + The sources are read under ``preprocessing`` and parsed as one project, + ordered by their dependencies. Compile-time values and type storage are + measured with ``probe``, which defaults to ``preprocessing``, and every + file is converted with the whole project as context. ``export_symbols``, + when given, narrows each file to the selected symbols and keeps the rest + as context. + """ + paths = tuple(Path(path) for path in source_paths) + texts = {path: read_fortran_source(path, preprocessing) for path in paths} + project = parse_fortran_project({str(path): text.source for path, text in texts.items()}) + probe = probe or FortranTypeProbe(preprocessing) + compile_time_values = _compile_time_values(project, probe) + type_facts = _type_facts(project, probe, compile_time_values) + + converted = fortran_project_to_semantic_files( + project, + compile_time_values=compile_time_values, + assume_intent_in_scalars=assume_intent_in_scalars, + **({"type_facts": type_facts} if type_facts is not None else {}), + ) + by_path = {Path(str(parsed.filename)): tuple(modules) for parsed, modules in converted} + files = tuple((path, by_path[path]) for path in paths) + modules = tuple(module for _parsed, file_modules in converted for module in file_modules) + context_modules: tuple[SemanticModule, ...] = () + if export_symbols is not None: + from prik.semantics.fortran_exports import select_fortran_export_symbols + + selection = select_fortran_export_symbols(modules, export_symbols) + selected = { + id(source): module + for source, module in zip(selection.primary_sources, selection.primary_modules, strict=True) + } + files = tuple( + (path, chosen) + for path, file_modules in files + if (chosen := tuple(selected[id(module)] for module in file_modules if id(module) in selected)) + ) + modules = tuple(selection.available_modules) + context_modules = tuple(selection.context_modules) + + return FortranSemanticSources( + project=project, + files=files, + modules=modules, + context_modules=context_modules, + recipes={path: text.recipe for path, text in texts.items()}, + dependencies=tuple( + dict.fromkeys( + dependency + for path, text in texts.items() + for dependency in semantic_dependency_paths(path, text.included_files) + ) + ), + ) + + +def semantic_dependency_paths(root: Path, included_files: Iterable[object]) -> tuple[Path, ...]: + """Return one source and every existing file its preprocessing read, in stable order.""" + dependencies = [root.resolve(strict=False)] + for item in included_files: + raw_path = item.get("path") if isinstance(item, Mapping) else getattr(item, "path", None) + if not isinstance(raw_path, str | Path) or str(raw_path).startswith("<"): + continue + path = Path(raw_path) + if not path.is_absolute(): + path = root.parent / path + path = path.resolve(strict=False) + if path.is_file(): + dependencies.append(path) + return tuple(dict.fromkeys(dependencies)) + + +def _compile_time_values(project: FortranProject, probe: FortranTypeProbe) -> dict[str, int] | None: + """Measure the compile-time values the project's declarations need, if any.""" + if not probe.available: + return None + requirements = collect_semantic_compile_time_requirements(project) + if not requirements: + return None + return evaluate_fortran_type_requirements(probe.preprocessing, requirements, **probe.options()) + + +def _type_facts( + project: FortranProject, + probe: FortranTypeProbe, + compile_time_values: dict[str, int] | None, +) -> dict[tuple[str, str | None], dict[str, object]] | None: + """Measure the native storage of the intrinsic types the project uses, if any.""" + if not probe.available: + return None + requirements = collect_fortran_type_storage_requirements(project, compile_time_values=compile_time_values) + if not requirements: + return None + return evaluate_fortran_type_facts(probe.preprocessing, requirements, **probe.options()) diff --git a/prik/preprocessing/__init__.py b/prik/preprocessing/__init__.py index 266d9008b..7f07d268e 100644 --- a/prik/preprocessing/__init__.py +++ b/prik/preprocessing/__init__.py @@ -3,6 +3,7 @@ from .source import ( CommandTemplateAdapter, CompilerAdapter, + FortranSourceText, GCCCompatibleCAdapter, GNUFortranAdapter, IncludedFile, @@ -21,6 +22,7 @@ build_template_preprocess_invocation, parse_linemarker_mappings, preprocess_source, + read_fortran_source, run_compiler_preprocessor, run_compiler_preprocessor_with_recipe, validate_macro_name, @@ -30,6 +32,7 @@ __all__ = ( "CommandTemplateAdapter", "CompilerAdapter", + "FortranSourceText", "GCCCompatibleCAdapter", "GNUFortranAdapter", "IncludedFile", @@ -49,6 +52,7 @@ "expand_native_fortran_includes", "parse_linemarker_mappings", "preprocess_source", + "read_fortran_source", "run_compiler_preprocessor", "run_compiler_preprocessor_with_recipe", "validate_macro_name", diff --git a/prik/preprocessing/languages.py b/prik/preprocessing/languages.py new file mode 100644 index 000000000..1e9f2d51a --- /dev/null +++ b/prik/preprocessing/languages.py @@ -0,0 +1,132 @@ +"""Which native language, and which Fortran source form, a source path names. + +Every stage that asks whether a path is a Fortran or C source, which Fortran +source form a file is written in, or which sources a list of files and +directories names reads the answer here, so discovery, parsing, +preprocessing, and builds never disagree about a path. +""" + +from __future__ import annotations + +from collections.abc import Iterable +from pathlib import Path + +#: Fortran sources written in fixed form: statements in columns 7 to 72, with +#: column 6 marking a continuation. ``.fpp`` is fixed-form source that is +#: preprocessed first, as the common Fortran compilers read it. +FORTRAN_FIXED_FORM_SUFFIXES = frozenset({".f", ".for", ".ftn", ".f77", ".fpp"}) +#: Fortran sources written in free form. +FORTRAN_FREE_FORM_SUFFIXES = frozenset({".f90", ".f95", ".f03", ".f08"}) +#: Every suffix a Fortran source may carry, in either case. +FORTRAN_SOURCE_SUFFIXES = FORTRAN_FIXED_FORM_SUFFIXES | FORTRAN_FREE_FORM_SUFFIXES +#: C inputs PRIK reads: implementation files, headers, and preprocessed units. +C_SOURCE_SUFFIXES = frozenset({".c", ".h", ".i"}) +#: C files a native build compiles. +C_IMPLEMENTATION_SUFFIXES = frozenset({".c"}) + + +def is_fortran_source(path: str | Path) -> bool: + """Return whether ``path`` names a Fortran source by its suffix.""" + return Path(path).suffix.casefold() in FORTRAN_SOURCE_SUFFIXES + + +def is_c_source(path: str | Path) -> bool: + """Return whether ``path`` names a C input by its suffix.""" + return Path(path).suffix.casefold() in C_SOURCE_SUFFIXES + + +def fortran_source_form(code: str, filename: str | None = None) -> str: + """Return ``"fixed"`` or ``"free"``, the form one Fortran source is written in. + + The suffix decides when it names a form. Otherwise the text does: a line + among the first twenty with blank columns 1 to 5 and a character in + column 6 is a fixed-form continuation. + """ + if filename: + suffix = Path(filename).suffix.casefold() + if suffix in FORTRAN_FIXED_FORM_SUFFIXES: + return "fixed" + if suffix in FORTRAN_FREE_FORM_SUFFIXES: + return "free" + for line in code.splitlines()[:20]: + if len(line) >= 6 and line[:5].strip() == "" and line[5:6].strip(): + return "fixed" + return "free" + + +def expand_source_paths(inputs: Iterable[str | Path], suffixes: Iterable[str]) -> tuple[Path, ...]: + """Return the sources a list of files and directories names, in the caller's order. + + A file is kept as given. A directory contributes every file below it + whose suffix is in ``suffixes``, in sorted order. A path named twice is + kept once, at its first position. Whether a given path exists or has a + supported suffix is for the caller to judge. + """ + wanted = frozenset(suffix.casefold() for suffix in suffixes) + paths: dict[Path, None] = {} + for raw in inputs: + path = Path(raw) + if path.is_dir(): + for candidate in sorted(path.rglob("*")): + if candidate.is_file() and candidate.suffix.casefold() in wanted: + paths.setdefault(candidate, None) + else: + paths.setdefault(path, None) + return tuple(paths) + + +class SourceInputError(ValueError): + """A list of wrapper inputs that does not name usable sources. + + ``reason`` is ``"empty"`` when nothing was named, ``"missing"`` for a file + that does not exist, ``"unsupported"`` for a file whose suffix is not a + source of the language, and ``"no_sources"`` for a directory holding + none. ``path`` is the offending input, when there is one. + """ + + def __init__(self, reason: str, label: str, path: Path | None = None) -> None: + messages = { + "empty": f"wrapper build requires at least one {label} source file or directory", + "missing": f"{label} source not found: {path}", + "unsupported": f"Unrecognized {label} source suffix: {path}", + "no_sources": f"No recognized {label} sources found under: {path}", + } + super().__init__(messages[reason]) + self.reason = reason + self.label = label + self.path = path + + +class MissingSourceError(SourceInputError, FileNotFoundError): + """A named wrapper input that does not exist.""" + + def __init__(self, label: str, path: Path) -> None: + super().__init__("missing", label, path) + + +def validated_source_paths( + inputs: str | Path | Iterable[str | Path], + suffixes: Iterable[str], + *, + label: str, +) -> tuple[Path, ...]: + """Return the sources wrapper inputs name, requiring each input to name some. + + This is :func:`expand_source_paths` for a build, which fails closed: every + file must exist with one of ``suffixes``, every directory must hold such a + file, and something must be named. ``label`` names the language in the + :class:`SourceInputError` raised otherwise. + """ + named = (Path(inputs),) if isinstance(inputs, str | Path) else tuple(Path(item) for item in inputs) + if not named: + raise SourceInputError("empty", label) + wanted = frozenset(suffix.casefold() for suffix in suffixes) + for path in named: + if path.is_dir(): + if not expand_source_paths([path], wanted): + raise SourceInputError("no_sources", label, path) + elif path.suffix.casefold() not in wanted: + raise SourceInputError("unsupported", label, path) + elif not path.is_file(): + raise MissingSourceError(label, path) + return expand_source_paths(named, wanted) diff --git a/prik/preprocessing/source.py b/prik/preprocessing/source.py index 85bcbe83f..b5f47ab76 100644 --- a/prik/preprocessing/source.py +++ b/prik/preprocessing/source.py @@ -30,6 +30,7 @@ from pathlib import Path from typing import ClassVar, Literal, Protocol +from prik.preprocessing.languages import is_fortran_source from prik.compiler.compiler_profiles import fortran_compiler_family @@ -50,8 +51,6 @@ # Compiler output syntax and supported source forms. _VALID_LANGUAGES = {"c", "fortran"} -_C_SOURCE_SUFFIXES = {".c", ".h", ".i"} -_FORTRAN_SOURCE_SUFFIXES = {".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08"} _DEFINE_RE = re.compile(r"^\s*#\s*define\s+([A-Za-z_]\w*)(\(([^)]*)\))?(?:\s+(.*))?$") _LINEMARKER_RE = re.compile( r'^\s*#\s+(?P\d+)\s+(?:"(?P(?:[^"\\]|\\.)*)"|(?P\S+))(?P(?:\s+\d+)*)\s*$' @@ -576,7 +575,7 @@ def _preprocessor_options( def _fortran_source_language_hint(source: Path) -> list[str]: """Return a source-form hint only for Fortran paths with unknown suffixes.""" - if source.suffix.lower() in _FORTRAN_SOURCE_SUFFIXES: + if is_fortran_source(source): return [] return ["-x", "f95-cpp-input"] @@ -1484,6 +1483,36 @@ def preprocess_source( return result +@dataclass(frozen=True) +class FortranSourceText: + """One Fortran source read the way the parser must see it. + + ``recipe`` records how the text was produced, or is ``None`` when it was + read as written with nothing to record; ``included_files`` lists what + compiler preprocessing pulled in. + """ + + source: str + recipe: dict[str, object] | None + included_files: tuple[IncludedFile, ...] = () + + +def read_fortran_source(source_path: Path | str, config: PreprocessingConfig) -> FortranSourceText: + """Return one Fortran source as the parser reads it, under ``config``. + + Compiler preprocessing expands the source and reports its recipe and + included files; otherwise the file is read as UTF-8 text with the + internal recipe its macros call for. Every route that parses a Fortran + path -- a build, ``prik generate``, a parse report, module discovery -- + reads it here. + """ + path = Path(source_path) + if config.uses_compiler: + result = preprocess_source(path, language="fortran", config=config) + return FortranSourceText(result.source, _recipe_from_result(result).to_dict(), tuple(result.included_files)) + return FortranSourceText(path.read_text(encoding="utf-8"), config.fortran_internal_recipe(path)) + + def run_compiler_preprocessor_with_recipe( source_path: Path | str, language: str, @@ -1518,6 +1547,7 @@ def run_compiler_preprocessor( __all__ = ( "CommandTemplateAdapter", "CompilerAdapter", + "FortranSourceText", "GCCCompatibleCAdapter", "GNUFortranAdapter", "IncludedFile", @@ -1536,6 +1566,7 @@ def run_compiler_preprocessor( "build_template_preprocess_invocation", "parse_linemarker_mappings", "preprocess_source", + "read_fortran_source", "run_compiler_preprocessor", "run_compiler_preprocessor_with_recipe", "validate_macro_name", diff --git a/prik/semantics/__init__.py b/prik/semantics/__init__.py index 4c22b2abb..8423ea6de 100644 --- a/prik/semantics/__init__.py +++ b/prik/semantics/__init__.py @@ -21,7 +21,6 @@ c_file_to_semantic_modules, c_function_to_semantic_function, c_parameter_to_semantic_argument, - c_project_to_semantic_module, c_project_to_semantic_modules, c_struct_to_semantic_class, c_type_to_semantic_type, @@ -35,7 +34,6 @@ "c_file_to_semantic_modules", "c_function_to_semantic_function", "c_parameter_to_semantic_argument", - "c_project_to_semantic_module", "c_project_to_semantic_modules", "c_struct_to_semantic_class", "c_type_to_semantic_type", diff --git a/prik/semantics/c2ir.py b/prik/semantics/c2ir.py index f8071d322..781400fe8 100644 --- a/prik/semantics/c2ir.py +++ b/prik/semantics/c2ir.py @@ -290,57 +290,6 @@ def _visit_CProject(self, project: CProject) -> list[SemanticModule]: self._classify_project_external_types(modules, project) return modules - def project_to_semantic_module( - self, - project: CProject, - *, - name: str = "c_project", - ) -> SemanticModule: - """Merge a project registry into one synthetic semantic module. - - This compatibility entrypoint converts project-level registries without - file-module exposure processing. It restores every converter registry - in ``finally`` so a reused converter has no project-state leakage. - """ - previous = self.typedefs, self.structs, self.unions, self.enums, self.opaque_standard_types - self.typedefs = dict(project.typedefs) - self.structs = dict(project.structs) - self.unions = dict(project.unions) - self.enums = dict(project.enums) - self.opaque_standard_types = set() - try: - semantic_functions = [self.visit(function) for function in project.functions.values()] - semantic_variables = [ - *[ - enumerator - for enum in self._project_enum_declarations(project) - for enumerator in self._enum_constants_for_enum(enum) - ], - *self._macro_constants_from_macros(list(project.macros.values())), - *[self.visit(variable) for variable in project.variables.values()], - ] - semantic_classes = [ - *[self.visit(struct) for struct in project.structs.values()], - *[self.visit(union) for union in project.unions.values()], - *self._opaque_standard_type_classes(), - ] - return SemanticModule( - name=self._identifier(name), - functions=semantic_functions, - classes=semantic_classes, - variables=semantic_variables, - metadata=self._project_metadata(project), - origin=SemanticOrigin( - source_language="c", - native_name=name, - native_scope=name, - source_kind="project", - metadata={"files": sorted(project.files)}, - ), - ) - finally: - self.typedefs, self.structs, self.unions, self.enums, self.opaque_standard_types = previous - def _visit_CFile( self, c_file: CFile, @@ -1518,26 +1467,6 @@ def _set_external_type_ref( "representation": "wrapped" if wrapped else "opaque", } - def _project_metadata(self, project: CProject) -> dict[str, Any]: - """Return stable language and aggregate-count metadata for a merged project module.""" - metadata: dict[str, Any] = { - "source_language": "c", - "counts": { - "files": len(project.files), - "functions": len(project.functions), - "structs": len(project.structs), - "unions": len(project.unions), - "enums": len(self._project_enum_declarations(project)), - "typedefs": len(project.typedefs), - "macros": len(project.macros), - "includes": len(project.includes), - "diagnostics": len(project.diagnostics), - }, - } - return metadata - - # Type lookup, target facts, and naming helpers - def _resolve_typedef(self, typedef: CTypedef, stack: tuple[str, ...] = ()) -> CTypedef | None: """Resolve typedef aliases through the current registry without following cycles. @@ -1773,22 +1702,6 @@ def _resolved_enum(self, enum: CEnum) -> CEnum: return self.enums[enum.name] return enum - @staticmethod - def _project_enum_declarations(project: CProject) -> list[CEnum]: - """Return project enums once, including anonymous declarations stored only on files.""" - declarations = list(project.enums.values()) - anonymous_ids: set[str | int] = {enum.anonymous_id or id(enum) for enum in declarations if enum.name is None} - for c_file in project.files.values(): - for enum in c_file.enums: - if enum.name is not None: - continue - identity: str | int = enum.anonymous_id or id(enum) - if identity in anonymous_ids: - continue - anonymous_ids.add(identity) - declarations.append(enum) - return declarations - def _typedef_alias_for_type(self, target: CType) -> str | None: """Find the first registry typedef whose target is the same parser type object.""" for typedef in self.typedefs.values(): @@ -2162,31 +2075,12 @@ def _apply_c_export_selection(module: SemanticModule, selected: set[str]) -> Non module.variables = [] -def c_project_to_semantic_module( - project: CProject, - *, - name: str = "c_project", - standard_type_report: Any | None = None, -) -> SemanticModule: - """Merge project registries into one synthetic semantic module. - - Use this compatibility entrypoint when consumers require one aggregate - module rather than file-level ownership and external references. ``name`` - is normalized into a semantic identifier; the project itself is not mutated. - """ - return CToIRConverter(standard_type_report=standard_type_report).project_to_semantic_module( - project, - name=name, - ) - - __all__ = ( "CToIRConverter", "c_file_to_semantic_module", "c_file_to_semantic_modules", "c_function_to_semantic_function", "c_parameter_to_semantic_argument", - "c_project_to_semantic_module", "c_project_to_semantic_modules", "c_struct_to_semantic_class", "c_type_to_semantic_type", diff --git a/tests/c/_support/fixture_outputs.py b/tests/c/_support/fixture_outputs.py index 3e9187648..0f165350a 100644 --- a/tests/c/_support/fixture_outputs.py +++ b/tests/c/_support/fixture_outputs.py @@ -7,9 +7,9 @@ from tempfile import TemporaryDirectory from prik.parsers.c import CParser -from prik.parsers.c.cli import attach_preprocessing_recipe +from prik.parsers.c.sources import attach_preprocessing_recipe from prik.preprocessing import PreprocessingConfig, preprocess_source -from prik.semantics.c2ir import c_project_to_semantic_module +from prik.semantics.c2ir import c_project_to_semantic_modules from prik.printers import emit_module from tests.c._support.paths import C_DATA_DIR, C_ROOT @@ -70,10 +70,16 @@ def parse_c_fixture_project(paths: list[Path]): def c_semantic_module_for_fixture_project(project_key: Path, paths: list[Path]): - return c_project_to_semantic_module( - parse_c_fixture_project(paths), - name=project_key.as_posix().replace("/", "_"), - ) + """Return the module PRIK converts from the project's primary input. + + Each file is converted with the project as context, exactly as a C route + converts it. A build wraps the implementation file, which includes its + header, so a project's contract is the ``.c`` file's module when it has + one and its header's otherwise. + """ + primary = min(paths, key=_c_fixture_sort_key).relative_to(C_DATA_DIR).as_posix() + modules = c_project_to_semantic_modules(parse_c_fixture_project(paths)) + return next(module for module in modules if module.origin.native_name == primary) def c_pyi_text_for_fixture_project(project_key: Path, paths: list[Path]) -> str: diff --git a/tests/c/data_types/semantics/test_types_and_constants.py b/tests/c/data_types/semantics/test_types_and_constants.py index 176a195a3..1c21f1c04 100644 --- a/tests/c/data_types/semantics/test_types_and_constants.py +++ b/tests/c/data_types/semantics/test_types_and_constants.py @@ -46,7 +46,6 @@ c_file_to_semantic_modules, c_function_to_semantic_function, c_parameter_to_semantic_argument, - c_project_to_semantic_module, c_project_to_semantic_modules, c_struct_to_semantic_class, c_type_to_semantic_type, @@ -439,9 +438,6 @@ def test_c_compatibility_helpers_forward_standard_type_reports(): assert _function( c_project_to_semantic_modules(project, standard_type_report=report)[0], "measure" ).return_type.name == ("UInt32") - assert _function( - c_project_to_semantic_module(project, standard_type_report=report), "measure" - ).return_type.name == ("UInt32") @pytest.mark.parametrize( diff --git a/tests/c/enumerations/semantics/test_c_enum_semantics.py b/tests/c/enumerations/semantics/test_c_enum_semantics.py index bc32d4778..123251bfe 100644 --- a/tests/c/enumerations/semantics/test_c_enum_semantics.py +++ b/tests/c/enumerations/semantics/test_c_enum_semantics.py @@ -12,7 +12,6 @@ CToIRConverter, c_file_to_semantic_module, c_file_to_semantic_modules, - c_project_to_semantic_module, c_project_to_semantic_modules, ) from prik.semantics.models import ( @@ -79,7 +78,7 @@ def test_c2ir_names_anonymous_typedef_enums_and_keeps_enumerators_unscoped(): parsed = parse_c_file(source, filename="flags.h") module = c_file_to_semantic_module(parsed) - project_module = c_project_to_semantic_module(parse_c_project({"flags.h": source}), name="flags") + project_module = c_project_to_semantic_modules(parse_c_project({"flags.h": source}))[0] assert module.classes == [] assert project_module.classes == [] diff --git a/tests/c/fixtures/pyi/general/modern_math_physics.pyi b/tests/c/fixtures/pyi/general/modern_math_physics.pyi index e092badc4..c574495b2 100644 --- a/tests/c/fixtures/pyi/general/modern_math_physics.pyi +++ b/tests/c/fixtures/pyi/general/modern_math_physics.pyi @@ -8,7 +8,7 @@ class modern_particle(CStruct): class vector3(CStruct): values: Float64[3] -modern_counter: Int +modern_counter: Int = 0 hidden_scale: private[Float64] = 1.0 diff --git a/tests/c/infrastructure/cli/pipeline/test_c_cli_output_contract.py b/tests/c/infrastructure/cli/pipeline/test_c_cli_output_contract.py index f9942b7cf..f060ac970 100644 --- a/tests/c/infrastructure/cli/pipeline/test_c_cli_output_contract.py +++ b/tests/c/infrastructure/cli/pipeline/test_c_cli_output_contract.py @@ -22,16 +22,6 @@ def test_prik_main_preserves_c_readable_stdout_contract(monkeypatch, capsys): "_build_preprocessing_config", lambda active_args, parser: preprocessing, ) - monkeypatch.setattr( - prik_cli, - "_c_parser_preprocessing_mode", - lambda active_preprocessing: "mode", - ) - monkeypatch.setattr( - prik_cli, - "_c_source_loader", - lambda active_preprocessing: "loader", - ) monkeypatch.setattr(prik_cli, "parse_c_report", lambda *args, **kwargs: parse_payload) monkeypatch.setattr( prik_cli, diff --git a/tests/c/infrastructure/cli/pipeline/test_c_cli_skeleton.py b/tests/c/infrastructure/cli/pipeline/test_c_cli_skeleton.py index 0c00b72d8..a00862079 100644 --- a/tests/c/infrastructure/cli/pipeline/test_c_cli_skeleton.py +++ b/tests/c/infrastructure/cli/pipeline/test_c_cli_skeleton.py @@ -12,6 +12,7 @@ from prik.parsers.c import CParseError from prik.parsers.c import cli as c_parser_cli +from prik.parsers.c import sources as c_sources import prik.cli as prik_cli from prik.preprocessing import PreprocessingConfig @@ -127,15 +128,15 @@ def test_cli_c_parse_preprocesses_macros_by_default(tmp_path: Path): def test_attach_preprocessing_recipe_filters_invalid_and_duplicate_macros(): - empty = c_parser_cli.CFile() - c_parser_cli.attach_preprocessing_recipe(empty, None) + empty = c_sources.CFile() + c_sources.attach_preprocessing_recipe(empty, None) assert empty.preprocessing_recipe is None - parsed = c_parser_cli.CFile( + parsed = c_sources.CFile( macros=[ - c_parser_cli.CMacro( + c_sources.CMacro( name="EXISTING", - source_location=c_parser_cli.CSourceLocation(filename="api.h", line=2), + source_location=c_sources.CSourceLocation(filename="api.h", line=2), ) ] ) @@ -149,7 +150,7 @@ def test_attach_preprocessing_recipe_filters_invalid_and_duplicate_macros(): ] } - c_parser_cli.attach_preprocessing_recipe(parsed, recipe) + c_sources.attach_preprocessing_recipe(parsed, recipe) assert parsed.preprocessing_recipe == recipe assert [macro.name for macro in parsed.macros] == ["EXISTING", "NEW", "WITH_LOC"] @@ -479,12 +480,8 @@ def test_c_parser_cli_module_handles_directory_loader_and_output_modes(tmp_path: (tmp_path / "ignored.txt").write_text("ignored\n", encoding="utf-8") assert c_parser_cli.expand_c_paths([str(tmp_path), str(header)]) == [header] - loaded = c_parser_cli.parse_c_report( - [str(header)], - source_loader=lambda _path: ("int generated(void);\n", {"mode": "test"}), - ) - assert loaded[str(header)]["functions"][0]["name"] == "generated" - assert loaded[str(header)]["preprocessing_recipe"] == {"mode": "test"} + loaded = c_parser_cli.parse_c_report([str(header)]) + assert loaded[str(header)]["functions"][0]["name"] == "add" assert c_parser_cli.main([str(header)]) == 0 assert "Functions: 1" in capsys.readouterr().out @@ -559,7 +556,7 @@ def preprocess(path, *, language, config): SimpleNamespace(to_dict=lambda: {"mode": "compiler", "compiler": config.compiler}), ) - monkeypatch.setattr(prik_cli, "run_compiler_preprocessor_with_recipe", preprocess) + monkeypatch.setattr(c_sources, "run_compiler_preprocessor_with_recipe", preprocess) config = PreprocessingConfig(mode="compiler", compiler="cc") semantics = prik_cli._semantic_report([str(header)], config, language="c") diff --git a/tests/c/infrastructure/cli/pipeline/test_c_cli_stage_dispatch.py b/tests/c/infrastructure/cli/pipeline/test_c_cli_stage_dispatch.py index 0811e535d..953fcbe65 100644 --- a/tests/c/infrastructure/cli/pipeline/test_c_cli_stage_dispatch.py +++ b/tests/c/infrastructure/cli/pipeline/test_c_cli_stage_dispatch.py @@ -5,6 +5,7 @@ import pytest import prik.cli as prik_cli +from prik.parsers.c import sources as c_sources from tests.c._support.cli import ( _install_main_parser, _main_args, @@ -18,8 +19,6 @@ class StopAfterDispatch(Exception): args = _main_args(language="requested", parse=True) _install_main_parser(monkeypatch, args) preprocessing = type("Preprocessing", (), {"include_dirs": ("include",)})() - parser_mode = object() - source_loader = object() parse_payload = {"parse": "payload"} calls = [] @@ -29,20 +28,10 @@ class StopAfterDispatch(Exception): "_build_preprocessing_config", lambda active_args, parser: preprocessing, ) - monkeypatch.setattr( - prik_cli, - "_c_parser_preprocessing_mode", - lambda active_preprocessing: calls.append(("mode", active_preprocessing)) or parser_mode, - ) - monkeypatch.setattr( - prik_cli, - "_c_source_loader", - lambda active_preprocessing: calls.append(("loader", active_preprocessing)) or source_loader, - ) monkeypatch.setattr( prik_cli, "parse_c_report", - lambda paths, **kwargs: calls.append(("parse", paths, kwargs)) or parse_payload, + lambda paths, active_preprocessing: calls.append(("parse", paths, active_preprocessing)) or parse_payload, ) monkeypatch.setattr( prik_cli, @@ -53,19 +42,8 @@ class StopAfterDispatch(Exception): with pytest.raises(StopAfterDispatch): prik_cli.main() - assert calls == [ - ("mode", preprocessing), - ("loader", preprocessing), - ( - "parse", - args.paths, - { - "include_dirs": preprocessing.include_dirs, - "preprocessing": parser_mode, - "source_loader": source_loader, - }, - ), - ] + # The C parse report receives the one preprocessing configuration the CLI built. + assert calls == [("parse", args.paths, preprocessing)] @pytest.mark.parametrize("stage", ["semantics", "pyi"]) @@ -92,10 +70,11 @@ class StopAfterDispatch(Exception): prik_cli.main() -def test_prik_parse_c_path_preserves_parser_and_preprocessing_arguments( +def test_one_c_parse_preserves_parser_and_preprocessing_arguments( tmp_path: Path, monkeypatch, ): + """Every C route parses a path through parse_c_source, raw or compiler-preprocessed.""" path = tmp_path / "api.h" raw_parsed = object() compiled_parsed = object() @@ -109,7 +88,7 @@ def parse_file(self, source, *, filename, include_dirs, preprocessing): return raw_parsed raw_config = prik_cli.PreprocessingConfig(include_dirs=["include"]) - assert prik_cli._parse_c_path(RawParser(), path, raw_config) is raw_parsed + assert c_sources.parse_c_source(path, raw_config, parser=RawParser()) is raw_parsed class Recipe: def to_dict(self): @@ -138,10 +117,10 @@ def attach_recipe(parsed, recipe): compiler="cc", include_dirs=["include"], ) - monkeypatch.setattr(prik_cli, "run_compiler_preprocessor_with_recipe", preprocess) - monkeypatch.setattr(prik_cli, "attach_preprocessing_recipe", attach_recipe) + monkeypatch.setattr(c_sources, "run_compiler_preprocessor_with_recipe", preprocess) + monkeypatch.setattr(c_sources, "attach_preprocessing_recipe", attach_recipe) - assert prik_cli._parse_c_path(CompilerParser(), path, compiler_config) is compiled_parsed + assert c_sources.parse_c_source(path, compiler_config, parser=CompilerParser()) is compiled_parsed def test_prik_main_preserves_c_parse_error_rendering_contract(monkeypatch, capsys): @@ -172,16 +151,6 @@ def test_prik_main_preserves_c_parse_error_rendering_contract(monkeypatch, capsy "format_diagnostic", lambda self, *, color, debug: calls.append(("render", color, debug)) or "rendered diagnostic", ) - monkeypatch.setattr( - prik_cli, - "_c_parser_preprocessing_mode", - lambda active_preprocessing: "mode", - ) - monkeypatch.setattr( - prik_cli, - "_c_source_loader", - lambda active_preprocessing: "loader", - ) monkeypatch.setattr( prik_cli, "parse_c_report", @@ -215,16 +184,6 @@ def test_prik_main_reraises_c_parse_errors_for_debug_environment(monkeypatch): "_env_flag", lambda name: calls.append(name) or name == "C_PARSER_DEBUG", ) - monkeypatch.setattr( - prik_cli, - "_c_parser_preprocessing_mode", - lambda active_preprocessing: "mode", - ) - monkeypatch.setattr( - prik_cli, - "_c_source_loader", - lambda active_preprocessing: "loader", - ) monkeypatch.setattr( prik_cli, "parse_c_report", diff --git a/tests/c/infrastructure/semantic_ir/semantics/test_projects_and_diagnostics.py b/tests/c/infrastructure/semantic_ir/semantics/test_projects_and_diagnostics.py index 044f76178..6f3e0bb50 100644 --- a/tests/c/infrastructure/semantic_ir/semantics/test_projects_and_diagnostics.py +++ b/tests/c/infrastructure/semantic_ir/semantics/test_projects_and_diagnostics.py @@ -21,14 +21,12 @@ CToIRConverter, c_file_to_semantic_module, c_parameter_to_semantic_argument, - c_project_to_semantic_module, c_project_to_semantic_modules, c_struct_to_semantic_class, c_type_to_semantic_type, ) from prik.semantics.models import SemanticArgument, SemanticModule, SemanticOrigin, SemanticType from tests.c._support.semantic_conversion import ( - _assert_c_origin, _function, ) @@ -156,33 +154,6 @@ def test_c2ir_visitor_and_project_compatibility_entrypoints_cover_supported_node assert default_argument.metadata == {"native_position": 0} assert c_struct_to_semantic_class(first.structs[0]).name == "point" assert [module.name for module in c_project_to_semantic_modules(project)] == ["a", "b"] - merged = c_project_to_semantic_module(project, name="42 api/project") - assert merged.name == "_42_api_project" - assert {function.name for function in merged.functions} == {"f", "g"} - assert [cls.name for cls in merged.classes] == ["point"] - assert [variable.name for variable in merged.variables] == ["value"] - assert merged.metadata == { - "source_language": "c", - "counts": { - "files": 2, - "functions": 2, - "structs": 1, - "unions": 0, - "enums": 0, - "typedefs": 0, - "macros": 0, - "includes": 0, - "diagnostics": 0, - }, - } - _assert_c_origin( - merged.origin, - native_name="42 api/project", - native_scope="42 api/project", - source_kind="project", - metadata={"files": ["a.h", "b.h"]}, - ) - assert converter.project_to_semantic_module(project).name == "c_project" typedef_project = parse_c_project( { "types.h": "typedef unsigned long count_t;\n", @@ -192,9 +163,6 @@ def test_c2ir_visitor_and_project_compatibility_entrypoints_cover_supported_node typedef_modules = {module.name: module for module in converter.visit(typedef_project)} assert _function(typedef_modules["api"], "count").return_type.name == "UInt64" assert _function(typedef_modules["api"], "count").return_type.metadata == {"c_typedefs": ["count_t"]} - typedef_merged = converter.project_to_semantic_module(typedef_project) - assert _function(typedef_merged, "count").return_type.name == "UInt64" - assert _function(typedef_merged, "count").return_type.metadata == {"c_typedefs": ["count_t"]} count_reference = CTypedef(name="global_count_t") count_function = CFunction(name="global_count", result_type=count_reference) reference_project = CProject( @@ -204,8 +172,6 @@ def test_c2ir_visitor_and_project_compatibility_entrypoints_cover_supported_node ) reference_modules = converter.visit(reference_project) assert _function(reference_modules[0], "global_count").return_type.name == "Int" - reference_merged = converter.project_to_semantic_module(reference_project) - assert _function(reference_merged, "global_count").return_type.name == "Int" record = CStruct(name="global_record", members=[CVariable(name="value", type=CInt())]) choice = CUnion(name="global_choice", members=[CVariable(name="value", type=CInt())]) registry_function = CFunction( @@ -225,10 +191,3 @@ def test_c2ir_visitor_and_project_compatibility_entrypoints_cover_supported_node registry_args = _function(registry_modules[0], "use_global_types").arguments assert registry_args[0].semantic_type.metadata == {"c_kind": "struct", "incomplete": False} assert registry_args[1].semantic_type.metadata["incomplete"] is False - registry_merged = converter.project_to_semantic_module(registry_project) - registry_merged_args = _function(registry_merged, "use_global_types").arguments - assert [arg.semantic_type.name for arg in registry_merged_args] == [ - "global_record", - "global_choice", - ] - assert [arg.semantic_type.metadata["incomplete"] for arg in registry_merged_args] == [False, False] diff --git a/tests/fortran/_support/wrapper_build.py b/tests/fortran/_support/wrapper_build.py index 560a0359c..1ff255fe2 100644 --- a/tests/fortran/_support/wrapper_build.py +++ b/tests/fortran/_support/wrapper_build.py @@ -27,11 +27,10 @@ NativeBuildPlan, _apply_source_python_exports, _build_generated_wrapper_extension, - _fortran_source_for_pipeline, _merge_wrapper_modules, _new_compiler, ) -from prik.preprocessing import PreprocessingConfig +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.pipeline.build import build_fortran_extension from prik.runtime.handles import AllocatableArray from prik.semantics.fortran2ir import fortran_project_to_semantic_modules @@ -349,10 +348,10 @@ def _build_source_wrapper_plan_and_import( ) parsed = parse_fortran_project( { - str(source): _fortran_source_for_pipeline( + str(source): read_fortran_source( source, PreprocessingConfig(mode="compiler", compiler=_compiler()), - ) + ).source } ) modules = fortran_project_to_semantic_modules(parsed) diff --git a/tests/fortran/allocatables/policy/test_allocatable_result_policy.py b/tests/fortran/allocatables/policy/test_allocatable_result_policy.py index f12b1c493..1db56137a 100644 --- a/tests/fortran/allocatables/policy/test_allocatable_result_policy.py +++ b/tests/fortran/allocatables/policy/test_allocatable_result_policy.py @@ -4,8 +4,8 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from tests.fortran._support.wrapper_build import wrapper_source from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _fortran_source_for_pipeline, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.semantics.models import ( RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, @@ -26,7 +26,7 @@ def _source_semantic_module(filename: str, *, module_name: str): source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) diff --git a/tests/fortran/callbacks/policy/test_callback_policy.py b/tests/fortran/callbacks/policy/test_callback_policy.py index d0b874c65..9bc2f209d 100644 --- a/tests/fortran/callbacks/policy/test_callback_policy.py +++ b/tests/fortran/callbacks/policy/test_callback_policy.py @@ -4,8 +4,8 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _fortran_source_for_pipeline, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.semantics.models import ( RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, @@ -25,7 +25,7 @@ def _source_semantic_module(filename: str, *, module_name: str, assume_intent_in_scalars: bool = False): source = FIXTURES / "native" / filename - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed, assume_intent_in_scalars=assume_intent_in_scalars) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) diff --git a/tests/fortran/functions/policy/test_function_result_policy.py b/tests/fortran/functions/policy/test_function_result_policy.py index 202fe1a61..f06ce93c7 100644 --- a/tests/fortran/functions/policy/test_function_result_policy.py +++ b/tests/fortran/functions/policy/test_function_result_policy.py @@ -4,8 +4,8 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from tests.fortran._support.wrapper_build import wrapper_source from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _fortran_source_for_pipeline, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.policy.ownership import ( NativeBarrierAction, @@ -22,7 +22,7 @@ def _source_semantic_module(filename: str, *, module_name: str): source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) diff --git a/tests/fortran/generic_interfaces/policy/test_generic_policy.py b/tests/fortran/generic_interfaces/policy/test_generic_policy.py index c3d550294..bca2e57da 100644 --- a/tests/fortran/generic_interfaces/policy/test_generic_policy.py +++ b/tests/fortran/generic_interfaces/policy/test_generic_policy.py @@ -4,8 +4,8 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from tests.fortran._support.wrapper_build import wrapper_source from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _fortran_source_for_pipeline, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.semantics.models import ( RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, @@ -17,7 +17,7 @@ def _source_semantic_module(filename: str, *, module_name: str): source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) diff --git a/tests/fortran/infrastructure/cli/pipeline/test_output_contract.py b/tests/fortran/infrastructure/cli/pipeline/test_output_contract.py index c5c793a9f..70da16fb8 100644 --- a/tests/fortran/infrastructure/cli/pipeline/test_output_contract.py +++ b/tests/fortran/infrastructure/cli/pipeline/test_output_contract.py @@ -14,6 +14,9 @@ import prik import pytest +from prik.preprocessing import PreprocessResult +from prik.preprocessing import source as preprocessing_source + import prik.cli as prik_cli from prik.parsers.fortran import cli as fortran_parser_cli from prik.preprocessing import ( @@ -602,7 +605,8 @@ def args(**overrides): upper_stub.write_text("def upper() -> None: ...\n", encoding="utf-8") (tmp_path / "notes.txt").write_text("ignore", encoding="utf-8") - assert prik_cli._expand_pyi_paths([str(tmp_path), str(stub)]) == [stub] + # A directory yields every contract it holds, whatever the suffix's case, and each once. + assert prik_cli._expand_pyi_paths([str(tmp_path), str(stub)]) == [stub, upper_stub] assert prik_cli._expand_pyi_paths([str(stub)]) == [stub] assert prik_cli._expand_pyi_paths([str(upper_stub)]) == [upper_stub] assert prik_cli._expand_pyi_paths([str(tmp_path / "notes.txt")]) == [] @@ -618,30 +622,25 @@ def args(**overrides): parser, ) - class Recipe: - def to_dict(self): - return {"mode": "compiler"} - def preprocess(path, *, language, config): assert path == source assert language == "fortran" assert config.compiler == "gfortran" - return "subroutine work()\nend subroutine work\n", Recipe() + return PreprocessResult( + source="subroutine work()\nend subroutine work\n", + recipe={"language": "fortran", "mode": "compiler"}, + ) + # A parse report reads each file through the shared reader, recipe included. source = tmp_path / "api.f90" source.write_text("subroutine ignored()\nend subroutine ignored\n", encoding="utf-8") - monkeypatch.setattr(prik_cli, "run_compiler_preprocessor_with_recipe", preprocess) - code, recipe = prik_cli._fortran_source_for_path( - source, - PreprocessingConfig(mode="compiler", compiler="gfortran"), - ) - assert "subroutine work" in code - assert recipe == {"mode": "compiler"} + monkeypatch.setattr(preprocessing_source, "preprocess_source", preprocess) report = prik_cli._parse_report( [str(source)], PreprocessingConfig(mode="compiler", compiler="gfortran"), ) - assert report[str(source)]["preprocessing_recipe"] == {"mode": "compiler"} + assert report[str(source)]["signatures"][0]["name"] == "work" + assert report[str(source)]["preprocessing_recipe"]["mode"] == "compiler" def test_cli_help_is_concise_and_points_to_detailed_help(): @@ -745,7 +744,6 @@ class Node: source.write_text("subroutine work(n)\n integer, intent(in) :: n\nend subroutine work\n", encoding="utf-8") (source.parent / "notes.txt").write_text("ignore", encoding="utf-8") - assert fortran_parser_cli._collect_extensions(tmp_path) == [source] report = fortran_parser_cli._parse_paths([str(tmp_path)]) assert list(report) == [str(source)] assert report[str(source)]["signatures"][0]["name"] == "work" @@ -909,7 +907,6 @@ class ParentFirstNode: source = tmp_path / "mini.f90" source.write_text("subroutine work(n)\n integer, intent(in) :: n\nend subroutine work\n", encoding="utf-8") - assert prik_cli._collect_extensions(tmp_path) == [source] assert prik_cli._expand_paths([str(tmp_path)]) == [source] monkeypatch.setattr(sys.stdout, "isatty", lambda: True) diff --git a/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py b/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py index 589b13fa4..704d8d948 100644 --- a/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py +++ b/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py @@ -12,6 +12,8 @@ from prik.cmake import cmake_module_dir from prik.parsers.fortran import FortranParseError +from prik.pipeline.sources import fortran_sources_to_semantic_modules +from prik.preprocessing import read_fortran_source import prik.cli as prik_cli from prik.parsers.fortran import cli as fortran_parser_cli from prik.preprocessing import ( @@ -232,7 +234,7 @@ def test_single_file_cli_resolves_direct_intrinsic_kind_rename_before_probing(tm encoding="utf-8", ) - project = prik_cli._parse_fortran_source_files([source], PreprocessingConfig()) + project = fortran_sources_to_semantic_modules([source], PreprocessingConfig()).project parsed = project.files[0] module = parsed.modules[0] @@ -265,10 +267,7 @@ def test_cli_cross_file_resolution_reaches_imported_derived_field_kinds(tmp_path encoding="utf-8", ) - project = prik_cli._parse_fortran_source_files( - [precision, records], - PreprocessingConfig(), - ) + project = fortran_sources_to_semantic_modules([precision, records], PreprocessingConfig()).project record_file = next(parsed for parsed in project.files if parsed.filename == str(records)) assert record_file.modules[0].derived_types[0].fields[0].kind == "8" @@ -669,26 +668,16 @@ def test_prik_main_public_api_modes_from_inline_source(tmp_path: Path, monkeypat assert "module m" in capsys.readouterr().out -def test_prik_fortran_source_for_path_raw_uses_utf8_and_internal_recipe(): - class RawPath: - def read_text(self, *, encoding): - assert encoding is not None - assert encoding.lower() == "utf-8" - return "subroutine raw()\nend subroutine raw\n" - - class RawPreprocessing: - uses_compiler = False +def test_a_fortran_source_read_as_written_carries_its_internal_recipe(tmp_path: Path): + """Without compiler preprocessing the text is read as UTF-8 and the macros are recorded.""" + path = tmp_path / "raw.f90" + path.write_text("subroutine raw()\n ! \u00e9\nend subroutine raw\n", encoding="utf-8") - def fortran_internal_recipe(self, received): - assert received is path - return {"mode": "internal"} + text = read_fortran_source(path, PreprocessingConfig(defines=["FLAG=1"])) - path = RawPath() - - assert prik_cli._fortran_source_for_path(path, RawPreprocessing()) == ( - "subroutine raw()\nend subroutine raw\n", - {"mode": "internal"}, - ) + assert text.source == path.read_text(encoding="utf-8") + assert text.recipe is not None and text.recipe["mode"] == "internal" + assert text.included_files == () def test_prik_probe_subcommand_dispatches_one_flag_driven_probe(monkeypatch, capsys): diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/assumed_shape_and_derived_args.json b/tests/fortran/infrastructure/parsing/fixtures/general/assumed_shape_and_derived_args.json index 54232eb2c..58c74de50 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/assumed_shape_and_derived_args.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/assumed_shape_and_derived_args.json @@ -2,7 +2,7 @@ "filename": "assumed_shape_and_derived_args.f90", "source": "subroutine fill_grid(x)\n integer, intent(inout) :: x(0:,0:)\nend subroutine fill_grid\n\nsubroutine update_plane(x)\n real, intent(inout), dimension(0:, 1:n) :: x\n integer, intent(in) :: n\nend subroutine update_plane\n\nsubroutine step(state)\n type(sim_state), intent(inout) :: state\nend subroutine step\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [], "submodules": [], "programs": [], diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/basic_subroutine.json b/tests/fortran/infrastructure/parsing/fixtures/general/basic_subroutine.json index ffc4b1fc9..06c978ea0 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/basic_subroutine.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/basic_subroutine.json @@ -2,7 +2,7 @@ "filename": "basic_subroutine.f90", "source": "module m1\ncontains\nsubroutine add1(n, x)\n integer, intent(in) :: n\n real(kind=8), intent(inout), dimension(n) :: x\nend subroutine add1\nend module m1\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [ { "name": "m1", diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_all_exprs.json b/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_all_exprs.json index 62bbe02f1..7cea2791b 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_all_exprs.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_all_exprs.json @@ -2,7 +2,7 @@ "filename": "compile_time_all_exprs.f90", "source": "module expr_mod\n integer, parameter :: a = 8\n integer, parameter :: b = 3\n integer, parameter :: c = 2\n integer, parameter :: p_add = a + b\n integer, parameter :: p_sub = a - b\n integer, parameter :: p_mul = b * c\n integer, parameter :: p_div = a / c\n integer, parameter :: p_pow = c ** b\n integer, parameter :: p_mix = (a + b) * c - 1\ncontains\n subroutine all_exprs(x1, x2, x3, x4, x5, x6, x7, x8, x9)\n integer, intent(inout) :: x1(1:p_add)\n integer, intent(inout) :: x2(1:p_sub)\n integer, intent(inout) :: x3(1:p_mul)\n integer, intent(inout) :: x4(1:p_div)\n integer, intent(inout) :: x5(1:p_pow)\n integer, intent(inout) :: x6(0:p_mix)\n integer, intent(inout) :: x7(1:-(-a + b))\n integer, intent(inout) :: x8(1:(a+b)*(c+1)-1)\n integer, intent(inout) :: x9(1:(a-b)*(a-c))\n end subroutine all_exprs\nend module expr_mod\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [ { "name": "expr_mod", diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_shape_exprs.json b/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_shape_exprs.json index 90ef0c153..12828c892 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_shape_exprs.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/compile_time_shape_exprs.json @@ -2,7 +2,7 @@ "filename": "compile_time_shape_exprs.f90", "source": "module dims_mod\n integer, parameter :: n0 = 4\n integer, parameter :: n1 = n0 + 2\ncontains\n subroutine use_expr(x, y)\n integer, intent(inout) :: x(0:n1-1)\n real, intent(inout), dimension(1:n0*2) :: y\n end subroutine use_expr\nend module dims_mod\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [ { "name": "dims_mod", diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/derived_type.json b/tests/fortran/infrastructure/parsing/fixtures/general/derived_type.json index 24262a6e3..d41a6e7a0 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/derived_type.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/derived_type.json @@ -2,7 +2,7 @@ "filename": "derived_type.f90", "source": "module particle_mod\n type :: particle\n integer :: id\n real(kind=8), dimension(3) :: x\n contains\n procedure :: move, reset\n end type particle\ncontains\nsubroutine touch(p)\n type(particle), intent(inout) :: p\nend subroutine touch\nend module particle_mod\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [ { "name": "particle_mod", diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/derived_types_and_methods.json b/tests/fortran/infrastructure/parsing/fixtures/general/derived_types_and_methods.json index 609631403..78b17c684 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/derived_types_and_methods.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/derived_types_and_methods.json @@ -2,7 +2,7 @@ "filename": "derived_types_and_methods.f90", "source": "module mesh_mod\n type :: node\n integer :: id\n real(kind=8), dimension(3) :: xyz\n contains\n procedure :: move\n end type node\n\n type :: mesh\n integer :: nnodes\n type(node), allocatable :: nodes(:)\n contains\n procedure :: init, clear\n end type mesh\nend module mesh_mod\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [ { "name": "mesh_mod", diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/f77_subroutine.json b/tests/fortran/infrastructure/parsing/fixtures/general/f77_subroutine.json index 98aeca83b..caac0c074 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/f77_subroutine.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/f77_subroutine.json @@ -2,7 +2,7 @@ "filename": "f77_subroutine.f", "source": " subroutine daxpy(n,a,x,y)\n integer n\n double precision a,x(n),y(n)\n do 10 i=1,n\n 1y(i)=y(i)+a*x(i)\n 10 continue\n return\n end\n", "encoding": "utf-8", - "format": "f77", + "format": "fixed", "modules": [], "submodules": [], "programs": [], diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/modern_pyi_example.json b/tests/fortran/infrastructure/parsing/fixtures/general/modern_pyi_example.json index 5e6ca98e4..c38659546 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/modern_pyi_example.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/modern_pyi_example.json @@ -2,7 +2,7 @@ "filename": "modern_pyi_example.f90", "source": "module modern_math_physics\n implicit none\n private\n public :: particle, vector3, counter, init_particle, kinetic_energy, scale_vector, dot3, fill_identity3, normalize_particle\n\n integer :: counter\n real(8) :: hidden_scale\n\n type :: particle\n integer :: id\n real(8) :: mass\n real(8), dimension(3) :: position\n end type particle\n\n type :: vector3\n real(8), dimension(3) :: values\n end type vector3\n\n type :: hidden_state\n integer :: code\n end type hidden_state\n\ncontains\n\n subroutine init_particle(p, pid, mass, x, y, z)\n type(particle), intent(out) :: p\n integer, intent(in) :: pid\n real(8), intent(in) :: mass, x, y, z\n p%id = pid\n p%mass = mass\n p%position = [x, y, z]\n end subroutine init_particle\n\n function kinetic_energy(p, vx, vy, vz) result(e)\n type(particle), intent(in) :: p\n real(8), intent(in) :: vx, vy, vz\n real(8) :: e\n e = 0.5d0 * p%mass * (vx*vx + vy*vy + vz*vz)\n end function kinetic_energy\n\n subroutine scale_vector(v, alpha)\n real(8), dimension(:), intent(inout) :: v\n real(8), intent(in) :: alpha\n v = alpha * v\n end subroutine scale_vector\n\n function dot3(a, b) result(s)\n real(8), dimension(3), intent(in) :: a, b\n real(8) :: s\n s = a(1)*b(1) + a(2)*b(2) + a(3)*b(3)\n end function dot3\n\n subroutine fill_identity3(a)\n real(8), dimension(3,3), intent(out) :: a\n a = 0.0d0\n a(1,1) = 1.0d0\n a(2,2) = 1.0d0\n a(3,3) = 1.0d0\n end subroutine fill_identity3\n\n subroutine normalize_particle(p)\n type(particle), intent(inout) :: p\n real(8) :: n\n n = sqrt(dot3(p%position, p%position))\n if (n > 0.0d0) p%position = p%position / n\n end subroutine normalize_particle\n\n subroutine hidden_proc(x)\n integer, intent(in) :: x\n end subroutine hidden_proc\n\nend module modern_math_physics\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [ { "name": "modern_math_physics", diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/module_vars_use.json b/tests/fortran/infrastructure/parsing/fixtures/general/module_vars_use.json index 25322bb60..56311a2aa 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/module_vars_use.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/module_vars_use.json @@ -2,7 +2,7 @@ "filename": "module_vars_use.f90", "source": "module constants_mod\n use iso_c_binding, only: c_int, c_double\n integer(kind=c_int), parameter :: nmax = 100\n real(kind=c_double), dimension(3) :: origin\nend module constants_mod\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [ { "name": "constants_mod", diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/procedures_and_functions.json b/tests/fortran/infrastructure/parsing/fixtures/general/procedures_and_functions.json index 498e3c573..dc8a57465 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/procedures_and_functions.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/procedures_and_functions.json @@ -2,7 +2,7 @@ "filename": "procedures_and_functions.f90", "source": "module math_mod\ncontains\n function norm2(x) result(res)\n real(kind=8), intent(in) :: x(:)\n real(kind=8) :: res\n end function norm2\n\n subroutine scale(a, x)\n real(kind=8), intent(in) :: a\n real(kind=8), intent(inout) :: x(:)\n end subroutine scale\nend module math_mod\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [ { "name": "math_mod", diff --git a/tests/fortran/infrastructure/parsing/fixtures/general/scope_name_reuse_combinations.json b/tests/fortran/infrastructure/parsing/fixtures/general/scope_name_reuse_combinations.json index 041f58329..6af745938 100644 --- a/tests/fortran/infrastructure/parsing/fixtures/general/scope_name_reuse_combinations.json +++ b/tests/fortran/infrastructure/parsing/fixtures/general/scope_name_reuse_combinations.json @@ -2,7 +2,7 @@ "filename": "scope_name_reuse_combinations.f90", "source": "module scope_name_reuse_combinations\n implicit none\n\n type :: same_name\n integer :: payload\n end type same_name\n\n integer :: same_name_i\n real :: same_name_r\n logical :: same_name_l\n complex :: same_name_c\n character(len=8) :: same_name_s\n\n interface do_work\n module procedure do_work_i\n module procedure do_work_r\n module procedure do_work_l\n end interface do_work\n\ncontains\n\n subroutine do_work_i(same_name)\n implicit none\n integer, intent(inout) :: same_name\n real :: shared\n\n shared = same_name\n call nested_block(shared)\n\n contains\n\n subroutine nested_block(shared)\n implicit none\n real, intent(inout) :: shared\n character(len=12) :: same_name\n\n same_name = 'inner-string'\n shared = shared + len_trim(same_name)\n end subroutine nested_block\n\n end subroutine do_work_i\n\n subroutine do_work_r(same_name)\n implicit none\n real, intent(in) :: same_name\n logical :: shared\n\n shared = same_name > 0.0\n end subroutine do_work_r\n\n subroutine do_work_l(same_name)\n implicit none\n logical, intent(in) :: same_name\n type(same_name) :: shared\n\n if (same_name) then\n shared%payload = 1\n else\n shared%payload = 0\n end if\n end subroutine do_work_l\n\n\n subroutine host_one(same_name)\n implicit none\n integer, intent(inout) :: same_name\n\n call helper(same_name)\n\n contains\n\n subroutine helper(same_name)\n implicit none\n integer, intent(inout) :: same_name\n same_name = same_name + 1\n end subroutine helper\n\n end subroutine host_one\n\n subroutine host_two(same_name)\n implicit none\n real, intent(inout) :: same_name\n\n call helper(same_name)\n\n contains\n\n subroutine helper(same_name)\n implicit none\n real, intent(inout) :: same_name\n same_name = same_name + 1.0\n end subroutine helper\n\n end subroutine host_two\n\n function convert_to_complex(same_name) result(shared)\n implicit none\n integer, intent(in) :: same_name\n complex :: shared\n\n shared = cmplx(real(same_name), -real(same_name))\n end function convert_to_complex\n\n function convert_to_char(same_name) result(shared)\n implicit none\n real, intent(in) :: same_name\n character(len=16) :: shared\n\n write(shared, '(f6.2)') same_name\n end function convert_to_char\n\n function convert_to_logical(same_name) result(shared)\n implicit none\n character(len=*), intent(in) :: same_name\n logical :: shared\n\n shared = len_trim(same_name) > 0\n end function convert_to_logical\n\nend module scope_name_reuse_combinations\n", "encoding": "utf-8", - "format": "modern", + "format": "free", "modules": [ { "name": "scope_name_reuse_combinations", diff --git a/tests/fortran/infrastructure/parsing/test_error_handling.py b/tests/fortran/infrastructure/parsing/test_error_handling.py index 65221dce5..14f835d38 100644 --- a/tests/fortran/infrastructure/parsing/test_error_handling.py +++ b/tests/fortran/infrastructure/parsing/test_error_handling.py @@ -272,7 +272,7 @@ def test_f77_source_with_module_keyword_is_parsed(): end module bad_module """ parsed = parse_fortran_file(code, filename="legacy.f77") - assert parsed.format == "f77" + assert parsed.format == "fixed" assert parsed.modules[0].name == "bad_module" diff --git a/tests/fortran/infrastructure/parsing/test_public_entrypoints.py b/tests/fortran/infrastructure/parsing/test_public_entrypoints.py index 1fc1fad8d..4b1ef48ab 100644 --- a/tests/fortran/infrastructure/parsing/test_public_entrypoints.py +++ b/tests/fortran/infrastructure/parsing/test_public_entrypoints.py @@ -91,9 +91,10 @@ def test_file_path_and_unknown_filename_public_parse_paths(tmp_path): parse_fortran_file(12345) assert parsed_from_path.filename == str(source_path) - assert parsed_from_path.format == "modern" + # A file records the source form its lexer read: by suffix, else by its text. + assert parsed_from_path.format == "free" assert parsed_from_path.procedures[0].name == "from_path" - assert parsed_unknown_suffix.format == "unknown" + assert parsed_unknown_suffix.format == "free" def test_public_instance_visitor_entrypoints_use_source_strings(): diff --git a/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py b/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py index b002dc991..15b281a68 100644 --- a/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py +++ b/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py @@ -134,9 +134,6 @@ def test_source_preparation_rejects_raw_cpp_and_preserves_root_units_and_source_ "subroutine global_step()", "end subroutine global_step", ] - assert parser._source_form("fixed.f") == "f77" - assert parser._source_form("modern.f90") == "modern" - assert parser._source_form(None) == "unknown" source_path = tmp_path / "path_input.f90" source_path.write_text("module from_path\nend module from_path\n", encoding="utf-8") diff --git a/tests/fortran/infrastructure/pipeline/test_source_route_parity.py b/tests/fortran/infrastructure/pipeline/test_source_route_parity.py new file mode 100644 index 000000000..0e9d4b35c --- /dev/null +++ b/tests/fortran/infrastructure/pipeline/test_source_route_parity.py @@ -0,0 +1,72 @@ +"""A build and ``prik generate`` turn the same Fortran sources into the same modules. + +Both read, parse, measure, convert, and select through one source route, so +the contract generated from a project describes what a build of it wraps. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from prik import cli as prik_cli +from prik.pipeline.build import _fortran_wrapper_module +from prik.preprocessing import PreprocessingConfig + +USER = """module user + use kinds, only: wp + implicit none +contains + subroutine scale(x, factor) + real(wp), intent(inout) :: x + real(wp), intent(in) :: factor + x = x * factor + end subroutine scale + subroutine hidden() + end subroutine hidden +end module user +""" +KINDS = "module kinds\n integer, parameter :: wp = 8\nend module kinds\n" + + +def _shape(modules) -> dict[str, list[tuple[str, list[str | None]]]]: + """Return each module's functions with their argument kinds.""" + return { + module.name: [ + (function.name, [argument.semantic_type.name for argument in function.arguments]) + for function in module.functions + ] + for module in modules + } + + +@pytest.mark.parametrize("exports", [None, ("user::scale",)], ids=["whole-project", "selected"]) +def test_generate_and_build_convert_sources_named_out_of_order_alike(tmp_path: Path, exports): + user = tmp_path / "user.f90" + user.write_text(USER, encoding="utf-8") + kinds = tmp_path / "kinds.f90" + kinds.write_text(KINDS, encoding="utf-8") + sources = (user, kinds) + preprocessing = PreprocessingConfig() + + generated = prik_cli._converted_semantic_files( + [str(path) for path in sources], + preprocessing, + language="fortran", + export_symbols=exports, + ) + _parsed, _module, built, _dependencies = _fortran_wrapper_module( + sources, + preprocessing=preprocessing, + type_probe_preprocessing=preprocessing, + output_name="parity", + fortran_type_report=None, + fortran_type_probe_runner=None, + fortran_type_probe_cache_dir=None, + refresh_fortran_type_probe=False, + export_symbols=exports, + ) + + assert _shape(generated.available_modules) == _shape(built) + assert ("scale", ["Float64", "Float64"]) in _shape(built)["user"] diff --git a/tests/fortran/infrastructure/policy/test_wrapper_policy.py b/tests/fortran/infrastructure/policy/test_wrapper_policy.py index 7216d7213..f0e6a5b91 100644 --- a/tests/fortran/infrastructure/policy/test_wrapper_policy.py +++ b/tests/fortran/infrastructure/policy/test_wrapper_policy.py @@ -6,8 +6,8 @@ from tests.fortran._support.wrapper_build import wrapper_source from prik.planning import WrapperPlanner from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _fortran_source_for_pipeline, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.pipeline.pyi import pyi_file_to_semantic_module from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.semantics.models import ( @@ -41,7 +41,7 @@ def _source_semantic_module(filename: str, *, module_name: str): source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) diff --git a/tests/fortran/infrastructure/preprocessing/test_source_languages.py b/tests/fortran/infrastructure/preprocessing/test_source_languages.py new file mode 100644 index 000000000..bb857a2b1 --- /dev/null +++ b/tests/fortran/infrastructure/preprocessing/test_source_languages.py @@ -0,0 +1,43 @@ +"""Every entry point names the same Fortran sources and reads them in the same form. + +Source suffixes, the fixed or free form of a file, and the sources a list of +files and directories names have one owner, so discovery, project parsing, +the CLIs, and a build agree on a path. +""" + +from __future__ import annotations + +from pathlib import Path + +from prik import cli as prik_cli +from prik.parsers.fortran import parse_fortran_project +from prik.parsers.fortran import cli as fortran_parser_cli +from prik.parsers.fortran.module_sources import _searched_files +from prik.preprocessing.languages import FORTRAN_SOURCE_SUFFIXES, fortran_source_form, validated_source_paths + + +def test_every_entry_point_names_the_same_sources_under_a_directory(tmp_path: Path): + """A ``.fpp`` file is a Fortran source everywhere; a text file is one nowhere.""" + root = tmp_path / "src" + (root / "nested").mkdir(parents=True) + free = root / "b.f90" + free.write_text("module b\nend module b\n", encoding="utf-8") + fixed = root / "nested" / "a.fpp" + fixed.write_text(" module a\n end module a\n", encoding="utf-8") + (root / "notes.txt").write_text("not a source\n", encoding="utf-8") + expected = {free, fixed} + + assert set(prik_cli._expand_paths([str(root)])) == expected + assert {Path(name) for name in fortran_parser_cli._parse_paths([str(root)])} == expected + assert {Path(parsed.filename) for parsed in parse_fortran_project(root).files} == expected + assert set(_searched_files([root])) == {path.resolve() for path in expected} + assert set(validated_source_paths([root], FORTRAN_SOURCE_SUFFIXES, label="Fortran")) == expected + + +def test_a_parsed_file_records_the_form_its_lexer_read(): + """``.fpp`` is fixed form; an unknown suffix is decided by column 6.""" + parsed = parse_fortran_project({"a.fpp": " module a\n &\n end module a\n"}).files[0] + + assert parsed.format == fortran_source_form(parsed.source, "a.fpp") == "fixed" + assert fortran_source_form(" x = 1\n & + 2\n", "legacy.src") == "fixed" + assert fortran_source_form("x = 1\n", "modern.src") == "free" diff --git a/tests/fortran/optional_arguments/policy/test_optional_policy.py b/tests/fortran/optional_arguments/policy/test_optional_policy.py index 41839ca55..294d9ac50 100644 --- a/tests/fortran/optional_arguments/policy/test_optional_policy.py +++ b/tests/fortran/optional_arguments/policy/test_optional_policy.py @@ -4,8 +4,8 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from tests.fortran._support.wrapper_build import wrapper_source from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _fortran_source_for_pipeline, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.pipeline.pyi import pyi_file_to_semantic_module from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.semantics.models import ( @@ -25,7 +25,7 @@ def _source_semantic_module(filename: str, *, module_name: str): source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) diff --git a/tests/fortran/raw_addresses/policy/test_raw_address_policy.py b/tests/fortran/raw_addresses/policy/test_raw_address_policy.py index 7037aad9d..4dc79be48 100644 --- a/tests/fortran/raw_addresses/policy/test_raw_address_policy.py +++ b/tests/fortran/raw_addresses/policy/test_raw_address_policy.py @@ -4,8 +4,8 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from tests.fortran._support.wrapper_build import wrapper_source from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _fortran_source_for_pipeline, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.semantics.models import ( RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, @@ -33,7 +33,7 @@ def _source_semantic_module(filename: str, *, module_name: str): source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) diff --git a/tests/fortran/strings/codegen/test_string_input_lowering.py b/tests/fortran/strings/codegen/test_string_input_lowering.py index 362533d43..99832e9ad 100644 --- a/tests/fortran/strings/codegen/test_string_input_lowering.py +++ b/tests/fortran/strings/codegen/test_string_input_lowering.py @@ -144,15 +144,14 @@ def _source_route_plan(tmp_path, text: str, module_name: str): from prik.parsers.fortran.parser import parse_fortran_project from prik.pipeline.build import ( _apply_source_python_exports, - _fortran_source_for_pipeline, _merge_wrapper_modules, ) - from prik.preprocessing import PreprocessingConfig + from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules source = tmp_path / f"{module_name}.f90" source.write_text(text, encoding="utf-8") - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) diff --git a/tests/fortran/strings/policy/test_string_wrapper_policy.py b/tests/fortran/strings/policy/test_string_wrapper_policy.py index 9e5746515..09be15cf6 100644 --- a/tests/fortran/strings/policy/test_string_wrapper_policy.py +++ b/tests/fortran/strings/policy/test_string_wrapper_policy.py @@ -5,8 +5,8 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from tests.fortran._support.wrapper_build import wrapper_source from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _fortran_source_for_pipeline, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.semantics.models import ( RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, @@ -39,7 +39,7 @@ def _source_semantic_module(filename: str, *, module_name: str): source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) @@ -135,7 +135,7 @@ def _semantic_module_from_text(source_text: str, tmp_path: Path, *, module_name: """Complete policy for one inline Fortran source without a shared fixture.""" source = tmp_path / f"{module_name}.f90" source.write_text(source_text, encoding="utf-8") - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) diff --git a/tests/fortran/subroutines/policy/test_subroutine_output_policy.py b/tests/fortran/subroutines/policy/test_subroutine_output_policy.py index 2b0b01692..54aee1ac7 100644 --- a/tests/fortran/subroutines/policy/test_subroutine_output_policy.py +++ b/tests/fortran/subroutines/policy/test_subroutine_output_policy.py @@ -4,8 +4,8 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from tests.fortran._support.paths import FORTRAN_ROOT from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _fortran_source_for_pipeline, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.policy.ownership import ( NativeBarrierAction, @@ -32,7 +32,7 @@ def _source_semantic_module(filename: str, *, module_name: str): source = CALLS_NATIVE / filename - parsed = parse_fortran_project({str(source): _fortran_source_for_pipeline(source, PreprocessingConfig())}) + parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) module = _merge_wrapper_modules(modules, name=module_name) From 65f75a983ef64da5c1206b8311fbab46227a5fe5 Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 22:11:35 +0100 Subject: [PATCH 23/47] Verify Open MPI by configure run, fix callback and selection bugs, rewrite the mpi_f08 tutorial - The Open MPI integration test requires the configured tree to record the installation's configure run (date, host, user, command line) and reports missing, failing, or incomplete Open MPI tools as unavailable. It runs the tutorial's CLI commands and program; workflow tests cover its availability rules with fake tools. - Export selection drops use associations only removed declarations named. - Generic specifics resolve callback prototypes imported inside interface bodies, prototypes resolve types through their body's own use statements, selected contract sets include modules a prototype names through a module import, generics dispatch Python callables, and processor types keep their spelling under procedure-local imports. - The Open MPI mpi_f08 tutorial is rewritten around the Python API it produces, with the tested 4.1.8 and 5.0.11 versions. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 26 +- docs/user/tutorials/openmpi-f08.md | 489 +++++++++++++++--- prik/codegen/c/binding.py | 2 + prik/pipeline/pyi.py | 15 +- prik/policy/completion.py | 6 +- prik/policy/models.py | 3 + prik/semantics/fortran2ir.py | 51 +- prik/semantics/fortran_exports.py | 28 + .../fixtures/runtime/mpi_example.py | 44 ++ .../fixtures/runtime/openmpi_basic.py | 64 --- .../end_to_end/test_openmpi_f08.py | 211 +++++--- .../generic_callback_prototype_imports.f90 | 47 ++ ...test_generic_callback_prototype_imports.py | 72 +++ .../test_callback_route_resolution.py | 36 ++ .../test_fortran_export_symbol_selection.py | 42 ++ tests/workflows/test_openmpi_integration.py | 118 +++++ 16 files changed, 1013 insertions(+), 241 deletions(-) create mode 100644 tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py delete mode 100644 tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py create mode 100644 tests/fortran/callbacks/end_to_end/fixtures/native/generic_callback_prototype_imports.f90 create mode 100644 tests/fortran/callbacks/end_to_end/test_generic_callback_prototype_imports.py create mode 100644 tests/workflows/test_openmpi_integration.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 7795615b6..b2834c1d9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -121,13 +121,25 @@ release tags add a leading `v` to the package version. declarations (including the component and parent types they declare) and native scalar storage views through `T[()]`. Source builds and generated contracts publish the same selected surface. -- The Open MPI `mpi_f08` tutorial and opt-in two-rank integration test build a - wrapper from a restricted generated `.pyi` against a matching prebuilt - Open MPI installation and exercise NumPy communication and in-place reduction; - it is verified with Open MPI 4.1.2 and 5.0.11. Pull-request validation runs - it in an Open MPI Integration lane that builds Open MPI 4.1.8 and 5.0.11 from - source, and the test requires the configured tree and the installation to - share their version and Fortran compiler. +- A tutorial turns a reviewed part of Open MPI's Fortran `mpi_f08` interface + into a Python MPI API: it generates a restricted `.pyi` contract from the + configured Open MPI sources, builds it with the CLI against the installed + Open MPI without compiling any Open MPI source, and runs a two-rank NumPy + program under `mpirun`. An opt-in integration test runs the same commands + and program; the Open MPI Integration lane runs it against Open MPI 4.1.8 and + 5.0.11 built from source. The test requires the configured tree to record the + same configure run as the installation, and reports missing or failing + Open MPI tools as unavailable -- a skip locally, a failure where Open MPI is + required. +- Export selection drops a use association that only unselected declarations + were written with, so a selected contract no longer imports an unused type + under a lowercase alias. +- A generic whose specific takes a callback dispatches any Python callable to + it. A callback prototype imported by a `use` inside an interface body is + resolved, a prototype's own argument types resolve through the `use` + statements in its body, and a selected contract set includes the modules a + prototype names that way. A processor type such as `c_ptr` keeps its own + spelling when a procedure-local `use` imports it. - Export selection accepts a generic that shares its name with one of its specifics; the name selects the generic instead of being reported ambiguous. - A contract keeps a dotted comparison spelling such as `operator(.EQ.)` in diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index e5c9a89f3..fc093af18 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -1,34 +1,100 @@ --- -title: Wrap a prebuilt Open MPI mpi_f08 installation -description: Generate a restricted Fortran contract and build a Python extension against Open MPI +title: Wrap Open MPI mpi_f08 for Python +description: Turn a reviewed part of Open MPI's Fortran mpi_f08 interface into a Python MPI API audience: users -prerequisites: configured Open MPI source tree and matching installed Open MPI development files -related: ../guide/wrapping-modules.md, ../reference/cli-commands.md, ../reference/pyi-format.md +prerequisites: an Open MPI installation with mpi_f08, and the configured Open MPI source and build trees it was built from +related: ../guide/wrapping-modules.md, ../guide/callbacks.md, ../reference/cli-commands.md, ../reference/pyi-format.md status: maintained publication: reviewed --- -# Wrap Open MPI `mpi_f08` +# Wrap Open MPI `mpi_f08` for Python -Use a configured Open MPI source tree and the corresponding installed Open MPI -toolchain. PRIK reads `mpi-f08.F90` and finds the sources of the modules it -uses in the source and build trees, whatever their names and layout in your -Open MPI version, to generate a contract; the extension compiles against the -installed modules and libraries. +This tutorial starts from Open MPI's real Fortran `mpi_f08` interface and turns +a small, reviewed part of it into a Python MPI API. At the end, an ordinary +Python program runs under `mpirun` and communicates through it: -Set `PRIK_OPENMPI_SOURCE` to the matching Open MPI source root and -`PRIK_OPENMPI_BUILD` to its build root. Use a tree that has been built with -`make`, which generates the Fortran includes the sources read, such as -`configure-fortran-output.h` and `sizeof_f08.h`; `configure` also generates some -module sources into the build tree, so search both trees. Confirm that -`mpifort --showme:version` reports the same Open MPI version as the source -tree, and configure the tree with the Fortran compiler the installation was -built with, which `ompi_info --parsable` reports as `compiler:fortran:absolute`. +```python +import numpy as np -Select the public facade's small initial API: +from prik_openmpi_f08 import mpi_f08 as mpi -```bash -cat > exports.txt <<'EOF' +mpi.mpi_init() +world = mpi.mpi_comm_world +rank, _ = mpi.mpi_comm_rank(world) + +values = np.array([rank + 1, rank + 2], dtype=np.int32) +reduced = np.empty_like(values) +mpi.mpi_allreduce(values, reduced, np.int32(values.size), mpi.mpi_int, mpi.mpi_sum, world) + +mpi.mpi_finalize() +``` + +Nothing in this API is written by hand. Every function, handle type, and +constant is generated from the declarations in Open MPI's Fortran sources, and +every call enters the installed Open MPI library. If you know `mpi4py`, the +result will feel familiar: Python code driving the native MPI implementation. +It is not `mpi4py`, though. PRIK exposes the selected Fortran interface as it +is declared, so the names and argument lists are those of `mpi_f08`. + +## 1. See what PRIK reads + +`mpi_f08` is an ordinary Fortran module that gathers other modules: + +```fortran +module mpi_f08 + use mpi_f08_types + use mpi_f08_interfaces + use pmpi_f08_interfaces + use mpi_f08_callbacks + use mpi_f08_interfaces_callbacks +end module mpi_f08 +``` + +The handle types and predefined objects are declared in those modules: + +```fortran +type(MPI_Comm), parameter :: MPI_COMM_WORLD = MPI_Comm(OMPI_MPI_COMM_WORLD) +type(MPI_Op), parameter :: MPI_SUM = MPI_Op(OMPI_MPI_SUM) +type(MPI_Datatype), parameter :: MPI_INT = MPI_Datatype(OMPI_MPI_INT) + +integer, bind(C, name="mpi_fortran_in_place_") :: MPI_IN_PLACE +type(MPI_Status), bind(C, name="mpi_fortran_status_ignore_") :: MPI_STATUS_IGNORE +``` + +and each MPI routine is a generic interface over its specific procedures: + +```fortran +interface MPI_Allreduce + subroutine MPI_Allreduce_f08(sendbuf, recvbuf, count, datatype, op, comm, ierror) + use :: mpi_f08_types, only : MPI_Datatype, MPI_Op, MPI_Comm + type(*), dimension(*), intent(in) :: sendbuf + type(*), dimension(*) :: recvbuf + integer, intent(in) :: count + type(MPI_Datatype), intent(in) :: datatype + type(MPI_Op), intent(in) :: op + type(MPI_Comm), intent(in) :: comm + integer, optional, intent(out) :: ierror + end subroutine MPI_Allreduce_f08 +end interface MPI_Allreduce +``` + +These excerpts are simplified from Open MPI 5.0. PRIK does not need to be +told where each declaration lives. It starts from the entry source +`mpi-f08.F90`, follows every `use` to the source that defines that module, and +continues through the modules those use. File names play no part: Open MPI +declares its types in `mpi_f08_types` in one release series and in a +configure-generated `mpi_types` module in another, and PRIK finds whichever +the sources declare. Some of these declarations, such as `MPI_IN_PLACE` above, +are in headers that Open MPI's `configure` writes into its build tree. + +## 2. Choose a small Python surface + +`mpi_f08` publishes hundreds of routines and constants. Start with a reviewed +subset instead of all of them. Save the Fortran identities to publish in +`mpi_exports.txt`: + +```text mpi_f08::MPI_Init mpi_f08::MPI_Finalize mpi_f08::MPI_Comm_rank @@ -43,67 +109,328 @@ mpi_f08::MPI_DOUBLE_PRECISION mpi_f08::MPI_SUM mpi_f08::MPI_IN_PLACE mpi_f08::MPI_STATUS_IGNORE -EOF +``` + +This one list covers every kind of declaration the example needs: generic +procedures such as `MPI_Allreduce`, derived-type module objects such as +`MPI_COMM_WORLD`, datatype and operator constants, status storage, and the +special native storage `MPI_IN_PLACE`. + +Selecting symbols this way is not an MPI feature. `--export-symbols` accepts +module-qualified public symbols from any Fortran project -- procedures, +generics, and module variables -- and here every name is qualified by +`mpi_f08`, the facade that re-exports it. PRIK publishes exactly those names +and keeps whatever supporting declarations they need, such as the handle types +their signatures name, without publishing the rest of the API. + +## 3. Generate the contract + +PRIK reads the Fortran declarations from a configured Open MPI source tree and +its build tree, which must be the ones your installed Open MPI was built from; +[why it must match](#why-the-configured-tree-must-match-the-installation) +comes later. Point two shell variables at them: + +```bash +OMPI_SRC=/path/to/openmpi-5.0.11 +OMPI_BUILD=/path/to/openmpi-5.0.11/build +``` + +Generate the contract from the one entry source, letting PRIK discover the +modules it uses under both trees: +```bash python3 -m prik generate --pyi \ - "$PRIK_OPENMPI_SOURCE/ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90" \ - --module-source-dir "$PRIK_OPENMPI_SOURCE" \ - --module-source-dir "$PRIK_OPENMPI_BUILD" \ - --export-symbols exports.txt --out contract --compiler mpifort \ - -I "$PRIK_OPENMPI_BUILD" \ - -I "$PRIK_OPENMPI_BUILD/ompi/mpi/fortran/use-mpi-f08" \ - -I "$PRIK_OPENMPI_BUILD/ompi/mpi/fortran/use-mpi-f08/mod" \ - -I "$PRIK_OPENMPI_SOURCE" \ - -I "$PRIK_OPENMPI_BUILD/ompi/include" \ - -I "$PRIK_OPENMPI_SOURCE/ompi/include" -``` - -Build from the generated `contract/__init__.pyi`. Query the installed wrapper -compiler for its compiler command, module and include directories, remaining -compile flags, and ordered link arguments. The compiler command must be a single -executable; a command such as `ccache gfortran` is not a compiler plus flags: + "$OMPI_SRC/ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90" \ + --module-source-dir "$OMPI_SRC" \ + --module-source-dir "$OMPI_BUILD" \ + --export-symbols mpi_exports.txt \ + --out contract \ + --compiler mpifort \ + -I "$OMPI_BUILD" \ + -I "$OMPI_BUILD/ompi/mpi/fortran/use-mpi-f08" \ + -I "$OMPI_BUILD/ompi/mpi/fortran/use-mpi-f08/mod" \ + -I "$OMPI_SRC" \ + -I "$OMPI_BUILD/ompi/include" \ + -I "$OMPI_SRC/ompi/include" +``` + +The Open MPI sources are preprocessed Fortran that includes headers from both +trees: the hand-written ones stay in the source tree, and the configured ones +are generated into the build tree beside them. The `-I` options name those +directories so PRIK reads each source exactly as the Fortran compiler did. + +PRIK reads these sources to learn their declarations. It does not compile +them. + +## 4. Read the contract + +The `contract/` directory holds one editable `.pyi` file per Fortran module the +selection needs. `contract/mpi_f08.pyi` publishes the selected names: + +```python +from .mpi_f08_types import mpi_comm_world, mpi_double_precision, mpi_in_place, mpi_int, mpi_status_ignore, mpi_sum +from .mpi_f08_interfaces import mpi_allreduce, mpi_barrier, mpi_comm_rank, mpi_comm_size, mpi_finalize, mpi_init, mpi_recv, mpi_send +from .mpi_types import Mpi_Comm, Mpi_Datatype, Mpi_Op, Mpi_Status +``` + +These excerpts come from Open MPI 5.0, which declares the handle types in +`mpi_types`. Open MPI 4.1 declares them in `mpi_f08_types`, so there they are +imported from that module and written as `mpi_f08_types.Mpi_Datatype` and so +on. The predefined objects become typed module attributes in +`contract/mpi_f08_types.pyi`: + +```python +mpi_comm_world: Final[Mpi_Comm] + +mpi_sum: Final[Mpi_Op] + +mpi_int: Final[Mpi_Datatype] + +mpi_double_precision: Final[Mpi_Datatype] + +mpi_in_place: Int32[()] + +mpi_status_ignore: Mpi_Status +``` + +and each selected routine keeps its Fortran interface in +`contract/mpi_f08_interfaces.pyi`: ```python -import shlex -import subprocess - -from prik.pipeline.build import NativeLinkItem, build_pyi_extension - - -def show(option): - return shlex.split(subprocess.check_output(["mpifort", option], text=True)) - - -command, compile_flags = show("--showme:command"), show("--showme:compile") -if len(command) != 1: - raise SystemExit(f"mpifort wraps the multi-token command {command}; set one compiler executable") -build_pyi_extension( - "contract/__init__.pyi", - input_compiler=command[0], - native_include_dirs=show("--showme:incdirs"), - wrapper_fortran_flags=compile_flags, - native_link_items=[NativeLinkItem("linker_argument", flag) for flag in show("--showme:link")], - native_linker_language="fortran", - output_name="prik_openmpi_f08", - output_dir="build/openmpi", -) -``` - -Run a Python program under the matching Open MPI launcher. The selected -functions live in `prik_openmpi_f08.mpi_f08`; Fortran `Int32` arguments such as -counts and ranks use `numpy.int32` values. NumPy arrays provide the storage for -choice buffers. In the tested Open MPI 4.1.2 and 5.0.11 configurations, `mpi_in_place` is -a concrete integer module object exposed as a live rank-zero NumPy view, so -pass that view directly to `mpi_allreduce`. -A two-rank example lives at -`tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py`. -Run it with `PYTHONPATH=build/openmpi orterun -n 2 python3` followed by that -path, or use the matching Open MPI `mpirun` launcher. It exercises -send/receive, ordinary and in-place all-reduce, and the selected communicator -and datatype objects. - -The choice-buffer ABI follows the configured interface declaration: -`TYPE(*), DIMENSION(*)` passes a raw address, while assumed-shape and -assumed-rank declarations pass a C descriptor. Arrays of derived MPI handles, -persistent callbacks, and nonblocking buffer-lifetime management are outside -this initial surface. +@bind("MPI_Allreduce") +@overload("mpi_allreduce_f08") +def mpi_allreduce( + sendbuf: Annotated[AnyNative[Flat], ReadOnly], + recvbuf: AnyNative[Flat], + count: Int32, + datatype: mpi_types.Mpi_Datatype, + op: mpi_types.Mpi_Op, + comm: mpi_types.Mpi_Comm, + ierror: Int32[()] = ... +) -> Returns["ierror", Int32[()]] | None: ... +``` + +Three mappings are worth a closer look. + +**Handles are concrete types.** `Mpi_Comm`, `Mpi_Datatype`, `Mpi_Op`, and +`Mpi_Status` are the derived types Open MPI declares, with their real +components. For example, `Mpi_Comm` holds the one integer handle Open MPI +stores: + +```python +class Mpi_Comm: + def __init__( + self, + *, + mpi_val: Int32 = ... + ) -> None: ... +``` + +A handle argument accepts only an object of its own type, and the predefined +handles are `Final` module constants. + +**`MPI_IN_PLACE` is native storage.** Its declaration is a C-bound integer +module variable, so it becomes rank-zero native integer storage, `Int32[()]`. +Python sees `mpi.mpi_in_place` as a live NumPy scalar view of Open MPI's own +`MPI_IN_PLACE` variable. Passing that view as a buffer passes the address of +that variable, which is how Open MPI recognizes an in-place operation. +Nothing here knows about MPI: it is the same mapping any Fortran module +variable declared this way receives. + +**Choice buffers are `AnyNative`.** The send and receive buffers of MPI +routines are `type(*)` dummies, which accept data of any type, so they become +`AnyNative[Flat]`: any NumPy array, passed as a raw address. `AnyNative` +appears only for these assumed-type dummies; `MPI_IN_PLACE` is a concrete +module object with a concrete type. + +The trailing `ierror` argument is optional. Leave it out and the call returns +`None` instead of the error code; Open MPI's default error handler aborts on +errors anyway. + +## 5. Build from the contract + +The contract, not the Open MPI sources, is what the extension is built from: + +```text +Open MPI Fortran sources + | +PRIK semantic analysis + | +restricted, editable .pyi contract + | +PRIK wrapper generation + | +Python extension linked to the installed Open MPI +``` + +Build `contract/__init__.pyi`, asking the installed `mpifort` wrapper for the +compiler it wraps, the flags that find Open MPI's Fortran modules, and the +libraries to link: + +```bash +python3 -m prik contract/__init__.pyi \ + --compiler "$(mpifort --showme:command)" \ + --wrapper-fortran-flags="$(mpifort --showme:compile)" \ + --native-library $(mpifort --showme:libs) \ + --native-library-dir $(mpifort --showme:libdirs) \ + --out prik_openmpi_f08 \ + --out-dir build +``` + +This writes `prik_openmpi_f08.so` in the current directory. It compiles only +the bridge and binding PRIK generates; no Open MPI source is compiled. The +installed Open MPI already provides the implementation, and the extension +links against its libraries. `--compiler` takes the compiler `mpifort` runs +rather than `mpifort` itself because PRIK identifies a Fortran compiler's +family from its executable name. + +## 6. Write an MPI program + +This program uses every selected routine: rank 0 sends a NumPy array to +rank 1, and then every rank takes part in three reductions, the last one in +place. + + +```python +import numpy as np + +from prik_openmpi_f08 import mpi_f08 as mpi + +mpi.mpi_init() +world = mpi.mpi_comm_world + +rank, _ = mpi.mpi_comm_rank(world) +size, _ = mpi.mpi_comm_size(world) +rank, size = int(rank), int(size) + +# Point to point: rank 0 sends four integers, rank 1 receives them. +if rank == 0: + sent = np.array([3, 5, 7, 11], dtype=np.int32) + mpi.mpi_send(sent, np.int32(sent.size), mpi.mpi_int, np.int32(1), np.int32(13), world) +elif rank == 1: + received = np.empty(4, dtype=np.int32) + mpi.mpi_recv( + received, + np.int32(received.size), + mpi.mpi_int, + np.int32(0), + np.int32(13), + world, + mpi.mpi_status_ignore, + ) + print(f"rank 1 received {received.tolist()}") + +# Collective: every rank contributes and every rank receives the sum. +values = np.array([rank + 1, rank + 2], dtype=np.int32) +reduced = np.empty_like(values) +mpi.mpi_allreduce(values, reduced, np.int32(values.size), mpi.mpi_int, mpi.mpi_sum, world) + +readings = np.array([20.0 + rank], dtype=np.float64) +total = np.empty_like(readings) +mpi.mpi_allreduce(readings, total, np.int32(readings.size), mpi.mpi_double_precision, mpi.mpi_sum, world) + +# In place: MPI_IN_PLACE as the send buffer reduces the receive buffer itself. +in_place = values.copy() +mpi.mpi_allreduce(mpi.mpi_in_place, in_place, np.int32(in_place.size), mpi.mpi_int, mpi.mpi_sum, world) + +mpi.mpi_barrier(world) +print(f"rank {rank} of {size}: sum {reduced.tolist()}, in place {in_place.tolist()}, total {total.tolist()}") +mpi.mpi_finalize() +``` + +Two details follow from the contract: + +- Counts, ranks, and tags are Fortran `INTEGER` values, which the contract + declares as `Int32`. Pass them as `np.int32`, the exact NumPy type, and use + `np.int32` arrays with `mpi_int` and `np.float64` arrays with + `mpi_double_precision`, so the element type matches the MPI datatype. +- `mpi_comm_rank` returns its result with the optional error code, which is + `None` when omitted, so `rank, _ = ...` unpacks it. `rank` is a NumPy + integer; `int(rank)` gives a plain Python value. + +Save it as `mpi_example.py` beside `prik_openmpi_f08.so`. + +## 7. Run it under Open MPI + +Start two ranks with the installed Open MPI launcher: + +```bash +mpirun -n 2 python3 mpi_example.py +``` + +The two ranks print these lines, in whichever order they finish: + +```text +rank 1 received [3, 5, 7, 11] +rank 0 of 2: sum [3, 5], in place [3, 5], total [41.0] +rank 1 of 2: sum [3, 5], in place [3, 5], total [41.0] +``` + +Here is what happened. `mpirun` started two Python processes as MPI ranks. +Each call went through PRIK's generated binding and bridge straight into the +installed Open MPI library. The NumPy arrays crossed the boundary as native +buffers: `mpi_recv` wrote into rank 1's `received`, and each `mpi_allreduce` +wrote into `reduced`, `total`, and `in_place` directly. No part of Open MPI +was rebuilt. + +## Why the configured tree must match the installation + +`mpi_f08` is not the same text in every Open MPI build. `configure` decides +details of the Fortran interface for the compiler and options it is given, and +writes some of the Fortran sources and headers PRIK reads -- the +`MPI_IN_PLACE` declaration shown earlier is one of them. Two builds of the +same Open MPI version with the same `gfortran` but different Fortran flags can +therefore declare different interfaces. PRIK has to read the declarations of +the build it links against, so the source and build trees must be the ones +that installation was configured from; the version number alone does not +guarantee that. + +Open MPI records each configure run in the installation and in the build tree, +so you can compare them. The installation reports when, where, by whom, and +with which command line it was configured: + +```bash +ompi_info --parsable | grep '^config:\(timestamp\|host\|user\|cli\)' +``` + +and the build tree records the same run: + +```bash +grep '^OPAL_CONFIGURE_\(DATE\|HOST\|USER\)' "$OMPI_BUILD/Makefile" +grep OPAL_CONFIGURE_CLI "$OMPI_BUILD/opal/include/opal/version.h" +``` + +The simplest way to have a matching pair is to build Open MPI yourself and +keep its trees: + +```bash +tar -xjf openmpi-5.0.11.tar.bz2 +mkdir openmpi-5.0.11/build && cd openmpi-5.0.11/build +../configure --prefix="$HOME/openmpi-5.0.11" --enable-mpi-fortran=usempif08 FC=gfortran +make -j4 && make install +``` + +PRIK's Open MPI integration test runs the commands in this tutorial and checks +this relationship before it starts: it requires the configured tree and the +installation to record the same configure run. Continuous integration runs it +against Open MPI 4.1.8 and 5.0.11 built this way. Those are the configurations +exercised in CI, not the only ones that can work. + +## Limitations + +This tutorial selected fourteen names; the rest of `mpi_f08` works the same +way when you select it, within these limits of what PRIK supports today: + +- **Arrays of handles.** Routines taking an array of derived-type values, such + as the request and status arrays of `MPI_Waitall`, are not supported: + building a contract that selects one stops with an error naming the + argument. +- **Stored callbacks.** PRIK passes a Python callable as a callback that is + valid only during the call it is passed to. MPI keeps some callbacks for + later -- the copy and delete functions of `MPI_Comm_create_keyval`, or the + function given to `MPI_Op_create` -- and calls them after that call has + returned, which is not supported. See [Callbacks](../guide/callbacks.md). +- **Nonblocking buffers.** Routines such as `MPI_Isend` wrap, but the + operation keeps using its buffer after the call returns. PRIK does not hold + on to that NumPy array, so your program must keep it alive and unchanged + until the operation completes. diff --git a/prik/codegen/c/binding.py b/prik/codegen/c/binding.py index 7591e8884..2c8850cf6 100644 --- a/prik/codegen/c/binding.py +++ b/prik/codegen/c/binding.py @@ -15876,6 +15876,8 @@ def _overload_required_argument_condition( namespace = self._type_namespace(match.derived_type_identity) expected = f"PyDict_GetItemString(PyModule_GetDict({namespace}), {class_name})" return f"{expected} != NULL && (PyObject *)Py_TYPE({value}) == {expected}" + if match.kind is OverloadMatchKind.CALLBACK: + return f"PyCallable_Check({value})" if match.kind is OverloadMatchKind.NUMPY_ARRAY: numpy_type = PrimitiveScalarTypeRegistry.type_for(match.semantic_type_name).numpy_type_macro return ( diff --git a/prik/pipeline/pyi.py b/prik/pipeline/pyi.py index b7d0531ff..37186bd8a 100644 --- a/prik/pipeline/pyi.py +++ b/prik/pipeline/pyi.py @@ -177,11 +177,16 @@ def _add_imported_contract_dependencies( pending = list(emitted_modules.values()) while pending: module = pending.pop(0) - dependency_names = { - statement.module.lstrip(".").casefold() - for statement in module.imports - if isinstance(statement, SemanticImport) and statement.module.startswith(".") - } + dependency_names = set() + for statement in module.imports: + if not isinstance(statement, SemanticImport) or not statement.module.startswith("."): + continue + package_module = statement.module.lstrip(".") + if package_module: + dependency_names.add(package_module.casefold()) + else: + # ``from . import name`` binds sibling modules, each a dependency. + dependency_names.update(item.source.casefold() for item in statement.items) for dependency_name in sorted(dependency_names): dependency = available.get(dependency_name) if dependency is None or dependency.name in emitted_modules: diff --git a/prik/policy/completion.py b/prik/policy/completion.py index 2b85bd301..36b13b32a 100644 --- a/prik/policy/completion.py +++ b/prik/policy/completion.py @@ -983,13 +983,17 @@ def _overload_argument_match( elif kind is ObjectKind.DERIVED_TYPE and argument.derived is not None: match_kind = OverloadMatchKind.DERIVED derived_identity = argument.derived.type_identity + elif argument.callback is not None: + match_kind = OverloadMatchKind.CALLBACK if match_kind is None: return None return OverloadArgumentPolicy( python_name=argument.python_name, kind=match_kind, optional=argument.optional_mode not in {OptionalMode.REQUIRED, OptionalMode.REQUIRED_DESCRIPTOR}, - semantic_type_name=argument.semantic_type_name, + # Every callable passes the same runtime test, so callbacks compare + # alike whatever prototype each names. + semantic_type_name="Callable" if match_kind is OverloadMatchKind.CALLBACK else argument.semantic_type_name, rank=argument.rank, derived_type_identity=derived_identity, scalar_actual_mode=argument.scalar_actual_mode, diff --git a/prik/policy/models.py b/prik/policy/models.py index 0bde49d44..4a6b52428 100644 --- a/prik/policy/models.py +++ b/prik/policy/models.py @@ -564,6 +564,9 @@ class OverloadMatchKind(str, Enum): NUMPY_ARRAY = "numpy_array" STRING = "string" DERIVED = "derived" + # Any Python callable. A callable carries no prototype to test, so two + # candidates that differ only in their callback types cannot be told apart. + CALLBACK = "callback" @dataclass(frozen=True) diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index ab6080989..182e17d7f 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -871,6 +871,28 @@ def _module_callback_interfaces( public = cls._module_public_names(module, modules) return {name: resolved for name, resolved in visible.items() if resolved.visible_name.casefold() in public} + @classmethod + def _procedure_callback_interfaces( + cls, + modules: dict[str, FortranModule], + module: FortranModule, + procedure: FortranProcedureSignature, + ) -> dict[str, _CallbackInterface]: + """Return the interfaces one procedure of ``module`` can name as a callback's type. + + A ``use`` written in the procedure, or in the interface body declaring + it, names an interface only there, so each procedure -- contained, + separate, or a generic's specific -- resolves against its module's + interfaces and its own imports. + """ + return cls._scope_callback_interfaces( + modules, + procedure.uses, + base=cls._module_callback_interfaces(modules, module), + owner=module, + scope_name=procedure.name, + ) + @classmethod def _scope_callback_interfaces( cls, @@ -1273,12 +1295,15 @@ def _module_prototypes( if identity in seen: continue seen.add(identity) - arguments = [self.visit(item, derived_type_context=context) for item in signature.arguments] + # An interface body reads types through its own ``use`` too, + # exactly as a callback argument naming it does. + signature_context = self._procedure_derived_type_context(signature, context) + arguments = [self.visit(item, derived_type_context=signature_context) for item in signature.arguments] for source_argument, argument in zip(signature.arguments, arguments, strict=True): self._normalize_callback_reference_storage(argument, source_argument) self._record_prototype_argument_intent(argument, source_argument) return_type = ( - self.visit(signature.result, derived_type_context=context, as_type=True) + self.visit(signature.result, derived_type_context=signature_context, as_type=True) if signature.result is not None else SemanticType("None", dtype="None") ) @@ -1561,22 +1586,13 @@ def _visit_FortranModule( index = module_index if module_index is not None else self._callback_module_index([module]) context = self._module_derived_type_context(module, index) self._record_abstract_type_names(module) - callback_interfaces = self._module_callback_interfaces(index, module) source_procedures = list(self._module_procedures(module)) semantic_functions = [ self.visit( proc, visibility=self._symbol_visibility(module, proc.name), derived_type_context=context, - # A procedure-local ``use`` names an interface only inside that - # procedure, so each one resolves against its own imports. - callback_interfaces=self._scope_callback_interfaces( - index, - proc.uses, - base=callback_interfaces, - owner=module, - scope_name=proc.name, - ), + callback_interfaces=self._procedure_callback_interfaces(index, module, proc), ) for proc in source_procedures ] @@ -2540,7 +2556,8 @@ def _derived_type_ref( return None wrapped = not origin.processor and (origin.module.lower(), origin.name.lower()) in self.wrapped_derived_types public_name = local_name - if origin.import_scope == "procedure": + # A processor type has no contract module to qualify it through. + if origin.import_scope == "procedure" and not origin.processor: public_name = f"{origin.module}.{origin.name}" metadata: dict[str, object] = { "name": origin.name, @@ -3342,6 +3359,7 @@ def _module_overload_sets( signature, visibility=self._symbol_visibility(module, signature.name), derived_type_context=context, + callback_interfaces=self._procedure_callback_interfaces(module_index or {}, module, signature), ) for signature in interface.procedures } @@ -3863,7 +3881,12 @@ def _inherited_generic_specifics( signature = signatures.get(name.casefold()) if signature is None or target.key in lookup: continue - function = self.visit(signature, visibility="private", derived_type_context=source_context) + function = self.visit( + signature, + visibility="private", + derived_type_context=source_context, + callback_interfaces=self._procedure_callback_interfaces(modules, source_module, signature), + ) lookup[target.key] = function inherited.append(target) return inherited, lookup diff --git a/prik/semantics/fortran_exports.py b/prik/semantics/fortran_exports.py index ed9788c3d..471e9d3b4 100644 --- a/prik/semantics/fortran_exports.py +++ b/prik/semantics/fortran_exports.py @@ -12,8 +12,10 @@ ProcedureOverloadSet, SemanticFunction, SemanticModule, + _module_semantic_types, _semantic_type_tree, ) +from prik.utilities.declaration_expressions import declaration_expression_identifiers _FORTRAN_IDENTIFIER = r"[A-Za-z][A-Za-z0-9_]*" @@ -78,6 +80,7 @@ def select_fortran_export_symbols( required_types = _with_dependent_types(_required_type_identities(primary_modules, selected), module_index) for module in primary_modules: _retain_required_types(module, module_index[_native_module_name(module)], required_types) + _drop_unused_imports(module) # Root selection owns only the requested symbol surface. Contract-import # completion already owns which available modules selected declarations @@ -89,6 +92,31 @@ def select_fortran_export_symbols( return FortranExportSelection(tuple(primary_sources), tuple(primary_modules), context_modules) +def _drop_unused_imports(module: SemanticModule) -> None: + """Keep only the use associations the declarations a selection retains still name. + + A module's imports record what its source declarations were written with. + Selection removes declarations, and a name only a removed declaration used + would otherwise stay imported with nothing in the contract to name it. + """ + used = {reexport.local_name.casefold() for reexport in module.reexports} + for semantic_type in _module_semantic_types(module): + used.add(semantic_type.name.casefold()) + reference = semantic_type.metadata.get(EXTERNAL_TYPE_REF_METADATA) + if isinstance(reference, dict): + used.update(str(reference.get(key) or "").casefold() for key in ("name", "local_name")) + array = semantic_type.storage.array if semantic_type.storage is not None else None + if array is None: + continue + for extent in (*array.shape, *array.lower_bounds, *array.upper_bounds): + if extent is not None: + used.update(name.casefold() for name in declaration_expression_identifiers(str(extent))) + used.update(item.name.casefold() for axis in array.expression_callables for item in axis) + for statement in module.imports: + statement.items = [item for item in statement.items if (item.target or item.source).casefold() in used] + module.imports = [statement for statement in module.imports if statement.items] + + def _selected_identities(requested, module_index): """Add the declaring identity of each name selected through a facade re-export.""" selected = set(requested) diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py new file mode 100644 index 000000000..efb295c05 --- /dev/null +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py @@ -0,0 +1,44 @@ +import numpy as np + +from prik_openmpi_f08 import mpi_f08 as mpi + +mpi.mpi_init() +world = mpi.mpi_comm_world + +rank, _ = mpi.mpi_comm_rank(world) +size, _ = mpi.mpi_comm_size(world) +rank, size = int(rank), int(size) + +# Point to point: rank 0 sends four integers, rank 1 receives them. +if rank == 0: + sent = np.array([3, 5, 7, 11], dtype=np.int32) + mpi.mpi_send(sent, np.int32(sent.size), mpi.mpi_int, np.int32(1), np.int32(13), world) +elif rank == 1: + received = np.empty(4, dtype=np.int32) + mpi.mpi_recv( + received, + np.int32(received.size), + mpi.mpi_int, + np.int32(0), + np.int32(13), + world, + mpi.mpi_status_ignore, + ) + print(f"rank 1 received {received.tolist()}") + +# Collective: every rank contributes and every rank receives the sum. +values = np.array([rank + 1, rank + 2], dtype=np.int32) +reduced = np.empty_like(values) +mpi.mpi_allreduce(values, reduced, np.int32(values.size), mpi.mpi_int, mpi.mpi_sum, world) + +readings = np.array([20.0 + rank], dtype=np.float64) +total = np.empty_like(readings) +mpi.mpi_allreduce(readings, total, np.int32(readings.size), mpi.mpi_double_precision, mpi.mpi_sum, world) + +# In place: MPI_IN_PLACE as the send buffer reduces the receive buffer itself. +in_place = values.copy() +mpi.mpi_allreduce(mpi.mpi_in_place, in_place, np.int32(in_place.size), mpi.mpi_int, mpi.mpi_sum, world) + +mpi.mpi_barrier(world) +print(f"rank {rank} of {size}: sum {reduced.tolist()}, in place {in_place.tolist()}, total {total.tolist()}") +mpi.mpi_finalize() diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py deleted file mode 100644 index 726f4aa33..000000000 --- a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/openmpi_basic.py +++ /dev/null @@ -1,64 +0,0 @@ -"""Executed by the Open MPI integration test under two or more ranks.""" - -import numpy as np - -from prik_openmpi_f08 import mpi_f08 as mpi - - -mpi.mpi_init() -assert isinstance(mpi.mpi_comm_world, mpi.Mpi_Comm) -assert isinstance(mpi.mpi_int, mpi.Mpi_Datatype) -assert isinstance(mpi.mpi_sum, mpi.Mpi_Op) -assert isinstance(mpi.mpi_status_ignore, mpi.Mpi_Status) -rank, rank_error = mpi.mpi_comm_rank(mpi.mpi_comm_world) -size, size_error = mpi.mpi_comm_size(mpi.mpi_comm_world) -rank, size = int(rank), int(size) -assert rank_error is None and size_error is None and size >= 2 -mpi.mpi_barrier(mpi.mpi_comm_world) - -if rank == 0: - sent = np.array([3, 5, 7, 11], dtype=np.int32) - mpi.mpi_send(sent, np.int32(sent.size), mpi.mpi_int, np.int32(1), np.int32(13), mpi.mpi_comm_world) -elif rank == 1: - received = np.empty(4, dtype=np.int32) - mpi.mpi_recv( - received, - np.int32(received.size), - mpi.mpi_int, - np.int32(0), - np.int32(13), - mpi.mpi_comm_world, - mpi.mpi_status_ignore, - ) - np.testing.assert_array_equal(received, [3, 5, 7, 11]) - -values = np.array([rank + 1, rank + 2], dtype=np.int32) -expected = np.array([size * (size + 1) // 2, size * (size + 3) // 2], dtype=np.int32) -reduced = np.empty_like(values) -mpi.mpi_allreduce(values, reduced, np.int32(values.size), mpi.mpi_int, mpi.mpi_sum, mpi.mpi_comm_world) -np.testing.assert_array_equal(reduced, expected) -floats = np.array([float(rank + 1)], dtype=np.float64) -float_reduced = np.empty_like(floats) -mpi.mpi_allreduce( - floats, - float_reduced, - np.int32(floats.size), - mpi.mpi_double_precision, - mpi.mpi_sum, - mpi.mpi_comm_world, -) -np.testing.assert_array_equal(float_reduced, [float(expected[0])]) - -in_place = values.copy() -assert isinstance(mpi.mpi_in_place, np.ndarray) and mpi.mpi_in_place.shape == () -mpi.mpi_allreduce( - mpi.mpi_in_place, - in_place, - np.int32(in_place.size), - mpi.mpi_int, - mpi.mpi_sum, - mpi.mpi_comm_world, -) -np.testing.assert_array_equal(in_place, expected) -mpi.mpi_finalize() -print(f"Open MPI rank {rank}: communication passed", flush=True) diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index dc940c2a8..41be8d965 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -1,23 +1,30 @@ -"""Real Open MPI source to generated contract to two-rank execution.""" +"""Real Open MPI source to generated contract to two-rank execution. + +The test runs the commands the Open MPI ``mpi_f08`` tutorial shows: generate +a restricted contract from the configured Open MPI sources, build it against +the installation without compiling any Open MPI source, and run the tutorial's +program under the Open MPI launcher. +""" from __future__ import annotations +import json import os import re import shlex import shutil import subprocess import sys +from collections.abc import Sequence from pathlib import Path from typing import NoReturn import pytest -from prik.pipeline.build import NativeLinkItem, build_pyi_extension - pytestmark = pytest.mark.fortran_end_to_end -RUNTIME = Path(__file__).parent / "fixtures" / "runtime" / "openmpi_basic.py" +# The tutorial displays this program; the test runs it as written. +RUNTIME = Path(__file__).parent / "fixtures" / "runtime" / "mpi_example.py" EXPORTS = ( "MPI_Init", "MPI_Finalize", @@ -34,6 +41,10 @@ "MPI_IN_PLACE", "MPI_STATUS_IGNORE", ) +# ``ompi_info`` reports these for the configure run that built the +# installation, and a configured tree records the same values, so they +# identify that run: its date, host, user, and exact command line. +CONFIGURE_IDENTITY = ("timestamp", "host", "user", "cli") def _unavailable(reason: str) -> NoReturn: @@ -43,15 +54,69 @@ def _unavailable(reason: str) -> NoReturn: pytest.skip(reason) -def _configured_openmpi() -> tuple[Path, Path, str, str, str]: - """Find matching configured sources, wrapper compiler, and Open MPI launcher.""" +def _tool_output(command: Sequence[str], purpose: str) -> str: + """Return one Open MPI helper command's output, or report the helper unavailable. + + A helper that is missing, cannot run, fails, or hangs leaves the + installation unusable for this test, which is not a test failure unless + Open MPI was required. + """ + try: + completed = subprocess.run(list(command), check=True, capture_output=True, text=True, timeout=60) + except (OSError, subprocess.CalledProcessError, subprocess.TimeoutExpired) as error: + _unavailable(f"{purpose} is unavailable: `{shlex.join(command)}` failed ({error})") + return completed.stdout + + +def _field(text: str, pattern: str, purpose: str) -> str: + """Return the one value ``pattern`` captures in ``text``, or report it unavailable.""" + match = re.search(pattern, text, flags=re.MULTILINE) + if match is None: + _unavailable(f"{purpose} is not recorded") + return match.group(1).strip() + + +def _installed_configuration(info: str) -> dict[str, object]: + """Return the installation's version and the identity of the configure run that built it.""" + identity: dict[str, object] = { + "version": _field(info, r"^ompi:version:full:(.+)$", "the installed Open MPI version"), + } + for key in CONFIGURE_IDENTITY: + value = _field(info, rf"^config:{key}:(.*)$", f"the installed Open MPI configure {key}") + identity[key] = shlex.split(value) if key == "cli" else value.strip('"') + return identity + + +def _tree_configuration(source: Path, build: Path) -> dict[str, object]: + """Return a configured tree's version and the identity of the configure run that produced it.""" + try: + version_text = (source / "VERSION").read_text(encoding="utf-8") + makefile = (build / "Makefile").read_text(encoding="utf-8") + version_header = (build / "opal/include/opal/version.h").read_text(encoding="utf-8") + except OSError as error: + _unavailable(f"the configured Open MPI tree records no configuration: {error}") + parts = [ + _field(version_text, rf"^{part}=(\d+)$", f"the Open MPI source {part} version") + for part in ("major", "minor", "release") + ] + cli = _field(version_header, r'^#define OPAL_CONFIGURE_CLI "(.*)"$', "the configured tree's configure command line") + identity: dict[str, object] = {"version": ".".join(parts), "cli": shlex.split(cli.replace("\\'", "'"))} + for key, variable in (("timestamp", "DATE"), ("host", "HOST"), ("user", "USER")): + identity[key] = _field( + makefile, rf"^OPAL_CONFIGURE_{variable} = (.*)$", f"the configured tree's configure {key}" + ) + return identity + + +def _configured_openmpi() -> tuple[Path, Path, str, str]: + """Find configured sources, and the installation they were configured for.""" source_text = os.environ.get("PRIK_OPENMPI_SOURCE") build_text = os.environ.get("PRIK_OPENMPI_BUILD") if not source_text or not build_text: _unavailable("set PRIK_OPENMPI_SOURCE and PRIK_OPENMPI_BUILD to a matching configured Open MPI tree") source, build = Path(source_text), Path(build_text) mpifort = os.environ.get("PRIK_OPENMPI_MPIFORT") or shutil.which("mpifort") - launcher = os.environ.get("PRIK_OPENMPI_LAUNCHER") or shutil.which("orterun") or shutil.which("mpirun") + launcher = os.environ.get("PRIK_OPENMPI_LAUNCHER") or shutil.which("mpirun") if not mpifort or not launcher: _unavailable("Open MPI Fortran compiler wrapper and launcher are required") # Only the entry source is named; the modules it uses are discovered. @@ -61,55 +126,37 @@ def _configured_openmpi() -> tuple[Path, Path, str, str, str]: ): if not path.is_file(): _unavailable(f"configured Open MPI semantic input is unavailable: {path}") - version_file = (source / "VERSION").read_text(encoding="utf-8") - parts = [re.search(rf"^{part}=(\d+)$", version_file, flags=re.MULTILINE) for part in ("major", "minor", "release")] - if any(part is None for part in parts): - _unavailable("Open MPI source version could not be read") - version = ".".join(part.group(1) for part in parts if part is not None) - compiler_version = subprocess.check_output([mpifort, "--showme:version"], text=True) - launcher_version = subprocess.check_output([launcher, "--version"], text=True) - if ( - f"Open MPI {version}" not in compiler_version - or version not in launcher_version - or not any(label in launcher_version for label in ("Open MPI", "OpenRTE")) - ): - _unavailable("configured sources, mpifort, and launcher must belong to the same Open MPI version") - _require_same_fortran_configuration(build, mpifort) - return source, build, mpifort, launcher, version - - -def _require_same_fortran_configuration(build: Path, mpifort: str) -> None: - """Require the configured tree and the installation to share their Fortran build. - One version configured twice can generate different Fortran sources and - headers, so the tree the contract is read from must have been configured - with the Fortran compiler the installation was built with, and the - installation must provide the ``mpi_f08`` module. - """ - ompi_info = Path(mpifort).with_name("ompi_info") - info = subprocess.check_output([str(ompi_info) if ompi_info.is_file() else "ompi_info", "--parsable"], text=True) - installed = re.search(r"^compiler:fortran:absolute:(.+)$", info, flags=re.MULTILINE) - configured_header = build / "opal/include/opal_config.h" - configured = ( - re.search(r'^#define OMPI_FC_ABSOLUTE "([^"]+)"', configured_header.read_text(encoding="utf-8"), re.MULTILINE) - if configured_header.is_file() - else None - ) - if re.search(r"^bindings:use_mpi_f08:yes", info, flags=re.MULTILINE) is None: + beside = Path(mpifort).with_name("ompi_info") + ompi_info = str(beside) if beside.is_file() else shutil.which("ompi_info") + if ompi_info is None: + _unavailable("ompi_info is unavailable beside mpifort or on PATH") + info = _tool_output([ompi_info, "--parsable"], "ompi_info") + if re.search(r"^bindings:use_mpi_f08:yes$", info, flags=re.MULTILINE) is None: _unavailable("the installed Open MPI does not provide the mpi_f08 module") - if ( - installed is None - or configured is None - or Path(installed.group(1).strip()).resolve() != Path(configured.group(1)).resolve() - ): - _unavailable("the configured Open MPI tree and the installation were built with different Fortran compilers") + installed = _installed_configuration(info) + configured = _tree_configuration(source, build) + # The generated Fortran sources and headers follow the configure run, so + # a tree of the same version configured differently -- other flags, other + # options, another compiler -- would describe another interface. + if configured != installed: + differing = sorted(key for key in installed if installed[key] != configured.get(key)) + _unavailable( + "the configured Open MPI tree is not the one the installation was configured from " + f"(differs in {', '.join(differing)})" + ) + + launcher_version = _tool_output([launcher, "--version"], "the Open MPI launcher") + if installed["version"] not in launcher_version: + _unavailable(f"the launcher does not belong to Open MPI {installed['version']}") + return source, build, mpifort, launcher def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) -> None: - """The selected facade and native storage survive a real .pyi replay build.""" - source, build, mpifort, launcher, _version = _configured_openmpi() + """The tutorial's contract, build, and program run against a real Open MPI.""" + source, build, mpifort, launcher = _configured_openmpi() contract = tmp_path / "contract" - exports = tmp_path / "exports.txt" + exports = tmp_path / "mpi_exports.txt" exports.write_text("".join(f"mpi_f08::{symbol}\n" for symbol in EXPORTS), encoding="utf-8") includes = ( build, @@ -142,7 +189,7 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) check=True, capture_output=True, text=True, - timeout=180, + timeout=300, ) facade = (contract / "mpi_f08.pyi").read_text(encoding="utf-8") types = (contract / "mpi_f08_types.pyi").read_text(encoding="utf-8") @@ -160,32 +207,55 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) assert all(f"class Mpi_{name}" in declarations for name in ("Comm", "Datatype", "Op", "Status")) assert "AnyNative[" in interfaces and '@overload("mpi_send_f08")\ndef mpi_send(' in interfaces - def show(flag: str) -> list[str]: - return shlex.split(subprocess.check_output([mpifort, flag], text=True)) + def showme(flag: str) -> list[str]: + return shlex.split(_tool_output([mpifort, f"--showme:{flag}"], f"mpifort --showme:{flag}")) # The underlying compiler command may take more than one token, such as a # launcher before the compiler, which is not a compiler plus flags. - command, compile_flags = show("--showme:command"), show("--showme:compile") + command = showme("command") if len(command) != 1: _unavailable(f"mpifort --showme:command is a multi-token command {command}; pass one compiler executable") - result = build_pyi_extension( - contract / "__init__.pyi", - input_compiler=command[0], - native_include_dirs=show("--showme:incdirs"), - wrapper_fortran_flags=compile_flags, - native_link_items=[NativeLinkItem("linker_argument", flag) for flag in show("--showme:link")], - native_linker_language="fortran", - output_name="prik_openmpi_f08", - output_dir=tmp_path / "extension", - jobs=2, + built = subprocess.run( + [ + sys.executable, + "-m", + "prik", + str(contract / "__init__.pyi"), + "--compiler", + command[0], + f"--wrapper-fortran-flags={shlex.join(showme('compile'))}", + "--native-library", + *showme("libs"), + "--native-library-dir", + *showme("libdirs"), + "--out", + "prik_openmpi_f08", + "--out-dir", + str(tmp_path / "extension"), + "--jobs", + "2", + "--json", + ], + check=True, + capture_output=True, + text=True, + timeout=300, + cwd=tmp_path, ) - assert result.native_build_plan is not None and not result.native_build_plan.compilation_units - bridge = next(path for path in result.generated_sources if path.suffix == ".f90").read_text(encoding="utf-8") + payload = json.loads(built.stdout) + # Only PRIK's bridge and binding compile; Open MPI's own sources do not. + assert payload["native_build_plan"]["compilation_units"] == [] + assert sorted(Path(path).name for path in payload["generated_files"] if path.endswith(".o")) == [ + "bind_c_prik_openmpi_f08_wrapper.o", + "prik_openmpi_f08_wrapper.o", + ] + bridge = (tmp_path / "extension" / "bind_c_prik_openmpi_f08_wrapper.f90").read_text(encoding="utf-8") assert "use mpi_f08_interfaces, only:" in bridge assert "=> MPI_Allreduce" in bridge and "=> MPI_Send" in bridge + env = os.environ.copy() - env["PYTHONPATH"] = os.pathsep.join(filter(None, (str(result.output_dir), env.get("PYTHONPATH", "")))) - env["LD_LIBRARY_PATH"] = os.pathsep.join((*show("--showme:libdirs"), env.get("LD_LIBRARY_PATH", ""))) + env["PYTHONPATH"] = os.pathsep.join(filter(None, (str(tmp_path / "extension"), env.get("PYTHONPATH", "")))) + env["LD_LIBRARY_PATH"] = os.pathsep.join((*showme("libdirs"), env.get("LD_LIBRARY_PATH", ""))) completed = subprocess.run( [launcher, "-n", "2", sys.executable, str(RUNTIME)], env=env, @@ -194,4 +264,7 @@ def show(flag: str) -> list[str]: text=True, timeout=120, ) - assert completed.stdout.count("communication passed") == 2 + output = completed.stdout.splitlines() + assert "rank 1 received [3, 5, 7, 11]" in output + for rank in (0, 1): + assert f"rank {rank} of 2: sum [3, 5], in place [3, 5], total [41.0]" in output diff --git a/tests/fortran/callbacks/end_to_end/fixtures/native/generic_callback_prototype_imports.f90 b/tests/fortran/callbacks/end_to_end/fixtures/native/generic_callback_prototype_imports.f90 new file mode 100644 index 000000000..91c3aba7e --- /dev/null +++ b/tests/fortran/callbacks/end_to_end/fixtures/native/generic_callback_prototype_imports.f90 @@ -0,0 +1,47 @@ +module handles + implicit none + type, bind(c) :: handle + integer :: val + end type +end module handles + +module cbs + implicit none + abstract interface + subroutine copy_fn(h, x) + use handles + implicit none + type(handle) :: h + integer :: x + end subroutine + end interface +end module cbs + +module ifaces + implicit none + interface register + subroutine register_impl(fn, key) + use :: cbs, only : copy_fn + implicit none + procedure(copy_fn) :: fn + integer, intent(out) :: key + end subroutine register_impl + end interface register +end module ifaces + +module facade + use ifaces +end module facade + +subroutine register_impl(fn, key) + use handles + use cbs, only: copy_fn + procedure(copy_fn) :: fn + integer, intent(out) :: key + type(handle) :: h + integer :: v + h%val = 7 + v = 4 + call fn(h, v) + key = v + h%val +end subroutine register_impl diff --git a/tests/fortran/callbacks/end_to_end/test_generic_callback_prototype_imports.py b/tests/fortran/callbacks/end_to_end/test_generic_callback_prototype_imports.py new file mode 100644 index 000000000..d6864cec6 --- /dev/null +++ b/tests/fortran/callbacks/end_to_end/test_generic_callback_prototype_imports.py @@ -0,0 +1,72 @@ +"""A generic's specific takes a callback whose prototype and types come from ``use`` in bodies. + +``register`` is a generic whose interface body imports its callback prototype +with a ``use`` of its own, and that prototype imports its argument's derived +type with a ``use`` in its body. Built from source and replayed from a +selected contract, the generic dispatches a Python callable to the callback. +""" + +from __future__ import annotations + +import subprocess +import sys +from pathlib import Path + +import pytest + +from prik import build_fortran_extension, build_pyi_extension +from tests.fortran._support.wrapper_build import _import_from_build_dir + +pytestmark = pytest.mark.fortran_end_to_end + +SOURCE = Path(__file__).parent / "fixtures" / "native" / "generic_callback_prototype_imports.f90" + + +def test_a_generic_dispatches_a_callable_to_a_prototype_imported_inside_bodies(tmp_path: Path): + contract = tmp_path / "contract" + symbols = tmp_path / "exports.txt" + symbols.write_text("facade::register\n", encoding="utf-8") + subprocess.run( + [ + sys.executable, + "-m", + "prik", + "generate", + "--pyi", + str(SOURCE), + "--export-symbols", + str(symbols), + "--out", + str(contract), + ], + check=True, + capture_output=True, + text=True, + ) + # The prototype names its argument's type through the module declaring it, + # and that module is part of the selected contract set. + prototypes = (contract / "cbs.pyi").read_text(encoding="utf-8") + assert "h: handles.Handle" in prototypes and (contract / "handles.pyi").is_file() + assert "from .cbs import copy_fn" in (contract / "ifaces.pyi").read_text(encoding="utf-8") + + source = build_fortran_extension( + SOURCE, + output_dir=tmp_path / "source", + output_name="callback_imports_source", + export_symbols=["facade::register"], + ) + replay = build_pyi_extension( + contract / "__init__.pyi", + native_fortran_sources=(SOURCE,), + output_dir=tmp_path / "replay", + output_name="callback_imports_replay", + ) + + def callback(handle, value): + value[()] = value + handle.val + + for result in (source, replay): + facade = _import_from_build_dir(result.module_name, result.output_dir).facade + assert facade.register(callback) == 18 + with pytest.raises(TypeError, match="no matching overload"): + facade.register(3) diff --git a/tests/fortran/callbacks/semantics/test_callback_route_resolution.py b/tests/fortran/callbacks/semantics/test_callback_route_resolution.py index 0882d7b00..659229566 100644 --- a/tests/fortran/callbacks/semantics/test_callback_route_resolution.py +++ b/tests/fortran/callbacks/semantics/test_callback_route_resolution.py @@ -8,6 +8,8 @@ from pathlib import Path +import pytest + from prik.parsers.fortran import parse_fortran_project from prik.semantics.fortran2ir import FortranToIRConverter @@ -158,3 +160,37 @@ def test_each_procedure_resolves_the_callback_it_declares(tmp_path: Path): seen[procedure.name] = [argument.base_type for argument in scope["cb"].signature.arguments] assert seen == {"first": ["integer"], "second": ["real"]} + + +@pytest.mark.parametrize( + ("module_use", "body_use"), + [ + pytest.param(" use, intrinsic :: iso_c_binding, only : c_ptr\n", " import :: c_ptr\n", id="module-use"), + pytest.param("", " use, intrinsic :: iso_c_binding, only : c_ptr\n", id="body-use"), + ], +) +def test_a_prototype_writes_a_processor_type_as_itself_however_it_is_imported(module_use: str, body_use: str): + """``c_ptr`` comes from the processor, so no contract module qualifies it.""" + from prik.parsers.fortran import parse_fortran_file + from prik.pipeline.pyi import emit_module_stubs + from prik.semantics.fortran2ir import fortran_file_to_semantic_modules + + source = ( + "module prototypes\n" + f"{module_use}" + " implicit none\n" + " abstract interface\n" + " subroutine user_fn(p, n)\n" + f"{body_use}" + " implicit none\n" + " type(c_ptr), value :: p\n" + " integer :: n\n" + " end subroutine\n" + " end interface\n" + "end module prototypes\n" + ) + modules = fortran_file_to_semantic_modules(parse_fortran_file(source, filename="prototypes.f90")) + + contract = emit_module_stubs(modules, normalize_public_names=True)["prototypes"] + + assert " p: Value(c_ptr)," in contract diff --git a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py index e647495a9..f55b2fc79 100644 --- a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py +++ b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py @@ -274,3 +274,45 @@ def test_a_generic_sharing_a_specific_name_is_one_selectable_name(): assert [overload.name for overload in selected.overload_sets] == ["area"] assert sorted(function.name for function in selected.functions) == ["area", "area_int"] assert selected.exported_names == ["area"] + + +def test_selection_drops_imports_only_unselected_declarations_used(tmp_path: Path): + """A name only a removed declaration was written with is not imported by the contract. + + Every type ``consts`` re-exports stays declared where it is; the selected + constant names one of them, so that is the one the contract binds, under + its class name. + """ + from prik.pipeline.pyi import emit_module_stubs + + source = tmp_path / "handles.f90" + source.write_text( + """module handles + implicit none + type, bind(c) :: Handle_A + integer :: val + end type + type, bind(c) :: Handle_B + integer :: val + end type +end module handles + +module consts + use handles + implicit none + type(Handle_A), parameter :: A_NULL = Handle_A(0) + type(Handle_B), parameter :: B_NULL = Handle_B(0) +end module consts +""", + encoding="utf-8", + ) + modules = fortran_project_to_semantic_modules(parse_fortran_project(tmp_path)) + selection = select_fortran_export_symbols(modules, ["consts::A_NULL"]) + + contract = emit_module_stubs( + list(selection.primary_modules), + available_modules=list(selection.available_modules), + normalize_public_names=True, + )["consts"] + + assert [line for line in contract.splitlines() if line.startswith("from .")] == ["from .handles import Handle_A"] diff --git a/tests/workflows/test_openmpi_integration.py b/tests/workflows/test_openmpi_integration.py new file mode 100644 index 000000000..29b2a2d21 --- /dev/null +++ b/tests/workflows/test_openmpi_integration.py @@ -0,0 +1,118 @@ +"""The Open MPI integration workflow runs only against a matching installation, and says why not. + +These tests stand in fake Open MPI tools and trees, so they run without Open +MPI: an unusable helper or a tree from another configure run skips the real +test locally, and fails it where ``PRIK_OPENMPI_REQUIRED=1`` says Open MPI is +provisioned. +""" + +from __future__ import annotations + +import os +import shutil +import stat +from pathlib import Path + +import pytest + +from tests.fortran.assumed_types.end_to_end.test_openmpi_f08 import _configured_openmpi + +CLI = "'--prefix=/opt/ompi' '--enable-mpi-fortran=usempif08' 'FC=gfortran'" +INFO = f"""ompi:version:full:5.0.11 +config:user:builder +config:timestamp:"Fri Sep 25 11:13:48 UTC 2026" +config:host:buildhost +config:cli: {CLI} +bindings:use_mpi_f08:yes +""" + + +def _executable(path: Path, script: str) -> Path: + path.write_text(f"#!/bin/sh\n{script}\n", encoding="utf-8") + path.chmod(path.stat().st_mode | stat.S_IXUSR) + return path + + +def _openmpi(tmp_path: Path, monkeypatch, *, ompi_info: str | None = None, cli: str = CLI) -> None: + """Lay out a configured tree and an installation, and point the test at them.""" + source, build, bin_dir = tmp_path / "source", tmp_path / "build", tmp_path / "bin" + for path in ( + source / "ompi/mpi/fortran/use-mpi-f08/mpi-f08.F90", + build / "ompi/mpi/fortran/configure-fortran-output.h", + ): + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text("", encoding="utf-8") + (source / "VERSION").write_text("major=5\nminor=0\nrelease=11\n", encoding="utf-8") + (build / "Makefile").write_text( + "OPAL_CONFIGURE_DATE = Fri Sep 25 11:13:48 UTC 2026\n" + "OPAL_CONFIGURE_HOST = buildhost\n" + "OPAL_CONFIGURE_USER = builder\n", + encoding="utf-8", + ) + header = build / "opal/include/opal/version.h" + header.parent.mkdir(parents=True) + header.write_text(f'#define OPAL_CONFIGURE_CLI " {cli.replace(chr(39), chr(92) + chr(39))}"\n', encoding="utf-8") + bin_dir.mkdir() + mpifort = _executable(bin_dir / "mpifort", "exit 0") + launcher = _executable(bin_dir / "mpirun", 'echo "mpirun (Open MPI) 5.0.11"') + if ompi_info is not None: + _executable(bin_dir / "ompi_info", ompi_info) + monkeypatch.setenv("PATH", str(bin_dir)) + monkeypatch.setenv("PRIK_OPENMPI_SOURCE", str(source)) + monkeypatch.setenv("PRIK_OPENMPI_BUILD", str(build)) + monkeypatch.setenv("PRIK_OPENMPI_MPIFORT", str(mpifort)) + monkeypatch.setenv("PRIK_OPENMPI_LAUNCHER", str(launcher)) + + +def _info(text: str) -> str: + """Return a script printing ``text``; ``PATH`` holds only the fake tools, so ``cat`` is named in full.""" + return f"{shutil.which('cat')} <<'EOF'\n{text}EOF" + + +@pytest.mark.parametrize( + ("ompi_info", "reason"), + [ + pytest.param(None, "ompi_info is unavailable", id="missing"), + pytest.param("exit 3", "ompi_info is unavailable", id="failing"), + pytest.param( + _info(INFO.replace("config:host:buildhost\n", "")), "configure host is not recorded", id="incomplete" + ), + pytest.param( + _info(INFO.replace("bindings:use_mpi_f08:yes\n", "")), + "does not provide the mpi_f08 module", + id="without-mpi-f08", + ), + ], +) +@pytest.mark.parametrize("required", [False, True], ids=["local", "required"]) +def test_an_unusable_open_mpi_helper_skips_locally_and_fails_when_required( + tmp_path: Path, monkeypatch, ompi_info: str | None, reason: str, required: bool +): + _openmpi(tmp_path, monkeypatch, ompi_info=ompi_info) + if required: + monkeypatch.setenv("PRIK_OPENMPI_REQUIRED", "1") + else: + monkeypatch.delenv("PRIK_OPENMPI_REQUIRED", raising=False) + + with pytest.raises(pytest.fail.Exception if required else pytest.skip.Exception, match=reason): + _configured_openmpi() + + +def test_a_tree_from_another_configure_run_does_not_match_the_installation(tmp_path: Path, monkeypatch): + """Same version and compiler, but other flags: another configure run describes another interface.""" + _openmpi(tmp_path, monkeypatch, ompi_info=_info(INFO), cli=CLI + " 'FCFLAGS=-fdefault-integer-8'") + monkeypatch.delenv("PRIK_OPENMPI_REQUIRED", raising=False) + + with pytest.raises( + pytest.skip.Exception, match=r"not the one the installation was configured from \(differs in cli\)" + ): + _configured_openmpi() + + +def test_the_tree_an_installation_was_configured_from_matches_it(tmp_path: Path, monkeypatch): + _openmpi(tmp_path, monkeypatch, ompi_info=_info(INFO)) + + source, build, mpifort, launcher = _configured_openmpi() + + assert (source.name, build.name) == ("source", "build") + assert os.path.basename(mpifort) == "mpifort" and os.path.basename(launcher) == "mpirun" From 8ed786b80ec931d8080d05976a6fecf6e454c46b Mon Sep 17 00:00:00 2001 From: said Date: Fri, 25 Sep 2026 23:34:39 +0100 Subject: [PATCH 24/47] Shape the mpi_f08 tutorial into an mpi4py-style API; resolve re-exported contract classes The tutorial now edits the generated facade contract to hide counts and error codes and return results, and adds a short Python module spelling mpi4py's COMM_WORLD, Send/Recv, pickled send/recv, Probe, Bcast, Reduce, and Allreduce. The tutorial's program runs unchanged under mpi4py but for its import, and the Open MPI integration test builds and runs it. A contract class imported through a contract module that re-exports it now resolves to its declaring module, which the edited facade relies on under Open MPI 5.0. The @bind reference no longer documents the removed module::symbol form. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 16 +- docs/user/reference/pyi-format.md | 2 +- docs/user/tutorials/openmpi-f08.md | 540 ++++++++++++++---- prik/semantics/pyi2ir.py | 60 +- .../fixtures/contracts/openmpi/mpi_f08.pyi | 129 +++++ .../fixtures/runtime/mpi_example.py | 71 ++- .../end_to_end/fixtures/runtime/prik_mpi.py | 118 ++++ .../end_to_end/test_openmpi_f08.py | 46 +- .../test_reexported_class_references.py | 50 ++ 9 files changed, 852 insertions(+), 180 deletions(-) create mode 100644 tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi create mode 100644 tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py create mode 100644 tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/semantics/test_reexported_class_references.py diff --git a/CHANGELOG.md b/CHANGELOG.md index b2834c1d9..dd1245312 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -97,6 +97,9 @@ release tags add a leading `v` to the package version. - A derived type a module reaches through another module's re-export is resolved to the module that declares it, so generated contracts name it instead of writing an undefined type. +- A `.pyi` contract class imported through a contract module that re-exports + it resolves to the declaring module's class, however long the chain, so the + build wraps it instead of failing with no completed wrapper type definition. - Fortran parse diagnostics on compiler-preprocessed sources report the line in the source file, or the `#include` line for text an included file contributes, instead of a line in the preprocessor output. @@ -122,11 +125,14 @@ release tags add a leading `v` to the package version. native scalar storage views through `T[()]`. Source builds and generated contracts publish the same selected surface. - A tutorial turns a reviewed part of Open MPI's Fortran `mpi_f08` interface - into a Python MPI API: it generates a restricted `.pyi` contract from the - configured Open MPI sources, builds it with the CLI against the installed - Open MPI without compiling any Open MPI source, and runs a two-rank NumPy - program under `mpirun`. An opt-in integration test runs the same commands - and program; the Open MPI Integration lane runs it against Open MPI 4.1.8 and + into an mpi4py-style Python MPI API: it generates a restricted `.pyi` + contract from the configured Open MPI sources, edits its facade to hide + counts and error codes and return results, builds it with the CLI against + the installed Open MPI without compiling any Open MPI source, adds a short + Python module spelling mpi4py's `COMM_WORLD`, `Send`/`Recv`, pickled + `send`/`recv`, `Bcast`, `Reduce`, and `Allreduce`, and runs a two-rank + program under `mpirun` that mpi4py also runs unchanged but for its import. + An opt-in integration test runs the same commands and program; the Open MPI Integration lane runs it against Open MPI 4.1.8 and 5.0.11 built from source. The test requires the configured tree to record the same configure run as the installation, and reports missing or failing Open MPI tools as unavailable -- a skip locally, a failure where Open MPI is diff --git a/docs/user/reference/pyi-format.md b/docs/user/reference/pyi-format.md index 767298298..680dd34f9 100644 --- a/docs/user/reference/pyi-format.md +++ b/docs/user/reference/pyi-format.md @@ -557,7 +557,7 @@ Python declaration and native callable names differ. | Decorator | Valid target | Language and meaning | | --- | --- | --- | | `@private` | Function or method | Shared: declaration remains available to contract dependencies but is not exported. | -| `@bind("symbol")` | Function, method, constructor, prototype, or destructor | Shared: select a different native name. For module-level Fortran procedures, `"module::symbol"` also selects the native module used for the call. | +| `@bind("symbol")` | Function, method, constructor, prototype, or destructor | Shared: select a different native name. A module-level Fortran procedure is called through the native module the contract module names, so the symbol may be any procedure or generic that module provides, including one it imports. | | `@native_abi("c")` | Function, method, or prototype | Fortran only: original declaration is `bind(C)`. | | `@standalone` | Module-level function | Fortran only: native procedure is outside a module. | | `@native_call([...], result=...)` | Function, method, or constructor | Shared: state the complete native argument order and optional native result mapping. | diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index fc093af18..201ce67bd 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -1,6 +1,6 @@ --- title: Wrap Open MPI mpi_f08 for Python -description: Turn a reviewed part of Open MPI's Fortran mpi_f08 interface into a Python MPI API +description: Turn a reviewed part of Open MPI's Fortran mpi_f08 interface into an mpi4py-style Python MPI API audience: users prerequisites: an Open MPI installation with mpi_f08, and the configured Open MPI source and build trees it was built from related: ../guide/wrapping-modules.md, ../guide/callbacks.md, ../reference/cli-commands.md, ../reference/pyi-format.md @@ -10,32 +10,75 @@ publication: reviewed # Wrap Open MPI `mpi_f08` for Python -This tutorial starts from Open MPI's real Fortran `mpi_f08` interface and turns -a small, reviewed part of it into a Python MPI API. At the end, an ordinary -Python program runs under `mpirun` and communicates through it: +This tutorial turns a reviewed part of Open MPI's real Fortran `mpi_f08` +interface into a Python MPI API modeled on [mpi4py](https://mpi4py.readthedocs.io), +the standard Python binding of MPI. At the end, this program runs under +`mpirun`: + ```python import numpy as np -from prik_openmpi_f08 import mpi_f08 as mpi +import prik_mpi as MPI -mpi.mpi_init() -world = mpi.mpi_comm_world -rank, _ = mpi.mpi_comm_rank(world) +comm = MPI.COMM_WORLD +rank = comm.Get_rank() +size = comm.Get_size() -values = np.array([rank + 1, rank + 2], dtype=np.int32) -reduced = np.empty_like(values) -mpi.mpi_allreduce(values, reduced, np.int32(values.size), mpi.mpi_int, mpi.mpi_sum, world) +# Python objects travel pickled. +if rank == 0: + comm.send({"a": 7, "b": 3.14}, dest=1, tag=11) +elif rank == 1: + data = comm.recv(source=0, tag=11) + print(f"rank 1 received {data}") -mpi.mpi_finalize() +# NumPy arrays travel as buffers, with an explicit MPI datatype ... +if rank == 0: + data = np.arange(4, dtype="i") + comm.Send([data, MPI.INT], dest=1, tag=77) +elif rank == 1: + data = np.empty(4, dtype="i") + status = MPI.Status() + comm.Recv([data, MPI.INT], source=MPI.ANY_SOURCE, tag=77, status=status) + print(f"rank 1 received {data.tolist()} from rank {status.Get_source()}") + +# ... or with the datatype taken from the array. +data = np.arange(3, dtype=np.float64) if rank == 0 else np.empty(3, dtype=np.float64) +comm.Bcast(data, root=0) + +# Collectives: every rank contributes. +values = np.array([rank + 1, rank + 2], dtype="i") +total = np.empty_like(values) +comm.Allreduce(values, total, op=MPI.SUM) +largest = np.empty_like(values) +comm.Reduce(values, largest, op=MPI.MAX, root=0) +comm.Allreduce(MPI.IN_PLACE, values, op=MPI.SUM) + +comm.Barrier() +print(f"rank {rank} of {size}: bcast {data.tolist()}, sum {total.tolist()}, in place {values.tolist()}") +if rank == 0: + print(f"rank 0 max {largest.tolist()}") ``` -Nothing in this API is written by hand. Every function, handle type, and -constant is generated from the declarations in Open MPI's Fortran sources, and -every call enters the installed Open MPI library. If you know `mpi4py`, the -result will feel familiar: Python code driving the native MPI implementation. -It is not `mpi4py`, though. PRIK exposes the selected Fortran interface as it -is declared, so the names and argument lists are those of `mpi_f08`. +If you know mpi4py, you know this program: `COMM_WORLD`, `Get_rank`, +lowercase `send`/`recv` for Python objects, uppercase `Send`/`Recv`/`Bcast`/ +`Reduce`/`Allreduce` for buffers, `[data, MPI.INT]` buffer specifications, +`MPI.IN_PLACE`, and `Status` are all spelled as mpi4py spells them. Replace +`import prik_mpi as MPI` with `from mpi4py import MPI` and the same program +runs under mpi4py and prints the same lines. + +Two layers make this work: + +- **The extension PRIK generates.** Every MPI routine, handle type, and + constant the program reaches comes from the declarations in Open MPI's + Fortran sources, and every call enters the installed Open MPI library through + generated code. You shape that native API by editing a generated `.pyi` + contract: hiding counts that follow from the buffers, turning error codes + into exceptions, returning results instead of filling output arguments. +- **A short Python module, `prik_mpi.py`.** It gives the native API mpi4py's + object model: a `Comm` class with methods, keyword defaults, datatypes + chosen from NumPy arrays, and pickled Python objects. It is ordinary Python + over the generated functions, with no C and no `ctypes`. ## 1. See what PRIK reads @@ -58,8 +101,10 @@ type(MPI_Comm), parameter :: MPI_COMM_WORLD = MPI_Comm(OMPI_MPI_COMM_WORLD) type(MPI_Op), parameter :: MPI_SUM = MPI_Op(OMPI_MPI_SUM) type(MPI_Datatype), parameter :: MPI_INT = MPI_Datatype(OMPI_MPI_INT) +integer MPI_ANY_SOURCE +parameter (MPI_ANY_SOURCE=-1) + integer, bind(C, name="mpi_fortran_in_place_") :: MPI_IN_PLACE -type(MPI_Status), bind(C, name="mpi_fortran_status_ignore_") :: MPI_STATUS_IGNORE ``` and each MPI routine is a generic interface over its specific procedures: @@ -88,7 +133,7 @@ configure-generated `mpi_types` module in another, and PRIK finds whichever the sources declare. Some of these declarations, such as `MPI_IN_PLACE` above, are in headers that Open MPI's `configure` writes into its build tree. -## 2. Choose a small Python surface +## 2. Choose a small native surface `mpi_f08` publishes hundreds of routines and constants. Start with a reviewed subset instead of all of them. Save the Fortran identities to publish in @@ -102,19 +147,26 @@ mpi_f08::MPI_Comm_size mpi_f08::MPI_Barrier mpi_f08::MPI_Send mpi_f08::MPI_Recv +mpi_f08::MPI_Probe +mpi_f08::MPI_Get_count +mpi_f08::MPI_Bcast +mpi_f08::MPI_Reduce mpi_f08::MPI_Allreduce mpi_f08::MPI_COMM_WORLD +mpi_f08::MPI_BYTE mpi_f08::MPI_INT -mpi_f08::MPI_DOUBLE_PRECISION +mpi_f08::MPI_DOUBLE mpi_f08::MPI_SUM +mpi_f08::MPI_MAX mpi_f08::MPI_IN_PLACE -mpi_f08::MPI_STATUS_IGNORE +mpi_f08::MPI_ANY_SOURCE +mpi_f08::MPI_ANY_TAG ``` -This one list covers every kind of declaration the example needs: generic -procedures such as `MPI_Allreduce`, derived-type module objects such as -`MPI_COMM_WORLD`, datatype and operator constants, status storage, and the -special native storage `MPI_IN_PLACE`. +These are the routines and objects behind the mpi4py names the program uses. +`MPI_Probe` and `MPI_Get_count` are not called by the program directly; they +let lowercase `recv` size its buffer before receiving a pickled object, as +mpi4py does. Selecting symbols this way is not an MPI feature. `--export-symbols` accepts module-qualified public symbols from any Fortran project -- procedures, @@ -162,14 +214,14 @@ directories so PRIK reads each source exactly as the Fortran compiler did. PRIK reads these sources to learn their declarations. It does not compile them. -## 4. Read the contract +## 4. Read the generated contract The `contract/` directory holds one editable `.pyi` file per Fortran module the selection needs. `contract/mpi_f08.pyi` publishes the selected names: ```python -from .mpi_f08_types import mpi_comm_world, mpi_double_precision, mpi_in_place, mpi_int, mpi_status_ignore, mpi_sum -from .mpi_f08_interfaces import mpi_allreduce, mpi_barrier, mpi_comm_rank, mpi_comm_size, mpi_finalize, mpi_init, mpi_recv, mpi_send +from .mpi_f08_types import mpi_any_source, mpi_any_tag, mpi_byte, mpi_comm_world, mpi_double, mpi_in_place, mpi_int, mpi_max, mpi_sum +from .mpi_f08_interfaces import mpi_allreduce, mpi_barrier, mpi_bcast, mpi_comm_rank, mpi_comm_size, mpi_finalize, mpi_get_count, mpi_init, mpi_probe, mpi_recv, mpi_reduce, mpi_send from .mpi_types import Mpi_Comm, Mpi_Datatype, Mpi_Op, Mpi_Status ``` @@ -180,17 +232,15 @@ on. The predefined objects become typed module attributes in `contract/mpi_f08_types.pyi`: ```python +mpi_any_source: Final[Int32] = -1 + mpi_comm_world: Final[Mpi_Comm] mpi_sum: Final[Mpi_Op] mpi_int: Final[Mpi_Datatype] -mpi_double_precision: Final[Mpi_Datatype] - mpi_in_place: Int32[()] - -mpi_status_ignore: Mpi_Status ``` and each selected routine keeps its Fortran interface in @@ -215,15 +265,21 @@ Three mappings are worth a closer look. **Handles are concrete types.** `Mpi_Comm`, `Mpi_Datatype`, `Mpi_Op`, and `Mpi_Status` are the derived types Open MPI declares, with their real components. For example, `Mpi_Comm` holds the one integer handle Open MPI -stores: +stores, and `Mpi_Status` the source, tag, and error of a message: ```python -class Mpi_Comm: +class Mpi_Status: def __init__( self, *, - mpi_val: Int32 = ... + mpi_source: Int32 = ..., + mpi_tag: Int32 = ..., + mpi_error: Int32 = ... ) -> None: ... + + mpi_source: Int32 + mpi_tag: Int32 + mpi_error: Int32 ``` A handle argument accepts only an object of its own type, and the predefined @@ -231,7 +287,7 @@ handles are `Final` module constants. **`MPI_IN_PLACE` is native storage.** Its declaration is a C-bound integer module variable, so it becomes rank-zero native integer storage, `Int32[()]`. -Python sees `mpi.mpi_in_place` as a live NumPy scalar view of Open MPI's own +Python sees `mpi_in_place` as a live NumPy scalar view of Open MPI's own `MPI_IN_PLACE` variable. Passing that view as a buffer passes the address of that variable, which is how Open MPI recognizes an in-place operation. Nothing here knows about MPI: it is the same mapping any Fortran module @@ -243,11 +299,165 @@ routines are `type(*)` dummies, which accept data of any type, so they become appears only for these assumed-type dummies; `MPI_IN_PLACE` is a concrete module object with a concrete type. -The trailing `ierror` argument is optional. Leave it out and the call returns -`None` instead of the error code; Open MPI's default error handler aborts on -errors anyway. +This contract already builds, and its functions are Fortran's: every count +is an argument, every routine takes and returns the optional `ierror`, and +`MPI_Comm_rank` returns the rank together with that error code. + +## 5. Edit the facade into a Python API + +The contract is yours to edit. Replace `contract/mpi_f08.pyi`, the facade the +extension publishes, with this one: -## 5. Build from the contract + +```python +from prik.contracts import Annotated, AnyNative, Arg, Flat, Hidden, Int32, ReadOnly, Return, bind, native_call, raises +from .mpi_f08_types import ( + Mpi_Comm, + Mpi_Datatype, + Mpi_Op, + Mpi_Status, + mpi_any_source, + mpi_any_tag, + mpi_byte, + mpi_comm_world, + mpi_double, + mpi_in_place, + mpi_int, + mpi_max, + mpi_sum, +) + +@raises(status="ierror", success=0) +@bind("MPI_Init") +@native_call([Hidden("ierror", Int32)]) +def init() -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Finalize") +@native_call([Hidden("ierror", Int32)]) +def finalize() -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Comm_rank") +@native_call([Arg(0), Return("rank", 0), Hidden("ierror", Int32)]) +def comm_rank(comm: Mpi_Comm) -> Int32: ... + +@raises(status="ierror", success=0) +@bind("MPI_Comm_size") +@native_call([Arg(0), Return("size", 0), Hidden("ierror", Int32)]) +def comm_size(comm: Mpi_Comm) -> Int32: ... + +@raises(status="ierror", success=0) +@bind("MPI_Barrier") +@native_call([Arg(0), Hidden("ierror", Int32)]) +def barrier(comm: Mpi_Comm) -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Send") +@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Arg(4), Hidden("ierror", Int32)]) +def send( + buf: Annotated[AnyNative[Flat], ReadOnly], + datatype: Mpi_Datatype, + dest: Int32, + tag: Int32, + comm: Mpi_Comm, +) -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Recv") +@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Arg(4), Return("status", 0), Hidden("ierror", Int32)]) +def recv( + buf: AnyNative[Flat], + datatype: Mpi_Datatype, + source: Int32, + tag: Int32, + comm: Mpi_Comm, +) -> Mpi_Status: ... + +@raises(status="ierror", success=0) +@bind("MPI_Probe") +@native_call([Arg(0), Arg(1), Arg(2), Return("status", 0), Hidden("ierror", Int32)]) +def probe(source: Int32, tag: Int32, comm: Mpi_Comm) -> Mpi_Status: ... + +@raises(status="ierror", success=0) +@bind("MPI_Get_count") +@native_call([Arg(0), Arg(1), Return("count", 0), Hidden("ierror", Int32)]) +def get_count(status: Mpi_Status, datatype: Mpi_Datatype) -> Int32: ... + +@raises(status="ierror", success=0) +@bind("MPI_Bcast") +@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Hidden("ierror", Int32)]) +def bcast(buffer: AnyNative[Flat], datatype: Mpi_Datatype, root: Int32, comm: Mpi_Comm) -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Reduce") +@native_call([Arg(0), Arg(1), Int32(Arg(1).size), Arg(2), Arg(3), Arg(4), Arg(5), Hidden("ierror", Int32)]) +def reduce( + sendbuf: Annotated[AnyNative[Flat], ReadOnly], + recvbuf: AnyNative[Flat], + datatype: Mpi_Datatype, + op: Mpi_Op, + root: Int32, + comm: Mpi_Comm, +) -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Allreduce") +@native_call([Arg(0), Arg(1), Int32(Arg(1).size), Arg(2), Arg(3), Arg(4), Hidden("ierror", Int32)]) +def allreduce( + sendbuf: Annotated[AnyNative[Flat], ReadOnly], + recvbuf: AnyNative[Flat], + datatype: Mpi_Datatype, + op: Mpi_Op, + comm: Mpi_Comm, +) -> None: ... + +__all__ = [ + "init", + "finalize", + "comm_rank", + "comm_size", + "barrier", + "send", + "recv", + "probe", + "get_count", + "bcast", + "reduce", + "allreduce", + "Mpi_Comm", + "Mpi_Datatype", + "Mpi_Op", + "Mpi_Status", + "mpi_any_source", + "mpi_any_tag", + "mpi_byte", + "mpi_comm_world", + "mpi_double", + "mpi_in_place", + "mpi_int", + "mpi_max", + "mpi_sum", +] +``` + +Each function still calls the Fortran routine it names; only its Python face +changes. `@native_call` lists the native arguments in Fortran order and says +where each one comes from: + +| Edit | Example | Effect in Python | +| --- | --- | --- | +| `@bind("MPI_Send")` on `def send` | every function | The Python name differs from the Fortran name it calls. | +| `Int32(Arg(0).size)` | `count` of `MPI_Send` | The count is computed from the buffer, so the caller does not pass it. | +| `Return("status", 0)` | `status` of `MPI_Recv` | The output argument becomes the return value: `recv` returns an `Mpi_Status`, `comm_rank` a rank. | +| `Hidden("ierror", Int32)` with `@raises(status="ierror", success=0)` | every function | The error code is not an argument; a nonzero code raises an exception. | + +The facade imports its handle types and constants from +`mpi_f08_types` in both Open MPI 4.1 and 5.0: in 5.0 that module re-exports +them from `mpi_types`, and PRIK follows the re-export to the declaration. So +the same edited file serves both release series. + +## 6. Build from the contract The contract, not the Open MPI sources, is what the extension is built from: @@ -256,7 +466,7 @@ Open MPI Fortran sources | PRIK semantic analysis | -restricted, editable .pyi contract +restricted .pyi contract, edited into the API you want | PRIK wrapper generation | @@ -284,94 +494,196 @@ links against its libraries. `--compiler` takes the compiler `mpifort` runs rather than `mpifort` itself because PRIK identifies a Fortran compiler's family from its executable name. -## 6. Write an MPI program +The extension can already run MPI: -This program uses every selected routine: rank 0 sends a NumPy array to -rank 1, and then every rank takes part in three reductions, the last one in -place. +```python +import numpy as np - +from prik_openmpi_f08 import mpi_f08 + +mpi_f08.init() +values = np.array([1, 2], dtype=np.int32) +total = np.empty_like(values) +mpi_f08.allreduce(values, total, mpi_f08.mpi_int, mpi_f08.mpi_sum, mpi_f08.mpi_comm_world) +mpi_f08.finalize() +``` + +## 7. Add the mpi4py-style layer + +A contract describes native calls. What mpi4py adds on top of MPI is a Python +object model, and that belongs in Python. Save this module as `prik_mpi.py` +beside `prik_openmpi_f08.so`: + + ```python +"""An mpi4py-style Python API over the PRIK-generated Open MPI extension.""" + +import atexit +import pickle + import numpy as np -from prik_openmpi_f08 import mpi_f08 as mpi +from prik_openmpi_f08 import mpi_f08 as _mpi -mpi.mpi_init() -world = mpi.mpi_comm_world +ANY_SOURCE = int(_mpi.mpi_any_source) +ANY_TAG = int(_mpi.mpi_any_tag) +IN_PLACE = _mpi.mpi_in_place +BYTE = _mpi.mpi_byte +INT = _mpi.mpi_int +DOUBLE = _mpi.mpi_double +SUM = _mpi.mpi_sum +MAX = _mpi.mpi_max -rank, _ = mpi.mpi_comm_rank(world) -size, _ = mpi.mpi_comm_size(world) -rank, size = int(rank), int(size) +# The MPI datatype of each NumPy element type, for buffers given without one. +_DATATYPES = {np.dtype(np.uint8): BYTE, np.dtype(np.int32): INT, np.dtype(np.float64): DOUBLE} -# Point to point: rank 0 sends four integers, rank 1 receives them. -if rank == 0: - sent = np.array([3, 5, 7, 11], dtype=np.int32) - mpi.mpi_send(sent, np.int32(sent.size), mpi.mpi_int, np.int32(1), np.int32(13), world) -elif rank == 1: - received = np.empty(4, dtype=np.int32) - mpi.mpi_recv( - received, - np.int32(received.size), - mpi.mpi_int, - np.int32(0), - np.int32(13), - world, - mpi.mpi_status_ignore, - ) - print(f"rank 1 received {received.tolist()}") - -# Collective: every rank contributes and every rank receives the sum. -values = np.array([rank + 1, rank + 2], dtype=np.int32) -reduced = np.empty_like(values) -mpi.mpi_allreduce(values, reduced, np.int32(values.size), mpi.mpi_int, mpi.mpi_sum, world) - -readings = np.array([20.0 + rank], dtype=np.float64) -total = np.empty_like(readings) -mpi.mpi_allreduce(readings, total, np.int32(readings.size), mpi.mpi_double_precision, mpi.mpi_sum, world) - -# In place: MPI_IN_PLACE as the send buffer reduces the receive buffer itself. -in_place = values.copy() -mpi.mpi_allreduce(mpi.mpi_in_place, in_place, np.int32(in_place.size), mpi.mpi_int, mpi.mpi_sum, world) - -mpi.mpi_barrier(world) -print(f"rank {rank} of {size}: sum {reduced.tolist()}, in place {in_place.tolist()}, total {total.tolist()}") -mpi.mpi_finalize() -``` -Two details follow from the contract: +def _message(buf): + """Return a buffer's array and MPI datatype; ``buf`` is an array or ``[array, datatype]``.""" + if isinstance(buf, list | tuple): + array, datatype = buf + return array, datatype + return buf, _DATATYPES[buf.dtype] + + +class Status: + """What MPI reports about a received message.""" + + def __init__(self): + self._native = None + + @property + def source(self): + return int(self._native.mpi_source) + + @property + def tag(self): + return int(self._native.mpi_tag) + + def Get_source(self): + return self.source + + def Get_tag(self): + return self.tag + + def Get_count(self, datatype=BYTE): + return int(_mpi.get_count(self._native, datatype)) + + +def _report(status, native): + if status is not None: + status._native = native + -- Counts, ranks, and tags are Fortran `INTEGER` values, which the contract - declares as `Int32`. Pass them as `np.int32`, the exact NumPy type, and use - `np.int32` arrays with `mpi_int` and `np.float64` arrays with - `mpi_double_precision`, so the element type matches the MPI datatype. -- `mpi_comm_rank` returns its result with the optional error code, which is - `None` when omitted, so `rank, _ = ...` unpacks it. `rank` is a NumPy - integer; `int(rank)` gives a plain Python value. +class Comm: + """A communicator, with the methods mpi4py spells for it.""" -Save it as `mpi_example.py` beside `prik_openmpi_f08.so`. + def __init__(self, handle): + self.handle = handle -## 7. Run it under Open MPI + def Get_rank(self): + return int(_mpi.comm_rank(self.handle)) -Start two ranks with the installed Open MPI launcher: + def Get_size(self): + return int(_mpi.comm_size(self.handle)) + + rank = property(Get_rank) + size = property(Get_size) + + def Barrier(self): + _mpi.barrier(self.handle) + + def Send(self, buf, dest, tag=0): + array, datatype = _message(buf) + _mpi.send(array, datatype, np.int32(dest), np.int32(tag), self.handle) + + def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG, status=None): + array, datatype = _message(buf) + _report(status, _mpi.recv(array, datatype, np.int32(source), np.int32(tag), self.handle)) + + def Probe(self, source=ANY_SOURCE, tag=ANY_TAG, status=None): + _report(status, _mpi.probe(np.int32(source), np.int32(tag), self.handle)) + return True + + def Bcast(self, buf, root=0): + array, datatype = _message(buf) + _mpi.bcast(array, datatype, np.int32(root), self.handle) + + def Reduce(self, sendbuf, recvbuf, op=SUM, root=0): + array, datatype = _message(recvbuf) + _mpi.reduce(sendbuf, array, datatype, op, np.int32(root), self.handle) + + def Allreduce(self, sendbuf, recvbuf, op=SUM): + array, datatype = _message(recvbuf) + _mpi.allreduce(sendbuf, array, datatype, op, self.handle) + + # Python objects travel pickled, as with mpi4py's lowercase methods. + def send(self, obj, dest, tag=0): + self.Send(np.frombuffer(pickle.dumps(obj), dtype=np.uint8), dest, tag) + + def recv(self, buf=None, source=ANY_SOURCE, tag=ANY_TAG, status=None): + status = status if status is not None else Status() + self.Probe(source, tag, status) + data = np.empty(status.Get_count(BYTE), dtype=np.uint8) + self.Recv(data, status.source, status.tag, status) + return pickle.loads(data.tobytes()) + + +COMM_WORLD = Comm(_mpi.mpi_comm_world) + +# Like mpi4py, MPI starts when this module is imported and stops at exit. +_mpi.init() +atexit.register(_mpi.finalize) +``` + +Everything in it calls the generated functions of step 5: + +- **Objects and methods.** `Comm` wraps an `Mpi_Comm` handle and spells + mpi4py's methods; `COMM_WORLD` wraps `mpi_comm_world`. `Status` keeps the + `Mpi_Status` that `recv` and `probe` return and answers `Get_source`, + `Get_tag`, and `Get_count` from it. +- **Buffers.** A buffer is a NumPy array, whose MPI datatype is chosen from + its element type, or an `[array, datatype]` pair naming the datatype + explicitly, as in mpi4py. +- **Defaults and Python integers.** `tag=0`, `source=ANY_SOURCE`, `root=0`, + and `op=SUM` are keyword defaults, and plain Python integers are converted + to the `np.int32` values the contract's `Int32` arguments take. +- **Python objects.** Lowercase `send` pickles an object into a byte array and + sends it; `recv` probes the incoming message, sizes a byte array with + `Get_count`, receives it, and unpickles it. +- **Lifetime.** As with mpi4py, importing the module initializes MPI, and MPI + is finalized when the interpreter exits. + +## 8. Run it under Open MPI + +Save the program from the top of this page as `mpi_example.py` beside +`prik_mpi.py` and `prik_openmpi_f08.so`, and start two ranks with the +installed Open MPI launcher: ```bash mpirun -n 2 python3 mpi_example.py ``` -The two ranks print these lines, in whichever order they finish: +The two ranks print these lines, each rank's lines in order but the ranks in +whichever order they finish: ```text -rank 1 received [3, 5, 7, 11] -rank 0 of 2: sum [3, 5], in place [3, 5], total [41.0] -rank 1 of 2: sum [3, 5], in place [3, 5], total [41.0] +rank 1 received {'a': 7, 'b': 3.14} +rank 1 received [0, 1, 2, 3] from rank 0 +rank 0 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5] +rank 0 max [2, 3] +rank 1 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5] ``` Here is what happened. `mpirun` started two Python processes as MPI ranks. -Each call went through PRIK's generated binding and bridge straight into the -installed Open MPI library. The NumPy arrays crossed the boundary as native -buffers: `mpi_recv` wrote into rank 1's `received`, and each `mpi_allreduce` -wrote into `reduced`, `total`, and `in_place` directly. No part of Open MPI -was rebuilt. +Each call went through `prik_mpi.py`, then PRIK's generated binding and +bridge, straight into the installed Open MPI library. The NumPy arrays crossed +the boundary as native buffers: `Recv` wrote into rank 1's `data`, `Bcast` +into every rank's `data`, and each reduction into its receive array directly. +No part of Open MPI was rebuilt. + +With `from mpi4py import MPI` in place of `import prik_mpi as MPI`, mpi4py +runs the same program and prints the same lines. ## Why the configured tree must match the installation @@ -418,7 +730,7 @@ exercised in CI, not the only ones that can work. ## Limitations -This tutorial selected fourteen names; the rest of `mpi_f08` works the same +This tutorial selected twenty-one names; the rest of `mpi_f08` works the same way when you select it, within these limits of what PRIK supports today: - **Arrays of handles.** Routines taking an array of derived-type values, such @@ -433,4 +745,14 @@ way when you select it, within these limits of what PRIK supports today: - **Nonblocking buffers.** Routines such as `MPI_Isend` wrap, but the operation keeps using its buffer after the call returns. PRIK does not hold on to that NumPy array, so your program must keep it alive and unchanged - until the operation completes. + until the operation completes. This is also why `prik_mpi.py` offers no + `Isend` or `Irecv`: mpi4py's request objects keep their buffers alive, and + a faithful imitation would need arrays of requests, the first limitation. + +`prik_mpi.py` imitates the part of mpi4py this program uses, not all of it. +It passes contiguous NumPy arrays only -- a strided view is refused with a +`TypeError` -- and picks a datatype for `int32`, `float64`, and `uint8` +arrays. It has none of mpi4py's other communicators, lowercase collectives, +or `MPI.Exception`: under Open MPI's default error handler an MPI error aborts +the job, and otherwise a nonzero `ierror` raises the exception the contract's +`@raises` produces. diff --git a/prik/semantics/pyi2ir.py b/prik/semantics/pyi2ir.py index 13fe6c825..b5a16a688 100644 --- a/prik/semantics/pyi2ir.py +++ b/prik/semantics/pyi2ir.py @@ -4060,14 +4060,26 @@ def _external_module_candidates(module_name: str) -> tuple[str, ...]: def _prototypes_with_reexports(modules: list[SemanticModule]) -> dict[tuple[str, str], SemanticPrototype]: - """Index every prototype name a contract module binds, declared or re-exported. + """Index every prototype name a contract module binds, declared or re-exported.""" + declared = { + (module.name, prototype.name): (module.name, prototype) for module in modules for prototype in module.prototypes + } + return {key: declaration for key, (_module, declaration) in _bound_with_reexports(modules, declared).items()} + - A module that imports a prototype and publishes it binds that name without - declaring it, so a consumer importing it from there must still resolve to - the declaring module. Repeating to a fixed point follows a chain of any - length. +def _bound_with_reexports( + modules: list[SemanticModule], + declared: dict[tuple[str, str], tuple[str, object]], +) -> dict[tuple[str, str], tuple[str, object]]: + """Index every name a contract module binds to a declaration, declared or re-exported. + + ``declared`` maps ``(module, name)`` to ``(declaring module, declaration)``. + A module that imports a declaration and publishes it binds that name + without declaring it, so a consumer importing it from there must still + resolve to the declaring module. Repeating to a fixed point follows a + chain of any length. """ - resolved = {(module.name, prototype.name): prototype for module in modules for prototype in module.prototypes} + resolved = dict(declared) changed = True while changed: changed = False @@ -4079,16 +4091,16 @@ def _prototypes_with_reexports(modules: list[SemanticModule]) -> dict[tuple[str, local_name = item.target or item.source if (module.name, local_name) in resolved: continue - prototype = next( + found = next( ( - found + match for candidate in _external_module_candidates(imported.module) - if (found := resolved.get((candidate, item.source))) is not None + if (match := resolved.get((candidate, item.source))) is not None ), None, ) - if prototype is not None: - resolved[(module.name, local_name)] = prototype + if found is not None: + resolved[(module.name, local_name)] = found changed = True return resolved @@ -4133,7 +4145,16 @@ def reconcile_external_type_refs(modules: list[SemanticModule]) -> list[Semantic classes are marked ``wrapped`` or ``opaque``. The same list is returned for pipeline chaining; absent external definitions remain opaque references. """ - definitions = {(module.name, declaration.name): declaration for module in modules for declaration in module.classes} + # A class imported through a module that re-exports it is the class its + # declaring module defines, so the reference names that module. + definitions = _bound_with_reexports( + modules, + { + (module.name, declaration.name): (module.name, declaration) + for module in modules + for declaration in module.classes + }, + ) declared_class_names = { module.name: frozenset(declaration.name for declaration in module.classes) for module in modules } @@ -4146,7 +4167,20 @@ def reconcile_external_type_refs(modules: list[SemanticModule]) -> list[Semantic continue if _bind_referenced_prototype(semantic_type, ref, prototypes, declared_class_names): continue - declaration = definitions.get((ref.get("origin_module"), ref.get("name"))) + candidates = _external_module_candidates(str(ref.get("origin_module") or "")) + found = next( + ( + match + for candidate in candidates + if (match := definitions.get((candidate, ref.get("name")))) is not None + ), + None, + ) + declaration = None + if found is not None: + declaring_module, declaration = found + if declaring_module not in candidates: + ref["origin_module"] = declaring_module wrapped = declaration is not None and ( not isinstance(declaration, SemanticClass) or "Opaque" not in declaration.base_classes ) diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi new file mode 100644 index 000000000..dd3d8c7dd --- /dev/null +++ b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi @@ -0,0 +1,129 @@ +from prik.contracts import Annotated, AnyNative, Arg, Flat, Hidden, Int32, ReadOnly, Return, bind, native_call, raises +from .mpi_f08_types import ( + Mpi_Comm, + Mpi_Datatype, + Mpi_Op, + Mpi_Status, + mpi_any_source, + mpi_any_tag, + mpi_byte, + mpi_comm_world, + mpi_double, + mpi_in_place, + mpi_int, + mpi_max, + mpi_sum, +) + +@raises(status="ierror", success=0) +@bind("MPI_Init") +@native_call([Hidden("ierror", Int32)]) +def init() -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Finalize") +@native_call([Hidden("ierror", Int32)]) +def finalize() -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Comm_rank") +@native_call([Arg(0), Return("rank", 0), Hidden("ierror", Int32)]) +def comm_rank(comm: Mpi_Comm) -> Int32: ... + +@raises(status="ierror", success=0) +@bind("MPI_Comm_size") +@native_call([Arg(0), Return("size", 0), Hidden("ierror", Int32)]) +def comm_size(comm: Mpi_Comm) -> Int32: ... + +@raises(status="ierror", success=0) +@bind("MPI_Barrier") +@native_call([Arg(0), Hidden("ierror", Int32)]) +def barrier(comm: Mpi_Comm) -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Send") +@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Arg(4), Hidden("ierror", Int32)]) +def send( + buf: Annotated[AnyNative[Flat], ReadOnly], + datatype: Mpi_Datatype, + dest: Int32, + tag: Int32, + comm: Mpi_Comm, +) -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Recv") +@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Arg(4), Return("status", 0), Hidden("ierror", Int32)]) +def recv( + buf: AnyNative[Flat], + datatype: Mpi_Datatype, + source: Int32, + tag: Int32, + comm: Mpi_Comm, +) -> Mpi_Status: ... + +@raises(status="ierror", success=0) +@bind("MPI_Probe") +@native_call([Arg(0), Arg(1), Arg(2), Return("status", 0), Hidden("ierror", Int32)]) +def probe(source: Int32, tag: Int32, comm: Mpi_Comm) -> Mpi_Status: ... + +@raises(status="ierror", success=0) +@bind("MPI_Get_count") +@native_call([Arg(0), Arg(1), Return("count", 0), Hidden("ierror", Int32)]) +def get_count(status: Mpi_Status, datatype: Mpi_Datatype) -> Int32: ... + +@raises(status="ierror", success=0) +@bind("MPI_Bcast") +@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Hidden("ierror", Int32)]) +def bcast(buffer: AnyNative[Flat], datatype: Mpi_Datatype, root: Int32, comm: Mpi_Comm) -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Reduce") +@native_call([Arg(0), Arg(1), Int32(Arg(1).size), Arg(2), Arg(3), Arg(4), Arg(5), Hidden("ierror", Int32)]) +def reduce( + sendbuf: Annotated[AnyNative[Flat], ReadOnly], + recvbuf: AnyNative[Flat], + datatype: Mpi_Datatype, + op: Mpi_Op, + root: Int32, + comm: Mpi_Comm, +) -> None: ... + +@raises(status="ierror", success=0) +@bind("MPI_Allreduce") +@native_call([Arg(0), Arg(1), Int32(Arg(1).size), Arg(2), Arg(3), Arg(4), Hidden("ierror", Int32)]) +def allreduce( + sendbuf: Annotated[AnyNative[Flat], ReadOnly], + recvbuf: AnyNative[Flat], + datatype: Mpi_Datatype, + op: Mpi_Op, + comm: Mpi_Comm, +) -> None: ... + +__all__ = [ + "init", + "finalize", + "comm_rank", + "comm_size", + "barrier", + "send", + "recv", + "probe", + "get_count", + "bcast", + "reduce", + "allreduce", + "Mpi_Comm", + "Mpi_Datatype", + "Mpi_Op", + "Mpi_Status", + "mpi_any_source", + "mpi_any_tag", + "mpi_byte", + "mpi_comm_world", + "mpi_double", + "mpi_in_place", + "mpi_int", + "mpi_max", + "mpi_sum", +] diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py index efb295c05..43284e28c 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py @@ -1,44 +1,41 @@ import numpy as np -from prik_openmpi_f08 import mpi_f08 as mpi +import prik_mpi as MPI -mpi.mpi_init() -world = mpi.mpi_comm_world +comm = MPI.COMM_WORLD +rank = comm.Get_rank() +size = comm.Get_size() -rank, _ = mpi.mpi_comm_rank(world) -size, _ = mpi.mpi_comm_size(world) -rank, size = int(rank), int(size) - -# Point to point: rank 0 sends four integers, rank 1 receives them. +# Python objects travel pickled. if rank == 0: - sent = np.array([3, 5, 7, 11], dtype=np.int32) - mpi.mpi_send(sent, np.int32(sent.size), mpi.mpi_int, np.int32(1), np.int32(13), world) + comm.send({"a": 7, "b": 3.14}, dest=1, tag=11) elif rank == 1: - received = np.empty(4, dtype=np.int32) - mpi.mpi_recv( - received, - np.int32(received.size), - mpi.mpi_int, - np.int32(0), - np.int32(13), - world, - mpi.mpi_status_ignore, - ) - print(f"rank 1 received {received.tolist()}") - -# Collective: every rank contributes and every rank receives the sum. -values = np.array([rank + 1, rank + 2], dtype=np.int32) -reduced = np.empty_like(values) -mpi.mpi_allreduce(values, reduced, np.int32(values.size), mpi.mpi_int, mpi.mpi_sum, world) - -readings = np.array([20.0 + rank], dtype=np.float64) -total = np.empty_like(readings) -mpi.mpi_allreduce(readings, total, np.int32(readings.size), mpi.mpi_double_precision, mpi.mpi_sum, world) - -# In place: MPI_IN_PLACE as the send buffer reduces the receive buffer itself. -in_place = values.copy() -mpi.mpi_allreduce(mpi.mpi_in_place, in_place, np.int32(in_place.size), mpi.mpi_int, mpi.mpi_sum, world) + data = comm.recv(source=0, tag=11) + print(f"rank 1 received {data}") -mpi.mpi_barrier(world) -print(f"rank {rank} of {size}: sum {reduced.tolist()}, in place {in_place.tolist()}, total {total.tolist()}") -mpi.mpi_finalize() +# NumPy arrays travel as buffers, with an explicit MPI datatype ... +if rank == 0: + data = np.arange(4, dtype="i") + comm.Send([data, MPI.INT], dest=1, tag=77) +elif rank == 1: + data = np.empty(4, dtype="i") + status = MPI.Status() + comm.Recv([data, MPI.INT], source=MPI.ANY_SOURCE, tag=77, status=status) + print(f"rank 1 received {data.tolist()} from rank {status.Get_source()}") + +# ... or with the datatype taken from the array. +data = np.arange(3, dtype=np.float64) if rank == 0 else np.empty(3, dtype=np.float64) +comm.Bcast(data, root=0) + +# Collectives: every rank contributes. +values = np.array([rank + 1, rank + 2], dtype="i") +total = np.empty_like(values) +comm.Allreduce(values, total, op=MPI.SUM) +largest = np.empty_like(values) +comm.Reduce(values, largest, op=MPI.MAX, root=0) +comm.Allreduce(MPI.IN_PLACE, values, op=MPI.SUM) + +comm.Barrier() +print(f"rank {rank} of {size}: bcast {data.tolist()}, sum {total.tolist()}, in place {values.tolist()}") +if rank == 0: + print(f"rank 0 max {largest.tolist()}") diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py new file mode 100644 index 000000000..715c3d027 --- /dev/null +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py @@ -0,0 +1,118 @@ +"""An mpi4py-style Python API over the PRIK-generated Open MPI extension.""" + +import atexit +import pickle + +import numpy as np + +from prik_openmpi_f08 import mpi_f08 as _mpi + +ANY_SOURCE = int(_mpi.mpi_any_source) +ANY_TAG = int(_mpi.mpi_any_tag) +IN_PLACE = _mpi.mpi_in_place +BYTE = _mpi.mpi_byte +INT = _mpi.mpi_int +DOUBLE = _mpi.mpi_double +SUM = _mpi.mpi_sum +MAX = _mpi.mpi_max + +# The MPI datatype of each NumPy element type, for buffers given without one. +_DATATYPES = {np.dtype(np.uint8): BYTE, np.dtype(np.int32): INT, np.dtype(np.float64): DOUBLE} + + +def _message(buf): + """Return a buffer's array and MPI datatype; ``buf`` is an array or ``[array, datatype]``.""" + if isinstance(buf, list | tuple): + array, datatype = buf + return array, datatype + return buf, _DATATYPES[buf.dtype] + + +class Status: + """What MPI reports about a received message.""" + + def __init__(self): + self._native = None + + @property + def source(self): + return int(self._native.mpi_source) + + @property + def tag(self): + return int(self._native.mpi_tag) + + def Get_source(self): + return self.source + + def Get_tag(self): + return self.tag + + def Get_count(self, datatype=BYTE): + return int(_mpi.get_count(self._native, datatype)) + + +def _report(status, native): + if status is not None: + status._native = native + + +class Comm: + """A communicator, with the methods mpi4py spells for it.""" + + def __init__(self, handle): + self.handle = handle + + def Get_rank(self): + return int(_mpi.comm_rank(self.handle)) + + def Get_size(self): + return int(_mpi.comm_size(self.handle)) + + rank = property(Get_rank) + size = property(Get_size) + + def Barrier(self): + _mpi.barrier(self.handle) + + def Send(self, buf, dest, tag=0): + array, datatype = _message(buf) + _mpi.send(array, datatype, np.int32(dest), np.int32(tag), self.handle) + + def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG, status=None): + array, datatype = _message(buf) + _report(status, _mpi.recv(array, datatype, np.int32(source), np.int32(tag), self.handle)) + + def Probe(self, source=ANY_SOURCE, tag=ANY_TAG, status=None): + _report(status, _mpi.probe(np.int32(source), np.int32(tag), self.handle)) + return True + + def Bcast(self, buf, root=0): + array, datatype = _message(buf) + _mpi.bcast(array, datatype, np.int32(root), self.handle) + + def Reduce(self, sendbuf, recvbuf, op=SUM, root=0): + array, datatype = _message(recvbuf) + _mpi.reduce(sendbuf, array, datatype, op, np.int32(root), self.handle) + + def Allreduce(self, sendbuf, recvbuf, op=SUM): + array, datatype = _message(recvbuf) + _mpi.allreduce(sendbuf, array, datatype, op, self.handle) + + # Python objects travel pickled, as with mpi4py's lowercase methods. + def send(self, obj, dest, tag=0): + self.Send(np.frombuffer(pickle.dumps(obj), dtype=np.uint8), dest, tag) + + def recv(self, buf=None, source=ANY_SOURCE, tag=ANY_TAG, status=None): + status = status if status is not None else Status() + self.Probe(source, tag, status) + data = np.empty(status.Get_count(BYTE), dtype=np.uint8) + self.Recv(data, status.source, status.tag, status) + return pickle.loads(data.tobytes()) + + +COMM_WORLD = Comm(_mpi.mpi_comm_world) + +# Like mpi4py, MPI starts when this module is imported and stops at exit. +_mpi.init() +atexit.register(_mpi.finalize) diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index 41be8d965..31e6a21e3 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -1,9 +1,10 @@ """Real Open MPI source to generated contract to two-rank execution. The test runs the commands the Open MPI ``mpi_f08`` tutorial shows: generate -a restricted contract from the configured Open MPI sources, build it against -the installation without compiling any Open MPI source, and run the tutorial's -program under the Open MPI launcher. +a restricted contract from the configured Open MPI sources, replace its facade +with the tutorial's edited one, build it against the installation without +compiling any Open MPI source, and run the tutorial's mpi4py-style program +under the Open MPI launcher. """ from __future__ import annotations @@ -23,8 +24,10 @@ pytestmark = pytest.mark.fortran_end_to_end -# The tutorial displays this program; the test runs it as written. -RUNTIME = Path(__file__).parent / "fixtures" / "runtime" / "mpi_example.py" +# The tutorial displays these files; the test uses them as written. +FIXTURES = Path(__file__).parent / "fixtures" +EDITED_FACADE = FIXTURES / "contracts" / "openmpi" / "mpi_f08.pyi" +PROGRAM = (FIXTURES / "runtime" / "prik_mpi.py", FIXTURES / "runtime" / "mpi_example.py") EXPORTS = ( "MPI_Init", "MPI_Finalize", @@ -33,13 +36,20 @@ "MPI_Barrier", "MPI_Send", "MPI_Recv", + "MPI_Probe", + "MPI_Get_count", + "MPI_Bcast", + "MPI_Reduce", "MPI_Allreduce", "MPI_COMM_WORLD", + "MPI_BYTE", "MPI_INT", - "MPI_DOUBLE_PRECISION", + "MPI_DOUBLE", "MPI_SUM", + "MPI_MAX", "MPI_IN_PLACE", - "MPI_STATUS_IGNORE", + "MPI_ANY_SOURCE", + "MPI_ANY_TAG", ) # ``ompi_info`` reports these for the configure run that built the # installation, and a configured tree records the same values, so they @@ -201,11 +211,13 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) assert "mpi_comm_world: Final[Mpi_Comm]" in types assert "mpi_sum: Final[Mpi_Op]" in types assert "mpi_int: Final[Mpi_Datatype]" in types - assert "mpi_status_ignore: Mpi_Status" in types + assert "mpi_any_source: Final[Int32]" in types # Where the handle types are declared depends on the Open MPI version. declarations = "".join(path.read_text(encoding="utf-8") for path in contract.glob("*.pyi")) assert all(f"class Mpi_{name}" in declarations for name in ("Comm", "Datatype", "Op", "Status")) assert "AnyNative[" in interfaces and '@overload("mpi_send_f08")\ndef mpi_send(' in interfaces + # The tutorial edits the generated facade into the Python API it wants. + shutil.copyfile(EDITED_FACADE, contract / "mpi_f08.pyi") def showme(flag: str) -> list[str]: return shlex.split(_tool_output([mpifort, f"--showme:{flag}"], f"mpifort --showme:{flag}")) @@ -250,21 +262,25 @@ def showme(flag: str) -> list[str]: "prik_openmpi_f08_wrapper.o", ] bridge = (tmp_path / "extension" / "bind_c_prik_openmpi_f08_wrapper.f90").read_text(encoding="utf-8") - assert "use mpi_f08_interfaces, only:" in bridge - assert "=> MPI_Allreduce" in bridge and "=> MPI_Send" in bridge + assert "native_allreduce => MPI_Allreduce" in bridge and "native_send => MPI_Send" in bridge + for path in PROGRAM: + shutil.copyfile(path, tmp_path / "extension" / path.name) env = os.environ.copy() env["PYTHONPATH"] = os.pathsep.join(filter(None, (str(tmp_path / "extension"), env.get("PYTHONPATH", "")))) env["LD_LIBRARY_PATH"] = os.pathsep.join((*showme("libdirs"), env.get("LD_LIBRARY_PATH", ""))) completed = subprocess.run( - [launcher, "-n", "2", sys.executable, str(RUNTIME)], + [launcher, "-n", "2", sys.executable, str(tmp_path / "extension" / "mpi_example.py")], env=env, check=True, capture_output=True, text=True, timeout=120, ) - output = completed.stdout.splitlines() - assert "rank 1 received [3, 5, 7, 11]" in output - for rank in (0, 1): - assert f"rank {rank} of 2: sum [3, 5], in place [3, 5], total [41.0]" in output + assert sorted(completed.stdout.splitlines()) == [ + "rank 0 max [2, 3]", + "rank 0 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5]", + "rank 1 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5]", + "rank 1 received [0, 1, 2, 3] from rank 0", + "rank 1 received {'a': 7, 'b': 3.14}", + ] diff --git a/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/semantics/test_reexported_class_references.py b/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/semantics/test_reexported_class_references.py new file mode 100644 index 000000000..8746e58c8 --- /dev/null +++ b/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/semantics/test_reexported_class_references.py @@ -0,0 +1,50 @@ +"""A contract class imported through a module that re-exports it is its declaration.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from prik.pipeline.pyi import pyi_paths_to_semantic_modules +from prik.semantics.models import EXTERNAL_TYPE_REF_METADATA + +HANDLES = """\ +from prik.contracts import Int32 + +class Handle: + def __init__( + self, + *, + value: Int32 = ... + ) -> None: ... + +__all__ = ["Handle"] +""" + +API = """\ +from prik.contracts import Int32 +from .{importer} import Handle + +def value_of(handle: Handle) -> Int32: ... + +__all__ = ["value_of"] +""" + + +@pytest.mark.parametrize("chain", [("facade",), ("facade", "outer")], ids=["one-reexport", "reexport-chain"]) +def test_class_imported_through_reexports_references_its_declaring_module(tmp_path: Path, chain: tuple[str, ...]): + (tmp_path / "handles.pyi").write_text(HANDLES, encoding="utf-8") + source = "handles" + for name in chain: + (tmp_path / f"{name}.pyi").write_text( + f'from .{source} import Handle\n\n__all__ = ["Handle"]\n', encoding="utf-8" + ) + source = name + (tmp_path / "api.pyi").write_text(API.format(importer=source), encoding="utf-8") + + modules = pyi_paths_to_semantic_modules(tmp_path) + + api = next(module for module in modules if module.name == "api") + ref = api.functions[0].arguments[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] + assert (ref["origin_module"], ref["wrapped"], ref["representation"]) == ("handles", True, "wrapped") From 9b24711b19d745844d6ea27561602040708bb72f Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 00:26:49 +0100 Subject: [PATCH 25/47] Call public interface-body specifics by their own names A specific declared by an interface body is a module entity like any other, so only a specific its module keeps private has to be called through the generic. Generated contracts for such generics, Open MPI's mpi_f08 among them, now link each candidate with @overload alone. The @bind reference and the functions guide now say that an @overload declaration calls the specific it links, not a procedure named like the Python function, and needs @bind only to call through a generic. Co-Authored-By: Claude Opus 5.5 --- docs/user/guide/wrapping-functions.md | 6 ++++ docs/user/reference/pyi-format.md | 11 +++++--- docs/user/tutorials/openmpi-f08.md | 5 +++- prik/semantics/fortran2ir.py | 19 ++++++------- .../native/interface_body_generic.f90 | 15 ++++++++++ .../native/interface_body_generic_impl.f90 | 13 +++++++++ .../end_to_end/test_generic_interfaces.py | 21 ++++++++++++++ .../test_fortran_generic_semantics.py | 28 +++++++++++++++++++ 8 files changed, 103 insertions(+), 15 deletions(-) create mode 100644 tests/fortran/generic_interfaces/end_to_end/fixtures/native/interface_body_generic.f90 create mode 100644 tests/fortran/generic_interfaces/end_to_end/fixtures/native/interface_body_generic_impl.f90 diff --git a/docs/user/guide/wrapping-functions.md b/docs/user/guide/wrapping-functions.md index 2e107e5cc..82327ea50 100644 --- a/docs/user/guide/wrapping-functions.md +++ b/docs/user/guide/wrapping-functions.md @@ -39,6 +39,12 @@ print(result) # 7.5 A contract declaration normally uses one name for both Python and the native procedure. Use `@bind("native_name")` only when those names differ. +An `@overload("specific")` declaration is the exception: it calls the specific +it links, not a procedure named like the Python function, so it needs +`@bind("generic")` only to call through the generic instead -- as when the +module keeps the specific private. See +[Generic Interfaces](generic-interfaces.md). + For example, rename the generated declaration to `multiply` and add `@bind("scale")`. The Python name changes, while the native target remains `scale`. The generated contract and its edited replacement are shown below. diff --git a/docs/user/reference/pyi-format.md b/docs/user/reference/pyi-format.md index 680dd34f9..a633df633 100644 --- a/docs/user/reference/pyi-format.md +++ b/docs/user/reference/pyi-format.md @@ -550,7 +550,9 @@ generated constructor form. Methods use the same rules plus an untyped `self`. `Pass()` places that object in an explicit native argument list. `@bind(...)` is needed only when the -Python declaration and native callable names differ. +Python declaration and native callable names differ. For an `@overload(...)` +declaration the native callable defaults to the linked specific, not the +Python name; see [Generic Procedure Overloads](#generic-procedure-overloads). ### Function And Method Decorators @@ -605,9 +607,10 @@ def convert(value: Int32) -> Int32: ... def convert(value: Float64) -> Float64: ... ``` -The linked concrete declaration owns `@native_call`. An overload-level -`@bind(...)` selects a public native generic when the specific itself is not the -link target. Runtime dispatch distinguishes exact scalar dtype, array element +The linked concrete declaration owns `@native_call`, and without `@bind(...)` +the candidate calls that specific by its own name, whatever the Python name. +An overload-level `@bind(...)` calls a public native generic instead, which is +needed when the module keeps the specific private. Runtime dispatch distinguishes exact scalar dtype, array element dtype and rank, or wrapped class; it does not use implicit numeric coercion. An `AnyNative` choice-buffer argument can appear in a generic with one selected candidate; its concrete wrapper validates the actual storage at the call. diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 201ce67bd..92a063dca 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -247,7 +247,6 @@ and each selected routine keeps its Fortran interface in `contract/mpi_f08_interfaces.pyi`: ```python -@bind("MPI_Allreduce") @overload("mpi_allreduce_f08") def mpi_allreduce( sendbuf: Annotated[AnyNative[Flat], ReadOnly], @@ -260,6 +259,10 @@ def mpi_allreduce( ) -> Returns["ierror", Int32[()]] | None: ... ``` +`MPI_Allreduce` is a generic whose one specific is `MPI_Allreduce_f08`, so +`mpi_allreduce` is an overload of the specific declared as `mpi_allreduce_f08` +in the same file, and a call to it calls that specific. + Three mappings are worth a closer look. **Handles are concrete types.** `Mpi_Comm`, `Mpi_Datatype`, `Mpi_Op`, and diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index 182e17d7f..fbfec2063 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -3302,20 +3302,21 @@ def _declared_specific(procedure: SemanticFunction) -> tuple[str, str]: ).key @staticmethod - def _bind_unreachable_specifics_through_generic( + def _bind_private_specifics_through_generic( overload_set: ProcedureOverloadSet, targets: list[_SpecificProcedure], lookup: dict[tuple[str, str], SemanticFunction], generic_name: str, - interface_body_targets: set[tuple[str, str]], ) -> None: - """Bind specifics without a module name through their reachable generic. + """Bind each private specific through the generic name that reaches it. - A private module procedure and a procedure declared only inside an - interface body are both callable through the generic name. + A specific its declaring module keeps private is unreachable by its own + name, while the generic -- or, for a constructor, the type name -- is + public and resolves to the same procedure. A public specific, including + one declared only by an interface body, is called by its own name. """ for target, candidate in zip(targets, overload_set.procedures, strict=True): - if lookup[target.key].visibility == "private" or target.key in interface_body_targets: + if lookup[target.key].visibility == "private": candidate.native_name = generic_name candidate.metadata[BIND_TARGET_METADATA] = generic_name @@ -3393,12 +3394,11 @@ def _module_overload_sets( # constructor, so its specifics become the class's own # `__init__` overload set rather than a module generic. constructor_set = self._normal_overload_set("__init__", procedures) - self._bind_unreachable_specifics_through_generic( + self._bind_private_specifics_through_generic( constructor_set, target_names, own_lookup | inline_lookup | inherited_lookup, interface.name, - set(inline_lookup), ) self._merge_overload_sets(constructor_class.overload_sets, [constructor_set]) self._mark_constructor_specifics(procedures, own_lookup, interface.name) @@ -3411,12 +3411,11 @@ def _module_overload_sets( else module.name, visibility=self._symbol_visibility(module, interface.name), ) - self._bind_unreachable_specifics_through_generic( + self._bind_private_specifics_through_generic( overload_set, target_names, own_lookup | inline_lookup | inherited_lookup, interface.name, - set(inline_lookup), ) overload_sets.append(overload_set) continue diff --git a/tests/fortran/generic_interfaces/end_to_end/fixtures/native/interface_body_generic.f90 b/tests/fortran/generic_interfaces/end_to_end/fixtures/native/interface_body_generic.f90 new file mode 100644 index 000000000..6c84e08d5 --- /dev/null +++ b/tests/fortran/generic_interfaces/end_to_end/fixtures/native/interface_body_generic.f90 @@ -0,0 +1,15 @@ +module interface_body_generic + implicit none + public + + interface scale + function scale_integer(value) result(output) + integer, intent(in) :: value + integer :: output + end function scale_integer + function scale_real(value) result(output) + real(8), intent(in) :: value + real(8) :: output + end function scale_real + end interface scale +end module interface_body_generic diff --git a/tests/fortran/generic_interfaces/end_to_end/fixtures/native/interface_body_generic_impl.f90 b/tests/fortran/generic_interfaces/end_to_end/fixtures/native/interface_body_generic_impl.f90 new file mode 100644 index 000000000..fc0a6911d --- /dev/null +++ b/tests/fortran/generic_interfaces/end_to_end/fixtures/native/interface_body_generic_impl.f90 @@ -0,0 +1,13 @@ +function scale_integer(value) result(output) + implicit none + integer, intent(in) :: value + integer :: output + output = 2 * value +end function scale_integer + +function scale_real(value) result(output) + implicit none + real(8), intent(in) :: value + real(8) :: output + output = 2.5_8 * value +end function scale_real diff --git a/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py b/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py index e74fbb36f..26db5bb1b 100644 --- a/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py +++ b/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py @@ -22,6 +22,10 @@ PRIVATE_INLINE_GENERIC_SUBMODULE = (NATIVE_FIXTURES / "private_inline_generic_impl.f90").read_text(encoding="utf-8") +INTERFACE_BODY_GENERIC_MODULE = (NATIVE_FIXTURES / "interface_body_generic.f90").read_text(encoding="utf-8") + +INTERFACE_BODY_GENERIC_IMPL = (NATIVE_FIXTURES / "interface_body_generic_impl.f90").read_text(encoding="utf-8") + @pytest.fixture def compiled_generic_module( @@ -102,6 +106,23 @@ def test_public_generic_dispatches_to_private_inline_submodule_specifics(tmp_pat assert "=> shift_real" not in bridge +def test_public_generic_calls_public_interface_body_specifics_by_their_own_names(tmp_path: Path): + """A public specific an interface body declares needs no route through the generic.""" + module, _payload = _build_sources_and_import( + [ + ("interface_body_generic.f90", INTERFACE_BODY_GENERIC_MODULE), + ("interface_body_generic_impl.f90", INTERFACE_BODY_GENERIC_IMPL), + ], + tmp_path, + ) + + assert module.interface_body_generic.scale(np.int32(4)) == np.int32(8) + assert module.interface_body_generic.scale(np.float64(4.0)) == np.float64(10.0) + bridge = (tmp_path / "bind_c_interface_body_generic_wrapper.f90").read_text(encoding="utf-8").lower() + assert "native__prik_overload_scale_0 => scale_integer" in bridge + assert "native__prik_overload_scale_1 => scale_real" in bridge + + EXTENDED_GENERIC_SOURCE = (NATIVE_FIXTURES / "extended_generic.f90").read_text(encoding="utf-8") diff --git a/tests/fortran/generic_interfaces/semantics/test_fortran_generic_semantics.py b/tests/fortran/generic_interfaces/semantics/test_fortran_generic_semantics.py index 001b3a2a4..6f31aa303 100644 --- a/tests/fortran/generic_interfaces/semantics/test_fortran_generic_semantics.py +++ b/tests/fortran/generic_interfaces/semantics/test_fortran_generic_semantics.py @@ -58,6 +58,34 @@ def test_public_generic_binds_private_inline_module_function_specifics_to_the_ge assert [candidate.metadata[BIND_TARGET_METADATA] for candidate in candidates] == ["shift", "shift"] +def test_public_interface_body_specifics_are_called_by_their_own_names(): + source = """ +module generic_mod + implicit none + interface shift + subroutine shift_integer(value) + integer, intent(inout) :: value + end subroutine shift_integer + subroutine shift_real(value) + real, intent(inout) :: value + end subroutine shift_real + end interface shift + private :: shift_real +end module generic_mod +""" + + module = FortranToIRConverter().visit(parse_fortran_source(source).modules[0]) + candidates = module.overload_sets[0].procedures + + # An interface body declares a module entity like any other, so only the + # specific the module keeps private must be reached through the generic. + assert [(candidate.name, candidate.native_name) for candidate in candidates] == [ + ("shift_integer", "shift_integer"), + ("shift_real", "shift"), + ] + assert [candidate.metadata.get(BIND_TARGET_METADATA) for candidate in candidates] == [None, "shift"] + + def test_converter_projects_a_generic_constructor_onto_its_class(): """An interface named for a derived type is that type's constructor. From 122fef643b2cd4eca2d2cf5df7263f3af05a42c2 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 06:46:56 +0100 Subject: [PATCH 26/47] Run the mpi_f08 tutorial in one directory; route an ignored status through MPI_STATUS_IGNORE The integration test now follows the tutorial step by step in one working directory: it builds with --out-dir build, imports the extension alias the CLI writes in that directory, and runs the Python files saved beside it with nothing added to the environment. The export list is one fixture the tutorial displays and the test reads. MPI_STATUS_IGNORE is selected again. The edited facade keeps recv's and probe's status as an argument, so the mpi4py-style Recv passes its own Mpi_Status or, given none, Open MPI's MPI_STATUS_IGNORE, as mpi4py does. A runtime check shows Open MPI recognizes that object, and a module test shows a module variable argument is passed as the variable itself. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 5 +- docs/user/tutorials/openmpi-f08.md | 85 ++++++++------ .../contracts/openmpi/mpi_exports.txt | 22 ++++ .../fixtures/contracts/openmpi/mpi_f08.pyi | 11 +- .../fixtures/runtime/mpi_example.py | 10 +- .../runtime/mpi_status_ignore_check.py | 27 +++++ .../end_to_end/fixtures/runtime/prik_mpi.py | 15 +-- .../end_to_end/test_openmpi_f08.py | 104 +++++++++--------- .../native/module_variable_arguments.f90 | 20 ++++ .../test_module_variables_and_state.py | 23 ++++ 10 files changed, 219 insertions(+), 103 deletions(-) create mode 100644 tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_exports.txt create mode 100644 tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_status_ignore_check.py create mode 100644 tests/fortran/modules/end_to_end/fixtures/native/module_variable_arguments.f90 diff --git a/CHANGELOG.md b/CHANGELOG.md index dd1245312..c09b003d0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -132,7 +132,10 @@ release tags add a leading `v` to the package version. Python module spelling mpi4py's `COMM_WORLD`, `Send`/`Recv`, pickled `send`/`recv`, `Bcast`, `Reduce`, and `Allreduce`, and runs a two-rank program under `mpirun` that mpi4py also runs unchanged but for its import. - An opt-in integration test runs the same commands and program; the Open MPI Integration lane runs it against Open MPI 4.1.8 and + `Recv` without a status passes Open MPI's own `MPI_STATUS_IGNORE`, a + generated `Mpi_Status` module object. An opt-in integration test follows + the tutorial's steps in one working directory and checks that Open MPI + recognizes that object; the Open MPI Integration lane runs it against Open MPI 4.1.8 and 5.0.11 built from source. The test requires the configured tree to record the same configure run as the installation, and reports missing or failing Open MPI tools as unavailable -- a skip locally, a failure where Open MPI is diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 92a063dca..c5fca3d53 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -29,8 +29,9 @@ size = comm.Get_size() if rank == 0: comm.send({"a": 7, "b": 3.14}, dest=1, tag=11) elif rank == 1: - data = comm.recv(source=0, tag=11) - print(f"rank 1 received {data}") + status = MPI.Status() + data = comm.recv(source=MPI.ANY_SOURCE, tag=11, status=status) + print(f"rank 1 received {data} from rank {status.Get_source()}") # NumPy arrays travel as buffers, with an explicit MPI datatype ... if rank == 0: @@ -38,9 +39,8 @@ if rank == 0: comm.Send([data, MPI.INT], dest=1, tag=77) elif rank == 1: data = np.empty(4, dtype="i") - status = MPI.Status() - comm.Recv([data, MPI.INT], source=MPI.ANY_SOURCE, tag=77, status=status) - print(f"rank 1 received {data.tolist()} from rank {status.Get_source()}") + comm.Recv([data, MPI.INT], source=0, tag=77) + print(f"rank 1 received {data.tolist()}") # ... or with the datatype taken from the array. data = np.arange(3, dtype=np.float64) if rank == 0 else np.empty(3, dtype=np.float64) @@ -105,6 +105,7 @@ integer MPI_ANY_SOURCE parameter (MPI_ANY_SOURCE=-1) integer, bind(C, name="mpi_fortran_in_place_") :: MPI_IN_PLACE +type(MPI_Status), bind(C, name="mpi_fortran_status_ignore_") :: MPI_STATUS_IGNORE ``` and each MPI routine is a generic interface over its specific procedures: @@ -139,6 +140,7 @@ are in headers that Open MPI's `configure` writes into its build tree. subset instead of all of them. Save the Fortran identities to publish in `mpi_exports.txt`: + ```text mpi_f08::MPI_Init mpi_f08::MPI_Finalize @@ -159,6 +161,7 @@ mpi_f08::MPI_DOUBLE mpi_f08::MPI_SUM mpi_f08::MPI_MAX mpi_f08::MPI_IN_PLACE +mpi_f08::MPI_STATUS_IGNORE mpi_f08::MPI_ANY_SOURCE mpi_f08::MPI_ANY_TAG ``` @@ -166,7 +169,8 @@ mpi_f08::MPI_ANY_TAG These are the routines and objects behind the mpi4py names the program uses. `MPI_Probe` and `MPI_Get_count` are not called by the program directly; they let lowercase `recv` size its buffer before receiving a pickled object, as -mpi4py does. +mpi4py does. Nor is `MPI_STATUS_IGNORE`: like mpi4py, `Recv` passes it when it +is given no `Status`. Selecting symbols this way is not an MPI feature. `--export-symbols` accepts module-qualified public symbols from any Fortran project -- procedures, @@ -220,7 +224,7 @@ The `contract/` directory holds one editable `.pyi` file per Fortran module the selection needs. `contract/mpi_f08.pyi` publishes the selected names: ```python -from .mpi_f08_types import mpi_any_source, mpi_any_tag, mpi_byte, mpi_comm_world, mpi_double, mpi_in_place, mpi_int, mpi_max, mpi_sum +from .mpi_f08_types import mpi_any_source, mpi_any_tag, mpi_byte, mpi_comm_world, mpi_double, mpi_in_place, mpi_int, mpi_max, mpi_status_ignore, mpi_sum from .mpi_f08_interfaces import mpi_allreduce, mpi_barrier, mpi_bcast, mpi_comm_rank, mpi_comm_size, mpi_finalize, mpi_get_count, mpi_init, mpi_probe, mpi_recv, mpi_reduce, mpi_send from .mpi_types import Mpi_Comm, Mpi_Datatype, Mpi_Op, Mpi_Status ``` @@ -241,6 +245,8 @@ mpi_sum: Final[Mpi_Op] mpi_int: Final[Mpi_Datatype] mpi_in_place: Int32[()] + +mpi_status_ignore: Mpi_Status ``` and each selected routine keeps its Fortran interface in @@ -263,7 +269,7 @@ def mpi_allreduce( `mpi_allreduce` is an overload of the specific declared as `mpi_allreduce_f08` in the same file, and a call to it calls that specific. -Three mappings are worth a closer look. +Four mappings are worth a closer look. **Handles are concrete types.** `Mpi_Comm`, `Mpi_Datatype`, `Mpi_Op`, and `Mpi_Status` are the derived types Open MPI declares, with their real @@ -296,6 +302,12 @@ that variable, which is how Open MPI recognizes an in-place operation. Nothing here knows about MPI: it is the same mapping any Fortran module variable declared this way receives. +**`MPI_STATUS_IGNORE` is an `Mpi_Status` object.** It is declared as a +C-bound module variable of type `MPI_Status`, so it keeps that type: it is not +a generic buffer, and it is accepted wherever an `Mpi_Status` is. Passing a +module variable passes the variable itself, not a copy, so Open MPI receives +its own `MPI_STATUS_IGNORE` and recognizes it by address. + **Choice buffers are `AnyNative`.** The send and receive buffers of MPI routines are `type(*)` dummies, which accept data of any type, so they become `AnyNative[Flat]`: any NumPy array, passed as a raw address. `AnyNative` @@ -327,6 +339,7 @@ from .mpi_f08_types import ( mpi_in_place, mpi_int, mpi_max, + mpi_status_ignore, mpi_sum, ) @@ -368,19 +381,20 @@ def send( @raises(status="ierror", success=0) @bind("MPI_Recv") -@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Arg(4), Return("status", 0), Hidden("ierror", Int32)]) +@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Arg(4), Arg(5), Hidden("ierror", Int32)]) def recv( buf: AnyNative[Flat], datatype: Mpi_Datatype, source: Int32, tag: Int32, comm: Mpi_Comm, -) -> Mpi_Status: ... + status: Mpi_Status, +) -> None: ... @raises(status="ierror", success=0) @bind("MPI_Probe") -@native_call([Arg(0), Arg(1), Arg(2), Return("status", 0), Hidden("ierror", Int32)]) -def probe(source: Int32, tag: Int32, comm: Mpi_Comm) -> Mpi_Status: ... +@native_call([Arg(0), Arg(1), Arg(2), Arg(3), Hidden("ierror", Int32)]) +def probe(source: Int32, tag: Int32, comm: Mpi_Comm, status: Mpi_Status) -> None: ... @raises(status="ierror", success=0) @bind("MPI_Get_count") @@ -440,6 +454,7 @@ __all__ = [ "mpi_in_place", "mpi_int", "mpi_max", + "mpi_status_ignore", "mpi_sum", ] ``` @@ -452,9 +467,13 @@ where each one comes from: | --- | --- | --- | | `@bind("MPI_Send")` on `def send` | every function | The Python name differs from the Fortran name it calls. | | `Int32(Arg(0).size)` | `count` of `MPI_Send` | The count is computed from the buffer, so the caller does not pass it. | -| `Return("status", 0)` | `status` of `MPI_Recv` | The output argument becomes the return value: `recv` returns an `Mpi_Status`, `comm_rank` a rank. | +| `Return("rank", 0)` | `rank` of `MPI_Comm_rank` | The output argument becomes the return value: `comm_rank` returns the rank, `get_count` the count. | | `Hidden("ierror", Int32)` with `@raises(status="ierror", success=0)` | every function | The error code is not an argument; a nonzero code raises an exception. | +`recv` and `probe` keep their `status` as an argument instead of returning +it. The caller passes either an `Mpi_Status`, which Open MPI fills in, or +`mpi_status_ignore`, which tells Open MPI not to. + The facade imports its handle types and constants from `mpi_f08_types` in both Open MPI 4.1 and 5.0: in 5.0 that module re-exports them from `mpi_types`, and PRIK follows the re-export to the declaration. So @@ -490,7 +509,9 @@ python3 -m prik contract/__init__.pyi \ --out-dir build ``` -This writes `prik_openmpi_f08.so` in the current directory. It compiles only +This compiles in `build/` and writes `prik_openmpi_f08.so`, a stable copy of +the extension, in the current directory; the rest of this tutorial works in +that directory and imports the extension from there. The build compiles only the bridge and binding PRIK generates; no Open MPI source is compiled. The installed Open MPI already provides the implementation, and the extension links against its libraries. `--compiler` takes the compiler `mpifort` runs @@ -515,7 +536,7 @@ mpi_f08.finalize() A contract describes native calls. What mpi4py adds on top of MPI is a Python object model, and that belongs in Python. Save this module as `prik_mpi.py` -beside `prik_openmpi_f08.so`: +in the same directory as `prik_openmpi_f08.so`: ```python @@ -531,6 +552,7 @@ from prik_openmpi_f08 import mpi_f08 as _mpi ANY_SOURCE = int(_mpi.mpi_any_source) ANY_TAG = int(_mpi.mpi_any_tag) IN_PLACE = _mpi.mpi_in_place +STATUS_IGNORE = _mpi.mpi_status_ignore BYTE = _mpi.mpi_byte INT = _mpi.mpi_int DOUBLE = _mpi.mpi_double @@ -550,10 +572,10 @@ def _message(buf): class Status: - """What MPI reports about a received message.""" + """What MPI reports about a received message, filled in by the call given it.""" def __init__(self): - self._native = None + self._native = _mpi.Mpi_Status() @property def source(self): @@ -573,9 +595,9 @@ class Status: return int(_mpi.get_count(self._native, datatype)) -def _report(status, native): - if status is not None: - status._native = native +def _native_status(status): + """Return the status MPI fills in; without one, MPI_STATUS_IGNORE, as in mpi4py.""" + return STATUS_IGNORE if status is None else status._native class Comm: @@ -602,10 +624,10 @@ class Comm: def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG, status=None): array, datatype = _message(buf) - _report(status, _mpi.recv(array, datatype, np.int32(source), np.int32(tag), self.handle)) + _mpi.recv(array, datatype, np.int32(source), np.int32(tag), self.handle, _native_status(status)) def Probe(self, source=ANY_SOURCE, tag=ANY_TAG, status=None): - _report(status, _mpi.probe(np.int32(source), np.int32(tag), self.handle)) + _mpi.probe(np.int32(source), np.int32(tag), self.handle, _native_status(status)) return True def Bcast(self, buf, root=0): @@ -642,9 +664,10 @@ atexit.register(_mpi.finalize) Everything in it calls the generated functions of step 5: - **Objects and methods.** `Comm` wraps an `Mpi_Comm` handle and spells - mpi4py's methods; `COMM_WORLD` wraps `mpi_comm_world`. `Status` keeps the - `Mpi_Status` that `recv` and `probe` return and answers `Get_source`, - `Get_tag`, and `Get_count` from it. + mpi4py's methods; `COMM_WORLD` wraps `mpi_comm_world`. `Status` owns an + `Mpi_Status` that `Recv` and `Probe` have Open MPI fill in, and answers + `Get_source`, `Get_tag`, and `Get_count` from it. Given no `Status`, they + pass `MPI_STATUS_IGNORE`, as mpi4py does. - **Buffers.** A buffer is a NumPy array, whose MPI datatype is chosen from its element type, or an `[array, datatype]` pair naming the datatype explicitly, as in mpi4py. @@ -659,9 +682,9 @@ Everything in it calls the generated functions of step 5: ## 8. Run it under Open MPI -Save the program from the top of this page as `mpi_example.py` beside -`prik_mpi.py` and `prik_openmpi_f08.so`, and start two ranks with the -installed Open MPI launcher: +Save the program from the top of this page as `mpi_example.py` in the same +directory, beside `prik_mpi.py` and `prik_openmpi_f08.so`, and start two ranks +there with the installed Open MPI launcher: ```bash mpirun -n 2 python3 mpi_example.py @@ -671,8 +694,8 @@ The two ranks print these lines, each rank's lines in order but the ranks in whichever order they finish: ```text -rank 1 received {'a': 7, 'b': 3.14} -rank 1 received [0, 1, 2, 3] from rank 0 +rank 1 received {'a': 7, 'b': 3.14} from rank 0 +rank 1 received [0, 1, 2, 3] rank 0 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5] rank 0 max [2, 3] rank 1 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5] @@ -733,7 +756,7 @@ exercised in CI, not the only ones that can work. ## Limitations -This tutorial selected twenty-one names; the rest of `mpi_f08` works the same +This tutorial selected twenty-two names; the rest of `mpi_f08` works the same way when you select it, within these limits of what PRIK supports today: - **Arrays of handles.** Routines taking an array of derived-type values, such diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_exports.txt b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_exports.txt new file mode 100644 index 000000000..61d2c4c67 --- /dev/null +++ b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_exports.txt @@ -0,0 +1,22 @@ +mpi_f08::MPI_Init +mpi_f08::MPI_Finalize +mpi_f08::MPI_Comm_rank +mpi_f08::MPI_Comm_size +mpi_f08::MPI_Barrier +mpi_f08::MPI_Send +mpi_f08::MPI_Recv +mpi_f08::MPI_Probe +mpi_f08::MPI_Get_count +mpi_f08::MPI_Bcast +mpi_f08::MPI_Reduce +mpi_f08::MPI_Allreduce +mpi_f08::MPI_COMM_WORLD +mpi_f08::MPI_BYTE +mpi_f08::MPI_INT +mpi_f08::MPI_DOUBLE +mpi_f08::MPI_SUM +mpi_f08::MPI_MAX +mpi_f08::MPI_IN_PLACE +mpi_f08::MPI_STATUS_IGNORE +mpi_f08::MPI_ANY_SOURCE +mpi_f08::MPI_ANY_TAG diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi index dd3d8c7dd..081bbbc76 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi +++ b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi @@ -12,6 +12,7 @@ from .mpi_f08_types import ( mpi_in_place, mpi_int, mpi_max, + mpi_status_ignore, mpi_sum, ) @@ -53,19 +54,20 @@ def send( @raises(status="ierror", success=0) @bind("MPI_Recv") -@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Arg(4), Return("status", 0), Hidden("ierror", Int32)]) +@native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Arg(4), Arg(5), Hidden("ierror", Int32)]) def recv( buf: AnyNative[Flat], datatype: Mpi_Datatype, source: Int32, tag: Int32, comm: Mpi_Comm, -) -> Mpi_Status: ... + status: Mpi_Status, +) -> None: ... @raises(status="ierror", success=0) @bind("MPI_Probe") -@native_call([Arg(0), Arg(1), Arg(2), Return("status", 0), Hidden("ierror", Int32)]) -def probe(source: Int32, tag: Int32, comm: Mpi_Comm) -> Mpi_Status: ... +@native_call([Arg(0), Arg(1), Arg(2), Arg(3), Hidden("ierror", Int32)]) +def probe(source: Int32, tag: Int32, comm: Mpi_Comm, status: Mpi_Status) -> None: ... @raises(status="ierror", success=0) @bind("MPI_Get_count") @@ -125,5 +127,6 @@ __all__ = [ "mpi_in_place", "mpi_int", "mpi_max", + "mpi_status_ignore", "mpi_sum", ] diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py index 43284e28c..6ec773072 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py @@ -10,8 +10,9 @@ if rank == 0: comm.send({"a": 7, "b": 3.14}, dest=1, tag=11) elif rank == 1: - data = comm.recv(source=0, tag=11) - print(f"rank 1 received {data}") + status = MPI.Status() + data = comm.recv(source=MPI.ANY_SOURCE, tag=11, status=status) + print(f"rank 1 received {data} from rank {status.Get_source()}") # NumPy arrays travel as buffers, with an explicit MPI datatype ... if rank == 0: @@ -19,9 +20,8 @@ comm.Send([data, MPI.INT], dest=1, tag=77) elif rank == 1: data = np.empty(4, dtype="i") - status = MPI.Status() - comm.Recv([data, MPI.INT], source=MPI.ANY_SOURCE, tag=77, status=status) - print(f"rank 1 received {data.tolist()} from rank {status.Get_source()}") + comm.Recv([data, MPI.INT], source=0, tag=77) + print(f"rank 1 received {data.tolist()}") # ... or with the datatype taken from the array. data = np.arange(3, dtype=np.float64) if rank == 0 else np.empty(3, dtype=np.float64) diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_status_ignore_check.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_status_ignore_check.py new file mode 100644 index 000000000..fd4ad0740 --- /dev/null +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_status_ignore_check.py @@ -0,0 +1,27 @@ +"""Open MPI receives MPI_STATUS_IGNORE as the predefined object itself. + +PRIK passes a module variable argument as that variable's own storage. Open +MPI fills in every status it is given except MPI_STATUS_IGNORE, which it +recognizes by that address, so a receive through it leaves it untouched while +an ordinary status is filled in. +""" + +import numpy as np + +from prik_openmpi_f08 import mpi_f08 as mpi + +mpi.init() +world = mpi.mpi_comm_world +ignore = mpi.mpi_status_ignore +if int(mpi.comm_rank(world)) == 0: + for tag in (21, 22): + mpi.send(np.arange(2, dtype=np.int32), mpi.mpi_int, np.int32(1), np.int32(tag), world) +else: + data = np.empty(2, dtype=np.int32) + status = mpi.Mpi_Status() + mpi.recv(data, mpi.mpi_int, np.int32(0), np.int32(21), world, status) + before = int(ignore.mpi_tag) + mpi.recv(data, mpi.mpi_int, np.int32(0), np.int32(22), world, ignore) + unchanged = int(ignore.mpi_tag) == before + print(f"{type(ignore).__name__}: status tag {int(status.mpi_tag)}, ignored status unchanged {unchanged}") +mpi.finalize() diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py index 715c3d027..68818706f 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py @@ -10,6 +10,7 @@ ANY_SOURCE = int(_mpi.mpi_any_source) ANY_TAG = int(_mpi.mpi_any_tag) IN_PLACE = _mpi.mpi_in_place +STATUS_IGNORE = _mpi.mpi_status_ignore BYTE = _mpi.mpi_byte INT = _mpi.mpi_int DOUBLE = _mpi.mpi_double @@ -29,10 +30,10 @@ def _message(buf): class Status: - """What MPI reports about a received message.""" + """What MPI reports about a received message, filled in by the call given it.""" def __init__(self): - self._native = None + self._native = _mpi.Mpi_Status() @property def source(self): @@ -52,9 +53,9 @@ def Get_count(self, datatype=BYTE): return int(_mpi.get_count(self._native, datatype)) -def _report(status, native): - if status is not None: - status._native = native +def _native_status(status): + """Return the status MPI fills in; without one, MPI_STATUS_IGNORE, as in mpi4py.""" + return STATUS_IGNORE if status is None else status._native class Comm: @@ -81,10 +82,10 @@ def Send(self, buf, dest, tag=0): def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG, status=None): array, datatype = _message(buf) - _report(status, _mpi.recv(array, datatype, np.int32(source), np.int32(tag), self.handle)) + _mpi.recv(array, datatype, np.int32(source), np.int32(tag), self.handle, _native_status(status)) def Probe(self, source=ANY_SOURCE, tag=ANY_TAG, status=None): - _report(status, _mpi.probe(np.int32(source), np.int32(tag), self.handle)) + _mpi.probe(np.int32(source), np.int32(tag), self.handle, _native_status(status)) return True def Bcast(self, buf, root=0): diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index 31e6a21e3..20a694703 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -1,10 +1,11 @@ """Real Open MPI source to generated contract to two-rank execution. -The test runs the commands the Open MPI ``mpi_f08`` tutorial shows: generate -a restricted contract from the configured Open MPI sources, replace its facade -with the tutorial's edited one, build it against the installation without -compiling any Open MPI source, and run the tutorial's mpi4py-style program -under the Open MPI launcher. +The test follows the Open MPI ``mpi_f08`` tutorial step by step, in one +working directory as a reader would: generate a restricted contract from the +configured Open MPI sources, replace its facade with the tutorial's edited +one, build it against the installation without compiling any Open MPI source, +save the tutorial's Python files beside the extension, and run its +mpi4py-style program there under the Open MPI launcher. """ from __future__ import annotations @@ -26,31 +27,11 @@ pytestmark = pytest.mark.fortran_end_to_end # The tutorial displays these files; the test uses them as written. FIXTURES = Path(__file__).parent / "fixtures" +EXPORT_LIST = FIXTURES / "contracts" / "openmpi" / "mpi_exports.txt" EDITED_FACADE = FIXTURES / "contracts" / "openmpi" / "mpi_f08.pyi" PROGRAM = (FIXTURES / "runtime" / "prik_mpi.py", FIXTURES / "runtime" / "mpi_example.py") -EXPORTS = ( - "MPI_Init", - "MPI_Finalize", - "MPI_Comm_rank", - "MPI_Comm_size", - "MPI_Barrier", - "MPI_Send", - "MPI_Recv", - "MPI_Probe", - "MPI_Get_count", - "MPI_Bcast", - "MPI_Reduce", - "MPI_Allreduce", - "MPI_COMM_WORLD", - "MPI_BYTE", - "MPI_INT", - "MPI_DOUBLE", - "MPI_SUM", - "MPI_MAX", - "MPI_IN_PLACE", - "MPI_ANY_SOURCE", - "MPI_ANY_TAG", -) +# Not shown in the tutorial: shows MPI_STATUS_IGNORE reaches Open MPI as itself. +STATUS_IGNORE_CHECK = FIXTURES / "runtime" / "mpi_status_ignore_check.py" # ``ompi_info`` reports these for the configure run that built the # installation, and a configured tree records the same values, so they # identify that run: its date, host, user, and exact command line. @@ -163,11 +144,10 @@ def _configured_openmpi() -> tuple[Path, Path, str, str]: def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) -> None: - """The tutorial's contract, build, and program run against a real Open MPI.""" + """The tutorial's steps, run in one directory, communicate through a real Open MPI.""" source, build, mpifort, launcher = _configured_openmpi() - contract = tmp_path / "contract" - exports = tmp_path / "mpi_exports.txt" - exports.write_text("".join(f"mpi_f08::{symbol}\n" for symbol in EXPORTS), encoding="utf-8") + shutil.copyfile(EXPORT_LIST, tmp_path / "mpi_exports.txt") + exports = [line.partition("::")[2] for line in EXPORT_LIST.read_text(encoding="utf-8").split()] includes = ( build, build / "ompi/mpi/fortran/use-mpi-f08", @@ -176,6 +156,7 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) build / "ompi/include", source / "ompi/include", ) + # Step 3: generate the contract. subprocess.run( [ sys.executable, @@ -189,9 +170,9 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) "--module-source-dir", str(build), "--export-symbols", - str(exports), + "mpi_exports.txt", "--out", - str(contract), + "contract", "--compiler", mpifort, *(part for include in includes for part in ("-I", str(include))), @@ -200,14 +181,19 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) capture_output=True, text=True, timeout=300, + cwd=tmp_path, ) + contract = tmp_path / "contract" facade = (contract / "mpi_f08.pyi").read_text(encoding="utf-8") types = (contract / "mpi_f08_types.pyi").read_text(encoding="utf-8") interfaces = (contract / "mpi_f08_interfaces.pyi").read_text(encoding="utf-8") - assert all(f'"{symbol.lower()}"' in facade for symbol in EXPORTS) + assert all(f'"{symbol.lower()}"' in facade for symbol in exports) assert all(f'"Mpi_{name}"' in facade for name in ("Comm", "Datatype", "Op", "Status")) assert "mpi_waitall" not in facade + # Predefined objects keep their declared types: MPI_IN_PLACE is native + # integer storage, and MPI_STATUS_IGNORE an Mpi_Status object. assert "mpi_in_place: Int32[()]" in types + assert "mpi_status_ignore: Mpi_Status" in types assert "mpi_comm_world: Final[Mpi_Comm]" in types assert "mpi_sum: Final[Mpi_Op]" in types assert "mpi_int: Final[Mpi_Datatype]" in types @@ -216,7 +202,7 @@ def test_openmpi_f08_contract_replay_and_two_rank_communication(tmp_path: Path) declarations = "".join(path.read_text(encoding="utf-8") for path in contract.glob("*.pyi")) assert all(f"class Mpi_{name}" in declarations for name in ("Comm", "Datatype", "Op", "Status")) assert "AnyNative[" in interfaces and '@overload("mpi_send_f08")\ndef mpi_send(' in interfaces - # The tutorial edits the generated facade into the Python API it wants. + # Step 5: replace the generated facade with the edited one. shutil.copyfile(EDITED_FACADE, contract / "mpi_f08.pyi") def showme(flag: str) -> list[str]: @@ -227,12 +213,14 @@ def showme(flag: str) -> list[str]: command = showme("command") if len(command) != 1: _unavailable(f"mpifort --showme:command is a multi-token command {command}; pass one compiler executable") + # Step 6: build, with the tutorial's options; --jobs and --json only bound + # the compiler processes and report the build. built = subprocess.run( [ sys.executable, "-m", "prik", - str(contract / "__init__.pyi"), + "contract/__init__.pyi", "--compiler", command[0], f"--wrapper-fortran-flags={shlex.join(showme('compile'))}", @@ -243,7 +231,7 @@ def showme(flag: str) -> list[str]: "--out", "prik_openmpi_f08", "--out-dir", - str(tmp_path / "extension"), + "build", "--jobs", "2", "--json", @@ -261,26 +249,32 @@ def showme(flag: str) -> list[str]: "bind_c_prik_openmpi_f08_wrapper.o", "prik_openmpi_f08_wrapper.o", ] - bridge = (tmp_path / "extension" / "bind_c_prik_openmpi_f08_wrapper.f90").read_text(encoding="utf-8") + bridge = (tmp_path / "build" / "bind_c_prik_openmpi_f08_wrapper.f90").read_text(encoding="utf-8") assert "native_allreduce => MPI_Allreduce" in bridge and "native_send => MPI_Send" in bridge - for path in PROGRAM: - shutil.copyfile(path, tmp_path / "extension" / path.name) + # The tutorial imports the extension from the working directory. + assert (tmp_path / "prik_openmpi_f08.so").is_file() - env = os.environ.copy() - env["PYTHONPATH"] = os.pathsep.join(filter(None, (str(tmp_path / "extension"), env.get("PYTHONPATH", "")))) - env["LD_LIBRARY_PATH"] = os.pathsep.join((*showme("libdirs"), env.get("LD_LIBRARY_PATH", ""))) - completed = subprocess.run( - [launcher, "-n", "2", sys.executable, str(tmp_path / "extension" / "mpi_example.py")], - env=env, - check=True, - capture_output=True, - text=True, - timeout=120, - ) - assert sorted(completed.stdout.splitlines()) == [ + # Steps 7 and 8: save the Python files beside the extension and run the + # program there, with nothing added to the environment. + for path in (*PROGRAM, STATUS_IGNORE_CHECK): + shutil.copyfile(path, tmp_path / path.name) + + def run(script: str) -> list[str]: + completed = subprocess.run( + [launcher, "-n", "2", sys.executable, script], + check=True, + capture_output=True, + text=True, + timeout=120, + cwd=tmp_path, + ) + return sorted(completed.stdout.splitlines()) + + assert run("mpi_example.py") == [ "rank 0 max [2, 3]", "rank 0 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5]", "rank 1 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5]", - "rank 1 received [0, 1, 2, 3] from rank 0", - "rank 1 received {'a': 7, 'b': 3.14}", + "rank 1 received [0, 1, 2, 3]", + "rank 1 received {'a': 7, 'b': 3.14} from rank 0", ] + assert run(STATUS_IGNORE_CHECK.name) == ["Mpi_Status: status tag 21, ignored status unchanged True"] diff --git a/tests/fortran/modules/end_to_end/fixtures/native/module_variable_arguments.f90 b/tests/fortran/modules/end_to_end/fixtures/native/module_variable_arguments.f90 new file mode 100644 index 000000000..c4f19c5f4 --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/native/module_variable_arguments.f90 @@ -0,0 +1,20 @@ +module module_variable_arguments + use iso_c_binding + implicit none + type, bind(C) :: box + integer(c_int) :: v + end type box + type(box), target :: shared + type(box), target, bind(C, name="module_variable_arguments_shared_c") :: shared_c +contains + function is_shared(b) result(r) + type(box), target, intent(in) :: b + logical :: r + r = c_associated(c_loc(b), c_loc(shared)) + end function is_shared + function is_shared_c(b) result(r) + type(box), target, intent(in) :: b + logical :: r + r = c_associated(c_loc(b), c_loc(shared_c)) + end function is_shared_c +end module module_variable_arguments diff --git a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py index 2896a2dc2..d36ce9848 100644 --- a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py +++ b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py @@ -731,3 +731,26 @@ def test_a_reexport_binds_one_callable_from_source_and_from_its_contract(tmp_pat # One wrapper defines the procedure on either route. generated = (result.output_dir / "reexport_contract_wrapper.c").read_text(encoding="utf-8") assert generated.count("static PyObject * wrap_scale_value") == 1 + + +def test_a_derived_module_variable_argument_is_the_variable_itself(pyi_parity_build_mode: str, tmp_path: Path): + """A procedure given a module variable receives that variable's storage, not a copy. + + Libraries recognize predefined objects by address -- Open MPI's + ``MPI_STATUS_IGNORE`` is one -- so passing one must pass the object itself. + """ + module = _build_source_or_generated_pyi_and_import( + NATIVE_FIXTURES / "module_variable_arguments.f90", + tmp_path, + { + "bind_c_module_variable_arguments_wrapper.f90", + "module_variable_arguments_wrapper.c", + "module_variable_arguments_wrapper.h", + }, + None, + pyi_parity_build_mode, + ) + + assert module.is_shared(module.shared) + assert module.is_shared_c(module.shared_c) + assert not module.is_shared(module.Box()) From d5f9f33390096c047b5d94b26fef0ad6eadc4567 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 11:38:25 +0100 Subject: [PATCH 27/47] Pass wrapped derived-type objects without per-call name strings or environment scans A generated binding read each wrapped object's native storage with PyObject_GetAttrString, which builds and hashes a new name string on every call, and scanned the environment for PRIK's own test failure hooks after every call taking a derived-type argument. It now reads those attributes through names interned once, and compiles the failure hooks only when built with -DPRIK_WRAPPER_FAULT_INJECTION, which the tests that inject failures pass and a default build does not. A call passing a wrapped handle costs about half as much: an 8-byte Open MPI round trip through the generated mpi_f08 extension drops from 2.1 to 1.1 us. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 7 +++ prik/codegen/c/binding.py | 24 ++++----- prik/runtime/native_support/prik_binding.h | 49 +++++++++++++++++++ tests/fortran/_support/wrapper_build.py | 21 ++++++-- .../arrays/end_to_end/test_array_results.py | 2 + .../test_scalar_actual_dummy_matrix.py | 3 +- .../end_to_end/test_edited_call_surfaces.py | 18 ++++++- .../end_to_end/test_character_boundaries.py | 2 + .../end_to_end/test_character_edge_cases.py | 3 ++ .../test_scalar_string_descriptors.py | 3 +- 10 files changed, 114 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c09b003d0..952d5f0b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,13 @@ release tags add a leading `v` to the package version. ## Unreleased +- Generated bindings pass a wrapped derived-type object about twice as fast: + they read its native storage through attribute names interned once instead + of building a new name string on every call, and no longer scan the + environment for PRIK's own test failure hooks, which a binding now compiles + only when built with `-DPRIK_WRAPPER_FAULT_INJECTION`. An Open MPI 8-byte + round trip through the generated `mpi_f08` extension drops from 2.1 to + 1.1 µs. - Fortran source commands accept `--module-source-dir DIR`, and `build_fortran_extension` accepts `module_source_dirs`: from the given entry sources, PRIK follows each `use` to the source under those directories that diff --git a/prik/codegen/c/binding.py b/prik/codegen/c/binding.py index 2c8850cf6..f78fb3cc4 100644 --- a/prik/codegen/c/binding.py +++ b/prik/codegen/c/binding.py @@ -1418,7 +1418,7 @@ def _callback_derived_result_nodes( CDeclaration( "callback_capsule", "PyObject *", - CodeExpression('PyObject_GetAttrString(callback_result, "_prik_capsule")'), + CodeExpression('prik_getattr_interned(callback_result, &prik_name_prik_capsule, "_prik_capsule")'), ), self._callback_abort_if_null( callback, @@ -1622,7 +1622,7 @@ def _derived_argument_extractor_function(self) -> CFunction: CDeclaration( "origin_object", "PyObject *", - CodeExpression('PyObject_GetAttrString(object, "_prik_origin")'), + CodeExpression('prik_getattr_interned(object, &prik_name_prik_origin, "_prik_origin")'), ), CIf( CodeExpression("origin_object == NULL"), @@ -1756,7 +1756,7 @@ def _derived_argument_ops_extraction_nodes(self) -> tuple: CDeclaration( "operation_map", "PyObject *", - CodeExpression('PyObject_GetAttrString(object, "_prik_ops")'), + CodeExpression('prik_getattr_interned(object, &prik_name_prik_ops, "_prik_ops")'), ), CIf( CodeExpression("operation_map == NULL"), @@ -1768,7 +1768,7 @@ def _derived_argument_ops_extraction_nodes(self) -> tuple: CDeclaration( "ops_capsule", "PyObject *", - CodeExpression('PyDict_GetItemString(operation_map, "_native_ops")'), + CodeExpression('prik_dict_getitem_interned(operation_map, &prik_name_native_ops, "_native_ops")'), ), CIf( CodeExpression("ops_capsule == NULL"), @@ -1830,7 +1830,7 @@ def _derived_argument_capsule_extraction_nodes(self) -> tuple: CDeclaration( "carrier_capsule", "PyObject *", - CodeExpression('PyObject_GetAttrString(object, "_prik_capsule")'), + CodeExpression('prik_getattr_interned(object, &prik_name_prik_capsule, "_prik_capsule")'), ), CIf( CodeExpression("carrier_capsule == NULL"), @@ -2088,7 +2088,7 @@ def _derived_origin_fault_declaration(name: str) -> CDeclaration: return CDeclaration( name, "const char *", - CodeExpression('getenv("PRIK_WRAPPER_FAIL_DERIVED_ORIGIN")'), + CodeExpression('prik_wrapper_fault_selector("PRIK_WRAPPER_FAIL_DERIVED_ORIGIN")'), ) def _derived_origin_fault_return( @@ -2745,7 +2745,7 @@ def _allocatable_holder_owner_nodes(self, type_name: str, *, setter: bool) -> tu CDeclaration( "owner_capsule", "PyObject *", - CodeExpression('PyObject_GetAttrString(owner_obj, "_prik_capsule")'), + CodeExpression('prik_getattr_interned(owner_obj, &prik_name_prik_capsule, "_prik_capsule")'), ), CIf(CodeExpression("owner_capsule == NULL"), body=(CReturn(CodeExpression("NULL")),)), CDeclaration( @@ -2835,7 +2835,7 @@ def _pointer_holder_owner_nodes(self, type_name: str, *, setter: bool) -> tuple: CDeclaration( "owner_capsule", "PyObject *", - CodeExpression('PyObject_GetAttrString(owner_obj, "_prik_capsule")'), + CodeExpression('prik_getattr_interned(owner_obj, &prik_name_prik_capsule, "_prik_capsule")'), ), CIf(CodeExpression("owner_capsule == NULL"), body=(CReturn(CodeExpression("NULL")),)), CDeclaration( @@ -3766,7 +3766,9 @@ def _derived_address_from_object_nodes(self, type_symbol: str, object_name: str, address = f"{prefix}_address" return ( CDeclaration( - capsule, "PyObject *", CodeExpression(f'PyObject_GetAttrString({object_name}, "_prik_capsule")') + capsule, + "PyObject *", + CodeExpression(f'prik_getattr_interned({object_name}, &prik_name_prik_capsule, "_prik_capsule")'), ), CIf(CodeExpression(f"{capsule} == NULL"), body=(CReturn(CodeExpression("NULL")),)), CIf( @@ -4277,7 +4279,7 @@ def _module_allocator_functions(self, required: bool) -> tuple[CFunction, ...]: CDeclaration( "fail_alloc", "const char *", - CodeExpression('getenv("PRIK_WRAPPER_FAIL_ALLOC")'), + CodeExpression('prik_wrapper_fault_selector("PRIK_WRAPPER_FAIL_ALLOC")'), ), CIf( CodeExpression("fail_alloc != NULL && fail_alloc[0] != '\\0' && fail_alloc[0] != '0'"), @@ -13252,7 +13254,7 @@ def _derived_after_native_failure_nodes( CDeclaration( fault, "const char *", - CodeExpression('getenv("PRIK_WRAPPER_FAIL_DERIVED_AFTER_NATIVE")'), + CodeExpression('prik_wrapper_fault_selector("PRIK_WRAPPER_FAIL_DERIVED_AFTER_NATIVE")'), ), CIf( CodeExpression(f"{fault} != NULL && {fault}[0] != '\\0' && {fault}[0] != '0'"), diff --git a/prik/runtime/native_support/prik_binding.h b/prik/runtime/native_support/prik_binding.h index 8371beae7..e325bb5e7 100644 --- a/prik/runtime/native_support/prik_binding.h +++ b/prik/runtime/native_support/prik_binding.h @@ -198,6 +198,55 @@ static inline const char *prik_derived_type_info_capsule_name(void) #define PRIK_MAYBE_UNUSED #endif +/* The fixed names a generated binding reads from its wrapper objects on every + * call. Each is interned once into its slot here, because building and hashing + * a new string per lookup, as PyObject_GetAttrString does, costs more than the + * rest of passing a wrapped object. */ +static PyObject *prik_name_prik_origin PRIK_MAYBE_UNUSED = NULL; +static PyObject *prik_name_prik_ops PRIK_MAYBE_UNUSED = NULL; +static PyObject *prik_name_prik_capsule PRIK_MAYBE_UNUSED = NULL; +static PyObject *prik_name_native_ops PRIK_MAYBE_UNUSED = NULL; + +static inline PyObject *prik_interned_name(PyObject **slot, const char *text) +{ + if (*slot == NULL) + *slot = PyUnicode_InternFromString(text); + return *slot; +} + +/* PyObject_GetAttrString through an interned name. */ +static inline PyObject *prik_getattr_interned(PyObject *object, PyObject **slot, const char *text) +{ + PyObject *name = prik_interned_name(slot, text); + return name == NULL ? NULL : PyObject_GetAttr(object, name); +} + +/* PyDict_GetItemString through an interned name: a borrowed item, or NULL + * without an exception when it is absent. */ +static inline PyObject *prik_dict_getitem_interned(PyObject *dict, PyObject **slot, const char *text) +{ + PyObject *name = prik_interned_name(slot, text); + if (name == NULL) { + PyErr_Clear(); + return NULL; + } + return PyDict_GetItem(dict, name); +} + +/* PRIK's own wrapper tests make a generated binding fail on purpose by naming + * the failure in the environment. Only a binding compiled with + * PRIK_WRAPPER_FAULT_INJECTION reads it; any other answers "no failure" without + * scanning the environment on every call. */ +static inline const char *prik_wrapper_fault_selector(const char *variable) +{ +#ifdef PRIK_WRAPPER_FAULT_INJECTION + return getenv(variable); +#else + (void)variable; + return NULL; +#endif +} + #ifdef PRIK_BINDING_CAPTURE_ADDRESS /* * Report the address a caller already passed by reference. diff --git a/tests/fortran/_support/wrapper_build.py b/tests/fortran/_support/wrapper_build.py index 1ff255fe2..f8dd61425 100644 --- a/tests/fortran/_support/wrapper_build.py +++ b/tests/fortran/_support/wrapper_build.py @@ -38,6 +38,10 @@ from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner +#: Compiles a wrapper's test-only failure hooks in, so a test can make it fail +#: by naming a ``PRIK_WRAPPER_FAIL_*`` failure in the environment. +FAULT_INJECTION_C_FLAGS = ("-DPRIK_WRAPPER_FAULT_INJECTION",) + WRAPPER_TEST_ROOT = Path(__file__).resolve().parent WRAPPER_SOURCE_PATHS = { "c_order_flat_buffer.f90": REPO_ROOT / "tests/fortran/functions/end_to_end/fixtures/native/c_order_flat_buffer.f90", @@ -115,7 +119,12 @@ def _run_captured_command( return result -def _build_and_import(source_template: Path, workdir: Path, expected_generated_sources: set[str]): +def _build_and_import( + source_template: Path, + workdir: Path, + expected_generated_sources: set[str], + wrapper_c_flags: tuple[str, ...] = (), +): source = workdir / source_template.name module_name = source_template.stem shutil.copyfile(source_template, source) @@ -129,6 +138,7 @@ def _build_and_import(source_template: Path, workdir: Path, expected_generated_s str(workdir), "--compiler", _compiler(), + *(f"--wrapper-c-flags={flag}" for flag in wrapper_c_flags), "--json", ] result = _run_captured_command(cmd, cwd=workdir) @@ -267,6 +277,7 @@ def _build_generated_pyi_and_import( source_template: Path, workdir: Path, expected_contract_package: Path | None = None, + wrapper_c_flags: tuple[str, ...] = (), ): """Generate a contract from source, then build and import through that contract.""" source_dir = workdir / "source" @@ -282,6 +293,7 @@ def _build_generated_pyi_and_import( native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=workdir / "pyi_build", + wrapper_c_flags=wrapper_c_flags, ) assert result.sources[0] == entry @@ -298,12 +310,15 @@ def _build_source_or_generated_pyi_and_import( expected_generated_sources: set[str], expected_contract_package: Path, build_mode: str, + wrapper_c_flags: tuple[str, ...] = (), ): if build_mode == "source": source_build_dir = workdir / "source_build" source_build_dir.mkdir(parents=True) - return _build_and_import(source_template, source_build_dir, expected_generated_sources) - return _build_generated_pyi_and_import(source_template, workdir / "generated_pyi_build", expected_contract_package) + return _build_and_import(source_template, source_build_dir, expected_generated_sources, wrapper_c_flags) + return _build_generated_pyi_and_import( + source_template, workdir / "generated_pyi_build", expected_contract_package, wrapper_c_flags + ) def _build_source_and_import( diff --git a/tests/fortran/arrays/end_to_end/test_array_results.py b/tests/fortran/arrays/end_to_end/test_array_results.py index 82f355bf3..c0c4718f7 100644 --- a/tests/fortran/arrays/end_to_end/test_array_results.py +++ b/tests/fortran/arrays/end_to_end/test_array_results.py @@ -7,6 +7,7 @@ import pytest from tests.fortran._support.wrapper_build import ( + FAULT_INJECTION_C_FLAGS, _build_source_or_generated_pyi_and_import, ) @@ -33,6 +34,7 @@ def test_array_results_follow_data_buffer_and_descriptor_handle_contracts( }, CONTRACT_FIXTURES / "farray_results_f90", pyi_parity_build_mode, + wrapper_c_flags=FAULT_INJECTION_C_FLAGS, ) fixed = module.fixed_vector() diff --git a/tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py b/tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py index bf341410a..605c2c478 100644 --- a/tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py +++ b/tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py @@ -14,7 +14,7 @@ import pytest from tests.fortran._support.paths import REPO_ROOT -from tests.fortran._support.wrapper_build import _import_from_build_dir +from tests.fortran._support.wrapper_build import FAULT_INJECTION_C_FLAGS, _import_from_build_dir from prik import build_pyi_extension FIXTURES = Path(__file__).parent / "fixtures" @@ -84,6 +84,7 @@ def scalar_matrix(tmp_path_factory) -> MatrixBuild: CONTRACT, native_fortran_sources=[SOURCE], output_dir=output_dir, + wrapper_c_flags=FAULT_INJECTION_C_FLAGS, ) package = _import_from_build_dir(result.module_name, result.output_dir) return MatrixBuild( diff --git a/tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_edited_call_surfaces.py b/tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_edited_call_surfaces.py index 6b2f01456..858a03853 100644 --- a/tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_edited_call_surfaces.py +++ b/tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_edited_call_surfaces.py @@ -6,6 +6,7 @@ import pytest from tests.fortran._support.wrapper_build import ( + FAULT_INJECTION_C_FLAGS, _compile_native_object, _import_from_build_dir, _sole_native_module, @@ -42,12 +43,13 @@ def character_literal_native_object(tmp_path_factory) -> Path: ) -def _build(case: str, native_object: Path, output_dir: Path): +def _build(case: str, native_object: Path, output_dir: Path, wrapper_c_flags: tuple[str, ...] = ()): result = build_pyi_extension( CONTRACTS / case / "__init__.pyi", native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=output_dir, + wrapper_c_flags=wrapper_c_flags, ) return _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) @@ -224,7 +226,7 @@ def test_hidden_fixed_shape_array_output_is_allocated_and_returned( tmp_path: Path, monkeypatch, ): - module = _build("hidden_array_output", output_native_object, tmp_path / "build") + module = _build("hidden_array_output", output_native_object, tmp_path / "build", FAULT_INJECTION_C_FLAGS) np.testing.assert_array_equal(module.fill_vector(np.int32(4)), np.array([2.0, 4.0, 6.0, 8.0])) assert module.fill_vector(np.int32(0)).shape == (0,) @@ -232,3 +234,15 @@ def test_hidden_fixed_shape_array_output_is_allocated_and_returned( monkeypatch.setenv("PRIK_WRAPPER_FAIL_ALLOC", "1") with pytest.raises(MemoryError, match="Unable to allocate copy-return output array"): module.fill_vector(np.int32(2)) + + +def test_a_wrapper_built_without_fault_injection_ignores_failure_requests( + output_native_object: Path, + tmp_path: Path, + monkeypatch, +): + """Only a build that compiles the failure hooks in reads them from the environment.""" + module = _build("hidden_array_output", output_native_object, tmp_path / "build") + + monkeypatch.setenv("PRIK_WRAPPER_FAIL_ALLOC", "1") + np.testing.assert_array_equal(module.fill_vector(np.int32(2)), np.array([2.0, 4.0])) diff --git a/tests/fortran/strings/end_to_end/test_character_boundaries.py b/tests/fortran/strings/end_to_end/test_character_boundaries.py index c9bfdc928..0df9bf596 100644 --- a/tests/fortran/strings/end_to_end/test_character_boundaries.py +++ b/tests/fortran/strings/end_to_end/test_character_boundaries.py @@ -6,6 +6,7 @@ import pytest from tests.fortran._support.wrapper_build import ( + FAULT_INJECTION_C_FLAGS, _build_source_or_generated_pyi_and_import, _compile_native_object, _import_from_build_dir, @@ -66,6 +67,7 @@ def test_modern_fortran_character_arguments_and_results( }, CONTRACT_FIXTURES / "fstrings_f90", pyi_parity_build_mode, + wrapper_c_flags=FAULT_INJECTION_C_FLAGS, ) _assert_modern_string_examples(module) diff --git a/tests/fortran/strings/end_to_end/test_character_edge_cases.py b/tests/fortran/strings/end_to_end/test_character_edge_cases.py index fdc1a43d4..5eab18456 100644 --- a/tests/fortran/strings/end_to_end/test_character_edge_cases.py +++ b/tests/fortran/strings/end_to_end/test_character_edge_cases.py @@ -5,6 +5,7 @@ import pytest from tests.fortran._support.wrapper_build import ( + FAULT_INJECTION_C_FLAGS, _build_source_or_generated_pyi_and_import, _compile_native_object, _import_from_build_dir, @@ -34,6 +35,7 @@ def compiled_character_edges_module( }, CONTRACT_FIXTURES / "fcharacter_edges_f90", pyi_parity_build_mode, + wrapper_c_flags=FAULT_INJECTION_C_FLAGS, ) @@ -103,6 +105,7 @@ def fixed_discard(name: String[8]) -> None: ... native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=tmp_path / "build", + wrapper_c_flags=FAULT_INJECTION_C_FLAGS, ) package = _import_from_build_dir(result.module_name, result.output_dir) module = package if hasattr(package, "fixed_replacement") else _sole_native_module(package) diff --git a/tests/fortran/strings/end_to_end/test_scalar_string_descriptors.py b/tests/fortran/strings/end_to_end/test_scalar_string_descriptors.py index 0ea216227..127003669 100644 --- a/tests/fortran/strings/end_to_end/test_scalar_string_descriptors.py +++ b/tests/fortran/strings/end_to_end/test_scalar_string_descriptors.py @@ -5,7 +5,7 @@ import numpy as np import pytest -from tests.fortran._support.wrapper_build import _build_source_or_generated_pyi_and_import +from tests.fortran._support.wrapper_build import FAULT_INJECTION_C_FLAGS, _build_source_or_generated_pyi_and_import FIXTURES = Path(__file__).parent / "fixtures" DESCRIPTOR_SOURCE = FIXTURES / "native" / "fstring_descriptors_f90.f90" @@ -32,6 +32,7 @@ def compiled_descriptor_module(pyi_parity_build_mode: str, tmp_path: Path): }, CONTRACT_FIXTURES / "fstring_descriptors_f90", pyi_parity_build_mode, + wrapper_c_flags=FAULT_INJECTION_C_FLAGS, ) From 11903daf0ca297a201b336368691a4f92b06ac23 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 11:51:34 +0100 Subject: [PATCH 28/47] Keep ranks and tags np.int32 from the start in the mpi_f08 tutorial's Python layer prik_mpi.py converted every rank and tag with np.int32(...) on every call, which cost more than a small MPI call, and checked each buffer against a list | tuple union before the common array case. Ranks and tags are now np.int32 throughout -- Get_rank, the Status fields, ANY_SOURCE, and the defaults keep the extension's np.int32 values, and the program writes its tags and root as np.int32 constants -- so they pass straight to the contract, and a plain array is recognized first. A plain Python int is refused with PRIK's TypeError, which the tutorial now lists as a limit. An 8-byte round trip through the layer drops from 4.1 to 2.3 us; the same program still runs under mpi4py with the same output. Co-Authored-By: Claude Opus 5.5 --- docs/user/tutorials/openmpi-f08.md | 77 +++++++++++-------- .../fixtures/runtime/mpi_example.py | 18 +++-- .../end_to_end/fixtures/runtime/prik_mpi.py | 46 ++++++----- 3 files changed, 83 insertions(+), 58 deletions(-) diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index c5fca3d53..68c57dc4f 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -25,33 +25,39 @@ comm = MPI.COMM_WORLD rank = comm.Get_rank() size = comm.Get_size() +# Ranks and tags are np.int32 from the start: Get_rank returns one, these +# constants are, and rank + 1 stays one. +ROOT = np.int32(0) +OBJECT_TAG = np.int32(11) +ARRAY_TAG = np.int32(77) + # Python objects travel pickled. if rank == 0: - comm.send({"a": 7, "b": 3.14}, dest=1, tag=11) + comm.send({"a": 7, "b": 3.14}, dest=rank + 1, tag=OBJECT_TAG) elif rank == 1: status = MPI.Status() - data = comm.recv(source=MPI.ANY_SOURCE, tag=11, status=status) + data = comm.recv(source=MPI.ANY_SOURCE, tag=OBJECT_TAG, status=status) print(f"rank 1 received {data} from rank {status.Get_source()}") # NumPy arrays travel as buffers, with an explicit MPI datatype ... if rank == 0: data = np.arange(4, dtype="i") - comm.Send([data, MPI.INT], dest=1, tag=77) + comm.Send([data, MPI.INT], dest=rank + 1, tag=ARRAY_TAG) elif rank == 1: data = np.empty(4, dtype="i") - comm.Recv([data, MPI.INT], source=0, tag=77) + comm.Recv([data, MPI.INT], source=rank - 1, tag=ARRAY_TAG) print(f"rank 1 received {data.tolist()}") # ... or with the datatype taken from the array. data = np.arange(3, dtype=np.float64) if rank == 0 else np.empty(3, dtype=np.float64) -comm.Bcast(data, root=0) +comm.Bcast(data, root=ROOT) # Collectives: every rank contributes. values = np.array([rank + 1, rank + 2], dtype="i") total = np.empty_like(values) comm.Allreduce(values, total, op=MPI.SUM) largest = np.empty_like(values) -comm.Reduce(values, largest, op=MPI.MAX, root=0) +comm.Reduce(values, largest, op=MPI.MAX, root=ROOT) comm.Allreduce(MPI.IN_PLACE, values, op=MPI.SUM) comm.Barrier() @@ -549,8 +555,13 @@ import numpy as np from prik_openmpi_f08 import mpi_f08 as _mpi -ANY_SOURCE = int(_mpi.mpi_any_source) -ANY_TAG = int(_mpi.mpi_any_tag) +# Ranks, tags, and counts are np.int32, the type the contract takes, from the +# start: the extension returns them as np.int32, the constants and defaults +# here are np.int32, and arithmetic with Python integers keeps the type. So +# they pass straight to the contract, never converted. +ANY_SOURCE = _mpi.mpi_any_source +ANY_TAG = _mpi.mpi_any_tag +_ZERO = np.int32(0) IN_PLACE = _mpi.mpi_in_place STATUS_IGNORE = _mpi.mpi_status_ignore BYTE = _mpi.mpi_byte @@ -562,13 +573,12 @@ MAX = _mpi.mpi_max # The MPI datatype of each NumPy element type, for buffers given without one. _DATATYPES = {np.dtype(np.uint8): BYTE, np.dtype(np.int32): INT, np.dtype(np.float64): DOUBLE} - def _message(buf): """Return a buffer's array and MPI datatype; ``buf`` is an array or ``[array, datatype]``.""" - if isinstance(buf, list | tuple): - array, datatype = buf - return array, datatype - return buf, _DATATYPES[buf.dtype] + if isinstance(buf, np.ndarray): + return buf, _DATATYPES[buf.dtype] + array, datatype = buf + return array, datatype class Status: @@ -579,11 +589,11 @@ class Status: @property def source(self): - return int(self._native.mpi_source) + return self._native.mpi_source @property def tag(self): - return int(self._native.mpi_tag) + return self._native.mpi_tag def Get_source(self): return self.source @@ -592,7 +602,7 @@ class Status: return self.tag def Get_count(self, datatype=BYTE): - return int(_mpi.get_count(self._native, datatype)) + return _mpi.get_count(self._native, datatype) def _native_status(status): @@ -607,10 +617,10 @@ class Comm: self.handle = handle def Get_rank(self): - return int(_mpi.comm_rank(self.handle)) + return _mpi.comm_rank(self.handle) def Get_size(self): - return int(_mpi.comm_size(self.handle)) + return _mpi.comm_size(self.handle) rank = property(Get_rank) size = property(Get_size) @@ -618,32 +628,32 @@ class Comm: def Barrier(self): _mpi.barrier(self.handle) - def Send(self, buf, dest, tag=0): + def Send(self, buf, dest, tag=_ZERO): array, datatype = _message(buf) - _mpi.send(array, datatype, np.int32(dest), np.int32(tag), self.handle) + _mpi.send(array, datatype, dest, tag, self.handle) def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG, status=None): array, datatype = _message(buf) - _mpi.recv(array, datatype, np.int32(source), np.int32(tag), self.handle, _native_status(status)) + _mpi.recv(array, datatype, source, tag, self.handle, _native_status(status)) def Probe(self, source=ANY_SOURCE, tag=ANY_TAG, status=None): - _mpi.probe(np.int32(source), np.int32(tag), self.handle, _native_status(status)) + _mpi.probe(source, tag, self.handle, _native_status(status)) return True - def Bcast(self, buf, root=0): + def Bcast(self, buf, root=_ZERO): array, datatype = _message(buf) - _mpi.bcast(array, datatype, np.int32(root), self.handle) + _mpi.bcast(array, datatype, root, self.handle) - def Reduce(self, sendbuf, recvbuf, op=SUM, root=0): + def Reduce(self, sendbuf, recvbuf, op=SUM, root=_ZERO): array, datatype = _message(recvbuf) - _mpi.reduce(sendbuf, array, datatype, op, np.int32(root), self.handle) + _mpi.reduce(sendbuf, array, datatype, op, root, self.handle) def Allreduce(self, sendbuf, recvbuf, op=SUM): array, datatype = _message(recvbuf) _mpi.allreduce(sendbuf, array, datatype, op, self.handle) # Python objects travel pickled, as with mpi4py's lowercase methods. - def send(self, obj, dest, tag=0): + def send(self, obj, dest, tag=_ZERO): self.Send(np.frombuffer(pickle.dumps(obj), dtype=np.uint8), dest, tag) def recv(self, buf=None, source=ANY_SOURCE, tag=ANY_TAG, status=None): @@ -671,9 +681,12 @@ Everything in it calls the generated functions of step 5: - **Buffers.** A buffer is a NumPy array, whose MPI datatype is chosen from its element type, or an `[array, datatype]` pair naming the datatype explicitly, as in mpi4py. -- **Defaults and Python integers.** `tag=0`, `source=ANY_SOURCE`, `root=0`, - and `op=SUM` are keyword defaults, and plain Python integers are converted - to the `np.int32` values the contract's `Int32` arguments take. +- **Defaults and `np.int32` values.** `tag`, `source=ANY_SOURCE`, `root`, + and `op=SUM` are keyword defaults. Ranks and tags are the `np.int32` values + the contract's `Int32` arguments take from the start -- `Get_rank` returns + one, `ANY_SOURCE` and the defaults are, and `rank + 1` stays one -- so they + pass straight through. Converting a plain integer on every call would cost + more than a small MPI call. - **Python objects.** Lowercase `send` pickles an object into a byte array and sends it; `recv` probes the incoming message, sizes a byte array with `Get_count`, receives it, and unpickles it. @@ -776,7 +789,9 @@ way when you select it, within these limits of what PRIK supports today: a faithful imitation would need arrays of requests, the first limitation. `prik_mpi.py` imitates the part of mpi4py this program uses, not all of it. -It passes contiguous NumPy arrays only -- a strided view is refused with a +Ranks and tags must be `np.int32`, as in the program above; a plain Python +`int` is refused with a `TypeError`, where mpi4py accepts one. It passes +contiguous NumPy arrays only -- a strided view is refused with a `TypeError` -- and picks a datatype for `int32`, `float64`, and `uint8` arrays. It has none of mpi4py's other communicators, lowercase collectives, or `MPI.Exception`: under Open MPI's default error handler an MPI error aborts diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py index 6ec773072..109e31baa 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py @@ -6,33 +6,39 @@ rank = comm.Get_rank() size = comm.Get_size() +# Ranks and tags are np.int32 from the start: Get_rank returns one, these +# constants are, and rank + 1 stays one. +ROOT = np.int32(0) +OBJECT_TAG = np.int32(11) +ARRAY_TAG = np.int32(77) + # Python objects travel pickled. if rank == 0: - comm.send({"a": 7, "b": 3.14}, dest=1, tag=11) + comm.send({"a": 7, "b": 3.14}, dest=rank + 1, tag=OBJECT_TAG) elif rank == 1: status = MPI.Status() - data = comm.recv(source=MPI.ANY_SOURCE, tag=11, status=status) + data = comm.recv(source=MPI.ANY_SOURCE, tag=OBJECT_TAG, status=status) print(f"rank 1 received {data} from rank {status.Get_source()}") # NumPy arrays travel as buffers, with an explicit MPI datatype ... if rank == 0: data = np.arange(4, dtype="i") - comm.Send([data, MPI.INT], dest=1, tag=77) + comm.Send([data, MPI.INT], dest=rank + 1, tag=ARRAY_TAG) elif rank == 1: data = np.empty(4, dtype="i") - comm.Recv([data, MPI.INT], source=0, tag=77) + comm.Recv([data, MPI.INT], source=rank - 1, tag=ARRAY_TAG) print(f"rank 1 received {data.tolist()}") # ... or with the datatype taken from the array. data = np.arange(3, dtype=np.float64) if rank == 0 else np.empty(3, dtype=np.float64) -comm.Bcast(data, root=0) +comm.Bcast(data, root=ROOT) # Collectives: every rank contributes. values = np.array([rank + 1, rank + 2], dtype="i") total = np.empty_like(values) comm.Allreduce(values, total, op=MPI.SUM) largest = np.empty_like(values) -comm.Reduce(values, largest, op=MPI.MAX, root=0) +comm.Reduce(values, largest, op=MPI.MAX, root=ROOT) comm.Allreduce(MPI.IN_PLACE, values, op=MPI.SUM) comm.Barrier() diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py index 68818706f..512149570 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py @@ -7,8 +7,13 @@ from prik_openmpi_f08 import mpi_f08 as _mpi -ANY_SOURCE = int(_mpi.mpi_any_source) -ANY_TAG = int(_mpi.mpi_any_tag) +# Ranks, tags, and counts are np.int32, the type the contract takes, from the +# start: the extension returns them as np.int32, the constants and defaults +# here are np.int32, and arithmetic with Python integers keeps the type. So +# they pass straight to the contract, never converted. +ANY_SOURCE = _mpi.mpi_any_source +ANY_TAG = _mpi.mpi_any_tag +_ZERO = np.int32(0) IN_PLACE = _mpi.mpi_in_place STATUS_IGNORE = _mpi.mpi_status_ignore BYTE = _mpi.mpi_byte @@ -20,13 +25,12 @@ # The MPI datatype of each NumPy element type, for buffers given without one. _DATATYPES = {np.dtype(np.uint8): BYTE, np.dtype(np.int32): INT, np.dtype(np.float64): DOUBLE} - def _message(buf): """Return a buffer's array and MPI datatype; ``buf`` is an array or ``[array, datatype]``.""" - if isinstance(buf, list | tuple): - array, datatype = buf - return array, datatype - return buf, _DATATYPES[buf.dtype] + if isinstance(buf, np.ndarray): + return buf, _DATATYPES[buf.dtype] + array, datatype = buf + return array, datatype class Status: @@ -37,11 +41,11 @@ def __init__(self): @property def source(self): - return int(self._native.mpi_source) + return self._native.mpi_source @property def tag(self): - return int(self._native.mpi_tag) + return self._native.mpi_tag def Get_source(self): return self.source @@ -50,7 +54,7 @@ def Get_tag(self): return self.tag def Get_count(self, datatype=BYTE): - return int(_mpi.get_count(self._native, datatype)) + return _mpi.get_count(self._native, datatype) def _native_status(status): @@ -65,10 +69,10 @@ def __init__(self, handle): self.handle = handle def Get_rank(self): - return int(_mpi.comm_rank(self.handle)) + return _mpi.comm_rank(self.handle) def Get_size(self): - return int(_mpi.comm_size(self.handle)) + return _mpi.comm_size(self.handle) rank = property(Get_rank) size = property(Get_size) @@ -76,32 +80,32 @@ def Get_size(self): def Barrier(self): _mpi.barrier(self.handle) - def Send(self, buf, dest, tag=0): + def Send(self, buf, dest, tag=_ZERO): array, datatype = _message(buf) - _mpi.send(array, datatype, np.int32(dest), np.int32(tag), self.handle) + _mpi.send(array, datatype, dest, tag, self.handle) def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG, status=None): array, datatype = _message(buf) - _mpi.recv(array, datatype, np.int32(source), np.int32(tag), self.handle, _native_status(status)) + _mpi.recv(array, datatype, source, tag, self.handle, _native_status(status)) def Probe(self, source=ANY_SOURCE, tag=ANY_TAG, status=None): - _mpi.probe(np.int32(source), np.int32(tag), self.handle, _native_status(status)) + _mpi.probe(source, tag, self.handle, _native_status(status)) return True - def Bcast(self, buf, root=0): + def Bcast(self, buf, root=_ZERO): array, datatype = _message(buf) - _mpi.bcast(array, datatype, np.int32(root), self.handle) + _mpi.bcast(array, datatype, root, self.handle) - def Reduce(self, sendbuf, recvbuf, op=SUM, root=0): + def Reduce(self, sendbuf, recvbuf, op=SUM, root=_ZERO): array, datatype = _message(recvbuf) - _mpi.reduce(sendbuf, array, datatype, op, np.int32(root), self.handle) + _mpi.reduce(sendbuf, array, datatype, op, root, self.handle) def Allreduce(self, sendbuf, recvbuf, op=SUM): array, datatype = _message(recvbuf) _mpi.allreduce(sendbuf, array, datatype, op, self.handle) # Python objects travel pickled, as with mpi4py's lowercase methods. - def send(self, obj, dest, tag=0): + def send(self, obj, dest, tag=_ZERO): self.Send(np.frombuffer(pickle.dumps(obj), dtype=np.uint8), dest, tag) def recv(self, buf=None, source=ANY_SOURCE, tag=ANY_TAG, status=None): From 363c1e8d83e22d3040b53ef89623bf2449fa91fe Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 11:55:55 +0100 Subject: [PATCH 29/47] Separate the tutorial's prik_mpi helpers by two blank lines Co-Authored-By: Claude Opus 5.5 --- docs/user/tutorials/openmpi-f08.md | 1 + .../assumed_types/end_to_end/fixtures/runtime/prik_mpi.py | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 68c57dc4f..a7cff572a 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -573,6 +573,7 @@ MAX = _mpi.mpi_max # The MPI datatype of each NumPy element type, for buffers given without one. _DATATYPES = {np.dtype(np.uint8): BYTE, np.dtype(np.int32): INT, np.dtype(np.float64): DOUBLE} + def _message(buf): """Return a buffer's array and MPI datatype; ``buf`` is an array or ``[array, datatype]``.""" if isinstance(buf, np.ndarray): diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py index 512149570..8e411dab7 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py @@ -25,6 +25,7 @@ # The MPI datatype of each NumPy element type, for buffers given without one. _DATATYPES = {np.dtype(np.uint8): BYTE, np.dtype(np.int32): INT, np.dtype(np.float64): DOUBLE} + def _message(buf): """Return a buffer's array and MPI datatype; ``buf`` is an array or ``[array, datatype]``.""" if isinstance(buf, np.ndarray): From 6c7c853b364ee6c937044212cada7b2ff0564403 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 12:09:31 +0100 Subject: [PATCH 30/47] Simplify the mpi_f08 tutorial's Python layer to an illustration prik_mpi.py now shows the shape of mpi4py and no more: every buffer is an np.int32 array sent as MPI_INT, and Recv always passes MPI_STATUS_IGNORE, so the datatype table, the [array, datatype] form, the Status class, and the pickled send/recv are gone, and each method is one generated call. The edited contract drops probe and get_count, the export list drops MPI_Probe, MPI_Get_count, MPI_BYTE, and MPI_DOUBLE, and the example uses np.int32 buffers throughout. mpi4py still runs the program unchanged but for its import. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 10 +- docs/user/tutorials/openmpi-f08.md | 226 +++++------------- .../contracts/openmpi/mpi_exports.txt | 4 - .../fixtures/contracts/openmpi/mpi_f08.pyi | 16 -- .../fixtures/runtime/mpi_example.py | 33 +-- .../end_to_end/fixtures/runtime/prik_mpi.py | 96 ++------ .../end_to_end/test_openmpi_f08.py | 5 +- 7 files changed, 99 insertions(+), 291 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 952d5f0b2..9aed10de9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -136,11 +136,11 @@ release tags add a leading `v` to the package version. contract from the configured Open MPI sources, edits its facade to hide counts and error codes and return results, builds it with the CLI against the installed Open MPI without compiling any Open MPI source, adds a short - Python module spelling mpi4py's `COMM_WORLD`, `Send`/`Recv`, pickled - `send`/`recv`, `Bcast`, `Reduce`, and `Allreduce`, and runs a two-rank - program under `mpirun` that mpi4py also runs unchanged but for its import. - `Recv` without a status passes Open MPI's own `MPI_STATUS_IGNORE`, a - generated `Mpi_Status` module object. An opt-in integration test follows + illustrative Python module spelling mpi4py's `COMM_WORLD`, `Send`/`Recv`, + `Bcast`, `Reduce`, and `Allreduce` over `np.int32` buffers, and runs a + two-rank program under `mpirun` that mpi4py also runs unchanged but for its + import. `Recv` passes Open MPI's own `MPI_STATUS_IGNORE`, a generated + `Mpi_Status` module object. An opt-in integration test follows the tutorial's steps in one working directory and checks that Open MPI recognizes that object; the Open MPI Integration lane runs it against Open MPI 4.1.8 and 5.0.11 built from source. The test requires the configured tree to record the diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index a7cff572a..01e7b6df7 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -25,35 +25,26 @@ comm = MPI.COMM_WORLD rank = comm.Get_rank() size = comm.Get_size() -# Ranks and tags are np.int32 from the start: Get_rank returns one, these -# constants are, and rank + 1 stays one. +# Buffers are np.int32 arrays, and ranks and tags are np.int32 too: +# Get_rank returns one, and rank + 1 stays one. ROOT = np.int32(0) -OBJECT_TAG = np.int32(11) -ARRAY_TAG = np.int32(77) +TAG = np.int32(77) -# Python objects travel pickled. +# Point to point: rank 0 sends four integers to rank 1. if rank == 0: - comm.send({"a": 7, "b": 3.14}, dest=rank + 1, tag=OBJECT_TAG) + data = np.arange(4, dtype=np.int32) + comm.Send(data, dest=rank + 1, tag=TAG) elif rank == 1: - status = MPI.Status() - data = comm.recv(source=MPI.ANY_SOURCE, tag=OBJECT_TAG, status=status) - print(f"rank 1 received {data} from rank {status.Get_source()}") - -# NumPy arrays travel as buffers, with an explicit MPI datatype ... -if rank == 0: - data = np.arange(4, dtype="i") - comm.Send([data, MPI.INT], dest=rank + 1, tag=ARRAY_TAG) -elif rank == 1: - data = np.empty(4, dtype="i") - comm.Recv([data, MPI.INT], source=rank - 1, tag=ARRAY_TAG) + data = np.empty(4, dtype=np.int32) + comm.Recv(data, source=rank - 1, tag=TAG) print(f"rank 1 received {data.tolist()}") -# ... or with the datatype taken from the array. -data = np.arange(3, dtype=np.float64) if rank == 0 else np.empty(3, dtype=np.float64) +# Broadcast: rank 0's values reach every rank. +data = np.arange(3, dtype=np.int32) if rank == 0 else np.empty(3, dtype=np.int32) comm.Bcast(data, root=ROOT) -# Collectives: every rank contributes. -values = np.array([rank + 1, rank + 2], dtype="i") +# Reductions: every rank contributes. +values = np.array([rank + 1, rank + 2], dtype=np.int32) total = np.empty_like(values) comm.Allreduce(values, total, op=MPI.SUM) largest = np.empty_like(values) @@ -67,11 +58,10 @@ if rank == 0: ``` If you know mpi4py, you know this program: `COMM_WORLD`, `Get_rank`, -lowercase `send`/`recv` for Python objects, uppercase `Send`/`Recv`/`Bcast`/ -`Reduce`/`Allreduce` for buffers, `[data, MPI.INT]` buffer specifications, -`MPI.IN_PLACE`, and `Status` are all spelled as mpi4py spells them. Replace -`import prik_mpi as MPI` with `from mpi4py import MPI` and the same program -runs under mpi4py and prints the same lines. +`Send`/`Recv`, `Bcast`, `Reduce`, `Allreduce`, `MPI.SUM`, and `MPI.IN_PLACE` +are all spelled as mpi4py spells them. Replace `import prik_mpi as MPI` with +`from mpi4py import MPI` and the same program runs under mpi4py and prints +the same lines. Two layers make this work: @@ -82,9 +72,10 @@ Two layers make this work: contract: hiding counts that follow from the buffers, turning error codes into exceptions, returning results instead of filling output arguments. - **A short Python module, `prik_mpi.py`.** It gives the native API mpi4py's - object model: a `Comm` class with methods, keyword defaults, datatypes - chosen from NumPy arrays, and pickled Python objects. It is ordinary Python - over the generated functions, with no C and no `ctypes`. + shape: a `Comm` class with methods and keyword defaults. It is ordinary + Python over the generated functions, with no C and no `ctypes`, and it is + kept small on purpose: an illustration of the approach, not a complete MPI + binding. ## 1. See what PRIK reads @@ -155,15 +146,11 @@ mpi_f08::MPI_Comm_size mpi_f08::MPI_Barrier mpi_f08::MPI_Send mpi_f08::MPI_Recv -mpi_f08::MPI_Probe -mpi_f08::MPI_Get_count mpi_f08::MPI_Bcast mpi_f08::MPI_Reduce mpi_f08::MPI_Allreduce mpi_f08::MPI_COMM_WORLD -mpi_f08::MPI_BYTE mpi_f08::MPI_INT -mpi_f08::MPI_DOUBLE mpi_f08::MPI_SUM mpi_f08::MPI_MAX mpi_f08::MPI_IN_PLACE @@ -173,10 +160,8 @@ mpi_f08::MPI_ANY_TAG ``` These are the routines and objects behind the mpi4py names the program uses. -`MPI_Probe` and `MPI_Get_count` are not called by the program directly; they -let lowercase `recv` size its buffer before receiving a pickled object, as -mpi4py does. Nor is `MPI_STATUS_IGNORE`: like mpi4py, `Recv` passes it when it -is given no `Status`. +The program does not name `MPI_STATUS_IGNORE`; `Recv` passes it, since this +small API reports no status. Selecting symbols this way is not an MPI feature. `--export-symbols` accepts module-qualified public symbols from any Fortran project -- procedures, @@ -230,8 +215,8 @@ The `contract/` directory holds one editable `.pyi` file per Fortran module the selection needs. `contract/mpi_f08.pyi` publishes the selected names: ```python -from .mpi_f08_types import mpi_any_source, mpi_any_tag, mpi_byte, mpi_comm_world, mpi_double, mpi_in_place, mpi_int, mpi_max, mpi_status_ignore, mpi_sum -from .mpi_f08_interfaces import mpi_allreduce, mpi_barrier, mpi_bcast, mpi_comm_rank, mpi_comm_size, mpi_finalize, mpi_get_count, mpi_init, mpi_probe, mpi_recv, mpi_reduce, mpi_send +from .mpi_f08_types import mpi_any_source, mpi_any_tag, mpi_comm_world, mpi_in_place, mpi_int, mpi_max, mpi_status_ignore, mpi_sum +from .mpi_f08_interfaces import mpi_allreduce, mpi_barrier, mpi_bcast, mpi_comm_rank, mpi_comm_size, mpi_finalize, mpi_init, mpi_recv, mpi_reduce, mpi_send from .mpi_types import Mpi_Comm, Mpi_Datatype, Mpi_Op, Mpi_Status ``` @@ -339,9 +324,7 @@ from .mpi_f08_types import ( Mpi_Status, mpi_any_source, mpi_any_tag, - mpi_byte, mpi_comm_world, - mpi_double, mpi_in_place, mpi_int, mpi_max, @@ -397,16 +380,6 @@ def recv( status: Mpi_Status, ) -> None: ... -@raises(status="ierror", success=0) -@bind("MPI_Probe") -@native_call([Arg(0), Arg(1), Arg(2), Arg(3), Hidden("ierror", Int32)]) -def probe(source: Int32, tag: Int32, comm: Mpi_Comm, status: Mpi_Status) -> None: ... - -@raises(status="ierror", success=0) -@bind("MPI_Get_count") -@native_call([Arg(0), Arg(1), Return("count", 0), Hidden("ierror", Int32)]) -def get_count(status: Mpi_Status, datatype: Mpi_Datatype) -> Int32: ... - @raises(status="ierror", success=0) @bind("MPI_Bcast") @native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Hidden("ierror", Int32)]) @@ -443,8 +416,6 @@ __all__ = [ "barrier", "send", "recv", - "probe", - "get_count", "bcast", "reduce", "allreduce", @@ -454,9 +425,7 @@ __all__ = [ "Mpi_Status", "mpi_any_source", "mpi_any_tag", - "mpi_byte", "mpi_comm_world", - "mpi_double", "mpi_in_place", "mpi_int", "mpi_max", @@ -473,11 +442,11 @@ where each one comes from: | --- | --- | --- | | `@bind("MPI_Send")` on `def send` | every function | The Python name differs from the Fortran name it calls. | | `Int32(Arg(0).size)` | `count` of `MPI_Send` | The count is computed from the buffer, so the caller does not pass it. | -| `Return("rank", 0)` | `rank` of `MPI_Comm_rank` | The output argument becomes the return value: `comm_rank` returns the rank, `get_count` the count. | +| `Return("rank", 0)` | `rank` of `MPI_Comm_rank` | The output argument becomes the return value: `comm_rank` returns the rank. | | `Hidden("ierror", Int32)` with `@raises(status="ierror", success=0)` | every function | The error code is not an argument; a nonzero code raises an exception. | -`recv` and `probe` keep their `status` as an argument instead of returning -it. The caller passes either an `Mpi_Status`, which Open MPI fills in, or +`recv` keeps its `status` as an argument. A caller that wants the status +passes an `Mpi_Status` for Open MPI to fill in; one that does not passes `mpi_status_ignore`, which tells Open MPI not to. The facade imports its handle types and constants from @@ -542,73 +511,33 @@ mpi_f08.finalize() A contract describes native calls. What mpi4py adds on top of MPI is a Python object model, and that belongs in Python. Save this module as `prik_mpi.py` -in the same directory as `prik_openmpi_f08.so`: +in the same directory as `prik_openmpi_f08.so`. It is deliberately small: one +datatype and no status, enough to show the shape. ```python -"""An mpi4py-style Python API over the PRIK-generated Open MPI extension.""" +"""An mpi4py-style Python API over the PRIK-generated Open MPI extension. + +It illustrates the shape of mpi4py rather than all of it: every buffer is an +np.int32 array sent as MPI_INT, and no receive reports a status. +""" import atexit -import pickle import numpy as np from prik_openmpi_f08 import mpi_f08 as _mpi -# Ranks, tags, and counts are np.int32, the type the contract takes, from the -# start: the extension returns them as np.int32, the constants and defaults -# here are np.int32, and arithmetic with Python integers keeps the type. So -# they pass straight to the contract, never converted. +# Ranks and tags are np.int32, the type the contract takes: the extension +# returns them as np.int32, and so are these constants and defaults. ANY_SOURCE = _mpi.mpi_any_source ANY_TAG = _mpi.mpi_any_tag -_ZERO = np.int32(0) IN_PLACE = _mpi.mpi_in_place -STATUS_IGNORE = _mpi.mpi_status_ignore -BYTE = _mpi.mpi_byte -INT = _mpi.mpi_int -DOUBLE = _mpi.mpi_double SUM = _mpi.mpi_sum MAX = _mpi.mpi_max - -# The MPI datatype of each NumPy element type, for buffers given without one. -_DATATYPES = {np.dtype(np.uint8): BYTE, np.dtype(np.int32): INT, np.dtype(np.float64): DOUBLE} - - -def _message(buf): - """Return a buffer's array and MPI datatype; ``buf`` is an array or ``[array, datatype]``.""" - if isinstance(buf, np.ndarray): - return buf, _DATATYPES[buf.dtype] - array, datatype = buf - return array, datatype - - -class Status: - """What MPI reports about a received message, filled in by the call given it.""" - - def __init__(self): - self._native = _mpi.Mpi_Status() - - @property - def source(self): - return self._native.mpi_source - - @property - def tag(self): - return self._native.mpi_tag - - def Get_source(self): - return self.source - - def Get_tag(self): - return self.tag - - def Get_count(self, datatype=BYTE): - return _mpi.get_count(self._native, datatype) - - -def _native_status(status): - """Return the status MPI fills in; without one, MPI_STATUS_IGNORE, as in mpi4py.""" - return STATUS_IGNORE if status is None else status._native +_ZERO = np.int32(0) +_INT = _mpi.mpi_int +_STATUS_IGNORE = _mpi.mpi_status_ignore class Comm: @@ -623,46 +552,23 @@ class Comm: def Get_size(self): return _mpi.comm_size(self.handle) - rank = property(Get_rank) - size = property(Get_size) - def Barrier(self): _mpi.barrier(self.handle) def Send(self, buf, dest, tag=_ZERO): - array, datatype = _message(buf) - _mpi.send(array, datatype, dest, tag, self.handle) + _mpi.send(buf, _INT, dest, tag, self.handle) - def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG, status=None): - array, datatype = _message(buf) - _mpi.recv(array, datatype, source, tag, self.handle, _native_status(status)) - - def Probe(self, source=ANY_SOURCE, tag=ANY_TAG, status=None): - _mpi.probe(source, tag, self.handle, _native_status(status)) - return True + def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG): + _mpi.recv(buf, _INT, source, tag, self.handle, _STATUS_IGNORE) def Bcast(self, buf, root=_ZERO): - array, datatype = _message(buf) - _mpi.bcast(array, datatype, root, self.handle) + _mpi.bcast(buf, _INT, root, self.handle) def Reduce(self, sendbuf, recvbuf, op=SUM, root=_ZERO): - array, datatype = _message(recvbuf) - _mpi.reduce(sendbuf, array, datatype, op, root, self.handle) + _mpi.reduce(sendbuf, recvbuf, _INT, op, root, self.handle) def Allreduce(self, sendbuf, recvbuf, op=SUM): - array, datatype = _message(recvbuf) - _mpi.allreduce(sendbuf, array, datatype, op, self.handle) - - # Python objects travel pickled, as with mpi4py's lowercase methods. - def send(self, obj, dest, tag=_ZERO): - self.Send(np.frombuffer(pickle.dumps(obj), dtype=np.uint8), dest, tag) - - def recv(self, buf=None, source=ANY_SOURCE, tag=ANY_TAG, status=None): - status = status if status is not None else Status() - self.Probe(source, tag, status) - data = np.empty(status.Get_count(BYTE), dtype=np.uint8) - self.Recv(data, status.source, status.tag, status) - return pickle.loads(data.tobytes()) + _mpi.allreduce(sendbuf, recvbuf, _INT, op, self.handle) COMM_WORLD = Comm(_mpi.mpi_comm_world) @@ -675,22 +581,18 @@ atexit.register(_mpi.finalize) Everything in it calls the generated functions of step 5: - **Objects and methods.** `Comm` wraps an `Mpi_Comm` handle and spells - mpi4py's methods; `COMM_WORLD` wraps `mpi_comm_world`. `Status` owns an - `Mpi_Status` that `Recv` and `Probe` have Open MPI fill in, and answers - `Get_source`, `Get_tag`, and `Get_count` from it. Given no `Status`, they - pass `MPI_STATUS_IGNORE`, as mpi4py does. -- **Buffers.** A buffer is a NumPy array, whose MPI datatype is chosen from - its element type, or an `[array, datatype]` pair naming the datatype - explicitly, as in mpi4py. + mpi4py's methods; `COMM_WORLD` wraps `mpi_comm_world`. Each method is one + call to a generated function. +- **One datatype.** Every buffer is an `np.int32` array, so every call passes + `MPI_INT`. +- **No status.** `Recv` always passes `MPI_STATUS_IGNORE`, as mpi4py does when + it is given no status. - **Defaults and `np.int32` values.** `tag`, `source=ANY_SOURCE`, `root`, and `op=SUM` are keyword defaults. Ranks and tags are the `np.int32` values the contract's `Int32` arguments take from the start -- `Get_rank` returns one, `ANY_SOURCE` and the defaults are, and `rank + 1` stays one -- so they pass straight through. Converting a plain integer on every call would cost more than a small MPI call. -- **Python objects.** Lowercase `send` pickles an object into a byte array and - sends it; `recv` probes the incoming message, sizes a byte array with - `Get_count`, receives it, and unpickles it. - **Lifetime.** As with mpi4py, importing the module initializes MPI, and MPI is finalized when the interpreter exits. @@ -708,11 +610,10 @@ The two ranks print these lines, each rank's lines in order but the ranks in whichever order they finish: ```text -rank 1 received {'a': 7, 'b': 3.14} from rank 0 rank 1 received [0, 1, 2, 3] -rank 0 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5] +rank 0 of 2: bcast [0, 1, 2], sum [3, 5], in place [3, 5] rank 0 max [2, 3] -rank 1 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5] +rank 1 of 2: bcast [0, 1, 2], sum [3, 5], in place [3, 5] ``` Here is what happened. `mpirun` started two Python processes as MPI ranks. @@ -770,7 +671,7 @@ exercised in CI, not the only ones that can work. ## Limitations -This tutorial selected twenty-two names; the rest of `mpi_f08` works the same +This tutorial selected eighteen names; the rest of `mpi_f08` works the same way when you select it, within these limits of what PRIK supports today: - **Arrays of handles.** Routines taking an array of derived-type values, such @@ -789,12 +690,13 @@ way when you select it, within these limits of what PRIK supports today: `Isend` or `Irecv`: mpi4py's request objects keep their buffers alive, and a faithful imitation would need arrays of requests, the first limitation. -`prik_mpi.py` imitates the part of mpi4py this program uses, not all of it. -Ranks and tags must be `np.int32`, as in the program above; a plain Python -`int` is refused with a `TypeError`, where mpi4py accepts one. It passes -contiguous NumPy arrays only -- a strided view is refused with a -`TypeError` -- and picks a datatype for `int32`, `float64`, and `uint8` -arrays. It has none of mpi4py's other communicators, lowercase collectives, -or `MPI.Exception`: under Open MPI's default error handler an MPI error aborts -the job, and otherwise a nonzero `ierror` raises the exception the contract's -`@raises` produces. +`prik_mpi.py` is an illustration, not a complete binding. Its buffers are +contiguous `np.int32` arrays only -- a strided view is refused with a +`TypeError` -- and it reports no status. Ranks and tags must be `np.int32`, +as in the program above; a plain Python `int` is refused with a `TypeError`, +where mpi4py accepts one. It has none of mpi4py's other datatypes, +communicators, pickled-object methods, or `MPI.Exception`: under Open MPI's +default error handler an MPI error aborts the job, and otherwise a nonzero +`ierror` raises the exception the contract's `@raises` produces. Each of these +is more Python over the same kind of generated calls, or more names in the +export list. diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_exports.txt b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_exports.txt index 61d2c4c67..e1c7e5c97 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_exports.txt +++ b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_exports.txt @@ -5,15 +5,11 @@ mpi_f08::MPI_Comm_size mpi_f08::MPI_Barrier mpi_f08::MPI_Send mpi_f08::MPI_Recv -mpi_f08::MPI_Probe -mpi_f08::MPI_Get_count mpi_f08::MPI_Bcast mpi_f08::MPI_Reduce mpi_f08::MPI_Allreduce mpi_f08::MPI_COMM_WORLD -mpi_f08::MPI_BYTE mpi_f08::MPI_INT -mpi_f08::MPI_DOUBLE mpi_f08::MPI_SUM mpi_f08::MPI_MAX mpi_f08::MPI_IN_PLACE diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi index 081bbbc76..0d8b3cd29 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi +++ b/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi @@ -6,9 +6,7 @@ from .mpi_f08_types import ( Mpi_Status, mpi_any_source, mpi_any_tag, - mpi_byte, mpi_comm_world, - mpi_double, mpi_in_place, mpi_int, mpi_max, @@ -64,16 +62,6 @@ def recv( status: Mpi_Status, ) -> None: ... -@raises(status="ierror", success=0) -@bind("MPI_Probe") -@native_call([Arg(0), Arg(1), Arg(2), Arg(3), Hidden("ierror", Int32)]) -def probe(source: Int32, tag: Int32, comm: Mpi_Comm, status: Mpi_Status) -> None: ... - -@raises(status="ierror", success=0) -@bind("MPI_Get_count") -@native_call([Arg(0), Arg(1), Return("count", 0), Hidden("ierror", Int32)]) -def get_count(status: Mpi_Status, datatype: Mpi_Datatype) -> Int32: ... - @raises(status="ierror", success=0) @bind("MPI_Bcast") @native_call([Arg(0), Int32(Arg(0).size), Arg(1), Arg(2), Arg(3), Hidden("ierror", Int32)]) @@ -110,8 +98,6 @@ __all__ = [ "barrier", "send", "recv", - "probe", - "get_count", "bcast", "reduce", "allreduce", @@ -121,9 +107,7 @@ __all__ = [ "Mpi_Status", "mpi_any_source", "mpi_any_tag", - "mpi_byte", "mpi_comm_world", - "mpi_double", "mpi_in_place", "mpi_int", "mpi_max", diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py index 109e31baa..3ebdfd48a 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/mpi_example.py @@ -6,35 +6,26 @@ rank = comm.Get_rank() size = comm.Get_size() -# Ranks and tags are np.int32 from the start: Get_rank returns one, these -# constants are, and rank + 1 stays one. +# Buffers are np.int32 arrays, and ranks and tags are np.int32 too: +# Get_rank returns one, and rank + 1 stays one. ROOT = np.int32(0) -OBJECT_TAG = np.int32(11) -ARRAY_TAG = np.int32(77) +TAG = np.int32(77) -# Python objects travel pickled. +# Point to point: rank 0 sends four integers to rank 1. if rank == 0: - comm.send({"a": 7, "b": 3.14}, dest=rank + 1, tag=OBJECT_TAG) + data = np.arange(4, dtype=np.int32) + comm.Send(data, dest=rank + 1, tag=TAG) elif rank == 1: - status = MPI.Status() - data = comm.recv(source=MPI.ANY_SOURCE, tag=OBJECT_TAG, status=status) - print(f"rank 1 received {data} from rank {status.Get_source()}") - -# NumPy arrays travel as buffers, with an explicit MPI datatype ... -if rank == 0: - data = np.arange(4, dtype="i") - comm.Send([data, MPI.INT], dest=rank + 1, tag=ARRAY_TAG) -elif rank == 1: - data = np.empty(4, dtype="i") - comm.Recv([data, MPI.INT], source=rank - 1, tag=ARRAY_TAG) + data = np.empty(4, dtype=np.int32) + comm.Recv(data, source=rank - 1, tag=TAG) print(f"rank 1 received {data.tolist()}") -# ... or with the datatype taken from the array. -data = np.arange(3, dtype=np.float64) if rank == 0 else np.empty(3, dtype=np.float64) +# Broadcast: rank 0's values reach every rank. +data = np.arange(3, dtype=np.int32) if rank == 0 else np.empty(3, dtype=np.int32) comm.Bcast(data, root=ROOT) -# Collectives: every rank contributes. -values = np.array([rank + 1, rank + 2], dtype="i") +# Reductions: every rank contributes. +values = np.array([rank + 1, rank + 2], dtype=np.int32) total = np.empty_like(values) comm.Allreduce(values, total, op=MPI.SUM) largest = np.empty_like(values) diff --git a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py index 8e411dab7..97e2187ff 100644 --- a/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py +++ b/tests/fortran/assumed_types/end_to_end/fixtures/runtime/prik_mpi.py @@ -1,66 +1,25 @@ -"""An mpi4py-style Python API over the PRIK-generated Open MPI extension.""" +"""An mpi4py-style Python API over the PRIK-generated Open MPI extension. + +It illustrates the shape of mpi4py rather than all of it: every buffer is an +np.int32 array sent as MPI_INT, and no receive reports a status. +""" import atexit -import pickle import numpy as np from prik_openmpi_f08 import mpi_f08 as _mpi -# Ranks, tags, and counts are np.int32, the type the contract takes, from the -# start: the extension returns them as np.int32, the constants and defaults -# here are np.int32, and arithmetic with Python integers keeps the type. So -# they pass straight to the contract, never converted. +# Ranks and tags are np.int32, the type the contract takes: the extension +# returns them as np.int32, and so are these constants and defaults. ANY_SOURCE = _mpi.mpi_any_source ANY_TAG = _mpi.mpi_any_tag -_ZERO = np.int32(0) IN_PLACE = _mpi.mpi_in_place -STATUS_IGNORE = _mpi.mpi_status_ignore -BYTE = _mpi.mpi_byte -INT = _mpi.mpi_int -DOUBLE = _mpi.mpi_double SUM = _mpi.mpi_sum MAX = _mpi.mpi_max - -# The MPI datatype of each NumPy element type, for buffers given without one. -_DATATYPES = {np.dtype(np.uint8): BYTE, np.dtype(np.int32): INT, np.dtype(np.float64): DOUBLE} - - -def _message(buf): - """Return a buffer's array and MPI datatype; ``buf`` is an array or ``[array, datatype]``.""" - if isinstance(buf, np.ndarray): - return buf, _DATATYPES[buf.dtype] - array, datatype = buf - return array, datatype - - -class Status: - """What MPI reports about a received message, filled in by the call given it.""" - - def __init__(self): - self._native = _mpi.Mpi_Status() - - @property - def source(self): - return self._native.mpi_source - - @property - def tag(self): - return self._native.mpi_tag - - def Get_source(self): - return self.source - - def Get_tag(self): - return self.tag - - def Get_count(self, datatype=BYTE): - return _mpi.get_count(self._native, datatype) - - -def _native_status(status): - """Return the status MPI fills in; without one, MPI_STATUS_IGNORE, as in mpi4py.""" - return STATUS_IGNORE if status is None else status._native +_ZERO = np.int32(0) +_INT = _mpi.mpi_int +_STATUS_IGNORE = _mpi.mpi_status_ignore class Comm: @@ -75,46 +34,23 @@ def Get_rank(self): def Get_size(self): return _mpi.comm_size(self.handle) - rank = property(Get_rank) - size = property(Get_size) - def Barrier(self): _mpi.barrier(self.handle) def Send(self, buf, dest, tag=_ZERO): - array, datatype = _message(buf) - _mpi.send(array, datatype, dest, tag, self.handle) - - def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG, status=None): - array, datatype = _message(buf) - _mpi.recv(array, datatype, source, tag, self.handle, _native_status(status)) + _mpi.send(buf, _INT, dest, tag, self.handle) - def Probe(self, source=ANY_SOURCE, tag=ANY_TAG, status=None): - _mpi.probe(source, tag, self.handle, _native_status(status)) - return True + def Recv(self, buf, source=ANY_SOURCE, tag=ANY_TAG): + _mpi.recv(buf, _INT, source, tag, self.handle, _STATUS_IGNORE) def Bcast(self, buf, root=_ZERO): - array, datatype = _message(buf) - _mpi.bcast(array, datatype, root, self.handle) + _mpi.bcast(buf, _INT, root, self.handle) def Reduce(self, sendbuf, recvbuf, op=SUM, root=_ZERO): - array, datatype = _message(recvbuf) - _mpi.reduce(sendbuf, array, datatype, op, root, self.handle) + _mpi.reduce(sendbuf, recvbuf, _INT, op, root, self.handle) def Allreduce(self, sendbuf, recvbuf, op=SUM): - array, datatype = _message(recvbuf) - _mpi.allreduce(sendbuf, array, datatype, op, self.handle) - - # Python objects travel pickled, as with mpi4py's lowercase methods. - def send(self, obj, dest, tag=_ZERO): - self.Send(np.frombuffer(pickle.dumps(obj), dtype=np.uint8), dest, tag) - - def recv(self, buf=None, source=ANY_SOURCE, tag=ANY_TAG, status=None): - status = status if status is not None else Status() - self.Probe(source, tag, status) - data = np.empty(status.Get_count(BYTE), dtype=np.uint8) - self.Recv(data, status.source, status.tag, status) - return pickle.loads(data.tobytes()) + _mpi.allreduce(sendbuf, recvbuf, _INT, op, self.handle) COMM_WORLD = Comm(_mpi.mpi_comm_world) diff --git a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py index 20a694703..35163f2ae 100644 --- a/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py +++ b/tests/fortran/assumed_types/end_to_end/test_openmpi_f08.py @@ -272,9 +272,8 @@ def run(script: str) -> list[str]: assert run("mpi_example.py") == [ "rank 0 max [2, 3]", - "rank 0 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5]", - "rank 1 of 2: bcast [0.0, 1.0, 2.0], sum [3, 5], in place [3, 5]", + "rank 0 of 2: bcast [0, 1, 2], sum [3, 5], in place [3, 5]", + "rank 1 of 2: bcast [0, 1, 2], sum [3, 5], in place [3, 5]", "rank 1 received [0, 1, 2, 3]", - "rank 1 received {'a': 7, 'b': 3.14} from rank 0", ] assert run(STATUS_IGNORE_CHECK.name) == ["Mpi_Status: status tag 21, ignored status unchanged True"] From 63ebb866ae7c01c1c770d4a0be17d66dd485edc1 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 12:27:25 +0100 Subject: [PATCH 31/47] Compare the mpi_f08 tutorial's API with mpi4py in a short table A "How fast it is" section gives, relative to mpi4py, how Allreduce, Barrier, and Get_rank perform through the generated functions and through prik_mpi.py. Co-Authored-By: Claude Opus 5.5 --- docs/user/tutorials/openmpi-f08.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 01e7b6df7..e37e506e2 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -626,6 +626,18 @@ No part of Open MPI was rebuilt. With `from mpi4py import MPI` in place of `import prik_mpi as MPI`, mpi4py runs the same program and prints the same lines. +## How fast it is + +Compared with mpi4py: + +| Call | Generated functions | `prik_mpi.py` | +| --- | --- | --- | +| `Allreduce`, 1 integer | 33% faster | 10% faster | +| `Allreduce`, 1,024 integers | 20% faster | 9% faster | +| `Allreduce`, large arrays | same speed | same speed | +| `Barrier` | 40% slower | 49% slower | +| `Get_rank` | about 9× slower | about 9× slower | + ## Why the configured tree must match the installation `mpi_f08` is not the same text in every Open MPI build. `configure` decides From a423f1ee73a4acdec432ae9de8cbc29eb1f2e586 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 12:47:31 +0100 Subject: [PATCH 32/47] Give the mpi_f08 tutorial's mpi4py comparison its timings and machine Co-Authored-By: Claude Opus 5.5 --- docs/user/tutorials/openmpi-f08.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index e37e506e2..f8bc63eb5 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -628,15 +628,20 @@ runs the same program and prints the same lines. ## How fast it is -Compared with mpi4py: - -| Call | Generated functions | `prik_mpi.py` | -| --- | --- | --- | -| `Allreduce`, 1 integer | 33% faster | 10% faster | -| `Allreduce`, 1,024 integers | 20% faster | 9% faster | -| `Allreduce`, large arrays | same speed | same speed | -| `Barrier` | 40% slower | 49% slower | -| `Get_rank` | about 9× slower | about 9× slower | +Time per call, compared with mpi4py: + +| Call | mpi4py | Generated functions | `prik_mpi.py` | +| --- | ---: | ---: | ---: | +| `Allreduce`, 1 integer | 1.26 µs | 0.85 µs (33% faster) | 1.14 µs (10% faster) | +| `Allreduce`, 1,024 integers | 2.62 µs | 2.10 µs (20% faster) | 2.40 µs (9% faster) | +| `Allreduce`, 1,048,576 integers | 3.04 ms | 2.82 ms (about the same) | 2.81 ms (about the same) | +| `Barrier` | 0.35 µs | 0.49 µs (41% slower) | 0.52 µs (49% slower) | +| `Get_rank` | 34 ns | 289 ns (about 9× slower) | 307 ns (about 9× slower) | + +These were measured on a local machine, with two ranks on it: an AMD Ryzen 5 +5600H laptop (x86-64, 6 cores and 12 threads, up to 4.28 GHz, 7 GB of memory) +running Ubuntu 22.04, with Python 3.10, NumPy 2.2, GCC and gfortran 11.4, +Open MPI 5.0.11, and mpi4py 4.1.2. ## Why the configured tree must match the installation From 203d73539d3d2926e3083b025d06793aae46e519 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 12:51:21 +0100 Subject: [PATCH 33/47] Explain in the mpi_f08 tutorial why small calls trail mpi4py Co-Authored-By: Claude Opus 5.5 --- docs/user/tutorials/openmpi-f08.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index f8bc63eb5..733a1e0fd 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -638,6 +638,14 @@ Time per call, compared with mpi4py: | `Barrier` | 0.35 µs | 0.49 µs (41% slower) | 0.52 µs (49% slower) | | `Get_rank` | 34 ns | 289 ns (about 9× slower) | 307 ns (about 9× slower) | +Every PRIK call pays a fixed cost of about a quarter of a microsecond: it +parses its arguments, unwraps the `Mpi_Comm` wrapper to reach its native +handle, and returns a new `np.int32`. mpi4py, written in Cython, keeps the C +handle inside its `Comm` object and returns a cached Python `int`. `Get_rank` +and `Barrier` do almost no work of their own, so that fixed cost is what +their times show; when data moves, it is small next to the work, and PRIK +comes out ahead. + These were measured on a local machine, with two ranks on it: an AMD Ryzen 5 5600H laptop (x86-64, 6 cores and 12 threads, up to 4.28 GHz, 7 GB of memory) running Ubuntu 22.04, with Python 3.10, NumPy 2.2, GCC and gfortran 11.4, From fa86b4bcfc0919553586922e02658e2714b71dbd Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 12:52:55 +0100 Subject: [PATCH 34/47] Shorten the mpi_f08 tutorial's note on small-call overhead Co-Authored-By: Claude Opus 5.5 --- docs/user/tutorials/openmpi-f08.md | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 733a1e0fd..cf7069fb0 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -638,13 +638,8 @@ Time per call, compared with mpi4py: | `Barrier` | 0.35 µs | 0.49 µs (41% slower) | 0.52 µs (49% slower) | | `Get_rank` | 34 ns | 289 ns (about 9× slower) | 307 ns (about 9× slower) | -Every PRIK call pays a fixed cost of about a quarter of a microsecond: it -parses its arguments, unwraps the `Mpi_Comm` wrapper to reach its native -handle, and returns a new `np.int32`. mpi4py, written in Cython, keeps the C -handle inside its `Comm` object and returns a cached Python `int`. `Get_rank` -and `Barrier` do almost no work of their own, so that fixed cost is what -their times show; when data moves, it is small next to the work, and PRIK -comes out ahead. +`Get_rank` and `Barrier` do almost no work, so their times are just the +overhead of making a call, which is higher through PRIK than through mpi4py. These were measured on a local machine, with two ranks on it: an AMD Ryzen 5 5600H laptop (x86-64, 6 cores and 12 threads, up to 4.28 GHz, 7 GB of memory) From 60ab5265247938978a38fc0e64eef5f900f7b78b Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 13:45:48 +0100 Subject: [PATCH 35/47] Re-measure the mpi_f08 tutorial's mpi4py comparison with one timer The earlier figures timed the generated functions through a Python lambda but mpi4py's Get_rank and Barrier directly, which charged PRIK an extra Python call. All three are now timed with mpi4py's MPI.Wtime, one call per loop iteration: a Barrier through the generated functions is as fast as mpi4py's, and Get_rank is about 7x slower rather than 9x. Co-Authored-By: Claude Opus 5.5 --- docs/user/tutorials/openmpi-f08.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index cf7069fb0..8fd23200e 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -632,19 +632,20 @@ Time per call, compared with mpi4py: | Call | mpi4py | Generated functions | `prik_mpi.py` | | --- | ---: | ---: | ---: | -| `Allreduce`, 1 integer | 1.26 µs | 0.85 µs (33% faster) | 1.14 µs (10% faster) | -| `Allreduce`, 1,024 integers | 2.62 µs | 2.10 µs (20% faster) | 2.40 µs (9% faster) | -| `Allreduce`, 1,048,576 integers | 3.04 ms | 2.82 ms (about the same) | 2.81 ms (about the same) | -| `Barrier` | 0.35 µs | 0.49 µs (41% slower) | 0.52 µs (49% slower) | -| `Get_rank` | 34 ns | 289 ns (about 9× slower) | 307 ns (about 9× slower) | - -`Get_rank` and `Barrier` do almost no work, so their times are just the -overhead of making a call, which is higher through PRIK than through mpi4py. - -These were measured on a local machine, with two ranks on it: an AMD Ryzen 5 -5600H laptop (x86-64, 6 cores and 12 threads, up to 4.28 GHz, 7 GB of memory) -running Ubuntu 22.04, with Python 3.10, NumPy 2.2, GCC and gfortran 11.4, -Open MPI 5.0.11, and mpi4py 4.1.2. +| `Allreduce`, 1 integer | 1.02 µs | 0.67 µs (35% faster) | 0.92 µs (10% faster) | +| `Allreduce`, 1,024 integers | 2.33 µs | 1.78 µs (23% faster) | 2.15 µs (7% faster) | +| `Allreduce`, 1,048,576 integers | 2.33 ms | 2.12 ms (about the same) | 2.37 ms (about the same) | +| `Barrier` | 0.32 µs | 0.32 µs (the same) | 0.50 µs (57% slower) | +| `Get_rank` | 19 ns | 134 ns (about 7× slower) | 297 ns (about 15× slower) | + +`Get_rank` does almost no work, so its time is just the overhead of making a +call, which is higher through PRIK than through mpi4py. + +These were measured on a local machine, with two ranks on it, and timed with +mpi4py's `MPI.Wtime` for all three: an AMD Ryzen 5 5600H laptop (x86-64, +6 cores and 12 threads, up to 4.28 GHz, 7 GB of memory) running Ubuntu 22.04, +with Python 3.10, NumPy 2.2, GCC and gfortran 11.4, Open MPI 5.0.11, and +mpi4py 4.1.2. ## Why the configured tree must match the installation From 2dca2af8dc1b4f51265bd1c75ecba5e7b60acbce Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 14:06:33 +0100 Subject: [PATCH 36/47] Describe Get_rank's time in the mpi_f08 tutorial as call overhead to optimize later Co-Authored-By: Claude Opus 5.5 --- docs/user/tutorials/openmpi-f08.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 8fd23200e..5e9f8949d 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -638,8 +638,9 @@ Time per call, compared with mpi4py: | `Barrier` | 0.32 µs | 0.32 µs (the same) | 0.50 µs (57% slower) | | `Get_rank` | 19 ns | 134 ns (about 7× slower) | 297 ns (about 15× slower) | -`Get_rank` does almost no work, so its time is just the overhead of making a -call, which is higher through PRIK than through mpi4py. +`Get_rank` is the cheapest call, so its time is almost all the overhead of +making a call. That overhead is small, but higher through PRIK than through +mpi4py, and can be optimized later. These were measured on a local machine, with two ranks on it, and timed with mpi4py's `MPI.Wtime` for all three: an AMD Ryzen 5 5600H laptop (x86-64, From bb1b994097a2df112080ad4403833c5af2d74534 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 19:31:34 +0100 Subject: [PATCH 37/47] codex: consolidate PRIK tests around behavioral evidence --- AGENTS.md | 50 +- CHANGELOG.md | 4 + docs/developer/packages/codegen.md | 4 +- docs/developer/packages/codegen/c-binding.md | 4 +- .../packages/codegen/fortran-bridge.md | 2 +- docs/developer/packages/compiler.md | 2 +- docs/developer/packages/planning.md | 2 +- docs/developer/packages/printers.md | 2 +- docs/developer/packages/runtime.md | 4 +- docs/developer/packages/semantics.md | 1 - docs/developer/packages/utilities.md | 4 +- docs/user/language-support/feature-matrix.md | 2 +- tests/c/data_types/probes/test_c_types.py | 23 - .../semantics/test_types_and_constants.py | 216 +-- .../semantics/test_c_enum_semantics.py | 9 - .../codegen/test_positional_only_lowering.py | 41 - .../test_direct_c_hidden_native_outputs.py | 128 +- .../end_to_end/test_export_symbol_workflow.py | 44 +- .../end_to_end/test_public_name_contract.py | 67 +- tests/c/functions/parsing/test_c_functions.py | 209 +- .../semantics/test_functions_and_callbacks.py | 222 +-- .../building/pipeline/test_c_build_cli.py | 148 +- .../pipeline/test_c_direct_rejections.py | 249 ++- .../pipeline/test_c_cli_argument_contract.py | 210 +- .../pipeline/test_c_cli_output_contract.py | 182 +- .../cli/pipeline/test_c_cli_skeleton.py | 564 ------ .../cli/pipeline/test_c_cli_stage_dispatch.py | 301 ++- .../c/infrastructure/parsing/test_c_corpus.py | 90 - .../test_c_declarations_and_declarators.py | 692 +++---- .../parsing/test_c_error_fixture_suite.py | 13 - .../parsing/test_c_fixture_suite.py | 145 +- .../parsing/test_c_json_sanity.py | 99 - .../parsing/test_c_lexer_preprocessor.py | 379 +--- .../parsing/test_c_model_serialization.py | 287 --- .../parsing/test_c_parser_properties.py | 11 - .../parsing/test_c_project_resolution.py | 290 ++- .../parsing/test_c_public_api_skeleton.py | 296 +-- .../preprocessing/test_c_preprocessing_cli.py | 277 +-- .../test_c_preprocessing_configuration.py | 351 +--- .../test_c_preprocessing_dependencies.py | 77 +- .../test_c_preprocessing_execution.py | 366 ++-- .../test_c_preprocessing_properties.py | 16 +- .../preprocessing/test_error_paths.py | 152 -- .../preprocessing/test_source_mappings.py | 35 - .../test_projects_and_diagnostics.py | 164 +- .../pipeline/test_c_pyi_contract_fixtures.py | 12 - .../test_runtime_rank_pointer_lowering.py | 33 - .../test_direct_c_pointer_contracts.py | 284 ++- .../test_runtime_rank_pointer_policy.py | 37 +- .../test_exact_native_scalar_lowering.py | 68 +- .../end_to_end/test_direct_c_runtime.py | 359 ++-- .../policy/test_direct_c_policy.py | 55 - .../test_exact_native_scalar_contract.py | 46 +- .../end_to_end/test_direct_c_strings.py | 359 ++-- .../parsing/test_c_structs_unions_typedefs.py | 183 +- .../semantics/test_c_record_semantics.py | 83 +- .../test_collision_adapter_lowering.py | 87 +- .../test_collision_adapter_runtime.py | 90 +- tests/docs/_structure_support.py | 19 - tests/docs/test_examples.py | 83 +- tests/docs/test_metadata_and_visibility.py | 46 +- tests/docs/test_navigation.py | 19 +- tests/docs/test_publication.py | 12 +- tests/docs/test_reference_and_codebase_map.py | 69 +- tests/fortran/CONTRACT_COVERAGE.md | 248 +-- tests/fortran/README.md | 2 +- tests/fortran/_support/printer_models.py | 4 - tests/fortran/_support/semantic_properties.py | 8 - .../codegen/test_allocatable_lowering.py | 273 +-- .../test_allocatable_cross_extension.py | 115 +- .../end_to_end/test_allocatable_handles.py | 97 +- .../test_deferred_character_handles.py | 5 + .../parsing/test_allocatable_declarations.py | 18 - .../test_allocatable_contract_printing.py | 38 - ...st_allocatable_module_contract_printing.py | 32 - .../policy/test_allocatable_handle_policy.py | 207 +- .../policy/test_allocatable_result_policy.py | 78 +- .../test_allocatable_contract_handles.py | 184 -- .../test_allocatable_handle_protocol.py | 249 --- .../test_fortran_allocatable_semantics.py | 55 - .../test_pyi_allocatable_semantics.py | 68 - .../codegen/test_array_buffer_lowering.py | 106 +- .../codegen/test_array_output_identity.py | 112 -- .../codegen/test_array_result_lowering.py | 88 +- .../test_dense_array_shape_lowering.py | 202 +- .../codegen/test_specialized_array_roles.py | 165 -- .../codegen/test_strided_array_lowering.py | 103 +- .../farray_contracts_f90.pyi | 83 +- .../fixtures/native/farray_contracts_f90.f90 | 84 - .../fixtures/native/fassumed_rank_f90.f90 | 137 +- .../fixtures/primitive_dtype_rank_matrix.py | 15 +- .../test_array_contract_validation.py | 6 +- .../end_to_end/test_array_wrapper_parity.py | 72 +- .../test_edited_pyi_layout_contract.py | 10 +- .../test_layout_and_strided_arrays.py | 121 +- .../test_native_handle_array_forms.py | 18 +- .../test_primitive_dtype_rank_matrix.py | 5 +- .../end_to_end/test_signed_stride_handoff.py | 55 +- .../test_array_declarations_and_shapes.py | 154 +- .../test_generated_array_contracts.py | 33 +- .../arrays/policy/test_array_shape_policy.py | 30 - .../arrays/semantics/test_array_semantics.py | 143 +- .../test_declaration_expression_utilities.py | 76 +- .../end_to_end/test_assumed_type_calls.py | 27 +- .../runtime/test_assumed_type_actual.py | 6 - .../semantics/test_assumed_type_semantics.py | 30 +- .../codegen/test_callback_planning.py | 303 +-- .../end_to_end/test_array_callbacks.py | 22 +- .../test_callback_scalar_storage.py | 59 +- .../test_direct_bind_c_callback_storage.py | 12 +- .../test_multi_file_contract_generation.py | 38 +- .../end_to_end/test_optional_callbacks.py | 14 +- .../test_generated_callback_contracts.py | 28 - .../callbacks/policy/test_callback_policy.py | 191 -- .../semantics/test_pyi_callback_semantics.py | 48 - .../test_default_logical_scalar_lowering.py | 57 - .../codegen/test_logical_storage_boundary.py | 32 - .../test_primitive_scalar_input_lowering.py | 32 - .../test_primitive_scalar_result_lowering.py | 44 - .../test_primitive_scalar_type_catalogue.py | 11 - .../codegen/test_scalar_boundary_plan.py | 76 - .../test_project_kind_alias_chain.py | 10 +- .../end_to_end/test_scalar_wrapper_parity.py | 94 +- .../test_generated_scalar_contract.py | 21 - .../test_scalar_generated_pyi_contracts.py | 26 - .../probes/test_fortran_type_probes.py | 90 +- .../test_fortran_scalar_semantics.py | 14 - .../semantics/test_scalar_type_catalogue.py | 12 - .../semantics/test_types_and_storage.py | 134 -- .../codegen/test_class_surfaces.py | 64 - .../test_derived_array_field_lowering.py | 26 - .../codegen/test_derived_lowering.py | 64 +- .../codegen/test_scalar_actual_dummy_plan.py | 181 +- .../module_aliased_proxy/__init__.pyi | 1 - .../fmodule_derived_alias_f90.pyi | 15 - .../opaque_boundary/__init__.pyi | 1 - .../opaque_boundary/fderived_boundary_f90.pyi | 29 - .../native/fborrowed_finalizer_runtime.f90 | 32 - .../end_to_end/test_abstract_hierarchy.py | 99 +- .../end_to_end/test_borrowed_components.py | 1 + .../end_to_end/test_derived_boundaries.py | 2 + .../test_derived_runtime_mechanisms.py | 147 +- .../end_to_end/test_generic_constructor.py | 21 +- .../end_to_end/test_module_derived_aliases.py | 5 + .../test_scalar_actual_dummy_matrix.py | 193 +- .../end_to_end/test_types_across_modules.py | 31 +- .../parsing/test_derived_field_syntax.py | 31 +- .../parsing/test_derived_type_declarations.py | 19 - .../parsing/test_derived_type_errors.py | 30 - .../parsing/test_type_bound_diagnostics.py | 26 - .../test_generated_derived_contracts.py | 39 - .../policy/test_derived_accessor_policy.py | 149 -- .../policy/test_derived_policy_defaults.py | 49 - .../policy/test_merged_contract_names.py | 56 - .../fixtures/native/derived_type.f90 | 20 - .../test_fortran_derived_semantics.py | 368 ---- .../test_imported_derived_semantics.py | 109 -- .../semantics/test_pyi_class_semantics.py | 20 - .../parsing/test_enum_diagnostics.py | 75 +- .../enumerations/parsing/test_enum_syntax.py | 27 +- .../pipeline/test_generated_enum_contract.py | 19 - .../codegen/test_runtime_envelope_lowering.py | 33 - .../codegen/test_status_error_lowering.py | 170 +- .../end_to_end/test_status_projection.py | 2 + .../pipeline/test_concise_cli_diagnostics.py | 25 +- .../pipeline/test_debug_cli_tracebacks.py | 23 - .../test_runtime_generated_contracts.py | 5 +- .../codegen/test_multiple_function_results.py | 37 - .../codegen/test_scalar_function_writeback.py | 51 - .../end_to_end/test_external_procedures.py | 54 - ...test_function_direct_entrypoint_routing.py | 20 - .../end_to_end/test_hidden_native_outputs.py | 77 +- ...est_procedure_and_interface_regressions.py | 20 - .../policy/test_function_result_policy.py | 70 - .../policy/test_positional_only_surface.py | 41 +- ...an_conversion_procedures_and_interfaces.py | 37 - .../test_fortran_function_semantics.py | 29 - .../codegen/test_overload_dispatch_plan.py | 56 +- .../end_to_end/test_generic_interfaces.py | 29 +- .../test_generated_generic_contracts.py | 12 +- .../compiling/test_compiler_verbose.py | 44 +- .../compiling/test_example_native_library.py | 6 +- .../compiling/test_logical_interop_flags.py | 59 +- .../compiling/test_support_probe_artifacts.py | 19 - .../building/end_to_end/test_cmake_builds.py | 1721 ++++++----------- .../end_to_end/test_multi_source_builds.py | 136 +- .../end_to_end/test_native_bundles.py | 103 +- .../end_to_end/test_runtime_compatibility.py | 73 +- .../end_to_end/test_source_build_modes.py | 78 +- .../fruntime_abi_f90/__init__.pyi | 3 - .../fruntime_abi_f90/fruntime_abi_f90.pyi | 9 - .../building/pipeline/test_cmake_packaging.py | 81 +- .../building/pipeline/test_pyi_build_modes.py | 64 +- .../building/pipeline/test_root_build_api.py | 13 - .../test_source_generated_contracts.py | 5 - .../infrastructure/cli/pipeline/_support.py | 114 -- .../cli/pipeline/test_argument_contract.py | 1384 ++++--------- .../cli/pipeline/test_output_contract.py | 1167 +++-------- .../cli/pipeline/test_stage_dispatch.py | 955 ++------- .../infrastructure/codegen/test_binding.py | 23 - .../infrastructure/codegen/test_checks.py | 135 +- .../codegen/test_native_entrypoint_routing.py | 30 +- .../test_ordinary_fortran_codegen_baseline.py | 42 - .../infrastructure/codegen/test_package.py | 13 +- .../infrastructure/codegen/test_planner.py | 165 +- .../infrastructure/codegen/test_visitor.py | 43 +- .../end_to_end/test_fortran_magic_runtime.py | 58 +- .../jupyter/test_fortran_magic.py | 232 +-- .../infrastructure/naming/test_policy.py | 44 +- .../errors/err_duplicate_argument_name.f90 | 4 - .../errors/err_duplicate_argument_name.json | 12 - .../err_duplicate_declaration_procedure.f90 | 4 - .../err_duplicate_declaration_procedure.json | 12 - .../err_duplicate_field_derived_type.f90 | 6 - .../err_duplicate_field_derived_type.json | 12 - .../errors/err_duplicate_parameter.f90 | 4 - .../errors/err_duplicate_parameter.json | 12 - .../errors/err_duplicate_procedure_global.f90 | 7 - .../err_duplicate_procedure_global.json | 12 - .../errors/err_duplicate_procedure_module.f90 | 9 - .../err_duplicate_procedure_module.json | 12 - .../errors/err_duplicate_variable_module.f90 | 4 - .../errors/err_duplicate_variable_module.json | 12 - .../err_implicit_none_undeclared_arg.f90 | 4 - .../err_implicit_none_undeclared_arg.json | 12 - .../err_implicit_none_undeclared_result.f90 | 4 - .../err_implicit_none_undeclared_result.json | 12 - ...err_parameter_without_type_implicit_none.f | 5 - ..._parameter_without_type_implicit_none.json | 12 - .../errors/err_result_shadows_argument.f90 | 3 - .../errors/err_result_shadows_argument.json | 12 - .../errors/err_unknown_function_result.f90 | 4 - .../errors/err_unknown_function_result.json | 12 - .../errors/err_unknown_type_derived_type.f90 | 5 - .../errors/err_unknown_type_derived_type.json | 12 - .../errors/err_unknown_type_module.f90 | 3 - .../errors/err_unknown_type_module.json | 12 - .../errors/err_unknown_type_procedure.f90 | 3 - .../errors/err_unknown_type_procedure.json | 12 - .../fixtures/json_sanity_allowlist.json | 3 - .../parsing/generate_error_goldens.py | 103 - .../parsing/generate_parser_goldens.py | 82 - .../test_declaration_and_interface_edges.py | 434 ++--- .../test_declaration_and_scope_regressions.py | 512 +---- .../test_derived_types_and_program_units.py | 350 ---- .../parsing/test_developer_tutorial.py | 86 - .../parsing/test_error_fixture_suite.py | 109 -- .../parsing/test_error_handling.py | 1153 +++++------ .../parsing/test_fortran_fixture_suite.py | 101 +- ...ortran_parser_procedures_and_interfaces.py | 491 +---- .../parsing/test_fortran_parser_properties.py | 30 +- .../parsing/test_json_sanity.py | 315 --- .../parsing/test_parser_benchmarks.py | 46 - .../parsing/test_public_entrypoints.py | 51 +- ...test_real_world_interaction_regressions.py | 25 - ...source_form_and_diagnostics_regressions.py | 603 +----- .../infrastructure/pipeline/test_wrapper.py | 28 - .../pipeline/test_wrapper_generator.py | 305 +-- .../policy/test_native_array_handles.py | 45 - .../infrastructure/policy/test_ownership.py | 113 +- .../policy/test_wrapper_policy.py | 471 ++--- .../infrastructure/preprocessing/_support.py | 15 - .../infrastructure/preprocessing/test_cli.py | 44 +- .../test_configuration_and_adapters.py | 41 - .../test_dependencies_and_includes.py | 99 +- .../preprocessing/test_execution.py | 31 - .../preprocessing/test_parser_boundaries.py | 215 +- .../test_preprocessing_properties.py | 91 - .../printers/test_source_printers.py | 300 +-- .../runtime/test_native_support.py | 103 +- .../semantics/test_compile_time_values.py | 88 +- .../test_semantic_conversion_smoke.py | 32 +- ...test_semantic_specialization_properties.py | 104 - .../test_declaring_namespace_publication.py | 14 +- .../test_method_and_constructor_contracts.py | 15 +- .../test_authoritative_contract_runtime.py | 65 +- .../parsing/test_python_ast_contracts.py | 56 +- .../native/emit_complex_fem_module.f90 | 30 - .../test_calls_and_policy_metadata.py | 505 ----- .../pipeline/test_classes_and_methods.py | 327 +--- .../pipeline/test_contract_loading.py | 13 - .../test_contract_package_generation.py | 88 +- .../pipeline/test_modern_example.py | 12 - .../pipeline/test_printer_diagnostics.py | 88 + .../test_pyi_printer_conversion_smoke.py | 15 +- .../test_pyi_printer_imports_and_packages.py | 144 +- .../pipeline/test_types_and_declarations.py | 704 ++----- .../semantics/test_calls_and_projections.py | 629 +----- .../semantics/test_imports_and_packages.py | 194 +- .../semantic_pyi/semantics/test_native_abi.py | 23 - .../semantics/test_round_trip_properties.py | 143 +- .../semantics/test_types_and_values.py | 328 +--- .../utilities/test_class_visitor.py | 37 +- .../infrastructure/utilities/test_strings.py | 30 - .../codegen/test_native_handle_planning.py | 345 +--- .../runtime/test_handle_lifecycle.py | 462 +++-- .../test_module_array_view_lowering.py | 28 +- .../test_scalar_module_variable_lowering.py | 140 +- .../end_to_end/test_logical_array_views.py | 18 +- .../test_module_array_storage_forms.py | 108 +- .../test_module_variable_reexport.py | 50 +- .../test_module_variables_and_state.py | 290 +-- .../parsing/test_project_scope_models.py | 232 +-- .../modules/parsing/test_scope_handling.py | 175 +- .../test_generated_module_contracts.py | 12 +- .../policy/test_module_variable_policy.py | 27 - .../fixtures/native/complex_module.f90 | 30 - .../semantics/test_declaration_publication.py | 46 +- .../test_module_contract_semantics.py | 22 +- .../semantics/test_modules_and_imports.py | 157 +- .../semantics/test_reexport_accessibility.py | 1306 ++++--------- .../codegen/test_optional_lowering.py | 58 +- ...test_optional_direct_entrypoint_routing.py | 22 - .../test_generated_optional_contracts.py | 11 +- .../policy/test_optional_policy.py | 169 +- .../test_optional_fortran_semantics.py | 228 +-- .../pointers/codegen/test_pointer_lowering.py | 133 +- .../end_to_end/test_pointer_handles.py | 83 +- ...e_policies_complete_before_ir_lowering.f90 | 1 + .../policy/test_pointer_ownership_policy.py | 548 ++---- .../runtime/test_pointer_contract_handles.py | 209 +- .../runtime/test_pointer_descriptor_abi.py | 211 +- .../runtime/test_pointer_handle_protocol.py | 553 ------ .../semantics/test_pointer_semantics.py | 89 +- .../codegen/test_raw_array_lowering.py | 41 - .../codegen/test_scalar_address_lowering.py | 112 +- .../codegen/test_string_address_lowering.py | 84 +- .../test_invalid_raw_address_contracts.py | 37 +- .../policy/test_raw_address_policy.py | 182 +- .../semantics/test_raw_address_semantics.py | 46 - .../fixtures/native/descriptor_local.f90 | 23 - .../codegen/test_character_array_lowering.py | 21 - .../test_fixed_string_result_lowering.py | 98 - .../codegen/test_fixed_string_writeback.py | 172 +- .../codegen/test_string_input_lowering.py | 246 +-- .../test_assumed_width_character_storage.py | 103 +- .../test_character_array_handle_arguments.py | 13 +- .../test_character_constant_quoting.py | 43 +- .../test_scalar_string_descriptors.py | 10 +- .../test_string_direct_entrypoint_routing.py | 20 - .../test_generated_string_contracts.py | 30 - .../policy/test_string_wrapper_policy.py | 439 +---- .../semantics/test_string_pyi_semantics.py | 22 - .../codegen/test_hidden_scalar_outputs.py | 30 - ..._scalar_subroutine_writeback_validation.py | 89 +- .../end_to_end/test_assumed_scalar_intent.py | 34 +- ...st_subroutine_direct_entrypoint_routing.py | 21 - .../policy/test_subroutine_output_policy.py | 50 +- .../fixtures/native/assumed_intent.f90 | 18 - .../test_subroutine_argument_projection.py | 148 +- tests/tools/test_build_time_benchmark.py | 14 - .../test_check_codegen_complexity_cli.py | 6 +- .../test_check_static_analysis_versions.py | 25 +- .../tools/test_direct_entrypoint_benchmark.py | 30 - tests/tools/test_generate_performance_docs.py | 71 +- tests/tools/test_runtime_benchmark.py | 69 +- .../test_warm_real_library_native_cache.py | 43 +- tests/workflows/test_openmpi_integration.py | 17 +- 358 files changed, 8817 insertions(+), 32548 deletions(-) delete mode 100644 tests/c/functions/codegen/test_positional_only_lowering.py delete mode 100644 tests/c/infrastructure/cli/pipeline/test_c_cli_skeleton.py delete mode 100644 tests/c/infrastructure/parsing/test_c_corpus.py delete mode 100644 tests/c/infrastructure/parsing/test_c_json_sanity.py delete mode 100644 tests/c/infrastructure/preprocessing/test_error_paths.py delete mode 100644 tests/c/infrastructure/preprocessing/test_source_mappings.py delete mode 100644 tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py delete mode 100644 tests/fortran/allocatables/parsing/test_allocatable_declarations.py delete mode 100644 tests/fortran/allocatables/pipeline/test_allocatable_contract_printing.py delete mode 100644 tests/fortran/allocatables/pipeline/test_allocatable_module_contract_printing.py delete mode 100644 tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py delete mode 100644 tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py delete mode 100644 tests/fortran/allocatables/semantics/test_fortran_allocatable_semantics.py delete mode 100644 tests/fortran/arrays/codegen/test_array_output_identity.py delete mode 100644 tests/fortran/arrays/codegen/test_specialized_array_roles.py delete mode 100644 tests/fortran/callbacks/pipeline/test_generated_callback_contracts.py delete mode 100644 tests/fortran/data_types/codegen/test_default_logical_scalar_lowering.py delete mode 100644 tests/fortran/data_types/codegen/test_logical_storage_boundary.py delete mode 100644 tests/fortran/data_types/codegen/test_scalar_boundary_plan.py delete mode 100644 tests/fortran/data_types/pipeline/test_generated_scalar_contract.py delete mode 100644 tests/fortran/data_types/pipeline/test_scalar_generated_pyi_contracts.py delete mode 100644 tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/module_aliased_proxy/__init__.pyi delete mode 100644 tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/module_aliased_proxy/fmodule_derived_alias_f90.pyi delete mode 100644 tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/opaque_boundary/__init__.pyi delete mode 100644 tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/opaque_boundary/fderived_boundary_f90.pyi delete mode 100644 tests/fortran/derived_types/end_to_end/fixtures/native/fborrowed_finalizer_runtime.f90 delete mode 100644 tests/fortran/derived_types/parsing/test_type_bound_diagnostics.py delete mode 100644 tests/fortran/derived_types/pipeline/test_generated_derived_contracts.py delete mode 100644 tests/fortran/derived_types/policy/test_merged_contract_names.py delete mode 100644 tests/fortran/derived_types/semantics/fixtures/native/derived_type.f90 delete mode 100644 tests/fortran/derived_types/semantics/test_fortran_derived_semantics.py delete mode 100644 tests/fortran/enumerations/pipeline/test_generated_enum_contract.py delete mode 100644 tests/fortran/error_handling/codegen/test_runtime_envelope_lowering.py delete mode 100644 tests/fortran/error_handling/pipeline/test_debug_cli_tracebacks.py delete mode 100644 tests/fortran/functions/codegen/test_scalar_function_writeback.py delete mode 100644 tests/fortran/functions/policy/test_function_result_policy.py delete mode 100644 tests/fortran/infrastructure/building/compiling/test_support_probe_artifacts.py delete mode 100644 tests/fortran/infrastructure/building/pipeline/fixtures/generated_contracts/source_builds/fruntime_abi_f90/__init__.pyi delete mode 100644 tests/fortran/infrastructure/building/pipeline/fixtures/generated_contracts/source_builds/fruntime_abi_f90/fruntime_abi_f90.pyi delete mode 100644 tests/fortran/infrastructure/cli/pipeline/_support.py delete mode 100644 tests/fortran/infrastructure/codegen/test_binding.py delete mode 100644 tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_argument_name.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_argument_name.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_declaration_procedure.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_declaration_procedure.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_field_derived_type.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_field_derived_type.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_parameter.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_parameter.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_global.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_global.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_module.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_module.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_variable_module.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_variable_module.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_arg.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_arg.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_result.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_result.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_parameter_without_type_implicit_none.f delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_parameter_without_type_implicit_none.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_result_shadows_argument.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_result_shadows_argument.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_function_result.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_function_result.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_derived_type.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_derived_type.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_module.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_module.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_procedure.f90 delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_procedure.json delete mode 100644 tests/fortran/infrastructure/parsing/fixtures/json_sanity_allowlist.json delete mode 100644 tests/fortran/infrastructure/parsing/generate_error_goldens.py delete mode 100644 tests/fortran/infrastructure/parsing/generate_parser_goldens.py delete mode 100644 tests/fortran/infrastructure/parsing/test_derived_types_and_program_units.py delete mode 100644 tests/fortran/infrastructure/parsing/test_developer_tutorial.py delete mode 100644 tests/fortran/infrastructure/parsing/test_error_fixture_suite.py delete mode 100644 tests/fortran/infrastructure/parsing/test_json_sanity.py delete mode 100644 tests/fortran/infrastructure/parsing/test_parser_benchmarks.py delete mode 100644 tests/fortran/infrastructure/pipeline/test_wrapper.py delete mode 100644 tests/fortran/infrastructure/policy/test_native_array_handles.py delete mode 100644 tests/fortran/infrastructure/preprocessing/test_preprocessing_properties.py delete mode 100644 tests/fortran/infrastructure/semantic_ir/semantics/test_semantic_specialization_properties.py delete mode 100644 tests/fortran/infrastructure/semantic_pyi/pipeline/fixtures/native/emit_complex_fem_module.f90 delete mode 100644 tests/fortran/infrastructure/semantic_pyi/pipeline/test_calls_and_policy_metadata.py create mode 100644 tests/fortran/infrastructure/semantic_pyi/pipeline/test_printer_diagnostics.py delete mode 100644 tests/fortran/infrastructure/utilities/test_strings.py delete mode 100644 tests/fortran/modules/semantics/fixtures/native/complex_module.f90 delete mode 100644 tests/fortran/pointers/runtime/test_pointer_handle_protocol.py delete mode 100644 tests/fortran/strings/codegen/fixtures/native/descriptor_local.f90 delete mode 100644 tests/fortran/strings/pipeline/test_generated_string_contracts.py delete mode 100644 tests/fortran/subroutines/semantics/fixtures/native/assumed_intent.f90 diff --git a/AGENTS.md b/AGENTS.md index df019dac9..37f665aa2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,6 +55,51 @@ When asked to change or move an API, import path, command, feature, or behavior, When updating tests, remove obsolete tests that only assert removed/old implementation behavior does not exist. Do not preserve rejection or absence checks for API/features that were intentionally removed unless explicitly requested. Do not add tests whose purpose is only to prove that removed or nonexistent features are rejected. Test supported behavior and meaningful validation boundaries instead. For example, if `ArrayCategory` is removed, delete its tests; do not add a test asserting that `ArrayCategory` now fails. +Optimize the test suite for maximum confidence per test and minimum +maintenance burden, not for test count. Treat end-to-end tests as the primary +proof that a feature works: where practical, demonstrate a feature through the +real workflow (source, preprocessing, parsing, semantic IR, `.pyi` contract, +replay or build, generated wrapper, compile and link, import, runtime call) and +finish by checking a concrete, repeatable result such as runtime values, native +state, generated contract or source text, or the native build plan. One strong +end-to-end test that covers several cooperating features should replace +several lower-level tests that only repeat pieces of the same behavior. + +Delete a test, rather than preserve it because it exists, when its only +purpose is to check implementation details, trivial getters, constructors, +dataclass fields, or plumbing; to repeat behavior a stronger end-to-end test +already proves; to assert an intermediate object only because it currently +exists; to test a tiny helper that is exercised thoroughly elsewhere; to repeat +one case at several stages; to lock internal architecture without protecting +user-visible behavior; or to add near-identical permutations that do not +represent distinct failure modes. + +Keep a focused isolated test only when it is the cheapest or clearest way to +protect a boundary that end-to-end tests do not cover economically, and when +it has a clear answer to: **what realistic regression does this catch that +would otherwise be difficult, expensive, or ambiguous to detect?** Typical +answers are parser grammar edge cases; preprocessing and source-discovery +rules; semantic transformations with many meaningful combinations; export and +re-export resolution; diagnostics and error locations; contract round trips; +compiler-independent behavior that would otherwise need many native builds; +subtle regressions whose end-to-end failure would not say which rule broke; +and negative validation paths that are cumbersome or unsafe to reproduce +through a full build. If there is no good answer, remove the test. In PRIK, +scrutinize especially tests of parser internals, semantic IR details, +policy and planning intermediates, generated-code string fragments, +source-versus-build route parity, and duplicated source-versus-generated-`.pyi` +assertions; where the two routes are meant to agree, prefer one shared parity +test over the same behavioral assertions in both. + +When fixing a real bug, first ask whether an existing end-to-end test can be +strengthened to cover the regression. If not, add the smallest focused +regression test at the layer where the bug reproduces clearly. Do not add a +unit test merely because production code changed. Before testing a subsystem +in isolation, list the distinct realistic ways it could fail and test those +behavioral boundaries with a small table of meaningful cases instead of +mirroring the implementation line by line. Do not change production behavior +to make a test easier to delete. + Treat tests as evidence for a named invariant, not as specifications merely because they already exist. Add or retain automated tests when they protect at least one of the following: @@ -87,7 +132,10 @@ and the earliest stage that can prove it. Keep the resulting evidence concise: - One test may assert several related consequences of the same setup and invariant. Do not create one test function per field or incidental detail. - Use parametrization when cases exercise the same operation and assertion - shape with different inputs, and give every row a descriptive ID. + shape with different inputs, and give every row a descriptive ID. Keep only + the rows that exercise distinct code paths instead of a full matrix. +- Prefer one end-to-end workflow that exercises several cooperating features + over a separate native build for every small operation. - Do not repeat the same invariant at adjacent stages. Add another stage test only when it protects a real handoff, completed decision, generated artifact, ABI mechanism, or runtime behavior. diff --git a/CHANGELOG.md b/CHANGELOG.md index 9aed10de9..5f997b7f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ release tags add a leading `v` to the package version. ## Unreleased +- The test suite consolidates overlapping checks around compiled workflows and + retains focused parser, semantic, diagnostic, and ABI boundary coverage; + contributor guidance now favors observable behavior over implementation shape. + - Generated bindings pass a wrapped derived-type object about twice as fast: they read its native storage through attribute names interned once instead of building a new name string on every call, and no longer scan the diff --git a/docs/developer/packages/codegen.md b/docs/developer/packages/codegen.md index b73bdcce5..26224f571 100644 --- a/docs/developer/packages/codegen.md +++ b/docs/developer/packages/codegen.md @@ -244,8 +244,8 @@ reaches into its dictionary before the script runs. | Evidence | What it establishes | | --- | --- | -| [Codegen infrastructure](../../../tests/fortran/infrastructure/codegen/test_binding.py) | C binding nodes, module assembly, source-independent lowering, and selected validation paths. | -| [Plan handoff and generated wrappers](../../../tests/fortran/infrastructure/pipeline/test_wrapper_generator.py) | Docstring rendering, plan freezing, cross-backend validation, node generation, and rendered-wrapper assembly. | +| [Codegen infrastructure](../../../tests/fortran/infrastructure/codegen/) | Direct and adapted entrypoint routing, backend facet ownership, package dependency direction, and planner boundaries. | +| [Plan handoff and generated wrappers](../../../tests/fortran/infrastructure/pipeline/test_wrapper_generator.py) | Rendered-wrapper assembly, revalidation of edited plans before lowering, binding/bridge edit ownership, and plan freezing after generation. | | [Primitive scalar lowering](../../../tests/fortran/data_types/codegen/test_primitive_scalar_type_catalogue.py) | Scalar spelling catalogue and exact C, Fortran, NumPy, and result representation selection. | | [Array lowering](../../../tests/fortran/arrays/codegen/test_array_buffer_lowering.py) | Planned buffer handoff and emitted binding/bridge operations. | | [Derived-type lowering](../../../tests/fortran/derived_types/codegen/test_derived_lowering.py) | Plan-selected native object, lifecycle, and bridge/binding mechanisms. | diff --git a/docs/developer/packages/codegen/c-binding.md b/docs/developer/packages/codegen/c-binding.md index e0735c27d..e5a942182 100644 --- a/docs/developer/packages/codegen/c-binding.md +++ b/docs/developer/packages/codegen/c-binding.md @@ -294,9 +294,9 @@ reverse-call entrypoints used by adapter-local callback procedures. | Evidence | What it establishes | | --- | --- | -| [Binding infrastructure](../../../../tests/fortran/infrastructure/codegen/test_binding.py) | Invalid NumPy scalar macros fail at the C binding helper boundary. | +| [Binding facet ownership](../../../../tests/fortran/infrastructure/codegen/test_consumer_facet_boundaries.py) | The binding reads only its own plan facet and consumes planned support inventories instead of re-deriving them. | | [Wrapper-generator handoff](../../../../tests/fortran/infrastructure/pipeline/test_wrapper_generator.py) | Frozen-plan validation and generated C binding, header, and wrapper assembly. | -| [Array lowering](../../../../tests/fortran/arrays/codegen/test_specialized_array_roles.py) | Plan-selected specialized array roles lower through the binding boundary. | +| [Array lowering](../../../../tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py) | Plan-selected specialized array roles lower through the binding boundary. | ## Failure Boundary diff --git a/docs/developer/packages/codegen/fortran-bridge.md b/docs/developer/packages/codegen/fortran-bridge.md index 94fb637e2..ea096bf9a 100644 --- a/docs/developer/packages/codegen/fortran-bridge.md +++ b/docs/developer/packages/codegen/fortran-bridge.md @@ -298,7 +298,7 @@ alias. This gives every imported module procedure a distinct bridge-local name. | --- | --- | | [Wrapper-generator handoff](../../../../tests/fortran/infrastructure/pipeline/test_wrapper_generator.py) | Frozen-plan validation and generated Fortran bridge assembly. | | [Module-variable lowering](../../../../tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py) | Matched C and Fortran scalar module-variable operations and their bridge procedures. | -| [Array lowering](../../../../tests/fortran/arrays/codegen/test_specialized_array_roles.py) | Plan-selected specialized array roles lower through the bridge ABI. | +| [Array lowering](../../../../tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py) | Plan-selected specialized array roles lower through the bridge ABI. | ## Failure Boundary diff --git a/docs/developer/packages/compiler.md b/docs/developer/packages/compiler.md index c145037a9..fe0aba161 100644 --- a/docs/developer/packages/compiler.md +++ b/docs/developer/packages/compiler.md @@ -200,7 +200,7 @@ and conditional support installation. | [Compiler profile and command construction](../../../tests/fortran/infrastructure/building/compiling/test_compiler_verbose.py) | Coherent C/Fortran driver selection, explicit overrides, profile and user-flag order, optional-flag probing, record-only mode, and preserved link-input order. | | [Generated-wrapper build handoff](../../../tests/fortran/infrastructure/building/pipeline/test_generated_wrapper_build.py) | Generated sources, conditional support installation, explicit C and Fortran object requests, and the final ordered link request passed from the pipeline. | | [Source build modes](../../../tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py) | The selected source-build mode produces an importable native extension. | -| [Native-support surface](../../../tests/fortran/infrastructure/runtime/test_native_support.py) | The bundled payload remains header-only and exposes the small native binding API expected by generated sources. | +| [Native-support surface](../../../tests/fortran/infrastructure/runtime/test_native_support.py) | The bundled support header keeps the native-array backend record's layout tag, release lifetime, address-capture linkage, and owner-identity checks. | | [C build integration](../../../tests/c/infrastructure/building/pipeline/test_c_build_cli.py) | C-only builds use the selected C compiler; mixed-language inputs select the required Fortran link driver. | ## Change Routes diff --git a/docs/developer/packages/planning.md b/docs/developer/packages/planning.md index e9f3763ba..666a48ad1 100644 --- a/docs/developer/packages/planning.md +++ b/docs/developer/packages/planning.md @@ -243,7 +243,7 @@ than reconstructed by either backend. | Evidence | What it establishes | | --- | --- | -| [Planner](../../../tests/fortran/infrastructure/codegen/test_planner.py) | Namespace grouping, exports, class lookup, array projection, and failure on missing, empty, or unsupported policy. | +| [Planner](../../../tests/fortran/infrastructure/codegen/test_planner.py) | Namespace grouping, publication ownership, folded-name symbols, entrypoint call order, and failure on missing, empty, or unsupported policy. | | [Plan ownership and freezing](../../../tests/fortran/infrastructure/pipeline/test_wrapper_generator.py) | A plan is editable before generation; the generator validates and freezes it before backend and printer use. | | [Overload dispatch plans](../../../tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py) | Candidate order, matching, and pre-call dispatch for supported generic interfaces. | | [Native handle planning](../../../tests/fortran/memory_management/codegen/test_native_handle_planning.py) | Descriptor-handle state, operation sets, required headers, and central plan validation. | diff --git a/docs/developer/packages/printers.md b/docs/developer/packages/printers.md index 3b79c1c45..cc9c3d818 100644 --- a/docs/developer/packages/printers.md +++ b/docs/developer/packages/printers.md @@ -171,7 +171,7 @@ publishes. | Evidence | What it establishes | | --- | --- | -| [Native source printers](../../../tests/fortran/infrastructure/printers/test_source_printers.py) | C and Fortran serialization, rejection of wrapper plans, line wrapping, literal preservation, and unsplittable-line diagnostics. | +| [Native source printers](../../../tests/fortran/infrastructure/printers/test_source_printers.py) | Fortran line wrapping at safe continuation points, literal preservation, and unsplittable-line diagnostics. | | [Semantic `.pyi` conversion smoke](../../../tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_conversion_smoke.py) | Emitted contract fixtures can be parsed and converted through the normal semantic-`.pyi` route. | | [`.pyi` imports and packages](../../../tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py) | Isolated emission state, imports, aliases, packages, name collisions, and opaque dependencies. | diff --git a/docs/developer/packages/runtime.md b/docs/developer/packages/runtime.md index ed06c0b1f..ef8568018 100644 --- a/docs/developer/packages/runtime.md +++ b/docs/developer/packages/runtime.md @@ -91,10 +91,10 @@ resizes it. The compiler installs the native header into a generated | Evidence | What it establishes | | --- | --- | -| [Allocatable runtime tests](../../../tests/fortran/allocatables/runtime/) | Allocation state, operations, descriptor handoffs, and NumPy views. | +| [Allocatable handle tests](../../../tests/fortran/allocatables/end_to_end/test_allocatable_handles.py) | Allocation state, operations, descriptor handoffs, and NumPy views. | | [Pointer runtime tests](../../../tests/fortran/pointers/runtime/) | Association, nullification, pointer descriptors, and views. | | [Memory-management runtime tests](../../../tests/fortran/memory_management/runtime/) | Owner retention, release, and array handoffs. | -| [Native-support tests](../../../tests/fortran/infrastructure/runtime/) | Bundled payload discovery and installation inputs. | +| [Native-support tests](../../../tests/fortran/infrastructure/runtime/) | Native-array backend record layout, release lifetime, and owner-identity checks in the bundled payload. | | [Compiled runtime compatibility](../../../tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py) | The payload and Python runtime working through a real extension. | An outstanding zero-copy NumPy view cannot be revoked after native diff --git a/docs/developer/packages/semantics.md b/docs/developer/packages/semantics.md index b3ee4ae31..f2b7f8bea 100644 --- a/docs/developer/packages/semantics.md +++ b/docs/developer/packages/semantics.md @@ -311,7 +311,6 @@ before policy completion or any backend lowering begins. | [C semantic IR conversion](../../../tests/c/infrastructure/semantic_ir/semantics/) | C-model conversion, exact target identities, export selection, and semantic graph properties. | | [Fortran datatype semantics](../../../tests/fortran/data_types/semantics/) | Stable scalar identities, storage facts, and compiler-measurement handoffs. | | [Semantic `.pyi` conversion](../../../tests/fortran/infrastructure/semantic_pyi/semantics/) | Contract constructs, imports, external references, projections, classes, overloads, and round trips. | -| [Native array handles](../../../tests/fortran/infrastructure/policy/test_native_array_handles.py) | Descriptor marking and separation of handle, data, and element facts. | | [Native contract validation](../../../tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py) | Native-contract preparation, validation, and diagnostic ownership. | ## Change Routes diff --git a/docs/developer/packages/utilities.md b/docs/developer/packages/utilities.md index 4161340ea..14741cae5 100644 --- a/docs/developer/packages/utilities.md +++ b/docs/developer/packages/utilities.md @@ -123,9 +123,9 @@ MRO fallback: expression:Expression | Evidence | What it establishes | | --- | --- | -| [Utility tests](../../../tests/fortran/infrastructure/utilities/) | Local-name allocation and generic visitor dispatch. | +| [Utility tests](../../../tests/fortran/infrastructure/utilities/) | Generic visitor dispatch. Local-name allocation is exercised through the [naming tests](../../../tests/fortran/infrastructure/naming/). | | [Declaration-expression tests](../../../tests/fortran/arrays/semantics/test_declaration_expression_utilities.py) | Translation, validation, role resolution, evaluation, and rendering. | -| [Wrapper freeze-boundary tests](../../../tests/fortran/infrastructure/pipeline/test_wrapper_generator.py) | Plans and generated nodes reject mutation after consumption. | +| [Wrapper freeze-boundary tests](../../../tests/fortran/infrastructure/pipeline/test_wrapper_generator.py) | A wrapper plan rejects mutation once the generator has consumed it. | Move a helper out of `utilities/` as soon as it starts selecting semantic policy, emitted mechanisms, or a pipeline action. diff --git a/docs/user/language-support/feature-matrix.md b/docs/user/language-support/feature-matrix.md index 12bab1cbc..4dc18877c 100644 --- a/docs/user/language-support/feature-matrix.md +++ b/docs/user/language-support/feature-matrix.md @@ -125,7 +125,7 @@ documented diagnostic-stage exception below. | Feature | Status | User docs | Evidence | Limitations | | --- | --- | --- | --- | --- | -| Unproved pointer lifetime and ownership-changing operations | Unsupported | [Pointer safety](../guide/pointers.md#safety-checklist) | [Pointer policy tests](../../../tests/fortran/pointers/policy/test_pointer_ownership_policy.py), [pointer runtime tests](../../../tests/fortran/pointers/runtime/test_pointer_handle_protocol.py) | Native targets must outlive every handle use; allocation, target deallocation, resize, and writable reassociation require explicit completed policy. | +| Unproved pointer lifetime and ownership-changing operations | Unsupported | [Pointer safety](../guide/pointers.md#safety-checklist) | [Pointer policy tests](../../../tests/fortran/pointers/policy/test_pointer_ownership_policy.py), [pointer handle tests](../../../tests/fortran/pointers/end_to_end/test_pointer_handles.py) | Native targets must outlive every handle use; allocation, target deallocation, resize, and writable reassociation require explicit completed policy. | | Persistent callbacks and procedure pointers | Unsupported | [Callback limitations](../guide/callbacks.md#important-limitations) | [Callback policy tests](../../../tests/fortran/callbacks/policy/test_callback_policy.py), [scalar callback tests](../../../tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py) | Callbacks are valid only during the wrapped call. | | Advanced multi-source dependency discovery and external-library integration | Unsupported | [Multiple source files](../guide/building-shared-library.md#multiple-source-files) | [Multi-source tests](../../../tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py) | PRIK does not discover sources you did not name, prebuilt module search paths, or external libraries. Dependency ordering among the sources it parses is supported. | | Blocked array forms | Unsupported | [Arrays](../guide/arrays.md) | [Array semantic tests](../../../tests/fortran/arrays/semantics/test_array_semantics.py), [diagnostics](../reference/diagnostic-codes.md) | Arrays of derived types and character arrays not representable as fixed-width bytes need missing runtime contracts. | diff --git a/tests/c/data_types/probes/test_c_types.py b/tests/c/data_types/probes/test_c_types.py index 45872e9d6..cd25e7f80 100644 --- a/tests/c/data_types/probes/test_c_types.py +++ b/tests/c/data_types/probes/test_c_types.py @@ -32,29 +32,6 @@ def _required_c_compiler() -> str: return _CC -def test_c_standard_type_probe_source_queries_standard_headers_without_layout_claims(): - source = build_c_standard_type_probe_source() - - assert "#include " in source - assert "#include " in source - assert "#include " in source - assert "#include " in source - assert "#include " in source - assert "#include " in source - assert 'PRIK_PRINT_ARITHMETIC("_Bool"' in source - assert "PRIK_PRINT_CHAR()" in source - assert 'PRIK_PRINT_ARITHMETIC("unsigned long"' in source - assert 'PRIK_PRINT_REAL("long double"' in source - assert 'PRIK_PRINT_COMPLEX("long double _Complex"' in source - assert 'PRIK_PRINT_ARITHMETIC("int"' in source - assert 'PRIK_PRINT_ARITHMETIC("size_t"' in source - assert 'PRIK_PRINT_ARITHMETIC("int64_t"' in source - assert 'PRIK_PRINT_ARITHMETIC("uint32_t"' in source - assert 'PRIK_PRINT_ARITHMETIC("time_t"' in source - assert "sizeof(FILE *)" in source - assert "sizeof(FILE)" not in source - - def test_c_standard_type_probe_requires_an_explicit_compiler(): with pytest.raises(CStandardTypeProbeError, match="exact compiler"): probe_c_standard_types(PreprocessingConfig(mode="compiler")) diff --git a/tests/c/data_types/semantics/test_types_and_constants.py b/tests/c/data_types/semantics/test_types_and_constants.py index 1c21f1c04..15118c567 100644 --- a/tests/c/data_types/semantics/test_types_and_constants.py +++ b/tests/c/data_types/semantics/test_types_and_constants.py @@ -1,7 +1,6 @@ """Tests split by stable ownership concept from `test_functions_and_callbacks.py`.""" from dataclasses import asdict -from typing import ClassVar import pytest @@ -69,68 +68,16 @@ def test_c2ir_maps_primitive_pointer_parameters_to_runtime_rank_storage_contract assert src.semantic_type.name == "Float64" assert src.semantic_type.storage.kind == "array" assert src.semantic_type.storage.read_only is True + assert src.semantic_type.storage.pointer_depth == 1 + assert src.semantic_type.storage.array.category == "runtime_rank" + assert src.semantic_type.storage.array.shape == ["..."] + assert src.semantic_type.storage.array.order == "ORDER_C" assert dst.semantic_type.name == "Float64" assert dst.semantic_type.storage.kind == "array" assert dst.semantic_type.storage.read_only is False assert asdict(src.semantic_type.ownership) == {"ownership": "borrowed", "mutable": False, "aliasing": True} assert asdict(dst.semantic_type.ownership) == {"ownership": "borrowed", "mutable": True, "aliasing": True} - assert asdict(src.semantic_type.storage) == { - "kind": "array", - "read_only": True, - "mutable": False, - "pointer_depth": 1, - "ownership": "borrowed", - "array": { - "rank": 1, - "shape": ["..."], - "lower_bounds": [], - "upper_bounds": [], - "source_shape": ["..."], - "category": "runtime_rank", - "order": "ORDER_C", - "copy_order": None, - "axes": ["dense"], - "contiguous": None, - "allocatable": False, - "pointer": False, - "metadata": {}, - }, - "calling_convention": None, - "metadata": { - "c_pointer_qualifiers": [[]], - "restrict": False, - "source_type": "const double *src", - }, - } - assert asdict(dst.semantic_type.storage) == { - "kind": "array", - "read_only": False, - "mutable": True, - "pointer_depth": 1, - "ownership": "borrowed", - "array": { - "rank": 1, - "shape": ["..."], - "lower_bounds": [], - "upper_bounds": [], - "source_shape": ["..."], - "category": "runtime_rank", - "order": "ORDER_C", - "copy_order": None, - "axes": ["dense"], - "contiguous": None, - "allocatable": False, - "pointer": False, - "metadata": {}, - }, - "calling_convention": None, - "metadata": { - "c_pointer_qualifiers": [[]], - "restrict": False, - "source_type": "double *dst", - }, - } restricted = CToIRConverter().visit( CComposedType( components=[CPointer(qualifiers=[CRestrict()]), CDouble()], @@ -166,62 +113,9 @@ def test_c2ir_uses_declared_c_array_bounds_before_parameter_adjustment(): assert matrix.semantic_type.shape == ["3", "4"] assert matrix.semantic_type.storage.array.shape == ["3", "4"] assert matrix.semantic_type.storage.array.order == "ORDER_C" - assert asdict(a.semantic_type.storage) == { - "kind": "array", - "read_only": False, - "mutable": True, - "pointer_depth": 1, - "ownership": "borrowed", - "array": { - "rank": 1, - "shape": ["4"], - "lower_bounds": [], - "upper_bounds": [], - "source_shape": ["4"], - "category": "c_array", - "order": None, - "copy_order": None, - "axes": ["dense"], - "contiguous": True, - "allocatable": False, - "pointer": False, - "metadata": { - "c_static_minimum": [True], - "c_variable_length": [False], - "c_flexible": [False], - }, - }, - "calling_convention": None, - "metadata": {"source_type": "double a[static 4]"}, - } - assert asdict(matrix.semantic_type.storage) == { - "kind": "array", - "read_only": False, - "mutable": True, - "pointer_depth": 1, - "ownership": "borrowed", - "array": { - "rank": 2, - "shape": ["3", "4"], - "lower_bounds": [], - "upper_bounds": [], - "source_shape": ["3", "4"], - "category": "c_array", - "order": "ORDER_C", - "copy_order": None, - "axes": ["dense", "dense"], - "contiguous": True, - "allocatable": False, - "pointer": False, - "metadata": { - "c_static_minimum": [False, False], - "c_variable_length": [False, False], - "c_flexible": [False, False], - }, - }, - "calling_convention": None, - "metadata": {"source_type": "int matrix[3][4]"}, - } + assert a.semantic_type.storage.array.category == "c_array" + assert a.semantic_type.storage.array.contiguous is True + assert matrix.semantic_type.storage.array.metadata["c_static_minimum"] == [False, False] def test_c2ir_converts_integer_expression_macro_constants_when_resolvable(): @@ -315,26 +209,6 @@ def test_c2ir_resolves_local_typedef_chains_and_standard_size_t_fallback(): assert cyclic.metadata == {} -def test_c2ir_uses_standard_type_probe_facts_when_supplied(): - parsed = parse_c_file("size_t count(void);\n", filename="probe.h") - converter = CToIRConverter( - standard_type_report={ - "types": { - "size_t": { - "available": True, - "kind": "integer", - "signed": False, - "bits": 32, - } - } - } - ) - - module = converter.visit(parsed) - - assert _function(module, "count").return_type.name == "UInt32" - - def test_c2ir_preserves_c_int_identity_and_stores_compiler_probed_precision(): converter = CToIRConverter( standard_type_report={ @@ -473,17 +347,26 @@ def test_c_primitive_precisions_map_to_semantic_types(ctype, expected_name, expe @pytest.mark.parametrize( ("name", "fact", "expected"), [ - ("int8_t", {"available": True, "kind": "integer", "signed": True, "bits": 8}, "Int8"), - ("int16_t", {"available": True, "kind": "integer", "signed": True, "bits": 16}, "Int16"), - ("int32_t", {"available": True, "kind": "integer", "signed": True, "bits": 32}, "Int32"), - ("int64_t", {"available": True, "kind": "integer", "signed": True, "bits": 64}, "Int64"), - ("uint8_t", {"available": True, "kind": "integer", "signed": False, "bits": 8}, "UInt8"), - ("uint16_t", {"available": True, "kind": "integer", "signed": False, "bits": 16}, "UInt16"), - ("uint32_t", {"available": True, "kind": "integer", "signed": False, "bits": 32}, "UInt32"), - ("uint64_t", {"available": True, "kind": "integer", "signed": False, "bits": 64}, "UInt64"), + pytest.param("int8_t", {"available": True, "kind": "integer", "signed": True, "bits": 8}, "Int8", id="signed"), + pytest.param( + "uint64_t", {"available": True, "kind": "integer", "signed": False, "bits": 64}, "UInt64", id="unsigned" + ), + pytest.param( + "size_t", + {"available": True, "kind": "integer", "signed": False, "bits": 32}, + "UInt32", + id="probe-replaces-size-t-fallback", + ), + pytest.param("real_size", {"kind": "real", "bits": 32}, "Float32", id="real"), + pytest.param( + "missing", + {"available": False, "kind": "integer", "signed": False, "bits": 32}, + "missing", + id="unavailable-stays-unresolved", + ), ], ) -def test_c_standard_integer_precision_facts_map_to_semantic_types(name, fact, expected): +def test_c_standard_type_facts_map_typedefs_to_semantic_types(name, fact, expected): semantic_type = CToIRConverter(standard_type_report={"types": {name: fact}}).visit(CTypedef(name=name)) assert semantic_type.name == expected @@ -575,49 +458,8 @@ def test_c2ir_reports_unsupported_type_and_declarator_compositions(): ) -def test_c2ir_standard_type_facts_and_numeric_constant_edge_cases(): - class Report: - types: ClassVar = { - "signed_size": {"kind": "integer", "signed": True, "bits": 16}, - "real_size": {"kind": "real", "bits": 32}, - "missing": {"available": False, "kind": "integer", "signed": False, "bits": 32}, - } - - converter = CToIRConverter(standard_type_report=Report()) - signed_size = converter._standard_semantic_type("signed_size") - real_size = converter._standard_semantic_type("real_size") - assert signed_size.name == "Int16" - assert signed_size.dtype == "Int16" - assert signed_size.metadata == { - "c_standard_type": "signed_size", - "c_standard_type_fact": {"kind": "integer", "signed": True, "bits": 16}, - } - assert real_size.name == "Float32" - assert real_size.dtype == "Float32" - assert real_size.metadata == { - "c_standard_type": "real_size", - "c_standard_type_fact": {"kind": "real", "bits": 32}, - } - fallback = CToIRConverter()._standard_semantic_type("size_t") - assert fallback.name == "SizeT" - assert fallback.dtype == "SizeT" - assert fallback.metadata == {"c_standard_type": "size_t", "c_standard_type_fallback": True} - assert converter._standard_semantic_type("missing") is None - assert converter._standard_semantic_type("not_standard") is None - opaque_converter = CToIRConverter( - standard_type_report={ - "implicit_handle": {"kind": "opaque_handle"}, - "missing_handle": {"available": False, "kind": "opaque_handle"}, - } - ) - assert opaque_converter._standard_semantic_type("implicit_handle").name == "implicit_handle" - assert opaque_converter._standard_semantic_type("missing_handle") is None - assert CToIRConverter._standard_type_facts(object()) == {} - assert CToIRConverter._integer_literal_value(None) is None - assert CToIRConverter._integer_literal_value("value") is None - assert CToIRConverter._integer_macro_expression("(MISSING + 1)", {}) is False - assert CToIRConverter._integer_macro_expression("(1 +)", {}) is False - +def test_c2ir_numeric_constant_edge_cases(): + converter = CToIRConverter() parsed = parse_c_file( "enum default_status { DEFAULT_OK, DEFAULT_NEXT };\nenum status { STATUS_EXPR = UNKNOWN, STATUS_NEXT };\n", filename="edge_constants.h", @@ -625,9 +467,13 @@ class Report: parsed.macros = [ CMacro(name="RATE", value="1.5"), CMacro(name="BAD", value="(MISSING + 1)"), + CMacro(name="MALFORMED", value="(1 +)"), ] constants = {variable.name: variable for variable in converter.visit(parsed).variables} assert constants["RATE"].semantic_type.name == "Float64" + # An expression that does not evaluate to an integer is not published as one. + assert "BAD" not in constants + assert "MALFORMED" not in constants assert constants["DEFAULT_OK"].default_value == "0" assert constants["DEFAULT_NEXT"].default_value == "1" assert constants["STATUS_EXPR"].default_value == "UNKNOWN" diff --git a/tests/c/enumerations/semantics/test_c_enum_semantics.py b/tests/c/enumerations/semantics/test_c_enum_semantics.py index 123251bfe..4ef1dfd2c 100644 --- a/tests/c/enumerations/semantics/test_c_enum_semantics.py +++ b/tests/c/enumerations/semantics/test_c_enum_semantics.py @@ -1,7 +1,5 @@ """C enum conversion into the semantic IR.""" -from dataclasses import asdict - from prik.printers import emit_module from prik.parsers.c import parse_c_file, parse_c_project from prik.parsers.c.models import ( @@ -14,9 +12,6 @@ c_file_to_semantic_modules, c_project_to_semantic_modules, ) -from prik.semantics.models import ( - SemanticVariable, -) from tests.c._support.semantic_conversion import ( _assert_c_origin, _function, @@ -39,12 +34,8 @@ def test_c2ir_converts_enum_constants_and_simple_macro_constants(): assert constants["STATUS_WARN"].default_value == "1" assert constants["STATUS_ERROR"].default_value == "10" api_version = constants["API_VERSION"] - assert isinstance(api_version, SemanticVariable) assert api_version.semantic_type.name == "Int32" assert api_version.semantic_type.dtype == "Int32" - assert [asdict(constraint) for constraint in api_version.semantic_type.constraints] == [ - {"name": "Constant", "arguments": []} - ] _assert_c_origin( api_version.origin, native_name="API_VERSION", diff --git a/tests/c/functions/codegen/test_positional_only_lowering.py b/tests/c/functions/codegen/test_positional_only_lowering.py deleted file mode 100644 index 05e0bcea5..000000000 --- a/tests/c/functions/codegen/test_positional_only_lowering.py +++ /dev/null @@ -1,41 +0,0 @@ -"""A positional-only binding parses its call tuple and installs no keyword table.""" - -from prik.parsers.c import parse_c_file -from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import WrapperPlanner -from prik.policy.completion import complete_semantic_policies -from prik.semantics.c2ir import c_file_to_semantic_module - -# Reserved parameter spellings are exactly what a real system header supplies. -_SOURCE = "double blend(double __x, double __y) { return __x + __y; }\n" - - -def _binding(**options) -> str: - module = c_file_to_semantic_module(parse_c_file(_SOURCE, filename="surface.c")) - complete_semantic_policies(module, **options) - generated = WrapperGenerator().generate(WrapperPlanner().build(module)) - return next(source.text for source in generated.sources if source.path.suffix == ".c") - - -def test_a_positional_only_binding_takes_no_keyword_dictionary(): - binding = _binding(positional_only=True) - - assert "static PyObject * wrap_blend(PyObject * self, PyObject * args) {" in binding - assert 'if (!PyArg_ParseTuple(args, "OO", &bound_arg0_obj, &bound_arg1_obj)) return NULL' in binding - assert "kwlist" not in binding - assert "METH_KEYWORDS" not in binding - - # The native declaration keeps the header's spelling; the Python surface does not. - assert "double blend(double __x, double __y);" in binding - assert "blend(arg0, arg1) -> float64" in binding - # Errors name the argument the way Python does. - assert '"arg0", &bound_arg0' in binding - assert "__x" not in binding.split("static PyObject * wrap_blend")[1] - - -def test_the_default_binding_still_accepts_keywords_under_the_declared_names(): - binding = _binding() - - assert "static PyObject * wrap_blend(PyObject * self, PyObject * args, PyObject * kwargs) {" in binding - assert 'static char * kwlist[] = {"__x", "__y", NULL};' in binding - assert "METH_VARARGS | METH_KEYWORDS" in binding diff --git a/tests/c/functions/end_to_end/test_direct_c_hidden_native_outputs.py b/tests/c/functions/end_to_end/test_direct_c_hidden_native_outputs.py index a2b1bf085..6fb752560 100644 --- a/tests/c/functions/end_to_end/test_direct_c_hidden_native_outputs.py +++ b/tests/c/functions/end_to_end/test_direct_c_hidden_native_outputs.py @@ -2,10 +2,10 @@ A hidden slot is passed to the native call like any other output, but it is not a Python result, so the return annotation states exactly what the caller gets. +Every output form shares one module-scoped build. """ import shutil -from pathlib import Path import numpy as np import pytest @@ -13,6 +13,9 @@ from prik import build_pyi_extension from tests.c._support.runtime import sole_native_module + +pytestmark = pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") + SOURCE = """void tally(int n, int *doubled, int *squared) { *doubled = n * 2; *squared = n * n; @@ -31,89 +34,19 @@ } """ - -def _build(tmp_path: Path, contract: str, name: str): - (tmp_path / f"{name}.pyi").write_text(contract, encoding="utf-8") - (tmp_path / f"{name}.c").write_text(SOURCE, encoding="utf-8") - return build_pyi_extension( - tmp_path / f"{name}.pyi", - native_language="c", - native_c_sources=[tmp_path / f"{name}.c"], - output_dir=tmp_path / f"build_{name}", - output_name=name, - ) - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_hidden_outputs_reach_the_native_call_without_becoming_results(tmp_path: Path): - """Every hidden slot is passed by address; none of them is returned.""" - result = _build( - tmp_path, - """from prik.contracts import Arg, Hidden, Int32, bind, native_call +CONTRACT = """from prik.contracts import Arg, Hidden, Int, Int32, Return, Returns, bind, native_call @bind("tally") @native_call([Arg(0), Hidden("doubled", Int32), Hidden("squared", Int32)]) -def tally(n: Int32) -> None: ... -""", - "all_hidden", - ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") - - assert "void tally(int32_t n, int32_t * doubled, int32_t * squared);" in binding - assert module.tally(np.int32(5)) is None - assert module.tally.__doc__.splitlines()[0] == "tally(n) -> None" - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_target_c_int_hidden_output_keeps_int_pointer_abi(tmp_path: Path): - """A projected ``Int`` output resolves storage without losing C identity.""" - result = _build( - tmp_path, - """from prik.contracts import Arg, Int, Return, Returns, native_call - -@native_call([Arg(0), Return("copy", 1)]) -def split_target_int(value: Int) -> tuple[Int, Returns["copy", Int]]: ... -""", - "target_int_output", - ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") - - assert "int split_target_int(int value, int * copy);" in binding - assert module.split_target_int(np.intc(7)) == (np.intc(0), np.intc(7)) - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_hidden_and_returned_outputs_share_one_native_call(tmp_path: Path): - """``Returns`` comes back and ``Hidden`` does not, from the same call.""" - result = _build( - tmp_path, - """from prik.contracts import Arg, Hidden, Int32, Return, Returns, bind, native_call +def tally_hidden(n: Int32) -> None: ... @bind("tally") @native_call([Arg(0), Return("doubled", 0), Hidden("squared", Int32)]) -def tally(n: Int32) -> Returns["doubled", Int32]: ... -""", - "mixed_hidden", - ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") +def tally_mixed(n: Int32) -> Returns["doubled", Int32]: ... - # Both outputs still cross the boundary; only one is a Python result. - assert "void tally(int32_t n, int32_t * doubled, int32_t * squared);" in binding - assert module.tally(np.int32(5)) == np.int32(10) - assert module.tally.__doc__.splitlines()[0] == "tally(n) -> int32" - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_four_returned_outputs_compile_and_use_shared_failure_cleanup(tmp_path: Path): - """A linear cleanup suffix preserves the successful four-result surface.""" - result = _build( - tmp_path, - """from prik.contracts import Arg, Int32, Return, Returns, bind, native_call +@native_call([Arg(0), Return("copy", 1)]) +def split_target_int(value: Int) -> tuple[Int, Returns["copy", Int]]: ... -@bind("split_four") @native_call([ Arg(0), Return("doubled", 0), @@ -127,11 +60,48 @@ def split_four(n: Int32) -> tuple[ Returns["quadrupled", Int32], Returns["quintupled", Int32], ]: ... -""", - "four_returned", +""" + + +@pytest.fixture(scope="module") +def outputs_build(tmp_path_factory): + tmp_path = tmp_path_factory.mktemp("c_hidden_outputs") + (tmp_path / "outputs.pyi").write_text(CONTRACT, encoding="utf-8") + (tmp_path / "outputs.c").write_text(SOURCE, encoding="utf-8") + result = build_pyi_extension( + tmp_path / "outputs.pyi", + native_language="c", + native_c_sources=[tmp_path / "outputs.c"], + output_dir=tmp_path / "build", + output_name="outputs", ) - module = sole_native_module(result.import_module()) binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + return binding, sole_native_module(result.import_module()) + + +def test_hidden_and_returned_outputs_reach_the_native_call(outputs_build): + """Every hidden slot is passed by address; only ``Returns`` comes back.""" + binding, module = outputs_build + + assert "void tally(int32_t n, int32_t * doubled, int32_t * squared);" in binding + assert module.tally_hidden(np.int32(5)) is None + assert module.tally_hidden.__doc__.splitlines()[0] == "tally_hidden(n) -> None" + # Both outputs still cross the boundary; only one is a Python result. + assert module.tally_mixed(np.int32(5)) == np.int32(10) + assert module.tally_mixed.__doc__.splitlines()[0] == "tally_mixed(n) -> int32" + + +def test_target_c_int_hidden_output_keeps_int_pointer_abi(outputs_build): + """A projected ``Int`` output resolves storage without losing C identity.""" + binding, module = outputs_build + + assert "int split_target_int(int value, int * copy);" in binding + assert module.split_target_int(np.intc(7)) == (np.intc(0), np.intc(7)) + + +def test_four_returned_outputs_compile_and_use_shared_failure_cleanup(outputs_build): + """A linear cleanup suffix preserves the successful four-result surface.""" + binding, module = outputs_build assert module.split_four(np.int32(5)) == tuple(np.int32(value) for value in (10, 15, 20, 25)) assert "goto prik_output_cleanup_4;" in binding diff --git a/tests/c/functions/end_to_end/test_export_symbol_workflow.py b/tests/c/functions/end_to_end/test_export_symbol_workflow.py index b8fb6931f..3ac5455a9 100644 --- a/tests/c/functions/end_to_end/test_export_symbol_workflow.py +++ b/tests/c/functions/end_to_end/test_export_symbol_workflow.py @@ -71,25 +71,13 @@ def _generate_contract(probe: Path, exports: Path, contract: Path) -> None: ) -def test_generate_pyi_selects_one_function_from_a_private_include(tmp_path: Path): - _header, probe, _implementation = _write_private_include_project(tmp_path) - exports = tmp_path / "exports.txt" - exports.write_text("# reviewed public surface\nincrement\n", encoding="utf-8") - contract = tmp_path / "api.pyi" - - _generate_contract(probe, exports, contract) - - text = contract.read_text(encoding="utf-8") - assert "def increment(" in text - assert "omitted" not in text - assert "private_state" not in text - assert _stated_exports(contract) == ["increment"] - - -def test_generated_all_states_the_selected_declarations_not_the_allowlist_lines(tmp_path: Path): +def test_generate_pyi_selects_functions_from_a_private_include(tmp_path: Path): """The allowlist selects declarations; __all__ states the Python names they publish.""" header = tmp_path / "reviewed_api.h" - header.write_text("int zulu(int __v);\nint alpha(int __v);\nint omitted(int __v);\n", encoding="utf-8") + header.write_text( + "extern int private_state;\nint zulu(int __v);\nint alpha(int __v);\nint omitted(int __v);\n", + encoding="utf-8", + ) probe = tmp_path / "probe.c" probe.write_text('#include "reviewed_api.h"\n', encoding="utf-8") exports = tmp_path / "exports.txt" @@ -98,25 +86,15 @@ def test_generated_all_states_the_selected_declarations_not_the_allowlist_lines( _generate_contract(probe, exports, contract) + text = contract.read_text(encoding="utf-8") + assert "def zulu(" in text + assert "def alpha(" in text + assert "omitted" not in text + assert "private_state" not in text # Declaration order, not allowlist order: the list follows the declarations # the selection kept, so comments, blank lines, and the file's own ordering # never reach it. assert _stated_exports(contract) == ["zulu", "alpha"] - assert "omitted" not in contract.read_text(encoding="utf-8") - - -def test_a_repeated_allowlist_name_is_rejected_rather_than_stated_twice(tmp_path: Path): - """A stated surface names each declaration once, so a repeat is a request error.""" - _header, probe, _implementation = _write_private_include_project(tmp_path) - exports = tmp_path / "exports.txt" - exports.write_text("increment\nincrement\n", encoding="utf-8") - contract = tmp_path / "api.pyi" - - with pytest.raises(subprocess.CalledProcessError) as exc_info: - _generate_contract(probe, exports, contract) - - assert "Repeated C function name in --export-symbols" in exc_info.value.stderr - assert not contract.exists() def test_source_build_reuses_selection_with_positional_and_collision_policies(tmp_path: Path): @@ -141,6 +119,8 @@ def test_source_build_reuses_selection_with_positional_and_collision_policies(tm module = sole_native_module(result.import_module()) assert module.increment(np.int32(4)) == np.int32(5) + # Positional-only publishes generated names, never the header's reserved ``__value``. + assert module.increment.__doc__.splitlines()[0] == "increment(arg0) -> int32" with pytest.raises(TypeError, match="keyword"): module.increment(arg0=np.int32(4)) assert {name for name in dir(module) if not name.startswith("_")} == {"increment"} diff --git a/tests/c/functions/end_to_end/test_public_name_contract.py b/tests/c/functions/end_to_end/test_public_name_contract.py index 3ce4dccd9..bb8cd2b21 100644 --- a/tests/c/functions/end_to_end/test_public_name_contract.py +++ b/tests/c/functions/end_to_end/test_public_name_contract.py @@ -54,66 +54,33 @@ def _stated_exports(contract: Path) -> list[str]: def test_a_contract_names_what_its_own_build_published(tmp_path: Path): - """One naming decision reaches both, so the stub is readable and accurate.""" + """One naming decision reaches the module and its readable, parseable stub.""" module, contract = _build( tmp_path, - "keyword_api", - "int lambda(int value);\nint lambda_(int value);\nint ordinary(int value);\n", + "public_names_api", + "int lambda(int value);\nint lambda_(int value);\nint ordinary(int value);\n" + "int Foo(int value);\nint foo(int value);\nint BarBaz(int value);\n", "int lambda(int v) { return v + 1; }\n" "int lambda_(int v) { return v + 2; }\n" - "int ordinary(int v) { return v + 3; }\n", - ["lambda", "lambda_", "ordinary"], + "int ordinary(int v) { return v + 3; }\n" + "int Foo(int v) { return v + 10; }\n" + "int foo(int v) { return v + 20; }\n" + "int BarBaz(int v) { return v + 30; }\n", + ["lambda", "lambda_", "ordinary", "Foo", "foo", "BarBaz"], ) + # ``_stated_exports`` parses the stub, so the contract is readable Python. published = {name for name in dir(module) if not name.startswith("_")} assert published == set(_stated_exports(contract)) # A name Python cannot bind is moved aside once, for the module and the # contract alike, and the C spelling is recorded rather than lost. - assert published == {"lambda_", "lambda__2", "ordinary"} + assert published == {"lambda_", "lambda__2", "ordinary", "Foo", "foo", "BarBaz"} text = contract.read_text(encoding="utf-8") assert '@bind("lambda")' in text assert '@bind("lambda_")' in text - - -def test_a_generated_contract_is_readable_python(tmp_path: Path): - """A contract exists to be re-read and edited, so it has to parse.""" - _module, contract = _build( - tmp_path, - "readable_api", - "int lambda(int value);\n", - "int lambda(int v) { return v + 1; }\n", - ["lambda"], - ) - - ast.parse(contract.read_text(encoding="utf-8"), filename=str(contract)) - - -def test_c_declarations_that_differ_only_in_case_stay_apart(tmp_path: Path): - """C spells its declarations exactly, so two spellings are two functions.""" - module, contract = _build( - tmp_path, - "case_api", - "int Foo(int value);\nint foo(int value);\n", - "int Foo(int v) { return v + 1; }\nint foo(int v) { return v + 2; }\n", - ["Foo", "foo"], - ) - - assert set(_stated_exports(contract)) == {"Foo", "foo"} - # Each Python name reaches the C function that spells itself that way. - assert module.Foo(np.int32(10)) == np.int32(11) - assert module.foo(np.int32(10)) == np.int32(12) - - -def test_a_mixed_case_c_name_keeps_its_spelling(tmp_path: Path): - """Folding case would rename a declaration C never asked to rename.""" - module, contract = _build( - tmp_path, - "mixed_case_api", - "int BarBaz(int value);\n", - "int BarBaz(int v) { return v + 1; }\n", - ["BarBaz"], - ) - - assert _stated_exports(contract) == ["BarBaz"] - assert module.BarBaz(np.int32(2)) == np.int32(3) - assert "barbaz" not in contract.read_text(encoding="utf-8") + # C spells its declarations exactly: two spellings are two functions, and + # a mixed-case name is never folded. + assert module.Foo(np.int32(1)) == np.int32(11) + assert module.foo(np.int32(1)) == np.int32(21) + assert module.BarBaz(np.int32(1)) == np.int32(31) + assert "barbaz" not in text diff --git a/tests/c/functions/parsing/test_c_functions.py b/tests/c/functions/parsing/test_c_functions.py index 831419878..4b705e6e1 100644 --- a/tests/c/functions/parsing/test_c_functions.py +++ b/tests/c/functions/parsing/test_c_functions.py @@ -124,7 +124,7 @@ def test_unnamed_builtin_parameter_prototype_is_not_an_old_style_definition(): def test_modern_prototype_before_old_style_definition_does_not_stop_knr_detection(): - from prik.parsers.c import CParseError, CParser, parse_c_file + from prik.parsers.c import CParseError, parse_c_file source = """ int modern(int value) @@ -148,53 +148,12 @@ def test_modern_prototype_before_old_style_definition_does_not_stop_knr_detectio assert error.column == 5 assert error.source_line == "int legacy(a)" - with pytest.raises(CParseError): - CParser()._raise_for_unsupported_old_style_definitions( - source, - "mixed_knr.c", - use_linemarkers=False, - normalize_compiler_extensions=False, - ) - - -def test_old_style_knr_scan_skips_directives_and_keeps_scanning(): - from prik.parsers.c import CParseError, CParser - - parser = CParser() - parser._raise_for_unsupported_old_style_definitions( - "#if defined(FEATURE)\nint value;\n", - "feature_guard.c", - use_linemarkers=False, - normalize_compiler_extensions=False, - ) - - with pytest.raises(CParseError): - parser._raise_for_unsupported_old_style_definitions( - "(not_a_declaration)\nint legacy(a)\nint a;\n", - "scan_through.c", - use_linemarkers=False, - normalize_compiler_extensions=False, - ) - - -def test_find_parameter_list_returns_outer_function_signature_bounds(): - from prik.parsers.c import CParser - - parser = CParser() - text = "int run(int (*callback)(char ch), const char *label) " - - assert parser._find_parameter_list(text) == (text.index("("), text.rstrip().rindex(")")) - assert parser._find_parameter_list("int value") is None - -def test_control_statement_parameter_lists_inside_function_bodies_are_not_knr_definitions(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - """ -int run(int value) -{ - if (value) +@pytest.mark.parametrize( + "body", + [ + pytest.param( + """ if (value) { return value; } @@ -211,11 +170,20 @@ def test_control_statement_parameter_lists_inside_function_bodies_are_not_knr_de default: return value; } - return 0; -} """, - filename="body_control.c", - ) + id="braced-control-statements-and-labels", + ), + pytest.param( + " if (value)\n value = 1;\n else if (value)\n value = 2;\n", id="else-if-chain" + ), + pytest.param(" value_type :: state;\n", id="non-c-tokens"), + pytest.param(" @@@\n", id="invalid-syntax"), + ], +) +def test_function_bodies_are_skipped_without_knr_or_syntax_checks(body): + from prik.parsers.c import parse_c_file + + parsed = parse_c_file(f"\nint run(int value)\n{{\n{body} return 0;\n}}\n", filename="body.c") assert [function.name for function in parsed.functions] == ["run"] @@ -223,13 +191,17 @@ def test_control_statement_parameter_lists_inside_function_bodies_are_not_knr_de @pytest.mark.parametrize( "source", [ - "def solve():\n return 0\n", - "lambda x: x\n", - "int add(int a, int b);\ninteger :: state;\n", - "int add(int a, int b);\ntype(c_ptr) :: handle;\n", + pytest.param("def solve():\n return 0\n", id="python-definition"), + pytest.param("int add(int a, int b);\ninteger :: state;\n", id="fortran-declaration-after-c"), + pytest.param("@@@\n", id="garbage-only"), + pytest.param("int run(void);\n@@@;\n", id="garbage-after-c"), + pytest.param("struct bad { @@@; };\n", id="struct-member"), + pytest.param("enum bad { OK, @@@ };\n", id="enum-constant"), + pytest.param("int run(@@@);\n", id="parameter-list"), + pytest.param("int run(int first, ..., int last);\n", id="ellipsis-before-last-parameter"), ], ) -def test_c_parser_rejects_non_c_top_level_syntax(source): +def test_c_parser_rejects_invalid_top_level_and_nested_syntax(source): from prik.parsers.c import CParseError, parse_c_file with pytest.raises(CParseError, match="Invalid C syntax") as exc_info: @@ -253,115 +225,6 @@ def test_c_parser_invalid_syntax_error_maps_preprocessed_source_location(): assert exc_info.value.line_number == 80 -def test_c_parser_skips_non_c_tokens_inside_function_body(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - """ -int run(void) -{ - value_type :: state; - return 0; -} -""", - filename="mixed_body.c", - ) - - assert [function.name for function in parsed.functions] == ["run"] - - -def test_c_parser_does_not_classify_valid_c_from_typedef_identifier_spelling(): - from prik.parsers.c import CTypedef, parse_c_file - - parsed = parse_c_file("subroutine solve(void);\n", filename="identifier_spelling.h") - - assert [function.name for function in parsed.functions] == ["solve"] - assert isinstance(parsed.functions[0].result_type, CTypedef) - assert parsed.functions[0].result_type.name == "subroutine" - - -@pytest.mark.parametrize("source", ["@@@\n", "int run(void);\n@@@;\n"]) -def test_c_parser_rejects_invalid_top_level_syntax(source): - from prik.parsers.c import CParseError, parse_c_file - - with pytest.raises(CParseError, match="Invalid C syntax") as exc_info: - parse_c_file(source, filename="invalid.c") - - assert exc_info.value.code == "CPARSE_INVALID_SYNTAX" - - -def test_c_parser_ignores_invalid_syntax_inside_function_body(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - """ -int run(void) -{ - @@@ - return 0; -} -""", - filename="invalid_body.c", - ) - - assert [function.name for function in parsed.functions] == ["run"] - - -@pytest.mark.parametrize( - "source", - [ - "struct bad { @@@; };\n", - "enum bad { OK, @@@ };\n", - "int run(@@@);\n", - "int run(int first, ..., int last);\n", - ], -) -def test_c_parser_rejects_invalid_nested_grammar_units(source): - from prik.parsers.c import CParseError, parse_c_file - - with pytest.raises(CParseError, match="Invalid C syntax") as exc_info: - parse_c_file(source, filename="invalid_nested.h") - - assert exc_info.value.code == "CPARSE_INVALID_SYNTAX" - - -def test_control_flow_conditions_inside_function_body_do_not_look_like_knr_definitions(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - """ -int evaluate(int value) -{ - if (value) - value = 1; - else if (value) - value = 2; - return value; -} -""", - filename="control_flow.c", - ) - - assert [function.name for function in parsed.functions] == ["evaluate"] - - -def test_function_pointer_parameter_is_a_callback_candidate_with_nameless_signature(): - from prik.parsers.c import CFunctionType, CInt, CPointer, parse_c_file - - parsed = parse_c_file( - "void sort_items(void *items, int (*compare)(const void *, const void *));\n", - filename="callbacks.h", - ) - - compare = parsed.functions[0].parameters[1] - assert compare.callback_candidate is True - assert compare.callback_policy is None - assert [type(component) for component in compare.type.components] == [CPointer, CFunctionType] - signature = compare.type.components[1] - assert isinstance(signature.result_type, CInt) - assert len(signature.parameter_types) == 2 - - def test_function_parameter_preserves_declaration_and_adjusts_to_callback_pointer(): from prik.parsers.c import CComposedType, CFunctionType, CPointer, parse_c_file @@ -480,23 +343,13 @@ def test_unsupported_function_declarator_is_reported_and_later_declarations_cont assert diagnostic.location.column == 1 -def test_conflicting_function_prototypes_report_diagnostic(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - "int work(int value);\ndouble work(double value);\n", - filename="conflicting_functions.h", - ) - - assert [function.name for function in parsed.functions] == ["work"] - assert any(diag.code == "C_CONFLICTING_FUNCTION_DECLARATION" for diag in parsed.diagnostics) - - def test_function_conflicts_consider_parameters_and_variadic_marker(): from prik.parsers.c import parse_c_file parsed = parse_c_file( """ +int work(int value); +double work(double value); int same_return(int value); int same_return(double value); int log_msg(const char *fmt); @@ -510,7 +363,7 @@ def test_function_conflicts_consider_parameters_and_variadic_marker(): for diagnostic in parsed.diagnostics if diagnostic.code == "C_CONFLICTING_FUNCTION_DECLARATION" ] - assert conflicts == ["same_return", "log_msg"] + assert conflicts == ["work", "same_return", "log_msg"] def test_duplicate_function_definitions_report_diagnostic(): diff --git a/tests/c/functions/semantics/test_functions_and_callbacks.py b/tests/c/functions/semantics/test_functions_and_callbacks.py index ab8ffd86d..a9b63c6dc 100644 --- a/tests/c/functions/semantics/test_functions_and_callbacks.py +++ b/tests/c/functions/semantics/test_functions_and_callbacks.py @@ -1,4 +1,4 @@ -"""Tests split by stable ownership concept from `test_functions_and_callbacks.py`.""" +"""C function declarations and their semantic projection.""" from prik.parsers.c import parse_c_file from prik.parsers.c.models import ( @@ -11,15 +11,12 @@ CInt, CParameter, CPointer, - CSourceLocation, - CStruct, - CTypedef, CUnknownType, CVariable, CVolatile, CVoid, ) -from prik.semantics.c2ir import CToIRConverter, c_file_to_semantic_modules, c_function_to_semantic_function +from prik.semantics.c2ir import CToIRConverter, c_file_to_semantic_modules from tests.c._support.semantic_conversion import ( _assert_c_origin, _function, @@ -34,87 +31,25 @@ def test_c2ir_converts_scalar_function_signatures_and_preserves_native_order(): scale = _function(module, "scale") assert module.name == "api" - assert [arg.name for arg in add.arguments] == ["a", "b"] - assert [arg.semantic_type.name for arg in add.arguments] == ["Int", "Int"] - assert [arg.semantic_type.dtype for arg in add.arguments] == ["Int32", "Int32"] - assert [arg.metadata for arg in add.arguments] == [{"native_position": 0}, {"native_position": 1}] + assert module.metadata["source_language"] == "c" + assert [(arg.name, arg.semantic_type.name, arg.semantic_type.dtype) for arg in add.arguments] == [ + ("a", "Int", "Int32"), + ("b", "Int", "Int32"), + ] assert add.native_name == "add" assert add.visibility == "public" - assert add.return_type.name == "Int" - assert add.return_type.dtype == "Int32" - assert [mapping.native_position for mapping in add.projection] == [0, 1] + assert (add.return_type.name, add.return_type.dtype) == ("Int", "Int32") assert scale.return_type.name == "Float64" - assert scale.arguments[0].semantic_type.metadata == {} - assert scale.arguments[0].semantic_type.origin.metadata["c_type"] == "CDouble" - assert module.metadata == { - "source_language": "c", - "counts": { - "functions": 2, - "structs": 0, - "unions": 0, - "enums": 0, - "typedefs": 0, - "macros": 0, - "includes": 0, - "diagnostics": 0, - }, - "preprocessing": "raw", - } - assert add.metadata == { - "storage": [], - "specifiers": [], - "prototype_style": "prototype", - "is_definition": False, - "c_abi": { - "calling_convention": "c", - "variadic": False, - "result": { - "source_spelling": "int", - "pointer_depth": 0, - "qualifiers": (), - "const": False, - "has_array_declarator": False, - "has_function_pointer": False, - }, - "parameters": [ - { - "source_spelling": "int", - "pointer_depth": 0, - "qualifiers": (), - "const": False, - "has_array_declarator": False, - "has_function_pointer": False, - }, - { - "source_spelling": "int", - "pointer_depth": 0, - "qualifiers": (), - "const": False, - "has_array_declarator": False, - "has_function_pointer": False, - }, - ], - }, - } assert [ - ( - mapping.python_name, - mapping.native_name, - mapping.native_position, - mapping.python_position, - ) + (mapping.python_name, mapping.native_name, mapping.native_position, mapping.python_position) for mapping in add.projection - ] == [ - ("a", "a", 0, 0), - ("b", "b", 1, 1), - ] - _assert_c_origin( - add.arguments[0].origin, - native_name="a", - native_scope="add", - source_kind="parameter", - source_type="int a", - ) + ] == [("a", "a", 0, 0), ("b", "b", 1, 1)] + # The exact C ABI spelling is recorded for direct-C policy to consume. + c_abi = add.metadata["c_abi"] + assert c_abi["variadic"] is False + assert c_abi["result"]["source_spelling"] == "int" + assert [parameter["source_spelling"] for parameter in c_abi["parameters"]] == ["int", "int"] + assert add.metadata["prototype_style"] == "prototype" _assert_c_origin( add.origin, native_name="add", @@ -127,23 +62,6 @@ def test_c2ir_converts_scalar_function_signatures_and_preserves_native_order(): "source_line": "int add(int a, int b);", }, ) - _assert_c_origin( - module.origin, - native_name="api.h", - native_scope="api.h", - source_kind="translation_unit", - metadata={"preprocessing": "raw"}, - ) - - -def test_c_function_compatibility_helper_accepts_parser_function(): - parsed = parse_c_file("float half(float value);\n", filename="helpers.h") - - function = c_function_to_semantic_function(parsed.functions[0]) - - assert function.name == "half" - assert function.arguments[0].semantic_type.name == "Float32" - assert function.return_type.name == "Float32" def test_c2ir_converts_qualifiers_callbacks_bitfields_and_unspecified_functions(): @@ -157,122 +75,32 @@ def test_c2ir_converts_qualifiers_callbacks_bitfields_and_unspecified_functions( converter = CToIRConverter() variable = converter.visit(CVariable(name="handler", type=callback, storage=["static"])) field = converter.visit(CVariable(name="bits", type=CInt(), bit_width="3")) - unresolved_variable = converter.visit( - CVariable(name="missing_value", type=CUnknownType(spelling="missing_t", source_text="missing_t")) - ) function = converter.visit(parse_c_file("static int legacy();\n", filename="legacy.h").functions[0]) qualified = converter.visit( CChar(qualifiers=[CConst(), CVolatile(), CAtomic()], source_text="const volatile _Atomic char") ) - unnamed = converter.visit(CParameter(name=None, type=CInt())) - located_parameter = converter.visit( - CParameter( - name="located", - type=CInt(), - source_location=CSourceLocation(filename="api.h", line=3, column=5, source_line="int located"), - ) - ) - ownerless_missing_parameter = converter.visit( - CParameter(name="missing", type=CUnknownType(spelling="missing_t", source_text="missing_t")) - ) callback_parameter = converter.visit(CParameter(name="callback", type=callback)) variadic = converter.visit(parse_c_file("int log_value(const char *fmt, ...);\n").functions[0]) - direct_callback = converter.visit(callback) - direct_function_type = converter.visit(CFunctionType(result_type=CVoid(), parameter_types=[CInt()])) void_type = converter.visit(CVoid()) - missing_parameter = converter.visit( - CParameter(name="missing", type=CUnknownType(spelling="missing_t", source_text="missing_t")), - owner="load", - ) - loose_struct = converter.visit(CStruct(name="loose"), as_type=True) missing_return = converter.visit( CFunction(name="missing_return", result_type=CUnknownType(spelling="missing_t", source_text="missing_t")) ) unnamed_function = converter.visit(CFunction(name="unnamed", parameters=[CParameter(name=None, type=CInt())])) + # ``static`` storage is translation-unit local, so it is never published. assert variable.visibility == "private" + assert function.visibility == "private" + assert function.metadata["prototype_style"] == "unspecified" + assert function.metadata["c_abi"]["parameters"] == [] assert variable.semantic_type.name == "CFunctionPointer" - assert variable.semantic_type.dtype == "CFunctionPointer" - assert variable.semantic_type.metadata == {"source_type": "void (*)(int)"} - _assert_c_origin( - variable.semantic_type.origin, - source_kind="function_pointer", - source_type="void (*)(int)", - ) + assert callback_parameter.semantic_type.metadata == {"source_type": "void (*)(int)"} assert field.semantic_type.metadata["c_primitive"] == "int" - assert field.semantic_type.metadata["c_type_fact"]["bits"] == 32 - assert field.visibility == "public" - assert unresolved_variable.semantic_type.name == "missing_t" - _assert_c_origin( - field.origin, - native_name="bits", - source_kind="variable", - source_type="CInt", - metadata={"storage": [], "bit_width": "3"}, - ) - assert function.visibility == "private" - assert function.metadata == { - "storage": ["static"], - "specifiers": [], - "prototype_style": "unspecified", - "is_definition": False, - "c_abi": { - "calling_convention": "c", - "variadic": False, - "result": { - "source_spelling": "int", - "pointer_depth": 0, - "qualifiers": (), - "const": False, - "has_array_declarator": False, - "has_function_pointer": False, - }, - "parameters": [], - }, - } + assert field.origin.metadata["bit_width"] == "3" assert qualified.name == "Int8" assert qualified.metadata["c_char_policy"] == "implementation-defined signed 8-bit code unit" - assert qualified.origin.metadata["c_type"] == "CChar" assert qualified.origin.metadata["qualifiers"] == ["const", "volatile", "_Atomic"] - assert unnamed.name == "arg0" - assert unnamed.metadata == {"native_position": 0} - assert located_parameter.origin.source_location == { - "filename": "api.h", - "line": 3, - "column": 5, - "source_line": "int located", - } - assert ownerless_missing_parameter.semantic_type.name == "missing_t" - assert callback_parameter.semantic_type.metadata == {"source_type": "void (*)(int)"} - assert unnamed_function.projection[0].native_name == "arg0" assert variadic.metadata["prototype_style"] == "prototype" - assert direct_callback.metadata == {"source_type": "void (*)(int)"} - _assert_c_origin( - direct_callback.origin, - source_kind="function_pointer", - source_type="void (*)(int)", - ) - assert direct_function_type.metadata == {"source_type": "CFunctionType"} - assert void_type.name == "Any" - assert void_type.dtype == "Any" - assert void_type.metadata == {"c_void_pointer_pointee": True} - _assert_c_origin( - void_type.origin, - source_kind="type", - source_type="CVoid", - metadata={"c_type": "CVoid"}, - ) - assert missing_parameter.semantic_type.name == "missing_t" - assert loose_struct.name == "loose" - assert loose_struct.dtype == "loose" - assert loose_struct.metadata == {"c_kind": "struct", "incomplete": False} - _assert_c_origin( - loose_struct.origin, - native_name="struct loose", - source_kind="type", - source_type="struct loose", - metadata={"c_type": "CStruct"}, - ) + assert variadic.metadata["c_abi"]["variadic"] is True + assert (void_type.name, void_type.metadata) == ("Any", {"c_void_pointer_pointee": True}) assert missing_return.return_type.name == "missing_t" - assert converter._return_type(CVoid(), owner="nothing") is None - assert CToIRConverter().visit(CTypedef(name="absent_t")).name == "absent_t" + assert unnamed_function.projection[0].native_name == "arg0" diff --git a/tests/c/infrastructure/building/pipeline/test_c_build_cli.py b/tests/c/infrastructure/building/pipeline/test_c_build_cli.py index 580cfc580..ef235a6ec 100644 --- a/tests/c/infrastructure/building/pipeline/test_c_build_cli.py +++ b/tests/c/infrastructure/building/pipeline/test_c_build_cli.py @@ -10,33 +10,19 @@ import numpy as np import pytest +import prik.cli as prik_cli + from prik import build_c_extension, build_pyi_extension, build_pyi_extension_from_manifest from tests.c._support.runtime import sole_native_module @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_cli_builds_a_c_source_only_when_the_c_language_is_explicit(tmp_path: Path): - source = tmp_path / "answer.c" - source.write_text("int answer(int value) { return value + 1; }\n", encoding="utf-8") +def test_cli_builds_a_c_source_only_when_the_c_language_is_explicit(tmp_path: Path, monkeypatch, capsys): + monkeypatch.chdir(tmp_path) + (tmp_path / "answer.c").write_text("int answer(int value) { return value + 1; }\n", encoding="utf-8") - completed = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "--language", - "c", - str(source), - "--out-dir", - str(tmp_path / "build"), - "--json", - ], - cwd=tmp_path, - capture_output=True, - text=True, - check=True, - ) - payload = json.loads(completed.stdout) + assert prik_cli.main(["--language", "c", "answer.c", "--out-dir", "build", "--json"]) == 0 + payload = json.loads(capsys.readouterr().out) assert Path(payload["shared_library"]) == tmp_path / "answer.so" assert Path(payload["shared_library"]).is_file() @@ -44,32 +30,20 @@ def test_cli_builds_a_c_source_only_when_the_c_language_is_explicit(tmp_path: Pa @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_cli_marks_a_source_free_pyi_contract_as_c_native_explicitly(tmp_path: Path): - contract = tmp_path / "api.pyi" - contract.write_text("from prik.contracts import Int\ndef add(value: Int) -> Int: ...\n", encoding="utf-8") - source = tmp_path / "implementation.c" - source.write_text("int add(int value) { return value + 1; }\n", encoding="utf-8") +def test_cli_marks_a_source_free_pyi_contract_as_c_native_explicitly(tmp_path: Path, monkeypatch, capsys): + monkeypatch.chdir(tmp_path) + (tmp_path / "api.pyi").write_text( + "from prik.contracts import Int\ndef add(value: Int) -> Int: ...\n", encoding="utf-8" + ) + (tmp_path / "implementation.c").write_text("int add(int value) { return value + 1; }\n", encoding="utf-8") - completed = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "--language", - "c", - str(contract), - "--native-c-sources", - str(source), - "--out-dir", - str(tmp_path / "build"), - "--json", - ], - cwd=tmp_path, - capture_output=True, - text=True, - check=True, + assert ( + prik_cli.main( + ["--language", "c", "api.pyi", "--native-c-sources", "implementation.c", "--out-dir", "build", "--json"] + ) + == 0 ) - payload = json.loads(completed.stdout) + payload = json.loads(capsys.readouterr().out) assert Path(payload["shared_library"]) == tmp_path / "api.so" assert payload["manifest"]["extension"]["native_language"] == "c" @@ -77,7 +51,7 @@ def test_cli_marks_a_source_free_pyi_contract_as_c_native_explicitly(tmp_path: P @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_native_manifest_replay_and_makefile_retain_the_c_language(tmp_path: Path): +def test_c_native_manifest_replay_and_makefile_retain_the_c_language_and_contract_graph(tmp_path: Path): contract = tmp_path / "api.pyi" contract.write_text("from prik.contracts import Int\ndef add(value: Int) -> Int: ...\n", encoding="utf-8") source = tmp_path / "implementation.c" @@ -103,41 +77,12 @@ def test_c_native_manifest_replay_and_makefile_retain_the_c_language(tmp_path: P module = sole_native_module(replay.import_module()) assert module.add(np.int32(4)) == np.int32(5) - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_manifest_replay_rejects_a_changed_contract_graph_before_compiler_selection(tmp_path: Path): - contract = tmp_path / "api.pyi" - contract.write_text("from prik.contracts import Int\ndef add(value: Int) -> Int: ...\n", encoding="utf-8") - source = tmp_path / "implementation.c" - source.write_text("int add(int value) { return value + 1; }\n", encoding="utf-8") - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build", - makefile=True, - ) + # A manifest whose contract graph changed is rejected before any compiler is selected. manifest = json.loads(result.build_manifest.read_text(encoding="utf-8")) manifest["contract_paths"].append("contract-that-was-not-recorded.pyi") result.build_manifest.write_text(json.dumps(manifest), encoding="utf-8") - with pytest.raises(ValueError, match="import graph does not match"): - build_pyi_extension_from_manifest( - result.build_manifest, - input_c_compiler="compiler-that-must-not-run", - ) - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_verbose_c_build_reports_c_compilation_and_link_commands(tmp_path: Path, capsys): - source = tmp_path / "answer.c" - source.write_text("int answer(int value) { return value + 1; }\n", encoding="utf-8") - - build_c_extension(source, output_dir=tmp_path / "build", verbose=True) - - output = capsys.readouterr().out - assert "cc" in output - assert "-shared" in output + build_pyi_extension_from_manifest(result.build_manifest, input_c_compiler="compiler-that-must-not-run") @pytest.mark.skipif( @@ -172,34 +117,31 @@ def test_c_direct_symbol_survives_a_mixed_language_link_with_the_fortran_driver( sys.platform == "win32" or shutil.which("make") is None or shutil.which("cc") is None, reason="requires GNU Make, a POSIX shell, and a C compiler", ) -def test_generated_c_makefile_builds_an_importable_extension_from_relative_paths(tmp_path: Path): +def test_generated_c_makefile_builds_an_importable_extension_from_relative_paths(tmp_path: Path, monkeypatch, capsys): """A generated Makefile must run on a clean tree, not only after a build.""" source = tmp_path / "src" / "answer.c" source.parent.mkdir() source.write_text("int answer(int value) { return value + 1; }\n", encoding="utf-8") - generated = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--makefile", - "--language", - "c", - "src/answer.c", - "--out-dir", - "build", - "--compiler", - "cc", - "--json", - ], - cwd=tmp_path, - capture_output=True, - text=True, - check=True, + monkeypatch.chdir(tmp_path) + assert ( + prik_cli.main( + [ + "generate", + "--makefile", + "--language", + "c", + "src/answer.c", + "--out-dir", + "build", + "--compiler", + "cc", + "--json", + ] + ) + == 0 ) - makefile = Path(json.loads(generated.stdout)["build_makefile"]) + makefile = Path(json.loads(capsys.readouterr().out)["build_makefile"]) subprocess.run( ["make", "-j4", "-f", str(makefile), "all"], cwd=tmp_path, capture_output=True, text=True, check=True ) @@ -215,7 +157,7 @@ def test_generated_c_makefile_builds_an_importable_extension_from_relative_paths @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_saved_c_contract_describes_only_the_wrapped_translation_unit(tmp_path: Path): +def test_saved_c_contract_describes_only_the_wrapped_translation_unit(tmp_path: Path, capsys): """Preprocessed headers stay inspection facts, not part of the built API.""" source = tmp_path / "mathlib.c" source.write_text( @@ -230,8 +172,12 @@ def test_saved_c_contract_describes_only_the_wrapped_translation_unit(tmp_path: encoding="utf-8", ) - result = build_c_extension(source, output_dir=tmp_path / "build", output_name="mathlib") + result = build_c_extension(source, output_dir=tmp_path / "build", output_name="mathlib", verbose=True) module = sole_native_module(result.import_module()) + # Verbose mode prints the C compile and shared-library link commands. + build_output = capsys.readouterr().out + assert "cc" in build_output + assert "-shared" in build_output contract_path = tmp_path / "build" / "contracts" / "mathlib.pyi" package_path = tmp_path / "build" / "contracts" / "__init__.pyi" contract = contract_path.read_text(encoding="utf-8") diff --git a/tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py b/tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py index 3ca0ac147..f068ffb98 100644 --- a/tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py +++ b/tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py @@ -1,4 +1,10 @@ -"""Pipeline boundary tests for fail-closed C direct adoption.""" +"""Pipeline boundary tests for fail-closed C direct adoption. + +Every row names a declaration outside the published direct-C boundary and +proves the build stops with its blocker code before writing build output. +Rows that pass ``compiler-that-must-not-run`` as the C compiler also prove the +blocker fires before the target ABI probe or any native compilation. +""" import shutil from pathlib import Path @@ -8,161 +14,122 @@ from prik import build_c_extension, build_pyi_extension from prik.preprocessing import PreprocessingConfig +_NO_COMPILER = "compiler-that-must-not-run" +_REQUIRES_CC = pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_unsupported_c_callback_fails_before_build_output_or_native_compilation(tmp_path: Path): - source = tmp_path / "callback.c" - output_dir = tmp_path / "build" - source.write_text( - "int identity(int value);\nvoid callback(void (*action)(int));\n", - encoding="utf-8", - ) - - with pytest.raises(ValueError, match="C_DIRECT_CALLBACK:action"): - build_c_extension( - source, - preprocessing=PreprocessingConfig(mode="compiler", compiler="cc"), - input_c_compiler="compiler-that-must-not-run", - output_dir=output_dir, - ) - - assert not output_dir.exists() - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_later_c_module_blocker_fails_before_earlier_module_abi_probe(tmp_path: Path): - primitive = tmp_path / "primitive.c" - callback = tmp_path / "callback.c" - output_dir = tmp_path / "build" - primitive.write_text("int identity(int value);\n", encoding="utf-8") - callback.write_text("void callback(void (*action)(int));\n", encoding="utf-8") - - with pytest.raises(ValueError, match="C_DIRECT_CALLBACK:action"): - build_c_extension( - [primitive, callback], - preprocessing=PreprocessingConfig(mode="compiler", compiler="cc"), - input_c_compiler="compiler-that-must-not-run", - output_dir=output_dir, - ) - - assert not output_dir.exists() - - -def test_volatile_c_access_fails_before_build_output_or_native_compilation(tmp_path: Path): - source = tmp_path / "volatile.c" - output_dir = tmp_path / "build" - source.write_text("void update(volatile int *value);\n", encoding="utf-8") - - with pytest.raises(ValueError, match="C_DIRECT_UNSUPPORTED_QUALIFIER:value"): - build_c_extension(source, output_dir=output_dir) - - assert not output_dir.exists() - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_aggregate_fails_before_target_probe_or_build_output(tmp_path: Path): - source = tmp_path / "aggregate.c" - output_dir = tmp_path / "build" - source.write_text( - "struct pair { int left; int right; };\nint accept_pair(struct pair value);\n", - encoding="utf-8", - ) +def _compiler_preprocessing() -> dict[str, object]: # Source preparation uses a working preprocessor; the target ABI probe uses # the executable that must never run, so reaching it would fail differently. - with pytest.raises(ValueError, match="C_DIRECT_AGGREGATE_TYPE:pair"): - build_c_extension( - source, - preprocessing=PreprocessingConfig(mode="compiler", compiler="cc"), - input_c_compiler="compiler-that-must-not-run", - output_dir=output_dir, - ) - - assert not output_dir.exists() - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_native_global_state_fails_before_any_generated_adapter_source(tmp_path: Path): - source = tmp_path / "globals.c" - output_dir = tmp_path / "build" - source.write_text("int scale(int value) { return value; }\nint gain = 2;\n", encoding="utf-8") - - with pytest.raises(ValueError, match="C_DIRECT_NATIVE_GLOBAL_STATE:gain"): - build_c_extension( - source, - preprocessing=PreprocessingConfig(mode="compiler", compiler="cc"), - input_c_compiler="compiler-that-must-not-run", - output_dir=output_dir, - ) - - assert not output_dir.exists() - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_enum_constants_fail_before_wrapper_planning(tmp_path: Path): - source = tmp_path / "enums.c" - output_dir = tmp_path / "build" - source.write_text("enum color { RED, GREEN };\nint pick(int value) { return value; }\n", encoding="utf-8") - - with pytest.raises(ValueError, match="C_DIRECT_ENUM_CONSTANT:RED"): - build_c_extension(source, output_dir=output_dir) - - assert not output_dir.exists() - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_source_free_contract_module_variable_generates_no_fortran_adapter(tmp_path: Path): - contract = tmp_path / "api.pyi" - implementation = tmp_path / "implementation.c" - output_dir = tmp_path / "build" - contract.write_text( - "from prik.contracts import Float64\n\ngain: Float64\n\ndef scale(value: Float64) -> Float64: ...\n", - encoding="utf-8", - ) - implementation.write_text("double gain = 2.0;\ndouble scale(double value) { return value; }\n", encoding="utf-8") - - with pytest.raises(ValueError, match="C_DIRECT_NATIVE_GLOBAL_STATE:gain"): - build_pyi_extension( - contract, - native_language="c", - native_c_sources=[implementation], - output_dir=output_dir, - ) - - assert not output_dir.exists() - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_unmodeled_c_declaration_is_not_silently_dropped_from_the_public_api(tmp_path: Path): - source = tmp_path / "attributes.c" + return {"preprocessing": PreprocessingConfig(mode="compiler", compiler="cc"), "input_c_compiler": _NO_COMPILER} + + +@pytest.mark.parametrize( + ("sources", "options", "blocker"), + [ + pytest.param( + {"callback.c": "int identity(int value);\nvoid callback(void (*action)(int));\n"}, + _compiler_preprocessing(), + "C_DIRECT_CALLBACK:action", + id="callback-parameter", + ), + pytest.param( + {"primitive.c": "int identity(int value);\n", "callback.c": "void callback(void (*action)(int));\n"}, + _compiler_preprocessing(), + "C_DIRECT_CALLBACK:action", + marks=_REQUIRES_CC, + id="later-module-blocker-before-earlier-module-abi-probe", + ), + pytest.param( + {"volatile.c": "void update(volatile int *value);\n"}, + {}, + "C_DIRECT_UNSUPPORTED_QUALIFIER:value", + id="volatile-access", + ), + pytest.param( + {"aggregate.c": "struct pair { int left; int right; };\nint accept_pair(struct pair value);\n"}, + _compiler_preprocessing(), + "C_DIRECT_AGGREGATE_TYPE:pair", + marks=_REQUIRES_CC, + id="aggregate-by-value", + ), + pytest.param( + {"globals.c": "int scale(int value) { return value; }\nint gain = 2;\n"}, + _compiler_preprocessing(), + "C_DIRECT_NATIVE_GLOBAL_STATE:gain", + marks=_REQUIRES_CC, + id="native-global-state", + ), + pytest.param( + {"enums.c": "enum color { RED, GREEN };\nint pick(int value) { return value; }\n"}, + {}, + "C_DIRECT_ENUM_CONSTANT:RED", + marks=_REQUIRES_CC, + id="enum-constants", + ), + pytest.param( + { + "attributes.c": "__attribute__((stdcall)) int convention(int value);\nint ordinary(int value) { return value; }\n" + }, + {}, + "C_DIRECT_UNMODELED_DECLARATION", + marks=_REQUIRES_CC, + id="unmodeled-declaration-is-not-silently-dropped", + ), + ], +) +def test_c_source_outside_the_direct_boundary_fails_before_build_output( + tmp_path: Path, sources: dict[str, str], options: dict[str, object], blocker: str +): + paths = [] + for name, text in sources.items(): + path = tmp_path / name + path.write_text(text, encoding="utf-8") + paths.append(path) output_dir = tmp_path / "build" - source.write_text( - "__attribute__((stdcall)) int convention(int value);\nint ordinary(int value) { return value; }\n", - encoding="utf-8", - ) - with pytest.raises(ValueError, match="C_DIRECT_UNMODELED_DECLARATION"): - build_c_extension(source, output_dir=output_dir) + with pytest.raises(ValueError, match=blocker): + build_c_extension(paths if len(paths) > 1 else paths[0], output_dir=output_dir, **options) assert not output_dir.exists() -def test_raw_c_contract_address_fails_before_target_probe_or_build_output(tmp_path: Path): - contract = tmp_path / "raw_address.pyi" - implementation = tmp_path / "implementation.c" +@pytest.mark.parametrize( + ("contract", "implementation", "options", "blocker"), + [ + pytest.param( + "from prik.contracts import Float64\n\ngain: Float64\n\ndef scale(value: Float64) -> Float64: ...\n", + "double gain = 2.0;\ndouble scale(double value) { return value; }\n", + {}, + "C_DIRECT_NATIVE_GLOBAL_STATE:gain", + marks=_REQUIRES_CC, + id="module-variable-generates-no-fortran-adapter", + ), + pytest.param( + "from prik.contracts import Addr, Int\n\ndef consume(value: Addr(Int)) -> Int: ...\n", + "int consume(int value) { return value; }\n", + {"input_c_compiler": _NO_COMPILER}, + "C_DIRECT_RAW_ADDRESS:value", + id="raw-address-before-target-probe", + ), + ], +) +def test_c_contract_outside_the_direct_boundary_fails_before_build_output( + tmp_path: Path, contract: str, implementation: str, options: dict[str, object], blocker: str +): + contract_path = tmp_path / "api.pyi" + implementation_path = tmp_path / "implementation.c" + contract_path.write_text(contract, encoding="utf-8") + implementation_path.write_text(implementation, encoding="utf-8") output_dir = tmp_path / "build" - contract.write_text( - "from prik.contracts import Addr, Int\n\ndef consume(value: Addr(Int)) -> Int: ...\n", - encoding="utf-8", - ) - implementation.write_text("int consume(int value) { return value; }\n", encoding="utf-8") - with pytest.raises(ValueError, match="C_DIRECT_RAW_ADDRESS:value"): + with pytest.raises(ValueError, match=blocker): build_pyi_extension( - contract, + contract_path, native_language="c", - native_c_sources=[implementation], - input_c_compiler="compiler-that-must-not-run", + native_c_sources=[implementation_path], output_dir=output_dir, + **options, ) assert not output_dir.exists() diff --git a/tests/c/infrastructure/cli/pipeline/test_c_cli_argument_contract.py b/tests/c/infrastructure/cli/pipeline/test_c_cli_argument_contract.py index d0c26abec..0d520ed06 100644 --- a/tests/c/infrastructure/cli/pipeline/test_c_cli_argument_contract.py +++ b/tests/c/infrastructure/cli/pipeline/test_c_cli_argument_contract.py @@ -1,154 +1,74 @@ -"""C input-language CLI argument contracts.""" +"""C input-language CLI argument contracts. + +Shared CLI validation, routing, and help completeness are proved once by +``tests/fortran/infrastructure/cli/pipeline/``; these rows are the diagnostics +only a C input or a C-only option can reach. They run real command lines +through ``prik.cli.main``. +""" from pathlib import Path -import types import pytest import prik.cli as prik_cli -from tests.c._support.cli import ( - _MainParserError, - _install_main_parser, - _main_args, -) - - -def test_prik_build_preprocessing_config_preserves_full_config_contract(monkeypatch): - args = types.SimpleNamespace( - defines=["USE_FAST=1"], - undefs=["LEGACY"], - compiler="cc", - compile_commands="compile_commands.json", - preprocessor_adapter="command-template", - preprocess_template="{compiler} -E {source}", - include_dirs=["include"], - std="c11", - compiler_args=["--target=test"], - include_exposure="all-project", - public_includes=["public.h"], - private_includes=["private.h"], - language="c", - ) - config = types.SimpleNamespace( - uses_compiler=True, - command_template=args.preprocess_template, - adapter=args.preprocessor_adapter, - compiler=args.compiler, - compile_commands=args.compile_commands, - include_dirs=args.include_dirs, - ) - calls = [] - - class Parser: - def error(self, message): - raise AssertionError(message) - - def validate(value, option): - calls.append(("validate", value, option)) - - def build(**kwargs): - calls.append(("build", kwargs)) - return config - monkeypatch.setattr(prik_cli, "validate_macro_name", validate) - monkeypatch.setattr(prik_cli, "PreprocessingConfig", build) - assert prik_cli._build_preprocessing_config(args, Parser()) is config - assert calls == [ - ("validate", "USE_FAST=1", "--define/-D"), - ("validate", "LEGACY", "--undef/-U"), - ( - "build", - { - "mode": "compiler", - "compiler": "cc", - "compile_commands": "compile_commands.json", - "adapter": "command-template", - "command_template": "{compiler} -E {source}", - "include_dirs": ["include"], - "defines": ["USE_FAST=1"], - "undefs": ["LEGACY"], - "std": "c11", - "compiler_args": ["--target=test"], - "include_exposure": "all-project", - "public_includes": ["public.h"], - "private_includes": ["private.h"], - }, +def _invoke(argv: list[str], capsys) -> tuple[int, str, str]: + """Run one command line in-process and return its exit code and output.""" + try: + code = prik_cli.main(argv) + except SystemExit as exc: + code = exc.code + captured = capsys.readouterr() + return code, captured.out, captured.err + + +@pytest.fixture +def c_inputs(tmp_path: Path, monkeypatch) -> Path: + """Write one input of each kind the rows name, relative to the working directory.""" + monkeypatch.chdir(tmp_path) + (tmp_path / "api.h").write_text("int add(int a, int b);\n", encoding="utf-8") + (tmp_path / "api.c").write_text("int increment(int value) { return value + 1; }\n", encoding="utf-8") + (tmp_path / "contract.pyi").write_text("from prik.contracts import Int\n", encoding="utf-8") + (tmp_path / "exports.txt").write_text("increment\n", encoding="utf-8") + return tmp_path + + +@pytest.mark.parametrize( + ("argv", "message"), + [ + pytest.param( + ["parse", "api.h", "--language", "c", "--show-vars"], + "--show-vars is Fortran-only and is not supported for --language c", + id="c-parse-with-fortran-only-option", ), - ] - - -def test_prik_main_rejects_fortran_only_c_parse_options(monkeypatch): - overrides = {"language": "c", "parse": True, "show_vars": True} - expected = "--show-vars is Fortran-only and is not supported for --language c" - args = _main_args(**overrides) - _install_main_parser(monkeypatch, args) - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: language) - monkeypatch.setattr( - prik_cli, - "_build_preprocessing_config", - lambda active_args, parser: object(), - ) - - with pytest.raises(_MainParserError) as exc_info: - prik_cli.main() - - assert str(exc_info.value) == expected - - -def test_prik_resolve_language_handles_c_input_edges(tmp_path: Path): - class ErrorParser: - def error(self, message): - raise ValueError(message) - - parser = ErrorParser() - input_dir = tmp_path / "inputs" - input_dir.mkdir() - c_header = tmp_path / "api.h" - c_header.write_text("int add(int x);\n", encoding="utf-8") - unknown = tmp_path / "notes.txt" - unknown.write_text("notes\n", encoding="utf-8") - - assert prik_cli._resolve_language([str(unknown)], "c", parser) == "c" - with pytest.raises(ValueError) as requested_error: - prik_cli._resolve_language([str(input_dir), str(c_header)], "fortran", parser) - assert str(requested_error.value) == ( - f"C input {c_header} is incompatible with --language fortran; pass --language c. Use --help for examples." - ) - - -def test_pyi_wrapper_build_rejects_export_symbols(tmp_path: Path, capsys): - """A contract already states its public surface, so the C allowlist has nothing to select.""" - contract = tmp_path / "api.pyi" - contract.write_text("from prik.contracts import Int\n", encoding="utf-8") - implementation = tmp_path / "api.c" - implementation.write_text("int increment(int value) { return value + 1; }\n", encoding="utf-8") - exports = tmp_path / "exports.txt" - exports.write_text("increment\n", encoding="utf-8") - - with pytest.raises(SystemExit) as exc_info: - prik_cli.main( - [ - str(contract), - "--native-c-sources", - str(implementation), - "--export-symbols", - str(exports), - ] - ) - - assert exc_info.value.code == 2 - message = capsys.readouterr().err - assert "--export-symbols selects the public surface" in message - assert "__all__" in message - - -def test_export_symbols_help_names_the_public_surface_it_selects(): - """The option and a contract's __all__ state the same thing, so the help says so.""" - build_help = prik_cli._build_parser(["input.h", "--language", "c", "--help"]).format_help() - generate_help = prik_cli._generate_parser(["--help"]).format_help() - - for help_text in (build_help, generate_help): - assert "--export-symbols" in help_text - assert "public surface" in help_text - assert "__all__" in help_text + pytest.param( + ["generate", "--pyi", "api.h", "--out"], + "C input api.h requires explicit --language c", + id="c-input-without-explicit-language", + ), + pytest.param( + ["generate", "--pyi", "api.h", "--language", "fortran", "--out"], + "C input api.h is incompatible with --language fortran; pass --language c", + id="c-input-with-fortran-frontend", + ), + pytest.param( + ["contract.pyi", "--native-c-sources", "api.c", "--export-symbols", "exports.txt"], + "--export-symbols selects the public surface while reading native source; " + "a semantic .pyi contract already states its public surface in __all__", + id="contract-with-c-export-allowlist", + ), + ], +) +def test_c_cli_rejects_invalid_invocation_with_its_documented_diagnostic( + c_inputs: Path, capsys, argv: list[str], message: str +): + before = sorted(c_inputs.iterdir()) + + code, out, err = _invoke(argv, capsys) + + assert code == 2 + assert out == "" + assert message in " ".join(err.split()) + # A rejected invocation writes nothing next to its inputs. + assert sorted(c_inputs.iterdir()) == before diff --git a/tests/c/infrastructure/cli/pipeline/test_c_cli_output_contract.py b/tests/c/infrastructure/cli/pipeline/test_c_cli_output_contract.py index f060ac970..5131c50b0 100644 --- a/tests/c/infrastructure/cli/pipeline/test_c_cli_output_contract.py +++ b/tests/c/infrastructure/cli/pipeline/test_c_cli_output_contract.py @@ -1,34 +1,166 @@ -"""C input-language CLI output contracts.""" +"""C input-language CLI report and contract outputs, through real command lines.""" -import types +import json +from pathlib import Path + +import pytest import prik.cli as prik_cli -from tests.c._support.cli import ( - _install_main_parser, - _main_args, +from prik.parsers.c import cli as c_parser_cli + + +@pytest.fixture +def c_headers(tmp_path: Path, monkeypatch) -> Path: + monkeypatch.chdir(tmp_path) + (tmp_path / "api.h").write_text("int add(int a, int b);\n", encoding="utf-8") + (tmp_path / "many.h").write_text("int first(void);\nint second(void);\n", encoding="utf-8") + return tmp_path + + +@pytest.mark.parametrize( + ("argv", "destination", "is_json", "fragments"), + [ + pytest.param( + ["parse", "api.h", "--language", "c"], + None, + False, + ("File: api.h", "Language: c", "Functions: 1"), + id="parse-text-stdout", + ), + pytest.param( + ["parse", "many.h", "--language", "c", "--print-limit", "1"], + None, + False, + ("Functions: 2", " - first", " ... 1 more functions"), + id="parse-text-print-limit", + ), + pytest.param( + ["parse", "api.h", "--language", "c", "--json"], + None, + True, + ('"language": "c"', '"name": "add"', '"diagnostics": []'), + id="parse-json-stdout", + ), + pytest.param( + ["parse", "api.h", "--language", "c", "--json", "--out", "report.json"], + "report.json", + True, + ('"language": "c"', '"name": "add"'), + id="parse-json-file", + ), + pytest.param( + ["semantics", "api.h", "--language", "c"], + None, + False, + ("File: api.h", "- add(a: Int32 in, b: Int32 in) -> Int32"), + id="semantics-text-stdout", + ), + pytest.param( + ["semantics", "api.h", "--language", "c", "--json"], + None, + True, + ('"name": "api"', '"dtype": "Int32"', '"c_type_fact_source": "compiler_probe"'), + id="semantics-json-stdout", + ), + pytest.param( + ["generate", "--pyi", "api.h", "--language", "c"], + None, + False, + ("File: api.h", "def add(", "a: Int"), + id="generate-pyi-stdout", + ), + pytest.param( + ["generate", "--pyi", "api.h", "--language", "c", "--out"], + "api.pyi", + False, + ("def add(",), + id="generate-pyi-adjacent", + ), + ], ) +def test_c_report_format_and_destination_are_chosen_independently( + c_headers: Path, capsys, argv, destination, is_json, fragments +): + assert prik_cli.main(argv) == 0 + out = capsys.readouterr().out + if destination is None: + text = out + else: + assert out == "" + text = (c_headers / destination).read_text(encoding="utf-8") + if is_json: + json.loads(text) + assert [fragment for fragment in fragments if fragment not in text] == [] -def test_prik_main_preserves_c_readable_stdout_contract(monkeypatch, capsys): - args = _main_args(language="c", parse=True, print_limit=2) - _install_main_parser(monkeypatch, args) - preprocessing = types.SimpleNamespace(include_dirs=()) - parse_payload = {"parse": {"node": 1}} - formats = [] - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: language) - monkeypatch.setattr( - prik_cli, - "_build_preprocessing_config", - lambda active_args, parser: preprocessing, - ) - monkeypatch.setattr(prik_cli, "parse_c_report", lambda *args, **kwargs: parse_payload) - monkeypatch.setattr( - prik_cli, - "format_c_report", - lambda payload, **kwargs: formats.append((payload, kwargs)) or "C REPORT", +def test_c_parse_preprocesses_with_the_default_c_compiler(tmp_path: Path, monkeypatch, capsys): + """Without preprocessing options a C parse runs ``cc -E`` and records that recipe.""" + monkeypatch.chdir(tmp_path) + (tmp_path / "api_types.h").write_text("typedef int api_int;\n", encoding="utf-8") + (tmp_path / "api.h").write_text( + '#include "api_types.h"\n#define API_DECL(ret) ret\nAPI_DECL(api_int) run(void);\n', + encoding="utf-8", ) - assert prik_cli.main() == 0 - assert capsys.readouterr().out == "C REPORT\n" - assert formats == [(parse_payload, {"print_limit": 2})] + assert prik_cli.main(["parse", "api.h", "--language", "c", "--json"]) == 0 + payload = json.loads(capsys.readouterr().out)["api.h"] + + assert payload["preprocessing"] == "compiler" + assert payload["preprocessing_recipe"]["compiler"] == "cc" + assert [function["name"] for function in payload["functions"]] == ["run"] + + +def test_c_generate_pyi_writes_one_contract_per_header_and_imports_shared_types(tmp_path: Path, capsys): + types = tmp_path / "types.h" + api = tmp_path / "api.h" + types.write_text("struct state { int id; };\n", encoding="utf-8") + api.write_text("struct state;\nvoid step(struct state *state);\n", encoding="utf-8") + + assert prik_cli.main(["generate", "--pyi", str(types), str(api), "--language", "c", "--out"]) == 0 + + assert capsys.readouterr().out == "" + assert "class state(CStruct):" in (tmp_path / "types.pyi").read_text(encoding="utf-8") + api_stub = (tmp_path / "api.pyi").read_text(encoding="utf-8") + assert "from .types import state" in api_stub + assert "class state" not in api_stub + assert "state: state" in api_stub + assert "Addr(state)" not in api_stub + + +def test_format_c_report_print_limit_expands_repeated_sections(): + report = { + "api.h": { + "language": "c", + "functions": [{"name": "add"}, {"name": "scale"}], + "structs": [{"reference": "struct context"}], + "unions": [], + "enums": [{"anonymous_id": "enum@api.h:1:1"}], + "typedefs": [], + "variables": [], + "macros": [], + "includes": [{"path": "api_types.h"}, {}], + "diagnostics": [ + { + "severity": "warning", + "code": "C_UNMODELED_COMPILER_EXTENSION", + "message": "attribute ignored", + } + ], + } + } + + output = c_parser_cli.format_c_report(report, print_limit=1) + + assert " Functions: 2" in output + assert " - add" in output + assert " - scale" not in output + assert " ... 1 more functions" in output + assert " Structs: 1" in output + assert " - struct context" in output + assert " Enums: 1" in output + assert " - enum@api.h:1:1" in output + assert " Includes: 2" in output + assert " - api_types.h" in output + assert " ... 1 more includes" in output + assert "warning: C_UNMODELED_COMPILER_EXTENSION: attribute ignored" in output diff --git a/tests/c/infrastructure/cli/pipeline/test_c_cli_skeleton.py b/tests/c/infrastructure/cli/pipeline/test_c_cli_skeleton.py deleted file mode 100644 index a00862079..000000000 --- a/tests/c/infrastructure/cli/pipeline/test_c_cli_skeleton.py +++ /dev/null @@ -1,564 +0,0 @@ -"""C parser CLI coverage for the current partial subset.""" - -import json -import os -import runpy -import subprocess -import sys -from pathlib import Path -from types import SimpleNamespace - -import pytest - -from prik.parsers.c import CParseError -from prik.parsers.c import cli as c_parser_cli -from prik.parsers.c import sources as c_sources -import prik.cli as prik_cli -from prik.preprocessing import PreprocessingConfig - -CONTRACT_IMPORT = "from prik.contracts import Int32\n\n" - - -def test_cli_parse_help_shows_explicit_c_language_mode(): - cmd = [sys.executable, "-m", "prik", "parse", "--help"] - - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert "--language {fortran,c}" in res.stdout - assert "Input language (default: fortran; use c for C inputs)" in " ".join(res.stdout.split()) - - -def test_cli_c_parse_human_tree_output_for_header(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("int add(int a, int b);\n", encoding="utf-8") - cmd = [sys.executable, "-m", "prik", "parse", str(header), "--language", "c"] - - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert f"File: {header}" in res.stdout - assert "Language: c" in res.stdout - assert "Functions: 1" in res.stdout - assert "Parser status" not in res.stdout - - -def test_format_c_report_print_limit_expands_repeated_sections(): - report = { - "api.h": { - "language": "c", - "functions": [{"name": "add"}, {"name": "scale"}], - "structs": [{"reference": "struct context"}], - "unions": [], - "enums": [{"anonymous_id": "enum@api.h:1:1"}], - "typedefs": [], - "variables": [], - "macros": [], - "includes": [{"path": "api_types.h"}, {}], - "diagnostics": [ - { - "severity": "warning", - "code": "C_UNMODELED_COMPILER_EXTENSION", - "message": "attribute ignored", - } - ], - } - } - - output = c_parser_cli.format_c_report(report, print_limit=1) - - assert " Functions: 2" in output - assert " - add" in output - assert " - scale" not in output - assert " ... 1 more functions" in output - assert " Structs: 1" in output - assert " - struct context" in output - assert " Enums: 1" in output - assert " - enum@api.h:1:1" in output - assert " Includes: 2" in output - assert " - api_types.h" in output - assert " ... 1 more includes" in output - assert "warning: C_UNMODELED_COMPILER_EXTENSION: attribute ignored" in output - - -def test_cli_c_parse_json_stdout_for_header(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("int add(int a, int b);\n", encoding="utf-8") - cmd = [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--json", - ] - - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - payload = json.loads(res.stdout) - file_payload = payload[str(header)] - - assert file_payload["language"] == "c" - assert "parser_status" not in file_payload - assert [fn["name"] for fn in file_payload["functions"]] == ["add"] - assert file_payload["structs"] == [] - assert file_payload["unions"] == [] - assert file_payload["enums"] == [] - assert file_payload["typedefs"] == [] - assert file_payload["variables"] == [] - assert file_payload["macros"] == [] - assert file_payload["includes"] == [] - assert file_payload["diagnostics"] == [] - - -def test_cli_c_parse_preprocesses_macros_by_default(tmp_path: Path): - header = tmp_path / "api.h" - types = tmp_path / "api_types.h" - types.write_text("typedef int api_int;\n", encoding="utf-8") - header.write_text( - '#include "api_types.h"\n#define API_DECL(ret) ret\nAPI_DECL(api_int) run(void);\n', - encoding="utf-8", - ) - cmd = [sys.executable, "-m", "prik", "parse", str(header), "--language", "c", "--json"] - - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - payload = json.loads(res.stdout)[str(header)] - - assert payload["preprocessing"] == "compiler" - assert [function["name"] for function in payload["functions"]] == ["run"] - - -def test_attach_preprocessing_recipe_filters_invalid_and_duplicate_macros(): - empty = c_sources.CFile() - c_sources.attach_preprocessing_recipe(empty, None) - assert empty.preprocessing_recipe is None - - parsed = c_sources.CFile( - macros=[ - c_sources.CMacro( - name="EXISTING", - source_location=c_sources.CSourceLocation(filename="api.h", line=2), - ) - ] - ) - recipe = { - "macros": [ - None, - {"name": ""}, - {"name": "EXISTING", "path": "api.h", "line": 2}, - {"name": "NEW", "value": 123, "function_like": 1, "path": 42, "line": "bad"}, - {"name": "WITH_LOC", "value": "1", "path": "api.h", "line": 4}, - ] - } - - c_sources.attach_preprocessing_recipe(parsed, recipe) - - assert parsed.preprocessing_recipe == recipe - assert [macro.name for macro in parsed.macros] == ["EXISTING", "NEW", "WITH_LOC"] - assert parsed.macros[1].value is None - assert parsed.macros[1].function_like is True - assert parsed.macros[1].source_location.filename is None - assert parsed.macros[2].source_location.line == 4 - - -def test_c_parser_cli_helpers_errors_and_module_entrypoint(monkeypatch, capsys): - monkeypatch.setenv("C_PARSER_TEST_FLAG", " on ") - assert c_parser_cli._env_flag("C_PARSER_TEST_FLAG") is True - monkeypatch.delenv("C_PARSER_TEST_FLAG") - assert c_parser_cli._env_flag("C_PARSER_TEST_FLAG") is False - - assert c_parser_cli._diagnostic_color_enabled(disabled=True) is False - monkeypatch.setenv("NO_COLOR", "1") - assert c_parser_cli._diagnostic_color_enabled(disabled=False) is False - monkeypatch.delenv("NO_COLOR") - assert c_parser_cli._diagnostic_color_enabled(disabled=False) is True - - def fail_parse(_paths): - raise CParseError("invalid", filename="bad.h", line_number=1, column=1, source_line="@@@") - - monkeypatch.setattr(c_parser_cli, "parse_c_report", fail_parse) - assert c_parser_cli.main(["bad.h", "--no-color"]) == 1 - assert "bad.h:1:1: error[CPARSE_ERROR]: invalid" in capsys.readouterr().err - - monkeypatch.setattr(c_parser_cli, "main", lambda _argv=None: 0) - monkeypatch.delitem(sys.modules, "prik.parsers.c.__main__", raising=False) - with pytest.raises(SystemExit) as exc_info: - runpy.run_module("prik.parsers.c.__main__", run_name="__main__") - assert exc_info.value.code == 0 - - -def test_cli_c_parse_json_out_writes_file_and_suppresses_stdout(tmp_path: Path): - header = tmp_path / "api.h" - output = tmp_path / "report.json" - header.write_text("double scale(double x);\n", encoding="utf-8") - cmd = [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--json", - "--out", - str(output), - ] - - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - payload = json.loads(output.read_text(encoding="utf-8")) - - assert res.stdout == "" - assert payload[str(header)]["language"] == "c" - assert "parser_status" not in payload[str(header)] - - -def test_cli_c_parse_out_with_json_writes_json_and_suppresses_stdout(tmp_path: Path): - header = tmp_path / "api.h" - output = tmp_path / "report.json" - header.write_text("int run(void);\n", encoding="utf-8") - cmd = [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--json", - "--out", - str(output), - ] - - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - payload = json.loads(output.read_text(encoding="utf-8")) - - assert res.stdout == "" - assert "parser_status" not in payload[str(header)] - - -def test_cli_c_semantics_stdout_for_header(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("int add(int a, int b);\n", encoding="utf-8") - cmd = [sys.executable, "-m", "prik", "semantics", str(header), "--language", "c"] - - summary = subprocess.run(cmd, capture_output=True, text=True, check=True) - assert summary.stdout.startswith(f"File: {header}") - assert "- add(a: Int32 in, b: Int32 in) -> Int32" in summary.stdout - - res = subprocess.run([*cmd, "--json"], capture_output=True, text=True, check=True) - payload = json.loads(res.stdout) - semantic_modules = payload[str(header)]["semantic_modules"] - - assert semantic_modules[0]["name"] == "api" - assert semantic_modules[0]["functions"][0]["name"] == "add" - argument_type = semantic_modules[0]["functions"][0]["arguments"][0]["semantic_type"] - assert argument_type["name"] == "Int" - assert argument_type["dtype"] == "Int32" - assert argument_type["metadata"]["c_type_fact_source"] == "compiler_probe" - - -def test_cli_c_pyi_human_output_for_header(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("int add(int a, int b);\n", encoding="utf-8") - cmd = [sys.executable, "-m", "prik", "generate", "--pyi", str(header), "--language", "c"] - - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert f"File: {header}" in res.stdout - assert "def add(" in res.stdout - assert "a: Int" in res.stdout - - -def test_cli_c_pyi_out_requires_explicit_language_and_writes_when_selected(tmp_path: Path): - header = tmp_path / "api.h" - output = tmp_path / "api.pyi" - header.write_text("int add(int a, int b);\n", encoding="utf-8") - - omitted = subprocess.run( - [sys.executable, "-m", "prik", "generate", "--pyi", str(header), "--out"], - capture_output=True, - text=True, - ) - assert omitted.returncode == 2 - assert "usage:" in omitted.stderr - assert "requires explicit --language c" in omitted.stderr - assert not output.exists() - - selected = subprocess.run( - [sys.executable, "-m", "prik", "generate", "--pyi", str(header), "--out", "--language", "c"], - capture_output=True, - text=True, - check=True, - ) - assert selected.stdout == "" - assert "def add(" in output.read_text(encoding="utf-8") - - -def test_cli_c_pyi_out_writes_explicit_multi_header_owner_stubs(tmp_path: Path): - types = tmp_path / "types.h" - api = tmp_path / "api.h" - types.write_text("struct state { int id; };\n", encoding="utf-8") - api.write_text("struct state;\nvoid step(struct state *state);\n", encoding="utf-8") - - result = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--pyi", - str(types), - str(api), - "--language", - "c", - "--out", - ], - capture_output=True, - text=True, - check=True, - ) - - assert result.stdout == "" - assert "class state(CStruct):" in (tmp_path / "types.pyi").read_text(encoding="utf-8") - api_stub = (tmp_path / "api.pyi").read_text(encoding="utf-8") - assert "from .types import state" in api_stub - assert "class state" not in api_stub - assert "state: state" in api_stub - assert "Addr(state)" not in api_stub - - -def test_cli_c_input_rejects_explicit_fortran_frontend(tmp_path: Path): - header = tmp_path / "api.h" - output = tmp_path / "api.pyi" - header.write_text("int add(int a, int b);\n", encoding="utf-8") - - result = subprocess.run( - [sys.executable, "-m", "prik", "generate", "--pyi", str(header), "--language", "fortran", "--out"], - capture_output=True, - text=True, - ) - - assert result.returncode == 2 - assert "incompatible with --language fortran" in result.stderr - assert "pass --language c" in result.stderr - assert not output.exists() - - -def test_cli_c_pyi_rejects_invalid_c_syntax(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text( - "int add(int a, int b);\nvalue_type :: state;\n", - encoding="utf-8", - ) - result = subprocess.run( - [sys.executable, "-m", "prik", "generate", "--pyi", str(header), "--language", "c"], - capture_output=True, - text=True, - ) - - assert result.returncode == 1 - assert "CPARSE_INVALID_SYNTAX" in result.stderr - assert "Invalid C syntax" in result.stderr - - -def test_cli_c_rejects_fortran_only_parse_flags(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("int add(int a, int b);\n", encoding="utf-8") - cmd = [sys.executable, "-m", "prik", "parse", str(header), "--language", "c", "--show-vars"] - - res = subprocess.run(cmd, capture_output=True, text=True) - - assert res.returncode != 0 - assert "show-vars" in res.stderr - assert "Fortran-only" in res.stderr - - -def test_cli_c_no_color_and_debug_flags_are_accepted(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("int run(void);\n", encoding="utf-8") - cmd = [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--no-color", - "--debug", - ] - - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert "Parser status" not in res.stdout - - -def test_cli_c_no_color_and_no_color_env_format_parse_errors_without_ansi(tmp_path: Path): - source = tmp_path / "old_style.c" - source.write_text( - """ -int add(a, b) -int a; -int b; -{ - return a + b; -} -""", - encoding="utf-8", - ) - - base_cmd = [sys.executable, "-m", "prik", "parse", str(source), "--language", "c"] - no_color_res = subprocess.run( - [*base_cmd, "--no-color"], - capture_output=True, - text=True, - ) - env = {**os.environ, "NO_COLOR": "1"} - env_res = subprocess.run(base_cmd, capture_output=True, text=True, env=env) - - assert no_color_res.returncode == 1 - assert "K&R style function definitions are not supported" in no_color_res.stderr - assert "\x1b[" not in no_color_res.stderr - assert env_res.returncode == 1 - assert "K&R style function definitions are not supported" in env_res.stderr - assert "\x1b[" not in env_res.stderr - - -def test_cli_c_invalid_primitive_specifier_sequence_is_fatal(tmp_path: Path): - header = tmp_path / "invalid_specifiers.h" - header.write_text("unsigned float value;\n", encoding="utf-8") - cmd = [sys.executable, "-m", "prik", "parse", str(header), "--language", "c", "--no-color"] - - res = subprocess.run(cmd, capture_output=True, text=True) - - assert res.returncode == 1 - assert "error[CPARSE_INVALID_SPECIFIER_SEQUENCE]: Invalid type specifier sequence 'unsigned float'." in res.stderr - assert "\x1b[" not in res.stderr - - -def test_cli_c_debug_reraises_parse_errors(tmp_path: Path): - header = tmp_path / "invalid_specifiers.h" - header.write_text("unsigned float value;\n", encoding="utf-8") - cmd = [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--debug", - ] - - res = subprocess.run(cmd, capture_output=True, text=True) - - assert res.returncode == 1 - assert "Traceback" in res.stderr - assert "CParseError" in res.stderr - - -def test_cli_c_debug_env_reraises_parse_errors(tmp_path: Path): - header = tmp_path / "invalid_specifiers.h" - header.write_text("unsigned float value;\n", encoding="utf-8") - cmd = [sys.executable, "-m", "prik", "parse", str(header), "--language", "c"] - - res = subprocess.run( - cmd, - capture_output=True, - text=True, - env={**os.environ, "C_PARSER_DEBUG": "1"}, - ) - - assert res.returncode == 1 - assert "Traceback" in res.stderr - assert "CParseError" in res.stderr - - -def test_c_parser_cli_module_handles_directory_loader_and_output_modes(tmp_path: Path, capsys): - header = tmp_path / "api.h" - output = tmp_path / "c-report.json" - header.write_text("int add(int a, int b);\n", encoding="utf-8") - (tmp_path / "ignored.txt").write_text("ignored\n", encoding="utf-8") - - assert c_parser_cli.expand_c_paths([str(tmp_path), str(header)]) == [header] - loaded = c_parser_cli.parse_c_report([str(header)]) - assert loaded[str(header)]["functions"][0]["name"] == "add" - - assert c_parser_cli.main([str(header)]) == 0 - assert "Functions: 1" in capsys.readouterr().out - - assert c_parser_cli.main([str(header), "--json"]) == 0 - assert json.loads(capsys.readouterr().out)[str(header)]["functions"][0]["name"] == "add" - - assert c_parser_cli.main([str(header), "--out", str(output)]) == 0 - assert capsys.readouterr().out == "" - assert "parser_status" not in json.loads(output.read_text(encoding="utf-8"))[str(header)] - - -def test_c_parser_module_entrypoint_and_exports(tmp_path: Path): - import prik.parsers.c.__main__ as c_module_entrypoint - from prik.parsers.c.parser import parse_c_project - - header = tmp_path / "api.h" - header.write_text("int run(void);\n", encoding="utf-8") - result = subprocess.run( - [sys.executable, "-m", "prik.parsers.c", str(header), "--json"], - capture_output=True, - text=True, - check=True, - ) - - assert json.loads(result.stdout)[str(header)]["functions"][0]["name"] == "run" - assert c_module_entrypoint.main is c_parser_cli.main - assert parse_c_project({"api.h": "int run(void);\n"}).functions["run"].name == "run" - - -def test_c_parser_module_formats_parse_errors_without_traceback(tmp_path: Path): - header = tmp_path / "invalid.h" - header.write_text("@@@;\n", encoding="utf-8") - - result = subprocess.run( - [sys.executable, "-m", "prik.parsers.c", str(header), "--no-color"], - capture_output=True, - text=True, - ) - - assert result.returncode == 1 - assert "error[CPARSE_INVALID_SYNTAX]" in result.stderr - assert "Traceback" not in result.stderr - - -def test_c_parser_module_debug_reraises_parse_errors(tmp_path: Path): - header = tmp_path / "invalid.h" - header.write_text("@@@;\n", encoding="utf-8") - - result = subprocess.run( - [sys.executable, "-m", "prik.parsers.c", str(header), "--debug"], - capture_output=True, - text=True, - ) - - assert result.returncode == 1 - assert "Traceback" in result.stderr - assert "CParseError" in result.stderr - - -def test_prik_c_compiler_source_loader_drives_semantics_and_pyi(tmp_path: Path, monkeypatch): - header = tmp_path / "api.h" - header.write_text("API(int) add(int a, int b);\n", encoding="utf-8") - calls: list[Path] = [] - - def preprocess(path, *, language, config): - calls.append(path) - assert language == "c" - assert config.compiler == "cc" - return ( - "int add(int a, int b);\n", - SimpleNamespace(to_dict=lambda: {"mode": "compiler", "compiler": config.compiler}), - ) - - monkeypatch.setattr(c_sources, "run_compiler_preprocessor_with_recipe", preprocess) - config = PreprocessingConfig(mode="compiler", compiler="cc") - - semantics = prik_cli._semantic_report([str(header)], config, language="c") - assert semantics[str(header)]["semantic_modules"][0]["functions"][0]["name"] == "add" - assert calls == [header] diff --git a/tests/c/infrastructure/cli/pipeline/test_c_cli_stage_dispatch.py b/tests/c/infrastructure/cli/pipeline/test_c_cli_stage_dispatch.py index 953fcbe65..fab639c39 100644 --- a/tests/c/infrastructure/cli/pipeline/test_c_cli_stage_dispatch.py +++ b/tests/c/infrastructure/cli/pipeline/test_c_cli_stage_dispatch.py @@ -1,196 +1,123 @@ -"""C input-language CLI dispatch contracts.""" +"""C input-language stage dispatch: parse-error presentation and the C parser module launcher.""" +import json from pathlib import Path +import subprocess +import sys import pytest import prik.cli as prik_cli -from prik.parsers.c import sources as c_sources -from tests.c._support.cli import ( - _install_main_parser, - _main_args, +from prik.parsers.c import CParseError +from prik.parsers.c import cli as c_parser_cli + +INVALID_SPECIFIERS = "unsigned float value;\n" +KNR_DEFINITION = "int add(a, b)\nint a;\nint b;\n{\n return a + b;\n}\n" +FOREIGN_SYNTAX = "int add(int a, int b);\nvalue_type :: state;\n" + + +@pytest.mark.parametrize( + ("entrypoint", "argv", "source", "environment", "concise"), + [ + pytest.param( + "prik", + ["parse", "bad.h", "--language", "c"], + INVALID_SPECIFIERS, + {}, + ("\033[", "CPARSE_INVALID_SPECIFIER_SEQUENCE"), + id="prik-colored-by-default", + ), + pytest.param( + "prik", + ["parse", "bad.h", "--language", "c", "--no-color"], + INVALID_SPECIFIERS, + {}, + ("error[CPARSE_INVALID_SPECIFIER_SEQUENCE]: Invalid type specifier sequence 'unsigned float'.",), + id="prik-no-color-flag", + ), + pytest.param( + "prik", + ["parse", "bad.h", "--language", "c"], + KNR_DEFINITION, + {"NO_COLOR": "1"}, + ("K&R style function definitions are not supported",), + id="prik-no-color-environment", + ), + pytest.param( + "prik", + ["generate", "--pyi", "bad.h", "--language", "c", "--no-color"], + FOREIGN_SYNTAX, + {}, + ("error[CPARSE_INVALID_SYNTAX]", "Invalid C syntax"), + id="prik-generate-pyi-invalid-syntax", + ), + pytest.param( + "prik", ["parse", "bad.h", "--language", "c", "--debug"], INVALID_SPECIFIERS, {}, None, id="prik-debug-flag" + ), + pytest.param( + "prik", + ["parse", "bad.h", "--language", "c"], + INVALID_SPECIFIERS, + {"C_PARSER_DEBUG": "1"}, + None, + id="prik-debug-environment", + ), + pytest.param( + "parser-module", + ["bad.h", "--no-color"], + "@@@;\n", + {}, + ("bad.h:1:1: error[CPARSE_INVALID_SYNTAX]",), + id="parser-module-concise", + ), + pytest.param("parser-module", ["bad.h", "--debug"], "@@@;\n", {}, None, id="parser-module-debug-flag"), + ], ) - - -def test_prik_main_preserves_c_parse_dispatch_contract(monkeypatch): - class StopAfterDispatch(Exception): - pass - - args = _main_args(language="requested", parse=True) - _install_main_parser(monkeypatch, args) - preprocessing = type("Preprocessing", (), {"include_dirs": ("include",)})() - parse_payload = {"parse": "payload"} - calls = [] - - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: "c") - monkeypatch.setattr( - prik_cli, - "_build_preprocessing_config", - lambda active_args, parser: preprocessing, - ) - monkeypatch.setattr( - prik_cli, - "parse_c_report", - lambda paths, active_preprocessing: calls.append(("parse", paths, active_preprocessing)) or parse_payload, - ) - monkeypatch.setattr( - prik_cli, - "_select_main_payload", - lambda *_args: (_ for _ in ()).throw(StopAfterDispatch), - ) - - with pytest.raises(StopAfterDispatch): - prik_cli.main() - - # The C parse report receives the one preprocessing configuration the CLI built. - assert calls == [("parse", args.paths, preprocessing)] - - -@pytest.mark.parametrize("stage", ["semantics", "pyi"]) -def test_prik_main_accepts_each_non_parse_c_stage(monkeypatch, stage): - class StopAfterDispatch(Exception): - pass - - args = _main_args(language="c", **{stage: True}) - _install_main_parser(monkeypatch, args) - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: language) - monkeypatch.setattr( - prik_cli, - "_build_preprocessing_config", - lambda active_args, parser: object(), - ) - monkeypatch.setattr(prik_cli, "_semantic_report", lambda *args, **kwargs: {}) - monkeypatch.setattr( - prik_cli, - "_select_main_payload", - lambda *_args: (_ for _ in ()).throw(StopAfterDispatch), - ) - - with pytest.raises(StopAfterDispatch): - prik_cli.main() - - -def test_one_c_parse_preserves_parser_and_preprocessing_arguments( - tmp_path: Path, - monkeypatch, +def test_c_parse_errors_are_concise_unless_debugging( + tmp_path: Path, monkeypatch, capsys, entrypoint, argv, source, environment, concise ): - """Every C route parses a path through parse_c_source, raw or compiler-preprocessed.""" - path = tmp_path / "api.h" - raw_parsed = object() - compiled_parsed = object() - - class RawParser: - def parse_file(self, source, *, filename, include_dirs, preprocessing): - assert source == path - assert filename == str(path) - assert include_dirs == ["include"] - assert preprocessing == "raw" - return raw_parsed - - raw_config = prik_cli.PreprocessingConfig(include_dirs=["include"]) - assert c_sources.parse_c_source(path, raw_config, parser=RawParser()) is raw_parsed - - class Recipe: - def to_dict(self): - return {"mode": "compiler"} - - def preprocess(received_path, *, language, config): - assert received_path == path - assert language == "c" - assert config is compiler_config - return "int add(int x);\n", Recipe() - - class CompilerParser: - def parse_file(self, source, *, filename, include_dirs, preprocessing): - assert source == "int add(int x);\n" - assert filename == str(path) - assert include_dirs == ["include"] - assert preprocessing == "compiler" - return compiled_parsed - - def attach_recipe(parsed, recipe): - assert parsed is compiled_parsed - assert recipe == {"mode": "compiler"} - - compiler_config = prik_cli.PreprocessingConfig( - mode="compiler", - compiler="cc", - include_dirs=["include"], - ) - monkeypatch.setattr(c_sources, "run_compiler_preprocessor_with_recipe", preprocess) - monkeypatch.setattr(c_sources, "attach_preprocessing_recipe", attach_recipe) - - assert c_sources.parse_c_source(path, compiler_config, parser=CompilerParser()) is compiled_parsed - - -def test_prik_main_preserves_c_parse_error_rendering_contract(monkeypatch, capsys): - args = _main_args(language="c", parse=True, no_color=True) - _install_main_parser(monkeypatch, args) - preprocessing = type("Preprocessing", (), {"include_dirs": ()})() - error = prik_cli.CParseError("bad parse") - calls = [] - - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: "c") - monkeypatch.setattr( - prik_cli, - "_build_preprocessing_config", - lambda active_args, parser: preprocessing, - ) - monkeypatch.setattr( - prik_cli, - "_env_flag", - lambda name: calls.append(("env", name)) or False, - ) - monkeypatch.setattr( - prik_cli, - "_diagnostic_color_enabled", - lambda *, disabled: calls.append(("color", disabled)) or "color-enabled", - ) - monkeypatch.setattr( - prik_cli.CParseError, - "format_diagnostic", - lambda self, *, color, debug: calls.append(("render", color, debug)) or "rendered diagnostic", + monkeypatch.chdir(tmp_path) + (tmp_path / "bad.h").write_text(source, encoding="utf-8") + for name in ("NO_COLOR", "C_PARSER_DEBUG", "PRIK_DEBUG"): + monkeypatch.delenv(name, raising=False) + for name, value in environment.items(): + monkeypatch.setenv(name, value) + main = prik_cli.main if entrypoint == "prik" else c_parser_cli.main + + if concise is None: + with pytest.raises(CParseError): + main(argv) + return + + assert main(argv) == 1 + captured = capsys.readouterr() + assert captured.out == "" + assert [fragment for fragment in concise if fragment not in captured.err] == [] + assert ("\033[" in captured.err) is ("NO_COLOR" not in environment and "--no-color" not in argv) + + +def test_c_parser_module_launcher_reports_every_mode(tmp_path: Path, capsys): + """``python -m prik.parsers.c`` prints the parse report, JSON, or a JSON file for C inputs only.""" + header = tmp_path / "api.h" + output = tmp_path / "c-report.json" + header.write_text("int add(int a, int b);\n", encoding="utf-8") + (tmp_path / "ignored.txt").write_text("ignored\n", encoding="utf-8") + + launched = subprocess.run( + [sys.executable, "-m", "prik.parsers.c", str(header), "--json"], + capture_output=True, + text=True, + check=True, ) - monkeypatch.setattr( - prik_cli, - "parse_c_report", - lambda *args, **kwargs: (_ for _ in ()).throw(error), - ) - - assert prik_cli.main() == 1 - assert capsys.readouterr().err == "rendered diagnostic\n" - assert calls == [ - ("env", "C_PARSER_DEBUG"), - ("color", True), - ("render", "color-enabled", False), - ] - - -def test_prik_main_reraises_c_parse_errors_for_debug_environment(monkeypatch): - args = _main_args(language="c", parse=True) - _install_main_parser(monkeypatch, args) - preprocessing = type("Preprocessing", (), {"include_dirs": ()})() - error = prik_cli.CParseError("bad parse") - calls = [] - - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: "c") - monkeypatch.setattr( - prik_cli, - "_build_preprocessing_config", - lambda active_args, parser: preprocessing, - ) - monkeypatch.setattr( - prik_cli, - "_env_flag", - lambda name: calls.append(name) or name == "C_PARSER_DEBUG", - ) - monkeypatch.setattr( - prik_cli, - "parse_c_report", - lambda *args, **kwargs: (_ for _ in ()).throw(error), - ) - - with pytest.raises(prik_cli.CParseError): - prik_cli.main() - - assert calls == ["C_PARSER_DEBUG"] + assert json.loads(launched.stdout)[str(header)]["functions"][0]["name"] == "add" + + # A directory expands to its C inputs; other files in it are ignored. + assert c_parser_cli.main([str(tmp_path)]) == 0 + report = capsys.readouterr().out + assert f"File: {header}" in report + assert "Functions: 1" in report + assert "ignored.txt" not in report + + assert c_parser_cli.main([str(header), "--out", str(output)]) == 0 + assert capsys.readouterr().out == "" + assert json.loads(output.read_text(encoding="utf-8"))[str(header)]["language"] == "c" diff --git a/tests/c/infrastructure/parsing/test_c_corpus.py b/tests/c/infrastructure/parsing/test_c_corpus.py deleted file mode 100644 index 1ebe04adf..000000000 --- a/tests/c/infrastructure/parsing/test_c_corpus.py +++ /dev/null @@ -1,90 +0,0 @@ -"""Active cJSON parser regression tests. - -cJSON is the first target corpus because it is small, realistic, and exercises -headers, typedef structs, recursive pointers, function declarations, macros, -constants, and callback hook fields without requiring a large build system. -""" - -import shutil - -import pytest -from tests.c._support.paths import C_DATA_DIR - -_CJSON_DIR = C_DATA_DIR / "json" - - -def _preprocessed_cjson_source(filename: str) -> str: - from prik.preprocessing import PreprocessingConfig, preprocess_source - - compiler = shutil.which("cc") - if compiler is None: - pytest.skip("cc is not available") - return preprocess_source( - _CJSON_DIR / filename, - language="c", - config=PreprocessingConfig(mode="compiler", compiler=compiler), - ).source - - -def test_cjson_regression_source_and_header_are_available(): - assert (_CJSON_DIR / "cJSON.h").exists() - assert (_CJSON_DIR / "cJSON.c").exists() - - -def test_cjson_header_raw_parse_requires_preprocessing(): - import pytest - - from prik.parsers.c import CParseError, parse_c_file - - with pytest.raises(CParseError, match="require compiler preprocessing") as exc_info: - parse_c_file(_CJSON_DIR / "cJSON.h") - assert exc_info.value.code == "CPARSE_PREPROCESSING_REQUIRED" - - -def test_cjson_header_preprocessed_mode_has_no_error_diagnostics(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - _preprocessed_cjson_source("cJSON.h"), - filename=str(_CJSON_DIR / "cJSON.h"), - preprocessing="compiler", - ) - - assert not any(diag.severity == "error" for diag in parsed.diagnostics) - - -def test_cjson_callback_hook_declarations_are_preprocessed_without_error_diagnostics(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - _preprocessed_cjson_source("cJSON.h"), - filename=str(_CJSON_DIR / "cJSON.h"), - preprocessing="compiler", - ) - - assert any(struct.name == "cJSON_Hooks" for struct in parsed.structs) - assert not any(diag.severity == "error" for diag in parsed.diagnostics) - - -def test_cjson_source_file_parse_skips_function_bodies_safely(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - _preprocessed_cjson_source("cJSON.c"), - filename=str(_CJSON_DIR / "cJSON.c"), - preprocessing="compiler", - ) - - assert any(fn.name == "parse_number" for fn in parsed.functions) - assert not any(hasattr(fn, "body") for fn in parsed.functions) - - -def test_cjson_project_parse_links_header_and_source(): - from prik.parsers.c import parse_c_project - - sources = {filename: _preprocessed_cjson_source(filename) for filename in ("cJSON.h", "cJSON.c")} - project = parse_c_project(sources, preprocessing="compiler") - - assert "cJSON.h" in project.files - assert "cJSON.c" in project.files - assert project.header_source_pairs["cJSON.h"] == {"cJSON.c"} diff --git a/tests/c/infrastructure/parsing/test_c_declarations_and_declarators.py b/tests/c/infrastructure/parsing/test_c_declarations_and_declarators.py index afb0434dc..3795b9789 100644 --- a/tests/c/infrastructure/parsing/test_c_declarations_and_declarators.py +++ b/tests/c/infrastructure/parsing/test_c_declarations_and_declarators.py @@ -3,126 +3,6 @@ import pytest -def test_primitive_specifiers_create_concrete_primitive_types(): - from prik.parsers.c import CBool, CShort, CUnsignedLongLong, parse_c_file - - parsed = parse_c_file( - """ -unsigned long long next_id(void); -signed short clamp_short(signed short value); -_Bool enabled(void); -""", - filename="primitives.h", - ) - - functions = {function.name: function for function in parsed.functions} - assert isinstance(functions["next_id"].result_type, CUnsignedLongLong) - assert isinstance(functions["clamp_short"].parameters[0].type, CShort) - assert isinstance(functions["enabled"].result_type, CBool) - - -@pytest.mark.parametrize( - ("spelling", "expected_name"), - [ - ("void", "CVoid"), - ("_Bool", "CBool"), - ("char", "CChar"), - ("signed char", "CSignedChar"), - ("unsigned char", "CUnsignedChar"), - ("short", "CShort"), - ("short int", "CShort"), - ("signed short", "CShort"), - ("signed short int", "CShort"), - ("unsigned short", "CUnsignedShort"), - ("unsigned short int", "CUnsignedShort"), - ("int", "CInt"), - ("signed", "CInt"), - ("signed int", "CInt"), - ("unsigned", "CUnsignedInt"), - ("unsigned int", "CUnsignedInt"), - ("long", "CLong"), - ("long int", "CLong"), - ("signed long", "CLong"), - ("signed long int", "CLong"), - ("unsigned long", "CUnsignedLong"), - ("unsigned long int", "CUnsignedLong"), - ("long long", "CLongLong"), - ("long long int", "CLongLong"), - ("signed long long", "CLongLong"), - ("signed long long int", "CLongLong"), - ("unsigned long long", "CUnsignedLongLong"), - ("unsigned long long int", "CUnsignedLongLong"), - ("float", "CFloat"), - ("double", "CDouble"), - ("long double", "CLongDouble"), - ("float _Complex", "CFloatComplex"), - ("_Complex", "CDoubleComplex"), - ("double _Complex", "CDoubleComplex"), - ("long double _Complex", "CLongDoubleComplex"), - ], -) -def test_every_supported_primitive_spelling_creates_a_concrete_ctype(spelling, expected_name): - import prik.parsers.c as c_parser - from prik.parsers.c import CType, parse_c_file - - function = parse_c_file(f"{spelling} primitive(void);\n", filename="primitive_table.h").functions[0] - expected = getattr(c_parser, expected_name) - - assert isinstance(function.result_type, expected) - assert isinstance(function.result_type, CType) - - -@pytest.mark.parametrize( - ("spelling", "expected_name"), - [ - ("int unsigned", "CUnsignedInt"), - ("int long unsigned", "CUnsignedLong"), - ("double long", "CLongDouble"), - ("_Complex float", "CFloatComplex"), - ], -) -def test_valid_reordered_primitive_specifiers_are_normalized(spelling, expected_name): - import prik.parsers.c as c_parser - from prik.parsers.c import parse_c_file - - function = parse_c_file(f"{spelling} primitive(void);\n", filename="reordered_primitives.h").functions[0] - - assert isinstance(function.result_type, getattr(c_parser, expected_name)) - assert function.result_type.source_text == spelling - - -@pytest.mark.parametrize( - ("source", "expected_column"), - [ - ("unsigned float value;\n", 1), - ("void bad(long char value);\n", 1), - ("struct bad { signed unsigned value; };\n", 14), - ("unsigned float bad(void) { return 0; }\n", 1), - ], -) -def test_invalid_primitive_specifier_sequences_raise_parse_errors(source, expected_column): - from prik.parsers.c import CParseError, parse_c_file - - with pytest.raises(CParseError, match="Invalid type specifier sequence") as error: - parse_c_file(source, filename="invalid_specifiers.h") - - assert error.value.code == "CPARSE_INVALID_SPECIFIER_SEQUENCE" - assert ( - f"invalid_specifiers.h:1:{expected_column}: error[CPARSE_INVALID_SPECIFIER_SEQUENCE]" - in error.value.format_diagnostic(color=False) - ) - - -def test_unresolved_single_typedef_name_is_preserved_until_resolution(): - from prik.parsers.c import CTypedef, parse_c_file - - parsed = parse_c_file("external_type value;\n", filename="deferred_typedef.h") - - assert isinstance(parsed.variables[0].type, CTypedef) - assert parsed.variables[0].type.name == "external_type" - assert parsed.diagnostics == [] - - def test_pointer_qualifiers_belong_to_the_component_they_qualify(): from prik.parsers.c import CComposedType, CConst, CDouble, CPointer, CRestrict, parse_c_file @@ -235,77 +115,6 @@ def test_typedefs_and_typedef_references_are_concrete_types(): assert isinstance(parsed.functions[1].parameters[0].type, CTypedef) -def test_repeated_file_scope_tentative_variable_declarations_merge(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file("int i;\nint i;\n", filename="tentative.c") - - assert [variable.name for variable in parsed.variables] == ["i"] - assert parsed.variables[0].initializer is None - assert [location.line for location in parsed.variables[0].declaration_locations] == [2] - assert parsed.diagnostics == [] - - -def test_tentative_variable_declaration_followed_by_definition_prefers_definition(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file("int i;\nint i = 1;\n", filename="definition.c") - - assert [variable.name for variable in parsed.variables] == ["i"] - assert parsed.variables[0].initializer.source_text == "1" - assert parsed.variables[0].source_location.line == 2 - assert [location.line for location in parsed.variables[0].declaration_locations] == [1] - assert parsed.diagnostics == [] - - -def test_duplicate_initialized_file_scope_variables_report_diagnostic(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file("int i = 1;\nint i = 2;\n", filename="duplicate_variables.c") - - assert [variable.name for variable in parsed.variables] == ["i"] - assert parsed.variables[0].initializer.source_text == "1" - assert any(diag.code == "C_DUPLICATE_VARIABLE_DEFINITION" for diag in parsed.diagnostics) - - -def test_conflicting_file_scope_variable_declarations_report_diagnostic(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file("int i;\ndouble i;\n", filename="conflicting_variables.c") - - assert [variable.name for variable in parsed.variables] == ["i"] - assert any(diag.code == "C_CONFLICTING_VARIABLE_DECLARATION" for diag in parsed.diagnostics) - - -def test_type_key_preserves_seen_state_for_recursive_composed_types(): - from prik.parsers.c import CComposedType, CParser, CPointer, CTypedef - - typedef = CTypedef(name="node") - recursive = CComposedType(components=[CPointer(), typedef]) - typedef.type = recursive - - assert CParser()._type_key(recursive) == ( - "CComposedType", - ( - ("CPointer", ()), - ("CTypedef", ("cycle", "CComposedType", None), ()), - ), - (), - ) - - -def test_compatible_repeated_typedefs_merge_but_conflicting_typedefs_diagnose(): - from prik.parsers.c import parse_c_file - - compatible = parse_c_file("typedef int count_t;\ntypedef int count_t;\n", filename="typedefs.h") - conflicting = parse_c_file("typedef int count_t;\ntypedef double count_t;\n", filename="bad_typedefs.h") - - assert [typedef.name for typedef in compatible.typedefs] == ["count_t"] - assert [location.line for location in compatible.typedefs[0].declaration_locations] == [2] - assert compatible.diagnostics == [] - assert any(diag.code == "C_CONFLICTING_TYPEDEF" for diag in conflicting.diagnostics) - - def test_variables_preserve_initializer_text_arrays_and_concrete_tag_types(): from prik.parsers.c import CArray, CEnum, CInt, CStruct, CUnion, parse_c_file @@ -333,47 +142,6 @@ def test_variables_preserve_initializer_text_arrays_and_concrete_tag_types(): assert variables["answer"].initializer.source_text == "42" -def test_parameters_preserve_concrete_struct_union_and_enum_uses(): - from prik.parsers.c import CEnum, CStruct, CUnion, parse_c_file - - parsed = parse_c_file( - "void consume(const struct state *s, union scalar *u, enum status status);\n", - filename="tag_params.h", - ) - - params = {parameter.name: parameter for parameter in parsed.functions[0].parameters} - assert isinstance(params["s"].type.components[-1], CStruct) - assert params["s"].type.components[-1].name == "state" - assert isinstance(params["u"].type.components[-1], CUnion) - assert isinstance(params["status"].type, CEnum) - - -def test_incomplete_structs_and_pointer_uses_are_concrete_objects(): - from prik.parsers.c import CComposedType, CPointer, CStruct, parse_c_file - - parsed = parse_c_file( - """ -struct handle; -struct handle *open_handle(void); -void close_handle(struct handle *handle); -""", - filename="opaque.h", - ) - - handle = parsed.structs[0] - assert isinstance(handle, CStruct) - assert handle.name == "handle" - assert handle.is_incomplete is True - assert handle.members == [] - - functions = {function.name: function for function in parsed.functions} - result = functions["open_handle"].result_type - assert isinstance(result, CComposedType) - assert isinstance(result.components[0], CPointer) - assert isinstance(result.components[1], CStruct) - assert result.components[1].name == "handle" - - def test_storage_is_declaration_metadata_and_qualifiers_are_type_metadata(): from prik.parsers.c import CAtomic, CConst, CUnsignedLong, CVolatile, parse_c_file @@ -428,23 +196,6 @@ def test_atomic_type_specifier_qualifies_the_declared_outermost_type(): assert parsed.diagnostics == [] -@pytest.mark.parametrize( - ("source", "message"), - [ - ("_Atomic(int) long value;\n", "Invalid type specifier sequence"), - ("_Atomic() value;\n", "Invalid _Atomic type-name"), - ("_Atomic(int named) value;\n", "Invalid _Atomic type-name"), - ], -) -def test_invalid_atomic_type_specifiers_raise_focused_errors(source, message): - from prik.parsers.c import CParseError, parse_c_file - - with pytest.raises(CParseError, match=message) as exc_info: - parse_c_file(source, filename="invalid_atomic.h") - - assert exc_info.value.code == "CPARSE_INVALID_SPECIFIER_SEQUENCE" - - def test_function_bodies_do_not_contribute_local_variables(): from prik.parsers.c import parse_c_file @@ -459,135 +210,6 @@ def test_function_bodies_do_not_contribute_local_variables(): assert [variable.name for variable in parsed.variables] == ["exported_value"] -def test_declarations_return_concrete_objects_instead_of_kind_fields(): - from prik.parsers.c import ( - CArray, - CFunction, - CFunctionType, - CInt, - CPointer, - CStruct, - CTypedef, - CVariable, - parse_c_file, - ) - - parsed = parse_c_file( - """ -struct handle; -typedef int (*compare_fn)(const void *, const void *); -extern int *values[4]; -extern int (*matrix)[4]; -int add(int a, int b); -void sort_items(int (*fallback)(const void *, const void *)); -""", - filename="declaration_matrix.h", - ) - - assert isinstance(parsed.structs[0], CStruct) - assert all(isinstance(typedef, CTypedef) for typedef in parsed.typedefs) - assert all(isinstance(variable, CVariable) for variable in parsed.variables) - assert all(isinstance(function, CFunction) for function in parsed.functions) - - compare = parsed.typedefs[0].type - assert [type(component) for component in compare.components] == [CPointer, CFunctionType] - values, matrix = parsed.variables - assert [type(component) for component in values.type.components] == [CArray, CPointer, CInt] - assert [type(component) for component in matrix.type.components] == [CPointer, CArray, CInt] - assert parsed.functions[1].parameters[0].callback_candidate is True - - -def test_composite_definitions_are_concrete_objects_and_static_assert_is_diagnostic(): - from prik.parsers.c import CEnum, CStruct, CUnion, CVariable, parse_c_file - - parsed = parse_c_file( - """ -struct point { double x; double y; }; -union value { int i; double d; }; -enum status { STATUS_OK = 0 }; -_Static_assert(sizeof(int) == 4, "expected int width"); -""", - filename="composites.h", - ) - - assert isinstance(parsed.structs[0], CStruct) - assert all(isinstance(member, CVariable) for member in parsed.structs[0].members) - assert [member.name for member in parsed.structs[0].members] == ["x", "y"] - assert isinstance(parsed.unions[0], CUnion) - assert isinstance(parsed.enums[0], CEnum) - assert [diagnostic.unit_kind for diagnostic in parsed.diagnostics] == ["static_assert"] - - -def test_parenthesized_declarators_preserve_pointer_array_order(): - from prik.parsers.c import CArray, CInt, CPointer, parse_c_file - - parsed = parse_c_file("extern int *values[4];\nextern int (*matrix)[4];\n", filename="paren_decl.h") - variables = {variable.name: variable for variable in parsed.variables} - - assert [type(component) for component in variables["values"].type.components] == [CArray, CPointer, CInt] - assert [type(component) for component in variables["matrix"].type.components] == [CPointer, CArray, CInt] - - -def test_function_type_discards_placeholder_parameter_names(): - from prik.parsers.c import CFunctionType, CPointer, parse_c_file - - parsed = parse_c_file( - "typedef int (*compare_fn)(const void *left, const void *right);\n", - filename="callback_typedef.h", - ) - - type_ = parsed.typedefs[0].type - assert isinstance(type_.components[0], CPointer) - signature = type_.components[1] - assert isinstance(signature, CFunctionType) - assert len(signature.parameter_types) == 2 - - -def test_conflicting_function_pointer_typedefs_report_diagnostic(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - "typedef int (*callback_fn)(int);\ntypedef double (*callback_fn)(double);\n", - filename="callback_typedef_conflict.h", - ) - - assert [typedef.name for typedef in parsed.typedefs] == ["callback_fn"] - assert [(diagnostic.code, diagnostic.unit_kind, diagnostic.unit_name) for diagnostic in parsed.diagnostics] == [ - ("C_CONFLICTING_TYPEDEF", "typedef", "callback_fn") - ] - - -def test_recursive_compositions_cover_tables_callback_arrays_and_function_results(): - from prik.parsers.c import CArray, CFunctionType, CInt, CPointer, parse_c_file - - parsed = parse_c_file( - """ -extern int *(*table)[4]; -typedef int (*callback_table[8])(int); -int (*factory(void))(int); -int direct(void), *value; -""", - filename="recursive_declarators.h", - ) - - variables = {variable.name: variable for variable in parsed.variables} - assert [type(component) for component in variables["table"].type.components] == [ - CPointer, - CArray, - CPointer, - CInt, - ] - assert [type(component) for component in variables["value"].type.components] == [CPointer, CInt] - callbacks = parsed.typedefs[0].type - assert [type(component) for component in callbacks.components] == [CArray, CPointer, CFunctionType] - functions = {function.name: function for function in parsed.functions} - assert set(functions) == {"factory", "direct"} - assert [type(component) for component in functions["factory"].result_type.components] == [ - CPointer, - CFunctionType, - ] - - def test_declaration_attributes_are_tolerated_and_layout_omissions_are_diagnosed(): from prik.parsers.c import parse_c_file @@ -627,59 +249,6 @@ def test_unsupported_top_level_declarator_is_reported_with_source_location(): assert diagnostic.location.source_line == "int value @@;" -@pytest.mark.parametrize( - ("text", "unit_kind", "message"), - [ - ("struct pending { int value; }", "struct_definition", "Struct definitions are not supported yet."), - ("union pending { int value; }", "union_definition", "Union definitions are not supported yet."), - ("enum pending { value }", "enum_definition", "Enum definitions are not supported yet."), - ("_Static_assert(sizeof(int) == 4)", "static_assert", "Static assertions are recorded but not evaluated."), - ("int value __attribute__((used))", "attribute_declaration", "Compiler-specific declaration attributes"), - ("int value __declspec(dllexport)", "attribute_declaration", "Compiler-specific declaration attributes"), - ("int value [[deprecated]]", "attribute_declaration", "Compiler-specific declaration attributes"), - ("_Alignas(16) int value", "alignment_declaration", "Declaration alignment specifiers"), - ("alignas(16) int value", "alignment_declaration", "Declaration alignment specifiers"), - ("int values[] = {1, 2, 3}", "brace_declaration", "Unsupported declaration containing braces."), - ], -) -def test_unsupported_declaration_diagnostic_classifies_known_shapes(text, unit_kind, message): - from prik.parsers.c import CParser - from prik.parsers.c.lexer import CTopLevelSegment - - segment = CTopLevelSegment( - text=text, - terminator=";", - filename="unsupported.h", - original_start_line=7, - original_start_column=3, - original_source_line=f" {text};", - ) - - diagnostic = CParser()._unsupported_declaration_diagnostic(segment) - - assert diagnostic is not None - assert diagnostic.code == "C_UNSUPPORTED_DECLARATION" - assert diagnostic.severity == "warning" - assert diagnostic.unit_kind == unit_kind - assert diagnostic.unit_name is None - assert message in diagnostic.message - assert diagnostic.location is not None - assert diagnostic.location.filename == "unsupported.h" - assert diagnostic.location.line == 7 - assert diagnostic.location.column == 3 - assert diagnostic.location.source_line == f" {text};" - - -def test_unsupported_declaration_diagnostic_ignores_empty_and_plain_declarations(): - from prik.parsers.c import CParser - from prik.parsers.c.lexer import CTopLevelSegment - - parser = CParser() - - assert parser._unsupported_declaration_diagnostic(CTopLevelSegment(text="", terminator=";")) is None - assert parser._unsupported_declaration_diagnostic(CTopLevelSegment(text="int value", terminator=";")) is None - - @pytest.mark.parametrize( "source", [ @@ -697,24 +266,6 @@ def test_non_c_top_level_grammar_is_rejected_without_language_guessing(source): assert exc_info.value.code == "CPARSE_INVALID_SYNTAX" -@pytest.mark.parametrize( - ("source", "name", "type_name"), - [ - ("class widget;\n", "widget", "class"), - ("namespace api = other;\n", "api", "namespace"), - ("using size_type = value;\n", "size_type", "using"), - ], -) -def test_identifier_spelling_does_not_trigger_foreign_language_detection(source, name, type_name): - from prik.parsers.c import CTypedef, parse_c_file - - parsed = parse_c_file(source, filename="identifier_spelling.h") - - assert [variable.name for variable in parsed.variables] == [name] - assert isinstance(parsed.variables[0].type, CTypedef) - assert parsed.variables[0].type.name == type_name - - def test_braced_and_designated_initializer_declarations_preserve_source_text(): from prik.parsers.c import CArray, CComposedType, parse_c_file @@ -762,3 +313,246 @@ def test_storage_class_and_inline_specifiers_are_recorded_on_functions(): assert functions["local_add"].storage == ["static"] assert "inline" in functions["local_add"].specifiers assert functions["exported_add"].storage == ["extern"] + + +@pytest.mark.parametrize( + ("spelling", "expected_name"), + [ + pytest.param("void", "CVoid", id="void"), + pytest.param("_Bool", "CBool", id="bool"), + pytest.param("char", "CChar", id="plain-char"), + pytest.param("signed char", "CSignedChar", id="signed-char-is-distinct"), + pytest.param("unsigned char", "CUnsignedChar", id="unsigned-char"), + pytest.param("signed short int", "CShort", id="short-with-sign-and-int"), + pytest.param("unsigned short", "CUnsignedShort", id="unsigned-short"), + pytest.param("signed", "CInt", id="bare-signed-is-int"), + pytest.param("unsigned", "CUnsignedInt", id="bare-unsigned-is-unsigned-int"), + pytest.param("long int", "CLong", id="long-int"), + pytest.param("unsigned long", "CUnsignedLong", id="unsigned-long"), + pytest.param("signed long long int", "CLongLong", id="long-long-counts-two-longs"), + pytest.param("unsigned long long", "CUnsignedLongLong", id="unsigned-long-long"), + pytest.param("float", "CFloat", id="float"), + pytest.param("double", "CDouble", id="double"), + pytest.param("long double", "CLongDouble", id="long-double"), + pytest.param("float _Complex", "CFloatComplex", id="float-complex"), + pytest.param("_Complex", "CDoubleComplex", id="bare-complex-is-double"), + pytest.param("long double _Complex", "CLongDoubleComplex", id="long-double-complex"), + pytest.param("int unsigned", "CUnsignedInt", id="reordered-int-unsigned"), + pytest.param("int long unsigned", "CUnsignedLong", id="reordered-int-long-unsigned"), + pytest.param("double long", "CLongDouble", id="reordered-double-long"), + pytest.param("_Complex float", "CFloatComplex", id="reordered-complex-float"), + ], +) +def test_primitive_specifier_spellings_create_one_concrete_ctype(spelling, expected_name): + import prik.parsers.c as c_parser + from prik.parsers.c import parse_c_file + + function = parse_c_file(f"{spelling} primitive(void);\n", filename="primitive_table.h").functions[0] + + assert isinstance(function.result_type, getattr(c_parser, expected_name)) + # The source spelling survives normalization, whatever the specifier order. + assert function.result_type.source_text == spelling + + +@pytest.mark.parametrize( + ("source", "message", "expected_column"), + [ + pytest.param("unsigned float value;\n", "Invalid type specifier sequence", 1, id="variable"), + pytest.param("void bad(long char value);\n", "Invalid type specifier sequence", 1, id="parameter"), + pytest.param( + "struct bad { signed unsigned value; };\n", "Invalid type specifier sequence", 14, id="struct-member" + ), + pytest.param("unsigned float bad(void) { return 0; }\n", "Invalid type specifier sequence", 1, id="definition"), + pytest.param("_Atomic(int) long value;\n", "Invalid type specifier sequence", None, id="atomic-plus-long"), + pytest.param("_Atomic() value;\n", "Invalid _Atomic type-name", None, id="empty-atomic"), + pytest.param("_Atomic(int named) value;\n", "Invalid _Atomic type-name", None, id="atomic-with-declarator"), + ], +) +def test_invalid_type_specifier_sequences_raise_located_parse_errors(source, message, expected_column): + from prik.parsers.c import CParseError, parse_c_file + + with pytest.raises(CParseError, match=message) as error: + parse_c_file(source, filename="invalid_specifiers.h") + + assert error.value.code == "CPARSE_INVALID_SPECIFIER_SEQUENCE" + if expected_column is not None: + assert ( + f"invalid_specifiers.h:1:{expected_column}: error[CPARSE_INVALID_SPECIFIER_SEQUENCE]" + in error.value.format_diagnostic(color=False) + ) + + +def test_declarators_compose_pointer_array_and_function_layers_in_c_binding_order(): + from prik.parsers.c import CArray, CFunctionType, CInt, CPointer, parse_c_file + + parsed = parse_c_file( + """ +extern int *values[4]; +extern int (*matrix)[4]; +extern int *(*table)[4]; +typedef int (*compare_fn)(const void *left, const void *right); +typedef int (*callback_table[8])(int); +int (*factory(void))(int); +int direct(void), *value; +void sort_items(int (*fallback)(const void *, const void *)); +""", + filename="recursive_declarators.h", + ) + + def layers(type_): + return [type(component) for component in type_.components] + + variables = {variable.name: variable for variable in parsed.variables} + assert layers(variables["values"].type) == [CArray, CPointer, CInt] + assert layers(variables["matrix"].type) == [CPointer, CArray, CInt] + assert layers(variables["table"].type) == [CPointer, CArray, CPointer, CInt] + assert layers(variables["value"].type) == [CPointer, CInt] + typedefs = {typedef.name: typedef for typedef in parsed.typedefs} + assert layers(typedefs["compare_fn"].type) == [CPointer, CFunctionType] + # Placeholder parameter names do not become part of a function type. + assert len(typedefs["compare_fn"].type.components[1].parameter_types) == 2 + assert layers(typedefs["callback_table"].type) == [CArray, CPointer, CFunctionType] + functions = {function.name: function for function in parsed.functions} + assert set(functions) == {"factory", "direct", "sort_items"} + assert layers(functions["factory"].result_type) == [CPointer, CFunctionType] + assert functions["sort_items"].parameters[0].callback_candidate is True + + +@pytest.mark.parametrize( + ("source", "section", "kept_lines", "initializer", "codes"), + [ + pytest.param("int i;\nint i;\n", "variables", [2], None, [], id="repeated-tentative-variable-merges"), + pytest.param( + "int i;\nint i = 1;\n", "variables", [1], "1", [], id="tentative-then-definition-keeps-definition" + ), + pytest.param( + "int i = 1;\nint i = 2;\n", + "variables", + None, + "1", + ["C_DUPLICATE_VARIABLE_DEFINITION"], + id="duplicate-definition", + ), + pytest.param( + "int i;\ndouble i;\n", + "variables", + None, + None, + ["C_CONFLICTING_VARIABLE_DECLARATION"], + id="conflicting-types", + ), + pytest.param("typedef int i;\ntypedef int i;\n", "typedefs", [2], None, [], id="compatible-typedef-merges"), + pytest.param( + "typedef int i;\ntypedef double i;\n", + "typedefs", + None, + None, + ["C_CONFLICTING_TYPEDEF"], + id="typedef-conflict", + ), + pytest.param( + "typedef int (*i)(int);\ntypedef double (*i)(double);\n", + "typedefs", + None, + None, + ["C_CONFLICTING_TYPEDEF"], + id="function-pointer-typedef-conflict", + ), + ], +) +def test_file_scope_redeclarations_merge_or_diagnose(source, section, kept_lines, initializer, codes): + from prik.parsers.c import parse_c_file + + parsed = parse_c_file(source, filename="redeclarations.c") + + declarations = getattr(parsed, section) + assert [declaration.name for declaration in declarations] == ["i"] + assert [diagnostic.code for diagnostic in parsed.diagnostics] == codes + if kept_lines is not None: + assert [location.line for location in declarations[0].declaration_locations] == kept_lines + if section == "variables": + kept = declarations[0].initializer + assert (kept.source_text if kept is not None else None) == initializer + + +@pytest.mark.parametrize( + ("text", "unit_kind", "message"), + [ + pytest.param( + "_Static_assert(sizeof(int) == 4)", + "static_assert", + "Static assertions are recorded but not evaluated.", + id="static-assert", + ), + pytest.param( + "int value __attribute__((used))", + "attribute_declaration", + "Compiler-specific declaration attributes", + id="gnu-attribute", + ), + pytest.param( + "int value __declspec(dllexport)", + "attribute_declaration", + "Compiler-specific declaration attributes", + id="declspec", + ), + pytest.param( + "int value [[deprecated]]", + "attribute_declaration", + "Compiler-specific declaration attributes", + id="standard-attribute", + ), + pytest.param( + "_Alignas(16) int value", "alignment_declaration", "Declaration alignment specifiers", id="alignas-keyword" + ), + pytest.param( + "alignas(16) int value", "alignment_declaration", "Declaration alignment specifiers", id="alignas-macro" + ), + pytest.param( + "struct packed { int a; } __attribute__((packed)) value", + "struct_definition", + "Struct definitions are not supported yet.", + id="attributed-struct-definition", + ), + ], +) +def test_raw_unsupported_declarations_warn_with_their_location_and_later_declarations_continue( + text, unit_kind, message +): + from prik.parsers.c import parse_c_file + + parsed = parse_c_file(f"int kept;\n {text};\nint later;\n", filename="unsupported.h") + + assert [variable.name for variable in parsed.variables] == ["kept", "later"] + assert len(parsed.diagnostics) == 1 + diagnostic = parsed.diagnostics[0] + assert diagnostic.code == "C_UNSUPPORTED_DECLARATION" + assert diagnostic.severity == "warning" + assert diagnostic.unit_kind == unit_kind + assert diagnostic.unit_name is None + assert message in diagnostic.message + assert diagnostic.location.filename == "unsupported.h" + assert diagnostic.location.line == 2 + assert diagnostic.location.column == 3 + assert diagnostic.location.source_line == f" {text};" + + +@pytest.mark.parametrize( + ("source", "name", "type_name"), + [ + pytest.param("external_type value;\n", "value", "external_type", id="unresolved-typedef-name"), + pytest.param("class widget;\n", "widget", "class", id="cpp-class-keyword"), + pytest.param("namespace api = other;\n", "api", "namespace", id="cpp-namespace-keyword"), + pytest.param("using size_type = value;\n", "size_type", "using", id="cpp-using-keyword"), + ], +) +def test_identifier_spelling_is_a_deferred_typedef_not_foreign_language_detection(source, name, type_name): + from prik.parsers.c import CTypedef, parse_c_file + + parsed = parse_c_file(source, filename="identifier_spelling.h") + + assert [variable.name for variable in parsed.variables] == [name] + assert isinstance(parsed.variables[0].type, CTypedef) + assert parsed.variables[0].type.name == type_name + assert parsed.variables[0].type.type is None + assert parsed.diagnostics == [] diff --git a/tests/c/infrastructure/parsing/test_c_error_fixture_suite.py b/tests/c/infrastructure/parsing/test_c_error_fixture_suite.py index 24c4207a9..d3ae2254d 100644 --- a/tests/c/infrastructure/parsing/test_c_error_fixture_suite.py +++ b/tests/c/infrastructure/parsing/test_c_error_fixture_suite.py @@ -41,19 +41,6 @@ def _update_mode_enabled() -> bool: return os.getenv("C_PARSER_UPDATE_GOLDENS", "0") == "1" -def test_c_error_fixture_suite_has_fixtures(): - fixtures = [path for path in _ERRORS_DIR.glob("*") if path.suffix.lower() in _SOURCE_SUFFIXES] - assert fixtures, "No C parser error fixtures found in tests/c/fixtures/native/errors/parser" - - -def test_c_error_fixtures_have_matching_expected_json(): - fixture_outputs = {f"{path.name}.json" for path in _ERRORS_DIR.glob("*") if path.suffix.lower() in _SOURCE_SUFFIXES} - expected_outputs = {path.name for path in _EXPECTED_ERRORS_DIR.glob("*.json")} - - assert not sorted(fixture_outputs - expected_outputs) - assert not sorted(expected_outputs - fixture_outputs) - - def test_c_error_fixture_suite_reports_expected_diagnostics(): from prik.parsers.c import CParseError, parse_c_file diff --git a/tests/c/infrastructure/parsing/test_c_fixture_suite.py b/tests/c/infrastructure/parsing/test_c_fixture_suite.py index c232a0b49..0205f3c9a 100644 --- a/tests/c/infrastructure/parsing/test_c_fixture_suite.py +++ b/tests/c/infrastructure/parsing/test_c_fixture_suite.py @@ -1,28 +1,22 @@ -"""C parser grouped-project fixture regression tests.""" +"""C parser grouped-project golden regression test. + +The goldens are the compiler-preprocessed Linux reference payloads for every +grouped fixture project (general, json, tinyexpr, linmath, nanosvg, stb). One +exact comparison proves the parse, project assembly, include graph, and JSON +shape for real-world headers; see ``tests/c/fixtures/parser/README.md``. +""" import importlib.util import json import shutil import sys -from pathlib import Path import pytest from tests.c._support.paths import C_ROOT -_C_ROOT = C_ROOT -_DATA_DIR = _C_ROOT / "fixtures" / "native" -_SOURCE_SUFFIXES = {".c", ".h", ".i"} -_SOURCE_ORDER = {".c": 0, ".h": 1, ".i": 2} -_FIXTURE_GROUPS = ("general", "json", "tinyexpr", "linmath", "nanosvg", "stb") -_PROJECT_OVERRIDES = { - "nanosvg": { - "nanosvg": ("nanosvg.h", "nanosvgrast.h"), - }, -} - def _load_golden_generator(): - module_path = _C_ROOT / "fixtures" / "parser" / "generate_c_parser_goldens.py" + module_path = C_ROOT / "fixtures" / "parser" / "generate_c_parser_goldens.py" spec = importlib.util.spec_from_file_location("generate_c_parser_goldens", module_path) if spec is None or spec.loader is None: raise RuntimeError(f"cannot load C golden generator from {module_path}") @@ -31,129 +25,6 @@ def _load_golden_generator(): return module -def _fixture_sort_key(fixture: Path) -> tuple[int, str]: - return (_SOURCE_ORDER.get(fixture.suffix.lower(), 99), fixture.as_posix()) - - -def _project_key(fixture: Path, root: Path) -> Path: - relative = fixture.relative_to(root) - for project_name, filenames in _PROJECT_OVERRIDES.get(root.name, {}).items(): - if relative.name in filenames: - return Path(project_name) - return relative.with_suffix("") - - -def _project_groups(root: Path) -> list[tuple[Path, list[Path]]]: - grouped: dict[Path, list[Path]] = {} - for fixture in sorted(root.rglob("*"), key=_fixture_sort_key): - if fixture.is_file() and fixture.suffix.lower() in _SOURCE_SUFFIXES: - grouped.setdefault(_project_key(fixture, root), []).append(fixture) - projects = [] - for project_key, fixtures in sorted(grouped.items()): - override = _PROJECT_OVERRIDES.get(root.name, {}).get(project_key.name) - if override is not None: - order = {filename: index for index, filename in enumerate(override)} - fixtures = sorted(fixtures, key=lambda fixture: order[fixture.name]) - else: - fixtures = sorted(fixtures, key=_fixture_sort_key) - projects.append((project_key, fixtures)) - return projects - - -@pytest.mark.parametrize("data_subdir", _FIXTURE_GROUPS) -def test_c_fixture_suite_has_inputs(data_subdir): - fixtures = sorted((_DATA_DIR / data_subdir).glob("*")) - assert any(path.suffix.lower() in _SOURCE_SUFFIXES for path in fixtures) - - -@pytest.mark.parametrize( - "fixture", - [ - _DATA_DIR / "json" / "cJSON.h", - _DATA_DIR / "tinyexpr" / "tinyexpr.h", - _DATA_DIR / "linmath" / "linmath.h", - _DATA_DIR / "nanosvg" / "nanosvg.h", - _DATA_DIR / "stb" / "stb_c_lexer.h", - ], -) -def test_c_fixture_headers_with_macros_require_preprocessing(fixture): - from prik.parsers.c import CParseError, parse_c_file - - with pytest.raises(CParseError, match="require compiler preprocessing") as exc_info: - parse_c_file(fixture) - - assert exc_info.value.code == "CPARSE_PREPROCESSING_REQUIRED" - - -@pytest.mark.parametrize( - ("fixture", "defines"), - [ - (_DATA_DIR / "json" / "jsmn.h", []), - (_DATA_DIR / "tinyexpr" / "tinyexpr.c", []), - (_DATA_DIR / "linmath" / "linmath.h", []), - (_DATA_DIR / "nanosvg" / "nanosvg.h", ["NANOSVG_IMPLEMENTATION"]), - (_DATA_DIR / "nanosvg" / "nanosvgrast.h", ["NANOSVGRAST_IMPLEMENTATION"]), - (_DATA_DIR / "stb" / "stb_ds.h", ["STB_DS_IMPLEMENTATION"]), - (_DATA_DIR / "stb" / "stb_dxt.h", ["STB_DXT_IMPLEMENTATION"]), - (_DATA_DIR / "stb" / "stb_image.h", ["STB_IMAGE_IMPLEMENTATION"]), - (_DATA_DIR / "stb" / "stb_rect_pack.h", ["STB_RECT_PACK_IMPLEMENTATION"]), - ], -) -def test_c_fixture_headers_parse_after_compiler_preprocessing(fixture, defines): - from prik.parsers.c import parse_c_file - from prik.preprocessing import PreprocessingConfig, preprocess_source - - compiler = shutil.which("cc") - if compiler is None: - pytest.skip("cc is not available") - preprocessed = preprocess_source( - fixture, - language="c", - config=PreprocessingConfig( - mode="compiler", - compiler=compiler, - include_dirs=[str(fixture.parent)], - defines=defines, - ), - ) - - parsed = parse_c_file( - preprocessed.source, - filename=str(fixture), - preprocessing="compiler", - ) - - assert parsed.preprocessing == "compiler" - assert parsed.functions - - -def test_c_fixture_suite_keeps_source_locations_stable_for_plain_source(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - _DATA_DIR / "general" / "basic_array_update.c", - ) - function = parsed.functions[0] - - assert function.source_location.filename == str(_DATA_DIR / "general" / "basic_array_update.c") - assert function.source_location.line >= 1 - assert function.source_location.column >= 1 - - -def test_c_fixture_suite_groups_matching_source_and_header_source_first(): - projects = dict(_project_groups(_DATA_DIR / "json")) - fixtures = projects[Path("cJSON")] - - assert [fixture.name for fixture in fixtures] == ["cJSON.c", "cJSON.h"] - - -def test_c_fixture_suite_groups_dependent_headers_dependency_first(): - projects = dict(_project_groups(_DATA_DIR / "nanosvg")) - fixtures = projects[Path("nanosvg")] - - assert [fixture.name for fixture in fixtures] == ["nanosvg.h", "nanosvgrast.h"] - - @pytest.mark.skipif( sys.platform != "linux", reason="C project goldens use the Linux compiler-preprocessing reference environment", diff --git a/tests/c/infrastructure/parsing/test_c_json_sanity.py b/tests/c/infrastructure/parsing/test_c_json_sanity.py deleted file mode 100644 index 57c322e13..000000000 --- a/tests/c/infrastructure/parsing/test_c_json_sanity.py +++ /dev/null @@ -1,99 +0,0 @@ -"""JSON schema sanity tests for legacy C parser project snapshots.""" - -import json -from tests.c._support.paths import PARSER_FIXTURE_ROOT - -_FIXTURES_DIR = PARSER_FIXTURE_ROOT / "fixtures" -_PARSER_FIXTURE_GROUPS = ("general", "json", "tinyexpr", "linmath", "nanosvg", "stb") - - -def _iter_project_payloads(): - for group in _PARSER_FIXTURE_GROUPS: - for path in (_FIXTURES_DIR / group).rglob("*.json"): - yield path, json.loads(path.read_text(encoding="utf-8")) - - -def _iter_file_payloads(): - for path, project in _iter_project_payloads(): - for filename, payload in project["files"].items(): - yield path, filename, payload - - -def test_c_json_fixtures_are_valid_json(): - for path in _FIXTURES_DIR.rglob("*.json"): - json.loads(path.read_text(encoding="utf-8")) - - -def test_c_json_project_fixtures_have_stable_top_level_shape(): - required_keys = { - "files", - "functions", - "structs", - "unions", - "enums", - "typedefs", - "variables", - "macros", - "includes", - "functions_by_file", - "enum_constants", - "include_graph", - "system_includes", - "unresolved_includes", - "header_source_pairs", - "diagnostics", - } - - for path, payload in _iter_project_payloads(): - assert required_keys <= set(payload), f"missing project keys in {path}" - - -def test_c_json_project_files_have_stable_c_file_shape(): - required_keys = { - "language", - "filename", - "functions", - "structs", - "unions", - "enums", - "typedefs", - "variables", - "macros", - "includes", - "diagnostics", - } - - for path, _, payload in _iter_file_payloads(): - assert required_keys <= set(payload), f"missing file keys in {path}" - assert payload["language"] == "c" - - -def test_c_json_functions_have_names_types_and_source_locations(): - for path, _, payload in _iter_file_payloads(): - for fn in payload.get("functions", []): - assert fn["name"], f"function without name in {path}" - assert fn["result_type"], f"function without result type in {path}" - assert isinstance(fn["parameters"], list) - assert fn["source_location"]["line"] >= 1 - assert fn["source_location"]["column"] >= 1 - - -def test_c_json_types_have_distinct_names_or_anonymous_ids(): - for path, _, payload in _iter_file_payloads(): - for key in ("structs", "unions", "enums"): - for entry in payload.get(key, []): - assert entry.get("name") or entry.get("anonymous_id") or entry.get("reference"), ( - f"anonymous {key} missing id in {path}" - ) - - -def test_c_json_diagnostics_have_codes_locations_and_severities(): - allowed = {"info", "warning", "error"} - - for _path, payload in _iter_project_payloads(): - for diagnostic in payload.get("diagnostics", []): - assert diagnostic["code"] - assert diagnostic["severity"] in allowed - assert diagnostic.get("message") - if diagnostic.get("location"): - assert diagnostic["location"]["line"] >= 1 diff --git a/tests/c/infrastructure/parsing/test_c_lexer_preprocessor.py b/tests/c/infrastructure/parsing/test_c_lexer_preprocessor.py index c85ff6a64..5dea7345a 100644 --- a/tests/c/infrastructure/parsing/test_c_lexer_preprocessor.py +++ b/tests/c/infrastructure/parsing/test_c_lexer_preprocessor.py @@ -3,292 +3,33 @@ import pytest -def test_lexer_removes_comments_without_changing_string_or_char_literals(): - from prik.parsers.c.lexer import lex_c_source - - tokens = lex_c_source( - r""" -const char *url = "https://example.invalid/a//b"; -char slash = '/'; -/* removed block comment */ -int value; // removed line comment -""", - filename="comments.c", - ) - - spellings = [token.text for token in tokens] - assert '"https://example.invalid/a//b"' in spellings - assert "'/'" in spellings - assert "comment" not in spellings - - -def test_lexer_removes_multiline_block_comments_but_preserves_following_line_numbers(): - from prik.parsers.c.lexer import lex_c_source - - tokens = lex_c_source( - "int first;\n/* removed\n block */\nint second;\n", - filename="block_comments.c", - ) - - identifiers = [token for token in tokens if token.kind == "identifier"] - assert [token.text for token in identifiers] == ["int", "first", "int", "second"] - assert identifiers[-2].line == 4 - assert identifiers[-1].column == 5 - - -def test_line_continuations_preserve_original_line_numbers(): - from prik.preprocessing.c import normalize_c_source - - normalized = normalize_c_source( - "#define SUM(a, b) \\\n ((a) + (b))\nint x;\n", - filename="continuations.h", - ) - - assert normalized.records[0].original_start_line == 1 - assert normalized.records[0].original_end_line == 2 - assert normalized.records[1].original_start_line == 3 - - -def test_top_level_split_helpers_ignore_nested_commas_and_function_bodies(): - from prik.parsers.c.lexer import split_top_level_c_source, top_level_split - - assert top_level_split("int (*cmp)(int, int), int value") == [ - "int (*cmp)(int, int)", - "int value", - ] - - segments = split_top_level_c_source( - 'int add(int a, int b) { const char *s = "{;}"; return a + b; }\nint next(void);\n', - filename="split.c", - ) - - assert [(segment.text, segment.terminator) for segment in segments] == [ - ("int add(int a, int b)", "block"), - ("int next(void)", ";"), - ] - - -def test_c_lexer_covers_linemarker_escapes_top_level_strings_and_eof_records(): - from prik.parsers.c import parse_c_file - from prik.parsers.c.lexer import ( - CLogicalRecord, - _unescape_linemarker_filename, - lex_c_source, - line_mappings_for_source, - normalize_c_source, - split_top_level_c_source, - ) - from prik.preprocessing.c import _record_location - - assert _unescape_linemarker_filename(r"a\nb\rc\td\\e\"f\x") == 'a\nb\rc\td\\e"fx' - assert _unescape_linemarker_filename("tail\\") == "tail\\" - - mappings = line_mappings_for_source( - '#line 7\nint local;\n# 3 "dir\\\\api\\".h"\nint named;\n', - filename="generated.i", - use_linemarkers=True, - ) - assert mappings[1].filename == "generated.i" - assert mappings[1].line == 7 - assert mappings[3].filename == 'dir\\api".h' - assert mappings[3].line == 3 - - segments = split_top_level_c_source('"literal";\nint unfinished', filename="odd.c") - assert [(segment.text, segment.terminator) for segment in segments] == [ - ('"literal"', ";"), - ("int unfinished", "eof"), - ] - - normalized = normalize_c_source("#define API \\\n", filename="defs.h") - assert normalized.records[0].text == "#define API" - assert ( - _record_location(CLogicalRecord(text="#define API", filename="defs.h", original_source_lines=())).source_line - is None - ) - assert ( - _record_location( - CLogicalRecord(text="define API", filename="defs.h", original_source_lines=("define API",)) - ).column - == 1 - ) - - token = lex_c_source(r'char *s = "unterminated\\', filename="bad.c")[-1] - assert token.kind == "string" - assert token.text == r'"unterminated\\' - - parsed = parse_c_file( - '#line 11 "dir\\\\api\\".h"\nstruct __attribute__((annotate("tag\\"ged"))) named { int value; };\n', - filename="generated.i", - preprocessing="preprocessed", - ) - assert parsed.structs[0].name == "named" - assert parsed.structs[0].source_location.filename == 'dir\\api".h' - - -def test_c_lexer_mapping_helpers_cover_boundaries_ranges_and_position_updates(): - from prik.parsers.c.lexer import ( - CLineMapping, - _advance_position, - _line_mapping, - _mapped_filenames, - _mapped_line_numbers, - _mapped_source_lines, - _source_line, - _source_lines, - ) - - source_lines = ["first", "second", "third"] - assert _source_line(source_lines, 1) == "first" - assert _source_line(source_lines, 3) == "third" - assert _source_line(source_lines, 0) is None - assert _source_line(source_lines, 4) is None - assert _source_lines(source_lines, 2, 3) == ("second", "third") - - mappings = [ - CLineMapping("first.h", 11, "first source"), - CLineMapping("second.h", 22, None), - ] - assert _line_mapping(mappings, 1, "fallback.h") == mappings[0] - assert _line_mapping(mappings, 2, "fallback.h") == mappings[1] - assert _line_mapping(mappings, 0, "fallback.h") == CLineMapping("fallback.h", 0, None) - assert _line_mapping(mappings, 3, "fallback.h") == CLineMapping("fallback.h", 3, None) - assert _mapped_source_lines(mappings, 1, 2) == ("first source", "") - assert _mapped_filenames(mappings, 1, 2) == ("first.h", "second.h") - assert _mapped_line_numbers(mappings, 1, 2) == (11, 22) - - assert _advance_position("x", 4, 7) == (4, 8) - assert _advance_position("\n", 4, 7) == (5, 1) - - -def test_c_lexer_linemarker_and_directive_helpers_cover_raw_and_preprocessed_modes(): - from prik.parsers.c.lexer import ( - CLineMapping, - _blank_preprocessor_directives, - _parse_linemarker, - line_mappings_for_source, - ) - - assert _parse_linemarker('# 12 "api.h" 1') == (12, "api.h") - assert _parse_linemarker("#line 7 generated.h") == (7, "generated.h") - assert _parse_linemarker("#line 9") == (9, None) - assert _parse_linemarker("#pragma once") is None - - raw = line_mappings_for_source("int first;\nint second;", filename="raw.h") - assert raw == [ - CLineMapping("raw.h", 1, "int first;"), - CLineMapping("raw.h", 2, "int second;"), - ] - - preprocessed = line_mappings_for_source( - '# 40 "api.h"\nint first;\n#line 7\nint second;\n', - filename="generated.i", - use_linemarkers=True, - ) - assert preprocessed == [ - CLineMapping("api.h", 40, '# 40 "api.h"'), - CLineMapping("api.h", 40, "int first;"), - CLineMapping("api.h", 7, "#line 7"), - CLineMapping("api.h", 7, "int second;"), - ] - - blanked = _blank_preprocessor_directives("#define VALUE \\\n continued\nint kept;\n#pragma once") - assert blanked.splitlines() == [ - " " * len("#define VALUE \\"), - " " * len(" continued"), - "int kept;", - " " * len("#pragma once"), - ] - - -def test_c_lexer_delimiter_helpers_cover_literals_nesting_offsets_and_validation(): - from prik.parsers.c.lexer import ( - _scan_code_states, - top_level_partition, - top_level_split, - top_level_split_with_offsets, - ) - - source = 'first("x,y", [a,b]), second' - assert top_level_split_with_offsets(source) == [ - ('first("x,y", [a,b])', 0), - ("second", source.index("second")), - ] - assert top_level_split(source) == ['first("x,y", [a,b])', "second"] - assert top_level_partition('name = call("=", [x=y])') == ("name", 'call("=", [x=y])') - assert top_level_partition("name") == ("name", None) - - states = list(_scan_code_states(source)) - quote_index = source.index('"') - nested_comma_index = source.index(",", source.index("[")) - top_level_comma_index = source.index(",", source.index("]")) - assert states[quote_index][3] == "normal" - assert states[quote_index + 1][3] == "string" - assert states[nested_comma_index][2] == ("(", "[") - assert states[top_level_comma_index][2] == () - - with pytest.raises(ValueError, match="single character"): - top_level_split_with_offsets("a, b", "::") - with pytest.raises(ValueError, match="single character"): - top_level_partition("a=b", "==") - - -def test_c_lexer_aggregate_attribute_helpers_preserve_shape_and_classify_headers(): - from prik.parsers.c.lexer import ( - _balanced_invocation_end, - _is_aggregate_definition_header, - _is_braced_declaration_header, - _strip_aggregate_header_attributes, - ) - - invocation = '(outer(")") + nested(1))' - assert _balanced_invocation_end(invocation, 0) == len(invocation) - prefixed_invocation = "ignored) (nested(1))" - assert _balanced_invocation_end(prefixed_invocation, prefixed_invocation.index("(")) == len(prefixed_invocation) - single_quoted_invocation = "(outer(')') + nested(1))" - assert _balanced_invocation_end(single_quoted_invocation, 0) == len(single_quoted_invocation) - empty_quoted_invocation = '(outer("") + nested(1))' - assert _balanced_invocation_end(empty_quoted_invocation, 0) == len(empty_quoted_invocation) - assert _balanced_invocation_end("(unterminated", 0) is None - - header = "struct __attribute__((packed)) packet" - stripped = _strip_aggregate_header_attributes(header) - assert len(stripped) == len(header) - assert stripped.split() == ["struct", "packet"] - spaced_header = "struct __attribute__ ((packed)) packet" - assert _strip_aggregate_header_attributes(spaced_header).split() == ["struct", "packet"] - bare_header = "struct __attribute__" - assert _strip_aggregate_header_attributes(bare_header).split() == ["struct"] - multiline_header = "struct __attribute__((\npacked)) packet" - assert _strip_aggregate_header_attributes(multiline_header).count("\n") == 1 - assert _is_aggregate_definition_header(header, tolerate_compiler_extensions=True) - assert not _is_aggregate_definition_header(header) - assert not _is_aggregate_definition_header("int factory(void)") - assert not _is_aggregate_definition_header("struct packet(value)") - assert not _is_aggregate_definition_header("struct packet = value") - assert _is_braced_declaration_header("int values =") - assert not _is_braced_declaration_header(header) - assert not _is_braced_declaration_header("int function(void)") - - -def test_c_lexer_comment_normalization_and_tokens_preserve_source_accounting(): +def test_c_lexer_strips_comments_and_joins_continuations_without_losing_source_accounting(): from prik.parsers.c.lexer import lex_c_source, normalize_c_source, strip_c_comments - source = 'int first; // removed\nchar *text = "/* kept */"; /* block\n removed */ int second;\n' + source = ( + 'int first; // removed\nchar *text = "/* kept */ a//b"; /* block\n removed */ int second;\n' + "char slash = '/';\n" + ) stripped = strip_c_comments(source) assert len(stripped) == len(source) assert stripped.count("\n") == source.count("\n") - assert '"/* kept */"' in stripped + assert '"/* kept */ a//b"' in stripped + assert "'/'" in stripped assert "removed" not in stripped assert "int second;" in stripped - normalized = normalize_c_source("int first = \\\n 1;\n\nint second;\n", filename="records.h") + normalized = normalize_c_source("int first = \\\n 1;\n\nint second;\n#define API \\\n", filename="records.h") assert [(record.text, record.original_start_line, record.original_end_line) for record in normalized.records] == [ ("int first = 1;", 1, 2), ("int second;", 4, 4), + ("#define API", 5, 5), ] assert normalized.records[0].original_source_lines == ("int first = \\", " 1;") - tokens = lex_c_source("int value = 12;\nvalue += 3;\nchar quote = '\\n';\n", filename="tokens.c") + tokens = lex_c_source( + "int value = 12;\nvalue += 3;\nchar quote = '\\n';\n/* removed\n block */ int after;\n", + filename="tokens.c", + ) assert [(token.text, token.kind, token.line, token.column) for token in tokens] == [ ("int", "identifier", 1, 1), ("value", "identifier", 1, 5), @@ -304,22 +45,56 @@ def test_c_lexer_comment_normalization_and_tokens_preserve_source_accounting(): ("=", "punctuation", 3, 12), ("'\\n'", "char", 3, 14), (";", "punctuation", 3, 18), + ("int", "identifier", 5, 13), + ("after", "identifier", 5, 17), + (";", "punctuation", 5, 22), ] assert all(token.filename == "tokens.c" for token in tokens) assert tokens[5].source_line == "value += 3;" + unterminated = lex_c_source(r'char *s = "unterminated\\', filename="bad.c")[-1] + assert (unterminated.kind, unterminated.text) == ("string", r'"unterminated\\') + + +def test_top_level_split_keeps_literal_statements_and_reports_an_unterminated_tail(): + from prik.parsers.c.lexer import split_top_level_c_source + + segments = split_top_level_c_source('"literal";\nint unfinished', filename="odd.c") + + assert [(segment.text, segment.terminator) for segment in segments] == [ + ('"literal"', ";"), + ("int unfinished", "eof"), + ] -def test_raw_mode_records_includes_without_expanding_them(): + +def test_linemarkers_map_escaped_filenames_and_bare_line_directives(): from prik.parsers.c import parse_c_file + from prik.parsers.c.lexer import CLineMapping, line_mappings_for_source - parsed = parse_c_file( - '#include "api_types.h"\n#include \nint run(void);\n', - filename="includes.h", - preprocessing="raw", + assert line_mappings_for_source("int first;\nint second;", filename="raw.h") == [ + CLineMapping("raw.h", 1, "int first;"), + CLineMapping("raw.h", 2, "int second;"), + ] + mappings = line_mappings_for_source( + '#line 7\nint local;\n# 3 "dir\\\\api\\".h"\nint named;\n#line 40\nint kept_file;\n', + filename="generated.i", + use_linemarkers=True, ) + # A bare #line keeps the current file; an escaped filename is unescaped. + assert [(mapping.filename, mapping.line) for mapping in mappings[1::2]] == [ + ("generated.i", 7), + ('dir\\api".h', 3), + ('dir\\api".h', 40), + ] - assert [include.target for include in parsed.includes] == ["api_types.h", "stddef.h"] - assert [include.kind for include in parsed.includes] == ["local", "system"] + parsed = parse_c_file( + '#line 11 "dir\\\\api\\".h"\nstruct __attribute__((annotate("tag\\"ged"))) named { int value; };\n', + filename="generated.i", + preprocessing="preprocessed", + ) + assert parsed.structs[0].name == "named" + assert parsed.structs[0].source_location.filename == 'dir\\api".h' + assert parsed.structs[0].source_location.line == 11 def test_raw_mode_resolves_local_includes_relative_to_path_input(tmp_path): @@ -336,52 +111,6 @@ def test_raw_mode_resolves_local_includes_relative_to_path_input(tmp_path): assert parsed.diagnostics == [] -def test_c_preprocessor_helpers_cover_include_dirs_and_filesystem_errors(tmp_path, monkeypatch): - from pathlib import Path - - from prik.parsers.c.lexer import CLogicalRecord - from prik.preprocessing.c import _record_location, _resolve_local_include - - include_dir = tmp_path / "include" - include_dir.mkdir() - types = include_dir / "api_types.h" - types.write_text("typedef int api_int;\n", encoding="utf-8") - - assert _resolve_local_include("api_types.h", None, [include_dir]) == str(types) - assert _resolve_local_include("missing.h", None, [include_dir]) is None - - unreadable_dir = tmp_path / "unreadable" - unreadable_dir.mkdir() - unreadable_candidate = unreadable_dir / "api_types.h" - original_is_file = Path.is_file - - def raise_one_os_error(path): - if path == unreadable_candidate: - raise OSError("unreadable") - return original_is_file(path) - - monkeypatch.setattr(Path, "is_file", raise_one_os_error) - assert _resolve_local_include("api_types.h", str(unreadable_dir / "api.h"), [include_dir]) == str(types) - - location = _record_location( - CLogicalRecord( - text=" #pragma once", - filename="api.h", - original_start_line=7, - original_source_lines=(" #pragma once",), - ) - ) - assert (location.filename, location.line, location.column, location.source_line) == ( - "api.h", - 7, - 3, - " #pragma once", - ) - assert ( - _record_location(CLogicalRecord(text="#pragma # once", original_source_lines=("#pragma # once",))).column == 1 - ) - - def test_collect_preprocessor_metadata_preserves_locations_and_diagnostics(tmp_path): from prik.preprocessing.c import collect_preprocessor_metadata diff --git a/tests/c/infrastructure/parsing/test_c_model_serialization.py b/tests/c/infrastructure/parsing/test_c_model_serialization.py index 5c8984e26..16f99ea66 100644 --- a/tests/c/infrastructure/parsing/test_c_model_serialization.py +++ b/tests/c/infrastructure/parsing/test_c_model_serialization.py @@ -1,10 +1,5 @@ """Minimal JSON-shape coverage for C parser model dataclasses.""" -from dataclasses import is_dataclass -import importlib -import inspect -from types import SimpleNamespace - import prik.parsers.c.models as models @@ -14,262 +9,6 @@ def _type_payload(model: str, **extra): return payload -def test_each_c_model_has_minimal_json_shape(): - cases = { - "CSourceLocation": ( - models.CSourceLocation(), - {"filename": None, "line": None, "column": None, "source_line": None}, - ), - "CDiagnostic": ( - models.CDiagnostic(code="C_TEST", message="test message"), - { - "code": "C_TEST", - "message": "test message", - "severity": "warning", - "location": None, - "unit_kind": None, - "unit_name": None, - }, - ), - "CQualifier": (models.CQualifier("const"), "const"), - "CConst": (models.CConst(), "const"), - "CVolatile": (models.CVolatile(), "volatile"), - "CRestrict": (models.CRestrict(), "restrict"), - "CAtomic": (models.CAtomic(), "_Atomic"), - "CType": (models.CType(), _type_payload("CType")), - "CUnknownType": (models.CUnknownType(), _type_payload("CUnknownType", spelling="unknown")), - "CVoid": (models.CVoid(), _type_payload("CVoid")), - "CBool": (models.CBool(), _type_payload("CBool")), - "CChar": (models.CChar(), _type_payload("CChar")), - "CSignedChar": (models.CSignedChar(), _type_payload("CSignedChar")), - "CUnsignedChar": (models.CUnsignedChar(), _type_payload("CUnsignedChar")), - "CShort": (models.CShort(), _type_payload("CShort")), - "CUnsignedShort": (models.CUnsignedShort(), _type_payload("CUnsignedShort")), - "CInt": (models.CInt(), _type_payload("CInt")), - "CUnsignedInt": (models.CUnsignedInt(), _type_payload("CUnsignedInt")), - "CLong": (models.CLong(), _type_payload("CLong")), - "CUnsignedLong": (models.CUnsignedLong(), _type_payload("CUnsignedLong")), - "CLongLong": (models.CLongLong(), _type_payload("CLongLong")), - "CUnsignedLongLong": (models.CUnsignedLongLong(), _type_payload("CUnsignedLongLong")), - "CFloat": (models.CFloat(), _type_payload("CFloat")), - "CDouble": (models.CDouble(), _type_payload("CDouble")), - "CLongDouble": (models.CLongDouble(), _type_payload("CLongDouble")), - "CFloatComplex": (models.CFloatComplex(), _type_payload("CFloatComplex")), - "CDoubleComplex": (models.CDoubleComplex(), _type_payload("CDoubleComplex")), - "CLongDoubleComplex": (models.CLongDoubleComplex(), _type_payload("CLongDoubleComplex")), - "CPointer": (models.CPointer(), _type_payload("CPointer")), - "CArray": ( - models.CArray(), - _type_payload( - "CArray", - bound=None, - is_static_minimum=False, - is_variable_length=False, - is_flexible=False, - ), - ), - "CFunctionType": ( - models.CFunctionType(), - _type_payload( - "CFunctionType", - result_type=_type_payload("CVoid"), - parameter_types=[], - is_variadic=False, - prototype_style=None, - ), - ), - "CComposedType": ( - models.CComposedType(), - _type_payload("CComposedType", components=[]), - ), - "CParameter": ( - models.CParameter(), - { - "name": None, - "type": _type_payload("CVoid"), - "declared_type": None, - "source_location": None, - "callback_policy": None, - }, - ), - "CFunction": ( - models.CFunction(name="run"), - { - "name": "run", - "result_type": _type_payload("CVoid"), - "parameters": [], - "storage": [], - "specifiers": [], - "is_variadic": False, - "is_definition": False, - "prototype_style": None, - "source_location": None, - "start": None, - "end": None, - "declaration_locations": [], - }, - ), - "CStruct": ( - models.CStruct(), - _type_payload( - "CStruct", - name=None, - members=[], - anonymous_id=None, - is_incomplete=False, - source_location=None, - ), - ), - "CUnion": ( - models.CUnion(), - _type_payload( - "CUnion", - name=None, - members=[], - anonymous_id=None, - is_incomplete=False, - source_location=None, - ), - ), - "CEnumerator": ( - models.CEnumerator(name="STATUS_OK"), - {"name": "STATUS_OK", "value": None, "source_location": None}, - ), - "CEnum": ( - models.CEnum(), - _type_payload("CEnum", name=None, constants=[], anonymous_id=None, source_location=None), - ), - "CTypedef": ( - models.CTypedef(name="api_int"), - _type_payload( - "CTypedef", - name="api_int", - type=None, - source_location=None, - declaration_locations=[], - ), - ), - "CInitializer": (models.CInitializer(source_text="42"), {"source_text": "42"}), - "CVariable": ( - models.CVariable(name="value"), - { - "name": "value", - "type": _type_payload("CVoid"), - "storage": [], - "initializer": None, - "bit_width": None, - "source_location": None, - "callback_policy": None, - "declaration_locations": [], - }, - ), - "CMacro": ( - models.CMacro(name="API"), - { - "name": "API", - "value": None, - "function_like": False, - "directive": "define", - "source_location": None, - }, - ), - "CRawDirective": ( - models.CRawDirective(directive="include"), - {"directive": "include", "argument": None, "source_location": None}, - ), - "CMacroDependency": ( - models.CMacroDependency(name="API"), - { - "name": "API", - "context": "declaration", - "source_location": None, - "source_text": "", - }, - ), - "CInclude": ( - models.CInclude(target="api.h"), - {"target": "api.h", "kind": "local", "resolved_path": None, "source_location": None}, - ), - "CFile": ( - models.CFile(filename="api.h"), - { - "filename": "api.h", - "language": "c", - "preprocessing": "raw", - "functions": [], - "structs": [], - "unions": [], - "enums": [], - "typedefs": [], - "variables": [], - "macros": [], - "includes": [], - "raw_directives": [], - "macro_dependencies": [], - "diagnostics": [], - }, - ), - "CProject": ( - models.CProject(), - { - "files": {}, - "functions": {}, - "structs": {}, - "unions": {}, - "enums": {}, - "typedefs": {}, - "variables": {}, - "macros": {}, - "includes": {}, - "functions_by_file": {}, - "enum_constants": {}, - "include_graph": {}, - "system_includes": {}, - "unresolved_includes": {}, - "header_source_pairs": {}, - "diagnostics": [], - }, - ), - } - - model_dataclasses = { - name - for name, obj in inspect.getmembers(models, inspect.isclass) - if obj.__module__ == models.__name__ and is_dataclass(obj) - } - assert set(cases) == model_dataclasses - - for model_name, (model, expected) in cases.items(): - assert models.c_model_to_dict(model) == expected, model_name - - -def test_c_model_helpers_cover_environment_color_and_windows_setup(monkeypatch): - monkeypatch.setenv("C_PARSER_DEBUG", " YES ") - assert models._env_flag("C_PARSER_DEBUG") - monkeypatch.delenv("C_PARSER_DEBUG") - assert not models._env_flag("C_PARSER_DEBUG") - - assert models._apply_color("value", "red", "bold", enabled=False) == "value" - assert models._apply_color("value", "red", "bold", enabled=True) == "\x1b[31m\x1b[1mvalue\x1b[0m" - - calls = [] - colorama = SimpleNamespace(just_fix_windows_console=lambda: calls.append("fixed")) - original_os_name = models.os.name - monkeypatch.setattr(models.os, "name", "nt") - try: - monkeypatch.setitem(models.sys.modules, "colorama", colorama) - models._enable_windows_ansi() - assert calls == ["fixed"] - - monkeypatch.delitem(models.sys.modules, "colorama") - monkeypatch.setattr(importlib.util, "find_spec", lambda name: object() if name == "colorama" else None) - monkeypatch.setattr(importlib, "import_module", lambda name: colorama if name == "colorama" else None) - models._enable_windows_ansi() - assert calls == ["fixed", "fixed"] - finally: - monkeypatch.setattr(models.os, "name", original_os_name) - - def _make_parse_error(**kwargs): return models.CParseError("unexpected token", **kwargs) @@ -310,29 +49,3 @@ def test_c_parse_error_diagnostic_rendering_contract(monkeypatch): assert blank.format_diagnostic(debug=False) == ( ": error[CPARSE_ERROR]: unexpected token\n |\n? | \n | " ) - - -def test_c_model_to_dict_preserves_seen_aggregates_through_dicts(): - node = models.CStruct(name="node") - - assert models.c_model_to_dict({"first": node, "again": node})["again"] == {"reference": "struct node"} - - -def test_c_parse_error_uses_immediate_stack_frame(monkeypatch): - frames = [ - None, - SimpleNamespace(filename="parser.py", lineno=7, function="raise_here"), - SimpleNamespace(filename="caller.py", lineno=9, function="caller"), - ] - monkeypatch.setattr(models.inspect, "stack", lambda: frames) - - error = models.CParseError("invalid") - - assert (error.parser_file, error.parser_line_number, error.parser_function) == ("parser.py", 7, "raise_here") - - -def test_callback_candidate_requires_function_type_after_pointer(): - callback_before_pointer = models.CComposedType(components=[models.CFunctionType(), models.CPointer()]) - - assert not models.CVariable(name="value", type=callback_before_pointer).callback_candidate - assert not models.CParameter(name="value", type=callback_before_pointer).callback_candidate diff --git a/tests/c/infrastructure/parsing/test_c_parser_properties.py b/tests/c/infrastructure/parsing/test_c_parser_properties.py index 0cc4ec8eb..a7e6951d1 100644 --- a/tests/c/infrastructure/parsing/test_c_parser_properties.py +++ b/tests/c/infrastructure/parsing/test_c_parser_properties.py @@ -82,17 +82,6 @@ def test_top_level_c_source_split_ignores_function_body_delimiters(names): ] -@pytest.mark.property -@given(function_name=_C_IDENTIFIERS) -def test_generated_c_visibility_attributes_are_tolerated(function_name): - source = f'int {function_name}(void) __attribute__((visibility("default")));\n' - - parsed = parse_c_file(source, filename="compiler.h", preprocessing="compiler") - - assert parsed.diagnostics == [] - assert [function.name for function in parsed.functions] == [function_name] - - @pytest.mark.fuzz @given(_FUZZ_TEXT) def test_c_parser_fuzz_fragments_only_raise_owned_errors(source): diff --git a/tests/c/infrastructure/parsing/test_c_project_resolution.py b/tests/c/infrastructure/parsing/test_c_project_resolution.py index b1c1699a5..f8cd61dca 100644 --- a/tests/c/infrastructure/parsing/test_c_project_resolution.py +++ b/tests/c/infrastructure/parsing/test_c_project_resolution.py @@ -2,6 +2,8 @@ from pathlib import Path +import pytest + def test_project_include_graph_tracks_local_system_missing_and_cycles(tmp_path: Path): from prik.parsers.c import parse_c_project @@ -94,34 +96,6 @@ def test_parse_c_project_directory_discovers_preprocessed_i_files(tmp_path: Path assert "origin" not in generated.to_dict()["functions"][0] -def test_project_indexes_functions_by_file_and_enum_constants(tmp_path: Path): - from prik.parsers.c import parse_c_project - - (tmp_path / "api.h").write_text( - "enum status { STATUS_OK = 0, STATUS_ERROR = -1 };\nint run(void);\nint stop(void);\n", - encoding="utf-8", - ) - - project = parse_c_project(tmp_path) - - assert project.functions_by_file["api.h"] == ["run", "stop"] - assert set(project.enum_constants) == {"STATUS_OK", "STATUS_ERROR"} - assert project.enum_constants["STATUS_OK"].value == "0" - - -def test_project_indexes_file_scope_variables(tmp_path: Path): - from prik.parsers.c import parse_c_project - - (tmp_path / "api.h").write_text( - "extern int global_count;\n", - encoding="utf-8", - ) - - project = parse_c_project(tmp_path) - - assert project.variables["global_count"].storage == ["extern"] - - def test_project_function_index_prefers_definition_over_compatible_prototype(tmp_path: Path): from prik.parsers.c import parse_c_project @@ -146,27 +120,6 @@ def test_project_reports_conflicting_function_declarations(tmp_path: Path): assert any(diag.code == "C_CONFLICTING_FUNCTION_DECLARATION" for diag in project.diagnostics) -def test_project_resolves_typedefs_and_struct_tags_across_files(tmp_path: Path): - from prik.parsers.c import CComposedType, CTypedef, parse_c_project - - (tmp_path / "types.h").write_text( - "typedef unsigned long api_size;\nstruct state { int id; };\n", - encoding="utf-8", - ) - (tmp_path / "api.h").write_text( - '#include "types.h"\napi_size count(void);\nvoid step(struct state *s);\n', - encoding="utf-8", - ) - - project = parse_c_project(tmp_path) - - assert isinstance(project.functions["count"].result_type, CTypedef) - assert project.functions["count"].result_type is project.typedefs["api_size"] - param_type = project.functions["step"].parameters[0].type - assert isinstance(param_type, CComposedType) - assert param_type.components[-1] is project.structs["state"] - - def test_project_completes_forward_struct_tags_regardless_of_file_order(): from prik.parsers.c import CComposedType, parse_c_project @@ -233,34 +186,6 @@ def test_project_resolves_typedefs_for_variables_and_aggregate_members(): assert project.structs["packet"].members[0].type is project.typedefs["api_size"] -def test_project_reports_each_typedef_cycle_once_with_structured_diagnostic(): - from prik.parsers.c import parse_c_project - - project = parse_c_project({"cycle.h": "typedef b a;\ntypedef a b;\n"}) - - assert len(project.diagnostics) == 1 - diagnostic = project.diagnostics[0] - assert diagnostic.code == "C_TYPEDEF_CYCLE" - assert diagnostic.message == "Typedef cycle detected: a -> b -> a." - assert diagnostic.severity == "error" - assert diagnostic.location.filename == "cycle.h" - assert diagnostic.unit_kind == "typedef" - assert diagnostic.unit_name == "a" - - -def test_project_reports_prefixed_typedef_cycle_without_including_acyclic_alias(): - from prik.parsers.c import parse_c_project - - project = parse_c_project( - {"cycle.h": ("typedef inner_a alias;\ntypedef inner_b inner_a;\ntypedef inner_a inner_b;\n")} - ) - - cycle_diagnostics = [diagnostic for diagnostic in project.diagnostics if diagnostic.code == "C_TYPEDEF_CYCLE"] - assert len(cycle_diagnostics) == 1 - assert cycle_diagnostics[0].message == "Typedef cycle detected: inner_a -> inner_b -> inner_a." - assert cycle_diagnostics[0].unit_name == "inner_a" - - def test_project_resolves_function_typedef_signature_references(): from prik.parsers.c import CComposedType, CFunctionType, parse_c_project @@ -283,119 +208,160 @@ def test_project_resolves_function_typedef_signature_references(): assert project.functions["select_measure"].result_type is project.typedefs["measure_fn"] -def test_project_resolves_parameter_declared_type_signature_references(): - from prik.parsers.c import CComposedType, CFunctionType, parse_c_project +def test_project_resolves_opaque_pointer_typedefs_across_files(tmp_path: Path): + from prik.parsers.c import CComposedType, CTypedef, parse_c_project - project = parse_c_project( - {"callbacks.h": ("typedef unsigned long api_size;\nvoid apply(api_size callback(api_size));\n")} + (tmp_path / "types.h").write_text( + "struct handle;\ntypedef struct handle *handle_t;\n", + encoding="utf-8", ) + (tmp_path / "api.h").write_text("handle_t open_handle(void);\n", encoding="utf-8") - callback = project.functions["apply"].parameters[0] - declared = callback.declared_type - assert isinstance(declared, CFunctionType) - assert declared.result_type is project.typedefs["api_size"] - assert declared.parameter_types == [project.typedefs["api_size"]] - assert isinstance(callback.type, CComposedType) - assert callback.type.components[1] is declared + project = parse_c_project(tmp_path) + assert project.typedefs["handle_t"].type.components[-1] is project.structs["handle"] + assert project.structs["handle"].is_incomplete is True + assert isinstance(project.functions["open_handle"].result_type, CTypedef) + assert project.functions["open_handle"].result_type is project.typedefs["handle_t"] + assert isinstance(project.typedefs["handle_t"].type, CComposedType) -def test_project_resolves_parameter_declared_array_references(): - from prik.parsers.c import CComposedType, parse_c_project - project = parse_c_project({"arrays.h": ("typedef unsigned long api_size;\nvoid collect(api_size values[4]);\n")}) +def test_project_serialization_keeps_include_indexes_json_stable(tmp_path: Path): + from prik.parsers.c import parse_c_project - values = project.functions["collect"].parameters[0] - assert isinstance(values.declared_type, CComposedType) - assert values.declared_type.components[-1] is project.typedefs["api_size"] - assert isinstance(values.type, CComposedType) - assert values.type.components[-1] is project.typedefs["api_size"] - assert values.declared_type is not values.type + (tmp_path / "api.h").write_text("#include \nint run(void);\n", encoding="utf-8") + + payload = parse_c_project(tmp_path).to_dict() + + assert payload["include_graph"] == {"api.h": []} + assert payload["system_includes"] == {"api.h": ["stddef.h"]} + assert payload["functions_by_file"] == {"api.h": ["run"]} -def test_project_reuses_typedef_cycle_state_across_resolved_use_sites(): +def test_project_indexes_functions_by_file_enum_constants_and_variables(tmp_path: Path): from prik.parsers.c import parse_c_project - project = parse_c_project( - { - "cycle_uses.h": ( - "typedef b a;\n" - "typedef a b;\n" - "typedef a (*cycle_callback)(a);\n" - "a get_value(void);\n" - "void set_value(a value);\n" - "extern a *global_value;\n" - "struct packet { a field; };\n" - ) - } + (tmp_path / "api.h").write_text( + "enum status { STATUS_OK = 0, STATUS_ERROR = -1 };\nint run(void);\nint stop(void);\nextern int global_count;\n", + encoding="utf-8", ) - assert [diagnostic.code for diagnostic in project.diagnostics].count("C_TYPEDEF_CYCLE") == 1 + project = parse_c_project(tmp_path) + + assert project.functions_by_file["api.h"] == ["run", "stop"] + assert set(project.enum_constants) == {"STATUS_OK", "STATUS_ERROR"} + assert project.enum_constants["STATUS_OK"].value == "0" + assert project.variables["global_count"].storage == ["extern"] -def test_project_resolves_union_and_enum_tag_references(tmp_path: Path): - from prik.parsers.c import CComposedType, parse_c_project +def test_project_resolves_typedefs_and_struct_union_enum_tags_across_files(tmp_path: Path): + from prik.parsers.c import CComposedType, CTypedef, parse_c_project (tmp_path / "types.h").write_text( - "union value { int i; };\nenum status { STATUS_OK = 0 };\n", + "typedef unsigned long api_size;\nstruct state { int id; };\nunion value { int i; };\n" + "enum status { STATUS_OK = 0 };\n", encoding="utf-8", ) (tmp_path / "api.h").write_text( + '#include "types.h"\napi_size count(void);\nvoid step(struct state *s);\n' "void set_value(union value *v);\nenum status current_status(void);\n", encoding="utf-8", ) project = parse_c_project(tmp_path) - value_type = project.functions["set_value"].parameters[0].type - assert isinstance(value_type, CComposedType) - assert value_type.components[-1] is project.unions["value"] + assert isinstance(project.functions["count"].result_type, CTypedef) + assert project.functions["count"].result_type is project.typedefs["api_size"] + for function, tag in (("step", project.structs["state"]), ("set_value", project.unions["value"])): + param_type = project.functions[function].parameters[0].type + assert isinstance(param_type, CComposedType) + assert param_type.components[-1] is tag assert project.functions["current_status"].result_type is project.enums["status"] -def test_project_resolves_opaque_pointer_typedefs_across_files(tmp_path: Path): - from prik.parsers.c import CComposedType, CTypedef, parse_c_project +def test_project_resolves_typedefs_inside_adjusted_parameter_declarations(): + from prik.parsers.c import CComposedType, CFunctionType, parse_c_project - (tmp_path / "types.h").write_text( - "struct handle;\ntypedef struct handle *handle_t;\n", - encoding="utf-8", + project = parse_c_project( + { + "callbacks.h": ( + "typedef unsigned long api_size;\n" + "void apply(api_size callback(api_size));\n" + "void collect(api_size values[4]);\n" + ) + } ) - (tmp_path / "api.h").write_text("handle_t open_handle(void);\n", encoding="utf-8") + api_size = project.typedefs["api_size"] - project = parse_c_project(tmp_path) + callback = project.functions["apply"].parameters[0] + assert isinstance(callback.declared_type, CFunctionType) + assert callback.declared_type.result_type is api_size + assert callback.declared_type.parameter_types == [api_size] + assert isinstance(callback.type, CComposedType) + assert callback.type.components[1] is callback.declared_type - assert project.typedefs["handle_t"].type.components[-1] is project.structs["handle"] - assert project.structs["handle"].is_incomplete is True - assert isinstance(project.functions["open_handle"].result_type, CTypedef) - assert project.functions["open_handle"].result_type is project.typedefs["handle_t"] - assert isinstance(project.typedefs["handle_t"].type, CComposedType) + values = project.functions["collect"].parameters[0] + assert values.declared_type.components[-1] is api_size + assert values.type.components[-1] is api_size + assert values.declared_type is not values.type -def test_project_preserves_unresolved_type_references_for_later_diagnostics(): - from prik.parsers.c import CTypedef, parse_c_project +@pytest.mark.parametrize( + ("source", "message", "unit_name"), + [ + pytest.param( + "typedef b a;\ntypedef a b;\n", "Typedef cycle detected: a -> b -> a.", "a", id="two-typedef-cycle" + ), + pytest.param( + "typedef inner_a alias;\ntypedef inner_b inner_a;\ntypedef inner_a inner_b;\n", + "Typedef cycle detected: inner_a -> inner_b -> inner_a.", + "inner_a", + id="acyclic-alias-into-a-cycle-is-not-part-of-it", + ), + pytest.param( + "typedef b a;\ntypedef a b;\ntypedef a (*cycle_callback)(a);\na get_value(void);\n" + "void set_value(a value);\nextern a *global_value;\nstruct packet { a field; };\n", + "Typedef cycle detected: a -> b -> a.", + "a", + id="every-use-site-reuses-one-cycle-report", + ), + ], +) +def test_project_reports_each_typedef_cycle_once_with_structured_diagnostic(source, message, unit_name): + from prik.parsers.c import parse_c_project - project = parse_c_project({"api.h": "missing_type value(void);\n"}) + project = parse_c_project({"cycle.h": source}) - assert isinstance(project.functions["value"].result_type, CTypedef) - assert project.functions["value"].result_type.name == "missing_type" - assert project.functions["value"].result_type.type is None + cycles = [diagnostic for diagnostic in project.diagnostics if diagnostic.code == "C_TYPEDEF_CYCLE"] + assert len(cycles) == 1 + assert cycles[0].message == message + assert cycles[0].severity == "error" + assert cycles[0].location.filename == "cycle.h" + assert cycles[0].unit_kind == "typedef" + assert cycles[0].unit_name == unit_name -def test_project_preserves_unresolved_tag_references_for_later_diagnostics(): - from prik.parsers.c import CComposedType, CEnum, CStruct, CUnion, parse_c_project +def test_project_preserves_unresolved_type_and_tag_references_for_later_diagnostics(): + from prik.parsers.c import CComposedType, CEnum, CStruct, CTypedef, CUnion, parse_c_project project = parse_c_project( - {"api.h": ("struct missing *get_struct(void);\nunion absent *get_union(void);\nenum unknown get_enum(void);\n")} + { + "api.h": ( + "missing_type value(void);\nstruct missing *get_struct(void);\n" + "union absent *get_union(void);\nenum unknown get_enum(void);\n" + ) + } ) - struct_type = project.functions["get_struct"].result_type - union_type = project.functions["get_union"].result_type + missing = project.functions["value"].result_type + assert isinstance(missing, CTypedef) + assert (missing.name, missing.type) == ("missing_type", None) + for function, model, name in (("get_struct", CStruct, "missing"), ("get_union", CUnion, "absent")): + result = project.functions[function].result_type + assert isinstance(result, CComposedType) + assert isinstance(result.components[-1], model) + assert result.components[-1].name == name enum_type = project.functions["get_enum"].result_type - assert isinstance(struct_type, CComposedType) - assert isinstance(struct_type.components[-1], CStruct) - assert struct_type.components[-1].name == "missing" - assert isinstance(union_type, CComposedType) - assert isinstance(union_type.components[-1], CUnion) - assert union_type.components[-1].name == "absent" assert isinstance(enum_type, CEnum) assert enum_type.name == "unknown" @@ -406,35 +372,13 @@ def test_project_header_source_pairs_use_matching_stems_and_direct_includes(tmp_ (tmp_path / "solver.h").write_text("int solve(void);\n", encoding="utf-8") (tmp_path / "solver.c").write_text('#include "solver.h"\n', encoding="utf-8") (tmp_path / "driver.h").write_text("int drive(void);\n", encoding="utf-8") - (tmp_path / "main.c").write_text('#include "driver.h"\n', encoding="utf-8") + (tmp_path / "shared.h").write_text("int shared(void);\n", encoding="utf-8") + (tmp_path / "main.c").write_text('#include "driver.h"\n#include "shared.h"\n', encoding="utf-8") + (tmp_path / "tool.c").write_text('#include "shared.h"\n', encoding="utf-8") project = parse_c_project(tmp_path) assert project.header_source_pairs["solver.h"] == {"solver.c"} assert project.header_source_pairs["driver.h"] == {"main.c"} - - -def test_project_header_source_pairs_preserve_many_to_many_relationships(tmp_path: Path): - from prik.parsers.c import parse_c_project - - (tmp_path / "a.h").write_text("int a(void);\n", encoding="utf-8") - (tmp_path / "b.h").write_text("int b(void);\n", encoding="utf-8") - (tmp_path / "one.c").write_text('#include "a.h"\n#include "b.h"\n', encoding="utf-8") - (tmp_path / "two.c").write_text('#include "a.h"\n', encoding="utf-8") - - project = parse_c_project(tmp_path) - - assert project.header_source_pairs["a.h"] == {"one.c", "two.c"} - assert project.header_source_pairs["b.h"] == {"one.c"} - - -def test_project_serialization_keeps_include_indexes_json_stable(tmp_path: Path): - from prik.parsers.c import parse_c_project - - (tmp_path / "api.h").write_text("#include \nint run(void);\n", encoding="utf-8") - - payload = parse_c_project(tmp_path).to_dict() - - assert payload["include_graph"] == {"api.h": []} - assert payload["system_includes"] == {"api.h": ["stddef.h"]} - assert payload["functions_by_file"] == {"api.h": ["run"]} + # One header can pair with many sources, and one source with many headers. + assert project.header_source_pairs["shared.h"] == {"main.c", "tool.c"} diff --git a/tests/c/infrastructure/parsing/test_c_public_api_skeleton.py b/tests/c/infrastructure/parsing/test_c_public_api_skeleton.py index 34fa19f96..c0b347f32 100644 --- a/tests/c/infrastructure/parsing/test_c_public_api_skeleton.py +++ b/tests/c/infrastructure/parsing/test_c_public_api_skeleton.py @@ -1,130 +1,41 @@ -"""C parser public API coverage for the current partial subset.""" +"""C parser public entry points and JSON serialization of parsed models.""" from pathlib import Path -def test_c_parser_path_and_include_key_helpers_preserve_boundary_contracts(monkeypatch): - from prik.parsers.c.parser import _include_key_from_current, _looks_like_existing_source_path +def test_c_parser_entry_points_accept_inline_path_mapping_and_directory_inputs(tmp_path: Path): + from prik.parsers.c import CFile, CParser, CProject, parse_c_file, parse_c_project - monkeypatch.setattr(Path, "is_file", lambda self: True) - - assert _looks_like_existing_source_path(Path("api.h")) is True - assert _looks_like_existing_source_path("api.h") is True - assert _looks_like_existing_source_path("") is False - assert _looks_like_existing_source_path("int answer(void);\n") is False - assert _looks_like_existing_source_path(object()) is False - assert _include_key_from_current("api.h", "types.h") == "types.h" - assert _include_key_from_current("src/api.h", "types.h") == "src/types.h" - - def raise_os_error(path): - raise OSError - - monkeypatch.setattr(Path, "is_file", raise_os_error) - - assert _looks_like_existing_source_path("api.h") is False - - -def test_c_parser_public_wrappers_forward_explicit_options(monkeypatch): - from prik.parsers.c import parse_c_file, parse_c_project - - calls = [] - - class RecordingParser: - def parse_file(self, *args, **kwargs): - calls.append(("file", args, kwargs)) - return "file-result" - - def parse_project(self, *args, **kwargs): - calls.append(("project", args, kwargs)) - return "project-result" - - monkeypatch.setattr("prik.parsers.c.parser._DEFAULT_PARSER", RecordingParser()) - - include_dirs = [Path("include")] - assert ( - parse_c_file( - "api source", - filename="api.h", - include_dirs=include_dirs, - preprocessing="preprocessed", - encoding="latin-1", - ) - == "file-result" - ) - assert ( - parse_c_project( - {"api.h": "api source"}, - include_dirs=include_dirs, - preprocessing="preprocessed", - encoding="latin-1", - ) - == "project-result" - ) - assert calls == [ - ( - "file", - ("api source",), - { - "filename": "api.h", - "include_dirs": include_dirs, - "preprocessing": "preprocessed", - "encoding": "latin-1", - }, - ), - ( - "project", - ({"api.h": "api source"},), - { - "include_dirs": include_dirs, - "preprocessing": "preprocessed", - "encoding": "latin-1", - }, - ), - ] - - -def test_parse_c_file_accepts_inline_source_and_returns_typed_model(): - from prik.parsers.c import CFile, parse_c_file - - parsed = parse_c_file("int add(int a, int b);\n", filename="inline.h") - - assert isinstance(parsed, CFile) - assert parsed.filename == "inline.h" - assert parsed.language == "c" - assert [fn.name for fn in parsed.functions] == ["add"] - - -def test_c_package_exports_file_and_project_entrypoints(): - from prik.parsers.c import CFile, CProject, parse_c_file, parse_c_project - - parsed = parse_c_file("int add(int left, int right);\n", filename="api.h") - project = parse_c_project({"api.h": "int add(int left, int right);\n"}) - - assert isinstance(parsed, CFile) - assert isinstance(project, CProject) - assert "add" in project.functions - - -def test_parse_c_file_accepts_path_input_and_preserves_filename(tmp_path: Path): - from prik.parsers.c import parse_c_file + inline = parse_c_file("int add(int a, int b);\n", filename="inline.h") + assert isinstance(inline, CFile) + assert (inline.filename, inline.language, [fn.name for fn in inline.functions]) == ("inline.h", "c", ["add"]) + empty = parse_c_file("", filename="empty.src") + assert (empty.functions, empty.diagnostics) == ([], []) header = tmp_path / "api.h" + source = tmp_path / "api.c" header.write_text("double scale(double x);\n", encoding="utf-8") + source.write_text('#include "api.h"\nint answer(void);\n', encoding="utf-8") + (tmp_path / "notes.txt").write_text("ignored\n", encoding="utf-8") + parsed_path = parse_c_file(header) + assert parsed_path.filename == str(header) + assert [fn.name for fn in parsed_path.functions] == ["scale"] - parsed = parse_c_file(header) - - assert parsed.filename == str(header) - assert [fn.name for fn in parsed.functions] == ["scale"] - - -def test_parse_c_file_accepts_empty_source_and_unknown_suffix(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file("", filename="empty.src") + mapping = parse_c_project( + {"types.h": "typedef int api_int;\n", "api.h": '#include "types.h"\napi_int answer(void);\n'} + ) + assert isinstance(mapping, CProject) + assert set(mapping.files) == {"types.h", "api.h"} + assert set(mapping.functions) == {"answer"} + assert set(parse_c_project(source).files) == {str(source)} + # A directory holds C inputs only, keyed relative to it. + assert set(parse_c_project(tmp_path).files) == {"api.h", "api.c"} - assert parsed.filename == "empty.src" - assert parsed.functions == [] - assert parsed.diagnostics == [] + parser = CParser() + assert parser.parse_file("int answer(void);\n", filename="api.h") == parse_c_file( + "int answer(void);\n", filename="api.h" + ) + assert parser.parse_project({"api.h": "int answer(void);\n"}) == parse_c_project({"api.h": "int answer(void);\n"}) def test_parse_c_file_rejects_unknown_preprocessing_mode(): @@ -136,98 +47,6 @@ def test_parse_c_file_rejects_unknown_preprocessing_mode(): parse_c_file("int answer(void);\n", filename="api.h", preprocessing="unknown") -def test_parse_c_project_accepts_mapping_sources(): - from prik.parsers.c import CProject, parse_c_project - - project = parse_c_project( - { - "types.h": "typedef int api_int;\n", - "api.h": '#include "types.h"\napi_int answer(void);\n', - } - ) - - assert isinstance(project, CProject) - assert set(project.files) == {"types.h", "api.h"} - assert project.files["api.h"].language == "c" - assert set(project.functions) == {"answer"} - - -def test_parse_c_project_accepts_single_file_path(tmp_path: Path): - from prik.parsers.c import parse_c_project - - source = tmp_path / "api.c" - source.write_text("int answer(void);\n", encoding="utf-8") - - project = parse_c_project(source) - - assert set(project.files) == {str(source)} - assert set(project.functions) == {"answer"} - - -def test_parse_c_project_indexes_forward_structs_by_tag_name(): - from prik.parsers.c import parse_c_project - - project = parse_c_project( - { - "types.h": "struct handle;\n", - "api.h": "struct handle *open_handle(void);\n", - } - ) - - assert set(project.structs) == {"handle"} - assert project.structs["handle"].is_incomplete is True - assert project.files["types.h"].structs[0].name == "handle" - - -def test_parse_c_project_indexes_named_union_and_enum_tags(): - from prik.parsers.c import parse_c_project - - project = parse_c_project( - { - "types.h": "union value { int i; }; enum status { STATUS_OK = 0 };", - } - ) - - assert set(project.unions) == {"value"} - assert set(project.enums) == {"status"} - - -def test_parse_c_project_accepts_directory_input_with_c_and_h_files(tmp_path: Path): - from prik.parsers.c import parse_c_project - - (tmp_path / "api.h").write_text("int add(int a, int b);\n", encoding="utf-8") - (tmp_path / "api.c").write_text('#include "api.h"\n', encoding="utf-8") - (tmp_path / "notes.txt").write_text("ignored\n", encoding="utf-8") - - project = parse_c_project(tmp_path) - - assert set(project.files) == {"api.h", "api.c"} - - -def test_c_file_serialization_is_json_stable(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file("", filename="empty.c") - - assert parsed.to_dict() == { - "filename": "empty.c", - "language": "c", - "preprocessing": "raw", - "functions": [], - "structs": [], - "unions": [], - "enums": [], - "typedefs": [], - "variables": [], - "macros": [], - "includes": [], - "raw_directives": [], - "macro_dependencies": [], - "diagnostics": [], - } - assert "preprocessing_recipe" not in parsed.to_dict() - - def test_concrete_type_serialization_preserves_semantic_type_fields_and_locations(): from prik.parsers.c import parse_c_file @@ -294,62 +113,3 @@ def test_unresolved_typedef_reference_metadata_is_preserved_in_json(): assert result_type["model"] == "CTypedef" assert result_type["name"] == "api_size" assert result_type["type"] is None - - -def test_c_parser_instance_entrypoints_match_public_functions(): - from prik.parsers.c import CParser, parse_c_file, parse_c_project - - source = "int answer(void);\n" - parser = CParser() - - assert parser.parse_file(source, filename="api.h") == parse_c_file(source, filename="api.h") - assert parser.parse_project({"api.h": source}) == parse_c_project({"api.h": source}) - assert parser._assemble_project({"api.h": parser.parse_file(source, filename="api.h")}) == parse_c_project( - {"api.h": source} - ) - - -def test_c_parse_error_attributes_and_diagnostic_formatting(): - from prik.parsers.c import CArray, CComposedType, CInt, CParseError, CPointer, CSourceLocation - - err = CParseError( - "unexpected token", - filename="bad.h", - line_number=2, - column=5, - source_line="int broken(;", - ) - - assert err.filename == "bad.h" - assert err.line_number == 2 - assert err.column == 5 - assert err.base_message == "unexpected token" - assert err.code == "CPARSE_ERROR" - - diagnostic = err.format_diagnostic(color=False, debug=True) - assert "bad.h:2:5: error[CPARSE_ERROR]: unexpected token" in diagnostic - assert "2 | int broken(;" in diagnostic - assert "note: parser raised at" in diagnostic - - assert CSourceLocation(filename="api.h").display == "api.h" - composed = CComposedType(components=[CPointer(), CArray(bound="4"), CInt()]) - assert composed.pointer_depth == 1 - assert composed.array_rank == 1 - - -def test_c_parse_error_color_and_no_color_formatting(): - from prik.parsers.c import CParseError - - err = CParseError( - "unexpected token", - filename="bad.h", - line_number=2, - column=5, - source_line="int broken(;", - ) - - plain = err.format_diagnostic(color=False) - colored = err.format_diagnostic(color=True) - - assert "\x1b[" not in plain - assert "\x1b[" in colored diff --git a/tests/c/infrastructure/preprocessing/test_c_preprocessing_cli.py b/tests/c/infrastructure/preprocessing/test_c_preprocessing_cli.py index 6707ca74d..306269358 100644 --- a/tests/c/infrastructure/preprocessing/test_c_preprocessing_cli.py +++ b/tests/c/infrastructure/preprocessing/test_c_preprocessing_cli.py @@ -1,156 +1,66 @@ -"""Tests split by stable ownership concept from `test_cli.py`.""" +"""C preprocessing options reach the compiler exactly, and its output reaches the parse report.""" import json from pathlib import Path -import subprocess -import sys +import pytest + +import prik.cli as prik_cli +from prik.preprocessing import PreprocessingError from tests.c._support.preprocessing import ( _failing_compiler, _fake_compiler, ) -def test_cli_c_default_compiler_mode_accepts_include_dirs(tmp_path: Path): - include_dir = tmp_path / "include" - include_dir.mkdir() - dependency = include_dir / "types.h" - header = tmp_path / "api.h" - dependency.write_text("typedef int api_int;\n", encoding="utf-8") - header.write_text('#include "types.h"\nint run(void);\n', encoding="utf-8") - - res = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--json", - "-I", - str(include_dir), - ], - capture_output=True, - text=True, - check=True, - ) - payload = json.loads(res.stdout)[str(header)] - - assert payload["preprocessing"] == "compiler" - assert payload["preprocessing_recipe"]["compiler"] == "cc" - - -def test_cli_c_default_compiler_mode_accepts_define_flags(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("int run(void);\n", encoding="utf-8") - - res = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "-D", - "USE_FAST", - ], - capture_output=True, - text=True, - ) - - assert res.returncode == 0 +def _use_fake_compiler(monkeypatch, tmp_path: Path, output: str) -> tuple[Path, Path]: + compiler, args_file, env = _fake_compiler(tmp_path, output) + for name in ("PRIK_FAKE_COMPILER_ARGS", "PRIK_FAKE_COMPILER_OUTPUT"): + monkeypatch.setenv(name, env[name]) + return compiler, args_file -def test_cli_compiler_mode_uses_default_c_compiler(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("int run(void);\n", encoding="utf-8") - - res = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--json", - ], - capture_output=True, - text=True, - ) - - assert res.returncode == 0 - assert json.loads(res.stdout)[str(header)]["preprocessing_recipe"]["compiler"] == "cc" - - -def test_cli_accepts_explicit_compiler_flag(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("int run(void);\n", encoding="utf-8") - - res = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--compiler", - "gcc-13", - ], - capture_output=True, - text=True, - ) - - assert res.returncode in {0, 1} - - -def test_cli_c_compiler_mode_runs_exact_compiler_and_parses_preprocessed_stdout(tmp_path: Path): +def test_cli_c_compiler_mode_runs_exact_compiler_and_parses_preprocessed_stdout(tmp_path: Path, monkeypatch, capsys): header = tmp_path / "api.h" header.write_text("#define API(ret) ret\nAPI(int) hidden(void);\n", encoding="utf-8") - compiler, args_file, env = _fake_compiler(tmp_path, '# 44 "include/api.h"\nint expanded(void);\n') + compiler, args_file = _use_fake_compiler( + monkeypatch, + tmp_path, + '# 44 "include/api.h"\n#define API_VERSION 3\nint expanded(void);\n', + ) - res = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--json", - "--compiler", - str(compiler), - "-I", - "include", - "-D", - "API_EXPORT=", - "-U", - "DEBUG", - "--std", - "c11", - "--compiler-arg=--sysroot=/opt/sdk", - ], - capture_output=True, - text=True, - check=True, - env=env, + assert ( + prik_cli.main( + [ + "parse", + str(header), + "--language", + "c", + "--json", + "--compiler", + str(compiler), + "-I", + "include", + "-D", + "API_EXPORT=", + "-U", + "DEBUG", + "--std", + "c11", + "--compiler-arg=--sysroot=/opt/sdk", + ] + ) + == 0 ) - payload = json.loads(res.stdout)[str(header)] + payload = json.loads(capsys.readouterr().out)[str(header)] compiler_args = args_file.read_text(encoding="utf-8").splitlines() assert payload["preprocessing"] == "compiler" assert [fn["name"] for fn in payload["functions"]] == ["expanded"] assert "origin" not in payload["functions"][0] assert payload["functions"][0]["source_location"]["filename"] == "include/api.h" - assert payload["functions"][0]["source_location"]["line"] == 44 + assert payload["functions"][0]["source_location"]["line"] == 45 + assert {macro["name"]: macro["value"] for macro in payload["macros"]} == {"API_VERSION": "3"} assert compiler_args == [ "-E", "-x", @@ -178,55 +88,28 @@ def test_cli_c_compiler_mode_runs_exact_compiler_and_parses_preprocessed_stdout( assert payload["original_source_paths"] == ["include/api.h"] -def test_cli_preprocessing_failure_has_category_without_traceback_unless_debug(tmp_path: Path): +def test_cli_preprocessing_failure_has_category_without_traceback_unless_debug(tmp_path: Path, monkeypatch, capsys): header = tmp_path / "api.h" header.write_text("int run(void);\n", encoding="utf-8") compiler = _failing_compiler(tmp_path, "bad option\n") + monkeypatch.delenv("PRIK_DEBUG", raising=False) + argv = ["parse", str(header), "--language", "c", "--compiler", str(compiler)] - res = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--compiler", - str(compiler), - ], - capture_output=True, - text=True, - ) - debug_res = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--compiler", - str(compiler), - "--debug", - ], - capture_output=True, - text=True, - ) + assert prik_cli.main(argv) == 1 + err = capsys.readouterr().err + assert "error[PREPROCESSOR_FAILED]" in err + assert "bad option" in err - assert res.returncode == 1 - assert "error[PREPROCESSOR_FAILED]" in res.stderr - assert "bad option" in res.stderr - assert "Traceback" not in res.stderr - assert debug_res.returncode == 1 - assert "Traceback" in debug_res.stderr + with pytest.raises(PreprocessingError, match="bad option"): + prik_cli.main([*argv, "--debug"]) -def test_cli_c_compile_commands_mode_uses_exact_database_compiler(tmp_path: Path): +def test_cli_c_compile_commands_mode_uses_exact_database_compiler(tmp_path: Path, monkeypatch, capsys): source = tmp_path / "api.c" source.write_text("API(int) hidden(void);\n", encoding="utf-8") - compiler, args_file, env = _fake_compiler(tmp_path, '#line 12 "generated/api.h"\nint from_database(void);\n') + compiler, args_file = _use_fake_compiler( + monkeypatch, tmp_path, '#line 12 "generated/api.h"\nint from_database(void);\n' + ) database = tmp_path / "compile_commands.json" database.write_text( json.dumps( @@ -249,25 +132,8 @@ def test_cli_c_compile_commands_mode_uses_exact_database_compiler(tmp_path: Path encoding="utf-8", ) - res = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "parse", - str(source), - "--language", - "c", - "--json", - "--compile-commands", - str(database), - ], - capture_output=True, - text=True, - check=True, - env=env, - ) - payload = json.loads(res.stdout)[str(source)] + assert prik_cli.main(["parse", str(source), "--language", "c", "--json", "--compile-commands", str(database)]) == 0 + payload = json.loads(capsys.readouterr().out)[str(source)] compiler_args = args_file.read_text(encoding="utf-8").splitlines() assert [fn["name"] for fn in payload["functions"]] == ["from_database"] @@ -281,34 +147,3 @@ def test_cli_c_compile_commands_mode_uses_exact_database_compiler(tmp_path: Path assert recipe["compile_commands"] == str(database) assert recipe["compile_commands_entry"]["file"] == str(source) assert recipe["compile_commands_entry"]["arguments"][0] == str(compiler) - - -def test_cli_c_compiler_mode_macro_metadata_flows_to_parse_report(tmp_path: Path): - header = tmp_path / "api.h" - header.write_text("#define API_VERSION 3\nint api(void);\n", encoding="utf-8") - compiler, _args_file, env = _fake_compiler( - tmp_path, - "#define API_VERSION 3\nint api(void);\n", - ) - - res = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "parse", - str(header), - "--language", - "c", - "--json", - "--compiler", - str(compiler), - ], - capture_output=True, - text=True, - check=True, - env=env, - ) - - macros = {macro["name"]: macro for macro in json.loads(res.stdout)[str(header)]["macros"]} - assert macros["API_VERSION"]["value"] == "3" diff --git a/tests/c/infrastructure/preprocessing/test_c_preprocessing_configuration.py b/tests/c/infrastructure/preprocessing/test_c_preprocessing_configuration.py index cbf9cfab1..0a6517cbe 100644 --- a/tests/c/infrastructure/preprocessing/test_c_preprocessing_configuration.py +++ b/tests/c/infrastructure/preprocessing/test_c_preprocessing_configuration.py @@ -56,163 +56,6 @@ def test_direct_c_preprocess_invocation_uses_exact_compiler_and_flags(tmp_path: ) -def test_preprocessing_metadata_models_and_adapter_helpers(tmp_path: Path): - source = tmp_path / "api.c" - source.write_text("int api(void);\n", encoding="utf-8") - diagnostic = preprocessing.PreprocessingDiagnostic( - category="PREPROCESSOR_FAILED", - message="bad flag", - path=str(source), - line=3, - command=["cc", "-E"], - ) - included = preprocessing.IncludedFile( - path=str(tmp_path / "public.h"), - included_by=str(source), - include_line=1, - ) - mapping = preprocessing.SourceMapping( - generated_line=2, - original_path=str(source), - original_line=7, - include_stack=[str(source)], - ) - macro = preprocessing.MacroDefinition( - name="SQR", - value="((x) * (x))", - function_like=True, - parameters=["x"], - path=str(source), - line=4, - ) - plan = preprocessing.PreprocessingPlan( - language="c", - source_path=str(source), - adapter="direct", - compiler="cc", - include_dirs=["include"], - defines=["API=1"], - undefs=["DEBUG"], - standard="c11", - compiler_args=["-Wall"], - ) - result = preprocessing.PreprocessResult( - source="#define SQR(x) ((x) * (x))\n", - recipe={"mode": "compiler"}, - included_files=[included], - source_mappings=[mapping], - macros=[macro], - diagnostics=[diagnostic], - ) - recipe = preprocessing.PreprocessingRecipe(language="c", compiler="cc", standard="c11") - - assert diagnostic.to_dict() == { - "category": "PREPROCESSOR_FAILED", - "message": "bad flag", - "severity": "error", - "path": str(source), - "line": 3, - "command": ["cc", "-E"], - } - assert plan.to_dict() == { - "language": "c", - "source_path": str(source), - "adapter": "direct", - "compiler": "cc", - "cwd": None, - "include_dirs": ["include"], - "defines": ["API=1"], - "undefs": ["DEBUG"], - "standard": "c11", - "compiler_args": ["-Wall"], - "compile_commands": None, - "command_template": None, - } - assert included.to_dict() == { - "path": str(tmp_path / "public.h"), - "included_by": str(source), - "include_line": 1, - "mechanism": "cpp_include", - "dependency_kind": "project", - "exposure": "public", - } - assert mapping.to_dict() == { - "generated_line": 2, - "original_path": str(source), - "original_line": 7, - "include_stack": [str(source)], - } - assert macro.to_dict() == { - "name": "SQR", - "value": "((x) * (x))", - "function_like": True, - "parameters": ["x"], - "path": str(source), - "line": 4, - "builtin": False, - } - assert result.to_dict() == { - "source": "#define SQR(x) ((x) * (x))\n", - "recipe": {"mode": "compiler"}, - "included_files": [included.to_dict()], - "source_mappings": [mapping.to_dict()], - "macros": [macro.to_dict()], - "diagnostics": [diagnostic.to_dict()], - } - assert recipe.std == "c11" - assert recipe.to_dict() == { - "language": "c", - "compiler": "cc", - "mode": "compiler", - "adapter": "direct", - "argv": [], - "cwd": None, - "include_dirs": [], - "defines": [], - "undefs": [], - "standard": "c11", - "std": "c11", - "compiler_args": [], - "source_path": None, - "source_file": None, - "compile_commands": None, - "compile_commands_entry": None, - "command_template": None, - "included_files": [], - "source_mappings": [], - "macros": [], - "diagnostics": [], - "capabilities": {}, - } - - adapter = preprocessing.GCCCompatibleCAdapter() - config = PreprocessingConfig(mode="compiler", compiler="cc") - assert adapter.build_preprocess_invocation(source, language="c", config=config) == preprocessing.Invocation( - argv=["cc", "-E", "-x", "c", str(source)], - cwd=None, - adapter="gcc-compatible-c", - language="c", - compiler="cc", - capabilities={"dependency_output": True, "macro_dump": True, "linemarkers": True}, - ) - assert adapter.collect_dependencies(result) == [included] - assert adapter.collect_macros(result) == [macro] - assert adapter.parse_linemarkers('#line 7 "dir\\\\api\\".h"\nint x;\n')[0].original_line == 7 - assert adapter.parse_linemarkers("int x;\n", filename="api.h")[0].original_path == "api.h" - - invocation = preprocessing.CommandTemplateAdapter().build_preprocess_invocation( - source, - language="c", - config=PreprocessingConfig( - mode="compiler", - compiler="vendor-cc", - adapter="command-template", - command_template="{compiler} --lang {language} {source}", - ), - ) - assert invocation.argv == ["vendor-cc", "--lang", "c", str(source)] - - def test_direct_preprocess_invocation_rejects_missing_compiler_and_unknown_language(tmp_path: Path): source = tmp_path / "input.txt" with pytest.raises(PreprocessingError) as exc_info: @@ -245,7 +88,7 @@ def test_compile_commands_invocation_uses_database_compiler_and_filters_compile_ [ { "directory": str(tmp_path), - "file": str(source), + "file": "src/api.c", "arguments": [ str(fake_compiler), "-Iproject/include", @@ -280,7 +123,7 @@ def test_compile_commands_invocation_uses_database_compiler_and_filters_compile_ compile_commands=str(database), compile_commands_entry={ "directory": str(tmp_path), - "file": str(source), + "file": "src/api.c", "arguments": [ str(fake_compiler), "-Iproject/include", @@ -356,132 +199,37 @@ def test_compile_commands_invocation_reports_missing_file_and_supports_command_s assert invocation.argv == ["clang", "-E", str(source)] -def test_compile_commands_internal_helpers_cover_paths_commands_and_selection(monkeypatch, tmp_path: Path): - source = tmp_path / "src" / "api.c" - source.parent.mkdir() - source.write_text("int api(void);\n", encoding="utf-8") - relative_entry = {"directory": str(tmp_path), "file": "src/api.c"} - absolute_entry = {"directory": "/ignored", "file": str(source)} - - assert preprocessing._entry_file_path(relative_entry) == source - assert preprocessing._entry_file_path(absolute_entry) == source - assert preprocessing._entry_file_path({"file": "api.c"}) == Path("api.c") - with pytest.raises(PreprocessingError) as exc_info: - preprocessing._entry_file_path({}) - _assert_preprocessing_error(exc_info, message="compile_commands entry is missing 'file'") - - assert preprocessing._compile_command_argv({"arguments": ["cc", 7]}) == ["cc", "7"] - assert preprocessing._compile_command_argv({"command": "cc -DNAME='two words' api.c"}) == [ - "cc", - "-DNAME=two words", - "api.c", - ] - for entry, message in [ - ({"arguments": "cc api.c"}, "compile_commands entry 'arguments' must contain a list"), - ({"command": ["cc", "api.c"]}, "compile_commands entry 'command' must contain a string"), - ({}, "compile_commands entry must contain 'arguments' or 'command'"), - ({"arguments": []}, "compile_commands entry has an empty command"), - ]: - with pytest.raises(PreprocessingError) as exc_info: - preprocessing._compile_command_argv(entry) - _assert_preprocessing_error(exc_info, message=message) - - args = [ - "-c", - "/c", - "-o", - "api.o", - "-oother.o", - "/Fowindows.obj", - "-MF", - "deps.d", - "-MT", - "api.o", - "-MQ", - "api.o", - "-MFdeps2.d", - "-MTapi", - "-MQapi", - "src/api.c", - "-Wall", - ] - assert preprocessing._filter_compile_only_args(args, source, tmp_path) == ["-Wall"] - assert preprocessing._compile_commands_entry(source, [relative_entry]) == relative_entry - with pytest.raises(PreprocessingError) as exc_info: - preprocessing._compile_commands_entry(source, [1]) - _assert_preprocessing_error(exc_info, message="compile_commands entries must be objects") - - original_resolve = Path.resolve - - def fail_resolve(path: Path): - raise OSError(f"cannot resolve {path}") - - monkeypatch.setattr(Path, "resolve", fail_resolve) - try: - assert preprocessing._same_source(source, source) - finally: - monkeypatch.setattr(Path, "resolve", original_resolve) - - -def test_load_compile_commands_uses_explicit_utf8_and_exact_errors(monkeypatch, tmp_path: Path): - database = tmp_path / "compile_commands.json" - seen_encodings = [] - - def read_empty_database(_path: Path, *, encoding: str): - seen_encodings.append(encoding) - return "[]" - - monkeypatch.setattr(Path, "read_text", read_empty_database) - assert preprocessing._load_compile_commands(database) == [] - assert seen_encodings == ["utf-8"] - - def fail_read(_path: Path, *, encoding: str): - raise OSError(f"denied with {encoding}") - - monkeypatch.setattr(Path, "read_text", fail_read) - with pytest.raises(PreprocessingError) as exc_info: - preprocessing._load_compile_commands(database) - _assert_preprocessing_error( - exc_info, - message=f"cannot read compile commands file {database}: denied with utf-8", - ) - - monkeypatch.setattr(Path, "read_text", lambda _path, *, encoding: "not json") - with pytest.raises(PreprocessingError) as exc_info: - preprocessing._load_compile_commands(database) - _assert_preprocessing_error( - exc_info, - message="invalid compile commands JSON: Expecting value: line 1 column 1 (char 0)", - ) - - monkeypatch.setattr(Path, "read_text", lambda _path, *, encoding: "{}") - with pytest.raises(PreprocessingError) as exc_info: - preprocessing._load_compile_commands(database) - _assert_preprocessing_error(exc_info, message="compile_commands.json must contain a list") - - @pytest.mark.parametrize( "compile_only_args", [ - ["-c"], - ["/c"], - ["-o", "api.o"], - ["-oapi.o"], - ["/Foapi.o"], - ["-MF", "deps.d"], - ["-MT", "api.o"], - ["-MQ", "api.o"], - ["-MFdeps.d"], - ["-MTapi.o"], - ["-MQapi.o"], + pytest.param(["-c"], id="compile-only"), + pytest.param(["/c"], id="msvc-compile-only"), + pytest.param(["-o", "api.o"], id="separate-output"), + pytest.param(["-oapi.o"], id="joined-output"), + pytest.param(["/Foapi.obj"], id="msvc-output"), + pytest.param(["-MF", "deps.d", "-MT", "api.o", "-MQ", "api.o"], id="separate-dependency-outputs"), + pytest.param(["-MFdeps.d", "-MTapi.o", "-MQapi.o"], id="joined-dependency-outputs"), ], ) -def test_compile_only_arg_filter_removes_each_flag_without_skipping_following_args( - compile_only_args: list[str], tmp_path: Path +def test_compile_commands_invocation_drops_compile_only_args_without_skipping_following_args( + tmp_path: Path, compile_only_args: list[str] ): source = tmp_path / "api.c" + source.write_text("int api(void);\n", encoding="utf-8") + database = tmp_path / "compile_commands.json" + database.write_text( + json.dumps( + [{"directory": str(tmp_path), "file": "api.c", "arguments": ["cc", *compile_only_args, "-Wall", "api.c"]}] + ), + encoding="utf-8", + ) - assert preprocessing._filter_compile_only_args([*compile_only_args, "-Wall"], source, tmp_path) == ["-Wall"] + invocation = build_compile_commands_invocation( + source, + config=PreprocessingConfig(mode="compiler", compile_commands=str(database)), + ) + + assert invocation.argv == ["cc", "-E", "-Wall", str(source)] def test_compile_commands_invocation_defaults_missing_directory_to_current_directory(tmp_path: Path): @@ -507,7 +255,10 @@ def test_command_template_preprocess_invocation_expands_placeholders(tmp_path: P config = PreprocessingConfig( mode="compiler", adapter="command-template", - command_template="vendor-cc --preprocess {include_dirs} {defines} {undefs} {standard} {compiler_args} {source}", + command_template=( + "vendor-cc --preprocess {include_dirs} {defines} {undefs} {standard} {compiler_args} " + "--meta={language}:{standard} {source}" + ), include_dirs=["include"], defines=["API_EXPORT="], undefs=["DEBUG"], @@ -526,6 +277,7 @@ def test_command_template_preprocess_invocation_expands_placeholders(tmp_path: P "-UDEBUG", "-std=c11", "--target=x86_64-linux", + "--meta=c:c11", str(source), ], adapter="command-template", @@ -535,49 +287,6 @@ def test_command_template_preprocess_invocation_expands_placeholders(tmp_path: P ) -def test_command_template_tokens_expand_exactly(tmp_path: Path): - source = tmp_path / "api.h" - config = PreprocessingConfig( - mode="compiler", - compiler="vendor-cc", - include_dirs=["include"], - defines=["API=1"], - undefs=["DEBUG"], - std="c11", - compiler_args=["--target=x86_64-linux"], - ) - - assert preprocessing._template_token_value("{source}", source, "c", config) == [str(source)] - assert preprocessing._template_token_value("{compiler}", source, "c", config) == ["vendor-cc"] - assert preprocessing._template_token_value("{language}", source, "c", config) == ["c"] - assert preprocessing._template_token_value("{include_dirs}", source, "c", config) == ["-Iinclude"] - assert preprocessing._template_token_value("{defines}", source, "c", config) == ["-DAPI=1"] - assert preprocessing._template_token_value("{undefs}", source, "c", config) == ["-UDEBUG"] - assert preprocessing._template_token_value("{standard}", source, "c", config) == ["-std=c11"] - assert preprocessing._template_token_value("{compiler_args}", source, "c", config) == ["--target=x86_64-linux"] - assert preprocessing._template_token_value("--std={standard}", source, "c", config) == ["--std=c11"] - assert preprocessing._template_token_value( - "--meta={source}:{compiler}:{language}:{standard}", - source, - "c", - config, - ) == [f"--meta={source}:vendor-cc:c:c11"] - assert preprocessing._template_token_value("{standard}", source, "c", PreprocessingConfig()) == [] - assert preprocessing._template_token_value("{compiler}", source, "c", PreprocessingConfig()) == [""] - assert preprocessing._template_token_value("{include_dirs}", source, "c", PreprocessingConfig()) == [] - assert preprocessing._template_token_value("{defines}", source, "c", PreprocessingConfig()) == [] - assert preprocessing._template_token_value("{undefs}", source, "c", PreprocessingConfig()) == [] - assert preprocessing._template_token_value("{compiler_args}", source, "c", PreprocessingConfig()) == [] - assert preprocessing._template_token_value( - "--meta={compiler}:{standard}", - source, - "c", - PreprocessingConfig(), - ) == ["--meta=:"] - with pytest.raises(KeyError, match="unknown"): - preprocessing._template_token_value("{unknown}", source, "c", PreprocessingConfig()) - - def test_command_template_validation_and_dispatch_edges(tmp_path: Path): source = tmp_path / "api.h" source.write_text("int api(void);\n", encoding="utf-8") diff --git a/tests/c/infrastructure/preprocessing/test_c_preprocessing_dependencies.py b/tests/c/infrastructure/preprocessing/test_c_preprocessing_dependencies.py index d8c99de99..c16df42fa 100644 --- a/tests/c/infrastructure/preprocessing/test_c_preprocessing_dependencies.py +++ b/tests/c/infrastructure/preprocessing/test_c_preprocessing_dependencies.py @@ -1,48 +1,9 @@ -"""Tests split by stable ownership concept from `test_cli.py`.""" +"""Compiler linemarkers become include dependencies, exposure, source mappings, and macros.""" -import json from pathlib import Path import prik.preprocessing.source as preprocessing -from prik.preprocessing import PreprocessingConfig, build_compile_commands_invocation - - -def test_compile_commands_filters_dependency_and_windows_compile_flags(tmp_path: Path): - source = tmp_path / "src" / "api.c" - source.parent.mkdir() - source.write_text("int api(void);\n", encoding="utf-8") - compiler = tmp_path / "cc" - database = tmp_path / "compile_commands.json" - database.write_text( - json.dumps( - [ - { - "directory": str(tmp_path), - "file": str(source), - "arguments": [ - str(compiler), - "-MF", - "deps.d", - "-MT", - "api.o", - "-MQtarget", - "-MFdeps2.d", - "/c", - "src/api.c", - "-Wall", - ], - } - ] - ), - encoding="utf-8", - ) - - invocation = build_compile_commands_invocation( - source, - config=PreprocessingConfig(mode="compiler", compile_commands=str(database)), - ) - - assert invocation.argv == [str(compiler), "-E", "-Wall", str(source)] +from prik.preprocessing import PreprocessingConfig def test_linemarker_dependency_exposure_and_macro_edges(tmp_path: Path): @@ -80,34 +41,9 @@ def test_linemarker_dependency_exposure_and_macro_edges(tmp_path: Path): assert mappings[0].original_line == 7 assert 'api".h' in mappings[0].original_path - assert preprocessing._unescape_linemarker_filename(r"a\nb\rc\td\\e\"f\x") == 'a\nb\rc\td\\e"fx' - assert preprocessing._unescape_linemarker_filename("trailing\\") == "trailing\\" - assert preprocessing._parse_linemarker('# 12 "api.h" 1 3') == (12, "api.h", [1, 3]) - assert preprocessing._parse_linemarker("#line 14 api.h") == (14, "api.h", []) - assert preprocessing._parse_linemarker("int api;") is None - assert preprocessing._dependency_kind("api.h", [3]) == "system" - assert preprocessing._dependency_kind("") == "system" - assert preprocessing._dependency_kind("api.h") == "project" - assert preprocessing._exposure_for( - "private/api.h", "project", PreprocessingConfig(private_includes=["private"]) - ) == ("private") - assert preprocessing._exposure_for("public/api.h", "project", PreprocessingConfig(public_includes=["public"])) == ( - "public" - ) - assert preprocessing._exposure_for("api.h", "system", PreprocessingConfig()) == "private" - assert preprocessing._exposure_for("api.h", "project", PreprocessingConfig(include_exposure="roots-only")) == ( - "private" - ) - assert preprocessing._exposure_for("api.h", "root", PreprocessingConfig(include_exposure="roots-only")) == "public" no_filename_mappings = preprocessing.parse_linemarker_mappings("#line 42\nint next;\n", filename=str(root)) assert no_filename_mappings[0].original_path == str(root) assert no_filename_mappings[0].original_line == 42 - assert preprocessing._included_files_from_linemarkers( - "#line 5\nint next;\n", - root_path=root, - language="c", - config=PreprocessingConfig(), - ) == [files[0]] assert macros[0].name == "BUILTIN" assert macros[0].builtin is True assert by_path[str(root)].dependency_kind == "root" @@ -202,15 +138,6 @@ def test_linemarker_dependency_exposure_and_macro_edges(tmp_path: Path): ] -def test_linemarker_parser_accepts_bare_filename(): - assert preprocessing._parse_linemarker("# 14 api.h") == (14, "api.h", []) - - -def test_dependency_kind_requires_both_system_filename_brackets(): - assert preprocessing._dependency_kind("") == "project" - - def test_linemarker_mapping_and_macro_helpers_cover_default_and_return_edges(): source = "\n".join( [ diff --git a/tests/c/infrastructure/preprocessing/test_c_preprocessing_execution.py b/tests/c/infrastructure/preprocessing/test_c_preprocessing_execution.py index 4f63fee24..06269a8f8 100644 --- a/tests/c/infrastructure/preprocessing/test_c_preprocessing_execution.py +++ b/tests/c/infrastructure/preprocessing/test_c_preprocessing_execution.py @@ -1,224 +1,155 @@ -"""Tests split by stable ownership concept from `test_cli.py`.""" +"""C compiler-preprocessor execution: failure categories, provenance, and recipe macros.""" -import json from pathlib import Path +import subprocess +import sys import pytest import prik.preprocessing.source as preprocessing -from prik.preprocessing import ( - PreprocessingConfig, - PreprocessingError, - run_compiler_preprocessor, - run_compiler_preprocessor_with_recipe, -) +from prik.parsers.c import sources as c_sources +from prik.preprocessing import PreprocessingConfig, PreprocessingError -def test_run_compiler_preprocessor_success_and_failures(monkeypatch, tmp_path: Path): - config = PreprocessingConfig(mode="compiler", compiler="cc") - source = tmp_path / "api.c" - source.write_text("int api(void);\n", encoding="utf-8") - calls = [] - - def succeed(*args, **kwargs): - calls.append((args, kwargs)) - return type("Done", (), {"returncode": 0, "stdout": "expanded", "stderr": ""})() - - monkeypatch.setattr(preprocessing.subprocess, "run", succeed) - expanded, recipe = run_compiler_preprocessor_with_recipe(source, language="c", config=config) - assert expanded == "expanded" - assert recipe.compiler == "cc" - assert run_compiler_preprocessor(source, language="c", config=config) == "expanded" - assert calls == [ - ( - (["cc", "-E", "-x", "c", str(source)],), - {"cwd": None, "capture_output": True, "text": True, "timeout": 60, "check": False}, - ), - ( - (["cc", "-E", "-x", "c", str(source)],), - {"cwd": None, "capture_output": True, "text": True, "timeout": 60, "check": False}, - ), - ] +def _completed(returncode: int, stdout: str = "", stderr: str = ""): + return lambda *_args, **_kwargs: subprocess.CompletedProcess([], returncode, stdout=stdout, stderr=stderr) - def raise_oserror(*_args, **_kwargs): - raise OSError("cannot start") - monkeypatch.setattr(preprocessing.subprocess, "run", raise_oserror) - with pytest.raises(PreprocessingError) as exc_info: - run_compiler_preprocessor(source, language="c", config=config) - assert str(exc_info.value) == "failed to run compiler preprocessor: cannot start" - assert exc_info.value.category == "PREPROCESSOR_FAILED" - assert [diagnostic.to_dict() for diagnostic in exc_info.value.diagnostics] == [ - { - "category": "PREPROCESSOR_FAILED", - "message": "failed to run compiler preprocessor: cannot start", - "severity": "error", - "path": None, - "line": None, - "command": ["cc", "-E", "-x", "c", str(source)], - } - ] +def _raise(error: Exception): + def run(*_args, **_kwargs): + raise error - monkeypatch.setattr( - preprocessing.subprocess, - "run", - lambda *_args, **_kwargs: type("Done", (), {"returncode": 1, "stdout": "", "stderr": "bad option"})(), - ) - with pytest.raises(PreprocessingError) as exc_info: - run_compiler_preprocessor(source, language="c", config=config) - assert str(exc_info.value) == "compiler preprocessing failed with exit code 1\nbad option" - assert exc_info.value.category == "PREPROCESSOR_FAILED" - assert [diagnostic.to_dict() for diagnostic in exc_info.value.diagnostics] == [ - { - "category": "PREPROCESSOR_FAILED", - "message": "bad option", - "severity": "error", - "path": None, - "line": None, - "command": ["cc", "-E", "-x", "c", str(source)], - } - ] + return run -def test_preprocess_source_preserves_exact_success_metadata(monkeypatch, tmp_path: Path): +@pytest.mark.parametrize( + ("compiler", "run", "category", "message", "diagnostic"), + [ + pytest.param( + None, + None, + "INVALID_COMPILER_ARGUMENTS", + "Compiler preprocessing not configured", + None, + id="not-configured", + ), + pytest.param( + "prik-definitely-missing-preprocessor", + None, + "PREPROCESSOR_NOT_FOUND", + "preprocessor not found: prik-definitely-missing-preprocessor", + "preprocessor not found: prik-definitely-missing-preprocessor", + id="executable-not-on-path", + ), + pytest.param( + "missing-cc", + _raise(FileNotFoundError("missing")), + "PREPROCESSOR_NOT_FOUND", + "preprocessor not found: {compiler}", + "preprocessor not found: {compiler}", + id="executable-vanished", + ), + pytest.param( + "cannot-start-cc", + _raise(OSError("cannot start")), + "PREPROCESSOR_FAILED", + "failed to run compiler preprocessor: cannot start", + "failed to run compiler preprocessor: cannot start", + id="executable-cannot-start", + ), + pytest.param( + "slow-cc", + _raise(subprocess.TimeoutExpired(cmd="cc", timeout=60)), + "PREPROCESSOR_FAILED", + "compiler preprocessing failed: timed out after 60 seconds", + "compiler preprocessing timed out after 60 seconds", + id="timeout", + ), + pytest.param( + "failing-cc", + _completed(1, stderr="bad option"), + "PREPROCESSOR_FAILED", + "compiler preprocessing failed with exit code 1\nbad option", + "bad option", + id="nonzero-exit-with-stderr", + ), + pytest.param( + "bad-cc", + _completed(2), + "PREPROCESSOR_FAILED", + "compiler preprocessing failed with exit code 2", + "compiler preprocessing failed with exit code 2", + id="silent-nonzero-exit", + ), + ], +) +def test_preprocess_source_reports_each_failure_with_its_category( + monkeypatch, tmp_path: Path, compiler, run, category, message, diagnostic +): source = tmp_path / "api.c" source.write_text("int api(void);\n", encoding="utf-8") - expanded = f'# 1 "{source}"\n#define API 1\nint value;\n' - monkeypatch.setattr( - preprocessing.subprocess, - "run", - lambda *_args, **_kwargs: type("Done", (), {"returncode": 0, "stdout": expanded, "stderr": ""})(), - ) - config = PreprocessingConfig( - mode="compiler", - compiler=str(tmp_path / "cc"), - include_dirs=["include"], - defines=["CLI=1"], - undefs=["DEBUG"], - std="c11", - compiler_args=["-dD"], - ) + if run is not None: + # A path-like executable skips the PATH lookup, so the stubbed launch decides the outcome. + compiler = str(tmp_path / compiler) + monkeypatch.setattr(preprocessing.subprocess, "run", run) + message = message.format(compiler=compiler) + diagnostic = diagnostic.format(compiler=compiler) + config = PreprocessingConfig() if compiler is None else PreprocessingConfig(mode="compiler", compiler=compiler) - result = preprocessing.preprocess_source(source, language="c", config=config) - - argv = [ - str(tmp_path / "cc"), - "-E", - "-x", - "c", - "-Iinclude", - "-DCLI=1", - "-UDEBUG", - "-std=c11", - "-dD", - str(source), - ] - included_files = [ - { - "path": str(source), - "included_by": None, - "include_line": None, - "mechanism": "c_include", - "dependency_kind": "root", - "exposure": "public", - } - ] - mappings = [ - { - "generated_line": 2, - "original_path": str(source), - "original_line": 1, - "include_stack": [str(source)], - }, - { - "generated_line": 3, - "original_path": str(source), - "original_line": 2, - "include_stack": [str(source)], - }, - ] - macros = [ - { - "name": "API", - "value": "1", - "function_like": False, - "parameters": None, - "path": str(source), - "line": 1, - "builtin": False, - } - ] - recipe = { - "language": "c", - "compiler": str(tmp_path / "cc"), - "mode": "compiler", - "adapter": "gcc-compatible-c", - "argv": argv, - "cwd": None, - "include_dirs": ["include"], - "defines": ["CLI=1"], - "undefs": ["DEBUG"], - "standard": "c11", - "std": "c11", - "compiler_args": ["-dD"], - "source_path": str(source), - "source_file": str(source), - "compile_commands": None, - "compile_commands_entry": None, - "command_template": None, - "included_files": included_files, - "source_mappings": mappings, - "macros": macros, - "diagnostics": [], - "capabilities": {"dependency_output": True, "macro_dump": True, "linemarkers": True}, - } - assert result.to_dict() == { - "source": expanded, - "recipe": recipe, - "included_files": included_files, - "source_mappings": mappings, - "macros": macros, - "diagnostics": [], - } - - -def test_run_compiler_preprocessor_with_recipe_restores_sparse_recipe_defaults(monkeypatch, tmp_path: Path): - source = tmp_path / "api.c" - result = preprocessing.PreprocessResult(source="expanded\n", recipe={"language": "c"}) - monkeypatch.setattr(preprocessing, "preprocess_source", lambda *_args, **_kwargs: result) - - expanded, recipe = run_compiler_preprocessor_with_recipe(source, language="c", config=PreprocessingConfig()) - - assert expanded == "expanded\n" - assert recipe.mode == "compiler" - assert recipe.adapter == "direct" - - -def test_preprocess_source_uses_compile_database_working_directory(monkeypatch, tmp_path: Path): + with pytest.raises(PreprocessingError) as exc_info: + preprocessing.preprocess_source(source, language="c", config=config) + + assert str(exc_info.value) == message + assert exc_info.value.category == category + expected = [] + if diagnostic is not None: + expected = [ + { + "category": category, + "message": diagnostic, + "severity": "error", + "path": None, + "line": None, + "command": [compiler, "-E", "-x", "c", str(source)], + } + ] + assert [item.to_dict() for item in exc_info.value.diagnostics] == expected + + +def test_preprocess_source_warns_when_the_adapter_output_has_no_source_mapping(monkeypatch, tmp_path: Path): source = tmp_path / "api.c" source.write_text("int api(void);\n", encoding="utf-8") - compiler = tmp_path / "cc" - database = tmp_path / "compile_commands.json" - database.write_text( - json.dumps([{"directory": str(tmp_path), "file": str(source), "arguments": [str(compiler), str(source)]}]), - encoding="utf-8", - ) - calls = [] - - def succeed(*args, **kwargs): - calls.append((args, kwargs)) - return type("Done", (), {"returncode": 0, "stdout": "int api(void);\n", "stderr": ""})() + monkeypatch.setattr(preprocessing.subprocess, "run", _completed(0)) - monkeypatch.setattr(preprocessing.subprocess, "run", succeed) - - result = preprocessing.preprocess_source( + template = preprocessing.preprocess_source( source, language="c", - config=PreprocessingConfig(mode="compiler", compile_commands=str(database)), + config=PreprocessingConfig( + mode="compiler", + adapter="command-template", + command_template=f"{sys.executable} {{source}}", + ), ) + direct_config = PreprocessingConfig(mode="compiler", compiler=str(tmp_path / "cc")) + empty_direct = preprocessing.preprocess_source(source, language="c", config=direct_config) + monkeypatch.setattr(preprocessing.subprocess, "run", _completed(0, stdout="int api(void);\n")) + direct = preprocessing.preprocess_source(source, language="c", config=direct_config) - assert result.source == "int api(void);\n" - assert result.source_mappings == [ + assert [item.to_dict() for item in template.diagnostics] == [ + { + "category": "PROVENANCE_UNAVAILABLE", + "message": "selected compiler adapter did not provide source linemarkers", + "severity": "warning", + "path": None, + "line": None, + "command": [sys.executable, str(source)], + } + ] + # A linemarker-capable adapter is trusted even when its output is empty. + assert empty_direct.diagnostics == [] + assert direct.diagnostics == [] + # Output without linemarkers maps line for line onto the root source. + assert direct.source_mappings == [ preprocessing.SourceMapping( generated_line=1, original_path=str(source), @@ -226,9 +157,36 @@ def succeed(*args, **kwargs): include_stack=[str(source)], ) ] - assert calls == [ - ( - ([str(compiler), "-E", str(source)],), - {"cwd": str(tmp_path), "capture_output": True, "text": True, "timeout": 60, "check": False}, - ) - ] + + +def test_attach_preprocessing_recipe_filters_invalid_and_duplicate_macros(): + empty = c_sources.CFile() + c_sources.attach_preprocessing_recipe(empty, None) + assert empty.preprocessing_recipe is None + + parsed = c_sources.CFile( + macros=[ + c_sources.CMacro( + name="EXISTING", + source_location=c_sources.CSourceLocation(filename="api.h", line=2), + ) + ] + ) + recipe = { + "macros": [ + None, + {"name": ""}, + {"name": "EXISTING", "path": "api.h", "line": 2}, + {"name": "NEW", "value": 123, "function_like": 1, "path": 42, "line": "bad"}, + {"name": "WITH_LOC", "value": "1", "path": "api.h", "line": 4}, + ] + } + + c_sources.attach_preprocessing_recipe(parsed, recipe) + + assert parsed.preprocessing_recipe == recipe + assert [macro.name for macro in parsed.macros] == ["EXISTING", "NEW", "WITH_LOC"] + assert parsed.macros[1].value is None + assert parsed.macros[1].function_like is True + assert parsed.macros[1].source_location.filename is None + assert parsed.macros[2].source_location.line == 4 diff --git a/tests/c/infrastructure/preprocessing/test_c_preprocessing_properties.py b/tests/c/infrastructure/preprocessing/test_c_preprocessing_properties.py index 018aae0bb..1de1651e9 100644 --- a/tests/c/infrastructure/preprocessing/test_c_preprocessing_properties.py +++ b/tests/c/infrastructure/preprocessing/test_c_preprocessing_properties.py @@ -3,26 +3,12 @@ import pytest from hypothesis import given, strategies as st -from prik.parsers.c import CParseError, parse_c_file +from prik.parsers.c import parse_c_file from tests.c._support.parser_properties import ( _C_IDENTIFIERS, ) -@pytest.mark.property -@given( - feature=_C_IDENTIFIERS, - function_names=st.lists(_C_IDENTIFIERS, min_size=2, max_size=2, unique=True), -) -def test_generated_c_raw_conditionals_require_preprocessing(feature, function_names): - source = f"#ifdef {feature}\nint {function_names[0]}(void);\n#else\nint {function_names[1]}(void);\n#endif\n" - - with pytest.raises(CParseError, match="require compiler preprocessing") as exc_info: - parse_c_file(source, filename="conditional.h", preprocessing="raw") - - assert exc_info.value.code == "CPARSE_PREPROCESSING_REQUIRED" - - @pytest.mark.property @given(line_number=st.integers(min_value=1, max_value=100_000), stem=_C_IDENTIFIERS) def test_generated_c_linemarkers_map_function_origin(line_number, stem): diff --git a/tests/c/infrastructure/preprocessing/test_error_paths.py b/tests/c/infrastructure/preprocessing/test_error_paths.py deleted file mode 100644 index 2d1909b17..000000000 --- a/tests/c/infrastructure/preprocessing/test_error_paths.py +++ /dev/null @@ -1,152 +0,0 @@ -"""Tests split by stable ownership concept from `test_cli.py`.""" - -from pathlib import Path -import subprocess -import sys - -import pytest - -import prik.preprocessing.source as preprocessing -from prik.preprocessing import ( - PreprocessingConfig, - PreprocessingError, -) - - -def test_preprocess_source_c_error_paths(monkeypatch, tmp_path: Path): - c_source = tmp_path / "api.c" - c_source.write_text("int api(void);\n", encoding="utf-8") - - with pytest.raises(PreprocessingError, match="not configured") as exc_info: - preprocessing.preprocess_source(c_source, language="c", config=PreprocessingConfig()) - assert str(exc_info.value) == "Compiler preprocessing not configured" - assert exc_info.value.category == "INVALID_COMPILER_ARGUMENTS" - assert exc_info.value.diagnostics == [] - - missing_name = "prik-definitely-missing-preprocessor" - with pytest.raises(PreprocessingError, match="preprocessor not found") as exc_info: - preprocessing.preprocess_source( - c_source, - language="c", - config=PreprocessingConfig(mode="compiler", compiler=missing_name), - ) - assert exc_info.value.category == "PREPROCESSOR_NOT_FOUND" - assert [diagnostic.to_dict() for diagnostic in exc_info.value.diagnostics] == [ - { - "category": "PREPROCESSOR_NOT_FOUND", - "message": f"preprocessor not found: {missing_name}", - "severity": "error", - "path": None, - "line": None, - "command": [missing_name, "-E", "-x", "c", str(c_source)], - } - ] - - def raise_file_not_found(*_args, **_kwargs): - raise FileNotFoundError("missing") - - monkeypatch.setattr(preprocessing.subprocess, "run", raise_file_not_found) - missing_path = str(tmp_path / "missing-cc") - with pytest.raises(PreprocessingError, match="preprocessor not found") as exc_info: - preprocessing.preprocess_source( - c_source, - language="c", - config=PreprocessingConfig(mode="compiler", compiler=missing_path), - ) - assert str(exc_info.value) == f"preprocessor not found: {missing_path}" - assert exc_info.value.category == "PREPROCESSOR_NOT_FOUND" - assert [diagnostic.to_dict() for diagnostic in exc_info.value.diagnostics] == [ - { - "category": "PREPROCESSOR_NOT_FOUND", - "message": f"preprocessor not found: {missing_path}", - "severity": "error", - "path": None, - "line": None, - "command": [missing_path, "-E", "-x", "c", str(c_source)], - } - ] - - def raise_timeout(*_args, **_kwargs): - raise subprocess.TimeoutExpired(cmd="cc", timeout=60) - - monkeypatch.setattr(preprocessing.subprocess, "run", raise_timeout) - slow_path = str(tmp_path / "slow-cc") - with pytest.raises(PreprocessingError, match="timed out") as exc_info: - preprocessing.preprocess_source( - c_source, - language="c", - config=PreprocessingConfig(mode="compiler", compiler=slow_path), - ) - assert str(exc_info.value) == "compiler preprocessing failed: timed out after 60 seconds" - assert exc_info.value.category == "PREPROCESSOR_FAILED" - assert [diagnostic.to_dict() for diagnostic in exc_info.value.diagnostics] == [ - { - "category": "PREPROCESSOR_FAILED", - "message": "compiler preprocessing timed out after 60 seconds", - "severity": "error", - "path": None, - "line": None, - "command": [slow_path, "-E", "-x", "c", str(c_source)], - } - ] - - monkeypatch.setattr( - preprocessing.subprocess, - "run", - lambda *_args, **_kwargs: type("Done", (), {"returncode": 2, "stdout": "", "stderr": ""})(), - ) - bad_path = str(tmp_path / "bad-cc") - with pytest.raises(PreprocessingError, match="exit code 2") as exc_info: - preprocessing.preprocess_source( - c_source, - language="c", - config=PreprocessingConfig(mode="compiler", compiler=bad_path), - ) - assert exc_info.value.category == "PREPROCESSOR_FAILED" - assert [diagnostic.to_dict() for diagnostic in exc_info.value.diagnostics] == [ - { - "category": "PREPROCESSOR_FAILED", - "message": "compiler preprocessing failed with exit code 2", - "severity": "error", - "path": None, - "line": None, - "command": [bad_path, "-E", "-x", "c", str(c_source)], - } - ] - - monkeypatch.setattr( - preprocessing.subprocess, - "run", - lambda *_args, **_kwargs: type("Done", (), {"returncode": 0, "stdout": "", "stderr": ""})(), - ) - result = preprocessing.preprocess_source( - c_source, - language="c", - config=PreprocessingConfig( - mode="compiler", - adapter="command-template", - command_template=f"{sys.executable} {{source}}", - ), - ) - assert [diagnostic.to_dict() for diagnostic in result.diagnostics] == [ - { - "category": "PROVENANCE_UNAVAILABLE", - "message": "selected compiler adapter did not provide source linemarkers", - "severity": "warning", - "path": None, - "line": None, - "command": [sys.executable, str(c_source)], - } - ] - - monkeypatch.setattr( - preprocessing.subprocess, - "run", - lambda *_args, **_kwargs: type("Done", (), {"returncode": 0, "stdout": "", "stderr": ""})(), - ) - result = preprocessing.preprocess_source( - c_source, - language="c", - config=PreprocessingConfig(mode="compiler", compiler=str(tmp_path / "cc")), - ) - assert result.diagnostics == [] diff --git a/tests/c/infrastructure/preprocessing/test_source_mappings.py b/tests/c/infrastructure/preprocessing/test_source_mappings.py deleted file mode 100644 index ec433ddfe..000000000 --- a/tests/c/infrastructure/preprocessing/test_source_mappings.py +++ /dev/null @@ -1,35 +0,0 @@ -"""Tests split by stable ownership concept from `test_cli.py`.""" - -from pathlib import Path - -import prik.preprocessing.source as preprocessing -from prik.preprocessing import PreprocessingConfig - - -def test_preprocess_source_preserves_plain_c_source_mapping(monkeypatch, tmp_path: Path): - c_source = tmp_path / "api.c" - c_source.write_text("int api(void);\n", encoding="utf-8") - - monkeypatch.setattr( - preprocessing.subprocess, - "run", - lambda *_args, **_kwargs: type( - "Done", - (), - {"returncode": 0, "stdout": "int api(void);\n", "stderr": ""}, - )(), - ) - - c_result = preprocessing.preprocess_source( - c_source, - language="c", - config=PreprocessingConfig(mode="compiler", compiler=str(tmp_path / "cc")), - ) - assert c_result.source_mappings == [ - preprocessing.SourceMapping( - generated_line=1, - original_path=str(c_source), - original_line=1, - include_stack=[str(c_source)], - ) - ] diff --git a/tests/c/infrastructure/semantic_ir/semantics/test_projects_and_diagnostics.py b/tests/c/infrastructure/semantic_ir/semantics/test_projects_and_diagnostics.py index 6f3e0bb50..d76c45e55 100644 --- a/tests/c/infrastructure/semantic_ir/semantics/test_projects_and_diagnostics.py +++ b/tests/c/infrastructure/semantic_ir/semantics/test_projects_and_diagnostics.py @@ -1,31 +1,14 @@ -"""Tests split by stable ownership concept from `test_functions_and_callbacks.py`.""" - -import pytest +"""C project semantic IR: cross-header type ownership and project-wide type resolution.""" from prik.pipeline.pyi import emit_module_stubs -from prik.parsers.c import parse_c_file, parse_c_project +from prik.parsers.c import parse_c_project from prik.parsers.c.models import ( - CEnum, - CFile, - CFunction, - CInt, - CParameter, - CProject, CSourceLocation, CStruct, - CTypedef, - CUnion, - CVariable, ) from prik.semantics.c2ir import ( - CToIRConverter, - c_file_to_semantic_module, - c_parameter_to_semantic_argument, c_project_to_semantic_modules, - c_struct_to_semantic_class, - c_type_to_semantic_type, ) -from prik.semantics.models import SemanticArgument, SemanticModule, SemanticOrigin, SemanticType from tests.c._support.semantic_conversion import ( _function, ) @@ -64,130 +47,25 @@ def test_c2ir_explicit_project_headers_import_types_from_their_owner_module(): assert "class state" not in stubs["api"] -def test_c2ir_classifies_external_types_after_owner_modules_without_rewriting_local_references(): - converter = CToIRConverter() - local_reference = SemanticArgument(name="local", semantic_type=SemanticType(name="state")) - external_reference = SemanticArgument(name="external", semantic_type=SemanticType(name="state")) - owner_module = SemanticModule( - name="types", - variables=[local_reference], - origin=SemanticOrigin(native_name="types.h"), - ) - consumer_module = SemanticModule( - name="api", - variables=[external_reference], - origin=SemanticOrigin(native_name="api.h"), - ) - project = CProject(structs={"state": CStruct(name="state", source_location=CSourceLocation(filename="types.h"))}) - - converter._classify_project_external_types([owner_module, consumer_module], project) - - assert "external_type_ref" not in local_reference.semantic_type.metadata - assert external_reference.semantic_type.metadata["external_type_ref"] == { - "name": "state", - "local_name": "state", - "origin_module": "types", - "wrapped": True, - "representation": "wrapped", - } - - -def test_c2ir_visitor_and_project_compatibility_entrypoints_cover_supported_nodes(): - first = parse_c_file("struct point { int x; };\nint value;\nint f(int x);\n", filename="a.h") - second = parse_c_file("double g(double y);\n", filename="b.h") - project = CProject( - files={"b.h": second, "a.h": first}, - functions={"f": first.functions[0], "g": second.functions[0]}, - structs={"point": first.structs[0]}, - variables={"value": first.variables[0]}, - ) - converter = CToIRConverter() - - assert [module.name for module in converter.visit(project)] == ["a", "b"] - assert converter.visit(first).name == "a" - assert converter.visit(first.functions[0]).name == "f" - assert converter.visit(first.functions[0].parameters[0], position=3).metadata["native_position"] == 3 - assert converter.visit(CParameter(name=None, type=CInt())).metadata["native_position"] == 0 - assert converter.visit(first.structs[0]).name == "point" - assert converter.visit(CUnion(name="loose_union")).name == "loose_union" - assert converter.visit(first.variables[0]).name == "value" - assert converter.visit(CInt()).name == "Int" - enum_type = converter.visit(CEnum(name="status")) - assert enum_type.name == "Int" - assert enum_type.dtype == "Int32" - assert enum_type.metadata["c_kind"] == "enum" - assert enum_type.metadata["c_enum"] == "enum status" - assert enum_type.metadata["c_enum_name"] == "status" - assert enum_type.metadata["c_underlying_type"] == "Int" - assert enum_type.origin.native_name == "enum status" - assert enum_type.origin.metadata["c_type"] == "CEnum" - with pytest.raises(TypeError) as error: - converter.visit(object()) - assert str(error.value) == "Unsupported C parse object: " - - contextual_union = CUnion(name="context_union", members=[CVariable(name="value", type=CInt())]) - contextual_file = CFile( - filename="contextual.h", - functions=[ - CFunction(name="contextual", result_type=CTypedef(name="contextual_t")), - CFunction( - name="use_context_union", - parameters=[CParameter(name="value", type=CUnion(name="context_union", is_incomplete=True))], - ), - ], - unions=[contextual_union], - ) - contextual_module = converter.visit( - contextual_file, - typedefs={"contextual_t": CTypedef(name="contextual_t", type=CInt())}, - unions={"context_union": contextual_union}, - ) - assert _function(contextual_module, "contextual").return_type.name == "Int" - assert _function(contextual_module, "use_context_union").arguments[0].semantic_type.metadata["incomplete"] is False - assert [cls.name for cls in contextual_module.classes] == ["context_union"] - - assert c_file_to_semantic_module(first).name == "a" - assert c_type_to_semantic_type(CInt()).name == "Int" - assert c_parameter_to_semantic_argument(CParameter(name=None, type=CInt()), position=2).name == "arg2" - default_argument = c_parameter_to_semantic_argument(CParameter(name=None, type=CInt())) - assert default_argument.name == "arg0" - assert default_argument.metadata == {"native_position": 0} - assert c_struct_to_semantic_class(first.structs[0]).name == "point" - assert [module.name for module in c_project_to_semantic_modules(project)] == ["a", "b"] - typedef_project = parse_c_project( +def test_c2ir_resolves_typedefs_and_forward_tags_through_the_project_registry(): + """A header that only forward-declares a tag or names a typedef sees the owner's definition.""" + project = parse_c_project( { - "types.h": "typedef unsigned long count_t;\n", - "api.h": "count_t count(void);\n", + "types.h": ( + "typedef unsigned long count_t;\nstruct record { int value; };\nunion choice { int value; };\n" + ), + "api.h": ( + "struct record;\nunion choice;\ncount_t count(void);\n" + "void use(struct record *record, union choice *choice);\n" + ), } ) - typedef_modules = {module.name: module for module in converter.visit(typedef_project)} - assert _function(typedef_modules["api"], "count").return_type.name == "UInt64" - assert _function(typedef_modules["api"], "count").return_type.metadata == {"c_typedefs": ["count_t"]} - count_reference = CTypedef(name="global_count_t") - count_function = CFunction(name="global_count", result_type=count_reference) - reference_project = CProject( - files={"api.h": CFile(filename="api.h", functions=[count_function])}, - functions={"global_count": count_function}, - typedefs={"global_count_t": CTypedef(name="global_count_t", type=CInt())}, - ) - reference_modules = converter.visit(reference_project) - assert _function(reference_modules[0], "global_count").return_type.name == "Int" - record = CStruct(name="global_record", members=[CVariable(name="value", type=CInt())]) - choice = CUnion(name="global_choice", members=[CVariable(name="value", type=CInt())]) - registry_function = CFunction( - name="use_global_types", - parameters=[ - CParameter(name="record", type=CStruct(name="global_record", is_incomplete=True)), - CParameter(name="choice", type=CUnion(name="global_choice", is_incomplete=True)), - ], - ) - registry_project = CProject( - files={"api.h": CFile(filename="api.h", functions=[registry_function])}, - functions={"use_global_types": registry_function}, - structs={"global_record": record}, - unions={"global_choice": choice}, - ) - registry_modules = converter.visit(registry_project) - registry_args = _function(registry_modules[0], "use_global_types").arguments - assert registry_args[0].semantic_type.metadata == {"c_kind": "struct", "incomplete": False} - assert registry_args[1].semantic_type.metadata["incomplete"] is False + + api = {module.name: module for module in c_project_to_semantic_modules(project)}["api"] + count = _function(api, "count").return_type + record, choice = (argument.semantic_type for argument in _function(api, "use").arguments) + + assert count.name == "UInt64" + assert count.metadata == {"c_typedefs": ["count_t"]} + assert (record.metadata["c_kind"], record.metadata["incomplete"]) == ("struct", False) + assert (choice.metadata["c_kind"], choice.metadata["incomplete"]) == ("union", False) diff --git a/tests/c/infrastructure/semantic_pyi/pipeline/test_c_pyi_contract_fixtures.py b/tests/c/infrastructure/semantic_pyi/pipeline/test_c_pyi_contract_fixtures.py index 7e90584a3..2ce84ac1d 100644 --- a/tests/c/infrastructure/semantic_pyi/pipeline/test_c_pyi_contract_fixtures.py +++ b/tests/c/infrastructure/semantic_pyi/pipeline/test_c_pyi_contract_fixtures.py @@ -17,18 +17,6 @@ C_FIXTURE_PROJECTS = iter_general_c_fixture_projects() -def test_c_pyi_fixture_suite_has_fixtures(): - assert C_FIXTURE_PROJECTS, "No C fixtures found in tests/c/fixtures/native/general" - - -def test_c_pyi_fixtures_match_general_c_projects_one_to_one(): - expected = {project_key.with_suffix(".pyi") for project_key, _fixtures in C_FIXTURE_PROJECTS} - actual = {path.relative_to(C_PYI_FIXTURE_DIR) for path in C_PYI_FIXTURE_DIR.rglob("*.pyi") if path.is_file()} - - assert not sorted(expected - actual) - assert not sorted(actual - expected) - - def test_c_pyi_fixtures_do_not_contain_unknown_types(): unknown_fixtures = [ str(path.relative_to(C_PYI_FIXTURE_DIR)) diff --git a/tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py b/tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py deleted file mode 100644 index 9341ade1f..000000000 --- a/tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py +++ /dev/null @@ -1,33 +0,0 @@ -"""Direct C pointer lowering consumes completed runtime-rank policy.""" - -from prik.parsers.c import parse_c_file -from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import WrapperPlanner -from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ArrayEntrypointABI, ArrayPythonLayout, NativeArraySourceKind -from prik.semantics.c2ir import c_file_to_semantic_module - - -def test_direct_c_binding_keeps_pointer_abi_and_uses_completed_runtime_rank_bounds(): - module = c_file_to_semantic_module( - parse_c_file("double native_read(const double *input) { return *input; }", filename="read.c") - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - generated = WrapperGenerator().generate(plan) - binding = next(source.text for source in generated.sources if source.path.suffix == ".c") - function = plan.namespaces[0].functions[0] - array = function.arguments[0].array - actual = function.arguments[0].native_array_actual - - assert plan.bridge is None - assert plan.entrypoint.native_languages == ("c",) - assert array.rank is None - assert (array.minimum_rank, array.maximum_rank) == (0, 15) - assert array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS - assert actual.accepted_sources == (NativeArraySourceKind.NDARRAY,) - assert "double native_read(const double * input);" in binding - assert array.python_layout is ArrayPythonLayout.ANY_STRIDED - assert ("prik_array_validate(bound_input_obj, NPY_FLOAT64, 0, 15, PRIK_ARRAY_LAYOUT_ANY_STRIDED, 0, 0") in binding - assert "result = native_read(bound_input);" in binding - assert "bind_c_read_wrapper" not in binding diff --git a/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py b/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py index 939feff12..3671b52e5 100644 --- a/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py +++ b/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py @@ -1,4 +1,8 @@ -"""Compiled scalar-reference and NumPy-array contracts for one-level C pointers.""" +"""Compiled scalar-reference and NumPy-array contracts for one-level C pointers. + +The source-free contract mechanisms share one module-scoped build; the +generated-contract route keeps its own build because it starts from C source. +""" import shutil import subprocess @@ -13,7 +17,84 @@ from tests.c._support.runtime import sole_native_module -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") +pytestmark = pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") + +_CONTRACT = """from prik.contracts import Addr, Arg, CLongLong, Float64, Int32, Int64, Return, Returns, native_call + +@native_call([Addr(Arg(0))]) +def scale_scalar(value: Float64) -> Returns["value", Float64]: ... + +def scale_zero(value: Float64[()]) -> None: ... + +def scale_vector(values: Float64[n], n: Int32) -> None: ... + +def scale_matrix(values: Float64[2, 2]) -> None: ... + +@native_call([Arg(0).size, Arg(0)]) +def scale_any_rank(values: Float64[...]) -> None: ... + +@native_call([Arg(0).shape[0], Int64(Arg(0).strides[0]), Arg(0)]) +def scale_strided(values: Float64[...]) -> None: ... + +@native_call([Arg(0).shape[0], Arg(0)]) +def scale_promoted(values: Float64[:]) -> None: ... + +@native_call([Addr(CLongLong(Arg(0)))]) +def increment_scalar(value: Int64) -> Returns["value", Int64]: ... + +@native_call([CLongLong(Arg(0)), Arg(1)]) +def increment(values: Int64[:], count: Int32) -> None: ... + +@native_call([CLongLong(Arg(0))]) +def increment_zero(value: Int64[()]) -> None: ... + +@native_call([CLongLong(Return("out", 0))]) +def read_exact() -> Int64: ... +""" + +_SOURCE = """#include +void scale_scalar(double *value) { *value *= 2.0; } +void scale_zero(double *value) { *value += 1.0; } +void scale_vector(double *values, int n) { for (int i = 0; i < n; ++i) values[i] *= 3.0; } +void scale_matrix(double *values) { for (int i = 0; i < 4; ++i) values[i] += 1.0; } +void scale_any_rank(size_t count, double *values) { + for (size_t index = 0; index < count; ++index) values[index] *= 2.0; +} +void scale_strided(size_t count, long long stride_bytes, double *values) { + char *base = (char *)values; + for (size_t index = 0; index < count; ++index) { + *(double *)(base + (ptrdiff_t)index * (ptrdiff_t)stride_bytes) *= 2.0; + } +} +void scale_promoted(size_t n, double *values) { for (size_t i = 0; i < n; ++i) values[i] *= 2.0; } +void increment_scalar(long long *value) { *value += 1; } +void increment(long long *values, int count) { + for (int i = 0; i < count; ++i) values[i] += 1; +} +void increment_zero(long long *value) { *value += 1; } +void read_exact(long long *out) { *out = 9000000000LL; } +""" + + +@pytest.fixture(scope="module") +def pointer_build(tmp_path_factory): + tmp_path = tmp_path_factory.mktemp("c_pointer_contracts") + contract = tmp_path / "pointers.pyi" + contract.write_text(_CONTRACT, encoding="utf-8") + source = tmp_path / "pointers.c" + source.write_text(_SOURCE, encoding="utf-8") + with warnings.catch_warnings(): + warnings.simplefilter("error") + result = build_pyi_extension( + contract, + native_language="c", + native_c_sources=[source], + output_dir=tmp_path / "build", + ) + binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + return result, binding, sole_native_module(result.import_module()) + + def test_generated_c_int_array_uses_its_probed_primitive_storage(tmp_path: Path): """The public ``Int`` spelling retains its probed dtype for array policy.""" source = tmp_path / "integer_array.c" @@ -56,40 +137,8 @@ def test_generated_c_int_array_uses_its_probed_primitive_storage(tmp_path: Path) np.testing.assert_array_equal(values, np.array([1, 2, 3, 4], dtype=np.intc)) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_pointer_supports_explicit_scalar_reference_and_exact_array_contracts(tmp_path: Path): - contract = tmp_path / "pointers.pyi" - contract.write_text( - """from prik.contracts import Addr, Arg, Float64, Int32, Returns, native_call - -@native_call([Addr(Arg(0))]) -def scale_scalar(value: Float64) -> Returns["value", Float64]: ... - -def scale_zero(value: Float64[()]) -> None: ... - -def scale_vector(values: Float64[n], n: Int32) -> None: ... - -def scale_matrix(values: Float64[2, 2]) -> None: ... -""", - encoding="utf-8", - ) - source = tmp_path / "pointers.c" - source.write_text( - """void scale_scalar(double *value) { *value *= 2.0; } -void scale_zero(double *value) { *value += 1.0; } -void scale_vector(double *values, int n) { for (int i = 0; i < n; ++i) values[i] *= 3.0; } -void scale_matrix(double *values) { for (int i = 0; i < 4; ++i) values[i] += 1.0; } -""", - encoding="utf-8", - ) - - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build", - ) - module = sole_native_module(result.import_module()) +def test_c_pointer_supports_explicit_scalar_reference_and_exact_array_contracts(pointer_build): + _result, _binding, module = pointer_build assert module.scale_scalar(np.float64(2.5)) == np.float64(5.0) zero = np.array(4.0, dtype=np.float64) @@ -107,38 +156,10 @@ def scale_matrix(values: Float64[2, 2]) -> None: ... module.scale_matrix(np.asfortranarray(matrix)) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_runtime_rank_c_pointer_uses_total_size_for_rank_zero_and_ranked_storage(tmp_path: Path): - contract = tmp_path / "runtime_rank.pyi" - contract.write_text( - """from prik.contracts import Arg, Float64, native_call +def test_runtime_rank_c_pointer_uses_total_size_for_rank_zero_and_ranked_storage(pointer_build): + _result, binding, module = pointer_build -@native_call([Arg(0).size, Arg(0)]) -def scale(values: Float64[...]) -> None: ... -""", - encoding="utf-8", - ) - source = tmp_path / "runtime_rank.c" - source.write_text( - """#include -void scale(size_t count, double *values) { - for (size_t index = 0; index < count; ++index) values[index] *= 2.0; -} -""", - encoding="utf-8", - ) - - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build_runtime_rank", - output_name="runtime_rank", - ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") - - assert "void scale(size_t size_0, double * values);" in binding + assert "void scale_any_rank(size_t size_0, double * values);" in binding assert "(size_t)PyArray_SIZE((PyArrayObject *)bound_values_obj)" in binding zero = np.array(3.0, dtype=np.float64) @@ -147,149 +168,60 @@ def scale(values: Float64[...]) -> None: ... empty = np.empty((2, 0), dtype=np.float64) for values in (zero, vector, matrix, empty): expected = values.copy() * 2.0 - assert module.scale(values) is None + assert module.scale_any_rank(values) is None np.testing.assert_allclose(values, expected) # Runtime-rank storage constrains neither rank nor strides, so a # Fortran-ordered actual reaches the same contiguous buffer. fortran = np.asfortranarray(matrix) expected = fortran.copy() * 2.0 - assert module.scale(fortran) is None + assert module.scale_any_rank(fortran) is None np.testing.assert_allclose(fortran, expected) with pytest.raises(TypeError, match=r"numpy\.ndarray"): - module.scale(np.float64(3.0)) + module.scale_any_rank(np.float64(3.0)) with pytest.raises(TypeError, match=r"compatible numpy\.ndarray"): - module.scale(np.ones((1,) * 16, dtype=np.float64)) + module.scale_any_rank(np.ones((1,) * 16, dtype=np.float64)) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_runtime_rank_c_pointer_passes_a_strided_view_with_its_projected_layout(tmp_path: Path): +def test_runtime_rank_c_pointer_passes_a_strided_view_with_its_projected_layout(pointer_build): """``T[...]`` states no layout, so projected extents and strides carry it.""" - contract = tmp_path / "strided_rank.pyi" - contract.write_text( - """from prik.contracts import Arg, Float64, Int64, native_call - -@native_call([Arg(0).shape[0], Int64(Arg(0).strides[0]), Arg(0)]) -def scale(values: Float64[...]) -> None: ... -""", - encoding="utf-8", - ) - source = tmp_path / "strided_rank.c" - source.write_text( - """#include -void scale(size_t count, long long stride_bytes, double *values) { - char *base = (char *)values; - for (size_t index = 0; index < count; ++index) { - *(double *)(base + (ptrdiff_t)index * (ptrdiff_t)stride_bytes) *= 2.0; - } -} -""", - encoding="utf-8", - ) - - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build_strided_rank", - output_name="strided_rank", - ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + _result, binding, module = pointer_build assert "PRIK_ARRAY_LAYOUT_ANY_STRIDED" in binding assert "(int64_t)PyArray_STRIDE((PyArrayObject *)bound_values_obj, 0)" in binding base = np.arange(6, dtype=np.float64) - assert module.scale(base[::2]) is None + assert module.scale_strided(base[::2]) is None np.testing.assert_allclose(base, np.array([0.0, 1.0, 4.0, 3.0, 8.0, 5.0])) # A projected axis cannot exist on rank-zero storage, so the caller is told # instead of the binding reading past the actual's shape. with pytest.raises(TypeError, match="has no axis 0"): - module.scale(np.array(1.0, dtype=np.float64)) + module.scale_strided(np.array(1.0, dtype=np.float64)) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_edited_c_array_contract_can_derive_the_native_extent_from_its_shape(tmp_path: Path): +def test_edited_c_array_contract_can_derive_the_native_extent_from_its_shape(pointer_build): """The documented promotion hides the count behind ``Arg(0).shape[0]``. The derived extent is a binding-owned producer, so it keeps its own ``size_t`` identity while the promoted buffer crosses by address. """ - contract = tmp_path / "promotion.pyi" - contract.write_text( - """from prik.contracts import Arg, Float64, native_call - -@native_call([Arg(0).shape[0], Arg(0)]) -def scale(values: Float64[:]) -> None: ... -""", - encoding="utf-8", - ) - source = tmp_path / "promotion.c" - source.write_text( - """#include -void scale(size_t n, double *values) { for (size_t i = 0; i < n; ++i) values[i] *= 2.0; } -""", - encoding="utf-8", - ) + result, binding, module = pointer_build - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build", - ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") - - assert "void scale(size_t shape_0, double * values);" in binding + assert "void scale_promoted(size_t shape_0, double * values);" in binding assert all(path.suffix != ".f90" for path in result.generated_sources) values = np.array([1.0, 2.0, 3.0], dtype=np.float64) - assert module.scale(values) is None + assert module.scale_promoted(values) is None np.testing.assert_allclose(values, np.array([2.0, 4.0, 6.0])) - assert module.scale(np.empty(0, dtype=np.float64)) is None - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_exact_long_long_scalar_address_converts_while_arrays_require_native_storage(tmp_path: Path): - contract = tmp_path / "exact_long_long.pyi" - contract.write_text( - """from prik.contracts import Addr, Arg, CLongLong, Int32, Int64, Returns, native_call + assert module.scale_promoted(np.empty(0, dtype=np.float64)) is None -@native_call([Addr(CLongLong(Arg(0)))]) -def increment_scalar(value: Int64) -> Returns["value", Int64]: ... -@native_call([CLongLong(Arg(0)), Arg(1)]) -def increment(values: Int64[:], count: Int32) -> None: ... - -@native_call([CLongLong(Arg(0))]) -def increment_zero(value: Int64[()]) -> None: ... -""", - encoding="utf-8", - ) - source = tmp_path / "exact_long_long.c" - source.write_text( - """void increment_scalar(long long *value) { *value += 1; } -void increment(long long *values, int count) { - for (int i = 0; i < count; ++i) values[i] += 1; -} -void increment_zero(long long *value) { *value += 1; } -""", - encoding="utf-8", - ) - - with warnings.catch_warnings(): - warnings.simplefilter("error") - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build", - ) - module = sole_native_module(result.import_module()) +def test_exact_long_long_scalar_address_converts_while_arrays_require_native_storage(pointer_build): + _result, binding, module = pointer_build + # A converted scalar is materialized as native storage before its address is taken. + assert "void increment_scalar(long long * value);" in binding scalar = module.increment_scalar(np.int64(4)) assert scalar == np.int64(5) assert scalar.dtype == np.dtype(np.int64) @@ -307,6 +239,12 @@ def increment_zero(value: Int64[()]) -> None: ... assert module.increment_zero(zero) is None assert zero[()] == np.longlong(5) + # An exact output parameter is native storage converted to the contract result. + assert "void read_exact(long long * out);" in binding + output = module.read_exact() + assert output == np.int64(9000000000) + assert output.dtype == np.dtype(np.int64) + if np.dtype(np.int64).num != np.dtype(np.longlong).num: with pytest.raises(TypeError, match=r"numpy\.longlong"): module.increment(np.array([1, 2, 3], dtype=np.int64), np.int32(3)) diff --git a/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py b/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py index 47bc1bb26..aadd1bbff 100644 --- a/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py +++ b/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py @@ -2,45 +2,10 @@ from prik.pipeline.pyi import pyi_text_to_semantic_module from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - ArrayEntrypointABI, - ArrayPythonLayout, - EntrypointPassingConvention, - EntrypointProjectionAction, -) +from prik.policy.models import ArrayPythonLayout from prik.semantics.native_contract import validate_pyi_native_contract -def test_c_runtime_rank_and_total_size_are_complete_before_planning(): - module = pyi_text_to_semantic_module( - """from prik.contracts import Arg, Float64, native_call -@native_call([Arg(0).size, Arg(0)]) -def scale(values: Float64[...]) -> None: ... -""", - module_name="runtime_rank", - native_language="c", - ) - validate_pyi_native_contract([module]) - complete_semantic_policies(module) - - policy = module.functions[0].metadata["resolved_function_wrapper_policy"] - array = policy.arguments[0].array - size_slot = policy.native_call_slots[0] - - assert array.rank is None - assert (array.minimum_rank, array.maximum_rank) == (0, 15) - assert array.order == "ORDER_C" - assert array.native_order == "ORDER_C" - assert array.contiguous is None - assert array.python_layout is ArrayPythonLayout.ANY_STRIDED - assert array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS - assert policy.native_call_slots[1].array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS - assert policy.arguments[0].entrypoint_passing is EntrypointPassingConvention.POINTER_REFERENCE - assert size_slot.semantic_type_name == "SizeT" - assert size_slot.projection_action is EntrypointProjectionAction.COMPUTED_SIZE - assert size_slot.entrypoint_passing is EntrypointPassingConvention.C_VALUE - - def test_contiguous_narrows_runtime_rank_storage_to_the_c_order_layout(): """``T[...]`` states no layout, so ``Contiguous`` is what asserts one.""" module = pyi_text_to_semantic_module( diff --git a/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py b/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py index 704c77531..7e5fb284f 100644 --- a/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py +++ b/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py @@ -1,4 +1,9 @@ -"""Binding lowering consumes exact scalar types completed before planning.""" +"""Binding lowering consumes exact array element types completed before planning. + +The rows are identities whose NumPy storage is not implied by width alone: +plain ``char``, ``int`` versus ``long`` versus ``long long``, and the +extended-precision real and complex types. +""" import pytest @@ -9,14 +14,6 @@ from prik.semantics.native_contract import validate_pyi_native_contract -def _binding(text: str) -> str: - module = pyi_text_to_semantic_module(text, module_name="exact", native_language="c") - validate_pyi_native_contract([module]) - complete_semantic_policies(module) - generated = WrapperGenerator().generate(WrapperPlanner().build(module)) - return next(source.text for source in generated.sources if source.path.suffix == ".c") - - def _plan_and_binding(text: str): module = pyi_text_to_semantic_module(text, module_name="exact", native_language="c") validate_pyi_native_contract([module]) @@ -27,61 +24,12 @@ def _plan_and_binding(text: str): return plan, binding -def test_exact_value_argument_and_result_use_native_prototype_and_directional_casts(): - binding = _binding( - """from prik.contracts import Arg, CLongLong, Int64, Return, native_call -@native_call([CLongLong(Arg(0))], result=CLongLong(Return(0))) -def convert(value: Int64) -> Int64: ... -""" - ) - - assert "long long convert(long long value);" in binding - assert "result = (int64_t)convert((long long)bound_value);" in binding - - -def test_exact_address_argument_materializes_native_storage_before_taking_its_address(): - binding = _binding( - """from prik.contracts import Addr, Arg, CLongLong, Int64, Returns, native_call -@native_call([Addr(CLongLong(Arg(0)))]) -def update(value: Int64) -> Returns["value", Int64]: ... -""" - ) - - assert "void update(long long * value);" in binding - assert "long long bound_value;" in binding - assert "bound_value = (long long)bound_value_converted;" in binding - assert "bound_value_storage = &bound_value;" in binding - assert "update(bound_value_storage);" in binding - assert "int64_t bound_value_contract = (int64_t)*bound_value_storage;" in binding - assert "prik_int64_to_numpy(&bound_value_contract)" in binding - - -def test_exact_output_parameter_uses_native_storage_then_converts_the_python_result(): - binding = _binding( - """from prik.contracts import CLongLong, Int64, Return, native_call -@native_call([CLongLong(Return("out", 0))]) -def read() -> Int64: ... -""" - ) - - assert "void read(long long * out);" in binding - assert "long long out;" in binding - assert "read(&out);" in binding - assert "int64_t out_contract = (int64_t)out;" in binding - - @pytest.mark.parametrize( ("native_type", "annotation", "c_type", "numpy_macro", "numpy_name"), [ ("CChar", "Int8", "char", "NPY_BYTE", "numpy.byte"), - ("CSignedChar", "Int8", "signed char", "NPY_BYTE", "numpy.byte"), - ("CUnsignedChar", "UInt8", "unsigned char", "NPY_UBYTE", "numpy.ubyte"), - ("CShort", "Int16", "short", "NPY_SHORT", "numpy.short"), - ("CUnsignedShort", "UInt16", "unsigned short", "NPY_USHORT", "numpy.ushort"), ("CInt", "Int32", "int", "NPY_INT", "numpy.intc"), - ("CUnsignedInt", "UInt32", "unsigned int", "NPY_UINT", "numpy.uintc"), ("CLong", "Int64", "long", "NPY_LONG", "numpy.long"), - ("CUnsignedLong", "UInt64", "unsigned long", "NPY_ULONG", "numpy.ulong"), ("CLongLong", "Int64", "long long", "NPY_LONGLONG", "numpy.longlong"), ( "CUnsignedLongLong", @@ -90,11 +38,7 @@ def read() -> Int64: ... "NPY_ULONGLONG", "numpy.ulonglong", ), - ("CFloat", "Float32", "float", "NPY_FLOAT", "numpy.single"), - ("CDouble", "Float64", "double", "NPY_DOUBLE", "numpy.double"), ("CLongDouble", "Float128", "long double", "NPY_LONGDOUBLE", "numpy.longdouble"), - ("CFloatComplex", "Complex64", "float _Complex", "NPY_CFLOAT", "numpy.csingle"), - ("CDoubleComplex", "Complex128", "double _Complex", "NPY_CDOUBLE", "numpy.cdouble"), ( "CLongDoubleComplex", "Complex256", diff --git a/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py b/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py index 0e8891d8d..067807fd6 100644 --- a/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py +++ b/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py @@ -1,7 +1,11 @@ -"""Compiled direct-C primitive scalar evidence.""" +"""Compiled direct-C primitive scalar evidence. + +One C source build and one source-free contract build carry every scalar +mechanism: source declarations, preprocessing, typedef resolution, the default +``T *`` storage, and contract-side binding, projection, and overloads. +""" import shutil -from pathlib import Path import numpy as np import pytest @@ -10,183 +14,116 @@ from tests.c._support.runtime import sole_native_module -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_source_build_calls_renamed_user_symbol_without_a_fortran_adapter(tmp_path: Path): - source = tmp_path / "scalar_api.c" - source.write_text( - """double native_add(double left, double right) { return left + right; } -double native_scale(double *value) { *value *= 2.0; return *value; } -""", - encoding="utf-8", - ) +pytestmark = pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") - result = build_c_extension(source, output_dir=tmp_path / "build", output_name="c_scalar_api") - module = sole_native_module(result.import_module()) +_C_SOURCE = """#include +#define PRIK_TEST_GAIN 3.0 - assert module.native_add(np.float64(1.5), np.float64(2.0)) == np.float64(3.5) - assert module.native_scale(np.array(3.0, dtype=np.float64)) == np.float64(6.0) - assert all(path.suffix != ".f90" for path in result.generated_sources) - binding = next(path for path in result.generated_sources if path.suffix == ".c") - text = binding.read_text(encoding="utf-8") - assert "double native_add(double left, double right);" in text - assert "native_add(" in text +typedef long my_int; +double native_add(double left, double right) { return left + right; } +double native_scale(double *value) { *value *= 2.0; return *value; } +double scaled(double value) { return value * PRIK_TEST_GAIN; } +size_t total(size_t value) { return value + 1; } +void twice(double *value) { *value *= 2.0; } +my_int alias_step(my_int value) { return value + 1; } +ptrdiff_t alias_offset(const ptrdiff_t *value) { return *value + 1; } +""" -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_native_language_is_explicit_for_a_source_free_pyi_contract(tmp_path: Path): - contract = tmp_path / "direct_c_contract.pyi" - contract.write_text( - """from prik.contracts import Float64, Int, bind +_CONTRACT = """from prik.contracts import ( + Addr, Arg, Float64, Int, Int32, Return, SizeT, Value, bind, native_call, overload, private, +) @bind("native_add") def add(left: Float64, right: Float64) -> Float64: ... @bind("native_increment") def increment(value: Int) -> Int: ... -""", - encoding="utf-8", - ) - source = tmp_path / "implementation.c" - source.write_text( - """double native_add(double left, double right) { return left + right; } -int native_increment(int value) { return value + 1; } -""", - encoding="utf-8", - ) - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build", - ) - module = sole_native_module(result.import_module()) - - assert module.add(np.float64(4.0), np.float64(2.5)) == np.float64(6.5) - assert module.increment(np.int32(4)) == np.int32(5) - assert result.native_build_plan.compilation_units[0].language == "c" - assert result.manifest["extension"]["native_language"] == "c" - assert result.manifest["compiler"]["c_flags"] == [] +def bump(value: Int32) -> Int32: ... +@bind("projected_native") +@native_call([Value(Arg(1)), Addr(Arg(0)), Int32(5)]) +def projected(left: Int32, right: Int32) -> Int32: ... -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_contract_defaults_matching_python_name_to_native_symbol(tmp_path: Path): - """A C contract needs ``@bind`` only when the names differ.""" - contract = tmp_path / "matching_name.pyi" - contract.write_text( - """from prik.contracts import Int32 +@bind("projected_output_native") +@native_call([Value(Arg(1)), Addr(Arg(0)), Int32(5), Return("output", 0)]) +def projected_output(left: Int32, right: Int32) -> Int32: ... -def increment(value: Int32) -> Int32: ... -""", - encoding="utf-8", - ) - source = tmp_path / "matching_name.c" - source.write_text("int increment(int value) { return value + 1; }\n", encoding="utf-8") +@native_call([Arg(1), Arg(0)]) +def combine(scale: Float64, count: Int32) -> Float64: ... - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build", - ) - module = sole_native_module(result.import_module()) +def total(value: SizeT) -> SizeT: ... - assert module.increment(np.int32(4)) == np.int32(5) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") - assert "int32_t increment(int32_t value);" in binding - assert "result = increment(bound_value);" in binding +@private +def scale_integer(value: Int32) -> Int32: ... +@private +def scale_real(value: Float64) -> Float64: ... -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_contract_reuses_direct_projection_value_address_literal_and_hidden_output_paths(tmp_path: Path): - contract = tmp_path / "projection.pyi" - contract.write_text( - """from prik.contracts import Addr, Arg, Int32, Return, Value, bind, native_call +@overload("scale_integer") +def scale(value: Int32) -> Int32: ... -@bind("projected_native") -@native_call([Value(Arg(1)), Addr(Arg(0)), Int32(5)]) -def projected(left: Int32, right: Int32) -> Int32: ... +@overload("scale_real") +def scale(value: Float64) -> Float64: ... +""" -@bind("projected_output_native") -@native_call([Value(Arg(1)), Addr(Arg(0)), Int32(5), Return("output", 0)]) -def projected_output(left: Int32, right: Int32) -> Int32: ... -""", - encoding="utf-8", - ) - source = tmp_path / "projection.c" - source.write_text( - """int projected_native(int right, int *left, int bias) { return 100 * right + 10 * *left + bias; } +_CONTRACT_SOURCE = """#include +double native_add(double left, double right) { return left + right; } +int native_increment(int value) { return value + 1; } +int bump(int value) { return value + 1; } +int projected_native(int right, int *left, int bias) { return 100 * right + 10 * *left + bias; } void projected_output_native(int right, int *left, int bias, int *output) { *output = 100 * right + 10 * *left + bias; } -""", - encoding="utf-8", - ) +double combine(int count, double scale) { return count * scale; } +size_t total(size_t value) { return value + 1; } +int scale_integer(int value) { return value * 2; } +double scale_real(double value) { return value * 2.0; } +""" - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build", - ) - module = sole_native_module(result.import_module()) - assert module.projected(np.int32(2), np.int32(3)) == np.int32(325) - assert module.projected_output(np.int32(2), np.int32(3)) == np.int32(325) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") - assert "int32_t projected_native(int32_t right, int32_t * left, int32_t literal_2);" in binding - assert ( - "void projected_output_native(int32_t right, int32_t * left, int32_t literal_2, int32_t * output);" in binding - ) +def _binding(result) -> str: + return next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_reordered_c_projection_keeps_each_argument_its_own_declared_type(tmp_path: Path): - """A route-neutral reorder must not resolve one argument against another.""" - contract = tmp_path / "reordered.pyi" - contract.write_text( - """from prik.contracts import Arg, Float64, Int32, native_call +@pytest.fixture(scope="module") +def source_build(tmp_path_factory): + tmp_path = tmp_path_factory.mktemp("c_scalar_source") + source = tmp_path / "scalar_api.c" + source.write_text(_C_SOURCE, encoding="utf-8") + result = build_c_extension(source, output_dir=tmp_path / "build", output_name="c_scalar_api") + return result, sole_native_module(result.import_module()) -@native_call([Arg(1), Arg(0)]) -def combine(scale: Float64, count: Int32) -> Float64: ... -""", - encoding="utf-8", - ) - source = tmp_path / "reordered.c" - source.write_text("double combine(int count, double scale) { return count * scale; }\n", encoding="utf-8") +@pytest.fixture(scope="module") +def contract_build(tmp_path_factory): + tmp_path = tmp_path_factory.mktemp("c_scalar_contract") + contract = tmp_path / "direct_c_contract.pyi" + contract.write_text(_CONTRACT, encoding="utf-8") + source = tmp_path / "implementation.c" + source.write_text(_CONTRACT_SOURCE, encoding="utf-8") result = build_pyi_extension( contract, native_language="c", native_c_sources=[source], output_dir=tmp_path / "build", ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + return result, sole_native_module(result.import_module()) - assert module.combine(np.float64(2.5), np.int32(4)) == np.float64(10.0) - assert "double combine(int32_t count, double scale);" in binding - with pytest.raises(TypeError, match=r"numpy\.float64 or rank-zero array for argument scale"): - module.combine(np.int32(4), np.int32(4)) +def test_c_source_build_calls_the_user_symbol_without_a_fortran_adapter(source_build): + result, module = source_build -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_source_directives_are_expanded_before_the_wrapper_reads_declarations(tmp_path: Path): - """A C wrapper build preprocesses its sources like the inspection routes.""" - source = tmp_path / "directives.c" - source.write_text( - """#include -#define PRIK_TEST_GAIN 3.0 + assert module.native_add(np.float64(1.5), np.float64(2.0)) == np.float64(3.5) + assert module.native_scale(np.array(3.0, dtype=np.float64)) == np.float64(6.0) + assert all(path.suffix != ".f90" for path in result.generated_sources) + assert "double native_add(double left, double right);" in _binding(result) -double scaled(double value) { return value * PRIK_TEST_GAIN; } -size_t total(size_t value) { return value + 1; } -""", - encoding="utf-8", - ) - result = build_c_extension(source, output_dir=tmp_path / "build", output_name="c_directives") - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") +def test_c_source_directives_are_expanded_before_the_wrapper_reads_declarations(source_build): + """A C wrapper build preprocesses its sources like the inspection routes.""" + result, module = source_build assert module.scaled(np.float64(2.0)) == np.float64(6.0) output = module.total(np.uintp(4)) @@ -195,17 +132,24 @@ def test_c_source_directives_are_expanded_before_the_wrapper_reads_declarations( assert module.total(output) == np.uintp(6) # A typedef-written parameter declares the exact underlying builtin, which # the binding can always spell; the typedef itself is source provenance. - assert "unsigned long total(unsigned long value);" in binding + assert "unsigned long total(unsigned long value);" in _binding(result) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_default_c_pointer_writes_back_into_caller_owned_runtime_rank_storage(tmp_path: Path): - """The ``T *`` default passes caller-owned storage of whatever rank arrived.""" - source = tmp_path / "in_place.c" - source.write_text("void twice(double *value) { *value *= 2.0; }\n", encoding="utf-8") +def test_c_typedef_declarations_resolve_to_their_exact_underlying_builtin(source_build): + """A user typedef only the source's headers define cannot enter the binding.""" + result, module = source_build + binding = _binding(result) + + assert "long alias_step(long value);" in binding + assert "long alias_offset(const long * value);" in binding + assert "my_int" not in binding + assert module.alias_step(np.int64(4)) == np.int64(5) + assert module.alias_offset(np.array(4, dtype=np.int64)) == np.int64(5) - result = build_c_extension(source, output_dir=tmp_path / "build", output_name="c_in_place") - module = sole_native_module(result.import_module()) + +def test_default_c_pointer_writes_back_into_caller_owned_runtime_rank_storage(source_build): + """The ``T *`` default passes caller-owned storage of whatever rank arrived.""" + _result, module = source_build value = np.array(3.0, dtype=np.float64) assert module.twice(value) is None @@ -222,100 +166,59 @@ def test_default_c_pointer_writes_back_into_caller_owned_runtime_rank_storage(tm module.twice(np.float64(3.0)) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_typedef_declarations_resolve_to_their_exact_underlying_builtin(tmp_path: Path): - """A user typedef only the source's headers define cannot enter the binding.""" - source = tmp_path / "aliases.c" - source.write_text( - """#include -typedef long my_int; -my_int alias_step(my_int value) { return value + 1; } -ptrdiff_t alias_offset(const ptrdiff_t *value) { return *value + 1; } -""", - encoding="utf-8", - ) - - result = build_c_extension(source, output_dir=tmp_path / "build", output_name="c_aliases") - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") - - assert "long alias_step(long value);" in binding - assert "long alias_offset(const long * value);" in binding - assert "my_int" not in binding - assert module.alias_step(np.int64(4)) == np.int64(5) - assert module.alias_offset(np.array(4, dtype=np.int64)) == np.int64(5) +def test_c_native_language_is_explicit_for_a_source_free_pyi_contract(contract_build): + result, module = contract_build + binding = _binding(result) + assert module.add(np.float64(4.0), np.float64(2.5)) == np.float64(6.5) + assert module.increment(np.int32(4)) == np.int32(5) + assert result.native_build_plan.compilation_units[0].language == "c" + assert result.manifest["extension"]["native_language"] == "c" + assert result.manifest["compiler"]["c_flags"] == [] + # A contract needs ``@bind`` only when the Python and native names differ. + assert module.bump(np.int32(4)) == np.int32(5) + assert "int32_t bump(int32_t value);" in binding + assert "result = bump(bound_value);" in binding -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_source_free_c_contract_keeps_the_standard_typedef_it_names(tmp_path: Path): - """``SizeT`` is a contract spelling, so the binding declares ``size_t``.""" - contract = tmp_path / "sizes.pyi" - contract.write_text( - """from prik.contracts import SizeT -def total(value: SizeT) -> SizeT: ... -""", - encoding="utf-8", - ) - source = tmp_path / "sizes.c" - source.write_text( - """#include -size_t total(size_t value) { return value + 1; } -""", - encoding="utf-8", - ) +def test_c_contract_reuses_direct_projection_value_address_literal_and_hidden_output_paths(contract_build): + result, module = contract_build + binding = _binding(result) - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build", + assert module.projected(np.int32(2), np.int32(3)) == np.int32(325) + assert module.projected_output(np.int32(2), np.int32(3)) == np.int32(325) + assert "int32_t projected_native(int32_t right, int32_t * left, int32_t literal_2);" in binding + assert ( + "void projected_output_native(int32_t right, int32_t * left, int32_t literal_2, int32_t * output);" in binding ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") - assert "size_t total(size_t value);" in binding - assert "#include " in binding - assert module.total(np.uint64(4)) == np.uint64(5) +def test_reordered_c_projection_keeps_each_argument_its_own_declared_type(contract_build): + """A route-neutral reorder must not resolve one argument against another.""" + result, module = contract_build -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_contract_supports_private_candidates_behind_one_overloaded_name(tmp_path: Path): - """An unexported concrete procedure is a shared contract feature, not a C limit.""" - contract = tmp_path / "overloads.pyi" - contract.write_text( - """from prik.contracts import Float64, Int32, overload, private + assert module.combine(np.float64(2.5), np.int32(4)) == np.float64(10.0) + assert "double combine(int32_t count, double scale);" in _binding(result) + with pytest.raises(TypeError, match=r"numpy\.float64 or rank-zero array for argument scale"): + module.combine(np.int32(4), np.int32(4)) -@private -def scale_integer(value: Int32) -> Int32: ... -@private -def scale_real(value: Float64) -> Float64: ... +def test_source_free_c_contract_keeps_the_standard_typedef_it_names(contract_build): + """``SizeT`` is a contract spelling, so the binding declares ``size_t``.""" + result, module = contract_build + binding = _binding(result) -@overload("scale_integer") -def scale(value: Int32) -> Int32: ... + assert "size_t total(size_t value);" in binding + assert "#include " in binding + assert module.total(np.uint64(4)) == np.uint64(5) -@overload("scale_real") -def scale(value: Float64) -> Float64: ... -""", - encoding="utf-8", - ) - source = tmp_path / "overloads.c" - source.write_text( - """int scale_integer(int value) { return value * 2; } -double scale_real(double value) { return value * 2.0; } -""", - encoding="utf-8", - ) - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build", - ) - module = sole_native_module(result.import_module()) +def test_c_contract_supports_private_candidates_behind_one_overloaded_name(contract_build): + """An unexported concrete procedure is a shared contract feature, not a C limit.""" + _result, module = contract_build assert module.scale(np.int32(21)) == np.int32(42) assert module.scale(np.float64(1.5)) == np.float64(3.0) - assert [name for name in dir(module) if not name.startswith("_")] == ["scale"] + published = {name for name in dir(module) if not name.startswith("_")} + assert "scale" in published + assert not {"scale_integer", "scale_real"} & published diff --git a/tests/c/primitive_scalars/policy/test_direct_c_policy.py b/tests/c/primitive_scalars/policy/test_direct_c_policy.py index dcad3f24b..61f299dca 100644 --- a/tests/c/primitive_scalars/policy/test_direct_c_policy.py +++ b/tests/c/primitive_scalars/policy/test_direct_c_policy.py @@ -9,61 +9,6 @@ from prik.semantics.native_contract import validate_pyi_native_contract -def _complete(source: str): - module = c_file_to_semantic_module(parse_c_file(source, filename="api.c")) - complete_semantic_policies(module) - return module.functions[0].metadata["resolved_function_wrapper_policy"] - - -def test_supported_c_scalar_policy_selects_direct_c_abi_without_a_bridge_facet(): - policy = _complete("double add(double left, double right) { return left + right; }\n") - - assert policy.supported is True - assert policy.entrypoint_action.value == "direct_c_abi" - assert policy.direct_c_abi.result.source_spelling == "double" - assert tuple(item.source_spelling for item in policy.direct_c_abi.parameters) == ("double", "double") - - -def test_source_free_exact_scalar_contract_completes_native_and_contract_storage_types(): - module = pyi_text_to_semantic_module( - """from prik.contracts import Arg, CLongLong, Int64, Return, native_call -@native_call([CLongLong(Arg(0))], result=CLongLong(Return(0))) -def convert(value: Int64) -> Int64: ... -""", - module_name="exact", - native_language="c", - ) - validate_pyi_native_contract([module]) - complete_semantic_policies(module) - - policy = module.functions[0].metadata["resolved_function_wrapper_policy"] - - assert policy.native_call_slots[0].native_scalar_c_type == "long long" - assert policy.direct_c_abi.parameters[0].source_spelling == "long long" - assert policy.direct_c_abi.result.source_spelling == "long long" - assert policy.direct_c_abi.result.converts_to_contract_storage is True - - -def test_source_free_exact_array_contract_requires_native_numpy_element_storage(): - module = pyi_text_to_semantic_module( - """from prik.contracts import Arg, CLongLong, Int64, native_call -@native_call([CLongLong(Arg(0))]) -def update(values: Int64[:]) -> None: ... -""", - module_name="exact_array", - native_language="c", - ) - validate_pyi_native_contract([module]) - complete_semantic_policies(module) - - policy = module.functions[0].metadata["resolved_function_wrapper_policy"] - - assert policy.arguments[0].native_array_element_c_type == "long long" - assert policy.native_call_slots[0].native_scalar_c_type == "long long" - assert policy.direct_c_abi.parameters[0].source_spelling == "long long *" - assert policy.direct_c_abi.parameters[0].converts_to_contract_storage is False - - def test_exact_c_bool_rank_zero_storage_fails_before_planning(): module = pyi_text_to_semantic_module( """from prik.contracts import Arg, Bool, CBool, native_call diff --git a/tests/c/primitive_scalars/semantics/test_exact_native_scalar_contract.py b/tests/c/primitive_scalars/semantics/test_exact_native_scalar_contract.py index e266e6ba3..932b017e2 100644 --- a/tests/c/primitive_scalars/semantics/test_exact_native_scalar_contract.py +++ b/tests/c/primitive_scalars/semantics/test_exact_native_scalar_contract.py @@ -2,7 +2,6 @@ import pytest -from prik.contracts import NATIVE_C_SCALAR_IDENTITIES from prik.parsers.c import parse_c_file from prik.pipeline.pyi import pyi_text_to_semantic_module from prik.printers.pyi import emit_module @@ -57,32 +56,35 @@ def test_same_width_long_and_int32_t_still_keep_their_distinct_c_identities(): assert ") -> Int32" in text -def test_exact_native_argument_and_result_contract_round_trip(): - text = """from prik.contracts import Arg, CLongLong, Int64, Return, native_call +@pytest.mark.parametrize( + ("text", "rendered"), + [ + pytest.param( + """from prik.contracts import Arg, CLongLong, Int64, Return, native_call @native_call([CLongLong(Arg(0))], result=CLongLong(Return(0))) def convert(value: Int64) -> Int64: ... -""" - - module = pyi_text_to_semantic_module(text, module_name="exact", native_language="c") - - assert module.functions[0].projection[0].native_c_identity == "CLongLong" - assert module.functions[0].return_type.metadata["native_c_scalar_identity"] == "CLongLong" - rendered = emit_module(module) - assert "@native_call([CLongLong(Arg(0))], result=CLongLong(Return(0)))" in rendered - - -def test_exact_native_array_element_contract_round_trips_without_a_public_c_type(): - text = """from prik.contracts import Arg, CLongLong, Int64, native_call +""", + ("@native_call([CLongLong(Arg(0))], result=CLongLong(Return(0)))", "value: Int64"), + id="scalar-argument-and-result", + ), + pytest.param( + """from prik.contracts import Arg, CLongLong, Int64, native_call @native_call([CLongLong(Arg(0))]) def update(values: Int64[:]) -> None: ... -""" - - module = pyi_text_to_semantic_module(text, module_name="exact_array", native_language="c") +""", + ("@native_call([CLongLong(Arg(0))])", "values: Int64[:]"), + id="array-element-without-a-public-c-type", + ), + ], +) +def test_exact_native_identity_contract_round_trips(text, rendered): + module = pyi_text_to_semantic_module(text, module_name="exact", native_language="c") assert module.functions[0].projection[0].native_c_identity == "CLongLong" - rendered = emit_module(module) - assert "@native_call([CLongLong(Arg(0))])" in rendered - assert "values: Int64[:]" in rendered + if module.functions[0].return_type is not None: + assert module.functions[0].return_type.metadata["native_c_scalar_identity"] == "CLongLong" + emitted = emit_module(module) + assert all(fragment in emitted for fragment in rendered) def test_native_c_identity_requires_exactly_one_positional_reference(): @@ -97,7 +99,7 @@ def invalid(value: Int64) -> None: ... ) -@pytest.mark.parametrize("native_name", sorted(NATIVE_C_SCALAR_IDENTITIES)) +@pytest.mark.parametrize("native_name", ["CChar", "CLongLong", "CLongDoubleComplex"]) def test_native_scalar_names_are_not_public_signature_types(native_name): with pytest.raises(ValueError, match="valid only inside @native_call"): pyi_text_to_semantic_module( diff --git a/tests/c/primitive_strings/end_to_end/test_direct_c_strings.py b/tests/c/primitive_strings/end_to_end/test_direct_c_strings.py index 2df72e9d3..f5b89dae5 100644 --- a/tests/c/primitive_strings/end_to_end/test_direct_c_strings.py +++ b/tests/c/primitive_strings/end_to_end/test_direct_c_strings.py @@ -1,4 +1,8 @@ -"""Compiled evidence for the adopted rank-zero C character contracts.""" +"""Compiled evidence for the adopted rank-zero C character contracts. + +Every supported form shares one module-scoped build; the fail-closed forms are +refused before planning and never reach a compiler. +""" import shutil from pathlib import Path @@ -9,18 +13,108 @@ from prik import build_pyi_extension from tests.c._support.runtime import sole_native_module -SOURCE = """#include + +pytestmark = pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") + +_CONTRACT = """from prik.contracts import ( + Arg, Float64, Hidden, Int32, Return, Returns, String, native_call, raises, +) + +def name_length(text: String) -> Int32: ... + +def shout_any(text: String, out: String[...][()]) -> None: ... + +def shout_fixed(text: String, out: String[32][()]) -> None: ... + +@raises(status="status", message="message", success=0) +@native_call([Arg(0), Return("root", 0), Hidden("status", Int32), Hidden("message", String[64])]) +def checked_sqrt(value: Float64) -> Returns["root", Float64]: ... + +@raises(status="status", message="message", success=0) +@native_call([Arg(0), Arg(1), Hidden("status", Int32)]) +def checked_visible(value: Float64, message: String[64][()]) -> None: ... + +@raises(status="status", message="message", success=0) +@native_call([Arg(0), Arg(1), Hidden("status", Int32)]) +def checked_borrowed(value: Float64, message: String) -> None: ... + +@raises(status="status", message="message", success=0) +@native_call([Arg(0), Arg(1), Hidden("status", Int32)]) +def checked_assumed(value: Float64, message: String[...][()]) -> None: ... + +@raises(status="status", message="message", success=0) +@native_call([Arg(0), Arg(1), Hidden("status", Int32)]) +def padded(value: Float64, message: String[64][()]) -> None: ... + +@raises(status="status", message="message", success=0) +@native_call([Arg(0), Hidden("status", Int32), Hidden("message", String[8])]) +def wide(value: Float64) -> None: ... +""" + +_SOURCE = """#include +#include #include int name_length(const char *text) { return (int)strlen(text); } -void shout(const char *text, char *out) { - size_t index = 0; - for (; text[index]; ++index) { - char value = text[index]; - out[index] = (value >= 'a' && value <= 'z') ? (char)(value - 32) : value; +#define PRIK_SHOUT(name) \\ + void name(const char *text, char *out) { \\ + size_t index = 0; \\ + for (; text[index]; ++index) { \\ + char value = text[index]; \\ + out[index] = (value >= 'a' && value <= 'z') ? (char)(value - 32) : value; \\ + } \\ + out[index] = '\\0'; \\ + } +PRIK_SHOUT(shout_any) +PRIK_SHOUT(shout_fixed) + +void checked_sqrt(double value, double *root, int *status, char *message) { + if (value < 0.0) { + *status = -1; + *root = 0.0; + strcpy(message, "value must not be negative"); + return; + } + *status = 0; + message[0] = '\\0'; + *root = value == 4.0 ? 2.0 : value; +} + +#define PRIK_CHECKED(name) \\ + void name(double value, char *message, int *status) { \\ + if (value < 0.0) { \\ + *status = -1; \\ + snprintf(message, 64, "bad value %g", value); \\ + return; \\ + } \\ + *status = 0; \\ + message[0] = '\\0'; \\ } - out[index] = '\\0'; +PRIK_CHECKED(checked_visible) +PRIK_CHECKED(checked_borrowed) +PRIK_CHECKED(checked_assumed) + +void padded(double value, char *message, int *status) { + int index = 0; + const char *text = "padded failure"; + if (value >= 0.0) { *status = 0; message[0] = '\\0'; return; } + *status = -1; + /* Fill the whole buffer with blanks, exactly as fixed-length native + character storage does, and leave no terminator. */ + for (; index < 64; ++index) { message[index] = ' '; } + for (index = 0; text[index]; ++index) { message[index] = text[index]; } +} + +void wide(double value, int *status, char *message) { + if (value < 0.0) { + *status = -1; + /* Fill the declared width with no terminator inside it. */ + memset(message, 'x', 8); + return; + } + *status = 0; + message[0] = '\\0'; } """ @@ -29,7 +123,7 @@ def _build(tmp_path: Path, contract_text: str, name: str): contract = tmp_path / f"{name}.pyi" contract.write_text(contract_text, encoding="utf-8") source = tmp_path / f"{name}.c" - source.write_text(SOURCE, encoding="utf-8") + source.write_text(_SOURCE, encoding="utf-8") return build_pyi_extension( contract, native_language="c", @@ -39,16 +133,16 @@ def _build(tmp_path: Path, contract_text: str, name: str): ) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_string_input_borrows_the_python_payload_as_a_const_char_pointer(tmp_path: Path): - """``String`` states a read-only input, so the prototype keeps ``const``.""" - result = _build( - tmp_path, - "from prik.contracts import Int32, String\n\ndef name_length(text: String) -> Int32: ...\n", - "text_in", - ) - module = sole_native_module(result.import_module()) +@pytest.fixture(scope="module") +def string_build(tmp_path_factory): + result = _build(tmp_path_factory.mktemp("c_strings"), _CONTRACT, "c_strings") binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + return binding, sole_native_module(result.import_module()) + + +def test_string_input_borrows_the_python_payload_as_a_const_char_pointer(string_build): + """``String`` states a read-only input, so the prototype keeps ``const``.""" + binding, module = string_build assert "int32_t name_length(const char * text);" in binding assert module.name_length("hello") == np.int32(5) @@ -58,44 +152,30 @@ def test_string_input_borrows_the_python_payload_as_a_const_char_pointer(tmp_pat assert module.name_length("a\0b") == np.int32(1) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_rank_zero_string_storage_is_written_in_place_at_any_declared_capacity(tmp_path: Path): +def test_rank_zero_string_storage_is_written_in_place_at_any_declared_capacity(string_build): """``String[...][()]`` passes the caller's bytes through untouched.""" - result = _build( - tmp_path, - "from prik.contracts import String\n\ndef shout(text: String, out: String[...][()]) -> None: ...\n", - "text_assumed", - ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + binding, module = string_build - assert "void shout(const char * text, char * out);" in binding + assert "void shout_any(const char * text, char * out);" in binding for width in ("S8", "S32"): buffer = np.array(b"", dtype=width) - assert module.shout("hello", buffer) is None + assert module.shout_any("hello", buffer) is None assert buffer[()] == b"HELLO" with pytest.raises(TypeError, match=r"rank-zero numpy\.ndarray"): - module.shout("hi", np.array([1.0])) + module.shout_any("hi", np.array([1.0])) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_declared_string_capacity_validates_the_caller_itemsize(tmp_path: Path): +def test_declared_string_capacity_validates_the_caller_itemsize(string_build): """``String[n][()]`` is the form that asks PRIK to check the width.""" - result = _build( - tmp_path, - "from prik.contracts import String\n\ndef shout(text: String, out: String[32][()]) -> None: ...\n", - "text_fixed", - ) - module = sole_native_module(result.import_module()) + _binding, module = string_build buffer = np.array(b"", dtype="S32") - assert module.shout("hello", buffer) is None + assert module.shout_fixed("hello", buffer) is None assert buffer[()] == b"HELLO" with pytest.raises(TypeError, match="itemsize 32"): - module.shout("hello", np.array(b"", dtype="S8")) + module.shout_fixed("hello", np.array(b"", dtype="S8")) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") def test_string_arrays_stay_outside_the_direct_c_lane(tmp_path: Path): """Only rank-zero character contracts have a completed C lowering.""" with pytest.raises(ValueError, match="C_DIRECT_UNSUPPORTED_STRING_CONTRACT:text"): @@ -106,47 +186,9 @@ def test_string_arrays_stay_outside_the_direct_c_lane(tmp_path: Path): ) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_raises_message_uses_a_binding_owned_buffer_without_an_adapter(tmp_path: Path): +def test_raises_message_uses_a_binding_owned_buffer_without_an_adapter(string_build): """Direct C owns the message buffer; only a bridged route allocates one.""" - contract = tmp_path / "checked.pyi" - contract.write_text( - """from prik.contracts import Arg, Float64, Hidden, Int32, Return, Returns, String, bind, native_call, raises - -@bind("checked_sqrt") -@raises(status="status", message="message", success=0) -@native_call([Arg(0), Return("root", 0), Hidden("status", Int32), Hidden("message", String[64])]) -def checked_sqrt(value: Float64) -> Returns["root", Float64]: ... -""", - encoding="utf-8", - ) - source = tmp_path / "checked.c" - source.write_text( - """#include - -void checked_sqrt(double value, double *root, int *status, char *message) { - if (value < 0.0) { - *status = -1; - *root = 0.0; - strcpy(message, "value must not be negative"); - return; - } - *status = 0; - message[0] = '\\0'; - *root = value == 4.0 ? 2.0 : value; -} -""", - encoding="utf-8", - ) - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build_message", - output_name="checked", - ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + binding, module = string_build # The callee receives the buffer itself, never the adapter's ``char **``. assert "void checked_sqrt(double value, double * root, int32_t * status, char * message);" in binding @@ -158,7 +200,6 @@ def checked_sqrt(value: Float64) -> Returns["root", Float64]: ... module.checked_sqrt(np.float64(-1.0)) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") @pytest.mark.parametrize("declaration", ["String", "String[...]", "String[:]"]) def test_raises_message_without_a_declared_capacity_stays_fail_closed(tmp_path: Path, declaration: str): """An assumed or deferred width leaves the binding no buffer size to emit. @@ -168,7 +209,7 @@ def test_raises_message_without_a_declared_capacity_stays_fail_closed(tmp_path: """ contract = f"""from prik.contracts import Arg, Float64, Hidden, Int32, String, bind, native_call, raises -@bind("checked") +@bind("checked_visible") @raises(status="status", message="message", success=0) @native_call([Arg(0), Hidden("status", Int32), Hidden("message", {declaration})]) def checked(value: Float64) -> None: ... @@ -177,172 +218,56 @@ def checked(value: Float64) -> None: ... _build(tmp_path, contract, "message") -CHECKED_SOURCE = """#include - -void checked(double value, char *message, int *status) { - if (value < 0.0) { - *status = -1; - snprintf(message, 64, "bad value %g", value); - return; - } - *status = 0; - message[0] = '\\0'; -} -""" - - -def _build_checked(tmp_path: Path, declaration: str, name: str): - contract = tmp_path / f"{name}.pyi" - contract.write_text( - f"""from prik.contracts import Arg, Float64, Hidden, Int32, String, bind, native_call, raises - -@bind("checked") -@raises(status="status", message="message", success=0) -@native_call([Arg(0), Arg(1), Hidden("status", Int32)]) -def checked(value: Float64, message: {declaration}) -> None: ... -""", - encoding="utf-8", - ) - source = tmp_path / f"{name}.c" - source.write_text(CHECKED_SOURCE, encoding="utf-8") - return build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / f"build_{name}", - output_name=name, - ) - - -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_raises_message_reads_a_caller_supplied_buffer(tmp_path: Path): +def test_raises_message_reads_a_caller_supplied_buffer(string_build): """A visible ``String[n][()]`` message carries its own capacity.""" - result = _build_checked(tmp_path, "String[64][()]", "visible") - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + binding, module = string_build # The caller owns the buffer, so the binding neither NULL-checks nor frees it. assert "free(bound_message)" not in binding - assert "void checked(double value, char * message, int32_t * status);" in binding + assert "void checked_visible(double value, char * message, int32_t * status);" in binding buffer = np.array(b"", dtype="S64") - assert module.checked(np.float64(9.0), buffer) is None + assert module.checked_visible(np.float64(9.0), buffer) is None assert buffer[()] == b"" with pytest.raises(RuntimeError, match="bad value -1"): - module.checked(np.float64(-1.0), buffer) + module.checked_visible(np.float64(-1.0), buffer) # Raising does not consume the buffer; the caller can still inspect it. assert buffer[()] == b"bad value -1" -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_raises_message_accepts_a_borrowed_string_payload(tmp_path: Path): +def test_raises_message_accepts_a_borrowed_string_payload(string_build): """``String`` states ``const char *``; PRIK does not police what C writes.""" - result = _build_checked(tmp_path, "String", "borrowed") - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + binding, module = string_build - assert "void checked(double value, const char * message, int32_t * status);" in binding + assert "void checked_borrowed(double value, const char * message, int32_t * status);" in binding scratch = "\0" * 64 - assert module.checked(np.float64(9.0), scratch) is None + assert module.checked_borrowed(np.float64(9.0), scratch) is None with pytest.raises(RuntimeError, match="bad value -1"): - module.checked(np.float64(-1.0), scratch) + module.checked_borrowed(np.float64(-1.0), scratch) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_visible_message_needs_no_declared_capacity(tmp_path: Path): +def test_visible_message_needs_no_declared_capacity(string_build): """The caller's storage supplies the width a hidden message must declare.""" - result = _build_checked(tmp_path, "String[...][()]", "assumed") - module = sole_native_module(result.import_module()) + _binding, module = string_build buffer = np.array(b"", dtype="S64") with pytest.raises(RuntimeError, match="bad value -2"): - module.checked(np.float64(-2.0), buffer) - - -PADDED_SOURCE = """void checked(double value, char *message, int *status) { - int index = 0; - const char *text = "padded failure"; - if (value >= 0.0) { *status = 0; message[0] = '\\0'; return; } - *status = -1; - /* Fill the whole buffer with blanks, exactly as fixed-length native - character storage does, and leave no terminator. */ - for (; index < 64; ++index) { message[index] = ' '; } - for (index = 0; text[index]; ++index) { message[index] = text[index]; } -} -""" + module.checked_assumed(np.float64(-2.0), buffer) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_visible_message_never_reads_past_the_caller_capacity(tmp_path: Path): +def test_visible_message_never_reads_past_the_caller_capacity(string_build): """An unterminated buffer is read as padded storage, not scanned for a NUL.""" - contract = tmp_path / "padded.pyi" - contract.write_text( - """from prik.contracts import Arg, Float64, Hidden, Int32, String, bind, native_call, raises - -@bind("checked") -@raises(status="status", message="message", success=0) -@native_call([Arg(0), Arg(1), Hidden("status", Int32)]) -def checked(value: Float64, message: String[64][()]) -> None: ... -""", - encoding="utf-8", - ) - source = tmp_path / "padded.c" - source.write_text(PADDED_SOURCE, encoding="utf-8") - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build_padded", - output_name="padded", - ) - module = sole_native_module(result.import_module()) + _binding, module = string_build buffer = np.array(b"", dtype="S64") with pytest.raises(RuntimeError, match=r"^padded failure$"): - module.checked(np.float64(-1.0), buffer) + module.padded(np.float64(-1.0), buffer) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_hidden_message_read_is_bounded_by_the_declared_capacity(tmp_path: Path): +def test_hidden_message_read_is_bounded_by_the_declared_capacity(string_build): """The binding reads at most the width the contract declared.""" - contract = tmp_path / "wide.pyi" - contract.write_text( - """from prik.contracts import Arg, Float64, Hidden, Int32, String, bind, native_call, raises - -@bind("wide") -@raises(status="status", message="message", success=0) -@native_call([Arg(0), Hidden("status", Int32), Hidden("message", String[8])]) -def wide(value: Float64) -> None: ... -""", - encoding="utf-8", - ) - source = tmp_path / "wide.c" - source.write_text( - """#include - -void wide(double value, int *status, char *message) { - if (value < 0.0) { - *status = -1; - /* Fill the declared width with no terminator inside it. */ - memset(message, 'x', 8); - return; - } - *status = 0; - message[0] = '\\0'; -} -""", - encoding="utf-8", - ) - result = build_pyi_extension( - contract, - native_language="c", - native_c_sources=[source], - output_dir=tmp_path / "build_wide", - output_name="wide", - ) - module = sole_native_module(result.import_module()) - binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + binding, module = string_build assert "prik_status_message_text" in binding with pytest.raises(RuntimeError, match=r"^x{8}$"): diff --git a/tests/c/records/parsing/test_c_structs_unions_typedefs.py b/tests/c/records/parsing/test_c_structs_unions_typedefs.py index b508b84ca..60cb970cc 100644 --- a/tests/c/records/parsing/test_c_structs_unions_typedefs.py +++ b/tests/c/records/parsing/test_c_structs_unions_typedefs.py @@ -19,17 +19,27 @@ def test_named_struct_members_are_variables_in_source_order(): assert point.members[2].type.components[0].bound == "2" -def test_typedef_struct_alias_refers_to_the_concrete_struct_object(): - from prik.parsers.c import parse_c_file +def test_typedef_aliases_refer_to_the_concrete_tag_objects(): + from prik.parsers.c import CUnion, parse_c_file parsed = parse_c_file( - "typedef struct point { double x; double y; } point_t;\n", - filename="typedef_struct.h", + "typedef struct point { double x; double y; } point_t;\n" + "typedef struct { int code; } result_t;\n" + "typedef union { int i; double d; } value_t;\n", + filename="typedef_tags.h", ) - assert parsed.structs[0].name == "point" - assert parsed.typedefs[0].name == "point_t" - assert parsed.typedefs[0].type is parsed.structs[0] + point, result = parsed.structs + typedefs = {typedef.name: typedef for typedef in parsed.typedefs} + assert point.name == "point" + assert typedefs["point_t"].type is point + assert result.name is None + assert result.anonymous_id + assert typedefs["result_t"].type is result + assert isinstance(parsed.unions[0], CUnion) + assert parsed.unions[0].anonymous_id + assert result.anonymous_id != parsed.unions[0].anonymous_id + assert typedefs["value_t"].type is parsed.unions[0] def test_forward_struct_declaration_is_completed_by_later_definition(): @@ -46,49 +56,6 @@ def test_forward_struct_declaration_is_completed_by_later_definition(): assert parsed.diagnostics == [] -def test_duplicate_complete_tag_definitions_report_diagnostics(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - "struct state { int id; };\nstruct state { int id; };\n", - filename="duplicate_struct.h", - ) - - assert [struct.name for struct in parsed.structs] == ["state"] - assert any(diag.code == "C_DUPLICATE_TAG_DEFINITION" for diag in parsed.diagnostics) - - -def test_anonymous_struct_typedef_gets_stable_anonymous_id(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file("typedef struct { int code; } result_t;\n", filename="anon_struct.h") - - assert parsed.structs[0].name is None - assert parsed.structs[0].anonymous_id - assert parsed.typedefs[0].type is parsed.structs[0] - - -def test_union_members_are_variables_without_struct_field_class(): - from prik.parsers.c import CUnion, CVariable, parse_c_file - - parsed = parse_c_file("union value { int i; double d; };\n", filename="union.h") - - value = parsed.unions[0] - assert isinstance(value, CUnion) - assert [member.name for member in value.members] == ["i", "d"] - assert all(isinstance(member, CVariable) for member in value.members) - - -def test_anonymous_union_typedef_refers_to_the_concrete_union_object(): - from prik.parsers.c import CUnion, parse_c_file - - parsed = parse_c_file("typedef union { int i; double d; } value_t;\n", filename="anon_union.h") - - assert isinstance(parsed.unions[0], CUnion) - assert parsed.unions[0].anonymous_id - assert parsed.typedefs[0].type is parsed.unions[0] - - def test_function_signatures_using_unions_by_value_report_diagnostics(): from prik.parsers.c import parse_c_file @@ -151,11 +118,13 @@ def test_incomplete_union_and_tag_typedef_aliases_use_concrete_tag_classes(): assert typedefs["payload_t"].type.name == "payload" -def test_repeated_union_and_enum_tags_normalize_with_duplicate_diagnostics(): +def test_repeated_struct_union_and_enum_tags_normalize_with_duplicate_diagnostics(): from prik.parsers.c import parse_c_file parsed = parse_c_file( """ +struct state { int id; }; +struct state { int id; }; union value; union value { int integer; }; union value { double real; }; @@ -167,7 +136,9 @@ def test_repeated_union_and_enum_tags_normalize_with_duplicate_diagnostics(): assert [member.name for member in parsed.unions[0].members] == ["integer"] assert [constant.name for constant in parsed.enums[0].constants] == ["STATUS_OK"] + assert [struct.name for struct in parsed.structs] == ["state"] assert [(diagnostic.code, diagnostic.unit_kind) for diagnostic in parsed.diagnostics] == [ + ("C_DUPLICATE_TAG_DEFINITION", "struct"), ("C_DUPLICATE_TAG_DEFINITION", "union"), ("C_DUPLICATE_TAG_DEFINITION", "enum"), ] @@ -372,88 +343,46 @@ def test_struct_field_missing_semicolon_reports_syntax_location(): assert error.source_line == "struct broken {" -def test_nested_aggregate_field_with_function_declarator_is_rejected(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - """struct outer { - struct inner { int x; } field, make(void); - int kept; -}; -""", - filename="nested_bad_field.h", - ) - - assert [member.name for member in parsed.structs[0].members] == ["kept"] - assert len(parsed.diagnostics) == 1 - diagnostic = parsed.diagnostics[0] - assert diagnostic.code == "C_UNSUPPORTED_FIELD_DECLARATION" - assert diagnostic.message == "Unsupported nested aggregate field declaration." - assert diagnostic.location is not None - assert diagnostic.location.line == 2 - assert diagnostic.location.column == 5 - - -def test_bad_field_declarator_does_not_stop_later_declarators(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - """struct bad { - int broken @@, kept; -}; -""", - filename="bad_field_multi.h", - ) - - assert [member.name for member in parsed.structs[0].members] == ["kept"] - assert len(parsed.diagnostics) == 1 - assert parsed.diagnostics[0].code == "C_UNSUPPORTED_FIELD_DECLARATION" - - -def test_unnamed_field_type_without_bit_width_reports_diagnostic(): - from prik.parsers.c import parse_c_file - - parsed = parse_c_file( - """struct bad { - int *; - int kept; -}; -""", - filename="unnamed_field.h", - ) - - assert [member.name for member in parsed.structs[0].members] == ["kept"] - assert len(parsed.diagnostics) == 1 - diagnostic = parsed.diagnostics[0] - assert diagnostic.code == "C_UNSUPPORTED_FIELD_DECLARATION" - assert diagnostic.message == "Unnamed field type is not supported." - assert diagnostic.location is not None - assert diagnostic.location.line == 2 - assert diagnostic.location.column == 5 - - -def test_unsupported_field_declarator_is_reported_at_member_location(): +@pytest.mark.parametrize( + ("field", "message", "members"), + [ + pytest.param( + "int broken @@;", + "Unsupported declarator syntax after parsed type layers: '@@'.", + ["kept"], + id="bad-declarator", + ), + pytest.param( + "int broken @@, kept_too;", + "Unsupported declarator syntax", + ["kept_too", "kept"], + id="bad-declarator-in-list", + ), + pytest.param("int *;", "Unnamed field type is not supported.", ["kept"], id="unnamed-field"), + pytest.param( + "struct inner { int x; } field, make(void);", + "Unsupported nested aggregate field declaration.", + ["kept"], + id="nested-aggregate-function-declarator", + ), + ], +) +def test_unsupported_field_declaration_is_reported_and_later_members_continue(field, message, members): from prik.parsers.c import parse_c_file - parsed = parse_c_file( - """struct bad { - int broken @@; - int kept; -}; -""", - filename="bad_field.h", - ) + parsed = parse_c_file(f"struct bad {{\n {field}\n int kept;\n}};\n", filename="bad_field.h") - assert [member.name for member in parsed.structs[0].members] == ["kept"] + assert [member.name for member in parsed.structs[0].members] == members assert len(parsed.diagnostics) == 1 diagnostic = parsed.diagnostics[0] assert diagnostic.code == "C_UNSUPPORTED_FIELD_DECLARATION" assert diagnostic.severity == "warning" assert diagnostic.unit_kind == "struct_field" - assert diagnostic.unit_name is None - assert diagnostic.message == "Unsupported declarator syntax after parsed type layers: '@@'." + assert message in diagnostic.message assert diagnostic.location is not None - assert diagnostic.location.filename == "bad_field.h" - assert diagnostic.location.line == 2 - assert diagnostic.location.column == 5 - assert diagnostic.location.source_line == " int broken @@;" + assert (diagnostic.location.filename, diagnostic.location.line, diagnostic.location.column) == ( + "bad_field.h", + 2, + 5, + ) + assert diagnostic.location.source_line == f" {field}" diff --git a/tests/c/records/semantics/test_c_record_semantics.py b/tests/c/records/semantics/test_c_record_semantics.py index cf9bd30a1..c981c9b52 100644 --- a/tests/c/records/semantics/test_c_record_semantics.py +++ b/tests/c/records/semantics/test_c_record_semantics.py @@ -7,7 +7,6 @@ CArray, CComposedType, CDouble, - CFile, CFunction, CInitializer, CInt, @@ -27,12 +26,7 @@ c_file_to_semantic_modules, ) from prik.semantics.models import ( - SemanticArgument, - SemanticClass, SemanticField, - SemanticModule, - SemanticOrigin, - SemanticType, ) from tests.c._support.semantic_conversion import ( _assert_c_origin, @@ -85,6 +79,7 @@ def test_c2ir_private_include_types_remain_available_as_opaque_handles(): # 1 "api.h" 2 struct private_context *make_context(void); void use_context(struct private_context *ctx); +void use_context_value(struct private_context ctx); """, filename="api.h", preprocessing="compiler", @@ -113,6 +108,8 @@ def test_c2ir_private_include_types_remain_available_as_opaque_handles(): "wrapped": False, "representation": "opaque", } + by_value = _function(module, "use_context_value").arguments[0].semantic_type + assert by_value.metadata["external_type_ref"]["representation"] == "opaque" assert "from .private import private_context" in stubs["api"] assert ( stubs["private"] @@ -163,65 +160,6 @@ def test_c2ir_preserves_anonymous_aggregate_members_as_nested_c_classes(): ] -def test_c2ir_private_include_opaque_struct_by_value_preserves_the_external_reference(): - parsed = parse_c_file( - """ -# 1 "private.h" 1 -struct private_context { int internal; }; -# 1 "api.h" 2 -void use_context(struct private_context ctx); -""", - filename="api.h", - preprocessing="compiler", - ) - parsed.preprocessing_recipe = { - "included_files": [ - {"path": "api.h", "dependency_kind": "root", "exposure": "public"}, - {"path": "private.h", "dependency_kind": "project", "exposure": "private"}, - ] - } - - module = c_file_to_semantic_modules(parsed)[0] - semantic_type = _function(module, "use_context").arguments[0].semantic_type - assert semantic_type.metadata["external_type_ref"]["representation"] == "opaque" - - -def test_c2ir_externalizes_only_private_opaque_classes_with_external_origins(): - converter = CToIRConverter() - public = SemanticClass(name="public", origin=SemanticOrigin(source_location={"filename": "api.h"})) - private_plain = SemanticClass( - name="private_plain", - visibility="private", - origin=SemanticOrigin(source_location={"filename": "private.h"}), - ) - private_without_location = SemanticClass( - name="private_without_location", visibility="private", base_classes=["Opaque"] - ) - private_external = SemanticClass( - name="private_external", - visibility="private", - base_classes=["Opaque"], - origin=SemanticOrigin(source_location={"filename": "private.h"}), - ) - reference = SemanticArgument(name="value", semantic_type=SemanticType(name="private_external")) - module = SemanticModule( - name="api", - classes=[public, private_plain, private_without_location, private_external], - variables=[reference], - ) - - converter._externalize_private_classes(module) - - assert [cls.name for cls in module.classes] == ["public", "private_plain", "private_without_location"] - assert reference.semantic_type.metadata["external_type_ref"] == { - "name": "private_external", - "local_name": "private_external", - "origin_module": "private", - "wrapped": False, - "representation": "opaque", - } - - def test_c2ir_uses_standard_type_probe_opaque_handle_facts(): parsed = parse_c_file("void close_file(FILE *stream);\n", filename="stdio_api.h") converter = CToIRConverter( @@ -342,7 +280,7 @@ def test_c2ir_models_pointer_to_arrays_unknown_extents_unions_and_anonymous_alia assert CToIRConverter().visit(CUnion(name="fresh_union"), as_type=True).name == "fresh_union" -def test_c2ir_preserves_nested_unresolved_owners_and_private_opaque_bases(): +def test_c2ir_preserves_nested_unresolved_owners(): converter = CToIRConverter() nested_array = converter.visit( CComposedType( @@ -369,24 +307,11 @@ def test_c2ir_preserves_nested_unresolved_owners_and_private_opaque_bases(): CComposedType(components=[CUnknownType(spelling="missing_t", source_text="missing_t")]), owner="singleton", ) - private_class = SemanticClass( - name="private_handle", - base_classes=["Opaque"], - origin=SemanticOrigin(source_language="c", source_location={"filename": "private.h"}), - ) - private_module = SemanticModule(name="api", classes=[private_class]) - private_file = CFile( - filename="api.h", - preprocessing_recipe={"included_files": [{"path": "private.h", "exposure": "private"}]}, - ) - converter._apply_include_exposure(private_module, private_file) assert nested_array.name == "missing_t" assert nested_pointer.name == "missing_t" assert nested_pointer_array.name == "missing_t" assert singleton.name == "missing_t" - assert private_class.visibility == "private" - assert private_class.base_classes == ["Opaque"] def test_c2ir_marks_incomplete_by_value_structs_and_preserves_initializer_locations(): diff --git a/tests/c/symbol_collisions/codegen/test_collision_adapter_lowering.py b/tests/c/symbol_collisions/codegen/test_collision_adapter_lowering.py index 654dc83c2..c5fc44125 100644 --- a/tests/c/symbol_collisions/codegen/test_collision_adapter_lowering.py +++ b/tests/c/symbol_collisions/codegen/test_collision_adapter_lowering.py @@ -1,14 +1,14 @@ """A collision-adapted symbol is reached from a unit that excludes Python.h.""" +import pytest + from prik.parsers.c import parse_c_file from prik.parsers.fortran import parse_fortran_file as parse_fortran_source -from prik.pipeline.pyi import pyi_text_to_semantic_module from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner from prik.policy.completion import complete_semantic_policies from prik.semantics.c2ir import c_file_to_semantic_module from prik.semantics.fortran2ir import fortran_file_to_semantic_modules -from prik.semantics.native_contract import validate_pyi_native_contract _SOURCE = """long long native_round(double value) { return (long long)value; } double native_add(double left, double right) { return left + right; } @@ -25,62 +25,39 @@ def _sources_by_name(generated): return {source.path.name: source.text for source in generated.sources if source.path.suffix == ".c"} -def test_unselected_symbols_keep_the_direct_declaration_and_emit_no_adapter_unit(): - sources = _sources_by_name(_generated()) - - assert "collide_adapters.c" not in sources - assert "long long native_round(double value);" in sources["collide_wrapper.c"] - - -def test_a_selected_symbol_moves_its_native_declaration_into_the_adapter_unit(): - sources = _sources_by_name(_generated(collision_adapters=("native_round",))) - binding = sources["collide_wrapper.c"] - adapters = sources["collide_adapters.c"] - - # The binding never declares the colliding identifier itself. - assert "long long native_round(double value);" not in binding - assert "long long prik_collision_adapter_native_round(double value);" in binding - assert "prik_collision_adapter_native_round(" in binding - - # The adapter unit declares it, forwards to it, and includes no Python header. - assert "long long native_round(double value);" in adapters - assert "return (native_round)(value);" in adapters - assert "Python.h" not in adapters - - # An unselected symbol in the same module keeps its direct declaration. - assert "double native_add(double left, double right);" in binding +_DECLARATIONS = { + "native_round": ("long long native_round(double value);", "return (native_round)(value);"), + "native_add": ("double native_add(double left, double right);", "return (native_add)(left, right);"), +} -def test_collision_adapter_all_selects_every_direct_c_symbol(): - sources = _sources_by_name(_generated(collision_adapter_all=True)) +@pytest.mark.parametrize( + ("planner_options", "adapted"), + [ + pytest.param({}, set(), id="unselected"), + pytest.param({"collision_adapters": ("native_round",)}, {"native_round"}, id="one-selected-symbol"), + pytest.param({"collision_adapter_all": True}, {"native_round", "native_add"}, id="all-direct-symbols"), + ], +) +def test_a_selected_symbol_moves_its_native_declaration_into_the_adapter_unit(planner_options, adapted): + sources = _sources_by_name(_generated(**planner_options)) binding = sources["collide_wrapper.c"] - adapters = sources["collide_adapters.c"] - - assert "prik_collision_adapter_native_round(" in binding - assert "prik_collision_adapter_native_add(" in binding - assert "return (native_add)(left, right);" in adapters - - -def test_two_callables_naming_one_symbol_define_the_forwarder_once(): - """Several Python names may bind one native symbol; the forwarder is one definition.""" - module = pyi_text_to_semantic_module( - """from prik.contracts import Float64, bind - -def native_add(left: Float64, right: Float64) -> Float64: ... - -@bind("native_add") -def add_alias(left: Float64, right: Float64) -> Float64: ... -""", - module_name="collide", - native_language="c", - ) - validate_pyi_native_contract([module]) - complete_semantic_policies(module) - generated = WrapperGenerator().generate(WrapperPlanner(collision_adapter_all=True).build(module)) - adapters = _sources_by_name(generated)["collide_adapters.c"] - - assert adapters.count("prik_collision_adapter_native_add(double left, double right) {") == 1 - assert adapters.count("double native_add(double left, double right);") == 1 + adapters = sources.get("collide_adapters.c") + + assert (adapters is not None) == bool(adapted) + if adapters is not None: + # The adapter unit reaches the user symbol from a unit without Python.h. + assert "Python.h" not in adapters + for symbol, (declaration, forward) in _DECLARATIONS.items(): + if symbol in adapted: + # The binding never declares the colliding identifier itself. + assert declaration not in binding + assert f"prik_collision_adapter_{symbol}(" in binding + assert declaration in adapters + assert forward in adapters + else: + assert declaration in binding + assert f"prik_collision_adapter_{symbol}" not in binding def test_collision_adapter_all_leaves_a_fortran_bind_c_entrypoint_alone(): diff --git a/tests/c/symbol_collisions/end_to_end/test_collision_adapter_runtime.py b/tests/c/symbol_collisions/end_to_end/test_collision_adapter_runtime.py index 7d63c5648..09facc971 100644 --- a/tests/c/symbol_collisions/end_to_end/test_collision_adapter_runtime.py +++ b/tests/c/symbol_collisions/end_to_end/test_collision_adapter_runtime.py @@ -10,7 +10,7 @@ import numpy as np import pytest -from prik import build_fortran_extension, build_pyi_extension, build_pyi_extension_from_manifest +from prik import build_pyi_extension, build_pyi_extension_from_manifest from tests.c._support.runtime import sole_native_module # This user API deliberately reuses the `Py_Initialize` identifier with a @@ -35,17 +35,6 @@ def Py_Initialize(value: Int64) -> Int64: ... def initialize_alias(value: Int64) -> Int64: ... """ -_BIND_C_SOURCE = """module m - use iso_c_binding - implicit none -contains - real(c_double) function scaled(x) bind(c, name="scaled") - real(c_double), intent(in), value :: x - scaled = 2.0_c_double * x - end function scaled -end module m -""" - def _contract(tmp_path: Path) -> Path: path = tmp_path / "libm_contract.pyi" @@ -65,12 +54,6 @@ def _aliased_contract(tmp_path: Path) -> Path: return path -def _bind_c_source(tmp_path: Path) -> Path: - path = tmp_path / "bind_c_collision.f90" - path.write_text(_BIND_C_SOURCE, encoding="utf-8") - return path - - @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") def test_a_symbol_declared_by_the_binding_headers_fails_to_compile_unadapted(tmp_path: Path): with pytest.raises(RuntimeError, match="conflicting types for"): @@ -84,59 +67,33 @@ def test_a_symbol_declared_by_the_binding_headers_fails_to_compile_unadapted(tmp @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_a_collision_adapted_symbol_compiles_and_calls_the_native_implementation(tmp_path: Path): +def test_a_collision_adapted_symbol_compiles_calls_and_replays_from_its_manifest(tmp_path: Path): + """Generation writes one forwarder for two callables; the manifest replay compiles and calls it.""" result = build_pyi_extension( - _contract(tmp_path), + _aliased_contract(tmp_path), native_language="c", native_c_sources=[_native_source(tmp_path)], collision_adapters=["Py_Initialize"], + makefile=True, output_dir=tmp_path / "adapted", output_name="libm_adapted", ) - module = sole_native_module(result.import_module()) - - assert module.Py_Initialize(np.int64(5)) == np.int64(12) - assert module.Py_Initialize(np.int64(-9)) == np.int64(-2) - binding = next(path for path in result.generated_sources if path.name.endswith("_wrapper.c")) adapters = next(path for path in result.generated_sources if path.name.endswith("_adapters.c")) assert "long long Py_Initialize(long long value);" not in binding.read_text(encoding="utf-8") adapter_text = adapters.read_text(encoding="utf-8") - assert "long long Py_Initialize(long long value);" in adapter_text - assert "return (Py_Initialize)(value);" in adapter_text - + assert adapter_text.count("long long Py_Initialize(long long value);") == 1 + assert adapter_text.count("return (Py_Initialize)(value);") == 1 -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_aliased_callables_compile_and_share_one_collision_adapter(tmp_path: Path): - result = build_pyi_extension( - _aliased_contract(tmp_path), - native_language="c", - native_c_sources=[_native_source(tmp_path)], - collision_adapter_all=True, - output_dir=tmp_path / "aliased", - output_name="aliased_collision", - ) - module = sole_native_module(result.import_module()) + assert result.build_manifest is not None + assert result.manifest["extension"]["collision_adapters"] == ["Py_Initialize"] + replay = build_pyi_extension_from_manifest(result.build_manifest) + module = sole_native_module(replay.import_module()) assert module.Py_Initialize(np.int64(5)) == np.int64(12) + assert module.Py_Initialize(np.int64(-9)) == np.int64(-2) assert module.initialize_alias(np.int64(-9)) == np.int64(-2) - - -@pytest.mark.skipif( - shutil.which("cc") is None or shutil.which("gfortran") is None, - reason="requires C and Fortran compilers", -) -def test_collision_adapter_all_builds_a_fortran_bind_c_module_without_an_adapter(tmp_path: Path): - result = build_fortran_extension( - _bind_c_source(tmp_path), - collision_adapter_all=True, - output_dir=tmp_path / "bind_c", - output_name="bind_c_collision", - ) - module = sole_native_module(result.import_module()) - - assert module.scaled(np.float64(3.0)) == np.float64(6.0) - assert not any(path.name.endswith("_adapters.c") for path in result.generated_sources) + assert any(path.name.endswith("_adapters.c") for path in replay.generated_sources) @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") @@ -153,27 +110,6 @@ def test_an_unknown_collision_adapter_name_fails_before_wrapper_planning(tmp_pat ) -@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_build_manifest_replay_retains_the_selected_collision_adapter(tmp_path: Path): - generated = build_pyi_extension( - _contract(tmp_path), - native_language="c", - native_c_sources=[_native_source(tmp_path)], - collision_adapters=["Py_Initialize"], - makefile=True, - output_dir=tmp_path / "replay", - output_name="collision_replay", - ) - - assert generated.build_manifest is not None - assert generated.manifest["extension"]["collision_adapters"] == ["Py_Initialize"] - replay = build_pyi_extension_from_manifest(generated.build_manifest) - module = sole_native_module(replay.import_module()) - - assert module.Py_Initialize(np.int64(5)) == np.int64(12) - assert any(path.name.endswith("_adapters.c") for path in replay.generated_sources) - - @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") def test_cli_selected_collision_adapter_builds_an_importable_extension(tmp_path: Path): output_dir = tmp_path / "cli" diff --git a/tests/docs/_structure_support.py b/tests/docs/_structure_support.py index f215c63af..595d0cce0 100644 --- a/tests/docs/_structure_support.py +++ b/tests/docs/_structure_support.py @@ -26,24 +26,6 @@ "not-yet-implemented", "planned-documentation", } -CLI_HELP_GROUP_HEADINGS = [ - "commands:", - "positional arguments:", - "input selection:", - "input options:", - "generation modes:", - "compiler and preprocessing options:", - "preprocessing options:", - "C include options:", - "report options:", - "compiler options:", - "wrapper options:", - "native options:", - "probe options:", - "execution options:", - "output options:", - "diagnostic options:", -] CLI_REFERENCE_OPTIONS = [ "paths", "--help-build", @@ -85,7 +67,6 @@ "--no-color", "--debug", ] -CLI_VISIBLE_HELP_OPTIONS = CLI_REFERENCE_OPTIONS FEATURE_MATRIX_STATUSES = { "Supported", "Partially supported", diff --git a/tests/docs/test_examples.py b/tests/docs/test_examples.py index 7fc3ccae9..dab8aa9ea 100644 --- a/tests/docs/test_examples.py +++ b/tests/docs/test_examples.py @@ -4,6 +4,7 @@ import ast from dataclasses import dataclass +import difflib import os import platform from pathlib import Path @@ -415,47 +416,75 @@ def test_documentation_has_automatically_verified_examples(): assert DOCUMENTED_SOURCES, "mark displayed fixture inputs with prik-doc-source" -@pytest.mark.parametrize("source", DOCUMENTED_SOURCES, ids=lambda source: source.test_id) -def test_documented_source_input(source: DocumentedSource): - assert source.source_path.is_file(), f"{source.test_id}: documented source does not exist: {source.source_path}" +def _documented_source_mismatch(source: DocumentedSource) -> str | None: + if not source.source_path.is_file(): + return f"{source.test_id}: documented source does not exist: {source.source_path}" file_text = source.source_path.read_text(encoding="utf-8") expected_text = file_text if source.selector is not None: tree = ast.parse(file_text, filename=str(source.source_path)) selected = [node for node in tree.body if isinstance(node, ast.FunctionDef) and node.name == source.selector] - assert len(selected) == 1, f"{source.test_id}: source selector {source.selector!r} did not name one function" + if len(selected) != 1: + return f"{source.test_id}: source selector {source.selector!r} did not name one function" function = selected[0] first_line = min((function.lineno, *(decorator.lineno for decorator in function.decorator_list))) expected_text = "\n".join(file_text.splitlines()[first_line - 1 : function.end_lineno]) - assert source.source_text.rstrip("\n") == expected_text.rstrip("\n") + if source.source_text.rstrip("\n") == expected_text.rstrip("\n"): + return None + diff = difflib.unified_diff( + expected_text.rstrip("\n").splitlines(), + source.source_text.rstrip("\n").splitlines(), + fromfile=str(source.source_path.relative_to(ROOT)), + tofile=source.test_id, + lineterm="", + ) + return f"{source.test_id}: displayed source differs from its fixture\n" + "\n".join(diff) -@pytest.mark.parametrize("block", DOCUMENTED_PYTHON_BLOCKS, ids=lambda block: block.test_id) -def test_documented_python_block_is_valid(block: DocumentedPythonBlock): - """Keep Python examples parseable and semantic contract examples loadable.""" - ast.parse(block.source, filename=block.test_id) +def test_documented_source_inputs_match_their_fixtures(): + mismatches = [message for source in DOCUMENTED_SOURCES if (message := _documented_source_mismatch(source))] + assert not mismatches, "\n\n".join(mismatches) + + +def _python_block_problem(block: DocumentedPythonBlock) -> str | None: + try: + ast.parse(block.source, filename=block.test_id) + except SyntaxError as error: + return f"{block.test_id}: invalid Python: {error}" if "from prik.contracts import" not in block.source: - assert not block.expects_contract_error, ( - f"{block.test_id}: prik-doc-contract: invalid marks a block that loads no contract" - ) - return + if block.expects_contract_error: + return f"{block.test_id}: prik-doc-contract: invalid marks a block that loads no contract" + return None + try: + pyi_text_to_semantic_module(block.source, module_name="documentation_example") + except ValueError as error: + if block.expects_contract_error: + return None + return f"{block.test_id}: contract does not load: {error}" if block.expects_contract_error: - with pytest.raises(ValueError): - pyi_text_to_semantic_module(block.source, module_name="documentation_example") - return - pyi_text_to_semantic_module(block.source, module_name="documentation_example") + return f"{block.test_id}: contract marked prik-doc-contract: invalid loads without error" + return None -@pytest.mark.parametrize("path", DOC_PATHS, ids=lambda path: str(path.relative_to(ROOT))) -def test_documented_expected_output_labels_are_automatically_verified(path: Path): - lines = path.read_text(encoding="utf-8").splitlines() - for index, line in enumerate(lines): - if line.strip() not in {"Expected output:", "Output:"}: - continue - marker_index = _next_nonempty_line(lines, index + 1) - assert marker_index < len(lines) and OUTPUT_MARKER.match(lines[marker_index]), ( - f"{path.relative_to(ROOT)}:{index + 1}: documented output must use prik-doc-test-output" - ) +def test_documented_python_blocks_are_valid(): + """Keep Python examples parseable and semantic contract examples loadable.""" + problems = [message for block in DOCUMENTED_PYTHON_BLOCKS if (message := _python_block_problem(block))] + assert not problems, "\n".join(problems) + + +def test_documented_expected_output_is_automatically_verified(): + problems = [] + for path in DOC_PATHS: + lines = path.read_text(encoding="utf-8").splitlines() + for index, line in enumerate(lines): + if line.strip() not in {"Expected output:", "Output:"}: + continue + marker_index = _next_nonempty_line(lines, index + 1) + if marker_index >= len(lines) or not OUTPUT_MARKER.match(lines[marker_index]): + problems.append( + f"{path.relative_to(ROOT)}:{index + 1}: documented output must use prik-doc-test-output" + ) + assert not problems, "\n".join(problems) @pytest.mark.parametrize("example", DOCUMENTATION_EXAMPLES, ids=lambda example: example.test_id) diff --git a/tests/docs/test_metadata_and_visibility.py b/tests/docs/test_metadata_and_visibility.py index 4f006d26f..4d5de8209 100644 --- a/tests/docs/test_metadata_and_visibility.py +++ b/tests/docs/test_metadata_and_visibility.py @@ -1,29 +1,45 @@ """Documentation metadata and visibility contracts.""" -from pathlib import Path - -import pytest - from tests.docs._structure_support import ( ALLOWED_PUBLICATION_STATES, ALLOWED_STATUSES, DOC_PATHS, + DOCS_ROOT, REQUIRED_METADATA, ROOT, _front_matter, ) -@pytest.mark.parametrize("path", DOC_PATHS, ids=lambda path: str(path.relative_to(ROOT))) -def test_documentation_page_metadata(path: Path) -> None: - metadata, _ = _front_matter(path) - missing = REQUIRED_METADATA - metadata.keys() - assert not missing, f"{path.relative_to(ROOT)}: missing metadata fields: {sorted(missing)}" +LANE_AUDIENCE_TERMS = { + "user": ("users",), + "developer": ("developers", "maintainers", "contributors"), +} + + +def test_documentation_pages_declare_valid_publication_metadata() -> None: + problems: list[str] = [] + for path in DOC_PATHS: + page = path.relative_to(ROOT) + metadata, _ = _front_matter(path) + missing = REQUIRED_METADATA - metadata.keys() + if missing: + problems.append(f"{page}: missing metadata fields: {sorted(missing)}") + problems.extend( + f"{page}: metadata field {key!r} is empty" + for key in sorted(REQUIRED_METADATA) + if key in metadata and not metadata[key] + ) + if metadata.get("status") not in ALLOWED_STATUSES: + problems.append(f"{page}: unknown status {metadata.get('status')!r}") + if metadata.get("publication") not in ALLOWED_PUBLICATION_STATES: + problems.append(f"{page}: unknown publication state {metadata.get('publication')!r}") - for key in REQUIRED_METADATA: - assert metadata[key], f"{path.relative_to(ROOT)}: metadata field {key!r} is empty" + lane = path.relative_to(DOCS_ROOT).parts[0] + audience = metadata.get("audience", "") + if lane in LANE_AUDIENCE_TERMS and not any(term in audience for term in LANE_AUDIENCE_TERMS[lane]): + problems.append(f"{page}: audience {audience!r} does not name the {lane} lane's readers") + if lane == "user" and "maintainers" in audience: + problems.append(f"{page}: user-lane page addresses maintainers") - assert metadata["status"] in ALLOWED_STATUSES, f"{path.relative_to(ROOT)}: unknown status {metadata['status']!r}" - assert metadata["publication"] in ALLOWED_PUBLICATION_STATES, ( - f"{path.relative_to(ROOT)}: unknown publication state {metadata['publication']!r}" - ) + assert not problems, "\n".join(problems) diff --git a/tests/docs/test_navigation.py b/tests/docs/test_navigation.py index 159f8ee98..e455e694b 100644 --- a/tests/docs/test_navigation.py +++ b/tests/docs/test_navigation.py @@ -1,23 +1,6 @@ """Published documentation-lane contracts.""" -import pytest - -from tests.docs._structure_support import DOCS_ROOT, ROOT, _front_matter - - -@pytest.mark.parametrize( - ("lane", "audience_terms"), - [ - ("user", ("users",)), - ("developer", ("developers", "maintainers", "contributors")), - ], -) -def test_documentation_lane_has_consistent_audience(lane: str, audience_terms: tuple[str, ...]) -> None: - for path in (DOCS_ROOT / lane).rglob("*.md"): - metadata, _ = _front_matter(path) - assert any(term in metadata["audience"] for term in audience_terms) - if lane == "user": - assert "maintainers" not in metadata["audience"] +from tests.docs._structure_support import ROOT def test_site_navigation_exposes_user_and_contributor_indexes() -> None: diff --git a/tests/docs/test_publication.py b/tests/docs/test_publication.py index 4e2745c36..44ae3e94c 100644 --- a/tests/docs/test_publication.py +++ b/tests/docs/test_publication.py @@ -117,11 +117,14 @@ def test_example_notebooks_are_served_from_the_site(tmp_path: Path) -> None: """A download button needs the notebook same-origin, not on GitHub. The browser honours ``download`` only for a same-origin file, so the site - serves a copy while the repository keeps the single source of truth. + serves a copy while the repository keeps the single source of truth. A + checkout without the notebook directory publishes none. """ docs_dir = tmp_path / "docs" notebook_dir = tmp_path / mkdocs_publication._EXAMPLE_NOTEBOOK_DIR docs_dir.mkdir() + assert mkdocs_publication._example_notebook_paths({"docs_dir": str(docs_dir)}) == [] + notebook_dir.mkdir(parents=True) (notebook_dir / "quickstart.ipynb").write_text("{}", encoding="utf-8") (notebook_dir / "notes.txt").write_text("not a notebook", encoding="utf-8") @@ -129,10 +132,3 @@ def test_example_notebooks_are_served_from_the_site(tmp_path: Path) -> None: published = mkdocs_publication._example_notebook_paths({"docs_dir": str(docs_dir)}) assert [path.name for path in published] == ["quickstart.ipynb"] - - -def test_publishing_example_notebooks_tolerates_a_missing_directory(tmp_path: Path) -> None: - docs_dir = tmp_path / "docs" - docs_dir.mkdir() - - assert mkdocs_publication._example_notebook_paths({"docs_dir": str(docs_dir)}) == [] diff --git a/tests/docs/test_reference_and_codebase_map.py b/tests/docs/test_reference_and_codebase_map.py index 1ee756809..dd7ac7d87 100644 --- a/tests/docs/test_reference_and_codebase_map.py +++ b/tests/docs/test_reference_and_codebase_map.py @@ -7,10 +7,8 @@ import prik from tests.docs._structure_support import ( - CLI_HELP_GROUP_HEADINGS, CLI_REFERENCE_OPTIONS, CLI_REFERENCE_PATH, - CLI_VISIBLE_HELP_OPTIONS, DOCS_ROOT, FEATURE_MATRIX_PATH, FEATURE_MATRIX_ROWS, @@ -26,56 +24,49 @@ DOCUMENTATION_PATH_REFERENCE = re.compile(r"`(docs/[^`]+\.md)`") -@pytest.mark.parametrize("heading", CLI_HELP_GROUP_HEADINGS) -def test_cli_help_uses_documented_option_groups(heading: str) -> None: - assert heading in _prik_cli_help() +def test_cli_reference_and_help_expose_every_public_option() -> None: + reference = CLI_REFERENCE_PATH.read_text(encoding="utf-8") + help_text = _prik_cli_help() + undocumented = [option for option in CLI_REFERENCE_OPTIONS if option not in reference] + hidden = [option for option in CLI_REFERENCE_OPTIONS if option not in help_text] + assert not undocumented, f"{CLI_REFERENCE_PATH.relative_to(ROOT)} does not document: {undocumented}" + assert not hidden, f"`python -m prik --help` output does not show: {hidden}" -@pytest.mark.parametrize("option", CLI_REFERENCE_OPTIONS) -def test_cli_reference_documents_public_option(option: str) -> None: - content = CLI_REFERENCE_PATH.read_text(encoding="utf-8") - assert option in content - -@pytest.mark.parametrize("option", CLI_VISIBLE_HELP_OPTIONS) -def test_cli_help_exposes_documented_public_option(option: str) -> None: - assert option in _prik_cli_help() - - -@pytest.mark.parametrize("name", sorted(prik.__all__)) -def test_python_api_reference_documents_public_export(name: str) -> None: +def test_python_api_reference_documents_every_public_export() -> None: content = PYTHON_API_REFERENCE_PATH.read_text(encoding="utf-8") - assert f"`{name}`" in content + missing = [name for name in sorted(prik.__all__) if f"`{name}`" not in content] + assert not missing, f"{PYTHON_API_REFERENCE_PATH.relative_to(ROOT)} does not document: {missing}" -def test_feature_matrix_has_rows() -> None: - assert FEATURE_MATRIX_ROWS - - -@pytest.mark.parametrize("row", FEATURE_MATRIX_ROWS, ids=lambda row: row["Feature"]) -def test_feature_matrix_support_claim_is_complete(row: dict[str, str]) -> None: - assert row["Status"] in FEATURE_MATRIX_STATUSES - for column in ["Feature", "Status", "User docs", "Evidence", "Limitations"]: - assert row[column] +def _feature_matrix_row_problems(row: dict[str, str]) -> list[str]: + feature = row["Feature"] or "" + problems = [f"{feature}: {column} is empty" for column in row if not row[column]] + if row["Status"] not in FEATURE_MATRIX_STATUSES: + problems.append(f"{feature}: unknown status {row['Status']!r}") for column in ["User docs", "Evidence"]: - assert MARKDOWN_LINK.search(row[column]), f"{row['Feature']}: {column} must contain a Markdown link" + targets = MARKDOWN_LINK.findall(row[column]) + if not targets: + problems.append(f"{feature}: {column} must contain a Markdown link") + for target in targets: + if target.startswith(("http://", "https://")): + continue + if not (FEATURE_MATRIX_PATH.parent / target).resolve().exists(): + problems.append(f"{feature}: {column} link target does not exist: {target}") if row["Status"] in {"Supported", "Partially supported"}: evidence_targets = [ (FEATURE_MATRIX_PATH.parent / target).resolve() for target in MARKDOWN_LINK.findall(row["Evidence"]) ] - assert any(target.is_relative_to(ROOT / "tests") for target in evidence_targets), ( - f"{row['Feature']}: support claims need direct test evidence" - ) + if not any(target.is_relative_to(ROOT / "tests") for target in evidence_targets): + problems.append(f"{feature}: support claims need direct test evidence") + return problems -@pytest.mark.parametrize("row", FEATURE_MATRIX_ROWS, ids=lambda row: row["Feature"]) -def test_feature_matrix_links_point_to_existing_files(row: dict[str, str]) -> None: - for column in ["User docs", "Evidence"]: - for target in MARKDOWN_LINK.findall(row[column]): - if target.startswith(("http://", "https://")): - continue - resolved_target = (FEATURE_MATRIX_PATH.parent / target).resolve() - assert resolved_target.exists(), f"{row['Feature']}: {column} link target does not exist: {target}" +def test_feature_matrix_support_claims_are_complete_and_linked() -> None: + assert FEATURE_MATRIX_ROWS, f"{FEATURE_MATRIX_PATH.relative_to(ROOT)}: feature matrix table has no rows" + problems = [problem for row in FEATURE_MATRIX_ROWS for problem in _feature_matrix_row_problems(row)] + assert not problems, "\n".join(problems) REVIEWED_CONTRIBUTOR_MAPS = [ diff --git a/tests/fortran/CONTRACT_COVERAGE.md b/tests/fortran/CONTRACT_COVERAGE.md index b23d921ea..404cda425 100644 --- a/tests/fortran/CONTRACT_COVERAGE.md +++ b/tests/fortran/CONTRACT_COVERAGE.md @@ -33,178 +33,178 @@ Authoritative sources: | Documentation contract | Status | Dimensions | Stage evidence | Runtime evidence | Negative evidence | CI lane | | --- | --- | --- | --- | --- | --- | --- | -| [`.pyi` Functions And Classes: Declare Native Destruction](../../docs/user/reference/pyi-contracts/functions-and-classes.md#declare-native-destruction) | Supported | language-neutral `@destroy` lifecycle role; excluded from callable methods; native name by default with optional `@bind`; one generated declaration per Fortran final procedure; old tuple metadata rejected | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_destroy_is_language_neutral_lifecycle_metadata`
`tests/fortran/derived_types/semantics/test_fortran_derived_semantics.py::test_derived_type_initializers_and_finalizers_reach_semantic_ir`
`tests/fortran/derived_types/policy/test_derived_accessor_policy.py::test_native_destructor_reaches_completed_derived_type_policy` | `tests/fortran/derived_types/end_to_end/test_default_constructors_and_finalizers.py::test_fortran_default_constructor_keywords_and_finalization[source]`
`tests/fortran/derived_types/end_to_end/test_default_constructors_and_finalizers.py::test_fortran_default_constructor_keywords_and_finalization[generated-pyi]` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_invalid_destroy_contracts_are_rejected` (`semantics`) | canonical | +| [`.pyi` Functions And Classes: Declare Native Destruction](../../docs/user/reference/pyi-contracts/functions-and-classes.md#declare-native-destruction) | Supported | language-neutral `@destroy` lifecycle role; excluded from callable methods; native name by default with optional `@bind`; one generated declaration per Fortran final procedure; old tuple metadata rejected | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_destroy_is_language_neutral_lifecycle_metadata` | `tests/fortran/derived_types/end_to_end/test_default_constructors_and_finalizers.py::test_fortran_default_constructor_keywords_and_finalization[source]`
`tests/fortran/derived_types/end_to_end/test_default_constructors_and_finalizers.py::test_fortran_default_constructor_keywords_and_finalization[generated-pyi]` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_invalid_destroy_contracts_are_rejected` (`semantics`) | canonical | | [Python API: Advanced Package Imports](../../docs/user/reference/python-api.md#advanced-package-imports) | Supported | public string, file, path-sequence, and project parser entry points; model traversal; stable source diagnostics | `tests/fortran/infrastructure/parsing/test_public_entrypoints.py::test_parser_public_entrypoint_aliases_and_singular_contracts_use_inline_sources` | — | — | canonical | | [CLI Commands: Compiler Preprocessing](../../docs/user/reference/cli-commands.md#compiler-preprocessing) | Supported | explicit compiler; include directories; macros; standard; compiler arguments; exact preprocessing recipe | `tests/fortran/infrastructure/preprocessing/test_configuration_and_adapters.py::test_direct_fortran_preprocess_invocation_uses_exact_compiler_and_cpp` | — | — | canonical | -| [CLI Commands: Parse And Semantics](../../docs/user/reference/cli-commands.md#parse-and-semantics) | Supported | public parser module and top-level command modes; parse, semantics, `.pyi`, and diagnostic dispatch | `tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py::test_fortran_parser_main_public_api_modes_from_inline_source` | — | — | canonical | +| [CLI Commands: Parse And Semantics](../../docs/user/reference/cli-commands.md#parse-and-semantics) | Supported | public parser module and top-level command modes; parse, semantics, `.pyi`, and diagnostic dispatch | `tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py::test_fortran_parser_module_launcher_reports_every_mode` | — | — | canonical | | [Semantics Stage: Inputs And Shared Representation](../../docs/developer/packages/semantics.md#inputs-and-shared-representation) | Supported | deterministic source-to-IR conversion; preserved wrapper-relevant facts; checked fixture serialization | `tests/fortran/infrastructure/semantic_ir/semantics/test_fortran_conversion_properties.py::test_generated_fortran_ast_to_semantic_ir_is_deterministic` | — | — | canonical | -| [Data Types: Example](../../docs/user/guide/data-types.md#example) | Supported | source generation; reviewed generated `.pyi`; source build; generated-`.pyi` replay | `tests/fortran/data_types/pipeline/test_generated_scalar_contract.py::test_generated_primitive_scalar_contract_matches_reviewed_package` | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]`
`tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[generated-pyi]` | — | canonical | -| [Data Types: Calling from Python](../../docs/user/guide/data-types.md#calling-from-python) | Supported | signed integer; real; complex; Boolean; exact visible values and scalar result types | `tests/fortran/data_types/codegen/test_primitive_scalar_result_lowering.py::test_direct_scalar_results_preserve_numpy_types_with_python_bool_as_the_exception[Complex128-NPY_COMPLEX128-numpy]` | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]` | — | canonical | +| [Data Types: Example](../../docs/user/guide/data-types.md#example) | Supported | source generation; reviewed generated `.pyi`; source build; generated-`.pyi` replay | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[generated-pyi]` | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]`
`tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[generated-pyi]` | — | canonical | +| [Data Types: Calling from Python](../../docs/user/guide/data-types.md#calling-from-python) | Supported | signed integer; real; complex; Boolean; exact visible values and scalar result types | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]` | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]` | — | canonical | | [Data Types: Scalar Type Mapping](../../docs/user/guide/data-types.md#scalar-type-mapping) | Supported | `Bool`/`Bool8/16/32/64`; `Int8/16/32/64`; `Float32/64`; `Complex64/128`; compiler-probed intrinsic, ISO environment, and ISO C kinds | `tests/fortran/data_types/semantics/test_fortran_scalar_semantics.py::test_intrinsic_builtin_kinds_map_to_semantic_types`
`tests/fortran/data_types/probes/test_fortran_type_probes.py::test_fortran_type_probe_evaluates_collected_semantic_requirements`
`tests/fortran/data_types/probes/test_fortran_type_probes.py::test_fortran_type_probe_resolves_supported_logical_storage_widths` | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]` | — | canonical | | [Data Types: Runtime Default Constructors](../../docs/user/guide/data-types.md#runtime-default-constructors) | Supported | Boolean; fixed-width signed/unsigned integer, real, and complex; `SizeT`; zero values | `tests/fortran/data_types/runtime/test_contract_scalar_constructors.py::test_concrete_primitive_default_constructors_return_zero_numpy_scalars` | `tests/fortran/data_types/runtime/test_contract_scalar_constructors.py::test_concrete_primitive_default_constructors_return_zero_numpy_scalars` | `tests/fortran/data_types/runtime/test_contract_scalar_constructors.py::test_primitive_contract_constructors_reject_values_and_array_annotations` (`runtime`) | canonical | -| [Data Types: Important Rules](../../docs/user/guide/data-types.md#important-rules) | Supported | exact NumPy scalar acceptance; Python `int`/`float` rejection; wrong NumPy dtype rejection; compiler-resolved kinds | `tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py::test_scalar_input_registry_lowers_completed_type_into_the_native_support_api[Int32-int32_t-NPY_INT32]` | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]` | — | canonical | -| [Data Types: Values And Native Storage](../../docs/user/guide/data-types.md#values-and-native-storage) | Supported | bare scalar values; immutable replacement; `Int32[()]`; `Float64[()]`; direct and hidden rank-zero results; wrong dtype and read-only rejection | `tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py::test_scalar_input_registry_lowers_completed_type_into_the_native_support_api[Float64-double-NPY_FLOAT64]` | `tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py::test_scalar_values_and_rank_zero_storage_cross_the_native_boundary` | — | canonical | +| [Data Types: Important Rules](../../docs/user/guide/data-types.md#important-rules) | Supported | exact NumPy scalar acceptance; Python `int`/`float` rejection; wrong NumPy dtype rejection; compiler-resolved kinds | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]` | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]` | — | canonical | +| [Data Types: Values And Native Storage](../../docs/user/guide/data-types.md#values-and-native-storage) | Supported | bare scalar values; immutable replacement; `Int32[()]`; `Float64[()]`; direct and hidden rank-zero results; wrong dtype and read-only rejection | `tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py::test_scalar_kind_coverage_uses_compiler_probed_wrapper_types[source]` | `tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py::test_scalar_values_and_rank_zero_storage_cross_the_native_boundary` | — | canonical | | [Data Types: Unsupported Widths And Forms](../../docs/user/guide/data-types.md#unsupported-widths-and-forms) | Blocked | real or complex mantissa wider than target C `long double`; logical storage outside 8/16/32/64 bits; unknown compiler-probed widths | — | — | `tests/fortran/data_types/semantics/test_fortran_scalar_semantics.py::test_unsupported_intrinsic_widths_fail_in_semantic_conversion` (`semantics`)
`tests/fortran/data_types/semantics/test_fortran_scalar_semantics.py::test_compiler_probed_unknown_storage_widths_fail_in_semantic_conversion` (`semantics`) | canonical | -| [Arrays: Complete Example](../../docs/user/guide/arrays.md#complete-example) | Supported | source and generated-contract builds; reviewed generated `.pyi`; visible extents; lower bound; assumed size; positive strides; automatic result; optional and no-`intent` storage | `tests/fortran/arrays/parsing/test_array_declarations_and_shapes.py::test_dimension_attribute_with_mixed_bounds_is_parsed`
`tests/fortran/arrays/semantics/test_array_semantics.py::test_explicit_bound_ranges_remain_shaped_storage_contracts`
`tests/fortran/arrays/pipeline/test_generated_array_contracts.py::test_array_generated_pyi_contract_matches_fixture[array_ops]` | `tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[source]`
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[generated-pyi]` | — | canonical | +| [Arrays: Complete Example](../../docs/user/guide/arrays.md#complete-example) | Supported | source and generated-contract builds; reviewed generated `.pyi`; visible extents; lower bound; assumed size; positive strides; automatic result; optional and no-`intent` storage | `tests/fortran/arrays/parsing/test_array_declarations_and_shapes.py::test_array_bounds_split_into_lower_and_upper[dimension-attribute-mixed-bounds]`
`tests/fortran/arrays/semantics/test_array_semantics.py::test_fortran_array_declarations_complete_category_shape_and_order` | `tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[source]`
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[generated-pyi]` | — | canonical | | [Arrays: Python Usage](../../docs/user/guide/arrays.md#python-usage) | Supported | `float64`; Fortran-order mutation; lower-bound normalization; flattened rank; returned NumPy storage | — | `tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[source]`
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[generated-pyi]` | — | canonical | -| [Arrays: What PRIK Validates](../../docs/user/guide/arrays.md#what-prik-validates) | Supported | exact dtype; rank; shape; layout; contiguity; alignment; byte order; writeability; positive strides; broadcasting and reversal rejection; zero size | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_dense_array_lowering_uses_planned_shape_checks_and_bridge_orientation`
`tests/fortran/arrays/codegen/test_strided_array_lowering.py::test_strided_array_lowering_validates_and_passes_one_explicit_bridge_slice` | `tests/fortran/arrays/end_to_end/test_array_contract_validation.py::test_remaining_array_contracts_are_validated_before_fortran_calls[source]` | `tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_rank2_assumed_shape_rejects_non_positive_strides[source]` (`runtime`)
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[source]` (`runtime`)
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[generated-pyi]` (`runtime`) | canonical | -| [Arrays: Layout Fortran First](../../docs/user/guide/arrays.md#layout-fortran-first) | Supported | dense rank two and three; explicit and assumed shape; Fortran contiguity | `tests/fortran/arrays/semantics/test_array_semantics.py::test_matrix_semantics` | `tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_rank2_contiguous_contract_requires_fortran_contiguous[source]` | `tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_rank3_contiguous_contract_requires_fortran_contiguous[source]` (`runtime`) | canonical | -| [Arrays: C-order Arrays](../../docs/user/guide/arrays.md#c-order-arrays) | Supported | edited semantic `.pyi`; direct `ORDER_C`; `ORDER_C` plus `COPY_F`; caller-visible copyback semantics | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_copy_f_is_one_binding_owned_transformation_lifecycle` | `tests/fortran/arrays/end_to_end/test_edited_pyi_layout_contract.py::test_edited_pyi_selects_direct_c_storage_or_fortran_copy_semantics` | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_copy_f_layer_edit_fails_central_validation` (`codegen`) | canonical | -| [Arrays: Flat Storage](../../docs/user/guide/arrays.md#flat-storage) | Supported | ranks 1-15; order-dependent contiguous storage; flat final axis with checked prefix; zero-sized storage | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_dense_array_plan_records_extent_dependencies_flat_storage_and_order` | `tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[source]`
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[generated-pyi]` | `tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[source]` (`runtime`)
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[generated-pyi]` (`runtime`) | canonical | -| [Arrays: Strided Views](../../docs/user/guide/arrays.md#strided-views) | Supported | dense, forward and reversed rank two and three input/output; descriptor and sectioned-address handoff | `tests/fortran/arrays/codegen/test_strided_array_lowering.py::test_strided_array_plan_selects_one_descriptor_without_parallel_stride_roles` | `tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_rank2_assumed_shape_accepts_fortran_ordered_strided_views[source]`
`tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_rank3_assumed_shape_accepts_fortran_ordered_strided_views[source]` | `tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_rank2_assumed_shape_accepts_reversed_axes_and_refuses_what_is_not_a_section[source]` (`runtime`) | canonical | -| [Arrays: Mutation and Results](../../docs/user/guide/arrays.md#mutation-and-results) | Supported | input; caller output; in-place `intent(inout)`; no-`intent`; immutable replacement; optional presence; fixed, automatic, rank 1-15, and zero-size results; exact-kind Boolean copy-in/copy-out | `tests/fortran/arrays/codegen/test_array_result_lowering.py::test_array_results_record_producer_shape_copy_ownership_and_shared_hidden_slot`
`tests/fortran/arrays/codegen/test_array_output_identity.py::test_projected_array_identity_uses_one_completed_in_place_copy_out_action` | `tests/fortran/arrays/end_to_end/test_array_results.py::test_array_results_follow_data_buffer_and_descriptor_handle_contracts[source]`
`tests/fortran/arrays/end_to_end/test_edited_pyi_layout_contract.py::test_edited_pyi_selects_direct_c_storage_or_fortran_copy_semantics`
`tests/fortran/arrays/end_to_end/test_logical_kind_array_conversions.py::test_boolean_arrays_copy_only_in_required_directions_for_every_supported_width` | — | canonical | -| [Arrays: Common Array Contracts](../../docs/user/guide/arrays.md#common-array-contracts) | Supported | every primitive dtype at concrete ranks 1-15; rank zero separately; assumed rank 1-15; dense, C-order, copy, flat, strided, fixed/open extents | `tests/fortran/arrays/semantics/test_array_semantics.py::test_array_constraints`
`tests/fortran/arrays/codegen/test_array_buffer_lowering.py::test_required_array_buffer_has_one_printable_editable_handoff_plan` | `tests/fortran/arrays/end_to_end/test_primitive_dtype_rank_matrix.py::test_every_primitive_dtype_at_every_concrete_rank_mutates_exact_storage`
`tests/fortran/arrays/end_to_end/test_assumed_rank_arrays.py::test_assumed_rank_arguments_dispatch_to_runtime_rank[source]`
`tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py::test_scalar_values_and_rank_zero_storage_cross_the_native_boundary` | `tests/fortran/arrays/end_to_end/test_assumed_rank_arrays.py::test_assumed_rank_arguments_dispatch_to_runtime_rank[source]` (`runtime`) | canonical | -| [Strings: Choose A String Boundary](../../docs/user/guide/strings.md#choose-a-string-boundary) | Supported | runtime-length `String`; fixed scalar value/replacement/discard; rank-zero `S8`; rank-one `S8`; raw address deferred to Raw Addresses | `tests/fortran/strings/semantics/test_string_pyi_semantics.py::test_string_length_and_shape_axes_round_trip`
`tests/fortran/strings/policy/test_string_wrapper_policy.py::test_wrapper_policy_completes_fixed_string_replacement_and_discarded_identity` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays`
`tests/fortran/strings/end_to_end/test_character_edge_cases.py::test_fixed_string_replacement_and_identity_use_canonical_plan` | — | canonical | -| [Strings: Complete Example](../../docs/user/guide/strings.md#complete-example) | Supported | source compilation; edited semantic `.pyi`; fixed scalar input/result; mutable scalar storage; fixed-width array input/output/inout | `tests/fortran/strings/pipeline/test_generated_string_contracts.py::test_string_generated_pyi_contract_matches_fixture[fstrings_f90]` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` | — | canonical | -| [Strings: Immutable Values](../../docs/user/guide/strings.md#immutable-values) | Supported | exact encoded width; trailing blanks; replacement identity; discarded mutation; hidden and direct results; allocation failure | `tests/fortran/strings/codegen/test_fixed_string_writeback.py::test_fixed_replacement_projects_completed_argument_and_lifecycle_facts`
`tests/fortran/strings/codegen/test_fixed_string_result_lowering.py::test_fixed_strings_reuse_ordered_result_plans_with_completed_length_and_copy_facts` | `tests/fortran/strings/end_to_end/test_character_edge_cases.py::test_fortran_character_edge_cases_follow_copy_in_copy_out_policy[source]` | `tests/fortran/strings/end_to_end/test_character_edge_cases.py::test_fortran_character_edge_cases_follow_copy_in_copy_out_policy[source]` (`runtime`) | canonical | -| [Strings: Mutable Scalar Storage](../../docs/user/guide/strings.md#mutable-scalar-storage) | Supported | rank-zero `S8`; same-object mutation; bytes result; exact itemsize/rank/writeability | `tests/fortran/strings/semantics/test_string_pyi_semantics.py::test_rank_zero_string_storage_round_trips_as_empty_tuple_array` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` (`runtime`) | canonical | -| [Strings: String Arrays](../../docs/user/guide/strings.md#string-arrays) | Supported | fixed itemsize; input and in-place mutation; fixed array result; rank/shape/dtype/writeability; zero size | `tests/fortran/strings/codegen/test_character_array_lowering.py::test_fixed_width_character_array_results_reuse_the_ordinary_array_copy_plan` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays`
`tests/fortran/strings/end_to_end/test_character_boundaries.py::test_modern_fortran_character_arguments_and_results[source]` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` (`runtime`) | canonical | -| [Strings: Length And Encoding](../../docs/user/guide/strings.md#length-and-encoding) | Supported | length 1, representative width 8, runtime length, Unicode UTF-8 byte length, blanks, empty values, embedded NUL rejection, conservative no-`intent`, ambiguous mutable deferred scalar rejection | `tests/fortran/strings/parsing/test_character_length_parsing.py::test_character_entity_lengths_and_assumed_bounds_are_preserved`
`tests/fortran/strings/codegen/test_string_input_lowering.py::test_required_string_values_reuse_argument_plan_with_character_handoff_facts` | `tests/fortran/strings/end_to_end/test_character_boundaries.py::test_modern_fortran_character_arguments_and_results[source]` | `tests/fortran/strings/semantics/test_string_pyi_semantics.py::test_bare_string_slice_is_rejected_as_ambiguous` (`semantics`)
`tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` (`runtime`) | canonical | -| [Wrapping Functions: Basic Scalar Function](../../docs/user/guide/wrapping-functions.md#basic-scalar-function) | Supported | direct scalar result; exact NumPy inputs; visible value | `tests/fortran/functions/semantics/test_fortran_function_semantics.py::test_function_result` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` (`runtime`) | canonical | +| [Arrays: What PRIK Validates](../../docs/user/guide/arrays.md#what-prik-validates) | Supported | exact dtype; rank; shape; layout; contiguity; alignment; byte order; writeability; positive strides; broadcasting and reversal rejection; zero size | `tests/fortran/arrays/codegen/test_specification_extent_check_lowering.py::test_the_binding_rejects_before_any_other_post_call_step` | `tests/fortran/arrays/end_to_end/test_array_contract_validation.py::test_remaining_array_contracts_are_validated_before_fortran_calls[source]` | `tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py::test_storage_that_is_not_a_described_section_stays_refused` (`runtime`)
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[source]` (`runtime`)
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[generated-pyi]` (`runtime`) | canonical | +| [Arrays: Layout Fortran First](../../docs/user/guide/arrays.md#layout-fortran-first) | Supported | dense rank two and three; explicit and assumed shape; Fortran contiguity | `tests/fortran/arrays/semantics/test_array_semantics.py::test_fortran_array_declarations_complete_category_shape_and_order` | `tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_rank2_contiguous_contract_requires_fortran_contiguous[source]` | `tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_rank2_explicit_shape_requires_fortran_contiguous[source]` (`runtime`) | canonical | +| [Arrays: C-order Arrays](../../docs/user/guide/arrays.md#c-order-arrays) | Supported | edited semantic `.pyi`; direct `ORDER_C`; `ORDER_C` plus `COPY_F`; caller-visible copyback semantics | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_copy_f_status_cleanup_clears_the_released_temporary_before_error_cleanup` | `tests/fortran/arrays/end_to_end/test_edited_pyi_layout_contract.py::test_edited_pyi_selects_direct_c_storage_or_fortran_copy_semantics` | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_copy_f_layer_edit_fails_central_validation` (`codegen`) | canonical | +| [Arrays: Flat Storage](../../docs/user/guide/arrays.md#flat-storage) | Supported | ranks 1-15; order-dependent contiguous storage; flat final axis with checked prefix; zero-sized storage | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_dense_array_plan_places_flat_storage_by_order[c-flat-first-axis-open-suffix]` | `tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[source]`
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[generated-pyi]` | `tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[source]` (`runtime`)
`tests/fortran/arrays/end_to_end/test_documented_array_journey.py::test_documented_array_build_validates_layout_flat_strides_mutation_and_results[generated-pyi]` (`runtime`) | canonical | +| [Arrays: Strided Views](../../docs/user/guide/arrays.md#strided-views) | Supported | dense, forward and reversed rank two and three input/output; descriptor and sectioned-address handoff | `tests/fortran/arrays/codegen/test_strided_array_lowering.py::test_descriptor_array_parallel_role_edit_fails_before_backend_lowering[stride_roles]` | `tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_assumed_shape_accepts_fortran_ordered_strided_views[source]`
`tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py::test_assumed_shape_accepts_fortran_ordered_strided_views[generated-pyi]` | `tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py::test_rank_two_numpy_views_keep_their_axis_order[both-axes-reversed]` (`runtime`) | canonical | +| [Arrays: Mutation and Results](../../docs/user/guide/arrays.md#mutation-and-results) | Supported | input; caller output; in-place `intent(inout)`; no-`intent`; immutable replacement; optional presence; fixed, automatic, rank 1-15, and zero-size results; exact-kind Boolean copy-in/copy-out | `tests/fortran/arrays/codegen/test_array_result_lowering.py::test_array_result_lowering_transfers_bridge_copy_to_capsule_owned_numpy_storage` | `tests/fortran/arrays/end_to_end/test_array_results.py::test_array_results_follow_data_buffer_and_descriptor_handle_contracts[source]`
`tests/fortran/arrays/end_to_end/test_edited_pyi_layout_contract.py::test_edited_pyi_selects_direct_c_storage_or_fortran_copy_semantics`
`tests/fortran/arrays/end_to_end/test_logical_kind_array_conversions.py::test_boolean_arrays_are_aliased_at_their_own_width_without_any_copy` | — | canonical | +| [Arrays: Common Array Contracts](../../docs/user/guide/arrays.md#common-array-contracts) | Supported | every primitive dtype at ranks 1-3 and float64 through rank 15; rank zero separately; assumed rank 1-15; dense, C-order, copy, flat, strided, fixed/open extents | `tests/fortran/arrays/semantics/test_array_semantics.py::test_fortran_array_declarations_complete_category_shape_and_order` | `tests/fortran/arrays/end_to_end/test_primitive_dtype_rank_matrix.py::test_every_primitive_dtype_at_representative_ranks_mutates_exact_storage`
`tests/fortran/arrays/end_to_end/test_assumed_rank_arrays.py::test_assumed_rank_arguments_dispatch_to_runtime_rank[source]`
`tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py::test_scalar_values_and_rank_zero_storage_cross_the_native_boundary` | `tests/fortran/arrays/end_to_end/test_assumed_rank_arrays.py::test_assumed_rank_arguments_dispatch_to_runtime_rank[source]` (`runtime`) | canonical | +| [Strings: Choose A String Boundary](../../docs/user/guide/strings.md#choose-a-string-boundary) | Supported | runtime-length `String`; fixed scalar value/replacement/discard; rank-zero `S8`; rank-one `S8`; raw address deferred to Raw Addresses | `tests/fortran/strings/semantics/test_string_pyi_semantics.py::test_string_length_and_shape_axes_round_trip`
`tests/fortran/strings/policy/test_string_wrapper_policy.py::test_character_dummy_attributes_select_local_release_and_update_lane` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays`
`tests/fortran/strings/end_to_end/test_character_edge_cases.py::test_fixed_string_replacement_and_identity_use_canonical_plan` | — | canonical | +| [Strings: Complete Example](../../docs/user/guide/strings.md#complete-example) | Supported | source compilation; edited semantic `.pyi`; fixed scalar input/result; mutable scalar storage; fixed-width array input/output/inout | `tests/fortran/strings/end_to_end/test_character_boundaries.py::test_modern_fortran_character_arguments_and_results[generated-pyi]` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` | — | canonical | +| [Strings: Immutable Values](../../docs/user/guide/strings.md#immutable-values) | Supported | exact encoded width; trailing blanks; replacement identity; discarded mutation; hidden and direct results; allocation failure | `tests/fortran/strings/end_to_end/test_character_edge_cases.py::test_fixed_string_replacement_and_identity_use_canonical_plan`
`tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` | `tests/fortran/strings/end_to_end/test_character_edge_cases.py::test_fortran_character_edge_cases_follow_copy_in_copy_out_policy[source]` | `tests/fortran/strings/end_to_end/test_character_edge_cases.py::test_fortran_character_edge_cases_follow_copy_in_copy_out_policy[source]` (`runtime`) | canonical | +| [Strings: Mutable Scalar Storage](../../docs/user/guide/strings.md#mutable-scalar-storage) | Supported | rank-zero `S8`; same-object mutation; bytes result; exact itemsize/rank/writeability | `tests/fortran/strings/semantics/test_string_pyi_semantics.py::test_string_length_and_shape_axes_round_trip` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` (`runtime`) | canonical | +| [Strings: String Arrays](../../docs/user/guide/strings.md#string-arrays) | Supported | fixed itemsize; input and in-place mutation; fixed array result; rank/shape/dtype/writeability; zero size | `tests/fortran/strings/codegen/test_character_array_lowering.py::test_fixed_width_character_array_results_lower_itemsize_into_both_backends` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays`
`tests/fortran/strings/end_to_end/test_character_boundaries.py::test_modern_fortran_character_arguments_and_results[source]` | `tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` (`runtime`) | canonical | +| [Strings: Length And Encoding](../../docs/user/guide/strings.md#length-and-encoding) | Supported | length 1, representative width 8, runtime length, Unicode UTF-8 byte length, blanks, empty values, embedded NUL rejection, conservative no-`intent`, ambiguous mutable deferred scalar rejection | `tests/fortran/strings/parsing/test_character_length_parsing.py::test_character_entity_lengths_and_assumed_bounds_are_preserved`
`tests/fortran/strings/end_to_end/test_character_boundaries.py::test_modern_fortran_character_arguments_and_results[source]` | `tests/fortran/strings/end_to_end/test_character_boundaries.py::test_modern_fortran_character_arguments_and_results[source]` | `tests/fortran/strings/semantics/test_string_pyi_semantics.py::test_shape_spellings_are_rejected_in_the_character_length_slot` (`semantics`)
`tests/fortran/strings/end_to_end/test_documented_string_journey.py::test_documented_edited_pyi_distinguishes_values_scalar_storage_and_string_arrays` (`runtime`) | canonical | +| [Wrapping Functions: Basic Scalar Function](../../docs/user/guide/wrapping-functions.md#basic-scalar-function) | Supported | direct scalar result; exact NumPy inputs; visible value | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` (`runtime`) | canonical | | [Wrapping Functions: Python And Native Names](../../docs/user/guide/wrapping-functions.md#python-and-native-names) | Supported | edited `.pyi`; standalone external; `@bind`; changed Python name; unchanged native ABI; exact signature | — | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_selects_child_flattened_aliased_and_bound_exports` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_selects_child_flattened_aliased_and_bound_exports` (`runtime`) | canonical | -| [Wrapping Functions: Array Return Values](../../docs/user/guide/wrapping-functions.md#array-return-values) | Supported | automatic shape; new NumPy array; Fortran layout; values | `tests/fortran/arrays/codegen/test_array_result_lowering.py::test_array_results_record_producer_shape_copy_ownership_and_shared_hidden_slot` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` | — | canonical | -| [Wrapping Functions: Functions with Output Arguments](../../docs/user/guide/wrapping-functions.md#functions-with-output-arguments) | Supported | direct result first; hidden scalar output second; caller array excluded from tuple; stable tuple order | `tests/fortran/functions/policy/test_function_result_policy.py::test_multiple_scalar_result_policy_completes_order_and_hidden_address_before_planning`
`tests/fortran/functions/codegen/test_multiple_function_results.py::test_multiple_scalar_results_lower_to_binding_tuple_and_one_bridge_function_call` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` | `tests/fortran/functions/codegen/test_multiple_function_results.py::test_multiple_scalar_result_validation_rejects_position_and_consumer_drift` (`codegen`) | canonical | -| [Wrapping Functions: Important Rules](../../docs/user/guide/wrapping-functions.md#important-rules) | Supported | exact dtype; array copy result; projected scalar tuple order; caller array mutation; conservative no-`intent` scalar replacement after direct result; Python-visible callable docstrings | `tests/fortran/functions/semantics/test_fortran_function_semantics.py::test_missing_intent_scalar_uses_conservative_replacement_projection`
`tests/fortran/functions/policy/test_function_result_policy.py::test_scalar_copy_in_out_policy_completes_writeback_before_planning`
`tests/fortran/functions/codegen/test_scalar_function_writeback.py::test_scalar_writeback_is_an_explicit_binding_lifecycle_result`
`tests/fortran/infrastructure/codegen/test_docstrings.py::test_codegen_renders_unresolved_plan_docstrings_before_freezing` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order`
`tests/fortran/functions/end_to_end/test_hidden_native_outputs.py::test_hidden_outputs_are_released_but_never_returned` | — | canonical | -| [Wrapping Subroutines: How Arguments Become Python Results](../../docs/user/guide/wrapping-subroutines.md#how-arguments-become-python-results) | Supported | input scalar/array; hidden scalar output; scalar replacement; caller array output/inout; visible derived object; hidden allocatable handle; conservative no-`intent` | `tests/fortran/subroutines/semantics/test_subroutine_argument_projection.py::test_primitive_scalar_inout_stays_visible_and_projects_replacement_return`
`tests/fortran/subroutines/semantics/test_subroutine_argument_projection.py::test_ordinary_array_output_stays_visible_without_result_projection`
`tests/fortran/subroutines/semantics/test_subroutine_argument_projection.py::test_scalar_derived_output_stays_visible_without_result_projection` | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules` | — | canonical | -| [Wrapping Subroutines: Complete Example](../../docs/user/guide/wrapping-subroutines.md#complete-example) | Supported | source build; hidden bounds tuple; in-place array scaling; scalar replacement; caller output storage | `tests/fortran/subroutines/policy/test_subroutine_output_policy.py::test_source_hidden_scalar_output_completes_call_local_address_before_planning` | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules` | — | canonical | +| [Wrapping Functions: Array Return Values](../../docs/user/guide/wrapping-functions.md#array-return-values) | Supported | automatic shape; new NumPy array; Fortran layout; values | `tests/fortran/arrays/codegen/test_array_result_lowering.py::test_array_result_lowering_transfers_bridge_copy_to_capsule_owned_numpy_storage` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` | — | canonical | +| [Wrapping Functions: Functions with Output Arguments](../../docs/user/guide/wrapping-functions.md#functions-with-output-arguments) | Supported | direct result first; hidden scalar output second; caller array excluded from tuple; stable tuple order | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` | `tests/fortran/functions/codegen/test_multiple_function_results.py::test_multiple_scalar_result_validation_rejects_position_and_consumer_drift` (`codegen`) | canonical | +| [Wrapping Functions: Important Rules](../../docs/user/guide/wrapping-functions.md#important-rules) | Supported | exact dtype; array copy result; projected scalar tuple order; caller array mutation; conservative no-`intent` scalar replacement after direct result; Python-visible callable docstrings | `tests/fortran/functions/semantics/test_fortran_function_semantics.py::test_missing_intent_scalar_uses_conservative_replacement_projection`
`tests/fortran/data_types/end_to_end/test_rank_zero_scalar_storage.py::test_scalar_values_and_rank_zero_storage_cross_the_native_boundary`
`tests/fortran/infrastructure/codegen/test_docstrings.py::test_codegen_renders_unresolved_plan_docstrings_before_freezing` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order`
`tests/fortran/functions/end_to_end/test_hidden_native_outputs.py::test_hidden_outputs_are_released_but_never_returned` | — | canonical | +| [Wrapping Subroutines: How Arguments Become Python Results](../../docs/user/guide/wrapping-subroutines.md#how-arguments-become-python-results) | Supported | input scalar/array; hidden scalar output; scalar replacement; caller array output/inout; visible derived object; hidden allocatable handle; conservative no-`intent` | `tests/fortran/subroutines/semantics/test_subroutine_argument_projection.py::test_output_dummies_stay_visible_and_only_primitive_scalars_project_a_result` | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules` | — | canonical | +| [Wrapping Subroutines: Complete Example](../../docs/user/guide/wrapping-subroutines.md#complete-example) | Supported | source build; hidden bounds tuple; in-place array scaling; scalar replacement; caller output storage | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules` | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules` | — | canonical | | [Wrapping Subroutines: Python Usage](../../docs/user/guide/wrapping-subroutines.md#python-usage) | Supported | exact NumPy values; scalar object unchanged; arrays mutated in place; visible outputs | — | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules` | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules` (`runtime`) | canonical | -| [Wrapping Subroutines: Key Rules](../../docs/user/guide/wrapping-subroutines.md#key-rules) | Supported | hidden scalar ordering; explicit scalar writeback lifecycle; ordinary arrays and derived objects excluded from result; native-created allocatable returned; `.pyi` projection authority | `tests/fortran/subroutines/codegen/test_hidden_scalar_outputs.py::test_hidden_scalar_result_is_one_bridge_output_and_one_python_result`
`tests/fortran/subroutines/codegen/test_scalar_subroutine_writeback_validation.py::test_generator_rejects_writeback_without_python_result_target` | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules` | `tests/fortran/subroutines/codegen/test_scalar_subroutine_writeback_validation.py::test_generator_rejects_writeback_from_an_unavailable_handoff` (`codegen`) | canonical | +| [Wrapping Subroutines: Key Rules](../../docs/user/guide/wrapping-subroutines.md#key-rules) | Supported | hidden scalar ordering; explicit scalar writeback lifecycle; ordinary arrays and derived objects excluded from result; native-created allocatable returned; `.pyi` projection authority | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules`
`tests/fortran/subroutines/codegen/test_scalar_subroutine_writeback_validation.py::test_generator_rejects_incomplete_scalar_writeback_plans` | `tests/fortran/subroutines/end_to_end/test_documented_subroutine_journey.py::test_subroutine_outputs_and_caller_storage_follow_documented_projection_rules` | `tests/fortran/subroutines/codegen/test_scalar_subroutine_writeback_validation.py::test_generator_rejects_incomplete_scalar_writeback_plans` (`codegen`) | canonical | | [Wrapping Modules: Basic Usage](../../docs/user/guide/wrapping-modules.md#basic-usage) | Supported | generated package entry; child native-module namespace; isolated import | `tests/fortran/modules/pipeline/test_generated_module_contracts.py::test_generated_module_contract_matches_fixture[module_exports]` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_selects_child_flattened_aliased_and_bound_exports` | — | canonical | | [Wrapping Modules: Procedures](../../docs/user/guide/wrapping-modules.md#procedures) | Supported | module functions; standalone external at root; multiple native modules in one source | `tests/fortran/modules/pipeline/test_generated_module_contracts.py::test_generated_module_contract_matches_fixture[module_exports]` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_selects_child_flattened_aliased_and_bound_exports` | — | canonical | -| [Wrapping Modules: Public Variables and Constants](../../docs/user/guide/wrapping-modules.md#public-variables-and-constants) | Supported | writable scalar state; true parameter; Python-local constant shadow; native state unchanged | `tests/fortran/modules/policy/test_module_variable_policy.py::test_scalar_module_variable_policy_completes_access_and_storage_before_planning`
`tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py::test_module_variable_plan_contains_only_completed_dispatch_facts` | `tests/fortran/modules/end_to_end/test_module_variables_and_state.py::test_scalar_module_variables_use_attributes_and_parameters_have_no_native_setter[source]` | — | canonical | +| [Wrapping Modules: Public Variables and Constants](../../docs/user/guide/wrapping-modules.md#public-variables-and-constants) | Supported | writable scalar state; true parameter; Python-local constant shadow; native state unchanged | `tests/fortran/modules/policy/test_module_variable_policy.py::test_scalar_module_variable_policy_completes_access_and_storage_before_planning` | `tests/fortran/modules/end_to_end/test_module_variables_and_state.py::test_scalar_module_variables_use_attributes_and_parameters_have_no_native_setter[source]` | — | canonical | | [Wrapping Modules: Module Arrays and Saved State](../../docs/user/guide/wrapping-modules.md#module-arrays-saved-state) | Supported | allocatable module array; persistent handle; live NumPy view; mutation; deallocation; procedure-local `save`; shared state across imports | `tests/fortran/modules/policy/test_module_variable_policy.py::test_scalar_module_variable_policy_completes_access_and_storage_before_planning` | `tests/fortran/modules/end_to_end/test_scalar_module_variable_plan.py::test_whole_scalar_module_variable_behavior_uses_canonical_plan`
`tests/fortran/modules/end_to_end/test_module_variables_and_state.py::test_scalar_module_variables_use_attributes_and_parameters_have_no_native_setter[source]` | — | canonical | | [Wrapping Modules: Shape the Module API With the Contract](../../docs/user/guide/wrapping-modules.md#shape-the-module-api-with-the-contract) | Supported | mutable literal initializer; hidden variable; private procedure; removed declaration; true `Final` constant | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/semantics/test_module_initializers.py::test_mutable_module_literal_defaults_are_preserved`
`tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/policy/test_export_and_initializer_policy.py::test_policy_completion_prunes_unexported_entry_declarations_before_lowering` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_visibility_and_initialization.py::test_editable_contract_removes_hides_and_initializes_module_declarations` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/semantics/test_module_initializers.py::test_mutable_module_expression_defaults_are_rejected[from prik.contracts import Int32\ncounter: Int32 = f(42)\n]` (`semantics`)
`tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/policy/test_export_and_initializer_policy.py::test_unsupported_module_variable_initializer_completes_an_unsupported_policy` (`policy`) | canonical | | [Wrapping Modules: Flatten Module Namespaces](../../docs/user/guide/wrapping-modules.md#flatten-module-namespaces) | Supported | child namespaces; wildcard flattening; selective imports; explicit aliases; unchanged native targets | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/policy/test_export_and_initializer_policy.py::test_policy_completion_prunes_unexported_entry_declarations_before_lowering` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_selects_child_flattened_aliased_and_bound_exports` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_rejects_colliding_wildcard_exports` (`pipeline`) | canonical | | [Wrapping Modules: Important Rules](../../docs/user/guide/wrapping-modules.md#important-rules) | Supported | private declarations hidden; common-block storage internal; shared native state; source-derived extension identity; public member module docstrings | `tests/fortran/modules/semantics/test_module_contract_semantics.py::test_module_common_block_storage_stays_internal` | `tests/fortran/modules/end_to_end/test_common_blocks.py::test_common_block_storage_stays_internal_to_wrapped_fortran[source]`
`tests/fortran/modules/end_to_end/test_module_variables_and_state.py::test_scalar_module_variables_use_attributes_and_parameters_have_no_native_setter[source]` | — | canonical | -| [Optional Arguments: Complete Example](../../docs/user/guide/optional-arguments.md#complete-example) | Supported | source generation; reviewed generated `.pyi`; optional scalar input; optional ordinary array output; native `present(...)` | `tests/fortran/optional_arguments/pipeline/test_generated_optional_contracts.py::test_generated_optional_contract_matches_fixture[foptional_f90]` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[generated-pyi]` | — | canonical | -| [Optional Arguments: Usage in Python](../../docs/user/guide/optional-arguments.md#usage-in-python) | Supported | omission; explicit `None`; positional value; keyword value; skipped earlier positions | `tests/fortran/optional_arguments/codegen/test_optional_lowering.py::test_optional_scalar_lowering_distinguishes_absent_or_none_from_value` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]` (`runtime`) | canonical | -| [Optional Arguments: Key Rules](../../docs/user/guide/optional-arguments.md#key-rules) | Supported | scalar; array; string; derived input; output visibility; conservative no-`intent`; presence without result inflation | `tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py::test_optional_without_intent_uses_visible_conservative_replacement_projection`
`tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_projected_array_keeps_nullable_value_separate_from_descriptor_storage` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_array_buffers_preserve_omission_and_identity` | — | canonical | -| [Optional Arguments: Scalar Allocatables and Pointers](../../docs/user/guide/optional-arguments.md#scalar-allocatables-and-pointers) | Supported | allocatable and pointer; omitted; present-unallocated/unassociated `None`; present concrete value; exact type rejection | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_descriptor_policy_completes_three_state_boundary_before_planning`
`tests/fortran/optional_arguments/codegen/test_optional_lowering.py::test_optional_descriptor_lowering_records_presence_and_nullable_value_handoffs` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_scalar_descriptors_distinguish_omitted_none_and_value` | — | canonical | -| [Optional Arguments: Optional Outputs](../../docs/user/guide/optional-arguments.md#optional-outputs) | Supported | ordinary scalar and array outputs; derived output remains visible; allocatable and pointer output visibility; absent, explicit `None`, caller storage; in-place identity; stable result projection | `tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py::test_optional_scalar_output_remains_visible_scalar_storage`
`tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py::test_optional_scalar_derived_output_stays_visible_without_result_projection`
`tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py::test_optional_allocatable_output_remains_visible`
`tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py::test_pointer_array_output_visibility_follows_intent_and_optional_presence` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_array_buffers_preserve_omission_and_identity` | — | canonical | -| [Optional Arguments: Limitations](../../docs/user/guide/optional-arguments.md#limitations) | Blocked | optional passed procedure; no invented native default; optional native literals cannot replace native procedure behavior | — | — | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_passed_procedure_is_blocked_before_codegen` (`policy`)
`tests/fortran/optional_arguments/codegen/test_optional_lowering.py::test_optional_arguments_with_hidden_literals_fail_during_shared_plan_validation` (`codegen`) | canonical | -| [Generic Interfaces: Complete Example](../../docs/user/guide/generic-interfaces.md#complete-example) | Supported | free-form source; fixed-form parsing and contract generation; reviewed generated `.pyi`; source build; generated-`.pyi` replay | `tests/fortran/generic_interfaces/parsing/test_generic_interface_syntax.py::test_fixed_form_generic_interface_preserves_specific_procedures`
`tests/fortran/generic_interfaces/pipeline/test_generated_generic_contracts.py::test_generated_generic_contract_matches_fixture[foverloads_f90]`
`tests/fortran/generic_interfaces/pipeline/test_generated_generic_contracts.py::test_generated_generic_contract_matches_fixture[foverloads_fixed]` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]`
`tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[generated-pyi]` | — | canonical | +| [Optional Arguments: Complete Example](../../docs/user/guide/optional-arguments.md#complete-example) | Supported | source generation; reviewed generated `.pyi`; optional scalar input; optional ordinary array output; native `present(...)` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[generated-pyi]` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[generated-pyi]` | — | canonical | +| [Optional Arguments: Usage in Python](../../docs/user/guide/optional-arguments.md#usage-in-python) | Supported | omission; explicit `None`; positional value; keyword value; skipped earlier positions | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]` (`runtime`) | canonical | +| [Optional Arguments: Key Rules](../../docs/user/guide/optional-arguments.md#key-rules) | Supported | scalar; array; string; derived input; output visibility; conservative no-`intent`; presence without result inflation | `tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py::test_optional_output_visibility_follows_intent_and_presence`
`tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_policy_completes_presence_mode_before_planning` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_array_buffers_preserve_omission_and_identity` | — | canonical | +| [Optional Arguments: Scalar Allocatables and Pointers](../../docs/user/guide/optional-arguments.md#scalar-allocatables-and-pointers) | Supported | allocatable and pointer; omitted; present-unallocated/unassociated `None`; present concrete value; exact type rejection | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_policy_completes_presence_mode_before_planning`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_scalar_descriptors_distinguish_omitted_none_and_value` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_scalar_descriptors_distinguish_omitted_none_and_value` | — | canonical | +| [Optional Arguments: Optional Outputs](../../docs/user/guide/optional-arguments.md#optional-outputs) | Supported | ordinary scalar and array outputs; derived output remains visible; allocatable and pointer output visibility; absent, explicit `None`, caller storage; in-place identity; stable result projection | `tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py::test_optional_output_visibility_follows_intent_and_presence` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_array_buffers_preserve_omission_and_identity` | — | canonical | +| [Optional Arguments: Limitations](../../docs/user/guide/optional-arguments.md#limitations) | Blocked | optional passed procedure; no invented native default; optional native literals cannot replace native procedure behavior | — | — | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_value_callback_dummy_is_blocked_before_codegen` (`policy`)
`tests/fortran/optional_arguments/codegen/test_optional_lowering.py::test_optional_arguments_with_hidden_literals_materialize_the_literal_in_the_binding` (`codegen`) | canonical | +| [Generic Interfaces: Complete Example](../../docs/user/guide/generic-interfaces.md#complete-example) | Supported | free-form source; fixed-form parsing and contract generation; reviewed generated `.pyi`; source build; generated-`.pyi` replay | `tests/fortran/generic_interfaces/parsing/test_generic_interface_syntax.py::test_fixed_form_generic_interface_preserves_specific_procedures`
`tests/fortran/generic_interfaces/pipeline/test_generated_generic_contracts.py::test_generated_generic_contract_matches_fixture[foverloads_fixed]` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]`
`tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[generated-pyi]` | — | canonical | | [Generic Interfaces: Generated Contract](../../docs/user/guide/generic-interfaces.md#complete-example) | Supported | private link targets; one exact overload candidate per declaration; public-generic `@bind`; native target precedence | `tests/fortran/generic_interfaces/semantics/test_pyi_overload_semantics.py::test_convert_pyi_to_ir_resolves_prik_overload_by_explicit_specific_name`
`tests/fortran/generic_interfaces/policy/test_generic_policy.py::test_module_overload_bind_takes_precedence_per_candidate` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]` | — | canonical | | [Generic Interfaces: Usage in Python](../../docs/user/guide/generic-interfaces.md#complete-example) | Supported | exact `Int32`, `Float64`, and `Complex128`; scalar and rank-one dispatch; generated-class dispatch; no implicit coercion | `tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py::test_plan_records_one_exact_numpy_scalar_predicate_per_candidate` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]` (`runtime`) | canonical | | [Generic Interfaces: Inspect the Overloads](../../docs/user/guide/generic-interfaces.md#inspect-the-overloads) | Supported | one public callable; all accepted signatures; hidden concrete procedures and internal names | — | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]` | — | canonical | | [Generic Interfaces: Extend an Overload Set](../../docs/user/guide/generic-interfaces.md#extend-an-overload-set) | Supported | edited `.pyi`; renamed public binding; added overload group; private-specific routing through public generic; absent candidate rejection | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_bind_selects_module_method_and_constructor_overload_targets` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/end_to_end/test_edited_class_surfaces.py::test_module_method_and_constructor_overloads_share_one_edited_contract` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/end_to_end/test_edited_class_surfaces.py::test_private_native_specific_without_overload_bind_fails_at_build[private_module_specifics_without_bind-missing_targets0]` (`compiling`) | canonical | | [Generic Interfaces: Key Rules](../../docs/user/guide/generic-interfaces.md#key-rules) | Supported | exact dtype/rank/class match; no-match `TypeError`; ambiguous signature rejection; exact-once specific links; `@bind`; private visibility; type-bound generics; defined operators; defined assignment | `tests/fortran/generic_interfaces/semantics/test_fortran_generic_semantics.py::test_converter_preserves_module_and_type_bound_generic_overload_sets`
`tests/fortran/generic_interfaces/semantics/test_fortran_generic_semantics.py::test_converter_preserves_defined_operators_assignment_and_type_bound_operators`
`tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py::test_plan_records_one_exact_numpy_scalar_predicate_per_candidate` | `tests/fortran/generic_interfaces/end_to_end/test_defined_operators.py::test_fortran_defined_operators_and_assignment_dispatch_in_generated_c_extension[source]` | `tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py::test_generator_rejects_ambiguous_edited_overload_plan_before_emission` (`codegen`)
`tests/fortran/generic_interfaces/semantics/test_pyi_overload_semantics.py::test_convert_pyi_to_ir_rejects_invalid_prik_overload_links[@overload("missing")\ndef convert(value: Int32) -> Int32: ...\n-missing specific procedure 'missing']` (`semantics`) | canonical | | [Generic Interfaces: Limitations](../../docs/user/guide/generic-interfaces.md#limitations) | Partially supported | only a type-named interface is a constructor; assumed-type `class(*)`; arrays of derived values | — | `tests/fortran/derived_types/end_to_end/test_generic_constructor.py::test_constructor_interface_overloads_init_from_its_specifics`
`tests/fortran/derived_types/end_to_end/test_generic_constructor.py::test_type_without_a_constructor_interface_keeps_keyword_fields` | `tests/fortran/generic_interfaces/parsing/test_generic_interface_syntax.py::test_assumed_type_generic_candidate_is_rejected_at_parsing` (`parsing`)
`tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py::test_generic_candidate_with_array_of_derived_values_is_blocked_before_lowering` (`codegen`) | canonical | -| [Wrapping Derived Types: Complete Example](../../docs/user/guide/wrapping-derived-types.md#complete-example) | Supported | derived declarations; public and nested fields; source generation; reviewed generated `.pyi`; source build; generated-`.pyi` replay | `tests/fortran/derived_types/parsing/test_derived_type_declarations.py::test_derived_type_fields_and_methods_detection`
`tests/fortran/derived_types/pipeline/test_generated_derived_contracts.py::test_generated_derived_contract_matches_fixture[fderived_boundary_f90]` | `tests/fortran/derived_types/end_to_end/test_derived_boundaries.py::test_scalar_derived_types_cross_procedure_boundaries[source]`
`tests/fortran/derived_types/end_to_end/test_derived_boundaries.py::test_scalar_derived_types_cross_procedure_boundaries[generated-pyi]` | — | canonical | -| [Wrapping Derived Types: Usage in Python](../../docs/user/guide/wrapping-derived-types.md#usage-in-python) | Supported | keyword construction; public field get/set; `intent(inout)` identity; owned result; nested borrowed component | `tests/fortran/derived_types/policy/test_derived_policy_defaults.py::test_recursive_module_policy_map_includes_nested_fields_and_functions` | `tests/fortran/derived_types/end_to_end/test_derived_boundaries.py::test_scalar_derived_types_cross_procedure_boundaries[source]` | — | canonical | +| [Wrapping Derived Types: Complete Example](../../docs/user/guide/wrapping-derived-types.md#complete-example) | Supported | derived declarations; public and nested fields; source generation; reviewed generated `.pyi`; source build; generated-`.pyi` replay | `tests/fortran/derived_types/parsing/test_derived_type_declarations.py::test_derived_type_fields_and_methods_detection` | `tests/fortran/derived_types/end_to_end/test_derived_boundaries.py::test_scalar_derived_types_cross_procedure_boundaries[source]`
`tests/fortran/derived_types/end_to_end/test_derived_boundaries.py::test_scalar_derived_types_cross_procedure_boundaries[generated-pyi]` | — | canonical | +| [Wrapping Derived Types: Usage in Python](../../docs/user/guide/wrapping-derived-types.md#usage-in-python) | Supported | keyword construction; public field get/set; `intent(inout)` identity; owned result; nested borrowed component | `tests/fortran/derived_types/codegen/test_derived_lowering.py::test_owned_derived_result_has_explicit_failure_and_release_lifecycle` | `tests/fortran/derived_types/end_to_end/test_derived_boundaries.py::test_scalar_derived_types_cross_procedure_boundaries[source]` | — | canonical | | [Wrapping Derived Types: Inspect the Class](../../docs/user/guide/wrapping-derived-types.md#inspect-the-class) | Supported | class, constructor, field, method, parameter, return, and overload docstrings; no native implementation names | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/policy/test_class_surface_policy.py::test_bound_constructor_and_method_reuse_completed_direct_function_plans`
`tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/policy/test_class_surface_policy.py::test_edited_overloads_complete_exact_dispatch_and_reject_ambiguous_plan` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/end_to_end/test_edited_class_surfaces.py::test_module_procedure_is_reused_by_bound_constructor_method_and_public_function` | — | canonical | -| [Wrapping Derived Types: Key Concepts](../../docs/user/guide/wrapping-derived-types.md#key-concepts) | Supported | Python-owned construction/result; parent-retained component; in-place output/inout/no-`intent`; primitive writable fields; nested types; keyword defaults; destruction | `tests/fortran/derived_types/policy/test_derived_accessor_policy.py::test_derived_field_setter_policy_uses_value_copy_write_through`
`tests/fortran/derived_types/codegen/test_derived_lowering.py::test_projected_derived_argument_returns_the_exact_caller_wrapper_without_release` | `tests/fortran/derived_types/end_to_end/test_default_constructors_and_finalizers.py::test_fortran_default_constructor_keywords_and_finalization[source]`
`tests/fortran/derived_types/end_to_end/test_borrowed_components.py::test_borrowed_child_wrapper_never_finalizes_native_component[source]` | — | canonical | +| [Wrapping Derived Types: Key Concepts](../../docs/user/guide/wrapping-derived-types.md#key-concepts) | Supported | Python-owned construction/result; parent-retained component; in-place output/inout/no-`intent`; primitive writable fields; nested types; keyword defaults; destruction | `tests/fortran/derived_types/policy/test_derived_accessor_policy.py::test_explicit_borrowed_derived_field_setter_rejects_replacement`
`tests/fortran/derived_types/codegen/test_derived_lowering.py::test_native_result_and_derived_writeback_share_ordered_output_aggregation` | `tests/fortran/derived_types/end_to_end/test_default_constructors_and_finalizers.py::test_fortran_default_constructor_keywords_and_finalization[source]`
`tests/fortran/derived_types/end_to_end/test_borrowed_components.py::test_borrowed_child_wrapper_never_finalizes_native_component[source]` | — | canonical | | [Wrapping Derived Types: Custom Constructor](../../docs/user/guide/wrapping-derived-types.md#custom-constructor) | Supported | edited `.pyi`; `@bind`; exactly one `Pass()`; reordered `Addr(Arg)` values; replacement of generated keyword initializer; constructor docs | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_bound_constructor_uses_explicit_pass_position_and_native_target`
`tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/policy/test_class_surface_policy.py::test_bound_constructor_pass_disambiguates_same_type_arguments_and_keeps_module_export` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/end_to_end/test_edited_class_surfaces.py::test_module_procedure_is_reused_by_bound_constructor_method_and_public_function` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_contradictory_constructor_declarations_are_rejected[\nclass state:\n @bind("init_state")\n @native_call([Addr(Arg(0))])\n def __init__(self, seed: Int32) -> None: ...\n-Bound constructor native_call requires exactly one Pass() entry]` (`semantics`) | canonical | -| [Wrapping Derived Types: Type-Bound Methods](../../docs/user/guide/wrapping-derived-types.md#type-bound-methods) | Supported | passed object becomes `self`; mutation preserves Python identity; direct and generated-`.pyi` replay | `tests/fortran/derived_types/semantics/test_fortran_derived_semantics.py::test_converter_covers_derived_dispatch_methods_and_kind_edges` | `tests/fortran/derived_types/end_to_end/test_type_bound_methods.py::test_modern_fortran_derived_type_exposes_class_and_type_bound_methods[source]`
`tests/fortran/derived_types/end_to_end/test_type_bound_methods.py::test_modern_fortran_derived_type_exposes_class_and_type_bound_methods[generated-pyi]` | — | canonical | +| [Wrapping Derived Types: Type-Bound Methods](../../docs/user/guide/wrapping-derived-types.md#type-bound-methods) | Supported | passed object becomes `self`; mutation preserves Python identity; direct and generated-`.pyi` replay | `tests/fortran/derived_types/parsing/test_derived_procedure_syntax.py::test_derived_type_procedure_and_generic_bindings` | `tests/fortran/derived_types/end_to_end/test_type_bound_methods.py::test_modern_fortran_derived_type_exposes_class_and_type_bound_methods[source]`
`tests/fortran/derived_types/end_to_end/test_type_bound_methods.py::test_modern_fortran_derived_type_exposes_class_and_type_bound_methods[generated-pyi]` | — | canonical | | [Wrapping Derived Types: Expose a Module Procedure as a Method](../../docs/user/guide/wrapping-derived-types.md#expose-a-module-procedure-as-a-method) | Supported | edited class method; `Pass()` receiver; independent module declaration; same or bound native name; optional private module surface; method docs | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_method_and_module_declarations_keep_native_targets_independent`
`tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/policy/test_class_surface_policy.py::test_module_procedure_method_visibility_is_completed_independently` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/end_to_end/test_edited_class_surfaces.py::test_module_procedure_is_reused_by_bound_constructor_method_and_public_function` | — | canonical | | [Wrapping Derived Types: Type-Bound Generics](../../docs/user/guide/wrapping-derived-types.md#type-bound-generics) | Supported | private specifics; public generic bind; exact `Int32`/`Float64` dispatch; wrapped receiver fixed by class; no trial calls | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_bind_selects_module_method_and_constructor_overload_targets`
`tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/policy/test_class_surface_policy.py::test_edited_overloads_complete_exact_dispatch_and_reject_ambiguous_plan` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/end_to_end/test_edited_class_surfaces.py::test_module_method_and_constructor_overloads_share_one_edited_contract` | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/end_to_end/test_edited_class_surfaces.py::test_module_method_and_constructor_overloads_share_one_edited_contract` (`runtime`) | canonical | | [Wrapping Derived Types: Defined Operators](../../docs/user/guide/wrapping-derived-types.md#defined-operators) | Supported | direct/reflected binary; unary; comparison; logical; named operators; defined assignment; exact wrapped/scalar dispatch; operator docstrings | `tests/fortran/generic_interfaces/semantics/test_fortran_generic_semantics.py::test_converter_preserves_defined_operators_assignment_and_type_bound_operators` | `tests/fortran/generic_interfaces/end_to_end/test_defined_operators.py::test_fortran_defined_operators_and_assignment_dispatch_in_generated_c_extension[source]` | `tests/fortran/generic_interfaces/end_to_end/test_defined_operators.py::test_fortran_defined_operators_and_assignment_dispatch_in_generated_c_extension[source]` (`runtime`) | canonical | -| [Wrapping Derived Types: Inheritance And Polymorphic Input Dispatch](../../docs/user/guide/wrapping-derived-types.md#inheritance-and-polymorphic-input-dispatch) | Partially supported | scalar extension inheritance; closed `class(base), intent(in)` dispatch; exact extension classes; unsupported polymorphic results, mutation, arrays, descriptor scalars, and assumed type | `tests/fortran/derived_types/codegen/test_class_surfaces.py::test_inheritance_and_polymorphism_are_completed_before_planning` | `tests/fortran/derived_types/end_to_end/test_inheritance_and_polymorphism.py::test_fortran_extension_types_generate_python_inheritance[source]`
`tests/fortran/derived_types/end_to_end/test_inheritance_and_polymorphism.py::test_fortran_extension_types_generate_python_inheritance[generated-pyi]` | `tests/fortran/derived_types/codegen/test_class_surfaces.py::test_invalid_class_graph_fails_before_emission` (`codegen`)
`tests/fortran/derived_types/policy/test_derived_accessor_policy.py::test_deferred_binding_without_an_abstract_type_is_refused` (`policy`) | canonical | -| [Wrapping Derived Types: Native Representation](../../docs/user/guide/wrapping-derived-types.md#native-representation) | Supported | opaque accessor storage for ordinary, `bind(C)`, and `sequence`; field get/set; by-value copy; no direct C aggregate access | `tests/fortran/derived_types/semantics/test_fortran_derived_semantics.py::test_bind_c_and_sequence_types_preserve_accessor_layout_metadata`
`tests/fortran/derived_types/codegen/test_derived_lowering.py::test_exact_typed_value_lowering_uses_fortran_value_semantics_and_opaque_binding` | `tests/fortran/derived_types/end_to_end/test_opaque_layout.py::test_bind_c_derived_types_use_accessors_and_fortran_value_copy[source]`
`tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_sequence_derived_value_uses_the_same_typed_opaque_call_path` | — | canonical | -| [Allocatables: Key Concepts](../../docs/user/guide/allocatables.md#key-concepts) | Supported | scalar value versus array handle; allocated, unallocated, and zero-sized states; live views; module, field, result, and caller-created descriptor origins | `tests/fortran/allocatables/semantics/test_pyi_allocatable_semantics.py::test_persistent_allocatable_descriptors_preserve_scalar_and_array_kinds`
`tests/fortran/allocatables/policy/test_allocatable_handle_policy.py::test_allocatable_array_field_is_wrapper_owned_borrowed_view` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]` | — | canonical | -| [Allocatables: When To Use An Allocatable Handle](../../docs/user/guide/allocatables.md#when-to-use-an-allocatable-handle) | Supported | descriptor arguments versus ordinary arrays; present-empty caller handle; dtype/rank compatibility; plain NumPy rejection | `tests/fortran/allocatables/runtime/test_allocatable_descriptor_abi.py::test_allocatable_descriptor_hook_accepts_unallocated_descriptor_without_numpy_conversion`
`tests/fortran/allocatables/runtime/test_allocatable_array_actual_abi.py::test_array_actual_argument_abi_packer_uses_allocatable_native_array_actual_without_numpy_conversion` | `tests/fortran/allocatables/end_to_end/test_external_allocatable.py::test_standalone_allocatable_argument_accepts_a_caller_created_handle` | `tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py::test_generated_storage_rejects_incompatible_allocatable_contract_handles[-float64-1-TypeError-fresh contract handle]` (`runtime`) | canonical | -| [Allocatables: Allocatable Array Handle API](../../docs/user/guide/allocatables.md#allocatable-array-handle-api) | Supported | default construction; allocated, shape, dtype, rank, `to_numpy`, resize, deallocate, close, and closed state; unavailable-operation gating | `tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py::test_contract_default_allocatable_constructor_preserves_dtype_rank_and_empty_state`
`tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py::test_allocatable_handle_reports_absent_state_and_routes_resize_deallocate` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_plain_allocatable_module_array_exposes_current_live_view[source]` | `tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py::test_allocatable_operations_are_gated_by_the_completed_ops_table` (`runtime`)
`tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py::test_non_array_allocatable_annotations_are_not_factories[-scalar allocatable contracts]` (`runtime`) | canonical | -| [Allocatables: Deallocate Versus Close](../../docs/user/guide/allocatables.md#deallocate-versus-close) | Supported | deallocate while open; owned descriptor close; idempotent close; borrowed module/field close no-op; closed-handle rejection | `tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py::test_close_is_a_noop_for_a_borrowed_allocatable_handle`
`tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py::test_writable_contract_handle_adopts_generated_storage_and_closes_once` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]` | `tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py::test_generated_storage_rejects_a_closed_contract_handle` (`runtime`) | canonical | -| [Allocatables: Module Variables And Derived Fields](../../docs/user/guide/allocatables.md#module-variables-and-derived-fields) | Supported | native-owned module descriptor; parent-retained component descriptor; stable handle identity; live allocation changes and mutation | `tests/fortran/allocatables/policy/test_allocatable_handle_policy.py::test_allocatable_array_field_is_wrapper_owned_borrowed_view`
`tests/fortran/allocatables/codegen/test_allocatable_lowering.py::test_plain_module_allocatable_uses_standard_descriptor_callback_without_copy` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[generated-pyi]` | — | canonical | -| [Allocatables: Function Results](../../docs/user/guide/allocatables.md#function-results) | Supported | owned result descriptor; allocated and zero-sized results; explicit maybe-unallocated result; rank one, rank two, and high-rank policy | `tests/fortran/allocatables/policy/test_allocatable_result_policy.py::test_direct_high_rank_allocatable_function_result_is_supported_before_codegen`
`tests/fortran/allocatables/codegen/test_allocatable_lowering.py::test_allocated_direct_result_assigns_then_moves_into_owned_descriptor` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_maybe_unallocated_direct_allocatable_results_preserve_unallocated_state` | `tests/fortran/allocatables/policy/test_allocatable_result_policy.py::test_direct_allocatable_scalar_function_result_is_blocked_before_codegen` (`policy`)
`tests/fortran/allocatables/codegen/test_allocatable_lowering.py::test_maybe_unallocated_is_only_valid_on_direct_allocatable_array_results` (`codegen`) | canonical | -| [Allocatables: Output And Inout Arguments](../../docs/user/guide/allocatables.md#output-and-inout-arguments) | Supported | hidden nonoptional output; visible optional output; visible inout; unallocated, allocated, reallocated, and deallocated states; same-object projection | `tests/fortran/allocatables/semantics/test_fortran_allocatable_semantics.py::test_allocatable_output_semantics_projects_a_hidden_descriptor_handle`
`tests/fortran/allocatables/policy/test_allocatable_handle_policy.py::test_visible_descriptor_writeback_completes_caller_handle_construction_lifecycle`
`tests/fortran/allocatables/pipeline/test_allocatable_output_contract_printing.py::test_emit_optional_allocatable_output_as_visible_argument` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]`
`tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py::test_caller_created_allocatable_crosses_separately_built_extensions` | — | canonical | +| [Wrapping Derived Types: Inheritance And Polymorphic Input Dispatch](../../docs/user/guide/wrapping-derived-types.md#inheritance-and-polymorphic-input-dispatch) | Partially supported | scalar extension inheritance; closed `class(base), intent(in)` dispatch; exact extension classes; unsupported polymorphic results, mutation, arrays, descriptor scalars, and assumed type | — | `tests/fortran/derived_types/end_to_end/test_inheritance_and_polymorphism.py::test_fortran_extension_types_generate_python_inheritance[source]`
`tests/fortran/derived_types/end_to_end/test_inheritance_and_polymorphism.py::test_fortran_extension_types_generate_python_inheritance[generated-pyi]` | `tests/fortran/derived_types/codegen/test_class_surfaces.py::test_invalid_class_graph_fails_before_emission` (`codegen`)
`tests/fortran/derived_types/policy/test_derived_accessor_policy.py::test_deferred_binding_without_an_abstract_type_is_refused` (`policy`) | canonical | +| [Wrapping Derived Types: Native Representation](../../docs/user/guide/wrapping-derived-types.md#native-representation) | Supported | opaque accessor storage for ordinary, `bind(C)`, and `sequence`; field get/set; by-value copy; no direct C aggregate access | — | `tests/fortran/derived_types/end_to_end/test_opaque_layout.py::test_bind_c_derived_types_use_accessors_and_fortran_value_copy[source]`
`tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_sequence_derived_value_uses_the_same_typed_opaque_call_path` | — | canonical | +| [Allocatables: Key Concepts](../../docs/user/guide/allocatables.md#key-concepts) | Supported | scalar value versus array handle; allocated, unallocated, and zero-sized states; live views; module, field, result, and caller-created descriptor origins | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]` | — | canonical | +| [Allocatables: When To Use An Allocatable Handle](../../docs/user/guide/allocatables.md#when-to-use-an-allocatable-handle) | Supported | descriptor arguments versus ordinary arrays; present-empty caller handle; dtype/rank compatibility; plain NumPy rejection | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_every_allocatable_handle_kind_reaches_read_only_and_writable_allocatable_dummies` | `tests/fortran/allocatables/end_to_end/test_external_allocatable.py::test_standalone_allocatable_argument_accepts_a_caller_created_handle` | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_descriptor_argument_refuses_a_handle_that_is_not_the_declared_one` | canonical | +| [Allocatables: Allocatable Array Handle API](../../docs/user/guide/allocatables.md#allocatable-array-handle-api) | Supported | default construction; allocated, shape, dtype, rank, `to_numpy`, resize, deallocate, close, and closed state; unavailable-operation gating | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_plain_allocatable_module_array_exposes_current_live_view[source]` | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_operations_outside_the_completed_capabilities_are_refused`
`tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_non_array_descriptor_and_ordinary_array_annotations_are_not_factories` | canonical | +| [Allocatables: Deallocate Versus Close](../../docs/user/guide/allocatables.md#deallocate-versus-close) | Supported | deallocate while open; owned descriptor close; idempotent close; borrowed module/field close no-op; closed-handle rejection | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_owned_handle_destroys_its_descriptor_exactly_once`
`tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_borrowed_handle_close_and_finalizer_do_not_destroy_native_storage` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]` | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_generated_storage_rejects_incompatible_contract_handles` | canonical | +| [Allocatables: Module Variables And Derived Fields](../../docs/user/guide/allocatables.md#module-variables-and-derived-fields) | Supported | native-owned module descriptor; parent-retained component descriptor; stable handle identity; live allocation changes and mutation | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_plain_allocatable_module_array_exposes_current_live_view` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[generated-pyi]` | — | canonical | +| [Allocatables: Function Results](../../docs/user/guide/allocatables.md#function-results) | Supported | owned result descriptor; allocated and zero-sized results; explicit maybe-unallocated result; rank one, rank two, and high-rank policy | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_maybe_unallocated_direct_allocatable_results_preserve_unallocated_state` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_maybe_unallocated_direct_allocatable_results_preserve_unallocated_state` | `tests/fortran/allocatables/policy/test_allocatable_result_policy.py::test_direct_allocatable_scalar_function_result_is_blocked_before_codegen` (`policy`)
`tests/fortran/allocatables/codegen/test_allocatable_lowering.py::test_maybe_unallocated_is_only_valid_on_direct_allocatable_array_results` (`codegen`) | canonical | +| [Allocatables: Output And Inout Arguments](../../docs/user/guide/allocatables.md#output-and-inout-arguments) | Supported | hidden nonoptional output; visible optional output; visible inout; unallocated, allocated, reallocated, and deallocated states; same-object projection | `tests/fortran/allocatables/pipeline/test_allocatable_output_contract_printing.py::test_emit_optional_allocatable_output_as_visible_argument`
`tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_every_allocatable_handle_kind_reaches_read_only_and_writable_allocatable_dummies` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]`
`tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py::test_caller_created_allocatable_crosses_separately_built_extensions` | — | canonical | | [Allocatables: Complete Example](../../docs/user/guide/allocatables.md#complete-example) | Supported | source build; reviewed generated `.pyi`; generated-`.pyi` replay; owned result; same-handle inout replacement; exact visible values | `tests/fortran/allocatables/pipeline/test_generated_allocatable_contract.py::test_generated_allocatable_contract_matches_fixture[fallocatable_views_f90]` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]`
`tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[generated-pyi]` | — | canonical | -| [Allocatables: Safety Checklist](../../docs/user/guide/allocatables.md#safety-checklist) | Supported | allocation checks; independent copies; fresh extraction after reallocation; explicit close; owner-only release; borrowed views and owner retention | `tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py::test_allocatable_to_numpy_explicit_copy_is_independent`
`tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py::test_allocatable_to_numpy_short_circuits_unallocated_state_before_generated_extraction` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_plain_allocatable_module_array_exposes_current_live_view[source]`
`tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]` | `tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py::test_generated_storage_rejects_a_closed_contract_handle` (`runtime`) | canonical | -| [Allocatables: Scalar Allocatables](../../docs/user/guide/allocatables.md#scalar-allocatables) | Supported | module scalar copy; argument input/inout/output; allocated value and unallocated `None`; source build and generated-`.pyi` replay | `tests/fortran/allocatables/semantics/test_pyi_allocatable_semantics.py::test_scalar_allocatable_calls_use_nullable_values_and_explicit_descriptor_projections`
`tests/fortran/allocatables/pipeline/test_generated_allocatable_contract.py::test_generated_allocatable_contract_matches_fixture[fscalar_allocatables_f90]` | `tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py::test_scalar_allocatables_project_values_and_unallocated_state[source]`
`tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py::test_scalar_allocatables_project_values_and_unallocated_state[generated-pyi]` | `tests/fortran/allocatables/semantics/test_pyi_allocatable_semantics.py::test_scalar_allocatable_calls_reject_descriptor_wrappers_as_python_values[from prik.contracts import Allocatable, Float64\ndef consume(value: Allocatable[Float64]) -> None: ...\n-Procedure scalar descriptors use nullable value annotations]` (`semantics`) | canonical | -| [Memory Management: The Python Object And Its Storage](../../docs/user/guide/memory-management.md#the-python-object-and-its-storage) | Supported | wrapper-owned result descriptor; native-owned module descriptor; parent-retained component; borrowed live view; caller-created descriptor; edited ownership policy; one release owner | `tests/fortran/memory_management/policy/test_memory_ownership_policy.py::test_contradictory_ownership_contract_fails_before_lowering`
`tests/fortran/allocatables/policy/test_allocatable_handle_policy.py::test_owned_allocatable_result_records_local_standard_c_descriptor_build_requirement` | `tests/fortran/memory_management/end_to_end/test_explicit_borrowed_owner.py::test_wrapper_owned_borrow_keeps_owner_alive_and_finalizes_exactly_once`
`tests/fortran/allocatables/end_to_end/test_edited_ownership.py::test_explicit_handle_ownership_uses_native_wrapper_and_result_lifetimes`
`tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py::test_allocatable_replacement_has_no_native_memory_errors[source]` | `tests/fortran/memory_management/policy/test_memory_ownership_policy.py::test_contradictory_ownership_contract_fails_before_lowering` (`policy`) | canonical | -| [Pointers: Key Concepts](../../docs/user/guide/pointers.md#key-concepts) | Supported | descriptor versus target ownership; scalar values versus array handles; association; live views; reassociation; close | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_hidden_pointer_handle_output_owns_descriptor_but_not_target_policy`
`tests/fortran/pointers/semantics/test_pointer_semantics.py::test_pyi_pointer_handles_preserve_rank_optionality_and_scalar_state` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | — | canonical | -| [Pointers: When To Use A Pointer Handle](../../docs/user/guide/pointers.md#when-to-use-a-pointer-handle) | Supported | descriptor parameter; ordinary array parameter; associated handle array actual; plain NumPy descriptor rejection; contiguity validation | `tests/fortran/pointers/runtime/test_pointer_array_actual_abi.py::test_pointer_array_actual_hook_requires_associated_state_without_to_numpy` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_descriptor_binding_helper_rejects_plain_arrays_none_and_wrong_kind` (`runtime`) | canonical | -| [Pointers: Pointer Array Handle API](../../docs/user/guide/pointers.md#pointer-array-handle-api) | Supported | default construction; association; shape; dtype; rank; live view; associate; nullify; allocate; deallocate; resize; close; unavailable operations | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_contract_default_handle_constructors_preserve_dtype_rank_and_empty_state`
`tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_handle_uses_common_base_and_nullify_operation` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_handle_tracks_native_output_association` | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_allocation_operations_are_policy_gated_by_ops_table` (`runtime`)
`tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_non_array_descriptor_and_ordinary_array_annotations_are_not_factories` (`runtime`) | canonical | -| [Pointers: Associate Two Pointers](../../docs/user/guide/pointers.md#associate-two-pointers) | Supported | same dtype/rank; associated and unassociated source; no copy; independent descriptor state; cross-extension ABI | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_associate_accepts_reassociation_and_an_unassociated_source`
`tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_generated_pointer_associate_packs_standard_descriptor_facts` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_crosses_separately_built_extensions` | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_associate_rejects_incompatible_sources[other1-TypeError-dtype]` (`runtime`)
`tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_associate_rejects_incompatible_sources[other2-ValueError-rank]` (`runtime`) | canonical | -| [Pointers: Nullify, Deallocate, And Close](../../docs/user/guide/pointers.md#nullify-deallocate-and-close) | Supported | association release; policy-gated target release; owned descriptor release; borrowed close no-op; closed state | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_pointer_policy_unsafe_deallocate_is_explicit_operation_opt_in`
`tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_allocation_operations_route_when_policy_ops_exist` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_pointer_association_rejects_closed_handles` (`runtime`) | canonical | -| [Pointers: Module Variables And Derived Fields](../../docs/user/guide/pointers.md#module-variables-and-derived-fields) | Supported | native module owner; parent-retained component; stable handle identity; live native reassociation; getter-only field | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_plain_pointer_array_container_policy_completes_default_handle_profile`
`tests/fortran/pointers/codegen/test_pointer_lowering.py::test_pointer_plans_complete_descriptor_ownership_and_operations_before_lowering` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_slice_shape_strides_and_parent_lifetime` | — | canonical | -| [Pointers: Function Results](../../docs/user/guide/pointers.md#function-results) | Supported | associated and unassociated result; wrapper-owned persistent descriptor; borrowed target; explicit close | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_hidden_pointer_handle_output_owns_descriptor_but_not_target_policy`
`tests/fortran/pointers/codegen/test_pointer_lowering.py::test_pointer_lowering_assigns_descriptors_without_target_deallocation` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[source]`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[generated-pyi]` | — | canonical | -| [Pointers: Output And Inout Arguments](../../docs/user/guide/pointers.md#output-and-inout-arguments) | Supported | hidden nonoptional output; visible inout; present unassociated caller descriptor; same-object update | `tests/fortran/pointers/codegen/test_pointer_lowering.py::test_pointer_plans_complete_descriptor_ownership_and_operations_before_lowering` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_handle_tracks_native_output_association`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | — | canonical | +| [Allocatables: Safety Checklist](../../docs/user/guide/allocatables.md#safety-checklist) | Supported | allocation checks; independent copies; fresh extraction after reallocation; explicit close; owner-only release; borrowed views and owner retention | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_plain_allocatable_module_array_exposes_current_live_view`
`tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_maybe_unallocated_direct_allocatable_results_preserve_unallocated_state` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_plain_allocatable_module_array_exposes_current_live_view[source]`
`tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]` | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_generated_storage_rejects_incompatible_contract_handles` | canonical | +| [Allocatables: Scalar Allocatables](../../docs/user/guide/allocatables.md#scalar-allocatables) | Supported | module scalar copy; argument input/inout/output; allocated value and unallocated `None`; source build and generated-`.pyi` replay | `tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py::test_scalar_allocatables_project_values_and_unallocated_state`
`tests/fortran/allocatables/pipeline/test_generated_allocatable_contract.py::test_generated_allocatable_contract_matches_fixture` | `tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py::test_scalar_allocatables_project_values_and_unallocated_state[source]`
`tests/fortran/allocatables/end_to_end/test_scalar_allocatables.py::test_scalar_allocatables_project_values_and_unallocated_state[generated-pyi]` | `tests/fortran/allocatables/semantics/test_pyi_allocatable_semantics.py::test_scalar_allocatable_calls_reject_descriptor_wrappers_as_python_values[from prik.contracts import Allocatable, Float64\ndef consume(value: Allocatable[Float64]) -> None: ...\n-Procedure scalar descriptors use nullable value annotations]` (`semantics`) | canonical | +| [Memory Management: The Python Object And Its Storage](../../docs/user/guide/memory-management.md#the-python-object-and-its-storage) | Supported | wrapper-owned result descriptor; native-owned module descriptor; parent-retained component; borrowed live view; caller-created descriptor; edited ownership policy; one release owner | `tests/fortran/memory_management/policy/test_memory_ownership_policy.py::test_contradictory_ownership_contract_fails_before_lowering`
`tests/fortran/memory_management/codegen/test_native_handle_planning.py::test_native_handle_plans_select_one_descriptor_abi_and_release_owner_per_origin` | `tests/fortran/memory_management/end_to_end/test_explicit_borrowed_owner.py::test_wrapper_owned_borrow_keeps_owner_alive_and_finalizes_exactly_once`
`tests/fortran/allocatables/end_to_end/test_edited_ownership.py::test_explicit_handle_ownership_uses_native_wrapper_and_result_lifetimes`
`tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py::test_allocatable_replacement_has_no_native_memory_errors[source]` | `tests/fortran/memory_management/policy/test_memory_ownership_policy.py::test_contradictory_ownership_contract_fails_before_lowering` (`policy`) | canonical | +| [Pointers: Key Concepts](../../docs/user/guide/pointers.md#key-concepts) | Supported | descriptor versus target ownership; scalar values versus array handles; association; live views; reassociation; close | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | — | canonical | +| [Pointers: When To Use A Pointer Handle](../../docs/user/guide/pointers.md#when-to-use-a-pointer-handle) | Supported | descriptor parameter; ordinary array parameter; associated handle array actual; plain NumPy descriptor rejection; contiguity validation | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_descriptor_argument_refuses_a_handle_that_is_not_the_declared_one` | canonical | +| [Pointers: Pointer Array Handle API](../../docs/user/guide/pointers.md#pointer-array-handle-api) | Supported | default construction; association; shape; dtype; rank; live view; associate; nullify; allocate; deallocate; resize; close; unavailable operations | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_native_array_handles_and_caller_created_pointers_use_canonical_plan` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_native_array_handles_and_caller_created_pointers_use_canonical_plan` | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_operations_outside_the_completed_capabilities_are_refused` | canonical | +| [Pointers: Associate Two Pointers](../../docs/user/guide/pointers.md#associate-two-pointers) | Supported | same dtype/rank; associated and unassociated source; no copy; independent descriptor state; cross-extension ABI | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_crosses_separately_built_extensions` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_crosses_separately_built_extensions` | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_pointer_associate_rejects_incompatible_sources` | canonical | +| [Pointers: Nullify, Deallocate, And Close](../../docs/user/guide/pointers.md#nullify-deallocate-and-close) | Supported | association release; policy-gated target release; owned descriptor release; borrowed close no-op; closed state | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_pointer_policy_unsafe_deallocate_is_explicit_operation_opt_in`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_pointer_association_rejects_closed_and_unattached_character_handles` | canonical | +| [Pointers: Module Variables And Derived Fields](../../docs/user/guide/pointers.md#module-variables-and-derived-fields) | Supported | native module owner; parent-retained component; stable handle identity; live native reassociation; getter-only field | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_pointer_container_ownership_is_fixed_by_its_native_parent` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | — | canonical | +| [Pointers: Function Results](../../docs/user/guide/pointers.md#function-results) | Supported | associated and unassociated result; wrapper-owned persistent descriptor; borrowed target; explicit close | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[source]`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[generated-pyi]` | — | canonical | +| [Pointers: Output And Inout Arguments](../../docs/user/guide/pointers.md#output-and-inout-arguments) | Supported | hidden nonoptional output; visible inout; present unassociated caller descriptor; same-object update | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_native_array_handles_and_caller_created_pointers_use_canonical_plan` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_native_array_handles_and_caller_created_pointers_use_canonical_plan`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association` | — | canonical | | [Pointers: Complete Module Example](../../docs/user/guide/pointers.md#complete-module-example) | Supported | source generation; reviewed generated `.pyi`; source and generated-`.pyi` builds; module handle; descriptor call; nullification | `tests/fortran/pointers/pipeline/test_generated_pointer_contract.py::test_generated_pointer_contract_matches_fixture[fpointers_f90]` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[generated-pyi]` | — | canonical | -| [Pointers: Contiguous And Strided Targets](../../docs/user/guide/pointers.md#contiguous-and-strided-targets) | Supported | decoded descriptor shape and strides; negative strides; live mutation; descriptor parameter; ordinary contiguous-array blocker | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_pointer_c_descriptor_helper_builds_strided_numpy_view_from_decoded_fields`
`tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_pointer_c_descriptor_helper_builds_negative_stride_numpy_view_from_decoded_fields` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_slice_shape_strides_and_parent_lifetime` | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_to_numpy_contiguous_view_policy_rejects_non_contiguous_storage` (`runtime`) | canonical | -| [Pointers: Check Association And Lifetime](../../docs/user/guide/pointers.md#check-association-and-lifetime) | Partially supported | unassociated short circuit; owner retention; descriptor state; external target lifetime remains native responsibility | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_to_numpy_short_circuits_unassociated_state_before_unsupported_policy` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_slice_shape_strides_and_parent_lifetime` | — | canonical | -| [Pointers: Do Not Return A Pointer To Expired Local Storage](../../docs/user/guide/pointers.md#do-not-return-a-pointer-to-expired-local-storage) | Partially supported | owned descriptor does not imply owned target; target lifetime policy; native expired-target prevention remains application responsibility | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_hidden_pointer_handle_output_owns_descriptor_but_not_target_policy` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[source]` | — | canonical | -| [Pointers: Copy Or Discard Views Before Target Changes](../../docs/user/guide/pointers.md#copy-or-discard-views-before-target-changes) | Supported | live shared view; explicit independent copy; mutation; reassociation and nullification state | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_to_numpy_descriptor_view_policy_never_copies_storage` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_slice_shape_strides_and_parent_lifetime` | — | canonical | -| [Pointers: Deallocate Only What This Pointer Allocated](../../docs/user/guide/pointers.md#deallocate-only-what-this-pointer-allocated) | Supported | default operation set; unsafe-deallocate opt-in; allocation/deallocation/resize routing; unavailable-operation diagnostic | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_pointer_policy_unsafe_deallocate_is_explicit_operation_opt_in`
`tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_allocation_operations_route_when_policy_ops_exist` | — | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_allocation_operations_are_policy_gated_by_ops_table` (`runtime`) | canonical | -| [Pointers: Nullifying One Pointer Does Not Change Other Pointers](../../docs/user/guide/pointers.md#nullifying-one-pointer-does-not-change-other-pointers) | Supported | descriptor association copied by value; source nullification; target remains associated | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_fresh_pointer_associate_copies_association_without_following_source_descriptor` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_crosses_separately_built_extensions` | — | canonical | -| [Pointers: Do Not Keep Using A Closed Handle](../../docs/user/guide/pointers.md#do-not-keep-using-a-closed-handle) | Supported | owned close; closed state; closed source and destination rejection; target not deallocated | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_pointer_association_rejects_closed_handles` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[source]` | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_pointer_association_rejects_closed_handles` (`runtime`) | canonical | -| [Pointers: Respect Contiguity Requirements](../../docs/user/guide/pointers.md#respect-contiguity-requirements) | Supported | strided descriptor view; contiguous-view policy; descriptor handoff versus array-actual handoff | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_pointer_descriptor_view_policy_uses_decoded_descriptor_fields_for_strided_view` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_to_numpy_contiguous_view_policy_rejects_non_contiguous_storage` (`runtime`) | canonical | -| [Pointers: Synchronize Target Changes](../../docs/user/guide/pointers.md#synchronize-target-changes) | Partially supported | live descriptor state and views; no implicit copy or lock; synchronization remains application responsibility | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_to_numpy_descriptor_view_policy_never_copies_storage` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | — | canonical | -| [Pointers: Scalar Pointers](../../docs/user/guide/pointers.md#scalar-pointers) | Supported | module snapshot; input/inout/output; nullable associated state; copied result; no handle API; source/generated-`.pyi` parity | `tests/fortran/pointers/semantics/test_pointer_semantics.py::test_fortran_pointer_arrays_and_scalars_preserve_descriptor_semantics`
`tests/fortran/pointers/semantics/test_pointer_semantics.py::test_pyi_pointer_handles_preserve_rank_optionality_and_scalar_state` | `tests/fortran/pointers/end_to_end/test_scalar_pointers.py::test_scalar_pointers_project_nullable_copied_values[source]`
`tests/fortran/pointers/end_to_end/test_scalar_pointers.py::test_scalar_pointers_project_nullable_copied_values[generated-pyi]` | `tests/fortran/pointers/semantics/test_pointer_semantics.py::test_scalar_pointer_results_reject_legacy_descriptor_spellings[def produce() -> Pointer[Float64]: ...\n-Procedure scalar descriptor results use a nullable value annotation]` (`semantics`) | canonical | +| [Pointers: Contiguous And Strided Targets](../../docs/user/guide/pointers.md#contiguous-and-strided-targets) | Supported | decoded descriptor shape and strides; negative strides; live mutation; descriptor parameter; ordinary contiguous-array blocker | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_native_array_handles_and_caller_created_pointers_use_canonical_plan` | canonical | +| [Pointers: Check Association And Lifetime](../../docs/user/guide/pointers.md#check-association-and-lifetime) | Partially supported | unassociated short circuit; owner retention; descriptor state; external target lifetime remains native responsibility | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | — | canonical | +| [Pointers: Do Not Return A Pointer To Expired Local Storage](../../docs/user/guide/pointers.md#do-not-return-a-pointer-to-expired-local-storage) | Partially supported | owned descriptor does not imply owned target; target lifetime policy; native expired-target prevention remains application responsibility | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[source]` | — | canonical | +| [Pointers: Copy Or Discard Views Before Target Changes](../../docs/user/guide/pointers.md#copy-or-discard-views-before-target-changes) | Supported | live shared view; explicit independent copy; mutation; reassociation and nullification state | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | — | canonical | +| [Pointers: Deallocate Only What This Pointer Allocated](../../docs/user/guide/pointers.md#deallocate-only-what-this-pointer-allocated) | Supported | default operation set; unsafe-deallocate opt-in; allocation/deallocation/resize routing; unavailable-operation diagnostic | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_pointer_policy_unsafe_deallocate_is_explicit_operation_opt_in` | — | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_operations_outside_the_completed_capabilities_are_refused` | canonical | +| [Pointers: Nullifying One Pointer Does Not Change Other Pointers](../../docs/user/guide/pointers.md#nullifying-one-pointer-does-not-change-other-pointers) | Supported | descriptor association copied by value; source nullification; target remains associated | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_crosses_separately_built_extensions` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_crosses_separately_built_extensions` | — | canonical | +| [Pointers: Do Not Keep Using A Closed Handle](../../docs/user/guide/pointers.md#do-not-keep-using-a-closed-handle) | Supported | owned close; closed state; closed source and destination rejection; target not deallocated | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_pointer_association_rejects_closed_and_unattached_character_handles` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[source]` | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_pointer_association_rejects_closed_and_unattached_character_handles` | canonical | +| [Pointers: Respect Contiguity Requirements](../../docs/user/guide/pointers.md#respect-contiguity-requirements) | Supported | strided descriptor view; contiguous-view policy; descriptor handoff versus array-actual handoff | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_native_array_handles_and_caller_created_pointers_use_canonical_plan` | canonical | +| [Pointers: Synchronize Target Changes](../../docs/user/guide/pointers.md#synchronize-target-changes) | Partially supported | live descriptor state and views; no implicit copy or lock; synchronization remains application responsibility | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | — | canonical | +| [Pointers: Scalar Pointers](../../docs/user/guide/pointers.md#scalar-pointers) | Supported | module live read-only view; input/inout/output; nullable associated state; copied result; no handle API; source/generated-`.pyi` parity | `tests/fortran/pointers/end_to_end/test_scalar_pointers.py::test_scalar_pointers_project_nullable_native_views` | `tests/fortran/pointers/end_to_end/test_scalar_pointers.py::test_scalar_pointers_project_nullable_native_views` | `tests/fortran/pointers/semantics/test_pointer_semantics.py::test_scalar_pointer_results_require_a_nullable_value_annotation` | canonical | | [`.pyi`: Pointer Array Handles](../../docs/user/reference/pyi-format.md#pointer-array-handles) | Supported | `Pointer[T[...]]`; optional absence; ten-fact `PointerPolicy`; operations; build requirements; result descriptor storage; strided extraction | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_complete_pointer_policy_metadata_round_trips_without_overriding_container_ownership`
`tests/fortran/pointers/pipeline/test_pointer_build_manifest.py::test_pyi_manifest_records_pointer_descriptor_interop_requirements` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[generated-pyi]` | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_pointer_policy_metadata_requires_every_fact` (`policy`) | canonical | -| [Pointers: Key Concepts](../../docs/user/guide/pointers.md#key-concepts) | Partially supported | scalar and array pointer boundaries; descriptor and target ownership; module/field handles; results; outputs; strided views; explicit policy blockers | `tests/fortran/pointers/codegen/test_pointer_lowering.py::test_pointer_lowering_assigns_descriptors_without_target_deallocation`
`tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_native_array_handle_build_requirements_are_selected_from_completed_policy` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]`
`tests/fortran/pointers/end_to_end/test_scalar_pointers.py::test_scalar_pointers_project_nullable_copied_values[source]` | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_allocation_operations_are_policy_gated_by_ops_table` (`runtime`) | canonical | -| [Memory Management: The Python Object And Its Storage](../../docs/user/guide/memory-management.md#the-python-object-and-its-storage) | Supported | Python object ownership; Python, native, wrapper, parent, descriptor, and target storage owners | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_runtime_handle_classes_are_public_api_exports`
`tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_generated_handle_factory_adapts_private_operations_to_runtime_protocol` | `tests/fortran/allocatables/end_to_end/test_edited_ownership.py::test_explicit_handle_ownership_uses_native_wrapper_and_result_lifetimes` | — | canonical | -| [Memory Management: Live Views And Copies](../../docs/user/guide/memory-management.md#live-views-and-copies) | Supported | live mutation; explicit detached copy; fresh extraction; reallocation and reassociation boundaries | `tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py::test_allocatable_to_numpy_explicit_copy_is_independent`
`tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_to_numpy_descriptor_view_policy_never_copies_storage` | `tests/fortran/derived_types/end_to_end/test_derived_runtime_mechanisms.py::test_plain_module_derived_proxy_reads_and_writes_live_members`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_slice_shape_strides_and_parent_lifetime` | — | canonical | -| [Memory Management: Allocatables And Pointers](../../docs/user/guide/memory-management.md#allocatables-and-pointers) | Supported | allocated versus associated; live view; allocation release; nullification; descriptor close; pointer target separation | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_generated_owned_handle_factory_passes_persistent_owner_to_every_operation` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_allocation_operations_are_policy_gated_by_ops_table` (`runtime`) | canonical | -| [Memory Management: Closing Handles](../../docs/user/guide/memory-management.md#closing-handles) | Supported | owned close; idempotence; finalizer; failed destruction; construction rollback; closed-use rejection; borrowed close no-op | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_owned_handle_close_calls_destroy_once_and_blocks_later_use`
`tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_owned_handle_finalizer_calls_destroy_once`
`tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_borrowed_handle_close_and_finalizer_do_not_destroy_native_storage` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[source]` | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_owned_handle_construction_requires_generated_destroy_operation` (`runtime`) | canonical | -| [Memory Management: Sharing Handles Between Extensions](../../docs/user/guide/memory-management.md#sharing-handles-between-extensions) | Supported | same descriptor kind, dtype, and rank; no-copy handoff; compatible runtime record; target lifetime unchanged | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_generated_storage_rejects_incompatible_contract_handles[-pointer-float64-1-TypeError-cannot attach pointer descriptor storage]` | `tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py::test_caller_created_allocatable_crosses_separately_built_extensions`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_crosses_separately_built_extensions` | `tests/fortran/pointers/runtime/test_pointer_handle_protocol.py::test_pointer_associate_rejects_incompatible_sources[other1-TypeError-dtype]` (`runtime`) | canonical | -| [Memory Management: Passing Objects To Functions](../../docs/user/guide/memory-management.md#passing-objects-to-functions) | Supported | caller ownership preserved; same-array mutation; same allocatable handle replacement; same pointer handle reassociation | `tests/fortran/arrays/codegen/test_array_output_identity.py::test_projected_array_identity_uses_one_completed_in_place_copy_out_action` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_handle_tracks_native_output_association` | — | canonical | -| [Memory Management: Derived Objects And Fields](../../docs/user/guide/memory-management.md#derived-objects-and-fields) | Supported | wrapper-owned instance; native-owned module object; parent-retained field; exactly-once finalization; user-defined `FINAL`; stale storage warning | `tests/fortran/memory_management/semantics/test_memory_contract_semantics.py::test_convert_pyi_to_ir_rejects_immutable_writable_borrowed_view_argument` | `tests/fortran/memory_management/end_to_end/test_explicit_borrowed_owner.py::test_wrapper_owned_borrow_keeps_owner_alive_and_finalizes_exactly_once`
`tests/fortran/derived_types/end_to_end/test_derived_runtime_mechanisms.py::test_borrowed_child_retains_owner_and_finalizes_exactly_once`
`tests/fortran/derived_types/end_to_end/test_default_constructors_and_finalizers.py::test_fortran_default_constructor_keywords_and_finalization[source]` | — | canonical | -| [Memory Management: Passing Objects To Functions](../../docs/user/guide/memory-management.md#passing-objects-to-functions) | Supported | wrapper and module origins; absent proxies; direct and scoped address; reversible `move_alloc`; pointer transaction; rollback and exactly-once restoration | `tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_generated_artifacts_keep_matrix_dispatch_linear_and_descriptor_free` | `tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_move_alloc_round_trip_preserves_target_association`
`tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_module_descriptor_transactions_preserve_empty_and_recreated_state`
`tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_later_acquisition_failure_rolls_back_earlier_origins` | `tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_reassociable_pointer_dummy_requires_pointer_storage[module_allocatable]` (`runtime`) | canonical | -| [Memory Management: Safety Checklist](../../docs/user/guide/memory-management.md#safety-checklist) | Supported | state checks; live views; explicit copies; owner-only deallocation; nullification; closed handles; caller synchronization responsibility | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_owned_handle_close_marks_closed_when_destroy_raises`
`tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py::test_allocatable_to_numpy_short_circuits_unallocated_state_before_generated_extraction` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_plain_allocatable_module_array_exposes_current_live_view[source]` | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_pointer_association_rejects_closed_handles` (`runtime`) | canonical | +| [Pointers: Key Concepts](../../docs/user/guide/pointers.md#key-concepts) | Partially supported | scalar and array pointer boundaries; descriptor and target ownership; module/field handles; results; outputs; strided views; explicit policy blockers | `tests/fortran/pointers/policy/test_pointer_ownership_policy.py::test_native_array_handle_build_requirements_are_selected_from_completed_policy` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association`
`tests/fortran/pointers/end_to_end/test_scalar_pointers.py::test_scalar_pointers_project_nullable_native_views` | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_operations_outside_the_completed_capabilities_are_refused` | canonical | +| [Memory Management: The Python Object And Its Storage](../../docs/user/guide/memory-management.md#the-python-object-and-its-storage) | Supported | Python object ownership; Python, native, wrapper, parent, descriptor, and target storage owners | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_owned_handle_destroys_its_descriptor_exactly_once` | `tests/fortran/allocatables/end_to_end/test_edited_ownership.py::test_explicit_handle_ownership_uses_native_wrapper_and_result_lifetimes` | — | canonical | +| [Memory Management: Live Views And Copies](../../docs/user/guide/memory-management.md#live-views-and-copies) | Supported | live mutation; explicit detached copy; fresh extraction; reallocation and reassociation boundaries | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_plain_allocatable_module_array_exposes_current_live_view`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | `tests/fortran/derived_types/end_to_end/test_derived_runtime_mechanisms.py::test_plain_module_derived_proxy_reads_and_writes_live_members`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime` | — | canonical | +| [Memory Management: Allocatables And Pointers](../../docs/user/guide/memory-management.md#allocatables-and-pointers) | Supported | allocated versus associated; live view; allocation release; nullification; descriptor close; pointer target separation | `tests/fortran/memory_management/codegen/test_native_handle_planning.py::test_native_handle_plans_select_one_descriptor_abi_and_release_owner_per_origin` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[source]`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_and_derived_pointer_handles_track_native_association[source]` | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_operations_outside_the_completed_capabilities_are_refused` | canonical | +| [Memory Management: Closing Handles](../../docs/user/guide/memory-management.md#closing-handles) | Supported | owned close; idempotence; finalizer; failed destruction; construction rollback; closed-use rejection; borrowed close no-op | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_owned_handle_destroys_its_descriptor_exactly_once`
`tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_borrowed_handle_close_and_finalizer_do_not_destroy_native_storage` | `tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_pointer_array_results_use_owned_descriptors_without_owning_targets[source]` | `tests/fortran/memory_management/runtime/test_handle_lifecycle.py::test_handle_construction_rejects_an_incomplete_generated_contract` | canonical | +| [Memory Management: Sharing Handles Between Extensions](../../docs/user/guide/memory-management.md#sharing-handles-between-extensions) | Supported | same descriptor kind, dtype, and rank; no-copy handoff; compatible runtime record; target lifetime unchanged | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_generated_storage_rejects_incompatible_contract_handles[-pointer-float64-1-TypeError-cannot attach pointer descriptor storage]` | `tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py::test_caller_created_allocatable_crosses_separately_built_extensions`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_caller_created_pointer_crosses_separately_built_extensions` | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_generated_storage_rejects_incompatible_contract_handles` | canonical | +| [Memory Management: Passing Objects To Functions](../../docs/user/guide/memory-management.md#passing-objects-to-functions) | Supported | caller ownership preserved; same-array mutation; same allocatable handle replacement; same pointer handle reassociation | — | `tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_every_actual_row_meets_every_dummy_form`
`tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_every_allocatable_handle_kind_reaches_read_only_and_writable_allocatable_dummies`
`tests/fortran/pointers/end_to_end/test_pointer_handles.py::test_module_native_array_handles_and_caller_created_pointers_use_canonical_plan` | — | canonical | +| [Memory Management: Derived Objects And Fields](../../docs/user/guide/memory-management.md#derived-objects-and-fields) | Supported | wrapper-owned instance; native-owned module object; parent-retained field; exactly-once finalization; user-defined `FINAL`; stale storage warning | `tests/fortran/memory_management/semantics/test_memory_contract_semantics.py::test_convert_pyi_to_ir_rejects_immutable_writable_borrowed_view_argument` | `tests/fortran/memory_management/end_to_end/test_explicit_borrowed_owner.py::test_wrapper_owned_borrow_keeps_owner_alive_and_finalizes_exactly_once`
`tests/fortran/derived_types/end_to_end/test_borrowed_components.py::test_borrowed_child_wrapper_never_finalizes_native_component[source]`
`tests/fortran/derived_types/end_to_end/test_default_constructors_and_finalizers.py::test_fortran_default_constructor_keywords_and_finalization[source]` | — | canonical | +| [Memory Management: Passing Objects To Functions](../../docs/user/guide/memory-management.md#passing-objects-to-functions) | Supported | wrapper and module origins; absent proxies; direct and scoped address; reversible `move_alloc`; pointer transaction; rollback and exactly-once restoration | `tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_generated_artifacts_keep_matrix_dispatch_linear_and_descriptor_free` | `tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_move_alloc_round_trip_preserves_target_association`
`tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_module_descriptor_transactions_preserve_empty_and_recreated_state`
`tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_later_acquisition_failure_rolls_back_earlier_origins` | `tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py::test_every_actual_row_meets_every_dummy_form[module_allocatable]` (`runtime`) | canonical | +| [Memory Management: Safety Checklist](../../docs/user/guide/memory-management.md#safety-checklist) | Supported | state checks; live views; explicit copies; owner-only deallocation; nullification; closed handles; caller synchronization responsibility | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_maybe_unallocated_direct_allocatable_results_preserve_unallocated_state` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_plain_allocatable_module_array_exposes_current_live_view[source]` | `tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py::test_pointer_association_rejects_closed_and_unattached_character_handles` | canonical | | [`.pyi`: Ownership, Transfer, And Destruction Policies](../../docs/user/reference/pyi-format.md#ownership-transfer-and-destruction-policies) | Supported | explicit ownership triple; by-value, call-local, in-place, copy-return, snapshot, borrowed-view, wrapper-instance, and blocked policy families; release responsibility | `tests/fortran/memory_management/policy/test_memory_ownership_policy.py::test_scalar_storage_rejects_incompatible_explicit_ownership_metadata`
`tests/fortran/memory_management/policy/test_memory_ownership_policy.py::test_contradictory_ownership_contract_fails_before_lowering` | `tests/fortran/allocatables/end_to_end/test_edited_ownership.py::test_explicit_handle_ownership_uses_native_wrapper_and_result_lifetimes`
`tests/fortran/memory_management/end_to_end/test_explicit_borrowed_owner.py::test_wrapper_owned_borrow_keeps_owner_alive_and_finalizes_exactly_once` | `tests/fortran/memory_management/semantics/test_memory_contract_semantics.py::test_convert_pyi_to_ir_rejects_immutable_writable_borrowed_view_argument` (`semantics`)
`tests/fortran/memory_management/policy/test_memory_ownership_policy.py::test_contradictory_ownership_contract_fails_before_lowering` (`policy`) | canonical | -| [Callbacks: The Short Version](../../docs/user/guide/callbacks.md#the-short-version) | Supported | primitive value and reference scalars; arrays; fixed strings; derived references and values | `tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_uses_value_default_and_explicit_reference_callbacks`
`tests/fortran/callbacks/codegen/test_callback_planning.py::test_callback_policy_completes_value_default_and_explicit_reference_before_planning` | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]`
`tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[generated-pyi]` | — | canonical | +| [Callbacks: The Short Version](../../docs/user/guide/callbacks.md#the-short-version) | Supported | primitive value and reference scalars; arrays; fixed strings; derived references and values | `tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_uses_value_default_and_explicit_reference_callbacks` | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]`
`tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[generated-pyi]` | — | canonical | | [Callbacks: What The Callable Sees](../../docs/user/guide/callbacks.md#what-the-callable-sees) | Supported | prototype boundary; outer native-call projection; exact NumPy scalar conversion | `tests/fortran/callbacks/semantics/test_fortran_callback_semantics.py::test_dummy_procedure_interfaces_become_complete_callable_contracts` | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` | — | canonical | -| [Callbacks: Small Example](../../docs/user/guide/callbacks.md#small-example) | Supported | immediate scalar reference callback; exact scalar result; source and generated-`.pyi` replay | `tests/fortran/callbacks/pipeline/test_generated_callback_contracts.py::test_callback_generated_pyi_contract_matches_fixture[fcallback_scalar_f90]` | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]`
`tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[generated-pyi]` | — | canonical | -| [Callbacks: Choosing The Prototype Spelling](../../docs/user/guide/callbacks.md#choosing-the-prototype-spelling) | Supported | native value/reference ABI; named arguments; shape dependencies; imported prototypes | `tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_uses_value_default_and_explicit_reference_callbacks`
`tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_preserves_prototype_argument_names_and_dimensions`
`tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_imported_prototype_resolves_as_module_interface_definition` | `tests/fortran/callbacks/end_to_end/test_array_callbacks.py::test_immediate_dummy_procedure_converts_array_arguments_and_results[source]` | `tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_rejects_invalid_prototype_address_wrappers[Addr(String[8])]` (`semantics`)
`tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_rejects_redundant_or_invalid_prototype_value_wrappers[Value(Float64)]` (`semantics`) | canonical | -| [Callbacks: Key Rules](../../docs/user/guide/callbacks.md#key-rules) | Supported | call scope; same-thread nested entry; exact scalar result; reference cleanup; live array and derived storage | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_callback_artifacts_use_linear_context_adapter_and_trampoline_paths` | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]`
`tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]` | — | canonical | -| [Callbacks: Important Limitations](../../docs/user/guide/callbacks.md#important-limitations) | Blocked | persistent, optional, asynchronous, and cross-thread callbacks; optional or descriptor prototype forms; fatal exceptions and invalid returns | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_optional_callback_retains_one_exact_policy_blocker` | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` | `tests/fortran/callbacks/policy/test_callback_policy.py::test_callback_descriptor_and_optional_forms_are_blocked_before_codegen[def callback_shape(value: Allocatable[Float64]) -> None: ...-callback argument 'value' uses unsupported allocatable, pointer, polymorphic, or assumed-type storage]` (`policy`)
`tests/fortran/callbacks/policy/test_callback_policy.py::test_callback_descriptor_and_optional_forms_are_blocked_before_codegen[def callback_shape(value: Float64 = ...) -> None: ...-callback argument 'value' cannot be optional]` (`policy`) | canonical | -| [Callbacks: Key Rules](../../docs/user/guide/callbacks.md#key-rules) | Supported | completed callback ABI, conversion, lifecycle, adapter symbols, bridge declaration, source/generated parity | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_callback_plan_projects_one_explicit_site_and_stable_roles_per_argument`
`tests/fortran/callbacks/codegen/test_callback_planning.py::test_every_callback_uses_the_shared_generated_abstract_prototype` | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]`
`tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[generated-pyi]` | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_optional_callback_retains_one_exact_policy_blocker` (`codegen`) | canonical | -| [Feature Matrix: Immediate Call-Scoped Python Callbacks](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | scalar, array, string, and derived conversions; entering thread; source/generated parity | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_callback_artifacts_use_linear_context_adapter_and_trampoline_paths` | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]` | — | canonical | -| [Feature Matrix: Persistent Callbacks And Procedure Pointers](../../docs/user/language-support/feature-matrix.md#unsupported-or-blocked-forms) | Blocked | stored or post-call invocation; call-scoped context | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_callback_plan_projects_one_explicit_site_and_stable_roles_per_argument` | — | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_optional_callback_retains_one_exact_policy_blocker` (`codegen`) | canonical | -| [Enumerations: Complete Example](../../docs/user/guide/enumerations.md#complete-example) | Supported | `enum, bind(C)`; explicit, implicit, negative, and symbolic values; source generation; reviewed contract; source/generated replay | `tests/fortran/enumerations/parsing/test_enum_syntax.py::test_valid_enum_subunit_accepts_optional_separator_and_multiple_enumerators`
`tests/fortran/enumerations/pipeline/test_generated_enum_contract.py::test_generated_enum_contract_matches_reviewed_package` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]`
`tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[generated-pyi]` | — | canonical | +| [Callbacks: Small Example](../../docs/user/guide/callbacks.md#small-example) | Supported | immediate scalar reference callback; exact scalar result; source and generated-`.pyi` replay | — | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]`
`tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[generated-pyi]` | — | canonical | +| [Callbacks: Choosing The Prototype Spelling](../../docs/user/guide/callbacks.md#choosing-the-prototype-spelling) | Supported | native value/reference ABI; named arguments; shape dependencies; imported prototypes | `tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_uses_value_default_and_explicit_reference_callbacks`
`tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_imported_prototype_resolves_as_module_interface_definition` | `tests/fortran/callbacks/end_to_end/test_array_callbacks.py::test_immediate_dummy_procedure_converts_array_arguments_and_results[source]` | `tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_rejects_invalid_prototype_address_wrappers[Addr(String[8])]` (`semantics`)
`tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_rejects_redundant_or_invalid_prototype_value_wrappers[Value(Float64)]` (`semantics`) | canonical | +| [Callbacks: Key Rules](../../docs/user/guide/callbacks.md#key-rules) | Supported | call scope; same-thread nested entry; exact scalar result; reference cleanup; live array and derived storage | — | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]`
`tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]` | — | canonical | +| [Callbacks: Important Limitations](../../docs/user/guide/callbacks.md#important-limitations) | Blocked | persistent, optional, asynchronous, and cross-thread callbacks; optional or descriptor prototype forms; fatal exceptions and invalid returns | — | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` | `tests/fortran/callbacks/policy/test_callback_policy.py::test_unsupported_callback_forms_are_blocked_before_codegen[def callback_shape(value: Allocatable[Float64]) -> None: ...-callback argument 'value' uses unsupported allocatable, pointer, polymorphic, or assumed-type storage]` (`policy`)
`tests/fortran/callbacks/policy/test_callback_policy.py::test_unsupported_callback_forms_are_blocked_before_codegen[def callback_shape(value: Float64 = ...) -> None: ...-callback argument 'value' cannot be both optional and passed by value; use a reference dummy so absence has a null-pointer ABI]` (`policy`) | canonical | +| [Callbacks: Key Rules](../../docs/user/guide/callbacks.md#key-rules) | Supported | completed callback ABI, conversion, lifecycle, adapter symbols, bridge declaration, source/generated parity | — | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]`
`tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[generated-pyi]` | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_callback_plan_edits_fail_central_validation_before_backend_emission[symbols-invalid-callback-symbols]` (`codegen`) | canonical | +| [Feature Matrix: Immediate Call-Scoped Python Callbacks](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | scalar, array, string, and derived conversions; entering thread; source/generated parity | — | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]` | — | canonical | +| [Feature Matrix: Persistent Callbacks And Procedure Pointers](../../docs/user/language-support/feature-matrix.md#unsupported-or-blocked-forms) | Blocked | stored or post-call invocation; call-scoped context | — | — | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_callback_plan_edits_fail_central_validation_before_backend_emission[lifecycle-unbalanced-callback-lifecycle]` (`codegen`) | canonical | +| [Enumerations: Complete Example](../../docs/user/guide/enumerations.md#complete-example) | Supported | `enum, bind(C)`; explicit, implicit, negative, and symbolic values; source generation; reviewed contract; source/generated replay | `tests/fortran/enumerations/parsing/test_enum_syntax.py::test_valid_enum_subunit_accepts_optional_separator_and_multiple_enumerators` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]`
`tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[generated-pyi]` | — | canonical | | [Enumerations: Usage In Python](../../docs/user/guide/enumerations.md#usage-in-python) | Supported | module constants; integer procedure input and result; exact `np.int32` values | `tests/fortran/enumerations/semantics/test_enum_semantics.py::test_fortran_enums_preserve_values_in_generated_pyi_contract` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]` | — | canonical | | [Enumerations: Key Points](../../docs/user/guide/enumerations.md#key-points) | Supported | `Final[Int32]`; native value stability; exact dtype; no closed-domain validation; integer fields and results | `tests/fortran/enumerations/semantics/test_enum_compile_time_values.py::test_resolve_semantic_compile_time_values_handles_enum_like_constants`
`tests/fortran/enumerations/semantics/test_pyi_enum_constants.py::test_convert_pyi_to_ir_round_trips_enum_like_integer_constants` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]` | — | canonical | | [Enumerations: Limitations](../../docs/user/guide/enumerations.md#limitations) | Blocked | no Python `Enum`/`IntEnum` class; integer application enums remain user-owned | `tests/fortran/enumerations/semantics/test_enum_semantics.py::test_fortran_enums_preserve_values_in_generated_pyi_contract` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]` | `tests/fortran/enumerations/semantics/test_unsupported_enum_classes.py::test_convert_pyi_to_ir_rejects_enum_classes` (`semantics`) | canonical | | [`.pyi`: Constants And Enums](../../docs/user/reference/pyi-format.md#constants-and-enums) | Supported | `Final[T]`; literal and symbolic initializers; round trip; ordinary integer arguments/results | `tests/fortran/enumerations/semantics/test_pyi_enum_constants.py::test_convert_pyi_to_ir_round_trips_enum_like_integer_constants` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[generated-pyi]` | `tests/fortran/enumerations/semantics/test_unsupported_enum_classes.py::test_convert_pyi_to_ir_rejects_enum_classes` (`semantics`) | canonical | | [Enumerations: Key Points](../../docs/user/guide/enumerations.md#key-points) | Supported | typed integer constants; `bind(C)` metadata; integer procedure and field surface; no generated enum class | `tests/fortran/enumerations/semantics/test_enum_semantics.py::test_fortran_enums_preserve_values_in_generated_pyi_contract` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]` | — | canonical | -| [Feature Matrix: Fortran Enum Constants](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | integer constants, field and procedure values, source/generated parity, malformed syntax diagnostics | `tests/fortran/enumerations/parsing/test_enum_diagnostics.py::test_enum_diagnostic_reports_first_invalid_line_after_valid_enumerator`
`tests/fortran/enumerations/semantics/test_enum_semantics.py::test_fortran_enums_preserve_values_in_generated_pyi_contract` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]` | `tests/fortran/enumerations/parsing/test_enum_syntax.py::test_enum_subunit_rejects_malformed_lines_and_nested_units[interface invalid]` (`parsing`) | canonical | -| [Raw Addresses: Checked Storage Or Raw Address](../../docs/user/guide/raw-addresses.md#checked-storage-or-raw-address) | Supported | checked rank-zero NumPy storage versus integer raw address; exact dtype/rank/itemsize/writeability validation only on checked storage | `tests/fortran/raw_addresses/policy/test_raw_address_policy.py::test_wrapper_policy_completes_fixed_string_storage_and_raw_address_ownership`
`tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py::test_scalar_storage_and_raw_address_plans_keep_explicit_boundary_facts` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | — | canonical | +| [Feature Matrix: Fortran Enum Constants](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | integer constants, field and procedure values, source/generated parity, malformed syntax diagnostics | `tests/fortran/enumerations/parsing/test_enum_diagnostics.py::test_enum_diagnostic_reports_the_first_invalid_line_after_a_valid_enumerator[declaration-statement]`
`tests/fortran/enumerations/semantics/test_enum_semantics.py::test_fortran_enums_preserve_values_in_generated_pyi_contract` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]` | `tests/fortran/enumerations/parsing/test_enum_diagnostics.py::test_enum_diagnostic_reports_the_first_invalid_line_after_a_valid_enumerator[nested-interface]` (`parsing`) | canonical | +| [Raw Addresses: Checked Storage Or Raw Address](../../docs/user/guide/raw-addresses.md#checked-storage-or-raw-address) | Supported | checked rank-zero NumPy storage versus integer raw address; exact dtype/rank/itemsize/writeability validation only on checked storage | `tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py::test_checked_scalar_storage_and_raw_address_complete_distinct_plans_and_guards` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | — | canonical | | [Raw Addresses: `Addr(T)` And `Addr(Arg(...))`](../../docs/user/guide/raw-addresses.md#addrt-and-addrarg) | Supported | type-level Python-visible integer address; native-call address projection; primitive-scalar projection restriction; no conflation of the two forms | `tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_public_raw_address_contract_round_trips`
`tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_native_call_addr_arg_rejects_non_primitive_scalar_values[Addr(Float64)]` | — | `tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_native_call_address_projection_rejects_non_argument_storage[Addr(Return(0))-Float64]` (`semantics`) | canonical | -| [Raw Addresses: Complete Example](../../docs/user/guide/raw-addresses.md#complete-example) | Supported | reviewed edited contract; primitive, vector, C-order matrix, Fortran-order matrix, and fixed-string address calls; checked-storage comparison | `tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py::test_raw_array_addresses_use_one_shared_transfer_and_shape_plan`
`tests/fortran/raw_addresses/codegen/test_string_address_lowering.py::test_string_addresses_dispatch_to_named_binding_and_bridge_lowering` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | — | canonical | -| [Raw Addresses: Primitive Address](../../docs/user/guide/raw-addresses.md#primitive-address) | Supported | integer pointer extraction; direct handoff; mutation; non-integer rejection; address-range overflow | `tests/fortran/raw_addresses/policy/test_raw_address_policy.py::test_wrapper_policy_completes_primitive_raw_address_handoff`
`tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py::test_scalar_storage_and_raw_address_lower_to_direct_named_paths` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` (`runtime`) | canonical | -| [Raw Addresses: Array Address](../../docs/user/guide/raw-addresses.md#array-address) | Supported | fully resolved rank and extents; C and Fortran orientation; same transfer action; mutation without descriptor or copy lifecycle | `tests/fortran/raw_addresses/policy/test_raw_address_policy.py::test_wrapper_policy_completes_required_raw_array_address_handoff`
`tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py::test_raw_array_addresses_reuse_integer_extraction_and_named_array_bridge_association` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | `tests/fortran/raw_addresses/policy/test_raw_address_policy.py::test_wrapper_policy_keeps_optional_raw_array_addresses_blocked` (`policy`)
`tests/fortran/raw_addresses/policy/test_raw_address_policy.py::test_wrapper_policy_keeps_projected_raw_array_addresses_blocked` (`policy`) | canonical | -| [Raw Addresses: Fixed-String Address](../../docs/user/guide/raw-addresses.md#fixed-string-address) | Supported | fixed scalar and rank-one string pointees; exact encoded width; integer address handoff; in-place mutation | `tests/fortran/raw_addresses/policy/test_raw_address_policy.py::test_wrapper_policy_completes_fixed_string_storage_and_raw_address_ownership`
`tests/fortran/raw_addresses/codegen/test_string_address_lowering.py::test_string_address_plans_keep_completed_ownership_length_and_copy_facts` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build`
`tests/fortran/raw_addresses/end_to_end/test_raw_fixed_string_arrays.py::test_raw_fixed_width_character_arrays_use_canonical_plan` | — | canonical | -| [Raw Addresses: Safety Rules](../../docs/user/guide/raw-addresses.md#safety-rules) | Supported | exact owner lifetime remains caller responsibility; no dtype, shape, layout, writeability, or target-lifetime validation; zero, negative, and arbitrary in-range integers are forwarded; unsafe dereference remains deliberately unexecuted | `tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_raw_address_policy_accepts_only_complete_primitive_layouts`
`tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py::test_raw_array_addresses_use_one_shared_transfer_and_shape_plan` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | `tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_raw_address_policy_rejects_incomplete_or_wrapped_pointees[Addr(Float64[:])-raw arrays require a fully resolved rank and shape]` (`semantics`) | canonical | -| [`.pyi`: Python And Native Boundaries](../../docs/user/reference/pyi-format.md#python-and-native-boundaries) | Supported | Python integer raw address; direct native address; distinct checked-storage and call-local address boundaries | `tests/fortran/raw_addresses/policy/test_raw_address_policy.py::test_wrapper_policy_completes_primitive_raw_address_handoff`
`tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py::test_scalar_storage_and_raw_address_plans_keep_explicit_boundary_facts` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | — | canonical | +| [Raw Addresses: Complete Example](../../docs/user/guide/raw-addresses.md#complete-example) | Supported | reviewed edited contract; primitive, vector, C-order matrix, Fortran-order matrix, and fixed-string address calls; checked-storage comparison | `tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py::test_raw_array_addresses_reuse_integer_extraction_and_named_array_bridge_association` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | — | canonical | +| [Raw Addresses: Primitive Address](../../docs/user/guide/raw-addresses.md#primitive-address) | Supported | integer pointer extraction; direct handoff; mutation; non-integer rejection; address-range overflow | `tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py::test_checked_scalar_storage_and_raw_address_complete_distinct_plans_and_guards` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` (`runtime`) | canonical | +| [Raw Addresses: Array Address](../../docs/user/guide/raw-addresses.md#array-address) | Supported | fully resolved rank and extents; C and Fortran orientation; same transfer action; mutation without descriptor or copy lifecycle | `tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py::test_raw_array_addresses_reuse_integer_extraction_and_named_array_bridge_association` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | `tests/fortran/raw_addresses/policy/test_raw_address_policy.py::test_wrapper_policy_blocks_raw_array_addresses_it_cannot_prove` | canonical | +| [Raw Addresses: Fixed-String Address](../../docs/user/guide/raw-addresses.md#fixed-string-address) | Supported | fixed scalar and rank-one string pointees; exact encoded width; integer address handoff; in-place mutation | `tests/fortran/raw_addresses/end_to_end/test_raw_fixed_string_arrays.py::test_raw_fixed_width_character_arrays_use_canonical_plan` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build`
`tests/fortran/raw_addresses/end_to_end/test_raw_fixed_string_arrays.py::test_raw_fixed_width_character_arrays_use_canonical_plan` | — | canonical | +| [Raw Addresses: Safety Rules](../../docs/user/guide/raw-addresses.md#safety-rules) | Supported | exact owner lifetime remains caller responsibility; no dtype, shape, layout, writeability, or target-lifetime validation; zero, negative, and arbitrary in-range integers are forwarded; unsafe dereference remains deliberately unexecuted | `tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_raw_address_policy_rejects_incomplete_or_wrapped_pointees`
`tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py::test_raw_array_plan_edits_fail_before_backend_lowering` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | `tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_raw_address_policy_rejects_incomplete_or_wrapped_pointees[Addr(Float64[:])-raw arrays require a fully resolved rank and shape]` (`semantics`) | canonical | +| [`.pyi`: Python And Native Boundaries](../../docs/user/reference/pyi-format.md#python-and-native-boundaries) | Supported | Python integer raw address; direct native address; distinct checked-storage and call-local address boundaries | `tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py::test_checked_scalar_storage_and_raw_address_complete_distinct_plans_and_guards` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | — | canonical | | [`.pyi`: Storage Contracts](../../docs/user/reference/pyi-format.md#storage-contracts) | Partially supported | `Addr(T)` for primitive scalar, fixed string, and fully resolved arrays; one-level pointer depth; deliberate wrapped, unresolved, optional, projected-array, and callable limits | `tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_public_raw_address_contract_round_trips`
`tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_raw_address_syntax_rejects_multiple_pointees_and_explicit_depth_one[value: Addr[1](Int32)\n-Addr[1](...) is invalid; use Addr(...)]` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | `tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py::test_raw_address_policy_rejects_incomplete_or_wrapped_pointees[Addr(String)-raw strings require a fixed length]` (`semantics`) | canonical | -| [Raw Addresses: Checked Storage Or Raw Address](../../docs/user/guide/raw-addresses.md#checked-storage-or-raw-address) | Supported | raw scalar addresses and checked rank-zero storage use distinct completed plans and named lowering paths | `tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py::test_scalar_storage_and_raw_address_lower_to_direct_named_paths` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | — | canonical | -| [Error Handling: Failure Stages](../../docs/user/guide/error-handling.md#failure-stages) | Supported | parsing, interface conversion, wrapper planning, compilation/linking, import, Python-call validation, native status, and callback-fatal boundaries remain distinct | `tests/fortran/error_handling/parsing/test_fortran_diagnostics.py::test_parse_error_formats_compiler_style_diagnostic`
`tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_planner_records_editable_native_runtime_and_status_error_facts` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` (`runtime`) | canonical | -| [Error Handling: Verbose Output And Tracebacks](../../docs/user/guide/error-handling.md#verbose-output-and-tracebacks) | Supported | concise expected diagnostic; full `--debug` traceback; replayable verbose native command | `tests/fortran/error_handling/compiling/test_verbose_commands.py::test_run_command_verbose_prints_replayable_command` | `tests/fortran/error_handling/pipeline/test_concise_cli_diagnostics.py::test_cli_formats_parse_errors_without_traceback`
`tests/fortran/error_handling/pipeline/test_debug_cli_tracebacks.py::test_cli_debug_flag_reraises_parse_errors` | — | canonical | -| [Error Handling: Status Projection Example](../../docs/user/guide/error-handling.md#status-projection-example) | Supported | edited `@native_call` hidden status/message projection; `@raises`; success value; `None` on success; exact `RuntimeError` message; repeated failure cleanup and recovery | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_runtime_policy_decorators_round_trip_through_pyi`
`tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_direct_bridge_lowering_projects_status_and_copies_fixed_message` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | — | canonical | +| [Raw Addresses: Checked Storage Or Raw Address](../../docs/user/guide/raw-addresses.md#checked-storage-or-raw-address) | Supported | raw scalar addresses and checked rank-zero storage use distinct completed plans and named lowering paths | `tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py::test_checked_scalar_storage_and_raw_address_complete_distinct_plans_and_guards` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build` | — | canonical | +| [Error Handling: Failure Stages](../../docs/user/guide/error-handling.md#failure-stages) | Supported | parsing, interface conversion, wrapper planning, compilation/linking, import, Python-call validation, native status, and callback-fatal boundaries remain distinct | `tests/fortran/error_handling/parsing/test_fortran_diagnostics.py::test_parse_error_formats_compiler_style_diagnostic`
`tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` (`runtime`) | canonical | +| [Error Handling: Verbose Output And Tracebacks](../../docs/user/guide/error-handling.md#verbose-output-and-tracebacks) | Supported | concise expected diagnostic; full `--debug` traceback; replayable verbose native command | `tests/fortran/error_handling/compiling/test_verbose_commands.py::test_run_command_verbose_prints_replayable_command` | `tests/fortran/error_handling/pipeline/test_concise_cli_diagnostics.py::test_cli_formats_parse_errors_concisely_and_reraises_with_debug` | — | canonical | +| [Error Handling: Status Projection Example](../../docs/user/guide/error-handling.md#status-projection-example) | Supported | edited `@native_call` hidden status/message projection; `@raises`; success value; `None` on success; exact `RuntimeError` message; repeated failure cleanup and recovery | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_runtime_policy_decorators_round_trip_through_pyi`
`tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | — | canonical | | [Error Handling: Common Python Exceptions](../../docs/user/guide/error-handling.md#common-python-exceptions) | Supported | boundary `TypeError`; contract/option and parse `ValueError`; projected native `RuntimeError`; native artifact import/load error taxonomy | `tests/fortran/error_handling/parsing/test_fortran_diagnostics.py::test_parse_error_is_subclass_of_value_error`
`tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_runtime_status_policy_rejects_invalid_output_contracts[@raises(status="status")\ndef solve(status: Int32) -> None: ...-status target must name a hidden output]` | `tests/fortran/raw_addresses/end_to_end/test_raw_native_addresses.py::test_primitive_array_and_fixed_string_raw_addresses_share_one_native_build`
`tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | — | canonical | -| [Error Handling: Best Practices](../../docs/user/guide/error-handling.md#best-practices) | Supported | full diagnostic first; verbose command replay; debug traceback only on demand; edited-contract inspection; risky callback isolation | `tests/fortran/error_handling/parsing/test_fortran_diagnostics.py::test_parse_error_message_includes_filename_and_lineno`
`tests/fortran/error_handling/compiling/test_verbose_commands.py::test_run_command_verbose_prints_replayable_command` | `tests/fortran/error_handling/pipeline/test_debug_cli_tracebacks.py::test_cli_debug_flag_reraises_parse_errors`
`tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` | — | canonical | -| [Error Handling: Failure Stages](../../docs/user/guide/error-handling.md#failure-stages) | Supported | ordinary wrapper exceptions; no inferred application convention; opt-in status/message projection; cleanup after failure; native termination remains unrecoverable | `tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_direct_binding_lowering_places_only_opted_in_native_call_outside_the_gil`
`tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_fixed_message_bridge_copy_requires_its_completed_reason` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` (`runtime`) | canonical | -| [Feature Matrix: Runtime Error Projection, GIL Policy, Recursion, OpenMP Path, And GNU ABI Checks](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | status error and message; completed GIL envelope; recursion/OpenMP/ABI remain separately owned; no caller synchronization inference | `tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_planner_records_editable_native_runtime_and_status_error_facts` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | — | canonical | -| [Building The Shared Library: Build](../../docs/user/guide/building-shared-library.md#build) | Supported | source input; default and explicit module names; build directory; generated sources; importable shared library | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_result_records_structured_native_plan`
`tests/fortran/infrastructure/building/pipeline/test_source_generated_contracts.py::test_source_build_generated_pyi_contract_matches_fixture[fruntime_abi_f90]` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_documented_readme_points_example_builds_and_imports` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_wrapper_build_rejects_empty_source_list` (`pipeline`)
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_wrapper_build_rejects_missing_source` (`pipeline`) | canonical | +| [Error Handling: Best Practices](../../docs/user/guide/error-handling.md#best-practices) | Supported | full diagnostic first; verbose command replay; debug traceback only on demand; edited-contract inspection; risky callback isolation | `tests/fortran/error_handling/parsing/test_fortran_diagnostics.py::test_parse_error_message_includes_filename_and_lineno`
`tests/fortran/error_handling/compiling/test_verbose_commands.py::test_run_command_verbose_prints_replayable_command` | `tests/fortran/error_handling/pipeline/test_concise_cli_diagnostics.py::test_cli_formats_parse_errors_concisely_and_reraises_with_debug`
`tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` | — | canonical | +| [Error Handling: Failure Stages](../../docs/user/guide/error-handling.md#failure-stages) | Supported | ordinary wrapper exceptions; no inferred application convention; opt-in status/message projection; cleanup after failure; native termination remains unrecoverable | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers`
`tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_status_error_plan_edits_fail_before_backend_lowering` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` (`runtime`) | canonical | +| [Feature Matrix: Runtime Error Projection, GIL Policy, Recursion, OpenMP Path, And GNU ABI Checks](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | status error and message; completed GIL envelope; recursion/OpenMP/ABI remain separately owned; no caller synchronization inference | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | — | canonical | +| [Building The Shared Library: Build](../../docs/user/guide/building-shared-library.md#build) | Supported | source input; default and explicit module names; build directory; generated sources; importable shared library | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_result_records_structured_native_plan`
`tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_generated_pyi_replay_matches_source_build_without_source_reparse` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_documented_readme_points_example_builds_and_imports` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_wrapper_build_rejects_empty_source_list` (`pipeline`)
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_wrapper_build_rejects_missing_source` (`pipeline`) | canonical | | [Building The Shared Library: Import](../../docs/user/guide/building-shared-library.md#import) | Supported | ABI-suffixed artifact; stable module import name; explicit output name; root-function name collision avoidance | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_names_importable_shared_library`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_default_module_name_does_not_collide_with_root_function` | — | canonical | -| [Building The Shared Library: Multiple Source Files](../../docs/user/guide/building-shared-library.md#multiple-source-files) | Supported | dependency-ordered compilation; contained-module namespaces; standalone externals; one merged extension; generated and edited contract parity | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_multi_source_pyi_out_writes_one_flat_combined_package` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_multi_file_modules_build_one_merged_extension`
`tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_multi_file_standalone_procedures_build_one_merged_extension` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_missing_module_directory_reports_compile_error` (`compiling`) | canonical | +| [Building The Shared Library: Multiple Source Files](../../docs/user/guide/building-shared-library.md#multiple-source-files) | Supported | dependency-ordered compilation; contained-module namespaces; standalone externals; one merged extension; generated and edited contract parity | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_multi_source_generated_contract_build_matches_source_runtime_and_link_order` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_multi_file_modules_build_one_merged_extension`
`tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_multi_file_standalone_procedures_build_one_merged_extension` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_missing_module_directory_reports_compile_error` (`compiling`) | canonical | | [Building The Shared Library: Use A Makefile](../../docs/user/guide/building-shared-library.md#use-a-makefile) | Supported | generation without compilation; editable compiler and flags; ordered source dependencies; GNU Make build; manifest regeneration and replay | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_makefile_manifest_and_replay_workflows` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_makefile_mode_reproduces_multi_source_build` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_wrapper_build_rejects_generation_verbose_combination[makefile]` (`pipeline`) | canonical | -| [Building The Shared Library: Compatibility](../../docs/user/guide/building-shared-library.md#compatibility) | Supported | target ABI; debug and optimized wrappers; top-level kind flags; platform-specific extension; rebuildable native artifacts | `tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py::test_top_level_native_kind_flags_drive_internal_type_measurement` | `tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py::test_debug_and_optimized_wrapper_builds_preserve_runtime_abi` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_incompatible_native_artifact_reports_linker_error` (`compiling`) | canonical | +| [Building The Shared Library: Compatibility](../../docs/user/guide/building-shared-library.md#compatibility) | Supported | target ABI; debug and optimized wrappers; top-level kind flags; platform-specific extension; rebuildable native artifacts | `tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py::test_top_level_native_kind_flags_drive_internal_type_measurement` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_makefile_mode_reproduces_multi_source_build` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_incompatible_native_artifact_reports_linker_error` (`compiling`) | canonical | | [Build Manifests And Makefiles: `prik-build.json`](../../docs/user/reference/configuration-files.md#prik-buildjson) | Supported | semantic sources separate from compilation units; produced and prebuilt artifacts; module/include/library directories; ordered object, archive, shared, named-library, and linker-argument items | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_native_link_plan_serializes_interleaved_item_kinds`
`tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_cli_preserves_explicit_ordered_link_items` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_mixed_module_external_bundle_resolves_all_native_input_kinds` | — | canonical | | [`.pyi`: Native Artifacts And Link Resolution](../../docs/user/reference/pyi-format.md#native-artifacts-and-link-resolution) | Supported | no filename inference; objects, archives, direct and named shared libraries; transitive providers; archive groups; missing/duplicate/incompatible artifact diagnostics | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_cli_preserves_explicit_ordered_link_items`
`tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_static_archive_groups_resolve_cyclic_archive_dependencies` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_mixed_module_external_bundle_resolves_all_native_input_kinds` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_missing_symbol_reports_native_link_or_loader_error` (`import`)
`tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_duplicate_native_definitions_report_linker_error` (`compiling`) | canonical | | [`.pyi`: Contract Imports](../../docs/user/reference/pyi-format.md#contract-imports) | Supported | explicit `prik.contracts` imports; arbitrary aliases; missing imports rejected; ordinary and relative imports preserved | `tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_convert_pyi_to_ir_requires_imported_contract_types`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_follows_arbitrary_contract_aliases` | — | `tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_convert_pyi_to_ir_requires_imported_contract_types` (`semantics`) | canonical | | [`.pyi`: Misuse, Diagnostics And Risk](../../docs/user/reference/pyi-format.md#misuse-diagnostics-and-risk) | Supported | syntax, semantic shape, native contract, policy, and unsafe-boundary diagnostics; filename-aware failures; no silent fallback | `tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_pyi_file_to_semantic_module_and_modules_forward_module_name_encoding_and_filename` | — | `tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py::test_pyi_parser_reports_unsupported_lines_and_invalid_helpers` (`parsing`)
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_pyi_python_api_rejects_invalid_projection_before_codegen` (`pipeline`) | canonical | -| [`.pyi`: File Shape](../../docs/user/reference/pyi-format.md#file-shape) | Supported | Python AST boundary; imports, annotated declarations, classes, ellipsis-only functions, and supported decorators | `tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py::test_pyi_parser_returns_python_ast_only`
`tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py::test_convert_pyi_to_ir_accepts_parsed_pyi_ast_only` | — | `tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py::test_pyi_parser_reports_unsupported_lines_and_invalid_helpers` (`parsing`) | canonical | +| [`.pyi`: File Shape](../../docs/user/reference/pyi-format.md#file-shape) | Supported | Python AST boundary; imports, annotated declarations, classes, ellipsis-only functions, and supported decorators | `tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py::test_pyi_parser_returns_python_ast_only`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py::test_contract_spellings_round_trip_through_semantic_ir[visibility-and-module-state]` | — | `tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py::test_pyi_parser_reports_unsupported_lines_and_invalid_helpers` (`parsing`) | canonical | | [`.pyi`: Imported Derived-Type Identity](../../docs/user/reference/pyi-format.md#imported-derived-type-identity) | Supported | direct, aliased, relative, qualified, opaque, and edited wrapped external type identity | `tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_pyi_paths_to_semantic_modules_reconciles_opaque_and_edited_external_types`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_pyi_paths_to_semantic_modules_reconciles_relative_namespace_type_refs` | — | `tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_pyi_paths_to_semantic_modules_handles_duplicate_roots_and_ambiguous_module_names` (`semantics`) | canonical | -| [`.pyi`: Contract Files And Native Procedure Placement](../../docs/user/reference/pyi-format.md#contract-files-and-native-procedure-placement) | Supported | entry contract; native module leaves; standalone root declarations; multiple modules; same-name module collision | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_multi_module_generation_keeps_each_native_namespace`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_same_named_module_uses_init_entry_and_keeps_externals_at_root` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback` | — | canonical | -| [`.pyi`: Contained Module Procedures](../../docs/user/reference/pyi-format.md#contained-module-procedures) | Supported | filename-selected native module scope; child Python namespace; exact native procedure name | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_module_generation_writes_explicit_package_entry_and_native_leaf`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_generated_native_scope_comes_from_contract_filename` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback` | — | canonical | -| [`.pyi`: Standalone Procedures](../../docs/user/reference/pyi-format.md#standalone-procedures) | Supported | `@standalone`; entry placement; multiple root procedures; no invented module scope | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_standalone_generation_writes_explicit_package_entry`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_generated_standalone_contract_retains_standalone_native_placement` | — | — | canonical | -| [`.pyi`: Source-To-Contract Layout](../../docs/user/reference/pyi-format.md#source-to-contract-layout) | Supported | module-only, standalone-only, mixed, multi-module, same-name, and transitive-import source layouts | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_import_graph_generation_writes_entry_and_native_leaves`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_multi_module_generation_keeps_each_native_namespace` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback` | — | canonical | +| [`.pyi`: Contract Files And Native Procedure Placement](../../docs/user/reference/pyi-format.md#contract-files-and-native-procedure-placement) | Supported | entry contract; native module leaves; standalone root declarations; multiple modules; same-name module collision | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_multi_module_generation_keeps_each_native_namespace`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_generated_contract_package_matches_reviewed_layout[contract_same_name]` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback` | — | canonical | +| [`.pyi`: Contained Module Procedures](../../docs/user/reference/pyi-format.md#contained-module-procedures) | Supported | filename-selected native module scope; child Python namespace; exact native procedure name | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_generated_contract_package_matches_reviewed_layout[contract_mixed_module_external]`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_generated_native_scope_comes_from_contract_filename` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback` | — | canonical | +| [`.pyi`: Standalone Procedures](../../docs/user/reference/pyi-format.md#standalone-procedures) | Supported | `@standalone`; entry placement; multiple root procedures; no invented module scope | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_generated_contract_package_matches_reviewed_layout[contract_standalone_only]`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_generated_native_scope_comes_from_contract_filename` | — | — | canonical | +| [`.pyi`: Source-To-Contract Layout](../../docs/user/reference/pyi-format.md#source-to-contract-layout) | Supported | module-only, standalone-only, mixed, multi-module, same-name, and transitive-import source layouts | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_generated_contract_package_matches_reviewed_layout[contract_import_graph]`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_multi_module_generation_keeps_each_native_namespace` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback` | — | canonical | | [`.pyi`: Root Export Contract](../../docs/user/reference/pyi-format.md#root-export-contract) | Supported | module import, selective symbol export, alias, support-import exclusion, and collision rejection | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_checked_entry_discovers_its_complete_contract_package[contract_import_graph]`
`tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/policy/test_export_and_initializer_policy.py::test_policy_completion_prunes_unexported_entry_declarations_before_lowering` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_selects_child_flattened_aliased_and_bound_exports` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_rejects_colliding_wildcard_exports` (`pipeline`) | canonical | -| [`.pyi`: Entry Contract And Extension Identity](../../docs/user/reference/pyi-format.md#entry-contract-and-extension-identity) | Supported | `__init__.pyi` parent identity; explicit output identity; leaf identity; ABI-suffixed shared object | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_same_named_module_uses_init_entry_and_keeps_externals_at_root` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias` | — | canonical | +| [`.pyi`: Entry Contract And Extension Identity](../../docs/user/reference/pyi-format.md#entry-contract-and-extension-identity) | Supported | `__init__.pyi` parent identity; explicit output identity; leaf identity; ABI-suffixed shared object | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_generated_contract_package_matches_reviewed_layout[contract_same_name]` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias` | — | canonical | | [`.pyi`: Contract Import Graph](../../docs/user/reference/pyi-format.md#contract-import-graph) | Supported | recursive relative imports; deterministic discovery order; parse cache; missing file and cycle diagnostics | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_pyi_contract_bundle_reuses_import_discovery_conversion_cache`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_checked_entry_discovers_its_complete_contract_package[contract_import_graph]` | — | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_recursive_graph_reports_missing_relative_contract_before_native_validation` (`pipeline`)
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_recursive_graph_reports_cycles_before_codegen` (`pipeline`) | canonical | -| [`.pyi`: Semantic Type Names](../../docs/user/reference/pyi-format.md#semantic-type-names) | Supported | canonical primitive, wrapper, nested, qualified, aliased, callback, and storage type spellings | `tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_dispatches_nested_and_qualified_semantic_types`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_accepts_aliased_contract_wrapper_names` | — | — | canonical | -| [`.pyi`: Metadata With `Annotated`](../../docs/user/reference/pyi-format.md#metadata-with-annotated) | Supported | constraints; source names; layout/copy; immutability; native descriptor and provenance metadata; stable round trip | `tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py::test_pyi_parser_preserves_generic_constraints_as_annotation_metadata`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_preserves_extended_array_metadata_and_nested_selector` | — | `tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_rejects_additional_invalid_storage_forms[value: Annotated[Int32, 'bad']\n-Unsupported Annotated metadata: "'bad'"]` (`semantics`) | canonical | +| [`.pyi`: Semantic Type Names](../../docs/user/reference/pyi-format.md#semantic-type-names) | Supported | canonical primitive, wrapper, nested, qualified, aliased, callback, and storage type spellings | `tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_dispatches_nested_and_qualified_semantic_types`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_follows_arbitrary_contract_aliases` | — | — | canonical | +| [`.pyi`: Metadata With `Annotated`](../../docs/user/reference/pyi-format.md#metadata-with-annotated) | Supported | constraints; source names; layout/copy; immutability; native descriptor and provenance metadata; stable round trip | `tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py::test_pyi_parser_preserves_generic_constraints_as_annotation_metadata`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_preserves_rank_selector_and_character_allocatable_marker` | — | `tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_rejects_additional_invalid_storage_forms[value: Annotated[Int32, 'bad']\n-Unsupported Annotated metadata: "'bad'"]` (`semantics`) | canonical | | [`.pyi`: Classes And Native ABI](../../docs/user/reference/pyi-format.md#classes-and-native-abi) | Supported | ordinary wrapped classes; opaque external classes; field declarations; class-level `@native_abi("c")` for `bind(C)` types | `tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_pyi_paths_to_semantic_modules_reconciles_opaque_and_edited_external_types`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_native_abi.py::test_native_abi_marks_fortran_bind_c_class_and_round_trips` | `tests/fortran/derived_types/end_to_end/test_derived_boundaries.py::test_scalar_derived_types_cross_procedure_boundaries[generated-pyi]` | — | canonical | | [`.pyi`: Functions, Methods And Returns](../../docs/user/reference/pyi-format.md#functions-methods-and-returns) | Supported | direct and tuple returns; named replacement outputs; native-order identity; method receiver; explicit projection | `tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py::test_plain_tuple_return_types_parse_component_returns`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py::test_native_order_outputs_do_not_get_projected_without_native_call` | `tests/fortran/functions/end_to_end/test_documented_function_journeys.py::test_function_results_outputs_arrays_and_no_intent_replacements_follow_documented_order` | — | canonical | | [`.pyi`: Generic Procedure Overloads](../../docs/user/reference/pyi-format.md#generic-procedure-overloads) | Supported | explicit specific links; private link targets; native bind; exact signature resolution; deterministic errors | `tests/fortran/generic_interfaces/semantics/test_pyi_overload_semantics.py::test_convert_pyi_to_ir_resolves_prik_overload_by_explicit_specific_name` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[generated-pyi]` | `tests/fortran/generic_interfaces/semantics/test_pyi_overload_semantics.py::test_convert_pyi_to_ir_rejects_invalid_prik_overload_links[@overload("missing")\ndef convert(value: Int32) -> Int32: ...\n-missing specific procedure 'missing']` (`semantics`) | canonical | | [`.pyi`: Defined Operators And Assignment](../../docs/user/reference/pyi-format.md#defined-operators-and-assignment) | Supported | direct/reflected/unary/comparison/named operators; explicit mutating assignment method | `tests/fortran/generic_interfaces/semantics/test_fortran_generic_semantics.py::test_converter_preserves_defined_operators_assignment_and_type_bound_operators` | `tests/fortran/generic_interfaces/end_to_end/test_defined_operators.py::test_fortran_defined_operators_and_assignment_dispatch_in_generated_c_extension[generated-pyi]` | — | canonical | -| [`.pyi`: Allocatable Array Handles](../../docs/user/reference/pyi-format.md#allocatable-array-handles) | Supported | persistent handle syntax; allocated/unallocated state; live views; field/module/result ownership; explicit copy | `tests/fortran/allocatables/semantics/test_pyi_allocatable_semantics.py::test_persistent_allocatable_descriptors_preserve_scalar_and_array_kinds` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[generated-pyi]` | `tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py::test_generated_storage_rejects_a_closed_contract_handle` (`runtime`) | canonical | +| [`.pyi`: Allocatable Array Handles](../../docs/user/reference/pyi-format.md#allocatable-array-handles) | Supported | persistent handle syntax; allocated/unallocated state; live views; field/module/result ownership; explicit copy | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles` | `tests/fortran/allocatables/end_to_end/test_allocatable_handles.py::test_allocatable_module_fields_and_results_expose_lifetime_safe_handles[generated-pyi]` | `tests/fortran/pointers/runtime/test_pointer_contract_handles.py::test_generated_storage_rejects_incompatible_contract_handles` | canonical | | [`.pyi`: Visibility And Names](../../docs/user/reference/pyi-format.md#visibility-and-names) | Supported | decorator and type-wrapper privacy; source-name metadata; generated Fortran normalization; strict-name rejection; native binding retained | `tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_preserves_user_private_bound_function_contract`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py::test_generated_pyi_escaping_round_trips_native_names`
`tests/fortran/infrastructure/naming/test_policy.py::test_python_keyword_is_renamed_only_at_the_python_boundary` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_visibility_and_initialization.py::test_editable_contract_removes_hides_and_initializes_module_declarations` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_visibility_naming.py::test_strict_wrapper_names_reject_python_name_fixes` (`pipeline`) | canonical | -| [`.pyi`: Projection Metadata](../../docs/user/reference/pyi-format.md#projection-metadata) | Supported | ordered `Arg`, `Addr`, `Value`, `Return`, descriptor, length, shape, presence, literal, pass, and workspace entries | `tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py::test_native_call_accepts_hidden_native_values`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py::test_emit_native_call_hidden_native_values` | — | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_pyi_python_api_rejects_invalid_projection_before_codegen` (`pipeline`) | canonical | -| [`.pyi`: Current Generated Coverage](../../docs/user/reference/pyi-format.md#current-generated-coverage) | Partially supported | canonical parser/printer round trip; reviewed package layout; authoritative runtime input; documented generated and loaded subsets | `tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py::test_generated_semantic_ir_round_trips_through_pyi`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_checked_contract_package_has_reviewed_files` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback` | — | canonical | +| [`.pyi`: Projection Metadata](../../docs/user/reference/pyi-format.md#projection-metadata) | Supported | ordered `Arg`, `Addr`, `Value`, `Return`, descriptor, length, shape, presence, literal, pass, and workspace entries | `tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py::test_native_call_accepts_hidden_native_values`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py::test_contract_spellings_round_trip_through_semantic_ir[hidden-native-values]` | — | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_pyi_python_api_rejects_invalid_projection_before_codegen` (`pipeline`) | canonical | +| [`.pyi`: Current Generated Coverage](../../docs/user/reference/pyi-format.md#current-generated-coverage) | Partially supported | canonical parser/printer round trip; reviewed package layout; authoritative runtime input; documented generated and loaded subsets | `tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py::test_generated_semantic_ir_round_trips_through_pyi`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_checked_contracts_round_trip_through_semantic_ir` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback` | — | canonical | | [`.pyi`: Rejected Or Not Yet Supported](../../docs/user/reference/pyi-format.md#rejected-or-not-yet-supported) | Blocked | unknown types; invalid subscriptions, depth, callable shapes, decorators, bodies, arguments, and overload/projection combinations | — | — | `tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py::test_convert_pyi_to_ir_rejects_invalid_projection_and_type_forms[value: Unknown\n-Unknown semantic type is not allowed in .pyi annotations]` (`semantics`)
`tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_rejects_additional_invalid_storage_forms[value: Float64[ORDER_F]\n-Non-dimensional type subscriptions are not supported; use Final[...] for constants and Annotated[...] for constraints or array metadata]` (`semantics`) | canonical | -| [`.pyi`: Remaining Format And Runtime Work](../../docs/user/reference/pyi-format.md#remaining-format-and-runtime-work) | Partially supported | implemented ordered projection and policy dispatch; broader polymorphism, pointer lifetimes, and IDE-only stub separation remain limited | `tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py::test_fortran_to_pyi_and_back_preserves_mixed_input_output_projection` | — | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_contradictory_constructor_declarations_are_rejected` (`semantics`)
`tests/fortran/allocatables/policy/test_allocatable_result_policy.py::test_direct_allocatable_scalar_function_result_is_blocked_before_codegen` (`policy`) | canonical | +| [`.pyi`: Remaining Format And Runtime Work](../../docs/user/reference/pyi-format.md#remaining-format-and-runtime-work) | Partially supported | implemented ordered projection and policy dispatch; broader polymorphism, pointer lifetimes, and IDE-only stub separation remain limited | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_types_and_declarations.py::test_fortran_declarations_emit_contract_spelling_that_reloads[scalar-arguments-and-results]` | — | `tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_contradictory_constructor_declarations_are_rejected` (`semantics`)
`tests/fortran/allocatables/policy/test_allocatable_result_policy.py::test_direct_allocatable_scalar_function_result_is_blocked_before_codegen` (`policy`) | canonical | | [`.pyi` Exports And Modules: Choose The Package Shape](../../docs/user/reference/pyi-contracts/exports-and-modules.md#choose-the-package-shape) | Supported | child namespaces; wildcard flattening; selective imports; symbol and module aliases; nested aliases; support-import exclusion; reachable declarations only | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/policy/test_export_and_initializer_policy.py::test_policy_completion_prunes_unexported_entry_declarations_before_lowering`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_checked_entry_discovers_its_complete_contract_package[contract_import_graph]` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_selects_child_flattened_aliased_and_bound_exports` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_rejects_colliding_wildcard_exports` (`pipeline`)
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_recursive_graph_reports_missing_relative_contract_before_native_validation` (`pipeline`)
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_recursive_graph_reports_cycles_before_codegen` (`pipeline`) | canonical | | [`.pyi` Exports And Modules: Remove Or Hide A Declaration](../../docs/user/reference/pyi-contracts/exports-and-modules.md#remove-or-hide-a-declaration) | Supported | deleted function and variable; `@private`; `private[...]`; class constructor suppression; later class/member/overload runtime owner retained | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/policy/test_export_and_initializer_policy.py::test_policy_completion_prunes_unexported_entry_declarations_before_lowering`
`tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py::test_removing_constructor_suppresses_generated_keyword_initialization` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_visibility_and_initialization.py::test_editable_contract_removes_hides_and_initializes_module_declarations` | — | canonical | | [`.pyi` Exports And Modules: Add Or Rename A Native Procedure](../../docs/user/reference/pyi-contracts/exports-and-modules.md#add-or-rename-a-native-procedure) | Supported | added module-leaf declaration; `@bind`; renamed standalone `@standalone`; unchanged native targets; no invented implementation | `tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_preserves_user_private_bound_function_contract` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_selects_child_flattened_aliased_and_bound_exports` | — | canonical | @@ -216,10 +216,10 @@ Authoritative sources: | [`.pyi` Calls And Results: Expose Native Arguments Directly](../../docs/user/reference/pyi-contracts/calls-and-results.md#expose-native-arguments-directly) | Supported | no `@native_call`; native-order scalar, rank-zero storage, array, fixed string, and derived object arguments; visible caller mutation and discarded string-temporary mutation | `tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/policy/test_call_and_result_policy.py::test_native_order_and_projected_result_positions_are_completed_before_planning` | `tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_edited_call_surfaces.py::test_native_order_exposes_writable_slots_without_projection` | — | canonical | | [`.pyi` Calls And Results: Reorder Arguments And Project Outputs](../../docs/user/reference/pyi-contracts/calls-and-results.md#reorder-arguments-and-project-outputs) | Supported | reordered `Arg`/`Addr(Arg)`; hidden scalar, fixed string, and fixed-array results; caller arrays and derived objects; multiple-result tuple order; typed literals and complete projection grammar | `tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/policy/test_call_and_result_policy.py::test_native_order_and_projected_result_positions_are_completed_before_planning`
`tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/codegen/test_call_and_result_lowering.py::test_plan_records_reordered_arguments_gil_behavior_and_hidden_result_slots`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py::test_native_call_accepts_hidden_native_values` | `tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_edited_call_surfaces.py::test_native_call_reorders_arguments_and_projects_mixed_results`
`tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_edited_call_surfaces.py::test_hidden_fixed_shape_array_output_is_allocated_and_returned` | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_pyi_python_api_rejects_invalid_projection_before_codegen` (`pipeline`) | canonical | | [`.pyi` Calls And Results: Control Mutation](../../docs/user/reference/pyi-contracts/calls-and-results.md#control-mutation) | Supported | immutable scalar, fixed string, array, and derived replacement results; unchanged Python inputs; copy-in/copy-out and identity writeback paths | `tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/policy/test_call_and_result_policy.py::test_immutable_replacement_policy_is_complete_before_ir_lowering`
`tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/codegen/test_call_and_result_lowering.py::test_replacement_writeback_dispatches_selected_scalar_result_behavior[copy_in_out]` | `tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_edited_call_surfaces.py::test_immutable_values_return_replacements_without_mutating_inputs` | `tests/fortran/memory_management/policy/test_memory_ownership_policy.py::test_contradictory_ownership_contract_fails_before_lowering` (`policy`) | canonical | -| [`.pyi` Calls And Results: Edit Types Shapes Layout And Optionality](../../docs/user/reference/pyi-contracts/calls-and-results.md#edit-types-shapes-layout-and-optionality) | Supported | fixed/open shapes; exact dtype, rank, layout, writeability, byte order, alignment, and zero-size checks; Fortran-order default; supported nullable/defaulted native optionals | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_dense_array_lowering_uses_planned_shape_checks_and_bridge_orientation`
`tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_scalar_policy_completes_nullable_value_presence_before_planning` | `tests/fortran/arrays/end_to_end/test_array_contract_validation.py::test_remaining_array_contracts_are_validated_before_fortran_calls[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]` | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_passed_procedure_is_blocked_before_codegen` (`policy`) | canonical | -| [`.pyi` Calls And Results: Translate Status Results Into Exceptions](../../docs/user/reference/pyi-contracts/calls-and-results.md#translate-status-results-into-exceptions) | Supported | named hidden scalar integer status; optional hidden string message; configurable success value; consumed projected outputs | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_status_projection_accepts_an_optional_missing_message_target`
`tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_runtime_plan_edits_dispatch_to_named_lowering_and_validate_roles` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_runtime_status_policy_rejects_invalid_output_contracts[@raises(status="status", message="message")\ndef solve() -> tuple[Returns["status", Int32], Returns["message", Int32]]: ...-must be a scalar string hidden output]` (`policy`) | canonical | -| [`.pyi` Calls And Results: Release The GIL For A Native Call](../../docs/user/reference/pyi-contracts/calls-and-results.md#release-the-gil-for-a-native-call) | Supported | ordinary held call; explicit released call; status conversion after reacquisition; callback trampoline reacquisition | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_runtime_policy_decorators_round_trip_through_pyi`
`tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_direct_binding_lowering_places_only_opted_in_native_call_outside_the_gil` | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]` | — | canonical | +| [`.pyi` Calls And Results: Edit Types Shapes Layout And Optionality](../../docs/user/reference/pyi-contracts/calls-and-results.md#edit-types-shapes-layout-and-optionality) | Supported | fixed/open shapes; exact dtype, rank, layout, writeability, byte order, alignment, and zero-size checks; Fortran-order default; supported nullable/defaulted native optionals | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_unavailable_dense_extent_role_fails_before_backend_lowering`
`tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_policy_completes_presence_mode_before_planning` | `tests/fortran/arrays/end_to_end/test_array_contract_validation.py::test_remaining_array_contracts_are_validated_before_fortran_calls[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]` | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_value_callback_dummy_is_blocked_before_codegen` (`policy`) | canonical | +| [`.pyi` Calls And Results: Translate Status Results Into Exceptions](../../docs/user/reference/pyi-contracts/calls-and-results.md#translate-status-results-into-exceptions) | Supported | named hidden scalar integer status; optional hidden string message; configurable success value; consumed projected outputs | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_status_projection_accepts_an_optional_missing_message_target`
`tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_status_error_plan_edits_fail_before_backend_lowering` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_runtime_status_policy_rejects_invalid_output_contracts[@raises(status="status", message="message")\ndef solve() -> tuple[Returns["status", Int32], Returns["message", Int32]]: ...-must be a scalar string hidden output]` (`policy`) | canonical | +| [`.pyi` Calls And Results: Release The GIL For A Native Call](../../docs/user/reference/pyi-contracts/calls-and-results.md#release-the-gil-for-a-native-call) | Supported | ordinary held call; explicit released call; status conversion after reacquisition; callback trampoline reacquisition | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_runtime_policy_decorators_round_trip_through_pyi`
`tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]` | — | canonical | | [Feature Matrix: Multi-Source Builds, Makefiles, Verbose Mode, And Output Placement](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | dependency-ordered compilation; direct and Makefile builds; replayable verbose commands; ABI artifact and stable alias placement | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_verbose_mode_prints_full_direct_build_commands` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_makefile_mode_reproduces_multi_source_build`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias` | — | canonical | -| [Feature Matrix: Fortran Source Wrapper Builds](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | ordered Fortran source inputs; generated contracts; structured native plan; ABI-compatible import | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_result_records_structured_native_plan`
`tests/fortran/infrastructure/building/pipeline/test_source_generated_contracts.py::test_source_build_generated_pyi_contract_matches_fixture[fdefault_output]` | `tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py::test_debug_and_optimized_wrapper_builds_preserve_runtime_abi` | — | canonical | -| [Feature Matrix: Semantic `.pyi` Wrapper Builds From Explicit Native Artifacts](../../docs/user/language-support/feature-matrix.md#supported-inspection-features) | Partially supported | exactly one entry contract; explicit native input; source-free object build; ordered link items; current runtime subset | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_python_api_accepts_exactly_one_entry_contract`
`tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_generated_pyi_fixture_builds_from_native_object_without_source_reparse` | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_scale_runtime_contract[generated-pyi]` | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_python_api_rejects_a_missing_native_artifact` (`pipeline`) | canonical | -| [Feature Matrix: Advanced Multi-Source Dependency Discovery And External-Library Integration](../../docs/user/language-support/feature-matrix.md#unsupported-or-blocked-forms) | Blocked | undeclared sources, prebuilt module paths, and external-library discovery are caller/build-system responsibilities; dependency ordering among parsed sources is PRIK's; explicit paths remain supported | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_reuses_native_plan_for_additional_compile_and_link_inputs` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_imported_contracts_resolve_from_one_archive_or_shared_library[archive]` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_missing_module_directory_reports_compile_error` (`compiling`) | canonical | +| [Feature Matrix: Fortran Source Wrapper Builds](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | ordered Fortran source inputs; generated contracts; structured native plan; ABI-compatible import | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_result_records_structured_native_plan`
`tests/fortran/infrastructure/building/pipeline/test_source_generated_contracts.py::test_source_build_generated_pyi_contract_matches_fixture[fdefault_output]` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_makefile_mode_reproduces_multi_source_build` | — | canonical | +| [Feature Matrix: Semantic `.pyi` Wrapper Builds From Explicit Native Artifacts](../../docs/user/language-support/feature-matrix.md#supported-inspection-features) | Partially supported | exactly one entry contract; explicit native input; source-free object build; ordered link items; current runtime subset | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_python_api_accepts_exactly_one_entry_contract` | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_generated_pyi_replay_matches_source_build_without_source_reparse` | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_python_api_rejects_a_missing_native_artifact` (`pipeline`) | canonical | +| [Feature Matrix: Advanced Multi-Source Dependency Discovery And External-Library Integration](../../docs/user/language-support/feature-matrix.md#unsupported-or-blocked-forms) | Blocked | undeclared sources, prebuilt module paths, and external-library discovery are caller/build-system responsibilities; dependency ordering among parsed sources is PRIK's; explicit paths remain supported | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_reuses_native_plan_for_additional_compile_and_link_inputs` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_imported_contracts_resolve_from_one_archive_or_shared_library[archive]` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_missing_module_directory_reports_compile_error` (`compiling`) | canonical | diff --git a/tests/fortran/README.md b/tests/fortran/README.md index dbca71e80..c2a51dce6 100644 --- a/tests/fortran/README.md +++ b/tests/fortran/README.md @@ -90,7 +90,7 @@ representation is supporting evidence, not the ownership rule. | `infrastructure/naming/` | Internal generated-name and public-name policy owned by `prik/naming/` | | `infrastructure/pipeline/` | Generated-wrapper orchestration and transport owned by `prik/pipeline/` | | `infrastructure/printers/` | Internal C and Fortran source serialization owned by `prik/printers/` | -| `infrastructure/utilities/` | Internal string and class-visitor helpers owned by `prik/utilities/` | +| `infrastructure/utilities/` | Internal class-visitor helper owned by `prik/utilities/` | Each infrastructure test module has an explicit production owner. New internal coverage goes in the matching `prik/` package directory; it must not recreate a diff --git a/tests/fortran/_support/printer_models.py b/tests/fortran/_support/printer_models.py index b0e699d0b..1ee62ecb8 100644 --- a/tests/fortran/_support/printer_models.py +++ b/tests/fortran/_support/printer_models.py @@ -56,7 +56,3 @@ def rendered_source(artifacts, suffix: str) -> str: matches = [source.text for source in artifacts.sources if source.path.suffix == suffix] assert len(matches) == 1 return matches[0] - - -def normalize(text: str) -> str: - return "\n".join(line.rstrip() for line in text.strip().splitlines()) diff --git a/tests/fortran/_support/semantic_properties.py b/tests/fortran/_support/semantic_properties.py index 75686bb1c..5b8fac711 100644 --- a/tests/fortran/_support/semantic_properties.py +++ b/tests/fortran/_support/semantic_properties.py @@ -28,14 +28,6 @@ ] ) -_FORTRAN_VALUE_TYPES = st.sampled_from( - [ - ("logical", "Bool"), - ("real(8)", "Float64"), - ("real", "Float32"), - ] -) - _SEMANTIC_SCALAR_TYPES = st.sampled_from(["Bool", "Float32", "Float64", "Int32"]) _PYI_IDENTIFIER_STEMS = st.from_regex(r"[a-z][a-z0-9_]{0,8}", fullmatch=True) diff --git a/tests/fortran/allocatables/codegen/test_allocatable_lowering.py b/tests/fortran/allocatables/codegen/test_allocatable_lowering.py index d56967787..a44cc3374 100644 --- a/tests/fortran/allocatables/codegen/test_allocatable_lowering.py +++ b/tests/fortran/allocatables/codegen/test_allocatable_lowering.py @@ -6,107 +6,33 @@ from prik.policy.completion import complete_semantic_policies from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -from prik.policy.models import NativeArrayDescriptorKind, NativeDescriptorHandoffABI -def _allocatable_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Addr, Allocatable, Annotated, Arg, Float64, Int32, MaybeUnallocated, native_call - -@native_call([Addr(Arg(0))]) -def make(n: Int32) -> Allocatable[Float64[:]]: ... - -@native_call([Addr(Arg(0))]) -def maybe_make(n: Int32) -> Annotated[Allocatable[Float64[:]], MaybeUnallocated]: ... -""", - module_name="allocatable_handles", - ) +def _c_source(contract: str, module_name: str) -> str: + module = parse_pyi_text(contract, module_name=module_name) complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def _module_allocatable_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Allocatable, Float64 - -plain_allocatable: Allocatable[Float64[:]] -""", - module_name="allocatable_module_handles", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def test_plain_module_allocatable_uses_standard_descriptor_callback_without_copy(): - artifacts = WrapperGenerator().generate(_module_allocatable_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "void (*callback)(CFI_cdesc_t *, void *)" in c_source - assert "prik_module_allocatable_module_handles_plain_allocatable_descriptor_callback_with_descriptor" in c_source - # Every inquiry runs one shared consumer over the descriptor the bridge - # supplies; nothing copies the descriptor out to be read in Python. - assert "prik_native_array_read_shape(void * descriptor, void * context)" in c_source - assert "source->base_addr" in c_source - # The capability tuple is the only Python object the module builds; no - # descriptor field is packed into Python values for the handle to read back. - built = [line.strip() for line in c_source.splitlines() if "Py_BuildValue" in line] - assert built and all("build_capabilities" in line for line in built) - assert "subroutine bind_c_plain_allocatable_descriptor(" in bridge_source - assert 'bind(c, name="bind_c_plain_allocatable_descriptor")' in bridge_source - assert "type(c_funptr), value :: callback_address" in bridge_source - assert "procedure(prik_plain_allocatable_descriptor_consumer), pointer :: callback" in bridge_source - assert "call callback(native_plain_allocatable, context)" in bridge_source + artifacts = WrapperGenerator().generate(WrapperPlanner().build(module)) + return next(source.text for source in artifacts.sources if source.path.suffix == ".c") -def test_allocated_direct_result_assigns_then_moves_into_owned_descriptor(): - bridge_source = next( - source.text - for source in WrapperGenerator().generate(_allocatable_plan()).sources - if source.path.suffix == ".f90" - ) - start = bridge_source.index("subroutine bind_c_make(") - end = bridge_source.index("end subroutine", start) - procedure = bridge_source[start:end] +ALLOCATABLE_ARGUMENT_CONTRACT = """ +from prik.contracts import Allocatable, Float64, native_call, nogil - assert "real(c_double), allocatable, dimension(:), intent(out) :: result" in procedure - assert "real(c_double), allocatable, dimension(:) :: result_value" in procedure - assert "result_value = native_make(n)" in procedure - assert "if (allocated(result_value)) then" in procedure - assert "call move_alloc(result_value, result)" in procedure - assert "if (allocated(result)) then" in procedure - assert "deallocate(result)" in procedure - assert "call prik_collect_allocatable_array_result(native_make(n), result)" not in procedure - assert "result = result_value" not in procedure - assert "subroutine bind_c_owned_result_" in bridge_source - assert "_deallocate(" in bridge_source - assert "real(c_double), allocatable, dimension(:), intent(inout) :: result" in bridge_source - assert "_destroy(" in bridge_source +@nogil +@native_call([]) +def total(values: Allocatable[Float64[:]]) -> Float64: ... +plain_allocatable: Allocatable[Float64[:]] +""" -def test_maybe_unallocated_direct_result_uses_collector_without_assignment(): - bridge_source = next( - source.text - for source in WrapperGenerator().generate(_allocatable_plan()).sources - if source.path.suffix == ".f90" - ) - start = bridge_source.index("subroutine bind_c_maybe_make(") - end = bridge_source.index("end subroutine", start) - procedure = bridge_source[start:end] +FIXED_CHARACTER_OWNER_CONTRACT = """ +from prik.contracts import Allocatable, Returns, String, nogil - assert "real(c_double), allocatable, dimension(:), intent(out) :: result" in procedure - assert "call prik_collect_allocatable_array_result(native_maybe_make(n), result)" in procedure - assert "real(c_double), allocatable, dimension(:) :: value" in procedure - assert "if (allocated(value)) then" in procedure - assert "call move_alloc(value, result)" in procedure - assert "if (allocated(result)) then" in procedure - assert "deallocate(result)" in procedure - assert "result_value = native_make(n)" not in procedure - assert "call move_alloc(result_value, result)" not in procedure - assert "result = result_value" not in procedure - assert "subroutine prik_collect_allocatable_array_result(" in procedure +@nogil +def rewrite( + values: Allocatable[String[4][:]], +) -> Returns["values", Allocatable[String[4][:]]]: ... +""" def test_maybe_unallocated_is_only_valid_on_direct_allocatable_array_results(): @@ -123,23 +49,6 @@ def invalid_argument(values: Annotated[Allocatable[Float64[:]], MaybeUnallocated complete_semantic_policies(module) -def _allocatable_argument_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Allocatable, Float64, native_call, nogil - -@nogil -@native_call([]) -def total(values: Allocatable[Float64[:]]) -> Float64: ... - -plain_allocatable: Allocatable[Float64[:]] -""", - module_name="allocatable_actuals", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - def test_no_generated_binding_establishes_an_allocated_allocatable_descriptor(): """CFI_establish reserves the allocatable descriptor for the Fortran runtime. @@ -147,10 +56,10 @@ def test_no_generated_binding_establishes_an_allocated_allocatable_descriptor(): ``CFI_attribute_allocatable``: an allocatable established from C must start unallocated. Pairing that attribute with a real address describes an already-allocated allocatable, which ifx rejects with - ``CFI_ERROR_BASE_ADDR_NOT_NULL`` while gfortran silently accepts it. + ``CFI_ERROR_BASE_ADDR_NOT_NULL`` while gfortran silently accepts it, so the + gfortran end-to-end suite cannot catch it. """ - artifacts = WrapperGenerator().generate(_allocatable_argument_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") + c_source = _c_source(ALLOCATABLE_ARGUMENT_CONTRACT, "allocatable_actuals") forged = [ line.strip() @@ -160,113 +69,35 @@ def test_no_generated_binding_establishes_an_allocated_allocatable_descriptor(): assert forged == [] -def test_allocatable_argument_uses_the_descriptor_the_runtime_built(): - """The binding passes on the runtime's descriptor rather than a record of its own. - - A C descriptor is the Fortran runtime's to build, so the binding neither - establishes one for an allocatable actual nor copies the one it is handed: - the call is made inside the consumer holding it, and only that pointer - crosses. - """ - plan = _allocatable_argument_plan() - functions = {function.binding.python_name: function for function in plan.namespaces[0].functions} - argument = functions["total"].arguments[0] - handle = argument.native_array_handle - - assert handle is not None - assert handle.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE - assert handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR - - artifacts = WrapperGenerator().generate(plan) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - assert "with_descriptor(" in c_source - copied = [line.strip() for line in c_source.splitlines() if "memcpy(" in line and "CFI_CDESC_T" in line] - assert copied == [] - - -def test_nogil_releases_only_while_the_descriptor_consumer_calls_fortran(): - c_source = next( - source.text - for source in WrapperGenerator().generate(_allocatable_argument_plan()).sources - if source.path.suffix == ".c" - ) - start = c_source.index("static void wrap_total_call_with_carrier_0(") - end = c_source.index("\n}\n", start) - consumer = c_source[start:end] - - begin = consumer.index("Py_BEGIN_ALLOW_THREADS") - call = consumer.index("bind_c_total(") - finish = consumer.index("Py_END_ALLOW_THREADS") +@pytest.mark.parametrize( + ("contract", "wrapper", "native_call", "lease"), + [ + pytest.param( + ALLOCATABLE_ARGUMENT_CONTRACT, + "static void wrap_total_call_with_carrier_0(", + "bind_c_total(", + False, + id="descriptor-consumer", + ), + pytest.param( + FIXED_CHARACTER_OWNER_CONTRACT, + "static PyObject * wrap_rewrite", + "bind_c_rewrite(", + True, + id="leased-fortran-owner", + ), + ], +) +def test_nogil_releases_only_around_the_native_call(contract: str, wrapper: str, native_call: str, lease: bool): + """The GIL is released only while Fortran runs, inside any owner lease.""" + c_source = _c_source(contract, "nogil_allocatables") + start = c_source.index(wrapper) + body = c_source[start:] + + begin = body.index("Py_BEGIN_ALLOW_THREADS") + call = body.index(native_call) + finish = body.index("Py_END_ALLOW_THREADS") assert begin < call < finish - - -def test_deferred_length_character_allocation_names_its_width_in_a_type_spec(): - """The generated allocation spells the width the plan carried to it. - - ``allocate(entity(n))`` is rejected outright for a deferred length type - parameter, so the bridge has to name the width in a type-spec. It takes it - from the planned ``element_length`` argument rather than choosing one. - """ - module = parse_pyi_text( - """ -deferred: Allocatable[String[:][:]] -numeric: Allocatable[Float64[:]] -""", - module_name="deferred_character_allocation_lowering", - ) - complete_semantic_policies(module) - - bridge_source = next( - source.text - for source in WrapperGenerator().generate(WrapperPlanner().build(module)).sources - if source.path.suffix == ".f90" - ) - - assert "subroutine bind_c_deferred_resize(extent_0, element_length)" in bridge_source - assert "integer(c_int64_t), value :: element_length" in bridge_source - assert "allocate(character(kind=c_char, len=element_length) :: native_deferred(extent_0))" in bridge_source - # A width is planned only where the standard requires one. - assert "subroutine bind_c_numeric_resize(extent_0)" in bridge_source - assert "allocate(native_numeric(extent_0))" in bridge_source - - -def test_fixed_character_argument_uses_a_leased_fortran_owner(): - module = parse_pyi_text( - """ -from prik.contracts import Allocatable, Returns, String, nogil - -@nogil -def rewrite( - values: Allocatable[String[4][:]], -) -> Returns["values", Allocatable[String[4][:]]]: ... -""", - module_name="fixed_character_owner", - ) - complete_semantic_policies(module) - artifacts = WrapperGenerator().generate(WrapperPlanner().build(module)) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "sequence" in bridge_source - assert "character(kind=c_char, len=4), allocatable, dimension(:) :: data" in bridge_source - assert "type(c_ptr), value :: values" in bridge_source - assert "call c_f_pointer(values, values_owner)" in bridge_source - assert "call native_rewrite(values_owner%data)" in bridge_source - assert "void bind_c_rewrite(void * values);" in c_source - assert "prik_native_array_forward_descriptor" in c_source - assert "&forwarded" in c_source - assert "(void *)consumer" not in c_source - assert "PRIK_NATIVE_ARRAY_CONTEXT_FORTRAN_OWNER, PRIK_FORTRAN_OWNER_ABI" in c_source - assert "function bind_c_owner_" in bridge_source - assert "deallocate(owner%data, stat=status)" in bridge_source - assert "allocate(owner%data(extent_0), stat=status)" in bridge_source - assert "failed to resize Fortran array owner" in c_source - assert "failed to deallocate Fortran array owner" in c_source - - wrapper = c_source[c_source.index("static PyObject * wrap_rewrite") :] - acquire = wrapper.index("prik_native_array_backend_acquire_call") - begin = wrapper.index("Py_BEGIN_ALLOW_THREADS") - call = wrapper.index("bind_c_rewrite(") - finish = wrapper.index("Py_END_ALLOW_THREADS") - release = wrapper.index("prik_native_array_backend_release_call") - assert acquire < begin < call < finish < release + if lease: + assert body.index("prik_native_array_backend_acquire_call") < begin + assert finish < body.index("prik_native_array_backend_release_call") diff --git a/tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py b/tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py index 9726bc08e..74379474a 100644 --- a/tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py +++ b/tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py @@ -77,12 +77,39 @@ def test_caller_created_allocatable_crosses_separately_built_extensions(tmp_path values.close() assert values.closed is True + # A reader refuses a capsule with another ABI name before reading it. + tampered = Allocatable[Float64[:]]() + first.select_a(tampered) + capsule_new = ctypes.pythonapi.PyCapsule_New + capsule_new.restype = ctypes.py_object + capsule_new.argtypes = (ctypes.c_void_p, ctypes.c_char_p, ctypes.c_void_p) + capsule_get = ctypes.pythonapi.PyCapsule_GetPointer + capsule_get.restype = ctypes.c_void_p + capsule_get.argtypes = (ctypes.py_object, ctypes.c_char_p) + capsule_name = ctypes.pythonapi.PyCapsule_GetName + capsule_name.restype = ctypes.c_char_p + capsule_name.argtypes = (ctypes.py_object,) + + published = capsule_name(tampered._native_backend) + assert published.startswith(b"prik.native_array_backend.v2.") + address = capsule_get(tampered._native_backend, published) + assert address + stranger = published[: published.rindex(b".")] + b".0000000000000000" + tampered._native_backend = capsule_new(address, stranger, None) + + # Only the capsule name differs; the reader must reject it before using the + # live backend address. + with pytest.raises(ValueError, match="PyCapsule_GetPointer called with incorrect name"): + second.total_b(tampered) + -def test_fortran_owned_character_handle_crosses_matching_extensions(tmp_path: Path): +def test_fortran_owned_character_handle_crosses_matching_extensions_only(tmp_path: Path): first_dir = tmp_path / "first" second_dir = tmp_path / "second" + wrong_width_dir = tmp_path / "wrong_width" first_dir.mkdir() second_dir.mkdir() + wrong_width_dir.mkdir() first = _build_text_and_import( CHARACTER_CROSS_A_SOURCE, "fcharacter_cross_a.f90", @@ -103,6 +130,16 @@ def test_fortran_owned_character_handle_crosses_matching_extensions(tmp_path: Pa "fcharacter_cross_b_wrapper.h", }, ) + wrong_width = _build_text_and_import( + CHARACTER_CROSS_WRONG_WIDTH_SOURCE, + "fcharacter_cross_wrong_width.f90", + wrong_width_dir, + { + "bind_c_fcharacter_cross_wrong_width_wrapper.f90", + "fcharacter_cross_wrong_width_wrapper.c", + "fcharacter_cross_wrong_width_wrapper.h", + }, + ) values = Allocatable[String[4][:]]() assert first.select_a(values) is values @@ -110,6 +147,9 @@ def test_fortran_owned_character_handle_crosses_matching_extensions(tmp_path: Pa assert second.select_b(values) is values assert first.state_a(values) == np.int32(304) assert values.to_numpy().tolist() == [b"red ", b"blue", b"sky "] + # An extension whose Fortran owner has another layout refuses the handle. + with pytest.raises(TypeError, match="owner does not match"): + wrong_width.state(values) values.close() @@ -148,40 +188,6 @@ def test_fortran_owned_character_handle_crosses_matching_extensions(tmp_path: Pa alias.close() -def test_fortran_owned_character_handle_refuses_a_different_owner_layout(tmp_path: Path): - producer_dir = tmp_path / "producer" - consumer_dir = tmp_path / "consumer" - producer_dir.mkdir() - consumer_dir.mkdir() - producer = _build_text_and_import( - CHARACTER_CROSS_A_SOURCE, - "fcharacter_cross_a.f90", - producer_dir, - { - "bind_c_fcharacter_cross_a_wrapper.f90", - "fcharacter_cross_a_wrapper.c", - "fcharacter_cross_a_wrapper.h", - }, - ) - consumer = _build_text_and_import( - CHARACTER_CROSS_WRONG_WIDTH_SOURCE, - "fcharacter_cross_wrong_width.f90", - consumer_dir, - { - "bind_c_fcharacter_cross_wrong_width_wrapper.f90", - "fcharacter_cross_wrong_width_wrapper.c", - "fcharacter_cross_wrong_width_wrapper.h", - }, - ) - values = Allocatable[String[4][:]]() - producer.select_a(values) - - with pytest.raises(TypeError, match="owner does not match"): - consumer.state(values) - - values.close() - - def test_fortran_owned_character_handle_refuses_a_different_compiler_abi( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, @@ -226,45 +232,6 @@ def test_fortran_owned_character_handle_refuses_a_different_compiler_abi( values.close() -def test_a_backend_capsule_from_another_producer_is_refused_not_interpreted(tmp_path: Path): - """A reader refuses a capsule with another ABI name before reading it.""" - module = _build_text_and_import( - ALLOCATABLE_CROSS_A_SOURCE, - "fallocatable_cross_a.f90", - tmp_path, - { - "bind_c_fallocatable_cross_a_wrapper.f90", - "fallocatable_cross_a_wrapper.c", - "fallocatable_cross_a_wrapper.h", - }, - ) - values = Allocatable[Float64[:]]() - module.select_a(values) - assert module.total_a(values) == np.float64(3.0) - - capsule_new = ctypes.pythonapi.PyCapsule_New - capsule_new.restype = ctypes.py_object - capsule_new.argtypes = (ctypes.c_void_p, ctypes.c_char_p, ctypes.c_void_p) - capsule_get = ctypes.pythonapi.PyCapsule_GetPointer - capsule_get.restype = ctypes.c_void_p - capsule_get.argtypes = (ctypes.py_object, ctypes.c_char_p) - capsule_name = ctypes.pythonapi.PyCapsule_GetName - capsule_name.restype = ctypes.c_char_p - capsule_name.argtypes = (ctypes.py_object,) - - published = capsule_name(values._native_backend) - assert published.startswith(b"prik.native_array_backend.v2.") - address = capsule_get(values._native_backend, published) - assert address - stranger = published[: published.rindex(b".")] + b".0000000000000000" - values._native_backend = capsule_new(address, stranger, None) - - # Only the capsule name differs; the reader must reject it before using the - # live backend address. - with pytest.raises(ValueError, match="PyCapsule_GetPointer called with incorrect name"): - module.total_a(values) - - @pytest.mark.skipif(shutil.which("valgrind") is None, reason="Valgrind is required for native ownership checks") def test_allocatable_replacement_has_no_native_memory_errors( pyi_parity_build_mode: str, diff --git a/tests/fortran/allocatables/end_to_end/test_allocatable_handles.py b/tests/fortran/allocatables/end_to_end/test_allocatable_handles.py index 1c40a9a77..d40256f11 100644 --- a/tests/fortran/allocatables/end_to_end/test_allocatable_handles.py +++ b/tests/fortran/allocatables/end_to_end/test_allocatable_handles.py @@ -187,6 +187,9 @@ def test_allocatable_module_fields_and_results_expose_lifetime_safe_handles( built_matrix.to_numpy(), np.array([[11.0, 21.0], [12.0, 22.0]], dtype=np.float64), ) + # A rank-two resize hands each extent to native code separately. + built_matrix.resize((3, 2)) + assert built_matrix.shape == (3, 2) empty_matrix = module.build_matrix(np.int32(0), np.int32(2)) assert isinstance(empty_matrix, AllocatableArray) assert empty_matrix.allocated is False @@ -232,6 +235,7 @@ def test_allocatable_module_fields_and_results_expose_lifetime_safe_handles( fresh = Allocatable[Float64[:]]() assert fresh.allocated is False + assert fresh.owned is True assert module.replace_values(fresh, np.int32(3)) is fresh np.testing.assert_allclose(fresh.to_numpy(), np.array([3.0, 6.0, 9.0], dtype=np.float64)) with pytest.raises(TypeError): @@ -301,6 +305,10 @@ def test_plain_allocatable_module_array_exposes_current_live_view( assert "void (*callback)(CFI_cdesc_t *, void *)" in wrapper_source_text assert "source->base_addr" in wrapper_source_text + # Inquiries read the descriptor in C; the capability tuple is the only + # Python value built, so no descriptor field is copied out to Python. + built = [line.strip() for line in wrapper_source_text.splitlines() if "Py_BuildValue" in line] + assert built and all("build_capabilities" in line for line in built) handle = module.values assert isinstance(handle, AllocatableArray) @@ -353,7 +361,8 @@ def test_module_allocatable_reports_its_real_lower_bound_with_or_without_target( the bounds, strides and element length then have to come from somewhere else. Reconstructing them hardcoded a lower bound of zero, which is wrong for every Fortran array — the default is one — and further wrong for a declared `(5:8)`. - Both declarations read the descriptor, so both report 5. + Both declarations read the descriptor, so both report 5. Fixed-width + character handles also report absence without an element width to guess. """ module = _build_text_and_import( LOWER_BOUND_SOURCE, @@ -367,6 +376,18 @@ def test_module_allocatable_reports_its_real_lower_bound_with_or_without_target( ) module.setup() + missing = module.missing_words + assert missing.allocated is False + assert missing.shape is None + assert missing.to_numpy() is None + assert missing.dtype == np.dtype("S5") + + missing_pointer = module.missing_pointer + assert missing_pointer.associated is False + assert missing_pointer.shape is None + missing_pointer.associate(missing_pointer) + assert missing_pointer.associated is False + # The bound is not a reported fact but part of the value: an allocatable # dummy adopts the bounds of the descriptor it is given, so a wrong one # makes the callee index the wrong elements. @@ -383,6 +404,10 @@ def test_module_allocatable_reports_its_real_lower_bound_with_or_without_target( assert getattr(module, name).dtype == np.dtype(f"S{width}"), name fixed = module.fixed_words + assert fixed.allocated is True + assert fixed.to_numpy().tolist() == [b"aaaaa"] * 4 + with pytest.raises(TypeError, match="descriptor attribute required by the dummy"): + module.deferred_word_bound_and_width(fixed) fixed.resize(2) assert fixed.dtype == np.dtype("S5") with pytest.raises(TypeError, match="fixed element width"): @@ -398,42 +423,10 @@ def test_module_allocatable_reports_its_real_lower_bound_with_or_without_target( assert module.deferred_word_bound_and_width(deferred) == np.int32(106) -def test_fixed_character_projection_reports_absence(tmp_path: Path): - module = _build_text_and_import( - LOWER_BOUND_SOURCE, - "falloc_lower_bounds_f90.f90", - tmp_path, - { - "bind_c_falloc_lower_bounds_f90_wrapper.f90", - "falloc_lower_bounds_f90_wrapper.c", - "falloc_lower_bounds_f90_wrapper.h", - }, - ) - module.setup() - - missing = module.missing_words - assert missing.allocated is False - assert missing.shape is None - assert missing.to_numpy() is None - assert missing.dtype == np.dtype("S5") - - missing_pointer = module.missing_pointer - assert missing_pointer.associated is False - assert missing_pointer.shape is None - missing_pointer.associate(missing_pointer) - assert missing_pointer.associated is False - - fixed = module.fixed_words - assert fixed.allocated is True - assert fixed.to_numpy().tolist() == [b"aaaaa"] * 4 - with pytest.raises(TypeError, match="descriptor attribute required by the dummy"): - module.deferred_word_bound_and_width(fixed) - - BORROWED_DESCRIPTOR_SOURCE = (NATIVE_FIXTURES / "fallocatable_borrowed_f90.f90").read_text(encoding="utf-8") -def test_every_allocatable_handle_kind_reaches_a_read_only_allocatable_dummy(tmp_path: Path): +def test_every_allocatable_handle_kind_reaches_read_only_and_writable_allocatable_dummies(tmp_path: Path): """An allocatable actual borrows the descriptor the Fortran runtime built. A read-only allocatable dummy requires an allocatable actual, and C may not @@ -442,13 +435,18 @@ def test_every_allocatable_handle_kind_reaches_a_read_only_allocatable_dummy(tmp rebuilding one from facts, so module, derived-field and result handles all reach the dummy on every compiler instead of only where an invalid descriptor happens to be tolerated. + + A callee that reallocates an ``intent(inout)`` dummy must update the caller. + The descriptor a module array or field hands out exists only while the + consumer holding it runs, so a callee handed a copy would reallocate the + copy and leave the caller's entity naming released storage. The call is + made inside that consumer instead. A handle owning its descriptor hands + that over directly. """ - workdir = tmp_path / "borrowed" - workdir.mkdir(parents=True) module = _build_text_and_import( BORROWED_DESCRIPTOR_SOURCE, "fallocatable_borrowed_f90.f90", - workdir, + tmp_path, { "bind_c_fallocatable_borrowed_f90_wrapper.f90", "fallocatable_borrowed_f90_wrapper.c", @@ -472,31 +470,6 @@ def test_every_allocatable_handle_kind_reaches_a_read_only_allocatable_dummy(tmp namespace.modvar.to_numpy()[:] = [100.0, 200.0, 300.0] assert namespace.total(namespace.modvar) == np.float64(600.0) - -def test_a_writable_allocatable_dummy_reaches_the_callers_entity(tmp_path: Path): - """A callee that reallocates an ``intent(inout)`` dummy updates the caller. - - The descriptor a module array or field hands out exists only while the - consumer holding it runs, so a callee handed a copy would reallocate the - copy and leave the caller's entity naming released storage. The call is - made inside that consumer instead, which is what lets the new allocation - reach the entity. A handle owning its descriptor hands that over directly - and needs no such arrangement. - """ - workdir = tmp_path / "writable" - workdir.mkdir(parents=True) - module = _build_text_and_import( - BORROWED_DESCRIPTOR_SOURCE, - "fallocatable_borrowed_f90.f90", - workdir, - { - "bind_c_fallocatable_borrowed_f90_wrapper.f90", - "fallocatable_borrowed_f90_wrapper.c", - "fallocatable_borrowed_f90_wrapper.h", - }, - ) - namespace = _sole_native_module(module) - # A module array: the callee replaces the allocation, and the module # variable names the new one afterwards. namespace.modvar.resize(2) diff --git a/tests/fortran/allocatables/end_to_end/test_deferred_character_handles.py b/tests/fortran/allocatables/end_to_end/test_deferred_character_handles.py index 89d44ccb7..e60844875 100644 --- a/tests/fortran/allocatables/end_to_end/test_deferred_character_handles.py +++ b/tests/fortran/allocatables/end_to_end/test_deferred_character_handles.py @@ -122,6 +122,11 @@ def replace_names( assert handle.dtype == np.dtype("S4") with pytest.raises(TypeError, match="needs an element_length"): handle.resize(2) + with pytest.raises(ValueError, match="must not be negative"): + handle.resize(2, element_length=-1) + with pytest.raises(TypeError, match="must be an integer"): + handle.resize(2, element_length=1.5) + assert handle.shape == (3,) caller_created = Allocatable[String[:][:]]() assert module.replace_names(caller_created) is caller_created diff --git a/tests/fortran/allocatables/parsing/test_allocatable_declarations.py b/tests/fortran/allocatables/parsing/test_allocatable_declarations.py deleted file mode 100644 index 562d7bda1..000000000 --- a/tests/fortran/allocatables/parsing/test_allocatable_declarations.py +++ /dev/null @@ -1,18 +0,0 @@ -"""Tests split by stable ownership concept from `test_procedures_and_interfaces.py`.""" - -from tests.fortran._support.parser_procedures import parse_fortran_module - - -def test_module_allocatable_target_attribute_is_preserved(): - module = parse_fortran_module( - """ -module alloc_target_mod - real(8), allocatable, target :: values(:) -end module alloc_target_mod -""" - ) - - values = module.variables[0] - assert values.name == "values" - assert values.allocatable is True - assert values.target is True diff --git a/tests/fortran/allocatables/pipeline/test_allocatable_contract_printing.py b/tests/fortran/allocatables/pipeline/test_allocatable_contract_printing.py deleted file mode 100644 index 46bea4e1d..000000000 --- a/tests/fortran/allocatables/pipeline/test_allocatable_contract_printing.py +++ /dev/null @@ -1,38 +0,0 @@ -"""Generated contract surface for allocatable outputs and results.""" - -from prik.printers import emit_module -from prik.semantics.fortran2ir import fortran_module_to_semantic_module -from prik.parsers.fortran import parse_fortran_file as parse_fortran_source - - -def _generate_pyi(source: str) -> str: - return emit_module(fortran_module_to_semantic_module(parse_fortran_source(source))) - - -def test_emit_allocatable(): - source = """ -module alloc_mod - -contains - -subroutine build(x) - - real(8), allocatable, intent(out) :: x(:) - -end subroutine - -function make_values() result(x) - - real(8), allocatable :: x(:) - -end function - -end module -""" - - code = _generate_pyi(source) - - assert "Allocatable" in code - assert "@native_call([Return('x', 0)])" in code - assert "def build() -> Allocatable[Float64[:]]: ..." in code - assert "def make_values() -> Allocatable[Float64[:]]: ..." in code diff --git a/tests/fortran/allocatables/pipeline/test_allocatable_module_contract_printing.py b/tests/fortran/allocatables/pipeline/test_allocatable_module_contract_printing.py deleted file mode 100644 index d6fa9aa0f..000000000 --- a/tests/fortran/allocatables/pipeline/test_allocatable_module_contract_printing.py +++ /dev/null @@ -1,32 +0,0 @@ -"""Generated and reparsed allocatable module and field declarations.""" - -from prik.printers import emit_module -from prik.semantics.fortran2ir import fortran_module_to_semantic_module -from prik.parsers.fortran import parse_fortran_file as parse_fortran_source -from prik.pipeline.pyi import pyi_text_to_semantic_module as parse_pyi_text - - -def _generate_pyi(source: str) -> str: - return emit_module(fortran_module_to_semantic_module(parse_fortran_source(source))) - - -def test_emit_and_load_allocatable_module_variable_declaration(): - source = """ -module alloc_view_mod - real(8), allocatable, target :: values(:) - type :: box - real(8), allocatable :: field(:) - end type box -end module alloc_view_mod -""" - code = _generate_pyi(source) - - assert "values: Annotated[Allocatable[Float64[:]], Aliased]" in code - assert "field: Allocatable[Float64[:]]" in code - - loaded = parse_pyi_text(code, module_name="alloc_view_mod") - assert [variable.name for variable in loaded.variables] == ["values"] - assert loaded.variables[0].semantic_type.storage.array.allocatable is True - assert loaded.variables[0].semantic_type.metadata["aliased"] is True - assert loaded.classes[0].fields[0].semantic_type.storage.array.allocatable is True - assert "aliased" not in loaded.classes[0].fields[0].semantic_type.metadata diff --git a/tests/fortran/allocatables/policy/test_allocatable_handle_policy.py b/tests/fortran/allocatables/policy/test_allocatable_handle_policy.py index 8307d7ac4..90cd7c71d 100644 --- a/tests/fortran/allocatables/policy/test_allocatable_handle_policy.py +++ b/tests/fortran/allocatables/policy/test_allocatable_handle_policy.py @@ -1,112 +1,10 @@ -"""Tests split by stable ownership concept from `test_handle_policy_dispatch.py`.""" +"""Allocatable descriptor-handle routing decisions completed before lowering.""" -from prik.semantics.models import ( - RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA, - RESOLVED_OWNERSHIP_POLICY_METADATA, -) -from prik.policy.native_array_handles import ( - NativeArrayBuildRequirement, - native_array_handle_build_requirements, -) -from prik.policy.ownership import ( - CodegenAction, - DestructionPolicy, - NativeBarrierAction, - OwnershipContext, - OwnershipOwner, - StorageMode, - TransferMode, - default_ownership_policy, -) -from prik.policy.completion import complete_semantic_policies from prik.parsers.fortran import parse_fortran_file from prik.planning import WrapperPlanner +from prik.policy.completion import complete_semantic_policies from prik.policy.models import NativeArrayOwnerStorage, NativeDescriptorHandoffABI, NativeEntrypointAction from prik.semantics.fortran2ir import fortran_module_to_semantic_module -from tests.fortran._support.ownership_policy import ( - _array_type, - parse_pyi_text, -) - - -def test_allocatable_array_field_is_wrapper_owned_borrowed_view(): - decision = default_ownership_policy.decide_semantic_type( - _array_type(allocatable=True), - OwnershipContext.field(), - ) - - assert decision.owner is OwnershipOwner.WRAPPER - assert decision.transfer is TransferMode.BORROWED_VIEW - assert decision.destruction is DestructionPolicy.WRAPPER_DEALLOC - assert decision.storage_mode is StorageMode.HEAP - assert decision.borrowed is True - assert decision.nullable is True - - -def test_hidden_allocatable_handle_output_completes_as_owned_result_before_lowering(): - module = parse_pyi_text( - """ -@native_call([Return("values", 0)]) -def make_values() -> Allocatable[Float64[:]]: ... -""", - module_name="hidden_allocatable_handle_result", - ) - complete_semantic_policies(module) - - argument = module.functions[0].arguments[0] - decision = argument.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA] - policy = argument.metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - - assert decision.owner is OwnershipOwner.WRAPPER - assert decision.transfer is TransferMode.WRAPPER_INSTANCE - assert decision.destruction is DestructionPolicy.WRAPPER_DEALLOC - assert decision.codegen_action is CodegenAction.WRAPPER_INSTANCE - assert decision.native_barrier_action is NativeBarrierAction.PASS_NATIVE_DESCRIPTOR - assert policy.handle_kind == "owned_result_descriptor" - assert policy.origin == "projected_result" - assert policy.owner_retention == "wrapper_owner_storage" - assert policy.descriptor_ownership == "owned" - assert policy.output_projection == "projected_handle" - - -def test_visible_descriptor_writeback_completes_caller_handle_construction_lifecycle(): - module = parse_pyi_text( - """ -@native_call([Arg(0)]) -def replace_values( - values: Allocatable[Float64[:]], -) -> Returns["values", Allocatable[Float64[:]]]: ... -""", - module_name="caller_created_handle", - ) - complete_semantic_policies(module) - - policy = module.functions[0].arguments[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - - assert policy.default_construction == "lazy_owned_descriptor" - assert policy.default_descriptor_ownership == "owned" - assert policy.default_release == "wrapper_dealloc" - assert policy.default_destroy_behavior == "handle_finalizer" - assert "destroy" in policy.default_operations - - -def test_deferred_character_writeback_uses_fortran_owner_storage(): - module = parse_pyi_text( - """ -def replace_names( - values: Allocatable[String[:][:]], -) -> Returns["values", Allocatable[String[:][:]]]: ... -""", - module_name="caller_created_deferred_character_handle", - ) - complete_semantic_policies(module) - - policy = module.functions[0].arguments[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - - assert policy.default_construction == "lazy_fortran_owner" - assert policy.owner_storage == "fortran_owner" - assert policy.element_length_argument is True - assert "destroy" in policy.default_operations def test_deferred_character_owner_storage_keeps_the_direct_descriptor_abi(): @@ -130,104 +28,3 @@ def test_deferred_character_owner_storage_keeps_the_direct_descriptor_abi(): assert handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR assert function.entrypoint.action is NativeEntrypointAction.DIRECT_C_ABI assert function.entrypoint.symbol_name == "rewrite" - - -def test_aliased_does_not_change_allocatable_live_view_semantics(): - """A module allocatable is reached through its descriptor either way. - - `Aliased` would allow `c_loc` on the variable, but that yields only a base - address: lower bounds, strides and element length would then have to be - assumed rather than read, and a non-default lower bound makes the assumption - wrong. Both declarations therefore complete to the same descriptor policy. - """ - module = parse_pyi_text( - """ -values: Allocatable[Float64[:]] -shared_values: Annotated[Allocatable[Float64[:]], Aliased] -""", - module_name="allocatable_numpy_policy", - ) - - complete_semantic_policies(module) - - values = module.variables[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - shared_values = module.variables[1].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - - assert values.to_numpy == shared_values.to_numpy == "descriptor_view" - assert values.descriptor_interop == shared_values.descriptor_interop == "module_allocatable_c_descriptor" - assert values.owner == shared_values.owner == "native" - assert values.borrowed is shared_values.borrowed is True - - -def test_owned_allocatable_result_records_local_standard_c_descriptor_build_requirement(): - module = parse_pyi_text( - """ -def make_values() -> Allocatable[Float64[:]]: ... -""", - module_name="owned_allocatable_build", - ) - complete_semantic_policies(module) - - requirements = native_array_handle_build_requirements(module) - - assert requirements.pointer_c_descriptor_interop is False - assert requirements.requires_iso_fortran_binding is True - assert requirements.headers == ("ISO_Fortran_binding.h",) - assert requirements.items == ( - NativeArrayBuildRequirement( - owner="owned_allocatable_build.make_values.return", - item="return", - descriptor_kind="allocatable", - handle_kind="owned_result_descriptor", - descriptor_interop="owned_allocatable_c_descriptor", - headers=("ISO_Fortran_binding.h",), - ), - ) - - -def test_deferred_length_character_allocation_plans_an_explicit_element_length(): - """A deferred length has no width to reuse, so allocation must be given one. - - The standard rejects an allocate-object with a deferred length type - parameter unless a type-spec, SOURCE or MOLD supplies the width, so policy - completes the width as a planned argument instead of leaving the bridge to - invent one. Every other entity already knows its element width. - """ - module = parse_pyi_text( - """ -deferred: Allocatable[String[:][:]] -fixed: Allocatable[String[8][:]] -numeric: Allocatable[Float64[:]] -""", - module_name="deferred_character_allocation", - ) - - complete_semantic_policies(module) - - deferred, fixed, numeric = ( - variable.metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] for variable in module.variables - ) - - assert deferred.element_length_argument is True - assert fixed.element_length_argument is False - assert numeric.element_length_argument is False - - -def test_a_deferred_length_character_array_can_be_resized(): - """Shape mutation is available once the width travels with the extents. - - Resize was previously withheld from these arrays because the generated - allocation had no width to name. That reason is gone, so withholding the - operation would only remove a capability the entity supports. - """ - module = parse_pyi_text( - "deferred: Allocatable[String[:][:]]", - module_name="deferred_character_resize", - ) - - complete_semantic_policies(module) - - policy = module.variables[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - - assert policy.allows("resize") - assert policy.allows("deallocate") diff --git a/tests/fortran/allocatables/policy/test_allocatable_result_policy.py b/tests/fortran/allocatables/policy/test_allocatable_result_policy.py index 1db56137a..cc5042358 100644 --- a/tests/fortran/allocatables/policy/test_allocatable_result_policy.py +++ b/tests/fortran/allocatables/policy/test_allocatable_result_policy.py @@ -1,57 +1,9 @@ -from pathlib import Path +"""Allocatable result forms that policy must block before codegen.""" - -from tests.fortran._support.ownership_policy import parse_pyi_text -from tests.fortran._support.wrapper_build import wrapper_source -from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig, read_fortran_source -from prik.semantics.fortran2ir import fortran_project_to_semantic_modules -from prik.semantics.models import ( - RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, - RESOLVED_OWNERSHIP_POLICY_METADATA, -) -from prik.policy.ownership import ( - NativeBarrierAction, -) from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - FunctionWrapperPolicy, - NativeArrayDescriptorKind, - NativeDescriptorHandoffABI, -) - -FMATH_CONTRACT = Path("tests/fortran/data_types/end_to_end/fixtures/contracts/fmath/__init__.pyi") - - -def _source_semantic_module(filename: str, *, module_name: str): - source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name=module_name) - complete_semantic_policies(module) - return module - - -def test_hidden_scalar_descriptor_result_keeps_descriptor_policy_instead_of_plain_address_storage(): - module = parse_pyi_text( - """ -@native_call([Allocatable(Return("value", 0))]) -def create_allocatable() -> Float64 | None: ... -""", - module_name="descriptor_result", - ) - function = module.functions[0] - result_argument = function.arguments[0] - - complete_semantic_policies(module) - - decision = result_argument.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA] - assert function.projection[0].value_kind == "allocatable" - assert result_argument.semantic_type.storage is None - assert decision.descriptor_boundary is True - assert decision.native_barrier_action is NativeBarrierAction.PASS_VALUE +from prik.policy.models import FunctionWrapperPolicy +from prik.semantics.models import RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA +from tests.fortran._support.ownership_policy import parse_pyi_text def test_direct_allocatable_scalar_function_result_is_blocked_before_codegen(): @@ -72,25 +24,3 @@ def maybe_allocatable(flag: Int32) -> Float64 | None: ... "direct allocatable scalar function results cannot preserve unallocated state; " "use an allocatable hidden output projection" ) in policy.blockers - - -def test_direct_high_rank_allocatable_function_result_is_supported_before_codegen(): - module = parse_pyi_text( - """ -@native_call([Addr(Arg(0)), Addr(Arg(1))]) -def make_matrix(n: Int32, m: Int32) -> Allocatable[Float64[:, :]]: ... -""", - module_name="direct_allocatable_matrix_result", - ) - - complete_semantic_policies(module) - - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - assert isinstance(policy, FunctionWrapperPolicy) - assert policy.supported is True - assert policy.blockers == () - result = policy.results[0] - assert result.rank == 2 - assert result.native_array_handle is not None - assert result.native_array_handle.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE - assert result.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE diff --git a/tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py b/tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py deleted file mode 100644 index 5e0600c72..000000000 --- a/tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py +++ /dev/null @@ -1,184 +0,0 @@ -"""Runtime constructors exposed by concrete prik contract annotations.""" - -import numpy as np -import pytest - -import prik.contracts as contracts -from prik.runtime.handles import ( - AllocatableArray, - _bind_contract_native_array_handle, - _native_array_backend_for_binding, -) -from tests.fortran._support.native_array_handles import _generated_handle_dispatch, _handle_dispatch - - -def test_fresh_contract_handle_has_no_descriptor_to_hand_over_on_its_own(): - """A handle the caller made owns no descriptor until the wrapper gives it one. - - Nothing rebuilds a descriptor from reported fields any more, so the only - thing such a handle can supply is storage a generated binder attached to - it. Reaching the call without that is a wrapper bug, not a caller error, - so it is refused rather than papered over with an empty descriptor. - """ - handle = contracts.Allocatable[contracts.Float64[:]]() - - with pytest.raises(TypeError, match="requires generated persistent descriptor storage"): - _native_array_backend_for_binding( - handle, - descriptor_kind="allocatable", - expected_dtype=np.float64, - expected_rank=1, - ) - - -def test_contract_default_allocatable_constructor_preserves_dtype_rank_and_empty_state(): - handle = contracts.Allocatable[contracts.Float64[:]]() - - assert isinstance(handle, AllocatableArray) - assert handle.dtype == np.dtype(np.float64) - assert handle.rank == 1 - assert handle.owned is True - assert handle.allocated is False - assert handle.shape is None - assert handle.to_numpy() is None - - -@pytest.mark.parametrize( - ("factory", "message"), - [ - ( - lambda: contracts.Allocatable[contracts.Float64](), - "scalar allocatable contracts", - ), - ( - lambda: contracts.Allocatable[contracts.Float64[()]](), - "positive array rank", - ), - ], -) -def test_non_array_allocatable_annotations_are_not_factories(factory, message: str): - with pytest.raises(TypeError, match=message): - factory() - - -@pytest.mark.parametrize( - ("prepare", "dtype", "rank", "error", "message"), - [ - ( - lambda: AllocatableArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: None, - "allocated": lambda _handle: False, - } - ), - to_numpy_policy="unsupported", - ), - "float64", - 1, - TypeError, - "fresh contract handle", - ), - ( - lambda: contracts.Allocatable[contracts.Float64[:]](), - "float64", - 2, - ValueError, - "does not match generated rank 2", - ), - ( - lambda: contracts.Allocatable[contracts.Float64[:]](), - "int32", - 1, - TypeError, - "does not match generated dtype", - ), - ], -) -def test_generated_storage_rejects_incompatible_allocatable_contract_handles( - prepare, - dtype: str, - rank: int, - error: type[Exception], - message: str, -): - handle = prepare() - - with pytest.raises(error, match=message): - operations = {} - _bind_contract_native_array_handle( - handle, - "allocatable", - dtype, - rank, - _generated_handle_dispatch(operations), - operations, - object(), - "owned", - "unsupported", - ) - - -def test_writable_contract_handle_adopts_generated_storage_and_closes_once(): - """Binding attaches storage, and the backend over it is what goes to the call. - - An owned handle's backend and its owner are the same capsule: the record - holds the descriptor the binder allocated, and every later call reads it - without coming back through Python. - """ - handle = contracts.Allocatable[contracts.Float64[:]]() - calls = [] - owner = object() - - def bind_default(value): - operations = { - "shape": lambda received_owner: calls.append(("shape", received_owner)) or None, - "allocated": lambda received_owner: False, - "destroy": lambda received_owner: calls.append(("destroy", received_owner)), - } - _bind_contract_native_array_handle( - value, - "allocatable", - "float64", - 1, - _generated_handle_dispatch(operations), - operations, - owner, - "owned", - "unsupported", - native_backend=owner, - ) - - assert _native_array_backend_for_binding( - handle, - descriptor_kind="allocatable", - expected_dtype=np.float64, - expected_rank=1, - bind_default=bind_default, - ) == (owner,) - assert handle.owner == owner - - handle.close() - handle.close() - assert calls == [("destroy", owner)] - - -def test_generated_storage_rejects_a_closed_contract_handle(): - handle = contracts.Allocatable[contracts.Float64[:]]() - handle.close() - - with pytest.raises(ReferenceError, match="handle is closed"): - operations = {} - _bind_contract_native_array_handle( - handle, - "allocatable", - "float64", - 1, - _generated_handle_dispatch(operations), - operations, - object(), - "owned", - "unsupported", - ) diff --git a/tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py b/tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py deleted file mode 100644 index ece7fb394..000000000 --- a/tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py +++ /dev/null @@ -1,249 +0,0 @@ -"""Allocatable handle state, live-view, and operation protocol.""" - -import numpy as np -import pytest -from prik.runtime.handles import ( - AllocatableArray, - NativeArrayHandleBase, -) -from tests.fortran._support.native_array_handles import ( - _ArrayState, - _common_ops, - _handle_dispatch, -) - - -def test_allocatable_handle_uses_common_metadata_shape_owner_and_numpy_dispatch(): - owner = object() - state = _ArrayState(shape=(2, 3), value=np.zeros((2, 3), dtype=np.float64)) - ops = { - **_common_ops(state), - "allocated": lambda _handle: state.shape is not None, - "deallocate": lambda _handle: setattr(state, "shape", None), - "resize": lambda _handle, shape: setattr(state, "shape", shape), - } - - handle = AllocatableArray( - dtype="float64", - rank=2, - **_handle_dispatch(ops), - owner=owner, - descriptor_ownership="borrowed", - generation=7, - ) - - assert isinstance(handle, NativeArrayHandleBase) - assert handle.descriptor_kind == "allocatable" - assert isinstance(handle.dtype, np.dtype) - assert handle.dtype == np.dtype("float64") - assert handle.rank == 2 - assert handle.shape == (2, 3) - assert handle.to_numpy() is state.value - assert handle.owner is owner - assert handle.borrowed is True - assert handle.owned is False - assert handle.to_numpy_policy == "borrowed_view" - assert handle.generation == 7 - assert handle.allocated is True - - -def test_allocatable_extraction_reports_unallocated_state_as_no_view(): - """Absence is reported by the extraction, not asked about beforehand. - - The generated extraction reads the descriptor, which is where whether the - storage exists is recorded, so nothing has to test allocation first. - """ - handle = AllocatableArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: None, - "to_numpy": lambda _handle: None, - "allocated": lambda _handle: pytest.fail("extraction must not need the allocation state"), - } - ), - ) - - assert handle.to_numpy() is None - - -def test_allocatable_handle_reports_absent_state_and_routes_resize_deallocate(): - state = _ArrayState() - ops = { - **_common_ops(state), - "allocated": lambda _handle: state.shape is not None, - "deallocate": lambda _handle: setattr(state, "shape", None), - "resize": lambda _handle, shape: setattr(state, "shape", shape), - } - handle = AllocatableArray(dtype="float64", rank=1, **_handle_dispatch(ops)) - - assert handle.allocated is False - assert handle.shape is None - assert handle.to_numpy() is None - - handle.resize(4) - assert handle.allocated is True - assert handle.shape == (4,) - - handle.deallocate() - assert handle.allocated is False - assert handle.shape is None - - -def test_allocatable_to_numpy_policy_returns_mutable_borrowed_view(): - source = np.array([1.0, 2.0, 3.0], dtype=np.float64) - state = _ArrayState(shape=source.shape, value=source) - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch( - { - **_common_ops(state), - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - } - ), - to_numpy_policy="borrowed_view", - ) - - view = handle.to_numpy() - - assert view is source - assert view.flags.writeable is True - view[1] = 8.0 - assert source[1] == 8.0 - - -def test_allocatable_to_numpy_explicit_copy_is_independent(): - source = np.array([1.0, 2.0, 3.0], dtype=np.float64) - state = _ArrayState(shape=source.shape, value=source) - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch( - { - **_common_ops(state), - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - } - ), - to_numpy_policy="descriptor_view", - ) - - view = handle.to_numpy() - independent = view.copy() - - assert np.shares_memory(view, source) is True - assert np.shares_memory(independent, source) is False - source[0] = 99.0 - assert view[0] == 99.0 - assert independent[0] == 1.0 - - -def test_allocatable_handle_requires_generated_allocated_operation(): - with pytest.raises(ValueError, match="requires generated operation 'allocated'"): - AllocatableArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (1,), - "to_numpy": lambda _handle: None, - } - ), - ) - - -def test_allocatable_operations_are_gated_by_completed_capabilities(): - handle = AllocatableArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: None, - "allocated": lambda _handle: False, - } - ), - to_numpy_policy="unsupported", - ) - - with pytest.raises(NotImplementedError, match="operation 'deallocate' is not available"): - handle.deallocate() - with pytest.raises(NotImplementedError, match="operation 'resize' is not available"): - handle.resize(2) - - -def test_close_is_a_noop_for_a_borrowed_allocatable_handle(): - owner = object() - handle = AllocatableArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: None, - "allocated": lambda _handle: False, - } - ), - owner=owner, - descriptor_ownership="borrowed", - to_numpy_policy="unsupported", - ) - - assert handle.close() is None - assert handle.closed is False - assert handle.owner is owner - assert handle.allocated is False - - -def test_deferred_length_character_allocation_requires_an_element_length(): - """The width is asked for exactly where the entity cannot supply one. - - A handle with no static dtype reads its width from the descriptor, which is - the same condition under which the standard refuses to allocate from a - shape alone. Passing a width anywhere else would be a second, conflicting - source for a width the entity already has. - """ - calls: list[tuple[str, tuple]] = [] - - def invoke(name, args): - calls.append((name, args)) - return 6 if name == "element_length" else None - - deferred = AllocatableArray( - invoke=invoke, - capabilities=("allocated", "deallocate", "element_length", "resize", "shape", "to_numpy"), - dtype=None, - rank=1, - to_numpy_policy="descriptor_view", - element_length_argument=True, - ) - - deferred.resize(3, element_length=4) - assert calls[-1][0] == "resize" - assert [int(value) for value in calls[-1][1]] == [3, 4] - - with pytest.raises(TypeError, match="needs an element_length"): - deferred.resize(3) - - with pytest.raises(ValueError, match="must not be negative"): - deferred.resize(3, element_length=-1) - - with pytest.raises(TypeError, match="must be an integer"): - deferred.resize(3, element_length=1.5) - - -def test_a_fixed_width_handle_refuses_an_element_length(): - """A handle that knows its width will not take a second one.""" - fixed = AllocatableArray( - invoke=lambda name, args: None, - capabilities=("allocated", "resize", "shape", "to_numpy"), - dtype="S8", - rank=1, - to_numpy_policy="descriptor_view", - ) - - with pytest.raises(TypeError, match="fixed element width"): - fixed.resize(4, element_length=8) diff --git a/tests/fortran/allocatables/semantics/test_fortran_allocatable_semantics.py b/tests/fortran/allocatables/semantics/test_fortran_allocatable_semantics.py deleted file mode 100644 index f9fe57322..000000000 --- a/tests/fortran/allocatables/semantics/test_fortran_allocatable_semantics.py +++ /dev/null @@ -1,55 +0,0 @@ -"""Tests split by stable ownership concept from `test_compile_time_values.py`.""" - -from prik.semantics.fortran2ir import ( - FortranToIRConverter, - fortran_module_to_semantic_module, -) -from prik.semantics.models import ProjectionMapping -from tests.fortran._support.semantic_conversion import array_contract -from prik.parsers.fortran import parse_fortran_file as parse_fortran_source - - -def test_converter_preserves_allocatable_target_metadata(): - source = """ -module alloc_target_mod - real(8), allocatable, target :: values(:) - type :: box - real(8), allocatable :: field(:) - end type box -end module alloc_target_mod -""" - module = FortranToIRConverter().visit(parse_fortran_source(source).modules[0]) - - values = module.variables[0] - assert values.name == "values" - assert values.semantic_type.storage.array.allocatable is True - assert values.semantic_type.metadata["aliased"] is True - - field = module.classes[0].fields[0] - assert field.semantic_type.storage.array.allocatable is True - assert "aliased" not in field.semantic_type.metadata - - -def test_allocatable_output_semantics_projects_a_hidden_descriptor_handle(): - source = """ -module alloc_mod -contains -subroutine build(x) - real(8), allocatable, intent(out) :: x(:) -end subroutine build -end module alloc_mod -""" - module = fortran_module_to_semantic_module(parse_fortran_source(source)) - function = module.functions[0] - output = function.arguments[0] - - assert array_contract(output.semantic_type).allocatable is True - assert function.projection == [ - ProjectionMapping( - python_name="x", - native_name="x", - native_position=0, - python_position=None, - result_position=0, - ) - ] diff --git a/tests/fortran/allocatables/semantics/test_pyi_allocatable_semantics.py b/tests/fortran/allocatables/semantics/test_pyi_allocatable_semantics.py index 8f86c4806..296f73923 100644 --- a/tests/fortran/allocatables/semantics/test_pyi_allocatable_semantics.py +++ b/tests/fortran/allocatables/semantics/test_pyi_allocatable_semantics.py @@ -4,66 +4,9 @@ import pytest -from prik.semantics.metadata import PROJECTED_OUTPUT_METADATA -from prik.semantics.native_array_handles import native_array_descriptor_kind from prik.pipeline.pyi import pyi_text_to_semantic_module as parse_pyi_text -def test_persistent_allocatable_descriptors_preserve_scalar_and_array_kinds(): - module = parse_pyi_text( - """ -from prik.contracts import Aliased, Allocatable, Annotated, Float64 - -scratch: Allocatable[Float64] -values: Annotated[Allocatable[Float64[:]], Aliased] - -class buffer: - field: Allocatable[Float64[:, :]] -""", - module_name="persistent_allocatables", - ) - - scratch, values = module.variables - field = module.classes[0].fields[0] - - assert scratch.semantic_type.name == "Float64" - assert scratch.semantic_type.rank == 0 - assert scratch.semantic_type.storage is None - assert scratch.semantic_type.metadata["fortran_allocatable"] is True - - assert native_array_descriptor_kind(values.semantic_type) == "allocatable" - assert values.semantic_type.rank == 1 - assert values.semantic_type.metadata["aliased"] is True - assert native_array_descriptor_kind(field.semantic_type) == "allocatable" - assert field.semantic_type.rank == 2 - assert "aliased" not in field.semantic_type.metadata - - -def test_scalar_allocatable_calls_use_nullable_values_and_explicit_descriptor_projections(): - module = parse_pyi_text( - """ -from prik.contracts import Allocatable, Arg, Float64, Return, Returns, native_call - -@native_call([Allocatable(Arg(0)), Allocatable(Return("created", 1))]) -def update( - value: Float64 | None, -) -> tuple[ - Returns["value", Float64] | None, - Returns["created", Float64] | None, -]: ... -""", - module_name="scalar_allocatable_calls", - ) - - function = module.functions[0] - value, created = function.arguments - assert value.semantic_type.metadata["fortran_allocatable"] is True - assert value.metadata[PROJECTED_OUTPUT_METADATA] is True - assert created.semantic_type.metadata["fortran_allocatable"] is True - assert created.metadata[PROJECTED_OUTPUT_METADATA] is True - assert [mapping.value_kind for mapping in function.projection] == ["allocatable", "allocatable"] - - @pytest.mark.parametrize( ("source", "message"), [ @@ -106,14 +49,3 @@ def test_plain_nullable_scalar_is_not_an_allocatable_descriptor(): semantic_type = module.variables[0].semantic_type assert semantic_type.name == "Float64 | None" assert "fortran_allocatable" not in semantic_type.metadata - - -def test_legacy_annotated_allocatable_array_spelling_is_rejected(): - with pytest.raises(ValueError, match="use Allocatable"): - parse_pyi_text( - """ -from prik.contracts import Allocatable, Annotated, Float64 -values: Annotated[Float64[:], Allocatable] -""", - module_name="legacy_allocatable_array", - ) diff --git a/tests/fortran/arrays/codegen/test_array_buffer_lowering.py b/tests/fortran/arrays/codegen/test_array_buffer_lowering.py index 8d639b8d1..82443e477 100644 --- a/tests/fortran/arrays/codegen/test_array_buffer_lowering.py +++ b/tests/fortran/arrays/codegen/test_array_buffer_lowering.py @@ -1,95 +1,23 @@ -"""Direct-plan required dense rank-one primitive-array input lowering.""" +"""Array handoff plans are validated before lowering, and lower to legal Fortran.""" from __future__ import annotations import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.ownership import ( - CodegenAction, - DestructionPolicy, - NativeBarrierAction, - ObjectKind, - OwnershipOwner, - PythonBarrierAction, - StorageMode, - TransferMode, -) from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - ArgumentHandoffMode, - ArrayEntrypointABI, - BridgeDataAction, - EntrypointPassingConvention, -) +from prik.policy.models import BridgeDataAction from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import ArrayHandoffPlan, WrapperPlanner -from prik.planning.models import DatatypeFamily +from prik.planning import WrapperPlanner -def _array_module(): +def _array_plan(): module = parse_pyi_text( "def sum_values(values: Float64[:]) -> Float64: ...\n", module_name="array_buffers", ) complete_semantic_policies(module) - return module - - -def _array_plan(): - return WrapperPlanner().build(_array_module()) - - -def test_required_array_buffer_has_one_printable_editable_handoff_plan(): - function = _array_plan().namespaces[0].functions[0] - argument = function.arguments[0] - - assert argument.object_kind is ObjectKind.NUMPY_ARRAY - assert argument.ownership_owner is OwnershipOwner.CALLER - assert argument.transfer_mode is TransferMode.IN_PLACE - assert argument.destruction_policy is DestructionPolicy.CALLER - assert argument.storage_mode is StorageMode.STACK - assert argument.boundary_storage_mode is StorageMode.STACK - assert argument.datatype_family is DatatypeFamily.REAL - assert argument.binding.python_action is PythonBarrierAction.ARRAY_STORAGE - assert argument.binding.codegen_action is CodegenAction.IN_PLACE_ARGUMENT - assert argument.bridge.native_action is NativeBarrierAction.PASS_ARRAY_BUFFER - assert argument.entrypoint.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER - assert argument.bridge.data_action is BridgeDataAction.ASSOCIATE_VIEW - - assert isinstance(argument.array, ArrayHandoffPlan) - assert argument.array is argument.projected_call_slot.array - assert argument.projected_call_slot.object_kind is ObjectKind.NUMPY_ARRAY - assert argument.array.rank == 1 - assert argument.array.shape == (":",) - assert argument.array.axes == ("dense",) - assert argument.array.contiguous is True - assert argument.array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR - assert argument.entrypoint.passing is EntrypointPassingConvention.C_DESCRIPTOR_POINTER - assert argument.entrypoint.pass_array_metadata is False - assert argument.array.flatten_python_storage is False - assert argument.array.flat_axis is None - assert argument.array.data_role == argument.entrypoint.handoff_role - assert argument.array.extent_roles == (f"{argument.owner_path}:extent:0",) - assert argument.array.lower_bound_roles == () - assert argument.array.upper_bound_roles == () - assert argument.array.stride_roles == () - - -def test_required_array_buffer_dispatches_through_named_binding_and_bridge_methods(): - artifacts = WrapperGenerator().generate(_array_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "double bind_c_sum_values(CFI_cdesc_t * values);" in c_source - assert "CFI_establish((CFI_cdesc_t *)&bound_values_section" in c_source - assert "prik_describe_numpy_array((CFI_cdesc_t *)&bound_values_parent" not in c_source - assert "call->result = bind_c_sum_values(call->descriptor_0);" in c_source - assert "bound_values_bind_fixed" not in c_source - - assert "real(c_double), dimension(:), contiguous :: values" in bridge_source - assert "call c_f_pointer(bound_values" not in bridge_source - assert "result = native_sum_values(values)" in bridge_source + return WrapperPlanner().build(module) @pytest.mark.parametrize( @@ -115,3 +43,27 @@ def test_array_handoff_plan_edits_fail_before_backend_lowering(edit: str, diagno with pytest.raises(ValueError, match=diagnostic): WrapperGenerator().generate(plan) + + +def test_high_rank_bool_array_bridge_stays_inside_the_fortran_line_limit(): + """Free-form Fortran caps a line at 132 columns, whatever the rank. + + A rank-15 logical descriptor dummy is the longest ordinary array + declaration PRIK emits and is where continuation would first be missed. + Not every compiler rejects a longer line, so this is checked on the text. + """ + shape = ", ".join(":" for _ in range(15)) + module = parse_pyi_text( + f""" +from prik.contracts import Bool + +def normalize(values: Bool[{shape}]) -> None: ... +""", + module_name="high_rank_logical_array", + ) + complete_semantic_policies(module) + artifacts = WrapperGenerator().generate(WrapperPlanner().build(module)) + bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") + + assert "dimension(:, :, :, :, :, :, :, :, :, :, :, :, :, :, :), contiguous :: values" in bridge_source + assert max(map(len, bridge_source.splitlines())) <= 132 diff --git a/tests/fortran/arrays/codegen/test_array_output_identity.py b/tests/fortran/arrays/codegen/test_array_output_identity.py deleted file mode 100644 index 5d149e181..000000000 --- a/tests/fortran/arrays/codegen/test_array_output_identity.py +++ /dev/null @@ -1,112 +0,0 @@ -"""Projected ordinary outputs preserve their original Python array identity.""" - -from __future__ import annotations - - -from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.ownership import CodegenAction, ObjectKind, OwnershipOwner, TransferMode -from prik.policy.completion import complete_semantic_policies -from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import WrapperPlanner -from prik.planning.models import WritebackPhase - - -def _output_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Float64, Int32, Returns - -def fill(n: Int32, values: Float64[n]) -> Returns["values", Float64[n]]: ... -def fill_two( - n: Int32, - left: Float64[n], - right: Float64[n], -) -> tuple[Returns["left", Float64[n]], Returns["right", Float64[n]]]: ... -""", - module_name="array_output_identity", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def _logical_output_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Bool, Int32 - -def invert_flags(n: Int32, values: Bool[n], out: Bool[n]) -> None: ... -""", - module_name="logical_arrays", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def _high_rank_logical_output_plan(): - shape = ", ".join(":" for _ in range(15)) - module = parse_pyi_text( - f""" -from prik.contracts import Bool - -def normalize(values: Bool[{shape}]) -> None: ... -""", - module_name="high_rank_logical_array", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def test_projected_array_identity_uses_one_completed_in_place_copy_out_action(): - function = _output_plan().namespaces[0].functions[0] - argument = function.arguments[-1] - action = function.writeback_actions[0] - - assert argument.object_kind is ObjectKind.NUMPY_ARRAY - assert argument.ownership_owner is OwnershipOwner.CALLER - assert argument.transfer_mode is TransferMode.IN_PLACE - assert argument.binding.codegen_action is CodegenAction.IN_PLACE_ARGUMENT - assert action.object_kind is ObjectKind.NUMPY_ARRAY - assert action.phase is WritebackPhase.COPY_OUT - assert action.binding is not None - assert action.binding.codegen_action is CodegenAction.IN_PLACE_ARGUMENT - - -def test_projected_array_lowering_increfs_original_objects_and_reuses_tuple_aggregation(): - artifacts = WrapperGenerator().generate(_output_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - - assert "PyObject * result_obj = bound_values_obj;" in c_source - assert "Py_INCREF(result_obj);" in c_source - assert "PyObject * result_0_obj = bound_left_obj;" in c_source - assert "PyObject * result_1_obj = bound_right_obj;" in c_source - assert "PyTuple_New(2)" in c_source - assert "PyTuple_SET_ITEM(result_obj, 0, result_0_obj)" in c_source - assert "PyTuple_SET_ITEM(result_obj, 1, result_1_obj)" in c_source - - -def test_mutable_bool_array_writeback_needs_no_normalization(): - """A Boolean array is written back like any other element type. - - Its elements already hold the zero or one a C `_Bool` is defined to hold, - because the compiler profiles request the option that guarantees it, so the - callee leaves nothing behind that has to be reduced afterwards. - """ - artifacts = WrapperGenerator().generate(_logical_output_plan()) - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "call native_invert_flags(n, values, out)" in bridge_source - assert "_logical_bytes" not in bridge_source - assert "iand(" not in bridge_source - - -def test_high_rank_bool_array_bridge_stays_inside_the_fortran_line_limit(): - """Free-form Fortran caps a line at 132 columns, whatever the rank. - - A rank-15 descriptor dummy is the longest ordinary array declaration PRIK - emits and is where continuation would first be missed. - """ - artifacts = WrapperGenerator().generate(_high_rank_logical_output_plan()) - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "dimension(:, :, :, :, :, :, :, :, :, :, :, :, :, :, :), contiguous :: values" in bridge_source - assert max(map(len, bridge_source.splitlines())) <= 132 diff --git a/tests/fortran/arrays/codegen/test_array_result_lowering.py b/tests/fortran/arrays/codegen/test_array_result_lowering.py index 5b10e3541..81573f2fa 100644 --- a/tests/fortran/arrays/codegen/test_array_result_lowering.py +++ b/tests/fortran/arrays/codegen/test_array_result_lowering.py @@ -1,13 +1,11 @@ -"""Fixed-shape direct and hidden ordinary array result lowering.""" +"""Direct and hidden ordinary array result ownership and plan validation.""" from __future__ import annotations import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.ownership import CodegenAction, NativeBarrierAction, ObjectKind, OwnershipOwner, TransferMode from prik.policy.completion import complete_semantic_policies -from prik.policy.models import BridgeDataAction, ORDINARY_ARRAY_RESULT_COPY_REASON from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner @@ -28,57 +26,18 @@ def hidden() -> Float64[3]: ... return WrapperPlanner().build(module) -def _array_property_result_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Float64 - -def vector(values: Float64[:]) -> Float64[values.size]: ... -def flattened(values: Float64[:, :]) -> Float64[values.size]: ... -def columns(values: Float64[:, :]) -> Float64[values.shape[1]]: ... -""", - module_name="size_intrinsic_results", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def test_array_results_record_producer_shape_copy_ownership_and_shared_hidden_slot(): - direct_function, hidden_function = _result_plan().namespaces[0].functions - direct = direct_function.results[0] - hidden = hidden_function.results[0] - - for result in (direct, hidden): - assert result.object_kind is ObjectKind.NUMPY_ARRAY - assert result.ownership_owner is OwnershipOwner.PYTHON - assert result.transfer_mode is TransferMode.COPY_RETURN - assert result.array is not None - assert result.array.rank == 1 - assert result.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - assert result.bridge.copy_reason == ORDINARY_ARRAY_RESULT_COPY_REASON - assert direct.source_kind == "direct_return" - assert direct.binding.codegen_action is CodegenAction.COPY_OUT - assert direct.bridge.native_action is NativeBarrierAction.NONE - assert direct.projected_call_slot is None - assert hidden.source_kind == "hidden_output" - assert hidden.binding.codegen_action is CodegenAction.COPY_OUT - assert hidden.bridge.native_action is NativeBarrierAction.PASS_ARRAY_BUFFER - assert hidden.array is hidden.projected_call_slot.array - assert hidden.projected_call_slot.object_kind is ObjectKind.NUMPY_ARRAY - - def test_array_result_lowering_transfers_bridge_copy_to_capsule_owned_numpy_storage(): + """The bridge's copy becomes the array's storage, released once by its capsule. + + `PyArray_SetBaseObject` steals the capsule even when it fails, so that + failure path must neither release the capsule nor free the storage again. + A double release there is not observable from a successful call. + """ artifacts = WrapperGenerator().generate(_result_plan()) c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - assert "void * bind_c_direct(int32_t n);" in c_source - assert ( - "PyArray_New(&PyArray_Type, 1, result_obj_dims, NPY_FLOAT64, NULL, result, 0, " - "NPY_ARRAY_C_CONTIGUOUS | NPY_ARRAY_WRITEABLE, NULL)" in c_source - ) assert "PyCapsule_New(result, NULL, prik_release_owned_memory)" in c_source - assert "PyArray_SetBaseObject((PyArrayObject *)result_obj, result_obj_base)" in c_source + assert "PyCapsule_New(out, NULL, prik_release_owned_memory)" in c_source assert "memcpy(PyArray_DATA((PyArrayObject *)result_obj), result" not in c_source base_failure = c_source.split( "if (PyArray_SetBaseObject((PyArrayObject *)result_obj, result_obj_base) < 0)", @@ -86,37 +45,6 @@ def test_array_result_lowering_transfers_bridge_copy_to_capsule_owned_numpy_stor )[1].split("}", maxsplit=1)[0] assert "Py_DECREF(result_obj_base)" not in base_failure assert "free(result)" not in base_failure - assert "void bind_c_hidden(void ** out);" in c_source - assert "PyCapsule_New(out, NULL, prik_release_owned_memory)" in c_source - assert "real(c_double), dimension(n) :: result_value" in bridge_source - assert "result = c_malloc(" in bridge_source - assert "size(result_value," in bridge_source - assert "storage_size(result_value," in bridge_source - assert "result_copy = reshape(result_value, [size(result_value)])" in bridge_source - assert "real(c_double), dimension(3) :: out_value" in bridge_source - assert "call native_hidden(out_value)" in bridge_source - - -def test_array_property_results_reuse_input_array_extent_roles_in_both_backends(): - plan = _array_property_result_plan() - vector, flattened, columns = plan.namespaces[0].functions - - assert vector.results[0].array.shape == ("__prik_extent_values_0",) - assert vector.results[0].array.extent_reference_tokens == (("__prik_extent_values_0",),) - assert vector.results[0].array.extent_reference_roles == (("size_intrinsic_results.vector.values:extent:0",),) - assert flattened.results[0].array.shape == ("__prik_extent_values_0 * __prik_extent_values_1",) - assert columns.results[0].array.shape == ("__prik_extent_values_1",) - - artifacts = WrapperGenerator().generate(plan) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "npy_intp result_obj_dims[] = {bound_values_extent_0};" in c_source - assert "npy_intp result_obj_dims[] = {bound_values_extent_0 * bound_values_extent_1};" in c_source - assert bridge_source.count("real(c_double), allocatable, dimension(:) :: result_value") == 3 - assert "allocate(result_value(size(values, 1)))" in bridge_source - assert "allocate(result_value(size(values, 1) * size(values, 2)))" in bridge_source - assert "allocate(result_value(size(values, 2)))" in bridge_source @pytest.mark.parametrize( diff --git a/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py b/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py index 6dadfc820..607a5d49c 100644 --- a/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py +++ b/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py @@ -7,9 +7,7 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from prik.policy.completion import complete_semantic_policies from prik.policy.models import ( - TransformationAction, TransformationLayer, - WritebackPhase, ) from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner @@ -61,23 +59,6 @@ def transform(values: Annotated[Float64[2, 3], ORDER_C, COPY_F]) -> None: ... return WrapperPlanner().build(module) -def _copy_f_lifecycle_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Annotated, COPY_F, Float64, ORDER_C, Returns - -def native_input(values: Annotated[Float64[2, 3], ORDER_C, COPY_F]) -> None: ... - -def projected( - values: Annotated[Float64[2, 3], ORDER_C, COPY_F] -) -> Returns["values", Float64[2, 3]]: ... -""", - module_name="copy_f_lifecycle", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - def _copy_f_status_plan(): module = parse_pyi_text( """ @@ -109,100 +90,29 @@ def late_extent(values: Float64[n], n: Annotated[Int32, Immutable] | None = ...) return WrapperPlanner().build(module) -def test_dense_array_plan_records_extent_dependencies_flat_storage_and_order(): +@pytest.mark.parametrize( + ("name", "shape", "order", "flat_axis"), + [ + pytest.param("flat_rank2_runtime", (":", ":"), "ORDER_F", 1, id="fortran-flat-last-axis-open-prefix"), + pytest.param("flat_rank2_fixed", ("3", ":"), "ORDER_F", 1, id="fortran-flat-last-axis-fixed-prefix"), + pytest.param("c_flat_rank2_runtime", (":", ":"), "ORDER_C", 0, id="c-flat-first-axis-open-suffix"), + pytest.param("c_flat_rank2_fixed", (":", "3"), "ORDER_C", 0, id="c-flat-first-axis-fixed-suffix"), + ], +) +def test_dense_array_plan_places_flat_storage_by_order(name, shape, order, flat_axis): + """The flat axis is the slowest-varying one for the declared order. + + Runtime tests cover rank-one and argument-sized Fortran-order flat storage; + the C-order placements and fixed-extent neighbours are only decided here. + """ functions = {function.binding.python_name: function for function in _dense_plan().namespaces[0].functions} - dense_f = functions["dense_f"].arguments[-1].array - dense_c = functions["dense_c"].arguments[-1].array - flat = functions["flat"].arguments[-1].array - flat_rank2_runtime = functions["flat_rank2_runtime"].arguments[-1].array - flat_rank2_fixed = functions["flat_rank2_fixed"].arguments[-1].array - c_flat_rank2_runtime = functions["c_flat_rank2_runtime"].arguments[-1].array - c_flat_rank2_fixed = functions["c_flat_rank2_fixed"].arguments[-1].array - bounded_flat = functions["bounded_flat"].arguments[-1].array - - assert dense_f is not None - assert dense_f.rank == 2 - assert dense_f.shape == ("rows", "cols") - assert dense_f.order == "ORDER_F" - assert dense_f.extent_reference_roles == ( - ("dense_array_shapes.dense_f.rows:value",), - ("dense_array_shapes.dense_f.cols:value",), - ) - assert dense_c is not None - assert dense_c.order == "ORDER_C" - assert flat is not None - assert flat.rank == 1 - assert flat.shape == (":",) - assert flat.category == "assumed_size" - assert flat.flatten_python_storage is True - assert flat.flat_axis == 0 - assert flat_rank2_runtime is not None - assert flat_rank2_runtime.rank == 2 - assert flat_rank2_runtime.shape == (":", ":") - assert flat_rank2_runtime.order == "ORDER_F" - assert flat_rank2_runtime.category == "assumed_size" - assert flat_rank2_runtime.flatten_python_storage is True - assert flat_rank2_runtime.flat_axis == 1 - assert flat_rank2_fixed is not None - assert flat_rank2_fixed.rank == 2 - assert flat_rank2_fixed.shape == ("3", ":") - assert flat_rank2_fixed.order == "ORDER_F" - assert flat_rank2_fixed.flatten_python_storage is True - assert flat_rank2_fixed.flat_axis == 1 - assert c_flat_rank2_runtime is not None - assert c_flat_rank2_runtime.rank == 2 - assert c_flat_rank2_runtime.shape == (":", ":") - assert c_flat_rank2_runtime.order == "ORDER_C" - assert c_flat_rank2_runtime.category == "assumed_size" - assert c_flat_rank2_runtime.flatten_python_storage is True - assert c_flat_rank2_runtime.flat_axis == 0 - assert c_flat_rank2_fixed is not None - assert c_flat_rank2_fixed.rank == 2 - assert c_flat_rank2_fixed.shape == (":", "3") - assert c_flat_rank2_fixed.order == "ORDER_C" - assert c_flat_rank2_fixed.flatten_python_storage is True - assert c_flat_rank2_fixed.flat_axis == 0 - assert bounded_flat is not None - assert bounded_flat.shape == ("ldb", ":") - assert bounded_flat.flatten_python_storage is True - assert bounded_flat.flat_axis == 1 - assert bounded_flat.extent_reference_roles == (("dense_array_shapes.bounded_flat.ldb:value",), ()) - - -def test_dense_array_lowering_uses_planned_shape_checks_and_bridge_orientation(): - artifacts = WrapperGenerator().generate(_dense_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") + array = functions[name].arguments[-1].array - assert "bound_values_bind_fixed[0] = (long long)(bound_rows);" in c_source - assert "bound_values_bind_fixed[1] = (long long)(bound_cols);" in c_source - assert ( - "prik_bind_array_or_handle(bound_values_obj, NPY_FLOAT64, 0, sizeof(double), 0, 2, 2, 2, PRIK_ARRAY_LAYOUT_F_CONTIGUOUS" - in c_source - ) - assert ( - "prik_bind_array_or_handle(bound_values_obj, NPY_FLOAT64, 0, sizeof(double), 0, 2, 2, 2, PRIK_ARRAY_LAYOUT_C_CONTIGUOUS" - in c_source - ) - assert "bound_values_bind_fixed[0] = -1;" in c_source - assert ( - "prik_bind_array_or_handle(bound_values_obj, NPY_FLOAT64, 0, sizeof(double), 0, 1, 1, 15, " - 'PRIK_ARRAY_LAYOUT_ANY_CONTIGUOUS, 1, 1, "numpy.float64", "values", 0, 0, ' - "bound_values_bind_fixed, &bound_values, bound_values_bind_extents, &bound_values_native_backend" - ) in c_source - assert ( - "prik_bind_array_or_handle(bound_values_obj, NPY_FLOAT64, 0, sizeof(double), 0, 2, 2, 15, " - 'PRIK_ARRAY_LAYOUT_F_CONTIGUOUS, 1, 1, "numpy.float64", "values", 1, 1, ' - "bound_values_bind_fixed, &bound_values, bound_values_bind_extents, &bound_values_native_backend" - ) in c_source - assert "call c_f_pointer(bound_values, values, [values_extent_0, values_extent_1])" in bridge_source - assert "call c_f_pointer(bound_values, values, [values_extent_1, values_extent_0])" in bridge_source - assert "subroutine bind_c_flat(n, bound_values, values_extent_0)" in bridge_source - assert "subroutine bind_c_flat_rank2_runtime(" in bridge_source - assert "subroutine bind_c_c_flat_rank2_runtime(" in bridge_source - assert "external :: flat_rank2_runtime" in bridge_source - assert "external :: c_flat_rank2_fixed" in bridge_source - assert "real(c_double), pointer, contiguous, dimension(:, :) :: values" in bridge_source + assert array.shape == shape + assert array.order == order + assert array.category == "assumed_size" + assert array.flatten_python_storage is True + assert array.flat_axis == flat_axis def test_external_interface_declares_late_extent_before_dependent_array(): @@ -245,44 +155,6 @@ def test_unavailable_dense_extent_role_fails_before_backend_lowering(): WrapperGenerator().generate(plan) -def test_copy_f_is_one_binding_owned_transformation_lifecycle(): - argument = _copy_f_plan().namespaces[0].functions[0].arguments[0] - - assert argument.array is not None - assert argument.array.order == "ORDER_C" - assert argument.array.native_order == "ORDER_F" - assert tuple(item.phase for item in argument.transformations) == ( - WritebackPhase.COPY_IN, - WritebackPhase.COPY_OUT, - WritebackPhase.CLEANUP, - ) - assert {item.layer for item in argument.transformations} == {TransformationLayer.BINDING} - assert tuple(item.action for item in argument.transformations) == ( - TransformationAction.COPY_ARRAY_REPRESENTATION, - TransformationAction.COPY_ARRAY_REPRESENTATION, - TransformationAction.RELEASE_TEMPORARY, - ) - - -def test_copy_f_lowering_keeps_numpy_copy_in_and_copy_out_out_of_the_bridge(): - artifacts = WrapperGenerator().generate(_copy_f_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "PRIK_ARRAY_LAYOUT_C_CONTIGUOUS, 1, 1" in c_source - assert ( - "bound_values_representation = PyArray_NewCopy((PyArrayObject *)bound_values_obj, NPY_FORTRANORDER)" in c_source - ) - assert "bound_values = PyArray_DATA((PyArrayObject *)bound_values_representation)" in c_source - assert ( - "PyArray_CopyInto((PyArrayObject *)bound_values_obj, (PyArrayObject *)bound_values_representation) < 0" - in c_source - ) - assert "Py_CLEAR(bound_values_representation)" in c_source - assert "call c_f_pointer(bound_values, values, [values_extent_0, values_extent_1])" in bridge_source - assert "COPY_F" not in bridge_source - - def test_copy_f_status_cleanup_clears_the_released_temporary_before_error_cleanup(): artifacts = WrapperGenerator().generate(_copy_f_status_plan()) c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") @@ -295,38 +167,6 @@ def test_copy_f_status_cleanup_clears_the_released_temporary_before_error_cleanu assert copyback < success_release < status_check < error_release -def test_copy_f_native_input_and_projected_identity_share_the_same_lifecycle_algorithm(): - functions = { - function.binding.python_name: function for function in _copy_f_lifecycle_plan().namespaces[0].functions - } - native_input = functions["native_input"].arguments[0] - projected = functions["projected"].arguments[0] - - assert tuple(item.phase for item in native_input.transformations) == ( - WritebackPhase.COPY_IN, - WritebackPhase.COPY_OUT, - WritebackPhase.CLEANUP, - ) - assert tuple(item.phase for item in projected.transformations) == ( - WritebackPhase.COPY_IN, - WritebackPhase.COPY_OUT, - WritebackPhase.CLEANUP, - ) - assert projected.projects_result is True - - artifacts = WrapperGenerator().generate(_copy_f_lifecycle_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - native_input_body, projected_body = c_source.split("static PyObject * wrap_projected", maxsplit=1) - assert "PyArray_NewCopy((PyArrayObject *)bound_values_obj, NPY_FORTRANORDER)" in native_input_body - assert "PyArray_CopyInto((PyArrayObject *)bound_values_obj" in native_input_body - assert "PyArray_CopyInto((PyArrayObject *)bound_values_obj" in projected_body - assert "PyObject * result_obj = bound_values_obj" in projected_body - assert "Py_INCREF(result_obj)" in projected_body - assert "COPY_F" not in bridge_source - - def test_copy_f_layer_edit_fails_central_validation(): plan = _copy_f_plan() argument = plan.namespaces[0].functions[0].arguments[0] diff --git a/tests/fortran/arrays/codegen/test_specialized_array_roles.py b/tests/fortran/arrays/codegen/test_specialized_array_roles.py deleted file mode 100644 index ab45e7cbe..000000000 --- a/tests/fortran/arrays/codegen/test_specialized_array_roles.py +++ /dev/null @@ -1,165 +0,0 @@ -"""Cross-feature optional, assumed-rank, and character-array lowering.""" - -from __future__ import annotations - - -from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - ArrayEntrypointABI, - EntrypointOptionalityAction, - EntrypointPassingConvention, - NativeArraySourceKind, - OptionalMode, -) -from prik.codegen import CBindingGenerator -from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import WrapperPlanner - - -def _later_array_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Float64, String - -def optional(values: Float64[:] = ...) -> None: ... -def optional_any_rank(values: Float64[...] = ...) -> None: ... -def any_rank(values: Float64[...]) -> Float64: ... -def labels(values: String[8][:]) -> None: ... -def labels_any_width(values: String[...][:]) -> None: ... -""", - module_name="later_array_buffers", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def _character_array_result_plan(): - module = parse_pyi_text( - """ -def direct_labels() -> String[5][3]: ... - -@native_call([Return("labels", 0)]) -def hidden_labels() -> String[4][2]: ... -""", - module_name="character_array_results", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def test_optional_assumed_rank_and_character_arrays_have_explicit_distinct_roles(): - functions = {function.binding.python_name: function for function in _later_array_plan().namespaces[0].functions} - optional = functions["optional"].arguments[0] - optional_assumed_argument = functions["optional_any_rank"].arguments[0] - optional_assumed = optional_assumed_argument.array - assumed_argument = functions["any_rank"].arguments[0] - assumed = assumed_argument.array - character_argument = functions["labels"].arguments[0] - character = character_argument.array - assumed_width_argument = functions["labels_any_width"].arguments[0] - handle_sources = ( - NativeArraySourceKind.NDARRAY, - NativeArraySourceKind.ALLOCATABLE_HANDLE, - NativeArraySourceKind.POINTER_HANDLE, - ) - - assert optional.binding.optional_mode is OptionalMode.NULLABLE_VALUE - assert optional.entrypoint.optional_mode is OptionalMode.NULLABLE_VALUE - assert optional.native_array_actual is not None - assert optional.native_array_actual.accepted_sources == handle_sources - assert optional_assumed_argument.binding.optional_mode is OptionalMode.NULLABLE_VALUE - assert optional_assumed_argument.entrypoint.optional_mode is OptionalMode.NULLABLE_VALUE - assert ( - optional_assumed_argument.entrypoint.optionality - is EntrypointOptionalityAction.EXPLICIT_PRESENCE_WITH_PLACEHOLDER_DESCRIPTOR - ) - assert optional_assumed_argument.entrypoint.presence_role is not None - assert optional_assumed is not None - assert optional_assumed.rank is None - assert optional_assumed.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR - assert optional_assumed_argument.entrypoint.passing is EntrypointPassingConvention.C_DESCRIPTOR_POINTER - assert assumed is not None - assert assumed.rank is None - assert assumed.contiguous is False - assert assumed.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR - assert assumed.signed_strides is True - assert assumed_argument.entrypoint.passing is EntrypointPassingConvention.C_DESCRIPTOR_POINTER - assert assumed_argument.entrypoint.pass_array_metadata is False - assert assumed.runtime_rank_role == "later_array_buffers.any_rank.values:rank" - assert len(assumed.extent_roles) == 15 - assert assumed_argument.native_array_actual is not None - assert assumed_argument.native_array_actual.rank is None - assert assumed_argument.native_array_actual.accepted_sources == handle_sources - assert character is not None - assert character.rank == 1 - assert character.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS - assert character.itemsize == 8 - assert character.itemsize_role == "later_array_buffers.labels.values:itemsize" - assert character_argument.native_array_actual is not None - assert character_argument.native_array_actual.accepted_sources == handle_sources - assert assumed_width_argument.native_array_actual is not None - assert assumed_width_argument.native_array_actual.dtype == "S" - assert assumed_width_argument.native_array_actual.accepted_sources == handle_sources - - -def test_optional_assumed_rank_and_character_lowering_follow_named_plan_fields(): - artifacts = WrapperGenerator().generate(_later_array_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "PyObject * bound_values_obj = Py_None;" in c_source - assert "if (bound_values_obj != Py_None)" in c_source - assert ( - "prik_array_validate_ndarray((PyArrayObject *)bound_values_obj, NPY_FLOAT64, 1, 15, " - "PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F" - ) in c_source - assert ( - "prik_native_array_backend_for_actual(bound_values_capsule, 1, 15, " - 'CFI_type_double, sizeof(double), "float64", "values")' - ) in c_source - assert "NPY_FLOAT64, 1, 15, PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F" in c_source - assert "bound_values_rank = (int64_t)PyArray_NDIM" in c_source - assert "void bind_c_optional_any_rank(CFI_cdesc_t * values, void * values_present);" in c_source - assert "bound_values_present = bound_values_obj != Py_None ? (void *)bound_values_obj : NULL;" in c_source - assert ( - "CFI_establish((CFI_cdesc_t *)&bound_values_section, &bound_values_placeholder, " - "CFI_attribute_other, CFI_type_double, sizeof(double), 0, NULL)" - ) in c_source - # Runtime character width is part of the raw bridge ABI. The shared binder - # returns it for either a NumPy array or a native handle. - assert "bound_values_itemsize" in c_source - assert "&bound_values_itemsize, CFI_type_char" in c_source - assert "real(c_double), dimension(..) :: values" in bridge_source - assert "select case (values_rank)" not in bridge_source - optional_any_rank = bridge_source.split("subroutine bind_c_optional_any_rank", maxsplit=1)[1].split( - "end subroutine bind_c_optional_any_rank", maxsplit=1 - )[0] - assert "real(c_double), dimension(..) :: values" in optional_any_rank - assert "type(c_ptr), value :: bound_values_present" in optional_any_rank - assert "if (c_associated(bound_values_present)) then" in optional_any_rank - assert "call native_optional_any_rank(values=values)" in optional_any_rank - assert "call native_optional_any_rank()" in optional_any_rank - assert "prik_optional_values_transport" not in optional_any_rank - assert "character(kind=c_char, len=8), pointer, contiguous, dimension(:) :: values" in bridge_source - assert max(map(len, bridge_source.splitlines())) <= 132 - - -def test_native_array_descriptor_result_unpacks_planned_runtime_rank_and_itemsize_roles(): - functions = {function.binding.python_name: function for function in _later_array_plan().namespaces[0].functions} - generator = CBindingGenerator() - - rank_function = functions["any_rank"] - rank_argument = rank_function.arguments[0] - rank_names = generator._function_context(rank_function).arguments[rank_argument.owner_path] - rank_nodes = generator._native_array_actual_unpack_nodes(rank_argument, rank_names) - - itemsize_function = functions["labels"] - itemsize_argument = itemsize_function.arguments[0] - itemsize_names = generator._function_context(itemsize_function).arguments[itemsize_argument.owner_path] - itemsize_nodes = generator._native_array_actual_unpack_nodes(itemsize_argument, itemsize_names) - - assert any(node.expression.text == "bound_values_rank = bound_values_actual.rank" for node in rank_nodes) - assert any( - node.expression.text == "bound_values_itemsize = bound_values_actual.itemsize" for node in itemsize_nodes - ) diff --git a/tests/fortran/arrays/codegen/test_strided_array_lowering.py b/tests/fortran/arrays/codegen/test_strided_array_lowering.py index faad673d2..b5dc1e2c6 100644 --- a/tests/fortran/arrays/codegen/test_strided_array_lowering.py +++ b/tests/fortran/arrays/codegen/test_strided_array_lowering.py @@ -1,4 +1,4 @@ -"""Signed-stride ordinary array view lowering.""" +"""Descriptor array dummies: contiguity and plan validation.""" from __future__ import annotations @@ -6,118 +6,41 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ArrayEntrypointABI, EntrypointPassingConvention from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -def _strided_plan(rank: int = 2): - dimensions = ", ".join("::" for _ in range(rank)) +def _plan(annotation: str): module = parse_pyi_text( f""" from prik.contracts import Float64 -def strided(values: Float64[{dimensions}]) -> None: ... +def call(values: {annotation}) -> None: ... """, - module_name="strided_arrays", + module_name="descriptor_arrays", ) complete_semantic_policies(module) return WrapperPlanner().build(module) -def _contiguous_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Float64 - -def contiguous(values: Float64[:]) -> None: ... -""", - module_name="contiguous_arrays", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def test_strided_array_plan_selects_one_descriptor_without_parallel_stride_roles(): - argument = _strided_plan().namespaces[0].functions[0].arguments[0] - array = argument.array - - assert array is not None - assert array.rank == 2 - assert array.axes == ("strided", "strided") - assert array.contiguous is False - assert array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR - assert array.signed_strides is True - assert argument.entrypoint.passing is EntrypointPassingConvention.C_DESCRIPTOR_POINTER - assert argument.entrypoint.pass_array_metadata is False - assert array.lower_bound_roles == () - assert array.upper_bound_roles == () - assert array.stride_roles == () - - -def test_strided_array_lowering_hands_over_one_descriptor_from_either_source(): - """A strided dummy is reached by a descriptor, whoever supplied the array. +def test_contiguous_descriptor_dummy_preserves_contiguity_for_native_call(): + """A contiguous descriptor dummy must not make a compiler temporary. - This is the direct-entrypoint answer for an assumed-shape dummy: a bind(C) - procedure with no bridge receives a ``CFI_cdesc_t *``, and the extents and - signed strides travel inside it. So the generated C describes a NumPy array - into one and enters a handle's own, and the bridge dummy is the array - itself -- there is nothing left to reconstruct on the Fortran side. + A temporary still computes the right values, so runtime tests cannot see it. """ - artifacts = WrapperGenerator().generate(_strided_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - # A handle is entered through its own descriptor entry point. - assert ( - "prik_native_array_backend_for_actual(bound_values_capsule, 2, 2, " - 'CFI_type_double, sizeof(double), "float64", "values")' - ) in c_source - # A contiguous NumPy array needs only CFI_establish; genuinely strided - # storage retains the CFI_section construction. - assert "PyArray_IS_F_CONTIGUOUS((PyArrayObject *)bound_values_obj)" in c_source - assert "CFI_establish((CFI_cdesc_t *)&bound_values_section" in c_source - assert "prik_describe_numpy_array((CFI_cdesc_t *)&bound_values_parent" in c_source - assert "CFI_section(section, parent, lower, upper, step)" in c_source - # Signed strides are what this layout accepts now. - assert "NPY_FLOAT64, 2, 2, PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F, 0, 1" in c_source - # One descriptor crosses, not an address with extents beside it. - assert "double bind_c_strided(CFI_cdesc_t * values)" in c_source or ( - "void bind_c_strided(CFI_cdesc_t * values)" in c_source - ) - assert "bound_values_upper_bound_0" not in c_source - - assert "real(c_double), dimension(:, :) :: values" in bridge_source - assert "call native_strided(values)" in bridge_source - # Nothing is rebuilt from an address any more. - assert "call c_f_pointer(" not in bridge_source - assert "values_base" not in bridge_source - assert max(map(len, bridge_source.splitlines())) <= 132 - - -def test_contiguous_descriptor_dummy_preserves_contiguity_for_native_call(): - """A contiguous descriptor dummy must not make a compiler temporary.""" - artifacts = WrapperGenerator().generate(_contiguous_plan()) + artifacts = WrapperGenerator().generate(_plan("Float64[:]")) bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") assert "real(c_double), dimension(:), contiguous :: values" in bridge_source -def test_descriptor_array_stride_role_edit_fails_before_backend_lowering(): - plan = _strided_plan() - array = plan.namespaces[0].functions[0].arguments[0].array - assert array is not None - array.stride_roles = (f"{array.data_role}:stride:0",) - - with pytest.raises(ValueError, match="unexpected-array-descriptor-roles"): - WrapperGenerator().generate(plan) - - -def test_descriptor_array_lower_bound_role_edit_fails_before_backend_lowering(): - plan = _strided_plan() +@pytest.mark.parametrize("roles", ["stride_roles", "lower_bound_roles"]) +def test_descriptor_array_parallel_role_edit_fails_before_backend_lowering(roles: str): + """Strides and bounds travel inside the descriptor, never beside it.""" + plan = _plan("Float64[::, ::]") array = plan.namespaces[0].functions[0].arguments[0].array assert array is not None - array.lower_bound_roles = (f"{array.data_role}:lower-bound:0",) + setattr(array, roles, (f"{array.data_role}:{roles}:0",)) with pytest.raises(ValueError, match="unexpected-array-descriptor-roles"): WrapperGenerator().generate(plan) diff --git a/tests/fortran/arrays/end_to_end/fixtures/contracts/farray_contracts_f90/farray_contracts_f90.pyi b/tests/fortran/arrays/end_to_end/fixtures/contracts/farray_contracts_f90/farray_contracts_f90.pyi index 281a7d421..5dd998c10 100644 --- a/tests/fortran/arrays/end_to_end/fixtures/contracts/farray_contracts_f90/farray_contracts_f90.pyi +++ b/tests/fortran/arrays/end_to_end/fixtures/contracts/farray_contracts_f90/farray_contracts_f90.pyi @@ -29,95 +29,14 @@ def shift1( out: Float64[::] ) -> None: ... -def shift2( - values: Float64[::, ::], - out: Float64[::, ::] -) -> None: ... - -def shift3( - values: Float64[::, ::, ::], - out: Float64[::, ::, ::] -) -> None: ... - def shift4( values: Float64[::, ::, ::, ::], out: Float64[::, ::, ::, ::] ) -> None: ... -def shift5( - values: Float64[::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::] -) -> None: ... - -def shift6( - values: Float64[::, ::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::, ::] -) -> None: ... - -def shift7( - values: Float64[::, ::, ::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::, ::, ::] -) -> None: ... - -def shift8( - values: Float64[::, ::, ::, ::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::, ::, ::, ::] -) -> None: ... - -def shift9( - values: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::] -) -> None: ... - -def shift10( - values: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::] -) -> None: ... - -def shift11( - values: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::] -) -> None: ... - -def shift12( - values: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::] -) -> None: ... - -def shift13( - values: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::] -) -> None: ... - -def shift14( - values: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::], - out: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::] -) -> None: ... - def shift15( values: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::], out: Float64[::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::, ::] ) -> None: ... -__all__ = [ - "sum_assumed_size", - "scale_lower", - "sum_in", - "bump_inout", - "fill_out", - "shift1", - "shift2", - "shift3", - "shift4", - "shift5", - "shift6", - "shift7", - "shift8", - "shift9", - "shift10", - "shift11", - "shift12", - "shift13", - "shift14", - "shift15", -] +__all__ = ["sum_assumed_size", "scale_lower", "sum_in", "bump_inout", "fill_out", "shift1", "shift4", "shift15"] diff --git a/tests/fortran/arrays/end_to_end/fixtures/native/farray_contracts_f90.f90 b/tests/fortran/arrays/end_to_end/fixtures/native/farray_contracts_f90.f90 index 83021c61f..94da505da 100644 --- a/tests/fortran/arrays/end_to_end/fixtures/native/farray_contracts_f90.f90 +++ b/tests/fortran/arrays/end_to_end/fixtures/native/farray_contracts_f90.f90 @@ -44,20 +44,6 @@ subroutine shift1(values, out) out = values + 1.0_8 end subroutine shift1 - subroutine shift2(values, out) - real(8), intent(in) :: values(:, :) - real(8), intent(out) :: out(:, :) - - out = values + 2.0_8 - end subroutine shift2 - - subroutine shift3(values, out) - real(8), intent(in) :: values(:, :, :) - real(8), intent(out) :: out(:, :, :) - - out = values + 3.0_8 - end subroutine shift3 - subroutine shift4(values, out) real(8), intent(in) :: values(:, :, :, :) real(8), intent(out) :: out(:, :, :, :) @@ -65,76 +51,6 @@ subroutine shift4(values, out) out = values + 4.0_8 end subroutine shift4 - subroutine shift5(values, out) - real(8), intent(in) :: values(:, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :) - - out = values + 5.0_8 - end subroutine shift5 - - subroutine shift6(values, out) - real(8), intent(in) :: values(:, :, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :, :) - - out = values + 6.0_8 - end subroutine shift6 - - subroutine shift7(values, out) - real(8), intent(in) :: values(:, :, :, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :, :, :) - - out = values + 7.0_8 - end subroutine shift7 - - subroutine shift8(values, out) - real(8), intent(in) :: values(:, :, :, :, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :, :, :, :) - - out = values + 8.0_8 - end subroutine shift8 - - subroutine shift9(values, out) - real(8), intent(in) :: values(:, :, :, :, :, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :, :, :, :, :) - - out = values + 9.0_8 - end subroutine shift9 - - subroutine shift10(values, out) - real(8), intent(in) :: values(:, :, :, :, :, :, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :, :, :, :, :, :) - - out = values + 10.0_8 - end subroutine shift10 - - subroutine shift11(values, out) - real(8), intent(in) :: values(:, :, :, :, :, :, :, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :, :, :, :, :, :, :) - - out = values + 11.0_8 - end subroutine shift11 - - subroutine shift12(values, out) - real(8), intent(in) :: values(:, :, :, :, :, :, :, :, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :, :, :, :, :, :, :, :) - - out = values + 12.0_8 - end subroutine shift12 - - subroutine shift13(values, out) - real(8), intent(in) :: values(:, :, :, :, :, :, :, :, :, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :, :, :, :, :, :, :, :, :) - - out = values + 13.0_8 - end subroutine shift13 - - subroutine shift14(values, out) - real(8), intent(in) :: values(:, :, :, :, :, :, :, :, :, :, :, :, :, :) - real(8), intent(out) :: out(:, :, :, :, :, :, :, :, :, :, :, :, :, :) - - out = values + 14.0_8 - end subroutine shift14 - subroutine shift15(values, out) real(8), intent(in) :: values(:, :, :, :, :, :, :, :, :, :, :, :, :, :, :) real(8), intent(out) :: out(:, :, :, :, :, :, :, :, :, :, :, :, :, :, :) diff --git a/tests/fortran/arrays/end_to_end/fixtures/native/fassumed_rank_f90.f90 b/tests/fortran/arrays/end_to_end/fixtures/native/fassumed_rank_f90.f90 index 198b25975..4e5bb0b50 100644 --- a/tests/fortran/arrays/end_to_end/fixtures/native/fassumed_rank_f90.f90 +++ b/tests/fortran/arrays/end_to_end/fixtures/native/fassumed_rank_f90.f90 @@ -1,7 +1,7 @@ module fassumed_rank_f90 real(8), target :: pointer_values(2) = [3.0_8, 4.0_8] - private :: pointer_values + private :: pointer_values, flat_sum, flat_add contains function allocatable_handle() result(values) real(8), allocatable :: values(:) @@ -33,49 +33,49 @@ real(8) function rank_weighted_sum(values) result(total) select rank(values) rank(1) - total = real(1, 8) + sum(values) + total = real(1, 8) + flat_sum(size(values), values) rank(2) - total = real(2, 8) + sum(values) + total = real(2, 8) + flat_sum(size(values), values) rank(3) - total = real(3, 8) + sum(values) + total = real(3, 8) + flat_sum(size(values), values) rank(4) - total = real(4, 8) + sum(values) + total = real(4, 8) + flat_sum(size(values), values) rank(5) - total = real(5, 8) + sum(values) + total = real(5, 8) + flat_sum(size(values), values) rank(6) - total = real(6, 8) + sum(values) + total = real(6, 8) + flat_sum(size(values), values) rank(7) - total = real(7, 8) + sum(values) + total = real(7, 8) + flat_sum(size(values), values) rank(8) - total = real(8, 8) + sum(values) + total = real(8, 8) + flat_sum(size(values), values) rank(9) - total = real(9, 8) + sum(values) + total = real(9, 8) + flat_sum(size(values), values) rank(10) - total = real(10, 8) + sum(values) + total = real(10, 8) + flat_sum(size(values), values) rank(11) - total = real(11, 8) + sum(values) + total = real(11, 8) + flat_sum(size(values), values) rank(12) - total = real(12, 8) + sum(values) + total = real(12, 8) + flat_sum(size(values), values) rank(13) - total = real(13, 8) + sum(values) + total = real(13, 8) + flat_sum(size(values), values) rank(14) - total = real(14, 8) + sum(values) + total = real(14, 8) + flat_sum(size(values), values) rank(15) - total = real(15, 8) + sum(values) + total = real(15, 8) + flat_sum(size(values), values) rank default total = -99.0_8 @@ -88,49 +88,49 @@ subroutine bump_assumed_rank(values) select rank(values) rank(1) - values = values + real(1, 8) + call flat_add(size(values), values, real(1, 8)) rank(2) - values = values + real(2, 8) + call flat_add(size(values), values, real(2, 8)) rank(3) - values = values + real(3, 8) + call flat_add(size(values), values, real(3, 8)) rank(4) - values = values + real(4, 8) + call flat_add(size(values), values, real(4, 8)) rank(5) - values = values + real(5, 8) + call flat_add(size(values), values, real(5, 8)) rank(6) - values = values + real(6, 8) + call flat_add(size(values), values, real(6, 8)) rank(7) - values = values + real(7, 8) + call flat_add(size(values), values, real(7, 8)) rank(8) - values = values + real(8, 8) + call flat_add(size(values), values, real(8, 8)) rank(9) - values = values + real(9, 8) + call flat_add(size(values), values, real(9, 8)) rank(10) - values = values + real(10, 8) + call flat_add(size(values), values, real(10, 8)) rank(11) - values = values + real(11, 8) + call flat_add(size(values), values, real(11, 8)) rank(12) - values = values + real(12, 8) + call flat_add(size(values), values, real(12, 8)) rank(13) - values = values + real(13, 8) + call flat_add(size(values), values, real(13, 8)) rank(14) - values = values + real(14, 8) + call flat_add(size(values), values, real(14, 8)) rank(15) - values = values + real(15, 8) + call flat_add(size(values), values, real(15, 8)) rank default return @@ -145,49 +145,49 @@ integer function rank_pair_score(left, right) result(score) select rank(left) rank(1) - score = score + 100 + int(sum(left)) + score = score + 100 + int(size(left)) rank(2) - score = score + 200 + int(sum(left)) + score = score + 200 + int(size(left)) rank(3) - score = score + 300 + int(sum(left)) + score = score + 300 + int(size(left)) rank(4) - score = score + 400 + int(sum(left)) + score = score + 400 + int(size(left)) rank(5) - score = score + 500 + int(sum(left)) + score = score + 500 + int(size(left)) rank(6) - score = score + 600 + int(sum(left)) + score = score + 600 + int(size(left)) rank(7) - score = score + 700 + int(sum(left)) + score = score + 700 + int(size(left)) rank(8) - score = score + 800 + int(sum(left)) + score = score + 800 + int(size(left)) rank(9) - score = score + 900 + int(sum(left)) + score = score + 900 + int(size(left)) rank(10) - score = score + 1000 + int(sum(left)) + score = score + 1000 + int(size(left)) rank(11) - score = score + 1100 + int(sum(left)) + score = score + 1100 + int(size(left)) rank(12) - score = score + 1200 + int(sum(left)) + score = score + 1200 + int(size(left)) rank(13) - score = score + 1300 + int(sum(left)) + score = score + 1300 + int(size(left)) rank(14) - score = score + 1400 + int(sum(left)) + score = score + 1400 + int(size(left)) rank(15) - score = score + 1500 + int(sum(left)) + score = score + 1500 + int(size(left)) rank default score = score - 100000 @@ -196,52 +196,67 @@ integer function rank_pair_score(left, right) result(score) select rank(right) rank(1) - score = score + 1 + int(sum(right)) + score = score + 1 + int(size(right)) rank(2) - score = score + 2 + int(sum(right)) + score = score + 2 + int(size(right)) rank(3) - score = score + 3 + int(sum(right)) + score = score + 3 + int(size(right)) rank(4) - score = score + 4 + int(sum(right)) + score = score + 4 + int(size(right)) rank(5) - score = score + 5 + int(sum(right)) + score = score + 5 + int(size(right)) rank(6) - score = score + 6 + int(sum(right)) + score = score + 6 + int(size(right)) rank(7) - score = score + 7 + int(sum(right)) + score = score + 7 + int(size(right)) rank(8) - score = score + 8 + int(sum(right)) + score = score + 8 + int(size(right)) rank(9) - score = score + 9 + int(sum(right)) + score = score + 9 + int(size(right)) rank(10) - score = score + 10 + int(sum(right)) + score = score + 10 + int(size(right)) rank(11) - score = score + 11 + int(sum(right)) + score = score + 11 + int(size(right)) rank(12) - score = score + 12 + int(sum(right)) + score = score + 12 + int(size(right)) rank(13) - score = score + 13 + int(sum(right)) + score = score + 13 + int(size(right)) rank(14) - score = score + 14 + int(sum(right)) + score = score + 14 + int(size(right)) rank(15) - score = score + 15 + int(sum(right)) + score = score + 15 + int(size(right)) rank default score = score - 100000 end select end function rank_pair_score + + pure real(8) function flat_sum(n, values) result(total) + integer, intent(in) :: n + real(8), intent(in) :: values(n) + + total = sum(values) + end function flat_sum + + pure subroutine flat_add(n, values, delta) + integer, intent(in) :: n + real(8), intent(inout) :: values(n) + real(8), intent(in) :: delta + + values = values + delta + end subroutine flat_add end module fassumed_rank_f90 diff --git a/tests/fortran/arrays/end_to_end/fixtures/primitive_dtype_rank_matrix.py b/tests/fortran/arrays/end_to_end/fixtures/primitive_dtype_rank_matrix.py index e7cfb8262..ff12802ed 100644 --- a/tests/fortran/arrays/end_to_end/fixtures/primitive_dtype_rank_matrix.py +++ b/tests/fortran/arrays/end_to_end/fixtures/primitive_dtype_rank_matrix.py @@ -1,4 +1,4 @@ -"""Generate one Fortran module covering every supported primitive array matrix cell.""" +"""Generate one Fortran module crossing every primitive dtype with representative ranks.""" from __future__ import annotations @@ -58,11 +58,20 @@ class PrimitiveArrayCase: ) +# The element-type mapping does not depend on rank, and descriptor rank handling +# does not depend on the element type. Every dtype therefore meets a vector, a +# matrix, and the first rank with an interior axis. The highest ranks are +# exercised for float64 by the contract-validation, array-result, and +# assumed-rank tests; optimizing whole-array expressions at every high rank for +# every dtype dominated this build without adding a failure mode. +PRIMITIVE_ARRAY_RANKS = (1, 2, 3) + + def primitive_dtype_rank_source() -> str: - """Return deterministic source for nine dtypes at concrete ranks 1 through 15.""" + """Return deterministic source for nine dtypes at the representative ranks.""" procedures: list[str] = [] for case in PRIMITIVE_ARRAY_CASES: - for rank in range(1, 16): + for rank in PRIMITIVE_ARRAY_RANKS: dimensions = ", ".join(":" for _ in range(rank)) procedures.append( "\n".join( diff --git a/tests/fortran/arrays/end_to_end/test_array_contract_validation.py b/tests/fortran/arrays/end_to_end/test_array_contract_validation.py index 6e3e0274a..f89351dd6 100644 --- a/tests/fortran/arrays/end_to_end/test_array_contract_validation.py +++ b/tests/fortran/arrays/end_to_end/test_array_contract_validation.py @@ -13,7 +13,9 @@ FIXTURES = Path(__file__).parent / "fixtures" ARRAY_CONTRACTS_F90_SOURCE = FIXTURES / "native" / "farray_contracts_f90.f90" CONTRACT_FIXTURES = FIXTURES / "contracts" -_MAX_WRAPPER_TEST_RANK = 15 +# The lowest rank, the rank the empty and zero-stride cases use, and the +# highest supported rank; every rank in between is a result-rank case. +_SHIFT_RANKS = (1, 4, 15) pytestmark = pytest.mark.fortran_end_to_end @@ -102,7 +104,7 @@ def test_remaining_array_contracts_are_validated_before_fortran_calls( assert zero_stride_empty_out.flags.f_contiguous assert module.shift4(zero_stride_empty, zero_stride_empty_out) is None - for rank in range(1, _MAX_WRAPPER_TEST_RANK + 1): + for rank in _SHIFT_RANKS: shape = (2, *([1] * (rank - 1))) source = np.asfortranarray(np.arange(np.prod(shape), dtype=np.float64).reshape(shape, order="F")) out = np.empty(shape, dtype=np.float64, order="F") diff --git a/tests/fortran/arrays/end_to_end/test_array_wrapper_parity.py b/tests/fortran/arrays/end_to_end/test_array_wrapper_parity.py index a9a90b9d3..6fa149b5c 100644 --- a/tests/fortran/arrays/end_to_end/test_array_wrapper_parity.py +++ b/tests/fortran/arrays/end_to_end/test_array_wrapper_parity.py @@ -1,20 +1,15 @@ """Primitive-array source/contract parity and wrapper-plan runtime tests.""" from pathlib import Path -import shutil -import numpy as np import pytest from tests.fortran._support.wrapper_build import ( _assert_array_rejects_strided_views, _assert_fmath_array_examples, + _build_and_import, _build_source_or_generated_pyi_and_import, - _compile_native_object, - _import_from_build_dir, - _sole_native_module, ) -from prik import build_pyi_extension FIXTURES = Path(__file__).parent / "fixtures" CONTRACTS = FIXTURES / "contracts" @@ -43,67 +38,26 @@ def test_fortran_array_wrapper_pipeline_matches_fmath_results_with_contiguous_ar _assert_array_rejects_strided_views(module, "SQUARE_R4") -def test_f90_array_wrapper_distinguishes_contiguous_and_strided_contracts( - pyi_parity_build_mode: str, - tmp_path: Path, -): - module = _build_source_or_generated_pyi_and_import( +def test_f90_array_wrapper_distinguishes_contiguous_and_strided_contracts(tmp_path: Path): + """The source build tells contiguous and strided dummies of every kind apart. + + This is the largest array fixture, so it builds once, from source. Its + generated contract is compared with the reviewed fixture by + `tests/fortran/arrays/pipeline/test_generated_array_contracts.py`, and + replaying generated contracts is proven by every other parity test here. + """ + source_dir = tmp_path / "source_build" + source_dir.mkdir() + module = _build_and_import( ARRAY_F90_SOURCE, - tmp_path, + source_dir, { "bind_c_fmath_arrays_f90_wrapper.f90", "fmath_arrays_f90_wrapper.c", "fmath_arrays_f90_wrapper.h", }, - CONTRACTS / "fmath_arrays_f90", - pyi_parity_build_mode, ) _assert_fmath_array_examples(module, suffix="_CONTIGUOUS", strided=False) _assert_array_rejects_strided_views(module, "SQUARE_R4_CONTIGUOUS") _assert_fmath_array_examples(module, suffix="_STRIDED", strided=True) - - -def test_required_array_buffers_use_canonical_wrapper_plan(tmp_path: Path): - """Replay one existing dense rank-one routine through a reduced contract.""" - native_object = _compile_native_object(ARRAY_F90_SOURCE, tmp_path / "native") - contract_package = tmp_path / "required_array" - shutil.copytree(CONTRACTS / "fmath_arrays_f90", contract_package) - (contract_package / "__init__.pyi").write_text( - "from .fmath_arrays_f90 import square_r8_contiguous\n", - encoding="utf-8", - ) - result = build_pyi_extension( - contract_package / "__init__.pyi", - native_objects=[native_object], - native_include_dirs=[native_object.parent], - output_dir=tmp_path / "build", - ) - package = _import_from_build_dir(result.module_name, result.output_dir) - module = package if hasattr(package, "square_r8_contiguous") else _sole_native_module(package) - - values = np.array([2.0, 3.0, -4.0], dtype=np.float64) - output = np.zeros_like(values) - assert module.square_r8_contiguous(np.int32(values.size), values, output) == np.int32(values.size) - np.testing.assert_array_equal(output, values**2) - - empty = np.empty(0, dtype=np.float64) - assert module.square_r8_contiguous(np.int32(0), empty, empty.copy()) == np.int32(0) - - valid = np.arange(4, dtype=np.float64) - output = np.zeros_like(valid) - invalid_cases = ( - np.arange(4, dtype=np.float32), - valid.reshape(2, 2), - np.arange(8, dtype=np.float64)[::2], - np.arange(4, dtype=">f8"), - np.ndarray(4, dtype=np.float64, buffer=bytearray(33), offset=1), - ) - for invalid in invalid_cases: - with pytest.raises((TypeError, ValueError)): - module.square_r8_contiguous(np.int32(4), invalid, output) - - read_only = valid.copy() - read_only.flags.writeable = False - with pytest.raises(TypeError, match="writeable"): - module.square_r8_contiguous(np.int32(4), read_only, output) diff --git a/tests/fortran/arrays/end_to_end/test_edited_pyi_layout_contract.py b/tests/fortran/arrays/end_to_end/test_edited_pyi_layout_contract.py index 17b2563e4..5685ae2a2 100644 --- a/tests/fortran/arrays/end_to_end/test_edited_pyi_layout_contract.py +++ b/tests/fortran/arrays/end_to_end/test_edited_pyi_layout_contract.py @@ -24,7 +24,7 @@ def test_edited_pyi_selects_direct_c_storage_or_fortran_copy_semantics(tmp_path: contract = tmp_path / "contract" contract.mkdir() (contract / "__init__.pyi").write_text( - "from .array_ops import scale_without_intent, sum_columns_copy_f, sum_columns_direct_c\n", + "from .array_ops import scale_in_place, scale_without_intent, sum_columns_copy_f, sum_columns_direct_c\n", encoding="utf-8", ) (contract / "array_ops.pyi").write_text( @@ -61,6 +61,9 @@ def sum_columns_copy_f( def scale_without_intent( values: Annotated[Float64[:], Immutable], ) -> Returns["values", Float64[:]]: ... + +@bind("scale_without_intent") +def scale_in_place(values: Float64[:]) -> Returns["values", Float64[:]]: ... """, encoding="utf-8", ) @@ -95,3 +98,8 @@ def scale_without_intent( np.testing.assert_array_equal(original, np.array([2.0, 5.0, 7.0])) np.testing.assert_array_equal(replacement, np.array([4.0, 10.0, 14.0])) assert replacement is not original + + # A mutable projected output is the caller's own array, mutated in place. + mutable = np.array([2.0, 5.0, 7.0], dtype=np.float64) + assert module.scale_in_place(mutable) is mutable + np.testing.assert_array_equal(mutable, np.array([4.0, 10.0, 14.0])) diff --git a/tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py b/tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py index 9c86a4f5c..253451768 100644 --- a/tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py +++ b/tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py @@ -59,16 +59,6 @@ def _c_ordered_strided_matrix(rows=4, cols=3): return base[:, ::2] -def _reversed_fortran_matrix(rows=4, cols=3): - base = _matrix(rows * 2, cols) - return base[::-2, :] - - -def _broadcast_fortran_like_matrix(rows=4, cols=3): - row = np.asfortranarray(np.arange(1, cols + 1, dtype=np.float64)[None, :]) - return np.broadcast_to(row, (rows, cols)) - - def _rank3(shape=(4, 3, 2)): data = np.arange(1, np.prod(shape) + 1, dtype=np.float64) return np.asfortranarray(data.reshape(shape, order="F")) @@ -99,6 +89,7 @@ def _c_ordered_strided_rank3(shape=(4, 3, 2)): def test_rank2_contiguous_contract_requires_fortran_contiguous(compiled_multid_array_module): + """Contiguous rank-two and rank-three dummies refuse C-ordered and strided storage.""" source = _matrix() out = np.zeros_like(source, order="F") @@ -119,8 +110,25 @@ def test_rank2_contiguous_contract_requires_fortran_contiguous(compiled_multid_a with pytest.raises(TypeError, match=r"expected ordering \(F\)"): compiled_multid_array_module.scale2_contiguous(strided_source, strided_out) + source = _rank3() + out = np.zeros_like(source, order="F") + + compiled_multid_array_module.shift3_contiguous(source, out) + + np.testing.assert_allclose(out, source + 10.0) + + c_order_source = np.array(source, order="C", copy=True) + with pytest.raises(TypeError, match=r"expected ordering \(F\)"): + compiled_multid_array_module.shift3_contiguous(c_order_source, out) + + strided_source = _strided_rank3() + strided_out = np.zeros_like(strided_source, order="F") + with pytest.raises(TypeError, match=r"expected ordering \(F\)"): + compiled_multid_array_module.shift3_contiguous(strided_source, strided_out) + -def test_rank2_assumed_shape_accepts_fortran_ordered_strided_views(compiled_multid_array_module): +def test_assumed_shape_accepts_fortran_ordered_strided_views(compiled_multid_array_module): + """Rank-two and rank-three assumed-shape dummies take positive-stride Fortran views.""" contiguous_source = _matrix() contiguous_out = np.zeros_like(contiguous_source, order="F") @@ -164,79 +172,6 @@ def test_rank2_assumed_shape_accepts_fortran_ordered_strided_views(compiled_mult with pytest.raises(TypeError, match=r"expected ordering \(F\)"): compiled_multid_array_module.scale2_strided(contiguous_source, c_order_out) - -def test_rank2_assumed_shape_accepts_reversed_axes_and_refuses_what_is_not_a_section( - compiled_multid_array_module, -): - """A reversed axis is described; a broadcast one has nothing to describe. - - The dummy is reached through a descriptor, which records a signed step per - axis, so an axis that runs backwards is passed on as it stands and the - callee reads the same elements the caller sees. A zero step is not a - direction, it is a repetition, and Fortran has no array section for it -- - so that one is still refused, and says so in its own terms. - """ - reversed_source = _reversed_fortran_matrix() - out = np.zeros_like(reversed_source, order="F") - checksum = np.zeros(1, dtype=np.float64) - - compiled_multid_array_module.scale2_strided(reversed_source, out) - np.testing.assert_allclose(out, 3.0 * reversed_source) - - compiled_multid_array_module.checksum2_strided(reversed_source, checksum) - np.testing.assert_allclose(checksum[0], _checksum2(reversed_source)) - - # Writing through a reversed view reaches the caller's own elements. - reversed_out = _reversed_fortran_matrix() - before = np.array(reversed_out, copy=True) - compiled_multid_array_module.scale2_strided(reversed_out, reversed_out) - np.testing.assert_allclose(reversed_out, 3.0 * before) - - broadcast_source = _broadcast_fortran_like_matrix() - assert broadcast_source.strides[0] == 0 - with pytest.raises(TypeError, match=r"not a Fortran array section"): - compiled_multid_array_module.scale2_strided(broadcast_source, out) - with pytest.raises(TypeError, match=r"not a Fortran array section"): - compiled_multid_array_module.checksum2_strided(broadcast_source, checksum) - - -def test_rank2_explicit_shape_requires_fortran_contiguous(compiled_multid_array_module): - source = _matrix() - rows, cols = source.shape - out = np.zeros_like(source, order="F") - - compiled_multid_array_module.scale2_explicit(np.int32(rows), np.int32(cols), source, out) - - np.testing.assert_allclose(out, 4.0 * source) - - c_order_source = np.array(source, order="C", copy=True) - with pytest.raises(TypeError, match=r"expected ordering \(F\)"): - compiled_multid_array_module.scale2_explicit(np.int32(rows), np.int32(cols), c_order_source, out) - - strided_source = _strided_matrix(rows, cols) - with pytest.raises(TypeError, match=r"expected ordering \(F\)"): - compiled_multid_array_module.scale2_explicit(np.int32(rows), np.int32(cols), strided_source, out) - - -def test_rank3_contiguous_contract_requires_fortran_contiguous(compiled_multid_array_module): - source = _rank3() - out = np.zeros_like(source, order="F") - - compiled_multid_array_module.shift3_contiguous(source, out) - - np.testing.assert_allclose(out, source + 10.0) - - c_order_source = np.array(source, order="C", copy=True) - with pytest.raises(TypeError, match=r"expected ordering \(F\)"): - compiled_multid_array_module.shift3_contiguous(c_order_source, out) - - strided_source = _strided_rank3() - strided_out = np.zeros_like(strided_source, order="F") - with pytest.raises(TypeError, match=r"expected ordering \(F\)"): - compiled_multid_array_module.shift3_contiguous(strided_source, strided_out) - - -def test_rank3_assumed_shape_accepts_fortran_ordered_strided_views(compiled_multid_array_module): contiguous_source = _rank3() contiguous_out = np.zeros_like(contiguous_source, order="F") @@ -270,3 +205,21 @@ def test_rank3_assumed_shape_accepts_fortran_ordered_strided_views(compiled_mult compiled_multid_array_module.shift3_strided(c_ordered_strided_source, contiguous_out) with pytest.raises(TypeError, match=r"expected ordering \(F\)"): compiled_multid_array_module.checksum3_strided(c_ordered_strided_source, contiguous_checksum) + + +def test_rank2_explicit_shape_requires_fortran_contiguous(compiled_multid_array_module): + source = _matrix() + rows, cols = source.shape + out = np.zeros_like(source, order="F") + + compiled_multid_array_module.scale2_explicit(np.int32(rows), np.int32(cols), source, out) + + np.testing.assert_allclose(out, 4.0 * source) + + c_order_source = np.array(source, order="C", copy=True) + with pytest.raises(TypeError, match=r"expected ordering \(F\)"): + compiled_multid_array_module.scale2_explicit(np.int32(rows), np.int32(cols), c_order_source, out) + + strided_source = _strided_matrix(rows, cols) + with pytest.raises(TypeError, match=r"expected ordering \(F\)"): + compiled_multid_array_module.scale2_explicit(np.int32(rows), np.int32(cols), strided_source, out) diff --git a/tests/fortran/arrays/end_to_end/test_native_handle_array_forms.py b/tests/fortran/arrays/end_to_end/test_native_handle_array_forms.py index 4d5b3de07..5a439c4d2 100644 --- a/tests/fortran/arrays/end_to_end/test_native_handle_array_forms.py +++ b/tests/fortran/arrays/end_to_end/test_native_handle_array_forms.py @@ -275,8 +275,8 @@ def record(frame, event, _arg): assert descriptor.shape == (2,) -def test_two_descriptor_dummies_reach_borrowed_and_owned_storage_alike(descriptor_matrix): - """Two descriptors are live at once, so both callees' writes reach their entities. +def test_several_descriptor_dummies_reach_borrowed_and_owned_storage_alike(descriptor_matrix): + """Several descriptors are live at once, so every callee write reaches its entity. Each argument is entered in turn and the call is made inside the last consumer, where every descriptor the Fortran runtime built is still valid. @@ -303,17 +303,13 @@ def test_two_descriptor_dummies_reach_borrowed_and_owned_storage_alike(descripto finally: owned_first.close() - -def test_three_descriptor_dummies_keep_every_borrowed_descriptor_live(descriptor_matrix): - first = descriptor_matrix.pair_left - second = descriptor_matrix.pair_right + # Three borrowed descriptors in one call are all live when it is made. third = descriptor_matrix.pair_third + descriptor_matrix.grow_three(left, right, third, np.int32(4)) - descriptor_matrix.grow_three(first, second, third, np.int32(4)) - - assert first.shape == second.shape == third.shape == (4,) - np.testing.assert_allclose(first.to_numpy(), np.full(4, 8.0)) - np.testing.assert_allclose(second.to_numpy(), np.full(4, 9.0)) + assert left.shape == right.shape == third.shape == (4,) + np.testing.assert_allclose(left.to_numpy(), np.full(4, 8.0)) + np.testing.assert_allclose(right.to_numpy(), np.full(4, 9.0)) np.testing.assert_allclose(third.to_numpy(), np.full(4, 10.0)) diff --git a/tests/fortran/arrays/end_to_end/test_primitive_dtype_rank_matrix.py b/tests/fortran/arrays/end_to_end/test_primitive_dtype_rank_matrix.py index 28e77253b..bea6a1a20 100644 --- a/tests/fortran/arrays/end_to_end/test_primitive_dtype_rank_matrix.py +++ b/tests/fortran/arrays/end_to_end/test_primitive_dtype_rank_matrix.py @@ -10,13 +10,14 @@ from tests.fortran._support.wrapper_build import _build_text_and_import from tests.fortran.arrays.end_to_end.fixtures.primitive_dtype_rank_matrix import ( PRIMITIVE_ARRAY_CASES, + PRIMITIVE_ARRAY_RANKS, primitive_dtype_rank_source, ) pytestmark = pytest.mark.fortran_end_to_end -def test_every_primitive_dtype_at_every_concrete_rank_mutates_exact_storage(tmp_path: Path): +def test_every_primitive_dtype_at_representative_ranks_mutates_exact_storage(tmp_path: Path): module = _build_text_and_import( primitive_dtype_rank_source(), "farray_dtype_rank_matrix.f90", @@ -29,7 +30,7 @@ def test_every_primitive_dtype_at_every_concrete_rank_mutates_exact_storage(tmp_ ) for case in PRIMITIVE_ARRAY_CASES: - for rank in range(1, 16): + for rank in PRIMITIVE_ARRAY_RANKS: shape = (2, *([1] * (rank - 1))) values = np.asfortranarray(np.asarray(case.values, dtype=case.dtype).reshape(shape, order="F")) diff --git a/tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py b/tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py index a00fa7b30..46f3eda36 100644 --- a/tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py +++ b/tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py @@ -55,7 +55,6 @@ def _checksum2(array): "view", [ pytest.param(lambda: _base()[::-1], id="rank-one-reversal"), - pytest.param(lambda: _base()[::-2], id="step-minus-2"), pytest.param(lambda: _base()[::2], id="step-2"), pytest.param(lambda: _base(), id="contiguous"), pytest.param(lambda: _base()[:0], id="zero-sized"), @@ -78,7 +77,6 @@ def test_rank_one_numpy_views_reach_an_assumed_shape_dummy(signed, view): pytest.param(lambda: _matrix()[:, ::-1], id="axis-1-reversed"), pytest.param(lambda: _matrix()[::-1, ::-1], id="both-axes-reversed"), pytest.param(lambda: _matrix(8, 3)[::-2, :], id="mixed-sign-strided"), - pytest.param(lambda: _matrix(8, 3)[::2, :], id="positive-strided"), pytest.param(lambda: _matrix(0, 3), id="zero-sized-axis"), ], ) @@ -122,8 +120,11 @@ def test_an_assumed_shape_dummy_rebases_every_actual_to_one(signed): assert signed.first_and_last(view) == pytest.approx(expected) -def test_a_reversed_pointer_handle_reaches_the_same_dummy(signed): - """A handle's descriptor already records its direction; it is entered as it is.""" +def test_reversed_pointer_and_field_handles_reach_the_same_dummy(signed): + """A handle's descriptor already records its direction; it is entered as it is. + + A field handle is entered through its parent, and keeps its direction too. + """ reversed_handle = signed.reversed_ptr strided_handle = signed.strided_ptr @@ -135,11 +136,7 @@ def test_a_reversed_pointer_handle_reaches_the_same_dummy(signed): assert signed.total1_bindc(reversed_handle) == pytest.approx(36.0) assert signed.total1(strided_handle) == pytest.approx(16.0) - -def test_a_reversed_derived_field_handle_reaches_the_same_dummy(signed): - """A field handle is entered through its parent, and keeps its direction.""" field = signed.parent.field_ptr - assert isinstance(field, PointerArray) assert field.shape == (6,) # store(6:1:-1) holds 6, 5, 4, 3, 2, 1. @@ -181,22 +178,20 @@ def test_assumed_rank_dummies_read_rank_and_size_from_the_descriptor(signed): assert signed.rank_and_size(signed.reversed_ptr) == np.int32(108) -def test_a_character_dummy_reports_its_own_width_from_either_source(signed): - """Character arrays keep their runtime element width on the portable path.""" - assert signed.word_width(signed.words) == np.int32(4) - assert signed.word_width(np.array([b"abcd", b"efgh"], dtype="S4")) == np.int32(4) - - -def test_a_character_dummy_accepts_a_reversed_section_from_either_source(signed): +def test_a_character_dummy_takes_width_and_signed_sections_from_either_source(signed): """A character array is reached by address, and a signed stride travels beside it. This is the direct-entrypoint answer for a character dummy. It cannot be ``bind(C)`` above length one, so a bridge is generated for it, and the - bridge is handed the buffer with a bound and a signed step per axis -- the - same triple a descriptor carries, in the form this ABI already had. + bridge is handed the buffer with its runtime element width, a bound and a + signed step per axis -- the same triple a descriptor carries, in the form + this ABI already had. A reversed handle's own descriptor is read into the + same bounds and step, so both sources arrive at the bridge in one shape. """ - words = np.array([b"abcd", b"efgh", b"ijkl", b"mnop"], dtype="S4") + assert signed.word_width(signed.words) == np.int32(4) + assert signed.word_width(np.array([b"abcd", b"efgh"], dtype="S4")) == np.int32(4) + words = np.array([b"abcd", b"efgh", b"ijkl", b"mnop"], dtype="S4") assert signed.word_width(words[::-1]) == np.int32(4) assert signed.word_join(words[::-1]).strip() == "mnopijklefghabcd" assert signed.word_join(words[::2]).strip() == "abcdijkl" @@ -204,6 +199,9 @@ def test_a_character_dummy_accepts_a_reversed_section_from_either_source(signed) assert signed.word_join(words[1::2]).strip() == "efghmnop" assert signed.word_join(words[:0]).strip() == "" + assert signed.word_join(signed.words).strip() == "abcdefghijklmnop" + assert signed.word_join(signed.reversed_words).strip() == "mnopijklefghabcd" + def test_a_character_dummy_writes_back_through_a_reversed_section(signed): """Only the elements the section names are written, in the caller's own storage.""" @@ -214,16 +212,6 @@ def test_a_character_dummy_writes_back_through_a_reversed_section(signed): assert list(words) == [b"3aaa", b"bbbb", b"2ccc", b"dddd", b"1eee"] -def test_a_character_dummy_accepts_a_reversed_handle(signed): - """A reversed handle reaches the same sectioned dummy a reversed view does. - - The handle's own descriptor is read into the same bounds and signed step, - so both sources arrive at the bridge in one shape. - """ - assert signed.word_join(signed.words).strip() == "abcdefghijklmnop" - assert signed.word_join(signed.reversed_words).strip() == "mnopijklefghabcd" - - def test_a_bound_handle_reaches_a_signed_stride_call_without_running_python(signed): """The reversed handoff costs no Python frame once the arguments are parsed.""" handle = signed.reversed_ptr @@ -268,8 +256,12 @@ def test_raw_address_dummies_refuse_what_an_address_cannot_convey(signed): assert signed.contig_total(_base()) == pytest.approx(36.0) -def test_layouts_that_are_not_array_sections_stay_refused(signed): - """A broadcast or overlapping view has no contiguous parent to be a section of.""" +def test_storage_that_is_not_a_described_section_stays_refused(signed): + """Broadcast, overlapping, absent and mismatched storage are refused in their own terms. + + A broadcast or overlapping view has no contiguous parent to be a section + of, and state and type checks are unchanged by how storage is handed over. + """ broadcast = np.broadcast_to(np.arange(1.0, 4.0), (4, 3)) assert broadcast.strides[0] == 0 with pytest.raises(TypeError, match=r"not a Fortran array section"): @@ -283,9 +275,6 @@ def test_layouts_that_are_not_array_sections_stay_refused(signed): with pytest.raises(TypeError, match=r"not a Fortran array section"): signed.checksum2(indivisible) - -def test_absent_and_mismatched_storage_stay_refused(signed): - """State and type checks are unchanged by how the storage is handed over.""" with pytest.raises(ValueError, match=r"unassociated"): signed.total1(signed.unassociated_ptr) with pytest.raises(TypeError, match=r"dtype"): diff --git a/tests/fortran/arrays/parsing/test_array_declarations_and_shapes.py b/tests/fortran/arrays/parsing/test_array_declarations_and_shapes.py index d353979a6..e9f34e865 100644 --- a/tests/fortran/arrays/parsing/test_array_declarations_and_shapes.py +++ b/tests/fortran/arrays/parsing/test_array_declarations_and_shapes.py @@ -1,134 +1,64 @@ -"""Tests split by stable ownership concept from `test_procedures_and_interfaces.py`.""" +"""Array declaration shapes and bounds as the Fortran parser records them.""" + +import pytest from prik.parsers.fortran import parse_fortran_file -from prik.parsers.fortran.models import ( - FortranUseMapping, - FortranVariable, -) -from tests.fortran._support.parser_procedures import ( - collect_project_procedure_signatures, - collect_signature_shape_symbols, - evaluate_signature_shapes, +from tests.fortran._support.parser_procedures import collect_project_procedure_signatures + + +@pytest.mark.parametrize( + ("declaration", "shape", "lower_bounds", "upper_bounds"), + [ + pytest.param( + "integer, intent(inout) :: x(0:,0:)", + ["0:", "0:"], + ["0", "0"], + [None, None], + id="assumed-shape-lower-bounds", + ), + pytest.param( + "real, intent(inout), dimension(0:, 1:n) :: x", + ["0:", "1:n"], + ["0", "1"], + [None, "n"], + id="dimension-attribute-mixed-bounds", + ), + pytest.param("real, intent(inout) :: x(n)", ["n"], ["1"], ["n"], id="explicit-extent-default-lower-bound"), + ], ) +def test_array_bounds_split_into_lower_and_upper(declaration, shape, lower_bounds, upper_bounds): + signature = parse_fortran_file(f"subroutine s(x)\n {declaration}\nend subroutine s\n").procedures[0] + argument = signature.arguments[0] + + assert argument.rank == len(shape) + assert argument.shape == shape + assert argument.lower_bounds == lower_bounds + assert argument.upper_bounds == upper_bounds -def test_compile_time_shape_eval_with_local_and_imported_params(): +def test_parameter_expressions_in_shapes_are_kept_across_local_and_imported_parameters(): files = { "kinds.f90": """ module k integer, parameter :: n = 8 end module k """, - "solver.f90": """ -subroutine step(x) - use k, only: n - integer, parameter :: m = n + 2 - real, intent(inout) :: x(m*2) -end subroutine step -""", - } - sig = collect_project_procedure_signatures(files)[0] - assert sig.arguments[0].shape == ["m*2"] - - -def test_assumed_shape_with_explicit_lower_bounds_is_preserved(): - code = """ -subroutine fill_grid(x) - integer, intent(inout) :: x(0:,0:) -end subroutine fill_grid -""" - sig = parse_fortran_file(code).procedures[0] - arg = sig.arguments[0] - assert arg.base_type == "integer" - assert arg.rank == 2 - assert arg.shape == ["0:", "0:"] - - -def test_dimension_attribute_with_mixed_bounds_is_parsed(): - code = """ -subroutine update_plane(x) - real, intent(inout), dimension(0:, 1:n) :: x -end subroutine update_plane -""" - sig = parse_fortran_file(code).procedures[0] - arg = sig.arguments[0] - assert arg.rank == 2 - assert arg.shape == ["0:", "1:n"] - assert arg.lower_bounds == ["0", "1"] - assert arg.upper_bounds == [None, "n"] - assert arg.lbound == ["0", "1"] - assert arg.ubound == [None, "n"] - assert arg.shape_info == [ - {"raw": "0:", "lower": "0", "upper": None}, - {"raw": "1:n", "lower": "1", "upper": "n"}, - ] - - -def test_shape_info_for_explicit_extent_dimension(): - code = """ -subroutine resize(x) - real, intent(inout) :: x(n) -end subroutine resize -""" - sig = parse_fortran_file(code).procedures[0] - arg = sig.arguments[0] - assert arg.shape_info == [ - {"raw": "n", "lower": "1", "upper": "n"}, - ] - assert arg.lower_bounds == ["1"] - assert arg.upper_bounds == ["n"] - assert arg.lbound == ["1"] - assert arg.ubound == ["n"] - - -def test_structured_shape_handles_empty_dimensions_and_use_mapping_equality(): - from prik.parsers.fortran.type_resolver import extract_kind_from_type_spec - - var = FortranVariable(name="empty", shape=[""]) - assert var.shape_info == [{"raw": "", "lower": None, "upper": None}] - shape = var.structured_shape - assert shape.raw == [""] - assert shape.dimensions == [None] - assert extract_kind_from_type_spec("real", "()") is None - assert extract_kind_from_type_spec("real", "(len=5)") is None - - renamed = FortranUseMapping(source="delete_input_list", target="delete_input") - assert renamed == "delete_input" - assert renamed == FortranUseMapping(source="delete_input_list", target="delete_input") - assert renamed != FortranUseMapping(source="delete_input_list") - assert renamed != object() - - -def test_compile_time_parameter_expressions_are_evaluated_in_shapes(): - files = { "dims.f90": """ module dims_mod - integer, parameter :: n0 = 4 - integer, parameter :: n1 = n0 + 2 + use k, only: n + integer, parameter :: m = n + 2 contains subroutine use_expr(x, y) - integer, intent(inout) :: x(0:n1-1) - real, intent(inout), dimension(1:n0*2) :: y + integer, intent(inout) :: x(0:m-1) + real, intent(inout), dimension(1:m*2) :: y end subroutine use_expr end module dims_mod -""" +""", } - sig = collect_project_procedure_signatures(files)[0] - assert sig.arguments[0].shape == ["0:n1-1"] - assert sig.arguments[1].shape == ["1:n0*2"] - - -def test_symbolic_shape_symbols_can_be_collected_and_later_evaluated(): - code = """ -subroutine s(a) - real, intent(inout) :: a(0:nx-1, 1:ny*2) -end subroutine s -""" - sig = parse_fortran_file(code).procedures[0] - assert collect_signature_shape_symbols(sig) == {"nx", "ny"} + signature = collect_project_procedure_signatures(files)[0] - evaluated = evaluate_signature_shapes(sig, {"nx": 6, "ny": 4}) - assert evaluated.arguments[0].shape == ["0:6-1", "1:4*2"] + assert signature.arguments[0].shape == ["0:m-1"] + assert signature.arguments[1].shape == ["1:m*2"] def test_balanced_extent_expressions_are_preserved_for_every_declaration_owner(): diff --git a/tests/fortran/arrays/pipeline/test_generated_array_contracts.py b/tests/fortran/arrays/pipeline/test_generated_array_contracts.py index 1b0a51769..532282d23 100644 --- a/tests/fortran/arrays/pipeline/test_generated_array_contracts.py +++ b/tests/fortran/arrays/pipeline/test_generated_array_contracts.py @@ -1,38 +1,27 @@ -"""Generated `.pyi` package fixtures for array wrapper inputs.""" +"""Generated `.pyi` package fixture for the source-only array wrapper input. + +Every other array contract fixture is compared with a freshly generated +contract by the generated-`.pyi` lane of the end-to-end test that replays it. +""" from __future__ import annotations from pathlib import Path -import pytest - from tests.fortran._support.generated_contracts import ( GeneratedContractCase, assert_generated_contract_matches_fixture, - contract_case_id, ) FIXTURE_ROOT = Path(__file__).resolve().parents[1] / "end_to_end" / "fixtures" -NATIVE_ROOT = FIXTURE_ROOT / "native" -CONTRACT_ROOT = FIXTURE_ROOT / "contracts" -CASES = tuple( - GeneratedContractCase( + + +def test_fmath_arrays_f90_generated_pyi_contract_matches_fixture(tmp_path: Path): + source = FIXTURE_ROOT / "native" / "fmath_arrays_f90.f90" + case = GeneratedContractCase( name=source.stem, inputs=(source,), - expected_package=CONTRACT_ROOT / source.stem, + expected_package=FIXTURE_ROOT / "contracts" / source.stem, ) - for source in ( - NATIVE_ROOT / "array_ops.f90", - NATIVE_ROOT / "farray_contracts_f90.f90", - NATIVE_ROOT / "farray_results_f90.f90", - NATIVE_ROOT / "fassumed_rank_f90.f90", - NATIVE_ROOT / "fmath_arrays.f", - NATIVE_ROOT / "fmath_arrays_f90.f90", - NATIVE_ROOT / "multid_arrays.f90", - ) -) - -@pytest.mark.parametrize("case", CASES, ids=contract_case_id) -def test_array_generated_pyi_contract_matches_fixture(case: GeneratedContractCase, tmp_path: Path): assert_generated_contract_matches_fixture(case, tmp_path) diff --git a/tests/fortran/arrays/policy/test_array_shape_policy.py b/tests/fortran/arrays/policy/test_array_shape_policy.py index e03105d54..87af53c10 100644 --- a/tests/fortran/arrays/policy/test_array_shape_policy.py +++ b/tests/fortran/arrays/policy/test_array_shape_policy.py @@ -34,36 +34,6 @@ def values() -> Float64[missing]: ... ) -def test_python_array_properties_and_integer_helpers_resolve_to_extent_roles(): - module = parse_pyi_text( - """ -from prik.contracts import Float64 - -def values(source: Float64[:, :]) -> Float64[ - source.size, - source.shape[1], - source.ndim, - len(source), - max(1, source.shape[0] - 1), - 2 ** source.shape[1], -]: ... -""", - module_name="property_extents", - ) - complete_semantic_policies(module) - - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - assert policy.supported is True - assert policy.results[0].array.shape == ( - "__prik_extent_source_0 * __prik_extent_source_1", - "__prik_extent_source_1", - "2", - "__prik_extent_source_0", - "max(1, __prik_extent_source_0 - 1)", - "2 ** __prik_extent_source_1", - ) - - def test_persistent_array_extents_reject_unavailable_runtime_values(): module = parse_pyi_text( """ diff --git a/tests/fortran/arrays/semantics/test_array_semantics.py b/tests/fortran/arrays/semantics/test_array_semantics.py index f650340b9..e0dac3ad1 100644 --- a/tests/fortran/arrays/semantics/test_array_semantics.py +++ b/tests/fortran/arrays/semantics/test_array_semantics.py @@ -1,4 +1,4 @@ -"""Tests split by stable ownership concept from `test_compile_time_values.py`.""" +"""Array declaration semantics and declaration-expression provenance.""" from pathlib import Path @@ -20,126 +20,43 @@ NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" -def test_array_constraints(): +def test_fortran_array_declarations_complete_category_shape_and_order(): + """Each declaration form becomes one array contract; bounds collapse to extents.""" source = """ -module array_mod - -contains - -subroutine scale(x) - - real(8), intent(inout) :: x(:) - -end subroutine - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - func = get_function(smod, "scale") - - x = func.arguments[0] - - assert x.semantic_type.name == "Float64" - - assert x.semantic_type.rank == 1 - - contract = array_contract(x.semantic_type) - assert contract.category == "assumed_shape" - assert contract.shape == ["::"] - assert contract.source_shape == [":"] - assert contract.order is None - - -def test_matrix_semantics(): - source = """ -module linalg_mod - +module array_forms contains - -subroutine matvec(A, x, y) - - real(8), intent(in) :: A(:, :) - real(8), intent(in) :: x(:) - real(8), intent(out) :: y(:) - -end subroutine - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - func = get_function(smod, "matvec") - - A = func.arguments[0] - - assert A.semantic_type.rank == 2 - - contract = array_contract(A.semantic_type) - assert A.semantic_type.shape == ["::", "::"] - assert contract.source_shape == [":", ":"] - assert contract.category == "assumed_shape" - assert contract.order == "ORDER_F" - - -def test_explicit_bound_ranges_remain_shaped_storage_contracts(): - source = """ -module bound_mod -contains -subroutine bounded(n, default_bound, zero_bound, shifted_bound) +subroutine forms(n, vector, matrix, fixed, default_bound, zero_bound, shifted_bound) integer, intent(in) :: n + real(8), intent(inout) :: vector(:) + real(8), intent(in) :: matrix(:, :) + real(8), intent(in) :: fixed(10, 20) real(8), intent(inout) :: default_bound(1:n) real(8), intent(inout) :: zero_bound(0:n-1) real(8), intent(inout) :: shifted_bound(2:n+1) -end subroutine bounded -end module bound_mod +end subroutine forms +end module array_forms """ module = fortran_module_to_semantic_module(parse_fortran_source(source)) - args = {arg.name: arg for arg in get_function(module, "bounded").arguments} - - default_bound = array_contract(args["default_bound"].semantic_type) - assert default_bound.category == "explicit_shape" - assert default_bound.shape == ["n"] - - zero_bound = array_contract(args["zero_bound"].semantic_type) - assert zero_bound.category == "explicit_shape" - assert zero_bound.shape == ["n"] - - shifted_bound = array_contract(args["shifted_bound"].semantic_type) - assert shifted_bound.category == "explicit_shape" - assert shifted_bound.shape == ["n"] - - -def test_explicit_shape(): - source = """ -module shape_mod - -contains - -subroutine foo(A) - - real(8), intent(in) :: A(10, 20) - -end subroutine - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - func = get_function(smod, "foo") - - A = func.arguments[0] - - assert A.semantic_type.shape == ["10", "20"] + arguments = {argument.name: argument.semantic_type for argument in get_function(module, "forms").arguments} + expected = { + "vector": ("assumed_shape", ["::"], [":"], None), + "matrix": ("assumed_shape", ["::", "::"], [":", ":"], "ORDER_F"), + "fixed": ("explicit_shape", ["10", "20"], None, "ORDER_F"), + "default_bound": ("explicit_shape", ["n"], None, None), + "zero_bound": ("explicit_shape", ["n"], None, None), + "shifted_bound": ("explicit_shape", ["n"], None, None), + } + + assert arguments["vector"].name == "Float64" + for name, (category, shape, source_shape, order) in expected.items(): + contract = array_contract(arguments[name]) + assert arguments[name].rank == len(shape), name + assert contract.category == category, name + assert contract.shape == shape, name + if source_shape is not None: + assert contract.source_shape == source_shape, name + if order is not None or len(shape) == 1: + assert contract.order == order, name def test_fortran_inquiries_become_python_array_expressions_and_keep_source_bounds(): diff --git a/tests/fortran/arrays/semantics/test_declaration_expression_utilities.py b/tests/fortran/arrays/semantics/test_declaration_expression_utilities.py index c04dc7efe..1c52e394a 100644 --- a/tests/fortran/arrays/semantics/test_declaration_expression_utilities.py +++ b/tests/fortran/arrays/semantics/test_declaration_expression_utilities.py @@ -146,12 +146,8 @@ def test_public_expression_grammar_rejects_unsupported_syntax(expression: str, e [ ("abs(-3)", 3), ("max(3, 7, 4)", 7), - ("min(3, 7, 4)", 3), ("modulo(8, 3)", 2), ("product((/ 2, 3, 4 /))", 24), - ("sum((/ 2, 3, 4 /))", 9), - ("maxval((/ 2, 3, 4 /))", 4), - ("minval((/ 2, 3, 4 /))", 2), ("merge(4, 2, .true.)", 4), ("int(2.9)", 2), ("len('abc')", 3), @@ -160,20 +156,14 @@ def test_public_expression_grammar_rejects_unsupported_syntax(expression: str, e ("2 ** 3", 8), ("3 if 1 < 2 else 4", 3), ("True and not False", 1), - ("False or True", 1), ("1 == 1 == 1", 1), - ("1 != 2", 1), - ("1 >= 1", 1), - ("1 <= 1", 1), ("+3", 3), ("1 // 1", 1), ("1 << 1", None), - ("~1", None), ("int(2.9, kind=4)", 2), ("int(2, base=10)", None), ("abs(1, 2)", None), ("iachar('')", None), - ("len(1)", None), ("sum((/ /))", None), ("1 / 0", None), ("max()", None), @@ -263,8 +253,6 @@ def test_role_resolution_reuses_completed_roles_and_names_blockers() -> None: ("n % 3", "fortran", "mod((native_n), (3))"), ("n % 3", "c", "native_n % 3"), ("n * (m + limit)", "c", "native_n * (native_m + native_limit)"), - ("n - (m - limit)", "fortran", "native_n - (native_m - native_limit)"), - ("(n ** m) ** limit", "fortran", "(native_n ** native_m) ** native_limit"), ("n ** (m ** limit)", "c", "prik_extent_power((native_n), (prik_extent_power((native_m), (native_limit))))"), ("-(n + m)", "c", "-(native_n + native_m)"), ("+n", "fortran", "+native_n"), @@ -272,17 +260,12 @@ def test_role_resolution_reuses_completed_roles_and_names_blockers() -> None: ("not flag", "c", "! (native_flag)"), ("n and m or flag", "c", "((((native_n) && (native_m))) || (native_flag))"), ("n and m", "fortran", "((native_n) .and. (native_m))"), - ("n or m", "fortran", "((native_n) .or. (native_m))"), ("n < m <= limit", "fortran", "(((native_n) .lt. (native_m)) .and. ((native_m) .le. (native_limit)))"), ("n == m", "c", "(((native_n) == (native_m)))"), ("n != m", "fortran", "(((native_n) .ne. (native_m)))"), - ("n > m", "c", "(((native_n) > (native_m)))"), - ("n >= m", "fortran", "(((native_n) .ge. (native_m)))"), ("n if flag else m", "c", "((native_flag) ? (native_n) : (native_m))"), ("n if flag else m", "fortran", "merge((native_n), (native_m), (native_flag))"), ("int(n)", "c", "((npy_intp)(native_n))"), - ("int(n)", "fortran", "int(native_n)"), - ("abs(n)", "fortran", "abs(native_n)"), ("abs(n)", "c", "((native_n) < 0 ? -(native_n) : (native_n))"), ("max(n, m, limit)", "fortran", "max(native_n, native_m, native_limit)"), ( @@ -290,8 +273,6 @@ def test_role_resolution_reuses_completed_roles_and_names_blockers() -> None: "c", "((native_n) > (native_m) ? (native_n) : (native_m))", ), - ("min(n, m)", "c", "((native_n) < (native_m) ? (native_n) : (native_m))"), - ("min(n, m)", "fortran", "min(native_n, native_m)"), ("extent_for(n)", "c", "native_extent_for(native_n)"), ("True", "c", "1"), ("False", "fortran", ".false."), @@ -324,36 +305,21 @@ def test_backend_renderer_rejects_invalid_target_and_unrenderable_syntax() -> No render_declaration_extent("[n]", {}, target="c") -def test_a_character_literal_references_no_name_it_happens_to_spell(): - """Parsing decides what is a reference, so a literal's contents are its value.""" - assert declaration_expression_identifiers('"box"') == () - assert declaration_expression_identifiers("'box'") == () - - -def test_an_expression_reports_the_names_it_reads(): - """A name used in a declaration is a reference wherever it appears.""" - assert declaration_expression_identifiers("crate") == ("crate",) - assert set(declaration_expression_identifiers("n * 2 + other")) == {"n", "other"} - assert set(declaration_expression_identifiers("size(values)")) == {"size", "values"} - - -def test_a_selector_keyword_names_a_slot_rather_than_an_entity(): - """`len` and `kind` are syntax, so only the value they carry is read.""" - assert declaration_expression_identifiers("len=3") == () - assert declaration_expression_identifiers("len=n") == ("n",) - assert declaration_expression_identifiers("kind=c_char") == ("c_char",) - assert declaration_expression_identifiers('kind="box"') == () - - -def test_each_selector_in_one_declaration_is_read_separately(): - """A character declaration carries both selectors in one stored string.""" - assert declaration_expression_identifiers("len=n, kind=c_char") == ("n", "c_char") - assert declaration_expression_identifiers("len=1, kind=c_char") == ("c_char",) - - -def test_a_comparison_is_not_read_as_a_selector(): - """`==` is an operator, so both sides are part of the expression.""" - assert set(declaration_expression_identifiers("a == b")) == {"a", "b"} +@pytest.mark.parametrize( + ("expression", "expected"), + [ + pytest.param('"box"', (), id="double-quoted-literal"), + pytest.param("n * 2 + other", ("n", "other"), id="operands"), + pytest.param("len=n", ("n",), id="selector-keyword-is-a-slot"), + pytest.param('kind="box"', (), id="selector-with-string"), + pytest.param("len=n, kind=c_char", ("n", "c_char"), id="each-selector-read-separately"), + pytest.param("a == b", ("a", "b"), id="comparison-is-not-a-selector"), + pytest.param("lambda(n) + 1", ("lambda", "n"), id="python-reserved-native-call"), + ], +) +def test_an_expression_reports_exactly_the_names_it_reads(expression: str, expected: tuple[str, ...]): + """Literal contents and `len`/`kind` selector keywords are not references.""" + assert sorted(declaration_expression_identifiers(expression)) == sorted(expected) def test_lexical_translation_leaves_character_literals_alone(): @@ -368,14 +334,12 @@ def test_lexical_translation_leaves_character_literals_alone(): assert _python_parseable_fortran_expression(".true.") == "True" -def test_a_native_name_python_reserves_is_still_read_as_a_call(): - """A Fortran function may be called `lambda`; the call is not invalid syntax.""" - assert declaration_expression_calls("lambda(n) + class(2)") == ("lambda", "class") - assert declaration_expression_identifiers("lambda(n) + 1") == ("lambda", "n") - - def test_respelling_changes_call_targets_and_nothing_else(): - """A variable or a literal spelled like the callee keeps its spelling.""" + """A variable or a literal spelled like the callee keeps its spelling. + + A Fortran function may be called `lambda`; it is still read as a call. + """ + assert declaration_expression_calls("lambda(n) + class(2)") == ("lambda", "class") assert rename_declaration_expression_calls("lambda(n)", {"lambda": "lambda_"}) == "lambda_(n)" assert ( rename_declaration_expression_calls("helper(n) + helper + len('helper(')", {"helper": "helper_2"}) diff --git a/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py b/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py index 626887cd1..08ed65f58 100644 --- a/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py +++ b/tests/fortran/assumed_types/end_to_end/test_assumed_type_calls.py @@ -6,10 +6,6 @@ import pytest from prik.pipeline.build import build_fortran_extension, build_pyi_extension -from prik.pipeline.pyi import pyi_file_to_semantic_module -from prik.planning import WrapperPlanner -from prik.policy import complete_semantic_policies -from prik.policy.models import ArrayEntrypointABI, EntrypointPassingConvention from tests.fortran._support.wrapper_build import _import_from_build_dir @@ -42,23 +38,7 @@ def calls(request, native_build, tmp_path_factory): def test_hand_authored_any_native_contract_loads_plans_and_calls(native_build, tmp_path): - semantic = pyi_file_to_semantic_module(AUTHORED_CONTRACT) - assert all(function.arguments[0].semantic_type.name == "AnyNative" for function in semantic.functions) - complete_semantic_policies(semantic) - plan = WrapperPlanner().build(semantic) - expected = { - "scalar": (EntrypointPassingConvention.POINTER_REFERENCE, None), - "assumed_size": (EntrypointPassingConvention.POINTER_REFERENCE, ArrayEntrypointABI.RAW_ADDRESS), - "assumed_shape": (EntrypointPassingConvention.C_DESCRIPTOR_POINTER, ArrayEntrypointABI.C_DESCRIPTOR), - "assumed_shape_two": (EntrypointPassingConvention.C_DESCRIPTOR_POINTER, ArrayEntrypointABI.C_DESCRIPTOR), - "assumed_shape_three": (EntrypointPassingConvention.C_DESCRIPTOR_POINTER, ArrayEntrypointABI.C_DESCRIPTOR), - "assumed_rank": (EntrypointPassingConvention.C_DESCRIPTOR_POINTER, ArrayEntrypointABI.C_DESCRIPTOR), - } - for function in plan.namespaces[0].functions: - passing, array_abi = expected[function.binding.python_name] - assert function.arguments[0].entrypoint.passing is passing - if array_abi is not None: - assert function.arguments[0].array.entrypoint_abi is array_abi + """Each dummy form picks its ABI: an address, or a descriptor that keeps strides.""" built = build_pyi_extension( AUTHORED_CONTRACT, native_objects=[native_build.output_dir / "assumed_type_calls.o"], @@ -146,11 +126,6 @@ def test_optional_absence_uses_dummy_specific_null_representation(calls): assert calls.optional_descriptor(np.arange(2, dtype=np.int64)) == 1 -def test_arbitrary_python_object_is_not_a_native_actual(calls): - with pytest.raises(TypeError, match="requires NumPy storage or a PRIK native object"): - calls.scalar(object()) - - def test_writable_scalar_requires_ndarray_storage(calls): with pytest.raises(TypeError, match="writable TYPE\\(\\*\\) requires a writable NumPy ndarray"): calls.modify_scalar(np.int64(7)) diff --git a/tests/fortran/assumed_types/runtime/test_assumed_type_actual.py b/tests/fortran/assumed_types/runtime/test_assumed_type_actual.py index 574b9764a..cccc354fe 100644 --- a/tests/fortran/assumed_types/runtime/test_assumed_type_actual.py +++ b/tests/fortran/assumed_types/runtime/test_assumed_type_actual.py @@ -144,9 +144,3 @@ def test_module_proxy_crosses_extension_boundary_without_type_enumeration(actual calls = _import_from_build_dir(built.module_name, built.output_dir).assumed_type_calls assert calls.scalar(derived.plain) == 10 assert calls.assumed_rank(derived.plain) == 0 - - -def test_arbitrary_class_has_no_native_representation(actuals): - probe, _ = actuals - with pytest.raises(TypeError, match="requires NumPy storage or a PRIK native object"): - probe.describe(object()) diff --git a/tests/fortran/assumed_types/semantics/test_assumed_type_semantics.py b/tests/fortran/assumed_types/semantics/test_assumed_type_semantics.py index 28385e231..67679cde4 100644 --- a/tests/fortran/assumed_types/semantics/test_assumed_type_semantics.py +++ b/tests/fortran/assumed_types/semantics/test_assumed_type_semantics.py @@ -111,8 +111,7 @@ def test_generated_contract_uses_public_any_native_forms(): assert arguments["shape_one"].semantic_type.metadata["fortran_target"] is True assert all(argument.semantic_type.name == "AnyNative" for argument in arguments.values()) - -def test_generated_contract_uses_bare_shape_forms_without_extra_metadata(): + # Without attributes, each shape is written bare, with no Annotated metadata. forms = ( ("scalar", "", "AnyNative"), ("raw", ", dimension(*)", "AnyNative[Flat]"), @@ -127,30 +126,23 @@ def test_generated_contract_uses_bare_shape_forms_without_extra_metadata(): + "\nend module" ) semantic = fortran_module_to_semantic_module(parse_fortran_file(source).modules[0]) - contract = PyiPrinter().emit(semantic) + bare_contract = PyiPrinter().emit(semantic) for _name, _shape, annotation in forms: - assert f"x: {annotation}\n" in contract - assert "Annotated" not in contract - - -def test_edited_contract_rejects_higher_rank_assumed_size(): - source = """from prik.contracts import AnyNative, Flat -def f(x: AnyNative[:, Flat]) -> None: ... -""" - with pytest.raises(ValueError, match="rank-one assumed-size"): - pyi_text_to_semantic_module(source, module_name="m") + assert f"x: {annotation}\n" in bare_contract + assert "Annotated" not in bare_contract @pytest.mark.parametrize( - "annotation", + ("annotation", "message"), [ - pytest.param("AnyNative[4]", id="explicit-shape"), - pytest.param("AnyNative[::]", id="strided-spelling"), + pytest.param("AnyNative[:, Flat]", "rank-one assumed-size", id="higher-rank-assumed-size"), + pytest.param("AnyNative[4]", "AnyNative", id="explicit-shape"), + pytest.param("AnyNative[::]", "AnyNative", id="strided-spelling"), ], ) -def test_any_native_shape_is_the_single_array_category_authority(annotation): - source = f"from prik.contracts import AnyNative\ndef f(x: {annotation}) -> None: ..." - with pytest.raises(ValueError, match="AnyNative"): +def test_edited_contract_rejects_any_native_shapes_a_dummy_cannot_declare(annotation, message): + source = f"from prik.contracts import AnyNative, Flat\ndef f(x: {annotation}) -> None: ..." + with pytest.raises(ValueError, match=message): pyi_text_to_semantic_module(source, module_name="m") diff --git a/tests/fortran/callbacks/codegen/test_callback_planning.py b/tests/fortran/callbacks/codegen/test_callback_planning.py index 866b3003a..6e81777dd 100644 --- a/tests/fortran/callbacks/codegen/test_callback_planning.py +++ b/tests/fortran/callbacks/codegen/test_callback_planning.py @@ -4,29 +4,18 @@ import pytest -from prik.parsers.fortran import parse_fortran_file as parse_fortran_source from prik.pipeline.pyi import pyi_file_to_semantic_module, pyi_text_to_semantic_module -from prik.semantics.fortran2ir import FortranToIRConverter -from prik.semantics import models from prik.policy.ownership import PythonBarrierAction from prik.policy.completion import complete_semantic_policies from prik.policy.models import ( - CallbackABIKind, - CallbackGILAction, - CallbackLifecycleAction, CallbackOptionalityAction, CallbackResultAction, - CallbackThreadAction, - CallbackTransferAction, - OptionalMode, ) from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import GeneratedSupportProcedureImplementationOwner, WrapperPlanner -from prik.planning.models import DatatypeFamily +from prik.planning import WrapperPlanner CONTRACT_ROOT = Path(__file__).parents[1] / "end_to_end" / "fixtures" / "contracts" CONTRACT = CONTRACT_ROOT / "fcallback_all_f90" / "fcallback_all_f90.pyi" -ARRAY_CONTRACT = CONTRACT.parents[1] / "fcallback_array_f90" / "fcallback_array_f90.pyi" def _module(): @@ -56,93 +45,6 @@ def _sources(plan): return c_source, bridge -def test_callback_policy_completes_value_default_and_explicit_reference_before_planning(): - module = _module() - policies = { - function.name: function.metadata[models.RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - for function in module.functions - } - - scalar = policies["apply_scalar_storage_callback"].arguments[0].callback - assert scalar.lifecycle == tuple(CallbackLifecycleAction) - assert scalar.thread_action is CallbackThreadAction.REQUIRE_ENTERING_THREAD - assert scalar.gil_actions == (CallbackGILAction.ACQUIRE_GIL, CallbackGILAction.RELEASE_GIL) - assert tuple(transfer.abi for transfer in scalar.arguments) == (CallbackABIKind.REFERENCE,) * 3 - # An undeclared intent permits the callee to read and modify the dummy, so - # it copies both ways rather than defaulting to copy-in. - assert tuple(transfer.adapter_action for transfer in scalar.arguments) == ( - CallbackTransferAction.COPY_IN_OUT, - CallbackTransferAction.COPY_OUT, - CallbackTransferAction.COPY_IN_OUT, - ) - # Every dummy the callee may write needs storage Python can write through. - assert tuple(transfer.python_action for transfer in scalar.arguments) == (PythonBarrierAction.SCALAR_STORAGE,) * 3 - - array = policies["apply_array_storage_callback"].arguments[0].callback - assert array.arguments[0].abi is CallbackABIKind.REFERENCE - assert array.arguments[0].adapter_action is CallbackTransferAction.COPY_IN - assert array.arguments[0].python_action is PythonBarrierAction.SCALAR_VALUE - assert array.arguments[1].abi is CallbackABIKind.DATA_AND_SHAPE - assert array.arguments[1].array.shape == ("count",) - - string = policies["apply_string_storage_callback"].arguments[0].callback - assert all(transfer.abi is CallbackABIKind.DATA_AND_LENGTH for transfer in string.arguments) - assert tuple(transfer.character_length for transfer in string.arguments) == (8, 8, 8) - - derived = policies["apply_point_callback"].arguments[0].callback - assert derived.arguments[0].derived_type_identity == ("fcallback_all_f90", "Point_T") - assert derived.result.action is CallbackResultAction.RETURN_DERIVED_ADDRESS - - -def test_callback_plan_projects_one_explicit_site_and_stable_roles_per_argument(): - plan = _plan() - callbacks = [ - argument.callback - for namespace in plan.namespaces - for function in namespace.functions - for argument in function.arguments - if argument.callback is not None - ] - - assert all( - _callback_argument(plan, function).datatype_family is DatatypeFamily.CALLBACK - for function in ( - "apply_value_callback", - "apply_scalar_storage_callback", - "apply_array_storage_callback", - "apply_string_storage_callback", - "apply_point_callback", - ) - ) - assert all( - not _function(plan, function).binding.release_gil - for function in ( - "apply_value_callback", - "apply_scalar_storage_callback", - "apply_array_storage_callback", - "apply_string_storage_callback", - "apply_point_callback", - ) - ) - assert len({callback.binding.context_current_symbol for callback in callbacks}) == len(callbacks) - assert len({callback.bridge.adapter_symbol for callback in callbacks}) == len(callbacks) - assert len({callback.entrypoint.support_procedure.symbol_name for callback in callbacks}) == len(callbacks) - assert all( - callback.entrypoint.support_procedure.implementation_owner - is GeneratedSupportProcedureImplementationOwner.BINDING - for callback in callbacks - ) - assert all( - next( - procedure - for procedure in plan.entrypoint.support_procedures - if procedure.key == callback.entrypoint.support_procedure.key - ) - is callback.entrypoint.support_procedure - for callback in callbacks - ) - - @pytest.mark.parametrize( ("edit", "diagnostic"), ( @@ -197,31 +99,6 @@ def test_callback_plan_edits_fail_central_validation_before_backend_emission(edi WrapperGenerator().generate(plan) -def test_callback_artifacts_use_linear_context_adapter_and_trampoline_paths(): - c_source, bridge = _sources(_plan()) - - assert "static _Thread_local" in c_source - assert "PyThread_get_thread_ident()" in c_source - assert "PyGILState_Ensure()" in c_source - assert "PyGILState_Release(" in c_source - assert "PyErr_PrintEx(0);" in c_source - assert "abort();" in c_source - assert "Py_BEGIN_ALLOW_THREADS" not in c_source - assert "Py_END_ALLOW_THREADS" not in c_source - - assert "integer(c_int32_t), value :: value" in bridge - assert "integer(c_int32_t) :: count" in bridge - assert "procedure(prik_" in bridge - assert 'bind(c, name="prik_callback_trampoline' in bridge - assert "size(values_callback_storage, dim=1, kind=c_int64_t)" in bridge - assert "int(len(read_label_callback_storage), kind=c_int64_t)" in bridge - assert "prik_int32_to_numpy(&value)" in c_source - assert "prik_int32_to_numpy(count_data)" in c_source - assert bridge.count("call native_apply_array_storage_callback(") == 1 - assert "call callback(" not in bridge - assert max(map(len, bridge.splitlines())) <= 132 - - def test_nogil_callback_call_releases_outer_envelope_and_reacquires_in_trampoline(): source = CONTRACT.read_text(encoding="utf-8") source = source.replace("native_call, prototype", "native_call, nogil, prototype", 1) @@ -245,184 +122,6 @@ def test_nogil_callback_call_releases_outer_envelope_and_reacquires_in_trampolin assert "PyGILState_Release(" in c_source -def test_every_callback_uses_the_shared_generated_abstract_prototype(): - module = pyi_file_to_semantic_module(ARRAY_CONTRACT, module_name="fcallback_array_f90") - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - - reduce = _callback_argument(plan, "apply_reduce").callback - transform = _callback_argument(plan, "apply_transform").callback - assert reduce.prototype.interface_symbol.startswith("prik_reduce_callback_") - assert transform.prototype.interface_symbol.startswith("prik_transform_callback_") - - _, bridge = _sources(plan) - assert f"procedure({reduce.prototype.interface_symbol}) :: {reduce.bridge.adapter_symbol}" in bridge - assert f"procedure({transform.prototype.interface_symbol}) :: {transform.bridge.adapter_symbol}" in bridge - assert "abstract interface" in bridge - assert "=> transform_callback" not in bridge - - -def test_optional_callback_uses_the_ordinary_presence_plan(): - module = pyi_file_to_semantic_module(CONTRACT, module_name="fcallback_all_f90") - function = next(item for item in module.functions if item.name == "apply_value_callback") - function.arguments[0].optional = True - complete_semantic_policies(module) - - plan = WrapperPlanner().build(module) - argument = _callback_argument(plan, "apply_value_callback") - - assert argument.binding.optional_mode is OptionalMode.NULLABLE_VALUE - assert argument.entrypoint.optional_mode is OptionalMode.NULLABLE_VALUE - assert argument.entrypoint.pass_callback_parameter is True - - c_source, bridge = _sources(plan) - assert "bound_callback_obj != Py_None ? prik_callback_trampoline_" in c_source - assert "if (c_associated(callback)) then" in bridge - assert "procedure(prik_value_callback_" in bridge - assert "callback=prik_optional_callback" in bridge - assert bridge.count("result = native_apply_value_callback(") == 1 - - -def test_direct_bind_c_callback_generates_no_fortran_callback_adapter(): - source = """ -module direct_callback - use iso_c_binding - implicit none - - abstract interface - subroutine report(value) bind(C) - import c_int - integer(c_int), value, intent(in) :: value - end subroutine report - end interface - -contains - - subroutine run(callback) bind(C) - procedure(report) :: callback - call callback(4_c_int) - end subroutine run -end module direct_callback -""" - module = FortranToIRConverter().visit(parse_fortran_source(source).modules[0]) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - callback = _callback_argument(plan, "run").callback - - artifacts = WrapperGenerator().generate(plan) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - assert callback.entrypoint.support_procedure.symbol_name in c_source - assert all(source.path.suffix != ".f90" for source in artifacts.sources) - - -def test_runtime_callback_extents_lower_to_assumed_shape_dummies_and_measured_copies(): - """Codegen spells a runtime extent instead of leaking the plan's marker. - - A runtime extent reaches the bridge as a public marker rather than an - expression, so the dummy takes the caller's descriptor and the contiguous - copy that backs ``c_loc`` is measured from that dummy. - """ - module = pyi_file_to_semantic_module(ARRAY_CONTRACT, module_name="fcallback_array_f90") - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - - callback = _callback_argument(plan, "apply_assumed_shape").callback - assert [transfer.array.shape for transfer in callback.arguments] == [("::",), ("::",)] - - _, bridge = _sources(plan) - assert "real(c_double), intent(in), dimension(:) :: values" in bridge - assert "real(c_double), target, dimension(size(values, 1)) :: values_callback_storage" in bridge - assert "real(c_double), intent(out), dimension(:) :: doubled" in bridge - assert "real(c_double), target, dimension(size(doubled, 1)) :: doubled_callback_storage" in bridge - - -def test_rank_zero_callback_storage_lowers_to_a_direction_correct_native_view(): - """Rank-zero storage aliases native memory instead of copying a value. - - Writeability follows the completed transfer direction, so only an ``out`` - or ``inout`` dummy can be written through. - """ - module = pyi_text_to_semantic_module( - """ -from prik.contracts import Float64, In, InOut, Out, prototype - -@prototype -def directions_callback( - read_value: In(Float64[()]), - update_value: InOut(Float64[()]), - write_value: Out(Float64[()]) -) -> None: ... - -def apply_directions(callback: directions_callback) -> None: ... -""", - module_name="callback_scalar_storage", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - - callback = _callback_argument(plan, "apply_directions").callback - assert [transfer.python_action for transfer in callback.arguments] == [PythonBarrierAction.SCALAR_STORAGE] * 3 - assert [transfer.abi for transfer in callback.arguments] == [CallbackABIKind.REFERENCE] * 3 - - c_source, _bridge = _sources(plan) - read_only = "PyArray_New(&PyArray_Type, 0, NULL, NPY_FLOAT64, NULL, read_value_data, 0, " - assert f"{read_only}NPY_ARRAY_F_CONTIGUOUS | NPY_ARRAY_ALIGNED, NULL)" in c_source - for parameter in ("update_value", "write_value"): - writable = f"PyArray_New(&PyArray_Type, 0, NULL, NPY_FLOAT64, NULL, {parameter}_data, 0, " - assert f"{writable}NPY_ARRAY_F_CONTIGUOUS | NPY_ARRAY_ALIGNED | NPY_ARRAY_WRITEABLE, NULL)" in c_source - - -MATRIX_CONTRACT = """ -from prik.contracts import Float64, In, Out, prototype - -@prototype -def matrix_callback( - input: In(Float64[::, ::]), - output: Out(Float64[::, ::]) -) -> None: ... - -def apply_matrix(callback: matrix_callback) -> None: ... -""" - - -def _matrix_plan(): - module = pyi_text_to_semantic_module(MATRIX_CONTRACT, module_name="callback_matrix") - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def test_multidimensional_runtime_extents_measure_every_axis_from_the_dummy(): - """Each axis of an assumed-shape callback array is lowered independently. - - A rank-one fix can silently ignore later axes, so the copy that backs - ``c_loc`` must be measured on every axis of the dummy it sits beside. - """ - plan = _matrix_plan() - callback = _callback_argument(plan, "apply_matrix").callback - assert [transfer.array.rank for transfer in callback.arguments] == [2, 2] - - _, bridge = _sources(plan) - assert "real(c_double), intent(in), dimension(:, :) :: input" in bridge - assert "real(c_double), target, dimension(size(input, 1), size(input, 2)) :: input_callback_storage" in bridge - assert "real(c_double), intent(out), dimension(:, :) :: output" in bridge - assert "real(c_double), target, dimension(size(output, 1), size(output, 2)) :: output_callback_storage" in bridge - - -def test_callback_docstrings_carry_array_rank_and_public_extents(): - """A callable's ABI depends on rank and shape, so both are documented. - - Extents use the spelling the `.pyi` contract uses, so the two descriptions - of the same array agree and no internal marker reaches the reader. - """ - plan = _matrix_plan() - c_source, _bridge = _sources(plan) - documentation = c_source.encode().decode("unicode_escape") - - assert "Called as: callback(input, output) -> None" in documentation - assert "input : ndarray[float64], rank 2, shape (::, ::), intent(in)" in documentation - assert "output : ndarray[float64], rank 2, shape (::, ::), intent(out)" in documentation - - def test_callback_array_result_diagnostic_uses_the_contract_spelling(): """A rejected shape is reported the way a contract would spell it. diff --git a/tests/fortran/callbacks/end_to_end/test_array_callbacks.py b/tests/fortran/callbacks/end_to_end/test_array_callbacks.py index a0f5d81ce..ba3bcfa0c 100644 --- a/tests/fortran/callbacks/end_to_end/test_array_callbacks.py +++ b/tests/fortran/callbacks/end_to_end/test_array_callbacks.py @@ -22,6 +22,11 @@ def test_immediate_dummy_procedure_converts_array_arguments_and_results( pyi_parity_build_mode: str, tmp_path: Path, ): + """Explicit-shape and assumed-shape callback arrays cross in both directions. + + An assumed-shape callback dummy carries its extent from the native + descriptor and reaches Python as a contiguous copy. + """ module = _build_source_or_generated_pyi_and_import( CALLBACK_ARRAY_F90_SOURCE, tmp_path, @@ -46,23 +51,6 @@ def test_immediate_dummy_procedure_converts_array_arguments_and_results( assert result is None np.testing.assert_array_equal(transformed, np.array([2.0, 4.0, 6.0], dtype=np.float64)) - -def test_assumed_shape_callback_arrays_cross_the_boundary_as_contiguous_copies( - pyi_parity_build_mode: str, - tmp_path: Path, -): - """An assumed-shape callback dummy carries its extent from the native descriptor.""" - module = _build_source_or_generated_pyi_and_import( - CALLBACK_ARRAY_F90_SOURCE, - tmp_path, - { - "bind_c_fcallback_array_f90_wrapper.f90", - "fcallback_array_f90_wrapper.c", - "fcallback_array_f90_wrapper.h", - }, - CONTRACT_FIXTURES / "fcallback_array_f90", - pyi_parity_build_mode, - ) values = np.asfortranarray(np.array([1.5, 2.5, 3.5, 4.5], dtype=np.float64)) doubled = np.zeros(4, dtype=np.float64) seen = [] diff --git a/tests/fortran/callbacks/end_to_end/test_callback_scalar_storage.py b/tests/fortran/callbacks/end_to_end/test_callback_scalar_storage.py index f3e177034..7a5207207 100644 --- a/tests/fortran/callbacks/end_to_end/test_callback_scalar_storage.py +++ b/tests/fortran/callbacks/end_to_end/test_callback_scalar_storage.py @@ -76,12 +76,15 @@ def callback(read_value, update_value, write_value): SOURCE_DEFAULT = NATIVE_FIXTURES / "fcallback_default_storage_f90.f90" -def test_out_scalar_callback_writes_back_without_editing_the_contract(tmp_path: Path): +def test_out_scalar_callback_writes_back_and_documents_the_callable_without_editing_the_contract(tmp_path: Path): """Wrapping Fortran source directly produces a callback that can answer. The generated default must be the spelling that works: an `intent(out)` scalar reaches Python as writable storage, so the value the callable - computes reaches the native caller with no contract edit. + computes reaches the native caller with no contract edit. The docstring is + the only callback description in the source-only workflow, and guessing a + callback signature wrong is fatal at the callback boundary, so `help()` + must state the arity, direction, and how an output is delivered. """ module = _build_source_and_import( SOURCE_DEFAULT, @@ -101,24 +104,7 @@ def objective_prik(x, f): assert module.evaluate(objective_prik, np.array([1.0, 2.0, 3.0])) == np.float64(14.0) - -def test_callback_docstring_states_the_callable_signature_and_write_through(tmp_path: Path): - """The docstring is the only callback description in the source-only workflow. - - Guessing a callback signature wrong is fatal at the callback boundary, so - `help()` must state the arity, direction, and how an output is delivered. - """ - module = _build_source_and_import( - SOURCE_DEFAULT, - tmp_path / "build", - { - "bind_c_fcallback_default_storage_f90_wrapper.f90", - "fcallback_default_storage_f90_wrapper.c", - "fcallback_default_storage_f90_wrapper.h", - }, - ) documentation = module.evaluate.__doc__ - assert "Called as: calfun(x, f) -> None" in documentation assert "x : ndarray[float64], rank 1, shape (::), intent(in)" in documentation assert "f : ndarray[float64], intent(out); assign through it (f[...] = value)" in documentation @@ -140,33 +126,18 @@ def _undeclared_intent_module(tmp_path: Path): ) -def test_callback_scalar_without_declared_intent_is_read_and_written(tmp_path: Path): +def test_undeclared_intent_is_read_and_written_from_source_and_its_generated_contract(tmp_path: Path): """An undeclared ``intent`` is conservatively both read and written. Fortran permits the callee to modify such a dummy, so the callable must observe the incoming value and see its own write reach the native caller. + The absence must survive source, contract, codegen and runtime: building + through PRIK's own generated contract proves the bare ``Float64[()]`` + spelling carries the conservative read/write transfer to the trampoline. """ - module = _undeclared_intent_module(tmp_path) - observed = [] - - def tweak(value): - observed.append(float(value)) - assert value.flags.writeable - value[...] = float(value) * 3.0 - - assert module.drive(tweak, np.float64(7.0)) == np.float64(21.0) - assert observed == [7.0] - - -def test_undeclared_intent_survives_the_generated_contract_round_trip(tmp_path: Path): - """The absent ``intent`` must survive source, contract, codegen and runtime. - - Building through PRIK's own generated contract proves the bare - ``Float64[()]`` spelling carries the conservative read/write transfer all - the way to the trampoline, rather than only appearing in the contract text. - """ + source_module = _undeclared_intent_module(tmp_path) workdir = tmp_path / "round_trip" - module = _build_generated_pyi_and_import(SOURCE_UNDECLARED, workdir) + contract_module = _build_generated_pyi_and_import(SOURCE_UNDECLARED, workdir) contract = (workdir / "contracts" / SOURCE_UNDECLARED.stem / f"{SOURCE_UNDECLARED.stem}.pyi").read_text( encoding="utf-8" @@ -179,10 +150,16 @@ def test_undeclared_intent_survives_the_generated_contract_round_trip(tmp_path: assert not any(f"intent({direction}) :: value" in bridge for direction in ("in", "out", "inout")) assert "value = value_callback_storage" in bridge + observed = [] + def tweak(value): + observed.append((float(value), bool(value.flags.writeable))) value[...] = float(value) * 3.0 - assert module.drive(tweak, np.float64(7.0)) == np.float64(21.0) + for module in (source_module, contract_module): + observed.clear() + assert module.drive(tweak, np.float64(7.0)) == np.float64(21.0) + assert observed == [(7.0, True)] def test_assume_intent_in_scalars_makes_an_undeclared_callback_scalar_input_only(tmp_path: Path): diff --git a/tests/fortran/callbacks/end_to_end/test_direct_bind_c_callback_storage.py b/tests/fortran/callbacks/end_to_end/test_direct_bind_c_callback_storage.py index 7451b3e1a..fa0d27b85 100644 --- a/tests/fortran/callbacks/end_to_end/test_direct_bind_c_callback_storage.py +++ b/tests/fortran/callbacks/end_to_end/test_direct_bind_c_callback_storage.py @@ -31,7 +31,8 @@ def test_direct_bind_c_callbacks_receive_writable_rank_zero_storage(tmp_path: Pa """The projection must work where no Fortran bridge exists at all. A direct entry point calls the trampoline as a plain C function pointer, so - writable storage has to be the binding's doing rather than an adapter's. + writable storage has to be the binding's doing rather than an adapter's, + and scalar callback storage must not drag a bridge onto the direct route. """ module = _direct_module(tmp_path) observed = {} @@ -48,11 +49,4 @@ def emit(value): assert module.drive_update(update, np.float64(5.0)) == np.float64(10.0) assert module.drive_emit(emit) == np.float64(42.0) assert observed == {"writeable": True, "incoming": 5.0, "emit_writeable": True} - - -def test_direct_bind_c_callback_storage_adds_no_fortran_bridge(tmp_path: Path): - """Scalar callback storage must not drag a bridge onto the direct route.""" - _direct_module(tmp_path) - generated = {path.name for path in (tmp_path / "build").glob("*_wrapper.f90")} - - assert generated == set() + assert not list((tmp_path / "build").glob("*_wrapper.f90")) diff --git a/tests/fortran/callbacks/end_to_end/test_multi_file_contract_generation.py b/tests/fortran/callbacks/end_to_end/test_multi_file_contract_generation.py index c8617b357..3dba49edb 100644 --- a/tests/fortran/callbacks/end_to_end/test_multi_file_contract_generation.py +++ b/tests/fortran/callbacks/end_to_end/test_multi_file_contract_generation.py @@ -7,7 +7,7 @@ import numpy as np import pytest -from tests.fortran._support.wrapper_build import _compiler, _import_from_build_dir +from tests.fortran._support.wrapper_build import _build_source_and_import, _compiler, _import_from_build_dir from prik import build_pyi_extension from prik.pipeline.pyi import pyi_text_to_semantic_module from prik.semantics.native_contract import native_contract_issues @@ -45,14 +45,15 @@ def _generate_contracts(tmp_path: Path) -> tuple[Path, list[Path]]: return contracts, sources -def test_multi_file_generation_places_the_prototype_with_its_declaring_module(tmp_path: Path): +def test_multi_file_generation_places_each_prototype_with_its_declaring_module_and_builds(tmp_path: Path): """Each module's contract records what that module declares or imports. The per-file CLI conversion path is where an imported interface previously degraded to an opaque placeholder, so this exercises that workflow rather - than whole-project conversion. + than whole-project conversion. PRIK must read back every contract it just + wrote, and the whole route must survive: source, contract, parse, build, call. """ - contracts, _sources = _generate_contracts(tmp_path) + contracts, sources = _generate_contracts(tmp_path) declaring = (contracts / "pintrf_mod.pyi").read_text(encoding="utf-8") assert "@prototype\ndef OBJ(" in declaring @@ -72,21 +73,12 @@ def test_multi_file_generation_places_the_prototype_with_its_declaring_module(tm assert "calfun: SCOPED_OBJ" in scoped assert "import SCOPED_OBJ" not in scoped.replace("OBJ as SCOPED_OBJ", "") - -def test_generated_multi_file_contracts_parse_without_native_contract_issues(tmp_path: Path): - """PRIK must be able to read back every contract it just wrote.""" - contracts, _sources = _generate_contracts(tmp_path) - for contract in sorted(contracts.glob("*.pyi")): if contract.name == "__init__.pyi": continue module = pyi_text_to_semantic_module(contract.read_text(encoding="utf-8"), module_name=contract.stem) assert native_contract_issues(module) == [], contract.name - -def test_building_from_generated_multi_file_contracts_runs_the_callback(tmp_path: Path): - """The whole route must survive: source, contract, parse, build, call.""" - contracts, sources = _generate_contracts(tmp_path) result = build_pyi_extension( contracts / "__init__.pyi", input_compiler=_compiler(), @@ -157,12 +149,16 @@ def test_imported_callback_returning_a_module_owned_type_builds(tmp_path: Path): RENAMED_CHAIN_SOURCE = NATIVE_FIXTURES / "chain.f90" -def test_renamed_reexport_chain_builds_through_its_generated_contracts(tmp_path: Path): +def test_renamed_reexport_chain_builds_from_its_generated_contracts_and_its_source(tmp_path: Path): """Each hop renames the interface, so only the declaring module names it. A rename and a re-export are covered separately elsewhere; combining them is what exposes a reference that followed the module back to the declaration while keeping an alias from somewhere along the way. + + Built directly from source, publishing an imported interface adds no + runtime name to alias: a signature is not an object Python holds, so the + chain reaches the build through prototype resolution alone. """ contracts = tmp_path / "contracts" subprocess.run( @@ -205,20 +201,10 @@ def objective(x, f): assert module.chain_consumer_mod.run_chain(objective, np.float64(6.0)) == np.float64(42.0) - -def test_renamed_reexport_chain_builds_directly_from_its_fortran_source(tmp_path: Path): - """Publishing an imported interface adds no runtime name to alias. - - A module publishing an imported prototype states where a callback signature - comes from, and a signature is not an object Python holds. Binding one at - runtime reaches for an attribute of a module that exports nothing at all, - so the chain has to reach the build through prototype resolution alone. - """ - from tests.fortran._support.wrapper_build import _build_source_and_import - + # The same chain built directly from its Fortran source. module = _build_source_and_import( RENAMED_CHAIN_SOURCE, - tmp_path / "build", + tmp_path / "source_build", {"bind_c_chain_wrapper.f90", "chain_wrapper.c", "chain_wrapper.h"}, ) diff --git a/tests/fortran/callbacks/end_to_end/test_optional_callbacks.py b/tests/fortran/callbacks/end_to_end/test_optional_callbacks.py index ae03dd8b5..c02b17f02 100644 --- a/tests/fortran/callbacks/end_to_end/test_optional_callbacks.py +++ b/tests/fortran/callbacks/end_to_end/test_optional_callbacks.py @@ -42,7 +42,11 @@ def optional_callback_module(pyi_parity_build_mode: str, tmp_path: Path): def test_optional_callback_and_optional_dummies_preserve_each_presence_state(optional_callback_module): - module, _build_dir, _build_mode = optional_callback_module + """Each presence state crosses the bridge and the direct `bind(C)` route alike. + + Exception propagation is unchanged for a supplied optional callback. + """ + module, build_dir, build_mode = optional_callback_module assert module.run(np.int32(0)) == np.int32(-1) assert module.run(np.int32(0), None) == np.int32(-1) @@ -70,10 +74,7 @@ def report(value, status, values, terminate): assert isinstance(observed[-1][3], np.ndarray) assert observed[-1][3].shape == () - -def test_optional_bind_c_callback_remains_direct_and_preserves_inner_presence(optional_callback_module): - module, build_dir, build_mode = optional_callback_module - + # A bind(C) optional callback stays on the direct route. assert module.direct_run(np.int32(0)) == np.int32(-1) assert module.direct_run(np.int32(0), None) == np.int32(-1) seen = [] @@ -88,9 +89,6 @@ def test_optional_bind_c_callback_remains_direct_and_preserves_inner_presence(op assert "direct_run(bound_mode, bound_callback_obj != Py_None ? prik_callback_trampoline_" in binding assert "function bind_c_direct_run" not in bridge.casefold() - -def test_exception_propagation_is_unchanged_for_a_supplied_optional_callback(optional_callback_module): - _module, build_dir, _build_mode = optional_callback_module result = subprocess.run( [ sys.executable, diff --git a/tests/fortran/callbacks/pipeline/test_generated_callback_contracts.py b/tests/fortran/callbacks/pipeline/test_generated_callback_contracts.py deleted file mode 100644 index cd2b4f45f..000000000 --- a/tests/fortran/callbacks/pipeline/test_generated_callback_contracts.py +++ /dev/null @@ -1,28 +0,0 @@ -"""Generated `.pyi` package fixtures for callback wrapper inputs.""" - -from __future__ import annotations - -from pathlib import Path - -import pytest - -from tests.fortran._support.generated_contracts import ( - GeneratedContractCase, - assert_generated_contract_matches_fixture, - contract_case_id, -) - -FIXTURES = Path(__file__).parents[1] / "end_to_end" / "fixtures" -CASES = tuple( - GeneratedContractCase( - name=name, - inputs=(FIXTURES / "native" / f"{name}.f90",), - expected_package=FIXTURES / "contracts" / name, - ) - for name in ("fcallback_all_f90", "fcallback_array_f90", "fcallback_scalar_f90") -) - - -@pytest.mark.parametrize("case", CASES, ids=contract_case_id) -def test_callback_generated_pyi_contract_matches_fixture(case: GeneratedContractCase, tmp_path: Path): - assert_generated_contract_matches_fixture(case, tmp_path) diff --git a/tests/fortran/callbacks/policy/test_callback_policy.py b/tests/fortran/callbacks/policy/test_callback_policy.py index 9bc2f209d..8f61e7af8 100644 --- a/tests/fortran/callbacks/policy/test_callback_policy.py +++ b/tests/fortran/callbacks/policy/test_callback_policy.py @@ -1,11 +1,8 @@ -from pathlib import Path - import pytest from tests.fortran._support.ownership_policy import parse_pyi_text from prik.parsers.fortran.parser import parse_fortran_project from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.semantics.models import ( RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, @@ -13,42 +10,8 @@ from prik.policy.completion import complete_semantic_policies from prik.policy.ownership import PythonBarrierAction from prik.policy.models import ( - CallbackABIKind, - CallbackOptionalityAction, - CallbackTransferAction, FunctionWrapperPolicy, ) -from prik.policy.construction import completed_function_wrapper_policy - -FIXTURES = Path(__file__).parents[1] / "end_to_end" / "fixtures" - - -def _source_semantic_module(filename: str, *, module_name: str, assume_intent_in_scalars: bool = False): - source = FIXTURES / "native" / filename - parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) - modules = fortran_project_to_semantic_modules(parsed, assume_intent_in_scalars=assume_intent_in_scalars) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name=module_name) - complete_semantic_policies(module) - return module - - -def test_source_callback_value_default_and_explicit_reference_are_completed(): - module = _source_semantic_module("fcallback_all_f90.f90", module_name="fcallback_all_f90") - function = next(item for item in module.functions if item.name == "apply_value_callback") - policy = completed_function_wrapper_policy(function) - transfer = policy.arguments[0].callback.arguments[0] - - assert transfer.abi is CallbackABIKind.VALUE - assert transfer.passed_by_value is True - assert transfer.adapter_action is CallbackTransferAction.COPY_IN - - array_function = next(item for item in module.functions if item.name == "apply_array_storage_callback") - array_policy = completed_function_wrapper_policy(array_function) - extent = array_policy.arguments[0].callback.arguments[0] - assert extent.abi is CallbackABIKind.REFERENCE - assert extent.passed_by_value is False - assert extent.adapter_action is CallbackTransferAction.COPY_IN @pytest.mark.parametrize( @@ -120,49 +83,6 @@ def test_procedure_interface_from_an_unsupplied_module_is_blocked_by_name(): ) -def test_writable_callback_scalars_use_rank_zero_storage_without_synthesizing_intent(): - """Every dummy the callee may write is projected as writable storage. - - Python has no writable scalar, so a dummy the native caller reads back must - reach the callable as rank-zero storage. An undeclared ``intent`` is - conservatively writable because Fortran permits the callee to modify it, - and the declaration keeps no intent of its own either way. - """ - module = _source_semantic_module("fcallback_all_f90.f90", module_name="fcallback_all_f90") - function = next(item for item in module.functions if item.name == "apply_scalar_storage_callback") - policy = completed_function_wrapper_policy(function) - transfers = policy.arguments[0].callback.arguments - - assert [transfer.intent for transfer in transfers] == ["inout", "out", None] - assert [transfer.python_action for transfer in transfers] == [PythonBarrierAction.SCALAR_STORAGE] * 3 - assert [transfer.adapter_action for transfer in transfers] == [ - CallbackTransferAction.COPY_IN_OUT, - CallbackTransferAction.COPY_OUT, - CallbackTransferAction.COPY_IN_OUT, - ] - assert policy.supported is True - - -def test_assume_intent_in_scalars_elects_the_input_only_default_for_an_undeclared_intent(): - """The flag chooses which default an undeclared ``intent`` receives. - - It narrows the conservative read/write default to input-only; it does not - give the dummy a declared direction, so the contract still carries none. - """ - module = _source_semantic_module( - "fcallback_all_f90.f90", - module_name="fcallback_all_f90", - assume_intent_in_scalars=True, - ) - function = next(item for item in module.functions if item.name == "apply_scalar_storage_callback") - transfers = completed_function_wrapper_policy(function).arguments[0].callback.arguments - - undeclared = transfers[2] - assert undeclared.intent is None - assert undeclared.python_action is PythonBarrierAction.SCALAR_VALUE - assert undeclared.adapter_action is CallbackTransferAction.COPY_IN - - @pytest.mark.parametrize( ("prototype", "blocker"), [ @@ -214,114 +134,3 @@ def apply(callback: callback_shape) -> None: ... policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] assert policy.supported is True assert policy.arguments[0].callback.arguments[0].python_action is PythonBarrierAction.SCALAR_VALUE - - -def test_optional_reference_callback_dummy_has_one_null_pointer_presence_decision(): - module = parse_pyi_text( - """ -@prototype -def callback_shape(value: In(Addr(Int32)) = ...) -> None: ... - -def apply(callback: callback_shape = ...) -> None: ... -""", - module_name="optional_callback", - ) - - complete_semantic_policies(module) - policy = completed_function_wrapper_policy(module.functions[0]) - - argument = policy.arguments[0] - assert argument.optional is True - assert argument.callback.arguments[0].optionality is CallbackOptionalityAction.NULL_DATA_POINTER - assert argument.callback.prototype.arguments[0].optional is True - - -def test_imported_interface_keeps_its_declaring_module_in_the_completed_identity(): - """A type an imported interface owns must not be attributed to the consumer. - - The consuming module never imports ``point_t``, so an identity taken from - the consuming scope names a type that module does not define and no wrapper - definition can satisfy it. - """ - sources = { - "callback_types.f90": """ -module callback_types - implicit none - type :: point_t - real(8) :: x - end type point_t - - abstract interface - subroutine move_point(p) - import :: point_t - implicit none - type(point_t), intent(inout) :: p - end subroutine move_point - end interface -end module callback_types -""", - "consumer.f90": """ -module consumer - use callback_types, only : move_point - implicit none -contains - subroutine run(f) - procedure(move_point) :: f - end subroutine run -end module consumer -""", - } - parsed = parse_fortran_project(sources) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name="merged") - complete_semantic_policies(module) - - function = next(item for item in module.functions if item.name == "run") - policy = completed_function_wrapper_policy(function) - - assert policy.supported is True - assert policy.arguments[0].callback.arguments[0].derived_type_identity == ("callback_types", "point_t") - - -def test_imported_interface_result_keeps_its_declaring_module_in_the_completed_identity(): - """A callback result's type identity must name the module that declares it.""" - sources = { - "callback_types.f90": """ -module callback_types - implicit none - type :: point_t - real(8) :: x - end type point_t - - abstract interface - function make_point(x) result(p) - import :: point_t - implicit none - real(8), intent(in) :: x - type(point_t) :: p - end function make_point - end interface -end module callback_types -""", - "consumer.f90": """ -module consumer - use callback_types, only : make_point - implicit none -contains - subroutine run(f) - procedure(make_point) :: f - end subroutine run -end module consumer -""", - } - parsed = parse_fortran_project(sources) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name="merged") - complete_semantic_policies(module) - - policy = completed_function_wrapper_policy(next(item for item in module.functions if item.name == "run")) - - assert policy.supported is True - assert policy.arguments[0].callback.result.transfer.derived_type_identity == ("callback_types", "point_t") diff --git a/tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py b/tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py index 017fb3e05..eb9ce3e08 100644 --- a/tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py +++ b/tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py @@ -58,53 +58,6 @@ def register( assert callback_arguments[8].semantic_type.storage.mutable is True -@pytest.mark.parametrize( - "annotation", - [ - "String[8]", - "String[8][()]", - ], -) -def test_callback_string_storage_contracts_complete(annotation: str): - module = parse_pyi_text( - f""" -@prototype -def string_callback(value: {annotation}) -> None: ... - -def register(callback: string_callback) -> None: ... -""", - module_name="callbacks", - ) - - complete_semantic_policies(module) - callback_type = module.functions[0].arguments[0].semantic_type - callback_argument = callback_type.metadata["callback_arguments"][0] - assert callback_argument.semantic_type.name == "String" - - -def test_convert_pyi_to_ir_preserves_prototype_argument_names_and_dimensions(): - module = parse_pyi_text( - """ -@prototype -def transform_callback( - count: Int32, - values: Float64[count], -) -> Float64[count]: ... - -def apply_transform( - callback: transform_callback -) -> None: ... -""", - module_name="callbacks", - ) - - callback_type = module.functions[0].arguments[0].semantic_type - callback_arguments = callback_type.metadata["callback_arguments"] - assert [arg.name for arg in callback_arguments] == ["count", "values"] - assert callback_type.metadata["return"].shape == ["count"] - assert callback_type.metadata["prototype_ref"]["name"] == "transform_callback" - - def test_prototype_is_one_exact_nonexported_signature_declaration(): module = parse_pyi_text( """ @@ -219,7 +172,6 @@ def test_convert_pyi_to_ir_rejects_invalid_prototype_address_wrappers(annotation "annotation", [ "Value(Float64)", - "Value(Int32)", "Value(String[8])", "Value(Allocatable[Float64])", "Value(Pointer[Float64])", diff --git a/tests/fortran/data_types/codegen/test_default_logical_scalar_lowering.py b/tests/fortran/data_types/codegen/test_default_logical_scalar_lowering.py deleted file mode 100644 index 412d380e1..000000000 --- a/tests/fortran/data_types/codegen/test_default_logical_scalar_lowering.py +++ /dev/null @@ -1,57 +0,0 @@ -"""Default-logical scalar kind adaptation through completed wrapper plans.""" - -from prik.parsers.fortran import parse_fortran_file -from prik.semantics.fortran2ir import fortran_module_to_semantic_module -from prik.policy.completion import complete_semantic_policies -from prik.policy.models import BridgeDataAction, ScalarLogicalABI -from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import WrapperPlanner - - -SOURCE = """ -module logical_args -contains -subroutine use_flags(input, output) - logical, intent(in) :: input - logical, intent(out) :: output - output = .not. input -end subroutine use_flags -end module logical_args -""" - - -def _logical_function_plan(): - parsed_module = parse_fortran_file(SOURCE).modules[0] - semantic_module = fortran_module_to_semantic_module(parsed_module) - complete_semantic_policies(semantic_module) - module_plan = WrapperPlanner().build(semantic_module) - return module_plan, module_plan.namespaces[0].functions[0] - - -def test_policy_completes_default_logical_input_and_output_kind_copies(): - _module_plan, function = _logical_function_plan() - input_plan = function.arguments[0] - output_slot = function.results[0].projected_call_slot - - assert input_plan.scalar_logical_abi is ScalarLogicalABI.NATIVE_KIND_COPY - assert input_plan.scalar_native_type == "logical" - assert input_plan.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - assert output_slot.scalar_logical_abi is ScalarLogicalABI.NATIVE_KIND_COPY - assert output_slot.scalar_native_type == "logical" - assert output_slot.adapter.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - - -def test_bridge_mechanically_lowers_completed_default_logical_kind_copies(): - module_plan, _function = _logical_function_plan() - - bridge_source = next( - source.text for source in WrapperGenerator().generate(module_plan).sources if source.path.suffix == ".f90" - ) - - assert "logical(c_bool), value :: input" in bridge_source - assert "logical :: input_native" in bridge_source - assert "input_native = input" in bridge_source - assert "logical(c_bool) :: output" in bridge_source - assert "logical :: output_value" in bridge_source - assert "call native_use_flags(input_native, output_value)" in bridge_source - assert "output = output_value" in bridge_source diff --git a/tests/fortran/data_types/codegen/test_logical_storage_boundary.py b/tests/fortran/data_types/codegen/test_logical_storage_boundary.py deleted file mode 100644 index 508f425c9..000000000 --- a/tests/fortran/data_types/codegen/test_logical_storage_boundary.py +++ /dev/null @@ -1,32 +0,0 @@ -"""Native-width logical storage stays interoperable at the generated bind(C) boundary.""" - -from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import WrapperPlanner -from prik.policy.completion import complete_semantic_policies - - -def test_wider_logical_crosses_bind_c_as_an_address_or_same_width_integer(): - """The logical kind exists only behind the boundary, for reference and value transport alike.""" - module = parse_pyi_text( - """from prik.contracts import Bool32, Int32, Returns - -def flip(flag: Bool32) -> Returns["flag", Bool32]: ... - -def count_true(flag: Bool32) -> Int32: ... -""", - module_name="logical_boundary", - ) - complete_semantic_policies(module) - - bridge = next( - source.text - for source in WrapperGenerator().generate(WrapperPlanner().build(module)).sources - if source.path.suffix == ".f90" - ) - - assert "type(c_ptr), value :: bound_flag" in bridge - assert "call c_f_pointer(bound_flag, flag)" in bridge - assert "integer(c_int32_t), value :: bound_flag" in bridge - assert "flag = transfer(bound_flag, flag)" in bridge - assert "logical(kind=4), pointer :: flag" in bridge diff --git a/tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py b/tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py index 6ddc49e61..28f6a2786 100644 --- a/tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py +++ b/tests/fortran/data_types/codegen/test_primitive_scalar_input_lowering.py @@ -2,44 +2,12 @@ from __future__ import annotations -import pytest - from tests.fortran._support.ownership_policy import parse_pyi_text from prik.policy.completion import complete_semantic_policies from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -@pytest.mark.parametrize( - ("type_name", "c_type", "helper_suffix"), - [ - ("Bool", "bool", "NPY_BOOL"), - ("Int8", "int8_t", "NPY_INT8"), - ("Int16", "int16_t", "NPY_INT16"), - ("Int32", "int32_t", "NPY_INT32"), - ("Int64", "int64_t", "NPY_INT64"), - ("Float32", "float", "NPY_FLOAT32"), - ("Float64", "double", "NPY_FLOAT64"), - ("Complex64", "float complex", "NPY_COMPLEX64"), - ("Complex128", "double complex", "NPY_COMPLEX128"), - ], -) -def test_scalar_input_registry_lowers_completed_type_into_the_native_support_api(type_name, c_type, helper_suffix): - module = parse_pyi_text(f"def identity(x: {type_name}) -> {type_name}: ...", module_name="scalar_input") - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - - artifacts = WrapperGenerator().generate(plan) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - - assert f"{c_type} bound_x;" in c_source - # The value, or rank-zero storage of the same dtype, crosses in one call - # to the type's native-support helper. - suffix = helper_suffix.casefold().removeprefix("npy_") - assert f"if (prik_{suffix}_or_storage(bound_x_obj, {helper_suffix}, " in c_source - assert "&bound_x, " in c_source - - def test_binding_locals_are_isolated_from_identifiers_imported_by_c_headers(): module = parse_pyi_text("def identity(complex: Float64) -> Float64: ...", module_name="header_names") complete_semantic_policies(module) diff --git a/tests/fortran/data_types/codegen/test_primitive_scalar_result_lowering.py b/tests/fortran/data_types/codegen/test_primitive_scalar_result_lowering.py index 13e86542d..8d8b7d7ba 100644 --- a/tests/fortran/data_types/codegen/test_primitive_scalar_result_lowering.py +++ b/tests/fortran/data_types/codegen/test_primitive_scalar_result_lowering.py @@ -2,8 +2,6 @@ from __future__ import annotations -import pytest - from tests.fortran._support.ownership_policy import parse_pyi_text from prik.policy.completion import complete_semantic_policies from prik.policy.models import DirectResultABI @@ -11,35 +9,6 @@ from prik.planning import WrapperPlanner -@pytest.mark.parametrize( - ("type_name", "numpy_type", "result_kind"), - [ - ("Bool", "NPY_BOOL", "python"), - ("Int8", "NPY_INT8", "numpy"), - ("Int16", "NPY_INT16", "numpy"), - ("Int32", "NPY_INT32", "numpy"), - ("Int64", "NPY_INT64", "numpy"), - ("Float32", "NPY_FLOAT32", "numpy"), - ("Float64", "NPY_FLOAT64", "numpy"), - ("Complex64", "NPY_COMPLEX64", "numpy"), - ("Complex128", "NPY_COMPLEX128", "numpy"), - ], -) -def test_direct_scalar_results_preserve_numpy_types_with_python_bool_as_the_exception( - type_name, - numpy_type, - result_kind, -): - module = parse_pyi_text(f"def identity(x: {type_name}) -> {type_name}: ...", module_name="scalar_result") - complete_semantic_policies(module) - artifacts = WrapperGenerator().generate(WrapperPlanner().build(module)) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - - helper_suffix = numpy_type.casefold().removeprefix("npy_") - assert f"PyObject * result_obj = prik_{helper_suffix}_to_{result_kind}(&result);" in c_source - assert "return result_obj;" in c_source - - def test_direct_bool_result_normalizes_the_fortran_truth_bit_before_c_conversion(): module = parse_pyi_text( "def not_flag(value: Bool) -> Bool: ...", @@ -62,16 +31,3 @@ def test_direct_bool_result_normalizes_the_fortran_truth_bit_before_c_conversion assert "c_result = native_not_flag(value)" in fortran_source # Reduced the way C converts to `_Bool`: any non-zero value is true. assert "result = merge(1_c_int8_t, 0_c_int8_t, transfer(c_result, 0_c_int8_t) /= 0_c_int8_t)" in fortran_source - - -def test_generator_rejects_a_non_normalized_direct_bool_result_abi(): - module = parse_pyi_text( - "def not_flag(value: Bool) -> Bool: ...", - module_name="logical_result", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - plan.namespaces[0].functions[0].results[0].entrypoint.direct_result_abi = DirectResultABI.NATIVE_SCALAR - - with pytest.raises(ValueError, match="invalid-direct-result-abi"): - WrapperGenerator().generate(plan) diff --git a/tests/fortran/data_types/codegen/test_primitive_scalar_type_catalogue.py b/tests/fortran/data_types/codegen/test_primitive_scalar_type_catalogue.py index b79e5ea57..91899872a 100644 --- a/tests/fortran/data_types/codegen/test_primitive_scalar_type_catalogue.py +++ b/tests/fortran/data_types/codegen/test_primitive_scalar_type_catalogue.py @@ -25,17 +25,6 @@ def test_numpy_projection_rejects_unresolved_and_unknown_semantic_dtypes(): NumpyDtypeRegistry.expression_for("Int") -def test_backend_catalogue_makes_each_emitted_representation_explicit(): - scalar = PrimitiveScalarTypeRegistry.type_for("Float64") - - assert scalar.semantic_name == "Float64" - assert scalar.c_spelling == "double" - assert scalar.fortran_spelling == "real(c_double)" - assert scalar.numpy_type_macro == "NPY_FLOAT64" - assert scalar.python_type_name == NumpyDtypeRegistry.TYPES["Float64"] - assert scalar.cfi_type_spelling == "CFI_type_double" - - def test_backend_catalogue_returns_detached_records(): scalar = PrimitiveScalarTypeRegistry.type_for("Int32") scalar.c_spelling = "changed" diff --git a/tests/fortran/data_types/codegen/test_scalar_boundary_plan.py b/tests/fortran/data_types/codegen/test_scalar_boundary_plan.py deleted file mode 100644 index 48fc35d86..000000000 --- a/tests/fortran/data_types/codegen/test_scalar_boundary_plan.py +++ /dev/null @@ -1,76 +0,0 @@ -"""Isolated compiled parity for primitive scalar boundary representations.""" - -from __future__ import annotations - -from pathlib import Path - -import numpy as np - -from tests.fortran._support.wrapper_build import ( - _compile_native_object, - _import_from_build_dir, - _sole_native_module, -) -from prik import build_pyi_extension - - -def _build_contract_module( - tmp_path: Path, - *, - module_name: str, - source_text: str, - contract_text: str, -): - source = tmp_path / f"{module_name}.f90" - source.write_text(source_text, encoding="utf-8") - contract = tmp_path / f"{module_name}.pyi" - contract.write_text(contract_text, encoding="utf-8") - native_object = _compile_native_object(source, tmp_path / "native") - - result = build_pyi_extension( - contract, - native_objects=[native_object], - native_include_dirs=[native_object.parent], - output_dir=tmp_path / "build", - ) - generated_c = (result.output_dir / f"{module_name}_wrapper.c").read_text(encoding="utf-8") - assert "static PyObject * wrap_" in generated_c - return _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) - - -def _build_multiple_scalar_result_modules(tmp_path: Path): - return ( - _build_contract_module( - tmp_path, - module_name="multiple_scalar_results_plan", - source_text=""" -module multiple_scalar_results_plan - use iso_c_binding, only: c_int32_t -contains - function with_scalar(n, status) result(value) - integer(c_int32_t), intent(in) :: n - integer(c_int32_t), intent(out) :: status - integer(c_int32_t) :: value - value = n * 2 - status = n + 3 - end function with_scalar -end module multiple_scalar_results_plan -""", - contract_text=""" -from prik.contracts import Addr, Arg, Int32, Return, native_call - -@native_call([Addr(Arg(0)), Return("status", 1)]) -def with_scalar(n: Int32) -> tuple[Int32, Int32]: ... -""", - ), - ) - - -def test_multiple_scalar_results_use_canonical_plan_without_array_blockers(tmp_path: Path): - modules = _build_multiple_scalar_result_modules(tmp_path) - - for module in modules: - result = module.with_scalar(np.int32(4)) - - assert result == (np.int32(8), np.int32(7)) - assert tuple(type(value) for value in result) == (np.int32, np.int32) diff --git a/tests/fortran/data_types/end_to_end/test_project_kind_alias_chain.py b/tests/fortran/data_types/end_to_end/test_project_kind_alias_chain.py index 7c0f6f878..e1928c1e4 100644 --- a/tests/fortran/data_types/end_to_end/test_project_kind_alias_chain.py +++ b/tests/fortran/data_types/end_to_end/test_project_kind_alias_chain.py @@ -19,13 +19,14 @@ KIND_ALIAS_SOURCE = (NATIVE_FIXTURES / "kinds.f90").read_text(encoding="utf-8") -def test_kind_alias_chain_reaches_the_probe_as_intrinsic_expressions(tmp_path: Path): +def test_kind_alias_chain_resolves_in_source_and_generated_contract_builds(tmp_path: Path): """A project names its kinds through its own parameters, and they resolve. Each `use` of one module adds to what the scope imported, and a parameter may name another, so `RP` reaches `REAL64` through `DP`. The probe measures target storage and is given expressions a compiler understands, never a - project name it has no way to evaluate. + project name it has no way to evaluate. The generated contract states the + resolved types, and rebuilding from it keeps the behavior. """ source = tmp_path / "kinds.f90" source.write_text(KIND_ALIAS_SOURCE, encoding="utf-8") @@ -48,11 +49,6 @@ def test_kind_alias_chain_reaches_the_probe_as_intrinsic_expressions(tmp_path: P ) assert module.consumer_mod.work(np.float64(2.5), np.int32(4)) == pytest.approx(10.0) - -def test_kind_alias_chain_survives_its_generated_contract(tmp_path: Path): - """The contract states resolved types, and rebuilding keeps the behavior.""" - source = tmp_path / "kinds.f90" - source.write_text(KIND_ALIAS_SOURCE, encoding="utf-8") contracts = tmp_path / "contracts" subprocess.run( [ diff --git a/tests/fortran/data_types/end_to_end/test_scalar_wrapper_parity.py b/tests/fortran/data_types/end_to_end/test_scalar_wrapper_parity.py index c45fa693d..06e04eeb5 100644 --- a/tests/fortran/data_types/end_to_end/test_scalar_wrapper_parity.py +++ b/tests/fortran/data_types/end_to_end/test_scalar_wrapper_parity.py @@ -1,4 +1,4 @@ -"""Primitive-scalar source/contract parity and wrapper-plan runtime tests.""" +"""Primitive-scalar source/contract parity for fixed-form externals and a free-form module.""" from pathlib import Path @@ -8,103 +8,41 @@ from tests.fortran._support.wrapper_build import ( _assert_fmath_examples, _build_source_or_generated_pyi_and_import, - _build_source_wrapper_plan_and_import, wrapper_source, ) DATA_TYPE_CONTRACTS = Path(__file__).parent / "fixtures" / "contracts" -SCALAR_FIXED_SOURCE = wrapper_source("fmath.f") -SCALAR_F90_SOURCE = wrapper_source("fmath_f90.f90") pytestmark = pytest.mark.fortran_end_to_end -def test_fortran_wrapper_pipeline_builds_importable_extension( +@pytest.mark.parametrize( + ("filename", "contract"), + [("fmath.f", "fmath"), ("fmath_f90.f90", "fmath_f90")], + ids=["fixed-form-externals", "free-form-module"], +) +def test_fmath_scalar_sources_build_from_source_and_generated_contract( + filename: str, + contract: str, pyi_parity_build_mode: str, tmp_path: Path, ): + source = wrapper_source(filename) module = _build_source_or_generated_pyi_and_import( - SCALAR_FIXED_SOURCE, + source, tmp_path, { - "bind_c_fmath_wrapper.f90", - "fmath_wrapper.c", - "fmath_wrapper.h", + f"bind_c_{source.stem}_wrapper.f90", + f"{source.stem}_wrapper.c", + f"{source.stem}_wrapper.h", }, - DATA_TYPE_CONTRACTS / "fmath", + DATA_TYPE_CONTRACTS / contract, pyi_parity_build_mode, ) _assert_fmath_examples(module) - - -@pytest.mark.parametrize( - "source", - [SCALAR_FIXED_SOURCE, SCALAR_F90_SOURCE], - ids=["fixed-form-externals", "free-form-module"], -) -def test_fmath_scalar_sources_use_canonical_wrapper_plan( - tmp_path: Path, - source: Path, -): - expected_generated_sources = { - f"bind_c_{source.stem}_wrapper.f90", - f"{source.stem}_wrapper.c", - f"{source.stem}_wrapper.h", - } - wrapper_root, wrapper_result = _build_source_wrapper_plan_and_import( - source, - tmp_path / "build", - unwrap_namespace=False, - ) - - if source == SCALAR_F90_SOURCE: - assert not hasattr(wrapper_root, "add_r8") - assert hasattr(wrapper_root, "fmath_f90") - module = wrapper_root.fmath_f90 - else: - assert hasattr(wrapper_root, "add_r8") - module = wrapper_root - - assert {path.name for path in wrapper_result.generated_sources} == expected_generated_sources - assert any(path.name == f"{source.stem}_wrapper.h" for path in wrapper_result.generated_files) - assert any( - path.name == "prik_binding.h" and path.parent.name == "binding_support" - for path in wrapper_result.generated_files - ) - assert wrapper_result.compiled is True - assert wrapper_result.shared_library.exists() - - _assert_fmath_examples(module) - error_type, message = _scalar_conversion_failure(module) - assert error_type is TypeError - assert "argument" in message - - -def _scalar_conversion_failure(module) -> tuple[type[BaseException], str]: - with pytest.raises(TypeError) as error_info: + with pytest.raises(TypeError, match="argument"): module.add_r8("not-a-real", np.float64(1.0)) - np.testing.assert_allclose( module.add_r8(np.float64(1.5), np.float64(2.25)), (np.float64(3.75), np.float64(1.5), np.float64(2.25)), ) - return type(error_info.value), str(error_info.value) - - -def test_f90_wrapper_pipeline_builds_importable_extension( - pyi_parity_build_mode: str, - tmp_path: Path, -): - module = _build_source_or_generated_pyi_and_import( - SCALAR_F90_SOURCE, - tmp_path, - { - "bind_c_fmath_f90_wrapper.f90", - "fmath_f90_wrapper.c", - "fmath_f90_wrapper.h", - }, - DATA_TYPE_CONTRACTS / "fmath_f90", - pyi_parity_build_mode, - ) - - _assert_fmath_examples(module) diff --git a/tests/fortran/data_types/pipeline/test_generated_scalar_contract.py b/tests/fortran/data_types/pipeline/test_generated_scalar_contract.py deleted file mode 100644 index 12fc8d566..000000000 --- a/tests/fortran/data_types/pipeline/test_generated_scalar_contract.py +++ /dev/null @@ -1,21 +0,0 @@ -"""Generated semantic-contract evidence for the primitive scalar fixture.""" - -from pathlib import Path - -from tests.fortran._support.generated_contracts import ( - GeneratedContractCase, - assert_generated_contract_matches_fixture, -) - - -FEATURE_ROOT = Path(__file__).parents[1] -FIXTURES = FEATURE_ROOT / "end_to_end" / "fixtures" -CASE = GeneratedContractCase( - name="fscalar_kinds_f90", - inputs=(FIXTURES / "native" / "fscalar_kinds_f90.f90",), - expected_package=FIXTURES / "contracts" / "fscalar_kinds_f90", -) - - -def test_generated_primitive_scalar_contract_matches_reviewed_package(tmp_path: Path): - assert_generated_contract_matches_fixture(CASE, tmp_path) diff --git a/tests/fortran/data_types/pipeline/test_scalar_generated_pyi_contracts.py b/tests/fortran/data_types/pipeline/test_scalar_generated_pyi_contracts.py deleted file mode 100644 index 258f2f313..000000000 --- a/tests/fortran/data_types/pipeline/test_scalar_generated_pyi_contracts.py +++ /dev/null @@ -1,26 +0,0 @@ -"""Generated `.pyi` package fixtures for scalar wrapper inputs.""" - -from __future__ import annotations - -from pathlib import Path - -import pytest - -from tests.fortran._support.generated_contracts import ( - GeneratedContractCase, - assert_generated_contract_matches_fixture, - contract_case_id, - source_contract_case, -) - -DATA_TYPE_CONTRACTS = Path(__file__).parents[1] / "end_to_end" / "fixtures" / "contracts" -CASES = ( - source_contract_case(DATA_TYPE_CONTRACTS, "fbind_value_f90.f90"), - source_contract_case(DATA_TYPE_CONTRACTS, "fmath.f"), - source_contract_case(DATA_TYPE_CONTRACTS, "fmath_f90.f90"), -) - - -@pytest.mark.parametrize("case", CASES, ids=contract_case_id) -def test_scalar_generated_pyi_contract_matches_fixture(case: GeneratedContractCase, tmp_path: Path): - assert_generated_contract_matches_fixture(case, tmp_path) diff --git a/tests/fortran/data_types/probes/test_fortran_type_probes.py b/tests/fortran/data_types/probes/test_fortran_type_probes.py index 83e52030c..97e765170 100644 --- a/tests/fortran/data_types/probes/test_fortran_type_probes.py +++ b/tests/fortran/data_types/probes/test_fortran_type_probes.py @@ -260,7 +260,6 @@ def test_fortran_type_probe_report_resolves_only_matching_parameter_requirements [ ([OSError("missing")], "failed to run Fortran type probe compiler"), ([SimpleNamespace(returncode=1, stderr="compile failed")], "compilation failed"), - ([SimpleNamespace(returncode=0, stderr=""), OSError("cannot execute")], "failed to execute"), ( [SimpleNamespace(returncode=0, stderr=""), SimpleNamespace(returncode=2, stderr="run failed")], "execution failed", @@ -269,10 +268,6 @@ def test_fortran_type_probe_report_resolves_only_matching_parameter_requirements [SimpleNamespace(returncode=0, stderr=""), SimpleNamespace(returncode=0, stdout="not json", stderr="")], "invalid JSON", ), - ( - [SimpleNamespace(returncode=0, stderr=""), SimpleNamespace(returncode=0, stdout="{}", stderr="")], - "missing 'values'", - ), ( [ SimpleNamespace(returncode=0, stderr=""), @@ -328,21 +323,6 @@ def test_fortran_type_probe_accepts_runner_and_cli_validates_macro_names(monkeyp fortran_type_probe.main(["--compiler", "gfortran", "-U", "=bad"]) -def test_fortran_type_probe_reports_values_from_native_compiler(): - compiler = _required_fortran_compiler() - report = probe_fortran_type_expressions( - PreprocessingConfig(mode="compiler", compiler=compiler), - ["selected_int_kind(9)", "selected_real_kind(12)", "kind(1.0d0)"], - ) - - assert report.values["selected_int_kind(9)"] > 0 - assert report.values["selected_real_kind(12)"] > 0 - assert report.values["kind(1.0d0)"] > 0 - assert report.recipe.compiler == compiler - assert "-cpp" in report.recipe.compile_argv - assert "selected_real_kind(12)" in report.source_text - - def test_fortran_type_probe_resolves_supported_logical_storage_widths(tmp_path): compiler = _required_fortran_compiler() @@ -389,34 +369,6 @@ def test_fortran_type_probe_carries_target_relevant_user_flags(tmp_path): assert report.recipe.compiler_args == ["-fno-range-check"] -def test_fortran_type_probe_maps_compiler_storage_facts(): - compiler = _required_fortran_compiler() - requirements = [ - { - "base_type": "integer", - "kind": None, - "expression": "storage_size(int(0))", - }, - { - "base_type": "real", - "kind": None, - "expression": "storage_size(real(0.0))", - }, - ] - - facts = evaluate_fortran_type_facts( - PreprocessingConfig( - mode="compiler", - compiler=compiler, - compiler_args=["-fdefault-integer-8", "-fdefault-real-8"], - ), - requirements, - ) - - assert facts[("integer", None)]["bits"] == 64 - assert facts[("real", None)]["bits"] == 64 - - def test_fortran_type_probe_evaluates_collected_semantic_requirements(): compiler = _required_fortran_compiler() parsed = parse_fortran_source( @@ -499,49 +451,14 @@ def test_fortran_type_probe_module_cli_emits_json_for_semantic_input(tmp_path): assert payload["source_text"].startswith("program prik_fortran_type_probe") -def test_prik_semantics_cli_evaluates_collected_fortran_type_requirements(tmp_path): - compiler = _required_fortran_compiler() - source = tmp_path / "solver.f90" - source.write_text( - """ -module solver_mod - integer, parameter :: rk = selected_real_kind(12) -contains -subroutine scale(x) - real(kind=rk), intent(inout) :: x -end subroutine scale -end module solver_mod -""", - encoding="utf-8", - ) - - completed = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "semantics", - str(source), - "--json", - "--compiler", - compiler, - ], - capture_output=True, - text=True, - check=True, - ) - - payload = json.loads(completed.stdout) - semantic_type = payload[str(source)]["semantic_modules"][0]["functions"][0]["arguments"][0]["semantic_type"] - assert semantic_type["name"] == "Float64" - - -def test_prik_semantics_cli_uses_compiler_dependent_default_fortran_kinds(tmp_path): +def test_prik_semantics_cli_uses_compiler_dependent_kinds_and_collected_requirements(tmp_path): compiler = _required_fortran_compiler() source = tmp_path / "defaults.f90" source.write_text( """ module defaults + integer, parameter :: rk = selected_real_kind(12) + real(kind=rk) :: selected integer :: count real :: scale complex :: value @@ -574,6 +491,7 @@ def test_prik_semantics_cli_uses_compiler_dependent_default_fortran_kinds(tmp_pa payload = json.loads(completed.stdout) variables = payload[str(source)]["semantic_modules"][0]["variables"] semantic_types = {variable["name"]: variable["semantic_type"] for variable in variables} + assert semantic_types["selected"]["name"] == "Float64" assert semantic_types["count"]["name"] == "Int64" assert semantic_types["scale"]["name"] == "Float64" assert semantic_types["value"]["name"] == "Complex128" diff --git a/tests/fortran/data_types/semantics/test_fortran_scalar_semantics.py b/tests/fortran/data_types/semantics/test_fortran_scalar_semantics.py index 920eb3cc2..15e2b9dcf 100644 --- a/tests/fortran/data_types/semantics/test_fortran_scalar_semantics.py +++ b/tests/fortran/data_types/semantics/test_fortran_scalar_semantics.py @@ -138,20 +138,6 @@ def test_fortran2ir_maps_probed_logical_storage_to_language_neutral_boolean_widt assert semantic_type.dtype == expected -def test_fortran2ir_rejects_compiler_storage_without_semantic_dtype(): - fact = { - "base_type": "integer", - "kind": None, - "bits": 48, - "expression": "storage_size(int(0))", - } - - with pytest.raises(ValueError, match="integer uses 48-bit storage"): - FortranToIRConverter(type_facts={("integer", None): fact}).visit( - FortranVariable(name="value", base_type="integer") - ) - - def test_compiler_probed_unknown_storage_widths_fail_in_semantic_conversion(): facts = ( {"base_type": "real", "kind": "3", "bits": 24}, diff --git a/tests/fortran/data_types/semantics/test_scalar_type_catalogue.py b/tests/fortran/data_types/semantics/test_scalar_type_catalogue.py index 45b5df597..f7db792cf 100644 --- a/tests/fortran/data_types/semantics/test_scalar_type_catalogue.py +++ b/tests/fortran/data_types/semantics/test_scalar_type_catalogue.py @@ -5,23 +5,11 @@ from prik.semantics.scalar_types import ( BOOLEAN_SEMANTIC_TYPE_NAMES, SEMANTIC_SCALAR_TYPES, - SEMANTIC_SCALAR_TYPE_NAMES, - SemanticScalarFamily, boolean_storage_bits, is_boolean_semantic_type_name, ) -def test_scalar_catalogue_exposes_semantic_family_and_storage_without_numpy_facts(): - assert SEMANTIC_SCALAR_TYPES["Int32"].family is SemanticScalarFamily.SIGNED_INTEGER - assert SEMANTIC_SCALAR_TYPES["Int32"].storage_bits == 32 - assert SEMANTIC_SCALAR_TYPES["Float64"].family is SemanticScalarFamily.REAL - assert SEMANTIC_SCALAR_TYPES["Float64"].storage_bits == 64 - assert SEMANTIC_SCALAR_TYPES["String"].family is SemanticScalarFamily.CHARACTER - assert SEMANTIC_SCALAR_TYPES["String"].storage_bits is None - assert frozenset(SEMANTIC_SCALAR_TYPES) == SEMANTIC_SCALAR_TYPE_NAMES - - def test_boolean_catalogue_preserves_native_widths_that_numpy_bool_cannot_distinguish(): names = ("Bool", "Bool8", "Bool16", "Bool32", "Bool64") diff --git a/tests/fortran/data_types/semantics/test_types_and_storage.py b/tests/fortran/data_types/semantics/test_types_and_storage.py index 900bc445b..b99ae0e83 100644 --- a/tests/fortran/data_types/semantics/test_types_and_storage.py +++ b/tests/fortran/data_types/semantics/test_types_and_storage.py @@ -1,16 +1,8 @@ """Tests split by stable ownership concept from `test_compile_time_values.py`.""" -import json from dataclasses import asdict from prik.parsers.fortran.models import ( FortranArgument, - FortranDerivedType, - FortranFile, - FortranModule, - FortranProcedureSignature, - FortranProject, - FortranUseMapping, - FortranUseStatement, FortranVariable, ) from prik.semantics.fortran2ir import ( @@ -20,108 +12,10 @@ from tests.fortran._support.semantic_conversion import ( array_contract, get_function, - has_constraint, ) from prik.parsers.fortran import parse_fortran_file as parse_fortran_source -def test_converter_visitor_and_compatibility_methods_cover_public_paths(): - converter = FortranToIRConverter() - scale = FortranVariable(name="scale", base_type="real", kind="8", is_parameter=True) - arg = FortranArgument( - name="x", - base_type="real", - kind="8", - allocatable=True, - pointer=True, - ) - proc = FortranProcedureSignature(name="work", kind="subroutine", arguments=[arg]) - base = FortranDerivedType(name="base_t") - dtype = FortranDerivedType( - name="child_t", - fields=[FortranArgument(name="payload", base_type="derived", kind="base_t")], - extends=base, - ) - module = FortranModule( - name="m", - uses=[ - FortranUseStatement("iso_c_binding", True, (FortranUseMapping(source="c_int", target="i32"),)), - FortranUseStatement("plain_import"), - ], - variables=[scale], - procedures=[proc], - derived_types=[dtype], - private_symbols=["work"], - ) - parsed = FortranFile(filename="/tmp/standalone_source.f90", modules=[module], procedures=[proc]) - - assert converter.visit(parsed)[0].name == "m" - assert converter.visit(module).functions[0].visibility == "private" - assert converter.visit(proc, visibility="private").visibility == "private" - assert converter.visit(proc).visibility == "public" - - semantic_arg = converter.visit(arg) - assert semantic_arg.semantic_type.storage.kind == "reference" - assert semantic_arg.semantic_type.storage.mutable is True - assert semantic_arg.visibility == "public" - assert semantic_arg.origin.source_language == "fortran" - assert semantic_arg.origin.native_name == "x" - assert semantic_arg.origin.source_kind == "argument" - - semantic_var = converter.visit(scale) - assert semantic_var.name == "Float64" - assert has_constraint(semantic_var, "Constant") - assert converter.visit(arg).name == "x" - assert converter.visit(proc).name == "work" - assert converter.visit(proc).visibility == "public" - assert converter.visit(dtype, procedure_lookup={}).base_classes == ["base_t"] - # No declaration is written with `i32`, and the compiler supplies - # `iso_c_binding`, so the module states no import for its contract. - assert converter.visit(module).imports == [] - - modules = converter.visit(parsed) - assert [module.name for module in modules] == ["m", "standalone_source"] - assert converter.visit(FortranProject(files=[parsed]))[0].name == "m" - - -def test_basic_scalar_arguments(): - source = """ -module math_mod - -contains - -subroutine add(a, b, c) - - real(8), intent(in) :: a - real(8), intent(in) :: b - real(8), intent(out) :: c - -end subroutine - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - assert smod.name == "math_mod" - - func = get_function(smod, "add") - - assert len(func.arguments) == 3 - - a = func.arguments[0] - c = func.arguments[2] - - assert a.name == "a" - - assert a.semantic_type.name == "Float64" - assert a.semantic_type.rank == 0 - - assert c.semantic_type.ownership.mutable is True - - def test_fortran_native_storage_contracts_cover_array_categories_and_scalars(): source = """ module contract_mod @@ -279,31 +173,3 @@ def test_fortran_native_storage_contracts_preserve_exact_bounds_and_member_flags assert plain_member.origin.source_kind == "variable" assert mixed_bounds.storage.array.lower_bounds == [None, None, "0"] assert mixed_bounds.storage.array.upper_bounds == [None, "4", "4"] - - -def test_semantic_ir_serialization(): - source = """ -module simple_mod - -contains - -subroutine hello(x) - - integer, intent(in) :: x - -end subroutine - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - data = asdict(smod) - - json_text = json.dumps(data, indent=2) - - assert "hello" in json_text - - assert "Int32" in json_text diff --git a/tests/fortran/derived_types/codegen/test_class_surfaces.py b/tests/fortran/derived_types/codegen/test_class_surfaces.py index 62857e54b..361fc3e01 100644 --- a/tests/fortran/derived_types/codegen/test_class_surfaces.py +++ b/tests/fortran/derived_types/codegen/test_class_surfaces.py @@ -4,10 +4,7 @@ import pytest -from prik.parsers.fortran import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules from prik.pipeline.pyi import pyi_file_to_semantic_module -from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.policy.completion import complete_semantic_policies from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner @@ -28,32 +25,6 @@ def _surface(plan, name: str): ) -def test_inheritance_and_polymorphism_are_completed_before_planning(): - plan = _plan(INHERITANCE) - base = _surface(plan, "Base_Shape") - circle = _surface(plan, "Circle") - derived = next( - item - for namespace in plan.namespaces - for item in namespace.derived_types - if item.type_identity == circle.type_identity - ) - describe = next( - function - for namespace in plan.namespaces - for function in namespace.functions - if function.binding.python_name == "describe_shape" - ) - - assert circle.base_identities == (base.type_identity,) - assert [field.name for field in derived.fields] == ["size", "radius"] - assert tuple(variant.type_identity for variant in describe.arguments[0].polymorphic.variants) == ( - _surface(plan, "Box").type_identity, - circle.type_identity, - base.type_identity, - ) - - def test_invalid_class_graph_fails_before_emission(): plan = _plan(INHERITANCE) _surface(plan, "Circle").base_identities = (("missing", "base"),) @@ -70,38 +41,3 @@ def test_a_type_defined_in_two_namespaces_fails_before_emission(): with pytest.raises(ValueError, match="duplicate-derived-type-identity"): WrapperGenerator().generate(plan) - - -EXTENDING_ANOTHER_MODULE = """\ -module zeta_base - implicit none - type :: shape - integer :: sides = 0 - end type shape -end module zeta_base - -module alpha_child - use zeta_base, only: shape - implicit none - type, extends(shape) :: square - integer :: edge = 1 - end type square -end module alpha_child -""" - - -def test_a_namespace_is_planned_after_the_one_defining_its_base(tmp_path: Path): - """A class extending another namespace's type is created once its base exists. - - Path order would put `alpha_child` first; inheritance overrides it only - where it has to. - """ - (tmp_path / "project.f90").write_text(EXTENDING_ANOTHER_MODULE, encoding="utf-8") - modules = fortran_project_to_semantic_modules(parse_fortran_project(str(tmp_path))) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name="package") - complete_semantic_policies(module) - - plan = WrapperPlanner().build(module) - - assert [namespace.python_path for namespace in plan.namespaces] == [(), ("zeta_base",), ("alpha_child",)] diff --git a/tests/fortran/derived_types/codegen/test_derived_array_field_lowering.py b/tests/fortran/derived_types/codegen/test_derived_array_field_lowering.py index 771edfbd1..a035232e3 100644 --- a/tests/fortran/derived_types/codegen/test_derived_array_field_lowering.py +++ b/tests/fortran/derived_types/codegen/test_derived_array_field_lowering.py @@ -40,32 +40,6 @@ def _bridge_source(): return _bridge_source_for(ARRAY_FIELD_SOURCE, "field_state") -def test_owned_array_field_takes_its_address_through_the_owner_pointer(): - """An owner reached as a pointer makes its components addressable. - - The owner arrives as an address and is associated with a Fortran pointer, so - its components are subobjects of a pointer target. `c_loc` may name them - whatever the field's own declaration said, and no capture is needed. - """ - source = _bridge_source() - - assert "result = c_loc(owner%grid)" in source - assert "extent_0 = int(size(owner%grid, 1), c_int64_t)" in source - - -def test_plain_module_object_field_captures_its_address_in_c(): - """A plain module object is named directly, so nothing about it is a target. - - `c_loc` cannot name a member of a module object that was declared without - `target`, so the address is taken on the C side, exactly as a non-addressable - module array's is. - """ - source = _bridge_source() - - assert "result = prik_capture_address(native_plain_box%grid)" in source - assert "c_loc(native_plain_box%grid)" not in source - - def _procedure(source: str, name: str) -> str: """Return the text of one generated procedure, by name.""" start = source.index(f"function {name}(") diff --git a/tests/fortran/derived_types/codegen/test_derived_lowering.py b/tests/fortran/derived_types/codegen/test_derived_lowering.py index 465bc7f9b..a08d9e0dd 100644 --- a/tests/fortran/derived_types/codegen/test_derived_lowering.py +++ b/tests/fortran/derived_types/codegen/test_derived_lowering.py @@ -7,7 +7,6 @@ import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.semantics.models import RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA from prik.policy.completion import complete_semantic_policies from prik.policy.models import ( BridgeDataAction, @@ -21,13 +20,11 @@ from prik.planning import WrapperPlanner -def _value_module(*, bind_c: bool = False): - decorator = '@native_abi("c")' if bind_c else "" +def _value_module(): module = parse_pyi_text( - f""" -from prik.contracts import Arg, Float64, Value, native_abi, native_call + """ +from prik.contracts import Arg, Float64, Value, native_call -{decorator} class point: x: Float64 @@ -51,32 +48,6 @@ def _sources(plan): return c_source, bridge_source -@pytest.mark.parametrize("bind_c", [False, True]) -def test_exact_typed_value_policy_projects_shared_canonical_derived_handoff(bind_c): - module = _value_module(bind_c=bind_c) - function_policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - policy = function_policy.arguments[0] - plan = WrapperPlanner().build(module) - argument = plan.namespaces[0].functions[0].arguments[0] - - assert policy.derived.type_identity == ("derived_value", "point") - assert policy.derived.native_handoff is DerivedNativeHandoff.TYPED_VALUE - assert policy.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - assert argument.derived is argument.projected_call_slot.derived - assert argument.derived.type_identity == ("derived_value", "point") - assert argument.derived.native_handoff is DerivedNativeHandoff.TYPED_VALUE - assert "type_identity=('derived_value', 'point')" in str(plan) - - -def test_exact_typed_value_lowering_uses_fortran_value_semantics_and_opaque_binding(): - c_source, bridge_source = _sources(_value_plan()) - - assert "struct point" not in c_source - assert "PyCapsule_GetPointer" in c_source - assert "type(prik_type_point), pointer :: value" in bridge_source - assert "native_score(value)" in bridge_source - - @pytest.mark.parametrize( ("edit", "diagnostic"), ( @@ -161,35 +132,6 @@ def make_point() -> Returns["value", point]: ... assert "allocate(value_value, stat=prik_allocation_status)" in bridge_source -def test_projected_derived_argument_returns_the_exact_caller_wrapper_without_release(): - module = parse_pyi_text( - """ -from prik.contracts import Float64, Returns - -class point: - x: Float64 - -def update(value: point) -> Returns["value", point]: ... -""", - module_name="derived_writeback", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - function = plan.namespaces[0].functions[0] - argument = function.arguments[0] - - assert argument.derived is argument.projected_call_slot.derived - assert argument.derived.origin is DerivedObjectOrigin.CALLER_WRAPPER - assert argument.derived.owner_retention is DerivedOwnerRetention.CALLER_WRAPPER - assert argument.derived.release is DerivedRelease.NONE - assert function.writeback_actions[2].binding.python_result_role.endswith(":python-result") - - c_source, _ = _sources(plan) - assert "PyObject * result_obj = bound_value_obj;" in c_source - assert "Py_INCREF(result_obj);" in c_source - assert "point_to_" not in c_source - - def test_derived_module_handoff_edit_fails_central_validation(): module = parse_pyi_text( """ diff --git a/tests/fortran/derived_types/codegen/test_scalar_actual_dummy_plan.py b/tests/fortran/derived_types/codegen/test_scalar_actual_dummy_plan.py index b64abad56..9a1ddf32a 100644 --- a/tests/fortran/derived_types/codegen/test_scalar_actual_dummy_plan.py +++ b/tests/fortran/derived_types/codegen/test_scalar_actual_dummy_plan.py @@ -79,104 +79,75 @@ def _actions(call): return {case.actual_storage: case.action for case in call.cases} -def _accesses(call): - return {case.actual_storage: case.access for case in call.cases} - - -@pytest.mark.parametrize( - ("function_name", "dummy"), - [ - ("object_dummy", DerivedDummyCategory.OBJECT), - ("target_dummy", DerivedDummyCategory.TARGET), - ("allocatable_dummy", DerivedDummyCategory.ALLOCATABLE), - ("allocatable_target_dummy", DerivedDummyCategory.ALLOCATABLE_TARGET), - ("pointer_dummy", DerivedDummyCategory.POINTER), - ("value_dummy", DerivedDummyCategory.VALUE), - ], -) -def test_every_dummy_form_has_one_exhaustive_completed_matrix(function_name, dummy): +S = DerivedObjectStorage +A = DerivedCallAction +_REFERENCE_ACTIONS = { + S.DIRECT: A.DIRECT_REFERENCE, + S.ALLOCATABLE_HOLDER: A.HOLDER_REFERENCE, + S.POINTER_HOLDER: A.POINTEE_REFERENCE, + S.MODULE_PROXY: A.SCOPED_REFERENCE, + S.MODULE_TARGET: A.MODULE_ADDRESS, + S.MODULE_ALLOCATABLE: A.SCOPED_REFERENCE, + S.MODULE_ALLOCATABLE_TARGET: A.MODULE_ADDRESS, + S.MODULE_POINTER: A.POINTEE_REFERENCE, +} +_ALLOCATABLE_ACTIONS = { + storage: A.INCOMPATIBLE + for storage in STORAGES + if storage not in {S.ALLOCATABLE_HOLDER, S.MODULE_ALLOCATABLE, S.MODULE_ALLOCATABLE_TARGET} +} | { + S.ALLOCATABLE_HOLDER: A.ALLOCATABLE_HOLDER, + S.MODULE_ALLOCATABLE: A.MODULE_ALLOCATABLE_TRANSACTION, + S.MODULE_ALLOCATABLE_TARGET: A.MODULE_ALLOCATABLE_TRANSACTION, +} +_POINTER_STORAGE_ACTIONS = {S.POINTER_HOLDER: A.POINTER_HOLDER, S.MODULE_POINTER: A.MODULE_POINTER_TRANSACTION} +_REQUIRED_PRESENT = { + S.ALLOCATABLE_HOLDER, + S.POINTER_HOLDER, + S.MODULE_ALLOCATABLE, + S.MODULE_ALLOCATABLE_TARGET, + S.MODULE_POINTER, +} + +# Each dummy form, the action completed for every actual storage, and which storages must carry a payload. +DUMMY_MATRIX = { + "object_dummy": (DerivedDummyCategory.OBJECT, _REFERENCE_ACTIONS, _REQUIRED_PRESENT), + "target_dummy": (DerivedDummyCategory.TARGET, _REFERENCE_ACTIONS, _REQUIRED_PRESENT), + "allocatable_dummy": (DerivedDummyCategory.ALLOCATABLE, _ALLOCATABLE_ACTIONS, set()), + "allocatable_target_dummy": (DerivedDummyCategory.ALLOCATABLE_TARGET, _ALLOCATABLE_ACTIONS, set()), + # A nonprojecting pointer dummy adapts nonpointer storage through a call-local pointer. + "pointer_dummy": ( + DerivedDummyCategory.POINTER, + dict.fromkeys(STORAGES, A.POINTER_INPUT_ADAPTER) | _POINTER_STORAGE_ACTIONS, + None, + ), + # Projected pointer writeback requires persistent pointer storage. + "projected_pointer_dummy": ( + DerivedDummyCategory.POINTER, + dict.fromkeys(STORAGES, A.INCOMPATIBLE) | _POINTER_STORAGE_ACTIONS, + None, + ), + # An exact typed value is not restricted to bind(C) layout. + "value_dummy": (DerivedDummyCategory.VALUE, dict.fromkeys(STORAGES, A.TYPED_VALUE_COPY), None), +} + + +@pytest.mark.parametrize("function_name", tuple(DUMMY_MATRIX)) +def test_every_dummy_form_completes_one_exhaustive_action_matrix(function_name): + dummy, expected_actions, required_present = DUMMY_MATRIX[function_name] call = _plans()[function_name] assert call.dummy_category is dummy assert tuple(case.actual_storage for case in call.cases) == STORAGES - assert len({case.abi_code for case in call.cases if case.action is not DerivedCallAction.INCOMPATIBLE}) <= 6 + assert _actions(call) == expected_actions + assert len({case.abi_code for case in call.cases if case.action is not A.INCOMPATIBLE}) <= 6 for case in call.cases: - incompatible = case.action is DerivedCallAction.INCOMPATIBLE + incompatible = case.action is A.INCOMPATIBLE assert incompatible is (case.access is DerivedActualAccess.NONE) assert incompatible is bool(case.failure_kind and case.failure_message) assert incompatible is (case.abi_code == 0) - - -@pytest.mark.parametrize("function_name", ["object_dummy", "target_dummy"]) -def test_object_and_target_dummies_cover_direct_scoped_holder_and_pointee_actuals(function_name): - call = _plans()[function_name] - actions = _actions(call) - - assert actions == { - DerivedObjectStorage.DIRECT: DerivedCallAction.DIRECT_REFERENCE, - DerivedObjectStorage.ALLOCATABLE_HOLDER: DerivedCallAction.HOLDER_REFERENCE, - DerivedObjectStorage.POINTER_HOLDER: DerivedCallAction.POINTEE_REFERENCE, - DerivedObjectStorage.MODULE_PROXY: DerivedCallAction.SCOPED_REFERENCE, - DerivedObjectStorage.MODULE_TARGET: DerivedCallAction.MODULE_ADDRESS, - DerivedObjectStorage.MODULE_ALLOCATABLE: DerivedCallAction.SCOPED_REFERENCE, - DerivedObjectStorage.MODULE_ALLOCATABLE_TARGET: DerivedCallAction.MODULE_ADDRESS, - DerivedObjectStorage.MODULE_POINTER: DerivedCallAction.POINTEE_REFERENCE, - } - required = {case.actual_storage for case in call.cases if case.requires_present} - assert required == { - DerivedObjectStorage.ALLOCATABLE_HOLDER, - DerivedObjectStorage.POINTER_HOLDER, - DerivedObjectStorage.MODULE_ALLOCATABLE, - DerivedObjectStorage.MODULE_ALLOCATABLE_TARGET, - DerivedObjectStorage.MODULE_POINTER, - } - - -@pytest.mark.parametrize("function_name", ["allocatable_dummy", "allocatable_target_dummy"]) -def test_allocatable_dummies_accept_only_holders_and_module_transactions(function_name): - call = _plans()[function_name] - actions = _actions(call) - compatible = { - storage: action for storage, action in actions.items() if action is not DerivedCallAction.INCOMPATIBLE - } - - assert compatible == { - DerivedObjectStorage.ALLOCATABLE_HOLDER: DerivedCallAction.ALLOCATABLE_HOLDER, - DerivedObjectStorage.MODULE_ALLOCATABLE: DerivedCallAction.MODULE_ALLOCATABLE_TRANSACTION, - DerivedObjectStorage.MODULE_ALLOCATABLE_TARGET: DerivedCallAction.MODULE_ALLOCATABLE_TRANSACTION, - } - assert all(not case.requires_present for case in call.cases if case.action is not DerivedCallAction.INCOMPATIBLE) - - -def test_nonprojecting_pointer_dummy_uses_call_local_adapters_for_nonpointer_storage(): - call = _plans()["pointer_dummy"] - actions = _actions(call) - - assert actions[DerivedObjectStorage.POINTER_HOLDER] is DerivedCallAction.POINTER_HOLDER - assert actions[DerivedObjectStorage.MODULE_POINTER] is DerivedCallAction.MODULE_POINTER_TRANSACTION - assert all( - actions[storage] is DerivedCallAction.POINTER_INPUT_ADAPTER - for storage in STORAGES - if storage not in {DerivedObjectStorage.POINTER_HOLDER, DerivedObjectStorage.MODULE_POINTER} - ) - - -def test_projected_pointer_writeback_requires_persistent_pointer_storage(): - call = _plans()["projected_pointer_dummy"] - actions = _actions(call) - - assert actions[DerivedObjectStorage.POINTER_HOLDER] is DerivedCallAction.POINTER_HOLDER - assert actions[DerivedObjectStorage.MODULE_POINTER] is DerivedCallAction.MODULE_POINTER_TRANSACTION - assert all( - actions[storage] is DerivedCallAction.INCOMPATIBLE - for storage in STORAGES - if storage not in {DerivedObjectStorage.POINTER_HOLDER, DerivedObjectStorage.MODULE_POINTER} - ) - - -def test_exact_typed_value_is_not_restricted_to_bind_c_layout(): - call = _plans()["value_dummy"] - assert all(case.action is DerivedCallAction.TYPED_VALUE_COPY for case in call.cases) + if required_present is not None: + assert {case.actual_storage for case in call.cases if case.requires_present} == required_present def test_module_actual_declarations_keep_distinct_runtime_storage(): @@ -195,19 +166,6 @@ def test_module_actual_declarations_keep_distinct_runtime_storage(): } -def test_pointer_result_uses_a_persistent_holder_instead_of_the_removed_blocker(): - module = _module() - result = next( - function.results[0] - for function in WrapperPlanner().build(module).namespaces[0].functions - if function.symbol_name == "make_pointer" - ) - - assert result.derived.storage is DerivedObjectStorage.POINTER_HOLDER - assert result.derived.target_owner_retention is DerivedOwnerRetention.NATIVE_MODULE - assert result.derived.target_release is DerivedRelease.NATIVE_OWNER - - def test_class_only_derived_methods_do_not_emit_unreachable_scoped_trampolines(): module = parse_pyi_text( """ @@ -253,20 +211,3 @@ def test_validation_rejects_a_backend_invented_matrix_gap(): with pytest.raises(ValueError, match="incomplete-derived-call-matrix"): WrapperGenerator().generate(plan) - - -def test_artifacts_emit_shared_holders_typed_origin_operations_and_one_native_call(): - artifacts = WrapperGenerator().generate(WrapperPlanner().build(_module())) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert bridge.count("type :: prik_item_allocatable_holder") == 1 - assert bridge.count("type :: prik_item_pointer_holder") == 1 - assert "abstract interface" in bridge - assert "c_f_procpointer" in bridge - assert "c_funloc(prik_derived_consumer" in bridge - assert "move_alloc" in bridge - assert bridge.count("native_object_dummy(") == 1 - assert "prik_derived_origin_ops" in c_source - assert "atomic_compare_exchange_strong" in c_source - assert "CFI_cdesc_t" not in bridge diff --git a/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/module_aliased_proxy/__init__.pyi b/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/module_aliased_proxy/__init__.pyi deleted file mode 100644 index 687107bfc..000000000 --- a/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/module_aliased_proxy/__init__.pyi +++ /dev/null @@ -1 +0,0 @@ -from . import fmodule_derived_alias_f90 diff --git a/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/module_aliased_proxy/fmodule_derived_alias_f90.pyi b/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/module_aliased_proxy/fmodule_derived_alias_f90.pyi deleted file mode 100644 index 09f8fe016..000000000 --- a/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/module_aliased_proxy/fmodule_derived_alias_f90.pyi +++ /dev/null @@ -1,15 +0,0 @@ -from prik.contracts import Addr, Aliased, Allocatable, Annotated, Arg, Float64, Int32, native_call - -class box: - values: Allocatable[Float64[:]] - -current: Annotated[box, Aliased] - -@native_call([Addr(Arg(0))]) -def allocate_current( - n: Int32 -) -> None: ... - -def deallocate_current() -> None: ... - -def current_sum() -> Float64: ... diff --git a/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/opaque_boundary/__init__.pyi b/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/opaque_boundary/__init__.pyi deleted file mode 100644 index d78156d8f..000000000 --- a/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/opaque_boundary/__init__.pyi +++ /dev/null @@ -1 +0,0 @@ -from . import fderived_boundary_f90 diff --git a/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/opaque_boundary/fderived_boundary_f90.pyi b/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/opaque_boundary/fderived_boundary_f90.pyi deleted file mode 100644 index e14b5d9ae..000000000 --- a/tests/fortran/derived_types/end_to_end/fixtures/edited_contracts/opaque_boundary/fderived_boundary_f90.pyi +++ /dev/null @@ -1,29 +0,0 @@ -from prik.contracts import Addr, Arg, Float64, native_call - -class point: - x: Float64 - y: Float64 - -def point_sum( - p: point -) -> Float64: ... - -@native_call([Arg(0), Addr(Arg(1)), Addr(Arg(2))]) -def move_point( - p: point, - dx: Float64, - dy: Float64 -) -> None: ... - -@native_call([Arg(0), Addr(Arg(1)), Addr(Arg(2))]) -def make_point_out( - p: point, - x: Float64, - y: Float64 -) -> None: ... - -@native_call([Addr(Arg(0)), Addr(Arg(1))]) -def make_point( - x: Float64, - y: Float64 -) -> point: ... diff --git a/tests/fortran/derived_types/end_to_end/fixtures/native/fborrowed_finalizer_runtime.f90 b/tests/fortran/derived_types/end_to_end/fixtures/native/fborrowed_finalizer_runtime.f90 deleted file mode 100644 index cd06df131..000000000 --- a/tests/fortran/derived_types/end_to_end/fixtures/native/fborrowed_finalizer_runtime.f90 +++ /dev/null @@ -1,32 +0,0 @@ -module derived_borrowed_finalizer - implicit none - integer :: final_count = 0 - - type :: child - integer :: marker = 0 - contains - final :: cleanup_child - end type child - - type :: parent - type(child) :: value - end type parent -contains - subroutine cleanup_child(self) - type(child) :: self - final_count = final_count + 1 - end subroutine cleanup_child - - function make_parent() result(value) - type(parent) :: value - end function make_parent - - function get_final_count() result(value) - integer :: value - value = final_count - end function get_final_count - - subroutine reset_final_count() - final_count = 0 - end subroutine reset_final_count -end module derived_borrowed_finalizer diff --git a/tests/fortran/derived_types/end_to_end/test_abstract_hierarchy.py b/tests/fortran/derived_types/end_to_end/test_abstract_hierarchy.py index 09ec762c8..3313cae5a 100644 --- a/tests/fortran/derived_types/end_to_end/test_abstract_hierarchy.py +++ b/tests/fortran/derived_types/end_to_end/test_abstract_hierarchy.py @@ -7,83 +7,82 @@ import numpy as np import pytest -from tests.fortran._support.wrapper_build import _build_source_and_import +from prik.pipeline.build import BUILD_CONTRACT_DIRECTORY_NAME, build_fortran_extension +from prik.preprocessing import PreprocessingConfig +from tests.fortran._support.wrapper_build import _compiler, _import_from_build_dir, _sole_native_module pytestmark = pytest.mark.fortran_end_to_end SOURCE = Path(__file__).parent / "fixtures" / "native" / "abstract_hierarchy.f90" -GENERATED = { - "bind_c_abstract_hierarchy_wrapper.f90", - "abstract_hierarchy_wrapper.c", - "abstract_hierarchy_wrapper.h", -} @pytest.fixture(scope="module") -def module(tmp_path_factory): - return _build_source_and_import(SOURCE, tmp_path_factory.mktemp("abstract_hierarchy"), GENERATED) +def build(tmp_path_factory): + return build_fortran_extension( + SOURCE, + output_dir=tmp_path_factory.mktemp("abstract_hierarchy"), + preprocessing=PreprocessingConfig(mode="compiler", compiler=_compiler()), + ) + + +@pytest.fixture(scope="module") +def module(build): + return _sole_native_module(_import_from_build_dir(build.module_name, build.output_dir)) -def test_abstract_type_cannot_be_instantiated(module): - """`type, abstract ::` has no instances, so its Python class has no constructor.""" +def test_abstract_type_has_no_constructor_and_publishes_only_public_members(module): + """`type, abstract ::` has no instances, so its Python class has no constructor. + + The hierarchy publishes only what its `private` statements allow. + """ with pytest.raises(TypeError, match="abstract native type and cannot be instantiated"): module.Shape_Base() - assert "__init__" not in module.Shape_Base.__dict__ + assert {name for name in dir(module.Shape_Base) if not name.startswith("_")} == { + "area", + "label", + "side_count", + "bump_sides", + } + assert {name for name in dir(module.Circle) if not name.startswith("_")} == { + "area", + "label", + "side_count", + "bump_sides", + "radius", + } + + +def test_extensions_subclass_the_abstract_base_and_dispatch_its_bindings(module): + """Fortran `extends` becomes real Python inheritance, not copied members. -def test_extensions_are_python_subclasses_of_the_abstract_base(module): - """Fortran `extends` becomes real Python inheritance, not copied members.""" + A deferred binding names a contract and the dynamic type selects the body; + an implemented binding on the abstract base serves its extensions. + """ assert issubclass(module.Circle, module.Shape_Base) assert issubclass(module.Square, module.Shape_Base) assert module.Circle.__mro__[:2] == (module.Circle, module.Shape_Base) - assert isinstance(module.Circle(radius=np.float64(1.0)), module.Shape_Base) - - -def test_deferred_bindings_dispatch_to_each_concrete_override(module): - """A deferred binding names a contract; the dynamic type selects the body.""" circle = module.Circle(radius=np.float64(2.0)) square = module.Square(side=np.float64(3.0)) + assert isinstance(circle, module.Shape_Base) assert circle.area() == pytest.approx(12.566370614, rel=1e-9) assert square.area() == pytest.approx(9.0) assert circle.label() == "circle " assert square.label() == "square " - # The base declares the same bindings, and they resolve through the caller's # concrete type rather than through anything the abstract type implements. assert module.Shape_Base.area(circle) == pytest.approx(circle.area()) assert module.Shape_Base.area(square) == pytest.approx(square.area()) - -def test_inherited_bindings_and_components_reach_every_extension(module): - """An implemented binding on the abstract base serves its extensions.""" - circle = module.Circle(radius=np.float64(1.0)) - assert circle.side_count() == np.int32(0) circle.bump_sides() circle.bump_sides() assert circle.side_count() == np.int32(2) -def test_private_components_stay_off_the_generated_classes(module): - """The hierarchy publishes only what its `private` statements allow.""" - assert {name for name in dir(module.Shape_Base) if not name.startswith("_")} == { - "area", - "label", - "side_count", - "bump_sides", - } - assert {name for name in dir(module.Circle) if not name.startswith("_")} == { - "area", - "label", - "side_count", - "bump_sides", - "radius", - } - - def test_interoperable_type_keeps_its_layout_beside_the_hierarchy(module): """A `bind(c)` type in the same module still wraps through its own accessors.""" box = module.Extent(width=np.float64(3.0), height=np.float64(4.0)) @@ -95,23 +94,13 @@ def test_interoperable_type_keeps_its_layout_beside_the_hierarchy(module): assert module.describe(box) == pytest.approx(20.0) -def test_build_writes_its_semantic_contract_beside_the_extension(tmp_path: Path): +def test_build_writes_its_semantic_contract_beside_the_extension(build): """Every build leaves the contract describing the API it just generated.""" - from prik.pipeline.build import BUILD_CONTRACT_DIRECTORY_NAME, build_fortran_extension - from prik.preprocessing import PreprocessingConfig - from tests.fortran._support.wrapper_build import _compiler - - result = build_fortran_extension( - SOURCE, - output_dir=tmp_path, - preprocessing=PreprocessingConfig(mode="compiler", compiler=_compiler()), - ) - - contracts = result.output_dir / BUILD_CONTRACT_DIRECTORY_NAME + contracts = build.output_dir / BUILD_CONTRACT_DIRECTORY_NAME module_contract = contracts / "abstract_hierarchy.pyi" package_contract = contracts / "__init__.pyi" - assert module_contract in result.generated_files - assert package_contract in result.generated_files + assert module_contract in build.generated_files + assert package_contract in build.generated_files assert module_contract.is_file() assert package_contract.read_text(encoding="utf-8").strip() == ("from . import abstract_hierarchy") diff --git a/tests/fortran/derived_types/end_to_end/test_borrowed_components.py b/tests/fortran/derived_types/end_to_end/test_borrowed_components.py index 8f5992679..a72ff26de 100644 --- a/tests/fortran/derived_types/end_to_end/test_borrowed_components.py +++ b/tests/fortran/derived_types/end_to_end/test_borrowed_components.py @@ -46,6 +46,7 @@ def test_borrowed_child_wrapper_never_finalizes_native_component( assert module.get_final_count() == np.int32(0) borrowed = owner.value + assert borrowed._prik_owner is owner del owner gc.collect() assert module.get_final_count() == np.int32(0) diff --git a/tests/fortran/derived_types/end_to_end/test_derived_boundaries.py b/tests/fortran/derived_types/end_to_end/test_derived_boundaries.py index cce197d1e..0be561870 100644 --- a/tests/fortran/derived_types/end_to_end/test_derived_boundaries.py +++ b/tests/fortran/derived_types/end_to_end/test_derived_boundaries.py @@ -43,6 +43,8 @@ def test_scalar_derived_types_cross_procedure_boundaries( assert id(point) == identity assert point.x == np.float64(5.0) assert point.y == np.float64(7.0) + with pytest.raises(TypeError, match="Expected"): + point.x = 12.0 out_point = module.Point() assert module.make_point_out(out_point, np.float64(8.0), np.float64(9.0)) is None diff --git a/tests/fortran/derived_types/end_to_end/test_derived_runtime_mechanisms.py b/tests/fortran/derived_types/end_to_end/test_derived_runtime_mechanisms.py index 1da775f0d..2970597d2 100644 --- a/tests/fortran/derived_types/end_to_end/test_derived_runtime_mechanisms.py +++ b/tests/fortran/derived_types/end_to_end/test_derived_runtime_mechanisms.py @@ -2,7 +2,6 @@ from __future__ import annotations -import gc from pathlib import Path import numpy as np @@ -20,12 +19,8 @@ FIXTURES = Path(__file__).parent / "fixtures" EDITED_CONTRACTS = FIXTURES / "edited_contracts" -DERIVED_BOUNDARY_F90_SOURCE = FIXTURES / "native" / "fderived_boundary_f90.f90" -CONTRACT = EDITED_CONTRACTS / "opaque_boundary" / "__init__.pyi" PLAIN_MODULE_SOURCE = FIXTURES / "native" / "fmodule_derived_snapshot_f90.f90" PLAIN_MODULE_CONTRACT = EDITED_CONTRACTS / "module_live_proxy" / "__init__.pyi" -ALIASED_MODULE_SOURCE = FIXTURES / "native" / "fmodule_derived_alias_f90.f90" -ALIASED_MODULE_CONTRACT = EDITED_CONTRACTS / "module_aliased_proxy" / "__init__.pyi" DERIVED_CONSTANT_SOURCE = FORTRAN_ROOT / "modules" / "end_to_end" / "fixtures" / "native" / "fmodule_vars_f90.f90" pytestmark = pytest.mark.fortran_end_to_end @@ -70,78 +65,6 @@ def optional_sum(value: point | None = ...) -> Float64: ... def update_point(value: point) -> Returns["value", point]: ... def fill_point(value: point) -> Returns["value", point]: ... """ -# GCC 13.2 PR113885 ICEs on function-result assignment when a finalizable type -# has no data components. The marker keeps this lifetime test on its intended path. -BORROWED_FINALIZER_SOURCE = (NATIVE_FIXTURES / "fborrowed_finalizer_runtime.f90").read_text(encoding="utf-8") -BORROWED_FINALIZER_CONTRACT = """\ -from prik.contracts import Int32, destroy - -class child: - @destroy - def cleanup_child(self) -> None: ... - -class parent: - value: child - -def make_parent() -> parent: ... -def get_final_count() -> Int32: ... -def reset_final_count() -> None: ... -""" - - -def _build_point_boundary(tmp_path: Path): - native_object = _compile_native_object(DERIVED_BOUNDARY_F90_SOURCE, tmp_path / "native") - result = build_pyi_extension( - CONTRACT, - native_objects=[native_object], - native_include_dirs=[native_object.parent], - output_dir=tmp_path / "build", - ) - module = _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) - return module, result - - -def _exercise_point_boundary(module): - point = module.make_point(np.float64(1.0), np.float64(2.0)) - assert isinstance(point, module.point) - assert point.x == np.float64(1.0) - assert point.y == np.float64(2.0) - assert module.point_sum(point) == np.float64(3.0) - - point.x = np.float64(4.0) - point.y = np.float64(5.0) - assert module.point_sum(point) == np.float64(9.0) - - identity = id(point) - assert module.move_point(point, np.float64(2.0), np.float64(3.0)) is None - assert id(point) == identity - assert point.x == np.float64(6.0) - assert point.y == np.float64(8.0) - - output = module.make_point(np.float64(0.0), np.float64(0.0)) - assert module.make_point_out(output, np.float64(10.0), np.float64(11.0)) is None - assert output.x == np.float64(10.0) - assert output.y == np.float64(11.0) - - with pytest.raises(TypeError, match="Expected"): - point.x = 12.0 - - -def test_scalar_derived_objects_use_canonical_plan(tmp_path: Path): - module, result = _build_point_boundary(tmp_path) - _exercise_point_boundary(module) - with pytest.raises(TypeError): - module.point() - - generated_c = (result.output_dir / "opaque_boundary_wrapper.c").read_text(encoding="utf-8") - generated_fortran = (result.output_dir / "bind_c_opaque_boundary_wrapper.f90").read_text(encoding="utf-8") - assert "static PyObject * wrap_point_sum" in generated_c - assert "@x.setter\\n def x(self, value):" in generated_c - assert "bind_c_prik_field_point_x_get" in generated_fortran - assert "bind_c_prik_field_point_x_set" in generated_fortran - assert "call native_make_point_out(p, x, y)" in generated_fortran - assert "result = c_null_ptr" in generated_fortran - assert "allocate(result_value, stat=prik_allocation_status)" in generated_fortran def test_plain_module_derived_proxy_reads_and_writes_live_members(tmp_path: Path): @@ -215,44 +138,6 @@ def test_plain_module_derived_proxy_reads_and_writes_live_members(tmp_path: Path assert "native_current%nested%id" in generated_fortran -def test_aliased_module_derived_object_uses_direct_live_field_handles(tmp_path: Path): - native_object = _compile_native_object(ALIASED_MODULE_SOURCE, tmp_path / "native") - result = build_pyi_extension( - ALIASED_MODULE_CONTRACT, - native_objects=[native_object], - native_include_dirs=[native_object.parent], - output_dir=tmp_path / "wrapper_plan", - ) - module = _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) - - first = module.current - second = module.current - assert isinstance(first, module.box) - assert first is not second - assert first._prik_owner is module - assert second._prik_owner is module - first_values = first.values - assert first_values.owner is first - assert first_values.to_numpy() is None - - module.allocate_current(np.int32(3)) - first_view = first_values.to_numpy() - np.testing.assert_allclose(first_view, np.array([1.0, 2.0, 3.0], dtype=np.float64)) - first_view[0] = np.float64(10.0) - assert module.current_sum() == np.float64(15.0) - np.testing.assert_allclose(second.values.to_numpy(), np.array([10.0, 2.0, 3.0], dtype=np.float64)) - - detached = first_values.to_numpy().copy() - module.deallocate_current() - assert first_values.to_numpy() is None - np.testing.assert_allclose(detached, np.array([10.0, 2.0, 3.0], dtype=np.float64)) - with pytest.raises(AttributeError): - module.current = second - - generated_fortran = (result.output_dir / "bind_c_module_aliased_proxy_wrapper.f90").read_text(encoding="utf-8") - assert "c_loc(native_current)" in generated_fortran - - def test_derived_module_constant_returns_independent_owned_values(tmp_path: Path): native_object = _compile_native_object(DERIVED_CONSTANT_SOURCE, tmp_path / "native") contract = tmp_path / "contract" / "fmodule_vars_f90.pyi" @@ -274,6 +159,8 @@ def test_derived_module_constant_returns_independent_owned_values(tmp_path: Path assert second.r == np.int32(0) assert module.black.r == np.int32(0) assert module.black_sum() == np.int32(0) + with pytest.raises(AttributeError): + module.black = second bridge = (result.output_dir / "bind_c_fmodule_vars_f90_wrapper.f90").read_text(encoding="utf-8") assert "result = c_null_ptr" in bridge @@ -367,33 +254,3 @@ def test_value_copy_and_optional_derived_inputs_match_source_oracle(tmp_path: Pa bridge = (result.output_dir / "bind_c_derived_value_arguments_wrapper.f90").read_text(encoding="utf-8") assert "type(prik_type_point), pointer :: value" in bridge assert "native_score_by_value(value)" in bridge - - -def test_borrowed_child_retains_owner_and_finalizes_exactly_once(tmp_path: Path): - source = tmp_path / "source" / "derived_borrowed_finalizer.f90" - source.parent.mkdir() - source.write_text(BORROWED_FINALIZER_SOURCE, encoding="utf-8") - native_object = _compile_native_object(source, tmp_path / "native") - contract = tmp_path / "contract" / "derived_borrowed_finalizer.pyi" - contract.parent.mkdir() - contract.write_text(BORROWED_FINALIZER_CONTRACT, encoding="utf-8") - result = build_pyi_extension( - contract, - native_objects=[native_object], - native_include_dirs=[native_object.parent], - output_dir=tmp_path / "build", - ) - module = _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) - - owner = module.make_parent() - module.reset_final_count() - borrowed = owner.value - assert borrowed._prik_owner is owner - del owner - gc.collect() - assert module.get_final_count() == np.int32(0) - - del borrowed - gc.collect() - gc.collect() - assert module.get_final_count() == np.int32(1) diff --git a/tests/fortran/derived_types/end_to_end/test_generic_constructor.py b/tests/fortran/derived_types/end_to_end/test_generic_constructor.py index 2e2f90f08..c7fdb68f3 100644 --- a/tests/fortran/derived_types/end_to_end/test_generic_constructor.py +++ b/tests/fortran/derived_types/end_to_end/test_generic_constructor.py @@ -32,7 +32,11 @@ def test_type_without_a_constructor_interface_keeps_keyword_fields(module): def test_constructor_interface_overloads_init_from_its_specifics(module): - """`interface `: each specific becomes an accepted signature.""" + """`interface `: each specific becomes an accepted signature. + + Each accepted signature produces its own wrapper-owned instance, and a call + matching no specific is refused rather than guessed at. + """ empty = module.Box() from_count = module.Box(np.int32(7)) from_value = module.Box(np.float64(2.5)) @@ -41,23 +45,14 @@ def test_constructor_interface_overloads_init_from_its_specifics(module): assert (from_count.count, from_count.value) == (np.int32(7), np.float64(7.0)) assert (from_value.count, from_value.value) == (np.int32(2), np.float64(2.5)) + from_count.count = np.int32(9) + assert empty.count == np.int32(0) + assert module.Box(np.int32(7)).count == np.int32(7) -def test_constructor_overload_rejects_an_unmatched_signature(module): - """A call matching no specific is refused rather than guessed at.""" with pytest.raises(TypeError, match="no matching overload"): module.Box("not a supported signature") -def test_constructed_instances_are_independent_wrapper_objects(module): - """Each accepted signature produces its own wrapper-owned instance.""" - first = module.Box(np.int32(1)) - second = module.Box(np.int32(2)) - - assert first is not second - first.count = np.int32(9) - assert second.count == np.int32(2) - - def test_constructor_contract_states_no_redundant_link_name(tmp_path: Path): """A constructor's native generic is named for its type, so `@bind` is omitted. diff --git a/tests/fortran/derived_types/end_to_end/test_module_derived_aliases.py b/tests/fortran/derived_types/end_to_end/test_module_derived_aliases.py index 39e5e6adc..a1260a808 100644 --- a/tests/fortran/derived_types/end_to_end/test_module_derived_aliases.py +++ b/tests/fortran/derived_types/end_to_end/test_module_derived_aliases.py @@ -43,6 +43,8 @@ def test_aliased_derived_module_object_borrows_native_state( current = module.current assert isinstance(current, module.Box) + assert current is not module.current + assert current._prik_owner is module values = current.values assert isinstance(values, AllocatableArray) assert values.owner is current @@ -76,7 +78,10 @@ def test_aliased_derived_module_object_borrows_native_state( assert "c_loc(native_current)" in bridge_source assert "bind_c_set_current" not in bridge_source + detached = values.to_numpy().copy() module.deallocate_current() + assert values.to_numpy() is None + np.testing.assert_allclose(detached, np.array([10.0, 2.0, 3.0], dtype=np.float64)) current_values = module.current.values assert isinstance(current_values, AllocatableArray) assert current_values.allocated is False diff --git a/tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py b/tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py index 605c2c478..4ec5379b3 100644 --- a/tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py +++ b/tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py @@ -126,119 +126,80 @@ def _matrix_cell_is_legal(case: ActualCase, dummy: str) -> bool: return case.storage == "allocatable" -@pytest.mark.parametrize("case", ACTUAL_CASES, ids=lambda case: case.name) -@pytest.mark.parametrize("dummy", tuple(DUMMY_READERS)) -def test_all_sixty_actual_dummy_cells(scalar_matrix, case: ActualCase, dummy: str): - """Exercise every cell of the documented 10-row by 6-column matrix.""" - module = scalar_matrix.module - module.reset_state() - actual = _actual(module, case) - reader = getattr(module, DUMMY_READERS[dummy]) - - if _matrix_cell_is_legal(case, dummy): - assert reader(actual) == case.expected - else: - with pytest.raises(TypeError, match="allocatable-derived-actual-required"): - reader(actual) - - -@pytest.mark.parametrize("case", ACTUAL_CASES, ids=lambda case: case.name) -def test_reassociable_pointer_dummy_requires_pointer_storage(scalar_matrix, case: ActualCase): - """Known pointer writeback accepts P rows and rejects every nonpointer row.""" - module = scalar_matrix.module - module.reset_state() - actual = _actual(module, case) - - if case.storage == "pointer": - returned = module.set_pointer(actual, np.int32(2)) - assert returned is actual - assert actual.value == 60 - else: - with pytest.raises(TypeError, match="pointer-derived-actual-required"): - module.set_pointer(actual, np.int32(2)) - - -@dataclass(frozen=True) -class EmptyCase: - name: str - storage: str - - -EMPTY_CASES = ( - EmptyCase("nonmodule_allocatable", "allocatable"), - EmptyCase("module_allocatable", "allocatable"), - EmptyCase("module_allocatable_target", "allocatable"), - EmptyCase("nonmodule_pointer", "pointer"), - EmptyCase("module_pointer", "pointer"), -) +# Rows with an empty state; a module row names its variable, which the fixture clears. +EMPTY_ROWS = { + "nonmodule_allocatable": None, + "module_allocatable": "allocatable_module", + "module_allocatable_target": "allocatable_target_module", + "nonmodule_pointer": None, + "module_pointer": "pointer_module", +} -def _empty_actual(module, case: EmptyCase): +def _empty_actual(module, case: ActualCase): + """Construct the row with no payload: an empty holder or a cleared module origin.""" module.reset_state() if case.name == "nonmodule_allocatable": return module.make_allocatable_item(np.int32(1), False) - if case.name == "module_allocatable": - value = module.allocatable_module - module.clear_allocatable_module() - return value - if case.name == "module_allocatable_target": - value = module.allocatable_target_module - module.clear_allocatable_target_module() - return value if case.name == "nonmodule_pointer": return module.make_pointer_item(np.int32(0)) - if case.name == "module_pointer": - value = module.pointer_module - module.clear_pointer_module() - return value - raise AssertionError(f"unhandled empty case {case.name!r}") + variable = EMPTY_ROWS[case.name] + value = getattr(module, variable) + getattr(module, f"clear_{variable}")() + return value -@pytest.mark.parametrize("case", EMPTY_CASES, ids=lambda case: case.name) -@pytest.mark.parametrize("dummy", tuple(DUMMY_READERS)) -def test_empty_descriptor_states_follow_dummy_requirements(scalar_matrix, case: EmptyCase, dummy: str): +def _assert_empty_row_follows_dummy_requirements(module, case: ActualCase) -> None: """Absent payloads fail only where a payload, not a descriptor, is required.""" - module = scalar_matrix.module - actual = _empty_actual(module, case) - reader = getattr(module, DUMMY_READERS[dummy]) - - if dummy in {"O", "T", "V"}: - with pytest.raises(ValueError, match=r"derived payload.*not present"): - reader(actual) - return - if dummy == "P": - if case.storage == "pointer": - assert reader(actual) == -1 + for dummy, reader_name in DUMMY_READERS.items(): + actual = _empty_actual(module, case) + reader = getattr(module, reader_name) + descriptor_matches = (dummy == "P" and case.storage == "pointer") or ( + dummy in {"A", "AT"} and case.storage == "allocatable" + ) + if descriptor_matches: + assert reader(actual) == -1, dummy + elif dummy in {"A", "AT"}: + with pytest.raises(TypeError, match="allocatable-derived-actual-required"): + reader(actual) else: with pytest.raises(ValueError, match=r"derived payload.*not present"): reader(actual) - return - if case.storage == "allocatable": - assert reader(actual) == -1 - else: - with pytest.raises(TypeError, match="allocatable-derived-actual-required"): - reader(actual) -def test_empty_module_getters_return_persistent_live_proxies(scalar_matrix): +@pytest.mark.parametrize("case", ACTUAL_CASES, ids=lambda case: case.name) +def test_every_actual_row_meets_every_dummy_form(scalar_matrix, case: ActualCase): + """One documented actual row against all six dummy columns, P writeback, and its empty state. + + The row is the storage carrier (direct, holder, or module origin); each + column is a distinct native interface, so all cells stay asserted. + """ module = scalar_matrix.module + for dummy, reader_name in DUMMY_READERS.items(): + module.reset_state() + actual = _actual(module, case) + reader = getattr(module, reader_name) + if _matrix_cell_is_legal(case, dummy): + assert reader(actual) == case.expected, dummy + else: + with pytest.raises(TypeError, match="allocatable-derived-actual-required"): + reader(actual) + + # Known pointer writeback accepts P rows and rejects every nonpointer row. module.reset_state() - old_allocatable = module.allocatable_module - old_pointer = module.pointer_module - module.clear_allocatable_module() - module.clear_pointer_module() + actual = _actual(module, case) + if case.storage == "pointer": + assert module.set_pointer(actual, np.int32(2)) is actual + assert actual.value == 60 + else: + with pytest.raises(TypeError, match="pointer-derived-actual-required"): + module.set_pointer(actual, np.int32(2)) - for value in (old_allocatable, module.allocatable_module): - assert isinstance(value, module.item) - with pytest.raises(ReferenceError, match="not currently present"): - _ = value.value - for value in (old_pointer, module.pointer_module): - assert isinstance(value, module.item) - with pytest.raises(ReferenceError, match="not currently present"): - _ = value.value + if case.name in EMPTY_ROWS: + _assert_empty_row_follows_dummy_requirements(module, case) -def test_wrapper_owned_empty_holders_can_be_filled_without_replacement(scalar_matrix): +def test_wrapper_owned_empty_holders_fill_in_place_and_pointer_holders_track_their_target(scalar_matrix): module = scalar_matrix.module module.reset_state() allocatable = module.make_allocatable_item(np.int32(1), False) @@ -257,12 +218,7 @@ def test_wrapper_owned_empty_holders_can_be_filled_without_replacement(scalar_ma assert module.set_pointer(pointer, np.int32(2)) is pointer assert (allocatable.value, allocatable_target.value, pointer.value) == (7, 8, 60) - -def test_pointer_holder_retains_native_owner_and_tracks_allocated_target_lifetime(scalar_matrix): - module = scalar_matrix.module - module.reset_state() - pointer = module.make_pointer_item(np.int32(0)) - + # A pointer holder retains its native owner and follows reassociation and nullification. assert pointer._prik_owner is module assert module.set_pointer(pointer, np.int32(3)) is pointer assert pointer.value == 70 @@ -277,6 +233,15 @@ def test_module_descriptor_transactions_preserve_empty_and_recreated_state(scala allocatable = module.allocatable_module pointer = module.pointer_module + # Clearing native storage leaves every proxy, old or new, a live typed proxy. + module.clear_allocatable_module() + module.clear_pointer_module() + for value in (allocatable, module.allocatable_module, pointer, module.pointer_module): + assert isinstance(value, module.item) + with pytest.raises(ReferenceError, match="not currently present"): + _ = value.value + + module.reset_state() assert module.set_allocatable(allocatable, np.int32(-1)) is allocatable with pytest.raises(ReferenceError, match="not currently present"): _ = allocatable.value @@ -343,6 +308,9 @@ def test_sequence_derived_value_uses_the_same_typed_opaque_call_path(scalar_matr assert isinstance(value, module.sequence_item) assert module.read_sequence_value(value) == 23 + # A contract class declared without `__init__` has no Python constructor. + with pytest.raises(TypeError): + module.item() def test_qualified_same_short_name_types_keep_exact_native_identity(scalar_matrix): @@ -360,15 +328,12 @@ def test_qualified_same_short_name_types_keep_exact_native_identity(scalar_matri with pytest.raises(TypeError, match=expected): module.read_qualified(*values) - -def test_module_origins_from_separate_modules_keep_type_specific_callbacks(scalar_matrix): - module = scalar_matrix.module - left = scalar_matrix.left_module.state - right = scalar_matrix.right_module.state - - assert module.read_qualified(left, right) == 307 + # Module-state origins from the two modules keep their type-specific origin callbacks. + left_state = scalar_matrix.left_module.state + right_state = scalar_matrix.right_module.state + assert module.read_qualified(left_state, right_state) == 307 with pytest.raises(TypeError, match=r"left_item.*left"): - module.read_qualified(right, left) + module.read_qualified(right_state, left_state) def test_duplicate_origins_share_reads_and_reject_writes_before_native_call(scalar_matrix): @@ -447,20 +412,6 @@ def hold(): pytest.fail("module object origin never entered its active scoped state") -def test_concurrent_origin_use_is_rejected_and_restored(scalar_matrix): - module = scalar_matrix.module - module.reset_state() - value = module.allocatable_module - thread, errors = _start_busy_origin(module, value) - try: - with pytest.raises(RuntimeError, match=r"origin failure.*status 2"): - module.read_allocatable(value) - finally: - thread.join() - assert errors == [] - assert value.value == 31 - - def test_later_acquisition_failure_rolls_back_earlier_origins(scalar_matrix): module = scalar_matrix.module module.reset_state() diff --git a/tests/fortran/derived_types/end_to_end/test_types_across_modules.py b/tests/fortran/derived_types/end_to_end/test_types_across_modules.py index 7c7b6560d..b26d01c8b 100644 --- a/tests/fortran/derived_types/end_to_end/test_types_across_modules.py +++ b/tests/fortran/derived_types/end_to_end/test_types_across_modules.py @@ -42,36 +42,30 @@ def modules(tmp_path_factory: pytest.TempPathFactory): return module.shapes, module.ops -def test_a_returned_type_is_the_declaring_module_class(modules): +def test_results_and_components_are_the_declaring_module_class(modules): + """A returned, allocatable-returned, or component value is the declaring module's class.""" shapes, ops = modules item = ops.boxed(np.int32(3)) - assert type(item) is shapes.Box assert item.value == 3 - -def test_an_allocatable_result_is_the_declaring_module_class(modules): - shapes, ops = modules - item = ops.maybe_box(np.int32(4)) - assert type(item) is shapes.Box assert item.value == 4 - -def test_a_callback_result_is_checked_against_the_declaring_module_class(modules): - shapes, ops = modules - - assert ops.total(lambda: shapes.Box(value=np.int32(9))) == 9 + holder = ops.Holder() + assert type(holder.inner) is shapes.Box + holder.inner = shapes.Box(value=np.int32(12)) + assert holder.inner.value == 12 -def test_a_callback_argument_is_the_declaring_module_class(modules): +def test_callback_arguments_and_results_use_the_declaring_module_class(modules): shapes, ops = modules seen = [] + assert ops.total(lambda: shapes.Box(value=np.int32(9))) == 9 ops.visit(lambda item: seen.append((type(item), int(item.value)))) - assert seen == [(shapes.Box, 41)] @@ -105,15 +99,6 @@ def test_a_generic_dispatches_on_the_declaring_module_class(modules): assert ops.weigh(np.int32(5)) == -5 -def test_a_component_of_another_module_type_is_that_module_class(modules): - shapes, ops = modules - holder = ops.Holder() - - assert type(holder.inner) is shapes.Box - holder.inner = shapes.Box(value=np.int32(12)) - assert holder.inner.value == 12 - - def test_two_modules_may_each_declare_a_type_spelled_alike(tmp_path: Path): """Each `box` keeps its own class, constructor, and helpers. diff --git a/tests/fortran/derived_types/parsing/test_derived_field_syntax.py b/tests/fortran/derived_types/parsing/test_derived_field_syntax.py index 4ddc11784..f850a4069 100644 --- a/tests/fortran/derived_types/parsing/test_derived_field_syntax.py +++ b/tests/fortran/derived_types/parsing/test_derived_field_syntax.py @@ -24,33 +24,22 @@ def test_derived_type_field_default_initializers_are_preserved(): assert fields["enabled"].symbolic_value == ".true." -def test_malformed_type_bound_declaration_raises(): - code = """ +@pytest.mark.parametrize("declaration", ["procedure broken_binding", "FINAL :: 123"]) +def test_malformed_type_bound_declaration_reports_its_location_and_code(declaration: str): + code = f""" module bad_binding_mod type :: t contains - procedure broken_binding + {declaration} end type t end module bad_binding_mod """ - with pytest.raises(FortranParseError, match="Unsupported or malformed type-bound declaration"): + with pytest.raises(FortranParseError) as error: parse_fortran_file(code, filename="bad_binding.f90") - -def test_bind_c_derived_type_attribute_and_component_order_are_preserved(): - code = """ -module bind_c_type_mod - use iso_c_binding - type, bind(C) :: sample - real(c_double) :: x - integer(c_int) :: tag - logical(c_bool) :: active - end type sample -end module bind_c_type_mod -""" - - dtype = parse_fortran_file(code, filename="bind_c_type.f90").modules[0].derived_types[0] - - assert dtype.attributes == ["bind(c)"] - assert [field.name for field in dtype.fields] == ["x", "tag", "active"] + assert error.value.base_message == f"Unsupported or malformed type-bound declaration in type 't': {declaration}" + assert error.value.filename == "bad_binding.f90" + assert error.value.line_number == 5 + assert error.value.source_line.strip() == declaration + assert error.value.code == "PARSE_UNSUPPORTED_TYPE_BOUND_DECLARATION" diff --git a/tests/fortran/derived_types/parsing/test_derived_type_declarations.py b/tests/fortran/derived_types/parsing/test_derived_type_declarations.py index 132b923c3..3d75427b5 100644 --- a/tests/fortran/derived_types/parsing/test_derived_type_declarations.py +++ b/tests/fortran/derived_types/parsing/test_derived_type_declarations.py @@ -26,22 +26,3 @@ def test_derived_type_fields_and_methods_detection(): assert t.fields[1].shape == ["3"] assert t.fields[2].base_type == "derived" assert t.fields[2].kind == "vector" - - -def test_derived_type_extends_and_attributes(): - code = """ -module m - type :: base_t - end type base_t - type, extends(base_t), abstract :: child_t - integer :: id - contains - procedure :: run - end type child_t -end module m -""" - dt = parse_fortran_file(code).modules[0].derived_types[1] - assert dt.name == "child_t" - assert dt.extends is not None - assert getattr(dt.extends, "name", None) == "base_t" - assert "abstract" in dt.attributes diff --git a/tests/fortran/derived_types/parsing/test_derived_type_errors.py b/tests/fortran/derived_types/parsing/test_derived_type_errors.py index be6864a99..f6ed698c8 100644 --- a/tests/fortran/derived_types/parsing/test_derived_type_errors.py +++ b/tests/fortran/derived_types/parsing/test_derived_type_errors.py @@ -33,21 +33,6 @@ def test_unknown_type_in_derived_type_field_raises_parse_error(): parse_fortran_file(code, filename="bad.f90") -def test_derived_type_fields_have_known_types(): - code = """ -module m - type :: point - real :: x - real :: y - integer :: id - end type point -end module m -""" - parsed = parse_fortran_file(code) - for field in parsed.modules[0].derived_types[0].fields: - assert field.base_type != "unknown" - - def test_duplicate_field_in_derived_type_raises_parse_error(): code = """ module m @@ -59,18 +44,3 @@ def test_duplicate_field_in_derived_type_raises_parse_error(): """ with pytest.raises(FortranParseError, match="Duplicate field"): parse_fortran_file(code, filename="dup_field.f90") - - -def test_derived_type_unique_fields_no_error(): - code = """ -module m - type :: point - real :: x - real :: y - real :: z - end type point -end module m -""" - parsed = parse_fortran_file(code, filename="ok.f90") - assert len(parsed.modules[0].derived_types) == 1 - assert len(parsed.modules[0].derived_types[0].fields) == 3 diff --git a/tests/fortran/derived_types/parsing/test_type_bound_diagnostics.py b/tests/fortran/derived_types/parsing/test_type_bound_diagnostics.py deleted file mode 100644 index 603293fde..000000000 --- a/tests/fortran/derived_types/parsing/test_type_bound_diagnostics.py +++ /dev/null @@ -1,26 +0,0 @@ -"""Tests split by stable ownership concept from `test_source_form_and_diagnostics_regressions.py`.""" - -import pytest -from prik.parsers.fortran import FortranParseError -from prik.parsers.fortran.models import FortranDerivedType -from prik.parsers.fortran.parser import FortranParser - - -def test_malformed_type_bound_declaration_diagnostic_preserves_public_metadata(): - parser = FortranParser() - dtype = FortranDerivedType("state_t") - - with pytest.raises(FortranParseError) as error: - parser._parse_derived_type_contains_line( - "FINAL :: 123", - dtype, - filename="type_bound_contract.f90", - lineno=9, - source_line="FINAL :: 123", - ) - - assert error.value.base_message == "Unsupported or malformed type-bound declaration in type 'state_t': FINAL :: 123" - assert error.value.filename == "type_bound_contract.f90" - assert error.value.line_number == 9 - assert error.value.source_line == "FINAL :: 123" - assert error.value.code == "PARSE_UNSUPPORTED_TYPE_BOUND_DECLARATION" diff --git a/tests/fortran/derived_types/pipeline/test_generated_derived_contracts.py b/tests/fortran/derived_types/pipeline/test_generated_derived_contracts.py deleted file mode 100644 index 6d2d1e779..000000000 --- a/tests/fortran/derived_types/pipeline/test_generated_derived_contracts.py +++ /dev/null @@ -1,39 +0,0 @@ -"""Reviewed generated contracts for derived-type feature subjects.""" - -from __future__ import annotations - -from pathlib import Path - -import pytest - -from tests.fortran._support.generated_contracts import ( - GeneratedContractCase, - assert_generated_contract_matches_fixture, - contract_case_id, -) - -FIXTURES = Path(__file__).parents[1] / "end_to_end" / "fixtures" -CONTRACT_ROOT = FIXTURES / "contracts" -CASES = tuple( - GeneratedContractCase( - name, - (FIXTURES / "native" / f"{name}.f90",), - CONTRACT_ROOT / name, - ) - for name in ( - "fbind_c_derived_layout_f90", - "fborrowed_finalizer_f90", - "fclasses_f90", - "fconstructors_f90", - "fderived_boundary_f90", - "finheritance_f90", - ) -) - - -@pytest.mark.parametrize("case", CASES, ids=contract_case_id) -def test_generated_derived_contract_matches_fixture( - case: GeneratedContractCase, - tmp_path: Path, -): - assert_generated_contract_matches_fixture(case, tmp_path) diff --git a/tests/fortran/derived_types/policy/test_derived_accessor_policy.py b/tests/fortran/derived_types/policy/test_derived_accessor_policy.py index 4e3fe2e01..3733e2d6e 100644 --- a/tests/fortran/derived_types/policy/test_derived_accessor_policy.py +++ b/tests/fortran/derived_types/policy/test_derived_accessor_policy.py @@ -1,73 +1,26 @@ """Tests split by stable ownership concept from `test_handle_policy_dispatch.py`.""" from prik.semantics.models import ( - RESOLVED_GETTER_OWNERSHIP_POLICY_METADATA, - RESOLVED_OWNERSHIP_POLICY_METADATA, RESOLVED_SETTER_OWNERSHIP_POLICY_METADATA, SemanticClass, - SemanticConstraint, - SemanticDestructor, SemanticField, SemanticModule, - SemanticVariable, ) from prik.policy.ownership import ( - AssignmentMode, - CodegenAction, - DestructionPolicy, ObjectKind, - OwnershipOwner, SetterAction, - StorageMode, TransferMode, ) from prik.semantics.ownership_metadata import set_ownership_metadata from prik.policy.completion import complete_semantic_policies from tests.fortran._support.ownership_policy import ( - _array_type, _derived_type, - _scalar_type, parse_pyi_text, ) from prik.semantics.models import ( RESOLVED_DERIVED_TYPE_POLICY_METADATA, - RESOLVED_MODULE_VARIABLE_POLICY_METADATA, ) -from prik.policy.models import ModuleObjectAccessMechanism - - -def test_abstract_type_completes_as_a_non_instantiable_derived_policy(): - """An abstract type is supported and records that it has no instances.""" - semantic_class = SemanticClass( - "shape", - metadata={ - "fortran_type_attributes": ["abstract"], - "fortran_deferred_bindings": ["area"], - }, - ) - module = SemanticModule("shapes", classes=[semantic_class]) - - complete_semantic_policies(module) - - policy = semantic_class.metadata[RESOLVED_DERIVED_TYPE_POLICY_METADATA] - assert policy.supported is True - assert policy.blockers == () - assert policy.abstract is True - assert policy.deferred_bindings == ("area",) - - -def test_native_destructor_reaches_completed_derived_type_policy(): - semantic_class = SemanticClass( - "owned_buffer", - destructors=[SemanticDestructor("release_owned_buffer")], - ) - module = SemanticModule("buffers", classes=[semantic_class]) - - complete_semantic_policies(module) - - policy = semantic_class.metadata[RESOLVED_DERIVED_TYPE_POLICY_METADATA] - assert policy.destructors == ("release_owned_buffer",) def test_deferred_binding_without_an_abstract_type_is_refused(): @@ -82,108 +35,6 @@ def test_deferred_binding_without_an_abstract_type_is_refused(): assert policy.blockers == ("deferred type-bound procedure 'area' needs a declaring abstract type",) -def test_derived_field_setter_policy_uses_value_copy_write_through(): - module = SemanticModule( - name="layout", - classes=[ - SemanticClass("point"), - SemanticClass("tagged_point", fields=[SemanticField("position", _derived_type("point"))]), - ], - ) - - complete_semantic_policies(module) - - setter = module.classes[1].fields[0].metadata[RESOLVED_SETTER_OWNERSHIP_POLICY_METADATA] - assert setter.kind is ObjectKind.DERIVED_TYPE - assert setter.assignment_mode is AssignmentMode.VALUE_COPY - assert setter.setter_action is SetterAction.WRITE_THROUGH - - -def test_aliased_derived_module_object_is_borrowed_and_rejects_replacement(): - module = SemanticModule( - name="state", - variables=[SemanticVariable("current", _derived_type("box", metadata={"aliased": True}))], - classes=[SemanticClass("box", fields=[SemanticField("value", _scalar_type())])], - ) - - complete_semantic_policies(module) - - variable = module.variables[0] - storage = variable.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA] - getter = variable.metadata[RESOLVED_GETTER_OWNERSHIP_POLICY_METADATA] - setter = variable.metadata[RESOLVED_SETTER_OWNERSHIP_POLICY_METADATA] - assert storage.owner is OwnershipOwner.NATIVE - assert storage.transfer is TransferMode.BORROWED_VIEW - assert storage.boundary_storage_mode is StorageMode.ALIAS - assert getter.codegen_action is CodegenAction.BORROWED_VIEW - assert setter.setter_action is SetterAction.REJECT_REPLACEMENT - - -def test_plain_derived_module_object_completes_live_member_proxy_policy(): - module = SemanticModule( - name="state", - variables=[SemanticVariable("current", _derived_type("box"))], - classes=[ - SemanticClass("point", fields=[SemanticField("x", _scalar_type())]), - SemanticClass( - "box", - fields=[ - SemanticField("value", _scalar_type()), - SemanticField("origin", _derived_type("point")), - SemanticField("values", _array_type(allocatable=True, metadata={"aliased": True})), - ], - ), - ], - ) - - complete_semantic_policies(module) - - variable = module.variables[0] - storage = variable.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA] - getter = variable.metadata[RESOLVED_GETTER_OWNERSHIP_POLICY_METADATA] - setter = variable.metadata[RESOLVED_SETTER_OWNERSHIP_POLICY_METADATA] - assert storage.owner is OwnershipOwner.NATIVE - assert storage.transfer is TransferMode.BORROWED_VIEW - assert storage.codegen_action is CodegenAction.BORROWED_VIEW - assert getter.codegen_action is CodegenAction.BORROWED_VIEW - assert setter.setter_action is SetterAction.REJECT_REPLACEMENT - policy = variable.metadata[RESOLVED_MODULE_VARIABLE_POLICY_METADATA] - assert policy.derived.access is ModuleObjectAccessMechanism.MEMBER_PROXY - assert policy.owner_path == "state.current" - - -def test_derived_module_constant_uses_wrapper_owned_copy_without_setter(): - constant_type = _derived_type("rgb_color") - constant_type.constraints.append(SemanticConstraint("Constant")) - module = SemanticModule( - name="colors", - variables=[SemanticVariable("black", constant_type)], - classes=[ - SemanticClass( - "rgb_color", - fields=[ - SemanticField("r", _scalar_type()), - SemanticField("g", _scalar_type()), - SemanticField("b", _scalar_type()), - ], - ) - ], - ) - - complete_semantic_policies(module) - - variable = module.variables[0] - storage = variable.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA] - getter = variable.metadata[RESOLVED_GETTER_OWNERSHIP_POLICY_METADATA] - setter = variable.metadata[RESOLVED_SETTER_OWNERSHIP_POLICY_METADATA] - assert storage.kind is ObjectKind.DERIVED_TYPE - assert storage.owner is OwnershipOwner.WRAPPER - assert storage.transfer is TransferMode.WRAPPER_INSTANCE - assert storage.destruction is DestructionPolicy.WRAPPER_DEALLOC - assert getter.transfer is TransferMode.WRAPPER_INSTANCE - assert setter.setter_action is SetterAction.OMIT - - def test_explicit_borrowed_derived_field_setter_rejects_replacement(): child_type = _derived_type("child") set_ownership_metadata( diff --git a/tests/fortran/derived_types/policy/test_derived_policy_defaults.py b/tests/fortran/derived_types/policy/test_derived_policy_defaults.py index 010baa160..5fde44c36 100644 --- a/tests/fortran/derived_types/policy/test_derived_policy_defaults.py +++ b/tests/fortran/derived_types/policy/test_derived_policy_defaults.py @@ -1,13 +1,5 @@ """Tests split by stable ownership concept from `test_handle_policy_dispatch.py`.""" -from prik.semantics.models import ( - SemanticArgument, - SemanticClass, - SemanticField, - SemanticFunction, - SemanticModule, - SemanticVariable, -) from prik.policy.ownership import ( CodegenAction, DestructionPolicy, @@ -16,10 +8,8 @@ default_ownership_policy, ) from tests.fortran._support.ownership_policy import ( - _array_type, _derived_type, _hidden_output_context, - _scalar_type, _writable_argument_context, ) @@ -43,42 +33,3 @@ def test_immutable_derived_output_selects_wrapper_instance_and_replacement_block ) assert replacement.is_blocked assert replacement.blocker == "immutable derived replacement is not implemented" - - -def test_recursive_module_policy_map_includes_nested_fields_and_functions(): - module = SemanticModule( - name="geometry", - variables=[ - SemanticVariable( - "values", - _array_type(allocatable=True, metadata={"aliased": True}), - ) - ], - classes=[ - SemanticClass( - "particle", - fields=[SemanticField("origin", _derived_type("point"))], - classes=[ - SemanticClass( - "buffer", - fields=[SemanticField("values", _array_type(allocatable=True))], - ) - ], - ) - ], - functions=[ - SemanticFunction( - "build", - arguments=[SemanticArgument("n", _scalar_type())], - return_type=_array_type(allocatable=True), - ) - ], - ) - - decisions = default_ownership_policy.decide_semantic_module(module) - - assert decisions["geometry.values"].owner is OwnershipOwner.NATIVE - assert decisions["geometry.particle.origin"].owner is OwnershipOwner.WRAPPER - assert decisions["geometry.particle.buffer.values"].transfer is TransferMode.BORROWED_VIEW - assert decisions["geometry.build.n"].transfer is TransferMode.CALL_LOCAL - assert decisions["geometry.build.return"].transfer is TransferMode.WRAPPER_INSTANCE diff --git a/tests/fortran/derived_types/policy/test_merged_contract_names.py b/tests/fortran/derived_types/policy/test_merged_contract_names.py deleted file mode 100644 index 3495d0ccf..000000000 --- a/tests/fortran/derived_types/policy/test_merged_contract_names.py +++ /dev/null @@ -1,56 +0,0 @@ -"""A build names a type once, however many of its modules use it. - -A build merges its source modules into one, so a type one of them imports from -another is declared by the merged module itself. It is not an import there, -and it must not compete with its own declaration for a name. -""" - -from pathlib import Path - -from prik.parsers.fortran import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules -from prik.policy.exports import complete_python_export_policy -from prik.semantics import models -from prik.semantics.fortran2ir import fortran_project_to_semantic_modules - -SOURCES = """\ -module shapes - implicit none - type :: box - integer :: value = 0 - end type box -end module shapes - -module ops - use shapes, only: box - implicit none - private - public :: boxed -contains - function boxed(v) result(out) - integer, intent(in) :: v - type(box) :: out - out%value = v - end function boxed -end module ops -""" - - -def test_a_merged_build_names_a_type_its_modules_share_once(tmp_path: Path): - """Named like the module declaring the type, the build still spells it `Box`. - - `ops` uses `box` without publishing it. Counting that use as an import put - a second `Box` in the ledger ahead of the declaration, which took `Box_2`. - """ - (tmp_path / "project.f90").write_text(SOURCES, encoding="utf-8") - modules = fortran_project_to_semantic_modules(parse_fortran_project(str(tmp_path))) - _apply_source_python_exports(modules) - merged = _merge_wrapper_modules(modules, name="shapes") - - complete_python_export_policy(merged) - - box = next(item for item in merged.classes if item.name == "box") - boxed = next(item for item in merged.functions if item.name == "boxed") - assert models.completed_contract_name(box) == "Box" - assert boxed.return_type.metadata[models.CONTRACT_NAME_METADATA] == "Box" - assert "box" not in merged.metadata[models.CONTRACT_IMPORT_NAMES_METADATA] diff --git a/tests/fortran/derived_types/semantics/fixtures/native/derived_type.f90 b/tests/fortran/derived_types/semantics/fixtures/native/derived_type.f90 deleted file mode 100644 index bdef8a83b..000000000 --- a/tests/fortran/derived_types/semantics/fixtures/native/derived_type.f90 +++ /dev/null @@ -1,20 +0,0 @@ -module sparse_mod - -type :: sparse_matrix - integer :: nrows - integer :: ncols -end type - -contains - -subroutine multiply(A, x, y) - - type(sparse_matrix), intent(in) :: A - - real(8), intent(in) :: x(:) - - real(8), intent(out) :: y(:) - -end subroutine - -end module diff --git a/tests/fortran/derived_types/semantics/test_fortran_derived_semantics.py b/tests/fortran/derived_types/semantics/test_fortran_derived_semantics.py deleted file mode 100644 index 17f3882c6..000000000 --- a/tests/fortran/derived_types/semantics/test_fortran_derived_semantics.py +++ /dev/null @@ -1,368 +0,0 @@ -"""Tests split by stable ownership concept from `test_compile_time_values.py`.""" - -from pathlib import Path - -import pytest -from prik.printers import emit_module -from prik.parsers.fortran.models import ( - FortranArgument, - FortranDerivedType, - FortranFile, - FortranProcedureSignature, - FortranVariable, -) -from prik.semantics.fortran2ir import ( - FortranToIRConverter, - fortran_module_to_semantic_module, -) -from prik.semantics.models import ( - SemanticArgument, - SemanticConstraint, - SemanticFunction, - SemanticMethod, - SemanticType, -) -from prik.semantics.native_contract import native_contract_issues -from tests.fortran._support.semantic_conversion import get_class -from prik.parsers.fortran import parse_fortran_file as parse_fortran_source -from prik.pipeline.pyi import pyi_text_to_semantic_module as parse_pyi_text - -NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" - - -def test_converter_rejects_unsupported_inputs_and_missing_derived_type_names(): - converter = FortranToIRConverter() - - with pytest.raises(TypeError) as error: - converter.visit(object()) - assert str(error.value) == "Unsupported Fortran parse object: " - - with pytest.raises(TypeError, match="Unsupported Fortran parse object"): - converter.first_module(object()) - - with pytest.raises(ValueError) as error: - converter.first_module(FortranFile()) - assert str(error.value) == "Expected at least one Fortran module in parsed file" - - from_list = converter.first_module( - [ - FortranProcedureSignature( - name="inside", - kind="subroutine", - module="legacy_mod", - in_interface=True, - ), - FortranProcedureSignature(name="outside", kind="subroutine"), - ] - ) - assert from_list.name == "legacy_mod" - assert [proc.name for proc in from_list.procedures] == ["outside"] - - with pytest.raises(ValueError, match="missing concrete type name"): - converter.visit(FortranVariable(name="state", base_type="derived")) - - with pytest.raises(ValueError, match="Unknown Fortran datatype"): - converter.visit(FortranVariable(name="x", base_type="unknown")) - - with pytest.raises(ValueError) as error: - converter.visit(FortranVariable(name="x", base_type="real", kind="selected_real_kind(33)")) - assert str(error.value) == "Unsupported Fortran semantic type for variable 'x': real(kind=selected_real_kind(33))" - - -def test_converter_covers_derived_dispatch_methods_and_kind_edges(): - converter = FortranToIRConverter() - callback = SemanticFunction( - name="advance", - native_name="advance_impl", - arguments=[SemanticArgument("state", SemanticType("particle_t"))], - return_type=SemanticType("Int32"), - contracts=[SemanticConstraint("Pure")], - visibility="private", - ) - dtype = FortranDerivedType( - name="particle_t", - fields=[FortranArgument(name="id", base_type="integer")], - methods=["missing_binding", "advance"], - ) - - semantic_class = converter.visit(dtype, procedure_lookup={"advance": callback}) - - assert semantic_class.methods == [ - SemanticMethod( - name="advance", - native_name="advance_impl", - arguments=callback.arguments, - return_type=callback.return_type, - contracts=callback.contracts, - visibility="private", - passed_object_name="state", - passed_object_position=0, - ) - ] - assert converter.visit(FortranVariable(name="count", base_type="integer")).name == "Int32" - - -def test_derived_type_initializers_and_finalizers_reach_semantic_ir(): - source = """ -module lifecycle_mod - type :: state - integer :: count = 7 - contains - final :: cleanup - end type state -contains - subroutine cleanup(self) - type(state), intent(inout) :: self - end subroutine cleanup -end module lifecycle_mod -""" - - parsed = parse_fortran_source(source) - module = fortran_module_to_semantic_module(parsed) - state = module.classes[0] - - assert state.fields[0].default_value == "7" - assert state.fields[0].metadata["fortran_initializer"] == "7" - assert [item.name for item in state.destructors] == ["cleanup"] - assert state.destructors[0].origin.source_language == "fortran" - assert state.destructors[0].origin.source_kind == "destructor" - emitted = emit_module(module) - assert " @destroy\n def cleanup(self) -> None: ..." in emitted - reloaded = parse_pyi_text(emitted, module_name=module.name) - assert reloaded.classes[0].methods == [] - assert [item.name for item in reloaded.classes[0].destructors] == ["cleanup"] - assert native_contract_issues(reloaded) == [] - - -def test_bind_c_and_sequence_types_preserve_accessor_layout_metadata(): - source = """ -module layout_mod - use iso_c_binding - type, bind(C) :: point - real(c_double) :: x - integer(c_int) :: axis - end type point - type, bind(C) :: tagged_point - type(point) :: position - logical(c_bool) :: active - complex(c_double_complex) :: weight - end type tagged_point - type :: ordered_pair - sequence - integer :: first - integer :: second - end type ordered_pair -end module layout_mod -""" - - module = fortran_module_to_semantic_module(parse_fortran_source(source)) - point, tagged, ordered = module.classes - rendered = emit_module(module) - - assert point.metadata["fortran_type_attributes"] == ["bind(c)"] - assert '@native_abi("c")\nclass point:' in rendered - assert '@native_abi("c")\nclass tagged_point:' in rendered - assert "native_type" not in rendered - assert point.metadata["fortran_bind_c"] is True - assert point.metadata["fortran_layout_policy"] == "accessors" - assert point.metadata["fortran_direct_layout"] is False - assert point.metadata["fortran_component_order"] == ["x", "axis"] - assert point.metadata["fortran_component_facts"] == [ - { - "name": "x", - "source_type": "real(kind=c_double)", - "kind": "c_double", - "rank": 0, - "shape": [], - "allocatable": False, - "pointer": False, - "target": False, - }, - { - "name": "axis", - "source_type": "integer(kind=c_int)", - "kind": "c_int", - "rank": 0, - "shape": [], - "allocatable": False, - "pointer": False, - "target": False, - }, - ] - assert [field.name for field in tagged.fields] == ["position", "active", "weight"] - assert tagged.fields[0].origin.source_type == "type(point)" - assert tagged.fields[1].origin.source_type == "logical(kind=c_bool)" - assert tagged.fields[2].origin.source_type == "complex(kind=c_double_complex)" - assert ordered.metadata["fortran_type_attributes"] == ["sequence"] - assert ordered.metadata["fortran_sequence"] is True - assert ordered.metadata["fortran_layout_policy"] == "accessors" - - reloaded = parse_pyi_text(rendered, module_name=module.name) - assert reloaded.classes[0].metadata["fortran_bind_c"] is True - assert "fortran_sequence" not in reloaded.classes[2].metadata - - -def test_bind_c_derived_value_argument_is_accessor_routed(): - interoperable_source = """ -module bind_c_value_mod - use iso_c_binding - type, bind(C) :: point - real(c_double) :: x - end type point -contains - subroutine consume(value) bind(C) - type(point), value :: value - end subroutine consume -end module bind_c_value_mod -""" - interoperable = fortran_module_to_semantic_module(parse_fortran_source(interoperable_source)) - assert interoperable.classes[0].name == "point" - - -def test_derived_type(): - source = (NATIVE_FIXTURES / "derived_type.f90").read_text(encoding="utf-8") - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - cls = get_class(smod, "sparse_matrix") - - assert cls.name == "sparse_matrix" - - assert len(cls.fields) == 2 - - field_names = {f.name for f in cls.fields} - - assert "nrows" in field_names - assert "ncols" in field_names - - -def test_derived_type_inheritance(): - source = """ -module inheritance_mod - -type :: base_matrix -end type - -type, extends(base_matrix) :: sparse_matrix -end type - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - cls = get_class(smod, "sparse_matrix") - - assert "base_matrix" in cls.base_classes - - -def test_class_declarations_preserve_polymorphic_source_fact(): - source = """ -module polymorphic_source_mod - type :: base - contains - procedure :: touch - end type base -contains - subroutine touch(self) - class(base), intent(inout) :: self - end subroutine touch - subroutine accept(value) - class(base), intent(in) :: value - end subroutine accept -end module polymorphic_source_mod -""" - - module = FortranToIRConverter().visit(parse_fortran_source(source).modules[0]) - touch_self = module.functions[0].arguments[0].semantic_type - accept_value = module.functions[1].arguments[0].semantic_type - - assert touch_self.origin.source_type == "class(base)" - assert touch_self.metadata["fortran_polymorphic"] is True - assert module.functions[0].metadata["fortran_type_bound_target"] is True - assert module.functions[0].metadata["fortran_passed_object_name"] == "self" - assert accept_value.origin.source_type == "class(base)" - assert accept_value.metadata["fortran_polymorphic"] is True - - -def test_declared_type_accessibility_wins_over_the_module_default(): - """`type, public ::` states the type's own accessibility. - - A module-level `private` default sets accessibility for symbols that do not - state one; it must not hide a type whose declaration says `public`. - """ - module = fortran_module_to_semantic_module( - parse_fortran_source( - """ -module exports_mod - implicit none - private - type,public :: exported - integer :: n = 0 - end type exported - type :: defaulted - integer :: n = 0 - end type defaulted -end module exports_mod -""" - ) - ) - - visibility = {semantic_class.name: semantic_class.visibility for semantic_class in module.classes} - assert visibility == {"exported": "public", "defaulted": "private"} - - -def test_private_components_carry_their_hidden_accessibility(): - """The type's `private` statement is the default accessibility of its components.""" - module = fortran_module_to_semantic_module( - parse_fortran_source( - """ -module hidden_mod - implicit none - type,public :: partly - private - integer :: hidden = 0 - integer,public :: shown = 0 - end type partly -end module hidden_mod -""" - ) - ) - - partly = module.classes[0] - assert {field.name: field.visibility for field in partly.fields} == { - "hidden": "private", - "shown": "public", - } - - -def test_private_type_bound_procedures_stay_off_the_generated_class_surface(): - """A binding hidden by the `private` statement after `contains` is not a method.""" - module = fortran_module_to_semantic_module( - parse_fortran_source( - """ -module bindings_mod - implicit none - type,public :: gated - integer :: n = 0 - contains - private - procedure :: internal_step - procedure,public :: step => internal_step - end type gated -contains - subroutine internal_step(self) - class(gated),intent(inout) :: self - self%n = self%n + 1 - end subroutine internal_step -end module bindings_mod -""" - ) - ) - - gated = module.classes[0] - assert [method.name for method in gated.methods if method.visibility == "public"] == ["step"] diff --git a/tests/fortran/derived_types/semantics/test_imported_derived_semantics.py b/tests/fortran/derived_types/semantics/test_imported_derived_semantics.py index 263b4f378..318f6cce6 100644 --- a/tests/fortran/derived_types/semantics/test_imported_derived_semantics.py +++ b/tests/fortran/derived_types/semantics/test_imported_derived_semantics.py @@ -1,6 +1,5 @@ """Tests split by stable ownership concept from `test_compile_time_values.py`.""" -from dataclasses import asdict from prik.parsers.fortran import parse_fortran_project from prik.parsers.fortran.models import ( FortranArgument, @@ -11,127 +10,19 @@ FortranProject, FortranUseMapping, FortranUseStatement, - FortranVariable, ) from prik.semantics.fortran2ir import ( - FortranToIRConverter, fortran_file_to_semantic_modules, fortran_module_to_semantic_module, fortran_project_to_semantic_modules, ) from prik.semantics.models import ( EXTERNAL_TYPE_REF_METADATA, - SemanticField, - SemanticVariable, ) from tests.fortran._support.semantic_conversion import get_function from prik.parsers.fortran import parse_fortran_file as parse_fortran_source -def test_converter_preserves_imported_derived_contexts_through_dispatch_paths(): - converter = FortranToIRConverter() - imported_type = FortranVariable(name="state", base_type="derived", kind="local_state") - imported_argument = FortranArgument(name="arg", base_type="derived", kind="local_state") - local_field = FortranArgument(name="nested", base_type="derived", kind="container_t") - dtype = FortranDerivedType( - name="container_t", - module="consumer", - fields=[FortranArgument(name="state", base_type="derived", kind="local_state"), local_field], - methods=["step"], - ) - dtype.visibility = "private" - proc = FortranProcedureSignature( - name="step", - kind="subroutine", - module="consumer", - arguments=[imported_argument], - ) - module = FortranModule( - name="consumer", - uses=[ - FortranUseStatement("plain_mod"), - FortranUseStatement("types_mod", True, (FortranUseMapping(source="state_t", target="local_state"),)), - ], - variables=[FortranVariable(name="module_state", base_type="derived", kind="local_state")], - procedures=[proc], - derived_types=[dtype], - private_symbols=["container_t"], - ) - parsed_file = FortranFile(modules=[module]) - project = FortranProject(files=[parsed_file]) - context = converter._module_derived_type_context(module) - - semantic_module = converter.visit(module) - semantic_class = converter.visit(dtype, derived_type_context=context) - external_ref = { - "name": "state_t", - "local_name": "local_state", - "origin_module": "types_mod", - "wrapped": False, - "representation": "opaque", - } - - assert converter.visit(imported_type, derived_type_context=context).metadata["external_type_ref"] == external_ref - assert ( - converter.visit(imported_argument, derived_type_context=context).semantic_type.metadata[ - EXTERNAL_TYPE_REF_METADATA - ] - == external_ref - ) - assert converter.visit(proc, derived_type_context=context).arguments[0].semantic_type.metadata[ - "external_type_ref" - ] == (external_ref) - assert ( - converter.visit(parsed_file)[0].classes[0].fields[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] - == external_ref - ) - assert ( - converter.visit(project)[0].classes[0].fields[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] - == external_ref - ) - assert semantic_module.classes[0].fields[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] == external_ref - assert "external_type_ref" not in semantic_module.classes[0].fields[1].semantic_type.metadata - assert semantic_class.fields[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] == external_ref - assert isinstance(semantic_class.fields[0], SemanticField) - assert semantic_class.visibility == "private" - assert semantic_class.origin.source_language == "fortran" - assert semantic_class.origin.native_name == "container_t" - assert semantic_class.origin.native_scope == "consumer" - assert semantic_class.origin.source_kind == "derived_type" - semantic_proc = semantic_module.functions[0] - assert semantic_proc.native_name == "step" - assert semantic_proc.locals == [] - assert semantic_proc.arguments[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] == external_ref - assert semantic_module.variables[0].semantic_type.metadata[EXTERNAL_TYPE_REF_METADATA] == external_ref - assert isinstance(semantic_module.variables[0], SemanticVariable) - assert [method.name for method in semantic_module.classes[0].methods] == ["step"] - assert semantic_module.classes[0].methods[0].projection == semantic_proc.projection - assert semantic_module.classes[0].methods[0].origin == semantic_proc.origin - assert semantic_proc.origin.source_language == "fortran" - assert semantic_proc.origin.native_name == "step" - assert semantic_proc.origin.native_scope == "consumer" - assert semantic_proc.origin.source_kind == "subroutine" - assert [asdict(mapping) for mapping in semantic_proc.projection] == [ - { - "python_name": "arg", - "native_name": "arg", - "native_position": 0, - "python_position": 0, - "result_position": None, - "value_kind": "", - "value": None, - "value_cast": None, - "native_c_identity": None, - } - ] - assert semantic_module.origin.source_language == "fortran" - assert semantic_module.origin.native_name == "consumer" - assert semantic_module.origin.native_scope == "consumer" - assert semantic_module.origin.source_kind == "module" - assert converter.visit(FortranDerivedType(name="default_t")).visibility == "public" - assert converter.visit(FortranVariable(name="local", base_type="derived", kind="state_t")).name == "state_t" - - def test_abstract_type_identity_is_module_qualified_and_available_project_wide(): abstract_type = FortranDerivedType( name="item_t", diff --git a/tests/fortran/derived_types/semantics/test_pyi_class_semantics.py b/tests/fortran/derived_types/semantics/test_pyi_class_semantics.py index 2dad4909c..1d6f0fefd 100644 --- a/tests/fortran/derived_types/semantics/test_pyi_class_semantics.py +++ b/tests/fortran/derived_types/semantics/test_pyi_class_semantics.py @@ -1,31 +1,11 @@ """Core `.pyi` class semantics retained by Derived Types.""" -from prik.printers import emit_module from prik.semantics.metadata import ( PROJECTED_OUTPUT_METADATA, - SUPPRESS_DEFAULT_CONSTRUCTOR_METADATA, ) from tests.fortran._support.pyi_conversion import parse_pyi_text -def test_convert_pyi_to_ir_self_only_generated_constructor_keeps_default_initializer(): - module = parse_pyi_text( - """ -class state: - def __init__(self) -> None: ... - - values: Allocatable[Float64[:]] -""", - module_name="edited", - ) - - cls = module.classes[0] - assert cls.origin.source_language == "fortran" - assert SUPPRESS_DEFAULT_CONSTRUCTOR_METADATA not in cls.origin.metadata - assert cls.methods == [] - assert " def __init__(self) -> None: ..." in emit_module(module) - - def test_compact_assignment_overload_projects_visible_destination_without_direction_label(): from_pyi = parse_pyi_text( """ diff --git a/tests/fortran/enumerations/parsing/test_enum_diagnostics.py b/tests/fortran/enumerations/parsing/test_enum_diagnostics.py index d2f9eaa7d..df915fe39 100644 --- a/tests/fortran/enumerations/parsing/test_enum_diagnostics.py +++ b/tests/fortran/enumerations/parsing/test_enum_diagnostics.py @@ -3,61 +3,60 @@ import pytest from prik.parsers.fortran import FortranParseError, parse_fortran_file -from prik.parsers.fortran.parser import FortranParser -from tests.fortran._support.parser_regressions import _unit -def test_enum_diagnostic_reports_first_invalid_line_after_valid_enumerator(): +@pytest.mark.parametrize( + ("invalid_line", "message"), + [ + pytest.param( + "enumerator :: valid = 1, 2invalid", + "Invalid Fortran syntax in enum specification part: enumerator :: valid = 1, 2invalid", + id="malformed-enumerator", + ), + pytest.param( + "integer :: invalid", + "Invalid Fortran syntax in enum specification part: integer :: invalid", + id="declaration-statement", + ), + pytest.param( + "interface invalid", + "Invalid Fortran syntax in enum specification part: interface invalid", + id="nested-interface", + ), + pytest.param( + "type :: nested\n end type nested", + "Invalid Fortran syntax in enum '' specification part: type :: nested", + id="nested-program-unit", + ), + ], +) +def test_enum_diagnostic_reports_the_first_invalid_line_after_a_valid_enumerator(invalid_line: str, message: str): + """The diagnostic names the offending line and its location, not the enum as a whole.""" with pytest.raises(FortranParseError) as error: parse_fortran_file( - """ + f""" module enum_contract enum, bind(c) enumerator :: valid = 1 - integer :: invalid + {invalid_line} end enum end module enum_contract """, filename="enum_contract.f90", ) - assert error.value.base_message == "Invalid Fortran syntax in enum specification part: integer :: invalid" + assert error.value.base_message == message assert error.value.filename == "enum_contract.f90" assert error.value.line_number == 5 - assert error.value.source_line.strip() == "integer :: invalid" + assert error.value.source_line.strip() == invalid_line.splitlines()[0] assert error.value.code == "PARSE_INVALID_SYNTAX" -def test_enum_diagnostic_rejects_nested_program_unit_with_source_metadata(): - with pytest.raises(FortranParseError) as error: - parse_fortran_file( - """ -module enum_contract - enum, bind(c) - type :: nested - end type nested - end enum -end module enum_contract -""", - filename="nested_enum_contract.f90", - ) - - assert error.value.base_message == "Invalid Fortran syntax in enum '' specification part: type :: nested" - assert error.value.filename == "nested_enum_contract.f90" - assert error.value.line_number == 4 - assert error.value.source_line.strip() == "type :: nested" - assert error.value.code == "PARSE_INVALID_SYNTAX" - - -def test_enum_validator_skips_preprocessed_linemarkers_before_enumerators(): - parser = FortranParser() - unit = _unit( - "enum", - None, - "enum, bind(c)", - '# 8 "generated.f90"', - "enumerator :: ready = 1", - "end enum", +def test_enum_accepts_preprocessed_linemarkers_before_enumerators(): + """A compiler preprocessor's line marker is provenance, not an enum statement.""" + parsed = parse_fortran_file( + 'module m\n enum, bind(c)\n# 8 "generated.f90"\n enumerator :: ready = 1\n end enum\nend module m\n', + filename="generated.f90", ) - parser._helper_validate_enum_unit(unit, filename="generated.f90") + assert [(item.name, item.value) for item in parsed.modules[0].enums[0].enumerators] == [("ready", "1")] diff --git a/tests/fortran/enumerations/parsing/test_enum_syntax.py b/tests/fortran/enumerations/parsing/test_enum_syntax.py index 463da4f7e..5031682c9 100644 --- a/tests/fortran/enumerations/parsing/test_enum_syntax.py +++ b/tests/fortran/enumerations/parsing/test_enum_syntax.py @@ -1,8 +1,6 @@ -"""Supported Fortran enum syntax and rejected declaration forms.""" +"""Supported Fortran enum syntax.""" -import pytest - -from prik.parsers.fortran import FortranParseError, parse_fortran_file +from prik.parsers.fortran import parse_fortran_file def test_valid_enum_subunit_accepts_optional_separator_and_multiple_enumerators(): @@ -28,24 +26,3 @@ def test_valid_enum_subunit_accepts_optional_separator_and_multiple_enumerators( ("third", "10", "10"), ("fourth", "11", None), ] - - -@pytest.mark.parametrize( - "invalid_line", - [ - "enumerator :: valid = 1, 2invalid", - "integer :: invalid", - "interface invalid", - ], -) -def test_enum_subunit_rejects_malformed_lines_and_nested_units(invalid_line): - code = f""" -module enum_invalid_mod - enum, bind(c) - {invalid_line} - end enum -end module enum_invalid_mod -""" - - with pytest.raises(FortranParseError, match="Invalid Fortran syntax"): - parse_fortran_file(code, filename="invalid_enum.f90") diff --git a/tests/fortran/enumerations/pipeline/test_generated_enum_contract.py b/tests/fortran/enumerations/pipeline/test_generated_enum_contract.py deleted file mode 100644 index 881b2fc1a..000000000 --- a/tests/fortran/enumerations/pipeline/test_generated_enum_contract.py +++ /dev/null @@ -1,19 +0,0 @@ -"""Reviewed generated semantic `.pyi` package for Fortran enumerations.""" - -from pathlib import Path - -from tests.fortran._support.generated_contracts import ( - GeneratedContractCase, - assert_generated_contract_matches_fixture, -) - -FIXTURES = Path(__file__).parents[1] / "end_to_end" / "fixtures" -CASE = GeneratedContractCase( - name="fenums_f90", - inputs=(FIXTURES / "native" / "fenums_f90.f90",), - expected_package=FIXTURES / "contracts" / "fenums_f90", -) - - -def test_generated_enum_contract_matches_reviewed_package(tmp_path: Path): - assert_generated_contract_matches_fixture(CASE, tmp_path) diff --git a/tests/fortran/error_handling/codegen/test_runtime_envelope_lowering.py b/tests/fortran/error_handling/codegen/test_runtime_envelope_lowering.py deleted file mode 100644 index 315fa8bb0..000000000 --- a/tests/fortran/error_handling/codegen/test_runtime_envelope_lowering.py +++ /dev/null @@ -1,33 +0,0 @@ -"""Native-call runtime-envelope lowering tests.""" - -from __future__ import annotations - -from pathlib import Path - -from prik.pipeline.pyi import pyi_file_to_semantic_module -from prik.policy.completion import complete_semantic_policies -from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import WrapperPlanner - - -RECURSION_CONTRACT = ( - Path("tests/fortran/error_handling/end_to_end/fixtures/contracts") - / "fruntime_recursion_f90" - / "fruntime_recursion_f90.pyi" -) - - -def _rendered_source(artifacts, suffix: str) -> str: - return next(source.text for source in artifacts.sources if source.path.name.endswith(suffix)) - - -def test_recursive_runtime_contract_keeps_the_gil_by_default(): - module = pyi_file_to_semantic_module(RECURSION_CONTRACT, module_name="fruntime_recursion_f90") - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - - assert plan.namespaces[0].functions - assert all(function.binding.release_gil is False for function in plan.namespaces[0].functions) - c_source = _rendered_source(WrapperGenerator().generate(plan), ".c") - assert "Py_BEGIN_ALLOW_THREADS" not in c_source - assert "Py_END_ALLOW_THREADS" not in c_source diff --git a/tests/fortran/error_handling/codegen/test_status_error_lowering.py b/tests/fortran/error_handling/codegen/test_status_error_lowering.py index ded967939..495f1c151 100644 --- a/tests/fortran/error_handling/codegen/test_status_error_lowering.py +++ b/tests/fortran/error_handling/codegen/test_status_error_lowering.py @@ -1,4 +1,4 @@ -"""Native-call runtime-envelope and status-error lowering tests.""" +"""Status-error plan validation before lowering.""" from __future__ import annotations @@ -9,7 +9,6 @@ from prik.pipeline.pyi import pyi_file_to_semantic_module from prik.policy.completion import complete_semantic_policies -from prik.policy.models import BridgeDataAction, PythonExceptionKind from prik.pipeline.wrapper import WrapperGenerator from prik.planning import DatatypeFamily, WrapperPlanner @@ -30,144 +29,51 @@ def _runtime_plan(): return WrapperPlanner().build(module) -def _rendered_source(artifacts, suffix: str) -> str: - return next(source.text for source in artifacts.sources if source.path.name.endswith(suffix)) - - -def _function_source(source: str, function_name: str, next_name: str | None = None) -> str: - start = source.index(f"static PyObject * wrap_{function_name}") - if next_name is None: - return source[start : source.index("PyMODINIT_FUNC", start)] - return source[start : source.index(f"static PyObject * wrap_{next_name}", start)] - - -def _edit_function(plan, function_name: str, edit): - root = plan.namespaces[0] - functions = tuple( - edit(function) if function.binding.python_name == function_name else function for function in root.functions - ) - return replace(plan, namespaces=(replace(root, functions=functions), *plan.namespaces[1:])) - - -def test_planner_records_editable_native_runtime_and_status_error_facts(): - plan = _runtime_plan() - functions = {function.binding.python_name: function for function in plan.namespaces[0].functions} - solve = functions["solve"] - - assert functions["pause_for_one_second"].binding.release_gil is True - assert functions["pause_with_gil"].binding.release_gil is False - assert solve.binding.release_gil is True - assert solve.binding.status_error is not None - assert solve.binding.status_error.success == 0 - assert solve.binding.status_error.exception_kind is PythonExceptionKind.RUNTIME_ERROR - assert solve.binding.status_error.status_role == solve.entrypoint.projected_slots[1].symbolic_role - assert solve.binding.status_error.message_role == solve.entrypoint.projected_slots[2].symbolic_role - assert [(result.parameter_name, result.native_result_role) for result in solve.entrypoint.results] == [ - ("status", solve.binding.status_error.status_role), - ("message", solve.binding.status_error.message_role), - ] - assert [parameter.source_kind for parameter in solve.entrypoint.parameters] == [ - "argument", - "hidden_result", - "hidden_result", - ] - WrapperGenerator().generate(plan) - assert "Raises\n------" in solve.binding.docstring - assert solve.binding.docstring.count("RuntimeError\n") == 1 - assert "If native status differs from the success value 0." in solve.binding.docstring - assert solve.entrypoint.projected_slots[1].semantic_type_name == "Int32" - assert solve.entrypoint.projected_slots[1].datatype_family is DatatypeFamily.INTEGER - assert solve.entrypoint.projected_slots[1].adapter.bridge_data_action is BridgeDataAction.DIRECT_TRANSFER - assert solve.entrypoint.projected_slots[1].adapter.bridge_copy_reason is None - assert solve.entrypoint.projected_slots[2].semantic_type_name == "String" - assert solve.entrypoint.projected_slots[2].datatype_family is DatatypeFamily.STRING - assert solve.entrypoint.projected_slots[2].character_length == 32 - assert solve.entrypoint.projected_slots[2].adapter.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - assert solve.entrypoint.projected_slots[2].adapter.bridge_copy_reason == ( - "copy fixed-length Fortran character output into C-owned null-terminated storage" +def _drop_message_copy_reason(function): + return replace( + function, + entrypoint=replace( + function.entrypoint, + projected_slots=tuple( + replace(slot, adapter=replace(slot.adapter, bridge_copy_reason=None)) + if slot.datatype_family is DatatypeFamily.STRING + else slot + for slot in function.entrypoint.projected_slots + ), + ), ) -def test_direct_binding_lowering_places_only_opted_in_native_call_outside_the_gil(): - artifacts = WrapperGenerator().generate(_runtime_plan()) - c_source = _rendered_source(artifacts, ".c") - released = _function_source(c_source, "pause_for_one_second", "pause_with_gil") - held = _function_source(c_source, "pause_with_gil", "solve") - solve = _function_source(c_source, "solve") - - assert released.index("Py_BEGIN_ALLOW_THREADS") < released.index("bind_c_pause_for_one_second()") - assert released.index("bind_c_pause_for_one_second()") < released.index("Py_END_ALLOW_THREADS") - assert "Py_BEGIN_ALLOW_THREADS" not in held - assert "Py_END_ALLOW_THREADS" not in held - assert solve.index("Py_BEGIN_ALLOW_THREADS") < solve.index("bind_c_solve(bound_value_storage, &status, &message)") - assert solve.index("bind_c_solve(bound_value_storage, &status, &message)") < solve.index("Py_END_ALLOW_THREADS") - assert solve.index("Py_END_ALLOW_THREADS") < solve.index("prik_status_message_text") - assert solve.index("prik_status_message_text") < solve.index("status != 0") - assert "PyErr_SetObject(PyExc_RuntimeError, message_obj)" in solve - assert "free(message)" in solve - - -def test_direct_bridge_lowering_projects_status_and_copies_fixed_message(): - artifacts = WrapperGenerator().generate(_runtime_plan()) - fortran_source = _rendered_source(artifacts, ".f90") - - assert "subroutine bind_c_solve(value, status, message)" in fortran_source - assert "integer(c_int32_t) :: status" in fortran_source - assert "type(c_ptr) :: message" in fortran_source - assert "character(kind=c_char, len=32) :: message_value" in fortran_source - assert "call native_solve(value, status, message_value)" in fortran_source - assert "message = c_malloc(33_c_size_t)" in fortran_source - assert "message_copy(33) = c_null_char" in fortran_source - - -def test_fixed_message_bridge_copy_requires_its_completed_reason(): - plan = _runtime_plan() - invalid = _edit_function( - plan, - "solve", - lambda function: replace( - function, - entrypoint=replace( - function.entrypoint, - projected_slots=tuple( - replace( - slot, - adapter=replace(slot.adapter, bridge_copy_reason=None), - ) - if slot.datatype_family is DatatypeFamily.STRING - else slot - for slot in function.entrypoint.projected_slots - ), - ), +def _drop_status_role(function): + return replace( + function, + binding=replace( + function.binding, + status_error=replace(function.binding.status_error, status_role="missing:status"), ), ) - with pytest.raises(ValueError, match="missing-bridge-copy-reason"): - WrapperGenerator().generate(invalid) +@pytest.mark.parametrize( + ("edit", "diagnostic"), + [ + (_drop_message_copy_reason, "missing-bridge-copy-reason"), + (_drop_status_role, "missing-status-result-role"), + ], + ids=["message-copy-without-reason", "status-error-without-status-result"], +) +def test_status_error_plan_edits_fail_before_backend_lowering(edit, diagnostic): + """A status error must name a produced status and a completed message copy. -def test_runtime_plan_edits_dispatch_to_named_lowering_and_validate_roles(): + Runtime status projection, message text, and GIL placement are proved in + ``end_to_end/test_status_projection.py``. + """ plan = _runtime_plan() - held = _edit_function( - plan, - "pause_for_one_second", - lambda function: replace(function, binding=replace(function.binding, release_gil=False)), - ) - c_source = _rendered_source(WrapperGenerator().generate(held), ".c") - released = _function_source(c_source, "pause_for_one_second", "pause_with_gil") - assert "Py_BEGIN_ALLOW_THREADS" not in released - assert "Py_END_ALLOW_THREADS" not in released - - invalid = _edit_function( - plan, - "solve", - lambda function: replace( - function, - binding=replace( - function.binding, - status_error=replace(function.binding.status_error, status_role="missing:status"), - ), - ), + root = plan.namespaces[0] + functions = tuple( + edit(function) if function.binding.python_name == "solve" else function for function in root.functions ) - with pytest.raises(ValueError, match="missing-status-result-role"): + invalid = replace(plan, namespaces=(replace(root, functions=functions), *plan.namespaces[1:])) + + with pytest.raises(ValueError, match=diagnostic): WrapperGenerator().generate(invalid) diff --git a/tests/fortran/error_handling/end_to_end/test_status_projection.py b/tests/fortran/error_handling/end_to_end/test_status_projection.py index e6dac38f1..e7ec770d2 100644 --- a/tests/fortran/error_handling/end_to_end/test_status_projection.py +++ b/tests/fortran/error_handling/end_to_end/test_status_projection.py @@ -53,6 +53,8 @@ def test_status_projection_consumes_outputs_raises_message_and_recovers(tmp_path with pytest.raises(RuntimeError, match="negative input"): module.solve(np.int32(-1)) assert module.solve(np.int32(2)) is None + assert "Raises\n------" in module.solve.__doc__ + assert "If native status differs from the success value 0." in module.solve.__doc__ binding = (result.output_dir / "fruntime_policy_f90_wrapper.c").read_text(encoding="utf-8") held = binding[ diff --git a/tests/fortran/error_handling/pipeline/test_concise_cli_diagnostics.py b/tests/fortran/error_handling/pipeline/test_concise_cli_diagnostics.py index 787558703..645ed2b80 100644 --- a/tests/fortran/error_handling/pipeline/test_concise_cli_diagnostics.py +++ b/tests/fortran/error_handling/pipeline/test_concise_cli_diagnostics.py @@ -1,11 +1,11 @@ -"""Concise CLI diagnostics for expected Fortran failures.""" +"""CLI diagnostics for expected Fortran failures: concise by default, a traceback with --debug.""" from pathlib import Path import subprocess import sys -def test_cli_formats_parse_errors_without_traceback(tmp_path: Path): +def test_cli_formats_parse_errors_concisely_and_reraises_with_debug(tmp_path: Path): f90 = tmp_path / "bad.f90" f90.write_text( """subroutine bad(x) @@ -15,12 +15,17 @@ def test_cli_formats_parse_errors_without_traceback(tmp_path: Path): encoding="utf-8", ) - cmd = [sys.executable, "-m", "prik", "parse", str(f90), "--no-color"] - res = subprocess.run(cmd, capture_output=True, text=True) + concise = subprocess.run( + [sys.executable, "-m", "prik", "parse", str(f90), "--no-color"], capture_output=True, text=True + ) + assert concise.returncode == 1 + assert concise.stdout == "" + assert "Traceback" not in concise.stderr + assert f"{f90}:" in concise.stderr + assert "error[PARSE_UNSUPPORTED_DECLARATION]:" in concise.stderr + assert "| weirdtype :: x" in concise.stderr - assert res.returncode == 1 - assert res.stdout == "" - assert "Traceback" not in res.stderr - assert f"{f90}:" in res.stderr - assert "error[PARSE_UNSUPPORTED_DECLARATION]:" in res.stderr - assert "| weirdtype :: x" in res.stderr + debug = subprocess.run([sys.executable, "-m", "prik", "parse", str(f90), "--debug"], capture_output=True, text=True) + assert debug.returncode == 1 + assert "Traceback" in debug.stderr + assert "FortranParseError" in debug.stderr diff --git a/tests/fortran/error_handling/pipeline/test_debug_cli_tracebacks.py b/tests/fortran/error_handling/pipeline/test_debug_cli_tracebacks.py deleted file mode 100644 index cd43285d4..000000000 --- a/tests/fortran/error_handling/pipeline/test_debug_cli_tracebacks.py +++ /dev/null @@ -1,23 +0,0 @@ -"""Debug CLI tracebacks for unexpected Fortran failures.""" - -from pathlib import Path -import subprocess -import sys - - -def test_cli_debug_flag_reraises_parse_errors(tmp_path: Path): - f90 = tmp_path / "bad.f90" - f90.write_text( - """subroutine bad(x) - weirdtype :: x -end subroutine bad -""", - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik", "parse", str(f90), "--debug"] - res = subprocess.run(cmd, capture_output=True, text=True) - - assert res.returncode == 1 - assert "Traceback" in res.stderr - assert "FortranParseError" in res.stderr diff --git a/tests/fortran/error_handling/pipeline/test_runtime_generated_contracts.py b/tests/fortran/error_handling/pipeline/test_runtime_generated_contracts.py index f4a3f8130..9fa4881e6 100644 --- a/tests/fortran/error_handling/pipeline/test_runtime_generated_contracts.py +++ b/tests/fortran/error_handling/pipeline/test_runtime_generated_contracts.py @@ -14,10 +14,7 @@ ) CONTRACT_ROOT = Path(__file__).parents[1] / "end_to_end" / "fixtures" / "contracts" -CASES = ( - source_contract_case(CONTRACT_ROOT, "fopenmp_runtime_f90.f90"), - source_contract_case(CONTRACT_ROOT, "fruntime_recursion_f90.f90"), -) +CASES = (source_contract_case(CONTRACT_ROOT, "fopenmp_runtime_f90.f90"),) @pytest.mark.parametrize("case", CASES, ids=contract_case_id) diff --git a/tests/fortran/functions/codegen/test_multiple_function_results.py b/tests/fortran/functions/codegen/test_multiple_function_results.py index d0ac0cb41..623192215 100644 --- a/tests/fortran/functions/codegen/test_multiple_function_results.py +++ b/tests/fortran/functions/codegen/test_multiple_function_results.py @@ -7,7 +7,6 @@ import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.ownership import CodegenAction, NativeBarrierAction from prik.policy.completion import complete_semantic_policies from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner @@ -41,42 +40,6 @@ def with_four_scalars(n: Int32) -> tuple[Int32, Int32, Int32, Int32]: ... return WrapperPlanner().build(module) -def test_multiple_scalar_result_plan_has_ordered_binding_consumers_and_shared_hidden_slot(): - function = _multiple_result_plan().namespaces[0].functions[0] - direct, hidden = function.results - - assert [(result.source_kind, result.result_position) for result in function.results] == [ - ("direct_return", 0), - ("hidden_output", 1), - ] - assert direct.projected_call_slot is None - assert hidden.projected_call_slot is function.entrypoint.projected_slots[hidden.projected_call_slot.native_position] - assert direct.binding.codegen_action is CodegenAction.DIRECT_VALUE - assert hidden.binding.codegen_action is CodegenAction.DIRECT_VALUE - assert hidden.bridge.native_action is NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS - assert direct.entrypoint.native_result_role in function.available_roles - assert hidden.entrypoint.native_result_role in function.available_roles - - -def test_multiple_scalar_results_lower_to_binding_tuple_and_one_bridge_function_call(): - artifacts = WrapperGenerator().generate(_multiple_result_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "int32_t bind_c_with_scalar(int32_t * n, int32_t * status);" in c_source - assert "result = bind_c_with_scalar(bound_n_storage, &status);" in c_source - assert "PyObject * result_0_obj = prik_int32_to_numpy(&result);" in c_source - assert "PyObject * result_1_obj = prik_int32_to_numpy(&status);" in c_source - assert "PyObject * result_obj = PyTuple_New(2);" in c_source - assert "PyTuple_SET_ITEM(result_obj, 0, result_0_obj);" in c_source - assert "PyTuple_SET_ITEM(result_obj, 1, result_1_obj);" in c_source - assert "Py_DECREF(result_0_obj);" in c_source - - assert 'function bind_c_with_scalar(n, status) result(result) bind(c, name="bind_c_with_scalar")' in bridge_source - assert "result = native_with_scalar(n, status)" in bridge_source - assert "PyTuple" not in bridge_source - - def test_four_scalar_results_share_one_linear_failure_cleanup_suffix(): artifacts = WrapperGenerator().generate(_four_result_plan()) c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") diff --git a/tests/fortran/functions/codegen/test_scalar_function_writeback.py b/tests/fortran/functions/codegen/test_scalar_function_writeback.py deleted file mode 100644 index 41103287e..000000000 --- a/tests/fortran/functions/codegen/test_scalar_function_writeback.py +++ /dev/null @@ -1,51 +0,0 @@ -"""Scalar function optional, descriptor, and writeback lowering tests.""" - -from __future__ import annotations - -from dataclasses import replace -from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.completion import complete_semantic_policies -from prik.policy.models import WritebackPhase -from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import WrapperPlanner - - -def _artifacts(module): - complete_semantic_policies(module) - return WrapperGenerator().generate(WrapperPlanner().build(module)) - - -def _source(artifacts, suffix: str) -> str: - return next(item.text for item in artifacts.sources if item.path.name.endswith(suffix)) - - -def _replace_root_function(plan, function): - root = plan.namespaces[0] - return replace(plan, namespaces=(replace(root, functions=(function,)), *plan.namespaces[1:])) - - -def test_scalar_writeback_is_an_explicit_binding_lifecycle_result(): - module = parse_pyi_text( - 'def bump(value: Annotated[Int32, Immutable]) -> Returns["value", Int32]: ...', - module_name="scalar_writeback", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - actions = plan.namespaces[0].functions[0].writeback_actions - - assert tuple(action.phase for action in actions) == tuple(WritebackPhase) - assert actions[0].binding is not None - assert actions[1].bridge is not None - assert actions[2].binding.python_result_role == "scalar_writeback.bump.value:python-result" - assert actions[3].binding is not None - - artifacts = WrapperGenerator().generate(plan) - c_source = _source(artifacts, ".c") - fortran_source = _source(artifacts, ".f90") - - assert "void bind_c_bump(int32_t * value);" in c_source - assert "bind_c_bump(&bound_value);" in c_source - assert "PyObject * result_obj = NULL;" in c_source - assert "result_obj = prik_int32_to_numpy(&bound_value);" in c_source - assert "subroutine bind_c_bump(value)" in fortran_source - assert "call native_bump(value)" in fortran_source diff --git a/tests/fortran/functions/end_to_end/test_external_procedures.py b/tests/fortran/functions/end_to_end/test_external_procedures.py index 30ad53c82..1ab02c83a 100644 --- a/tests/fortran/functions/end_to_end/test_external_procedures.py +++ b/tests/fortran/functions/end_to_end/test_external_procedures.py @@ -195,60 +195,6 @@ def test_one_source_with_several_standalone_procedures_exports_each_at_root(bund assert bundled_external_module.offset_value(np.int32(4)) == np.int32(14) -def test_generated_standalone_contracts_are_non_empty_root_fragments(tmp_path: Path): - for source in (FIXED_EXTERNAL, FREE_EXTERNAL, EXTERNAL_BUNDLE): - copied = _copy_sources((source,), tmp_path / source.stem) - entry = _generate_contract( - copied, - tmp_path / f"{source.stem}_contracts", - _generated_contract_fixture(source.stem), - ) - text = entry.read_text(encoding="utf-8") - - assert entry.name == "__init__.pyi" - assert text.strip() - assert text.count("@standalone") == len([line for line in text.splitlines() if line.startswith("def ")]) - assert sorted(path.name for path in entry.parent.glob("*.pyi")) == ["__init__.pyi"] - - -def test_classic_external_bridge_uses_implicit_declaration_and_no_module_use(tmp_path: Path): - sources = _copy_sources((FREE_EXTERNAL,), tmp_path / "sources") - module, result, entry = _build_generated_contract( - sources, - tmp_path, - output_name=FREE_EXTERNAL.stem, - expected_package=_generated_contract_fixture(FREE_EXTERNAL.stem), - ) - - bridge = (result.output_dir / f"bind_c_{result.module_name}_wrapper.f90").read_text(encoding="utf-8").lower() - assert module.free_square(np.int32(3)) == np.int32(9) - assert entry.read_text(encoding="utf-8").startswith( - "from prik.contracts import Addr, Arg, Int32, native_call, standalone\n\n@standalone\n" - ) - assert "integer(c_int32_t), external :: free_square" in bridge - assert "function free_square(" not in bridge - assert "result = free_square(value)" in bridge - assert "private\n" not in bridge - assert "public :: bind_c_free_square" not in bridge - assert "use free_external" not in bridge - - -def test_module_procedure_bridge_uses_native_module_scope(tmp_path: Path): - source = _copy_sources((BASIC_SOURCE,), tmp_path / "sources") - entry = _generate_contract(source, tmp_path / "contracts", _generated_contract_fixture(BASIC_SOURCE.stem)) - native_objects = _compile_native_objects(source, tmp_path / "native") - result = build_pyi_extension( - entry, - native_objects=native_objects, - native_include_dirs=[native_objects[0].parent], - output_dir=tmp_path / "pyi_build", - ) - - bridge = (result.output_dir / f"bind_c_{result.module_name}_wrapper.f90").read_text(encoding="utf-8").lower() - assert "use m1, only:" in bridge - assert "add1" in bridge - - def test_handwritten_c_order_flat_contract_passes_rank_preserving_bridge_view(tmp_path: Path): source = _copy_sources((C_ORDER_FLAT_BUFFER,), tmp_path / "sources") native_objects = _compile_native_objects(source, tmp_path / "native") diff --git a/tests/fortran/functions/end_to_end/test_function_direct_entrypoint_routing.py b/tests/fortran/functions/end_to_end/test_function_direct_entrypoint_routing.py index c0ecb4756..bcef791f9 100644 --- a/tests/fortran/functions/end_to_end/test_function_direct_entrypoint_routing.py +++ b/tests/fortran/functions/end_to_end/test_function_direct_entrypoint_routing.py @@ -6,7 +6,6 @@ import pytest from tests.fortran._support.wrapper_build import ( - _build_inline_pyi_contract_module, _build_source_or_generated_pyi_and_import, ) @@ -63,22 +62,3 @@ def test_standalone_mixed_route_adapts_only_ordinary_external( ) assert "bind_c_standalone_adapted" in bridge assert "standalone_direct" not in bridge - - -def test_standalone_mixed_route_matches_edited_source_free_contract(tmp_path: Path): - stem = "standalone_mixed_bind_c_f90" - source = (FIXTURES / "native" / f"{stem}.f90").read_text(encoding="utf-8") - contract = (FIXTURES / "contracts" / stem / "__init__.pyi").read_text(encoding="utf-8") - contract = contract.replace("from prik.contracts import ", "from prik.contracts import nogil, ") - contract = contract.replace("def standalone_direct(", "@nogil\ndef standalone_direct(").replace( - "def standalone_adapted(", "@nogil\ndef standalone_adapted(" - ) - module, result = _build_inline_pyi_contract_module( - tmp_path, module_name=stem, source_text=source, contract_text=contract - ) - - assert module.standalone_direct(np.int32(4)) == np.int32(6) - assert module.standalone_adapted(np.int32(4)) == np.int32(7) - bridge = (result.output_dir / f"bind_c_{stem}_wrapper.f90").read_text(encoding="utf-8").casefold() - assert "bind_c_standalone_adapted" in bridge - assert "function bind_c_standalone_direct" not in bridge diff --git a/tests/fortran/functions/end_to_end/test_hidden_native_outputs.py b/tests/fortran/functions/end_to_end/test_hidden_native_outputs.py index 13095734e..499d34c6f 100644 --- a/tests/fortran/functions/end_to_end/test_hidden_native_outputs.py +++ b/tests/fortran/functions/end_to_end/test_hidden_native_outputs.py @@ -5,6 +5,7 @@ differs: it builds no Python result from it. """ +import sys from pathlib import Path import numpy as np @@ -27,74 +28,46 @@ """ -def _build(tmp_path: Path, name: str, contract: str): +CONTRACT = """from prik.contracts import Arg, Hidden, Int32, Return, Returns, String, bind, native_call + +@native_call([Arg(0), Hidden("doubled", Int32), Hidden("note", String[16])]) +def tally(n: Int32) -> None: ... + +@bind("tally") +@native_call([Arg(0), Return("doubled", 0), Hidden("note", String[16])]) +def tally_doubled(n: Int32) -> Returns["doubled", Int32]: ... +""" + + +def test_hidden_outputs_are_released_but_never_returned(tmp_path: Path): + """The adapter still allocates the string, so the binding still frees it. + + A hidden output and a returned one share one bridge; only the binding + distinguishes them, and a discarded output must not leak its adapter + allocation or a reference across repeated calls. + """ + name = "hidden_outputs" (tmp_path / f"{name}.f90").write_text(SOURCE.format(name=name), encoding="utf-8") - (tmp_path / f"{name}.pyi").write_text(contract, encoding="utf-8") + (tmp_path / f"{name}.pyi").write_text(CONTRACT, encoding="utf-8") result = build_pyi_extension( tmp_path / f"{name}.pyi", native_fortran_sources=[tmp_path / f"{name}.f90"], - output_dir=tmp_path / f"build_{name}", + output_dir=tmp_path / "build", output_name=name, ) binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") bridge = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".f90") - return result, binding, bridge - - -def test_hidden_outputs_are_released_but_never_returned(tmp_path: Path): - """The adapter still allocates the string, so the binding still frees it.""" - result, binding, bridge = _build( - tmp_path, - "hidden_all", - """from prik.contracts import Arg, Hidden, Int32, String, native_call - -@native_call([Arg(0), Hidden("doubled", Int32), Hidden("note", String[16])]) -def tally(n: Int32) -> None: ... -""", - ) module = result.import_module() - # The bridge is the ordinary owned-allocation adapter for a character output. + # The bridge is the ordinary owned-allocation adapter for a character output ... assert "note = c_malloc(17_c_size_t)" in bridge # ... so the binding must still release it even though nothing is published. assert "free(note)" in binding assert module.tally(np.int32(5)) is None assert module.tally.__doc__.splitlines()[0] == "tally(n) -> None" - - -def test_hidden_and_returned_outputs_share_one_bridge(tmp_path: Path): - """Only the binding distinguishes them; the native call is the same.""" - result, _, bridge = _build( - tmp_path, - "hidden_mixed", - """from prik.contracts import Arg, Hidden, Int32, Return, Returns, String, native_call - -@native_call([Arg(0), Return("doubled", 0), Hidden("note", String[16])]) -def tally(n: Int32) -> Returns["doubled", Int32]: ... -""", - ) - module = result.import_module() - - assert 'subroutine bind_c_tally(n, doubled, note) bind(c, name="bind_c_tally")' in bridge - assert module.tally(np.int32(5)) == np.int32(10) - assert module.tally.__doc__.splitlines()[0] == "tally(n) -> int32" - - -def test_hidden_outputs_do_not_leak_across_repeated_calls(tmp_path: Path): - """A discarded output must not leak its adapter allocation or a reference.""" - result, _, _ = _build( - tmp_path, - "hidden_leak", - """from prik.contracts import Arg, Hidden, Int32, String, native_call - -@native_call([Arg(0), Hidden("doubled", Int32), Hidden("note", String[16])]) -def tally(n: Int32) -> None: ... -""", - ) - module = result.import_module() - - import sys + assert module.tally_doubled(np.int32(5)) == np.int32(10) + assert module.tally_doubled.__doc__.splitlines()[0] == "tally_doubled(n) -> int32" def refcount_growth(calls: int) -> int: """Return how much ``None``'s refcount moved across ``calls`` calls.""" diff --git a/tests/fortran/functions/parsing/test_procedure_and_interface_regressions.py b/tests/fortran/functions/parsing/test_procedure_and_interface_regressions.py index 1e71a7e15..44672f23a 100644 --- a/tests/fortran/functions/parsing/test_procedure_and_interface_regressions.py +++ b/tests/fortran/functions/parsing/test_procedure_and_interface_regressions.py @@ -27,26 +27,6 @@ def test_function_result_assignment_name_with_intrinsic_prefix_starts_execution_ assert proc.result.base_type == "real" -def test_procedure_bind_c_name_and_value_argument_are_preserved(): - parsed = parse_fortran_file( - """ -module c_api - use iso_c_binding -contains - integer(c_int) function renamed(n) bind(C, name="prik_renamed") result(res) - integer(c_int), value, intent(in) :: n - res = n - end function renamed -end module c_api -""" - ) - - proc = parsed.modules[0].procedures[0] - assert proc.attributes == ["bind(c)"] - assert proc.bind_name == "prik_renamed" - assert proc.arguments[0].pass_by_value is True - - def test_procedure_children_exclude_execution_text_and_internal_procedures(): unit = _unit( "procedure", diff --git a/tests/fortran/functions/policy/test_function_result_policy.py b/tests/fortran/functions/policy/test_function_result_policy.py deleted file mode 100644 index f06ce93c7..000000000 --- a/tests/fortran/functions/policy/test_function_result_policy.py +++ /dev/null @@ -1,70 +0,0 @@ -from pathlib import Path - - -from tests.fortran._support.ownership_policy import parse_pyi_text -from tests.fortran._support.wrapper_build import wrapper_source -from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig, read_fortran_source -from prik.semantics.fortran2ir import fortran_project_to_semantic_modules -from prik.policy.ownership import ( - NativeBarrierAction, -) -from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - ArgumentConversionPhase, - WritebackPhase, -) -from prik.policy.construction import completed_function_wrapper_policy - -FMATH_CONTRACT = Path("tests/fortran/data_types/end_to_end/fixtures/contracts/fmath/__init__.pyi") - - -def _source_semantic_module(filename: str, *, module_name: str): - source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name=module_name) - complete_semantic_policies(module) - return module - - -def test_scalar_copy_in_out_policy_completes_writeback_before_planning(): - module = parse_pyi_text( - 'def bump(value: Annotated[Int32, Immutable]) -> Returns["value", Int32]: ...', - module_name="scalar_writeback", - ) - complete_semantic_policies(module) - - policy = completed_function_wrapper_policy(module.functions[0]) - - assert policy.supported is True - assert policy.results == () - assert policy.native_is_subroutine is True - assert policy.arguments[0].conversion_phase is ArgumentConversionPhase.IMMEDIATE - assert tuple(action.phase for action in policy.writeback_actions) == tuple(WritebackPhase) - assert {action.source_role for action in policy.writeback_actions} == {"scalar_writeback.bump.value:value"} - assert {action.result_position for action in policy.writeback_actions} == {0} - - -def test_multiple_scalar_result_policy_completes_order_and_hidden_address_before_planning(): - module = parse_pyi_text( - """ -@native_call([Addr(Arg(0)), Return("status", 1)]) -def with_scalar(n: Int32) -> tuple[Int32, Int32]: ... -""", - module_name="multiple_scalar_results", - ) - complete_semantic_policies(module) - - policy = completed_function_wrapper_policy(module.functions[0]) - - assert policy.supported is True - assert [(result.source_kind, result.result_position) for result in policy.results] == [ - ("direct_return", 0), - ("hidden_output", 1), - ] - hidden = policy.results[1] - assert hidden.native_barrier_action is NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS - assert policy.native_call_slots[1].owner_path == hidden.owner_path diff --git a/tests/fortran/functions/policy/test_positional_only_surface.py b/tests/fortran/functions/policy/test_positional_only_surface.py index 45580b518..b3235e1b5 100644 --- a/tests/fortran/functions/policy/test_positional_only_surface.py +++ b/tests/fortran/functions/policy/test_positional_only_surface.py @@ -21,27 +21,26 @@ def _policies(source: str, **options): return {function.name: completed_function_wrapper_policy(function) for function in module.functions} -def test_an_all_required_function_becomes_positional_and_is_renamed_by_position(): - policy = _policies(_SOURCE, positional_only=True)["required_only"] - - assert policy.accepts_keyword_arguments is False - assert [argument.python_name for argument in policy.arguments] == ["arg0", "arg1"] - # The native declaration keeps its own names; only the Python surface changes. - assert [argument.name for argument in policy.arguments] == ["alpha", "beta"] - - -def test_an_optional_argument_keeps_keywords_because_skipping_one_requires_naming_the_rest(): - policy = _policies(_SOURCE, positional_only=True)["has_optional"] - - assert policy.accepts_keyword_arguments is True - assert [argument.python_name for argument in policy.arguments] == ["value", "scale"] - - -def test_the_default_surface_is_unchanged(): - policies = _policies(_SOURCE) - - assert policies["required_only"].accepts_keyword_arguments is True - assert [argument.python_name for argument in policies["required_only"].arguments] == ["alpha", "beta"] +def test_positional_only_surface_drops_keywords_only_where_policy_does_not_owe_them(): + """All-required functions become positional and are renamed by position. + + An optional argument keeps keywords, because skipping one requires naming + the rest, and the default surface is unchanged. The native declaration + keeps its own names; only the Python surface changes. + """ + positional = _policies(_SOURCE, positional_only=True) + required = positional["required_only"] + assert required.accepts_keyword_arguments is False + assert [argument.python_name for argument in required.arguments] == ["arg0", "arg1"] + assert [argument.name for argument in required.arguments] == ["alpha", "beta"] + + optional = positional["has_optional"] + assert optional.accepts_keyword_arguments is True + assert [argument.python_name for argument in optional.arguments] == ["value", "scale"] + + default = _policies(_SOURCE)["required_only"] + assert default.accepts_keyword_arguments is True + assert [argument.python_name for argument in default.arguments] == ["alpha", "beta"] def test_an_overload_set_cannot_become_positional_only_because_it_dispatches_on_keywords(): diff --git a/tests/fortran/functions/semantics/test_fortran_conversion_procedures_and_interfaces.py b/tests/fortran/functions/semantics/test_fortran_conversion_procedures_and_interfaces.py index 84379cd5a..208780f76 100644 --- a/tests/fortran/functions/semantics/test_fortran_conversion_procedures_and_interfaces.py +++ b/tests/fortran/functions/semantics/test_fortran_conversion_procedures_and_interfaces.py @@ -1,8 +1,6 @@ """Tests split by stable ownership concept from `test_compile_time_values.py`.""" -from prik.parsers.fortran.models import FortranProcedureSignature from prik.semantics.fortran2ir import ( - FortranToIRConverter, fortran_file_to_semantic_modules, fortran_module_to_semantic_module, ) @@ -10,12 +8,10 @@ ProjectionMapping, SemanticArgument, SemanticFunction, - SemanticMethod, SemanticType, ) from tests.fortran._support.semantic_conversion import get_function from prik.parsers.fortran import parse_fortran_file as parse_fortran_source -from prik.semantics import models as semantic_models def test_bind_c_name_and_value_calling_convention_reach_semantic_ir(): @@ -61,39 +57,6 @@ def test_converter_preserves_abstract_and_deferred_type_facts_for_policy_complet assert module.classes[0].metadata["fortran_deferred_bindings"] == ["area"] -def test_semantic_model_helpers_cover_projection_and_canonical_edge_cases(): - converter = FortranToIRConverter() - assert ( - converter.first_module([FortranProcedureSignature(name="hidden", kind="subroutine", in_interface=True)]).name - == "" - ) - assert FortranToIRConverter._literal_kind_key("kind(1.0q0)") == "16" - assert FortranToIRConverter._literal_kind_key("kind(1)") is None - assert SemanticFunction("f") != SemanticMethod("f") - assert semantic_models._semantic_type_key(None, {}) is None - assert semantic_models._canonical_expression( - ["n", ("m",), {"extent": "n + m"}], - {"n": "$0", "m": "$1"}, - ) == ["$0", ("$1",), {"extent": "$0 + $1"}] - - projection = [ - ProjectionMapping(native_position=0, python_position=1), - ProjectionMapping(native_position=1, python_position=None), - ProjectionMapping(native_position=2, result_position=0), - ProjectionMapping(native_position=3, python_position=None), - ProjectionMapping( - native_position=4, - value_kind="shape", - value={"value": ["n", ("m",)], "dim": {"extent": "n + m"}}, - ), - ] - - key = semantic_models._projection_key(projection, {"n": "$0", "m": "$1"}) - - assert len(key) == len(projection) - assert key[-1][4] == (("dim", (("extent", "$0 + $1"),)), ("value", ("$0", ("$1",)))) - - def test_scalar_descriptors_record_native_projection_kind(): source = """ module scalar_descriptor_mod diff --git a/tests/fortran/functions/semantics/test_fortran_function_semantics.py b/tests/fortran/functions/semantics/test_fortran_function_semantics.py index 7901e5b7b..59c25b553 100644 --- a/tests/fortran/functions/semantics/test_fortran_function_semantics.py +++ b/tests/fortran/functions/semantics/test_fortran_function_semantics.py @@ -2,7 +2,6 @@ from prik.semantics.fortran2ir import ( fortran_file_to_semantic_modules, - fortran_module_to_semantic_module, ) from prik.semantics.models import ProjectionMapping from tests.fortran._support.semantic_conversion import get_function @@ -32,31 +31,3 @@ def test_missing_intent_scalar_uses_conservative_replacement_projection(): result_position=1, ) ] - - -def test_function_result(): - source = """ -module func_mod - -contains - -function norm2(x) result(r) - - real(8), intent(in) :: x(:) - - real(8) :: r - -end function - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - func = get_function(smod, "norm2") - - assert func.return_type is not None - - assert func.return_type.name == "Float64" diff --git a/tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py b/tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py index 0dbcc9440..d8217fd21 100644 --- a/tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py +++ b/tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py @@ -12,7 +12,6 @@ from prik.codegen import CBindingGenerator from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -from prik.codegen.c.naming import CBindingNames DEFINED_OPERATORS = Path(__file__).parents[1] / "end_to_end/fixtures/contracts/foperators_f90/foperators_f90.pyi" @@ -74,35 +73,12 @@ def test_policy_completes_builtin_scalar_family_only_for_reflected_dispatch(): ) -def test_binding_lowers_public_overload_to_candidate_id_switch(): - plan = _plan() - overload = plan.namespaces[0].overloads[0] - - artifacts = WrapperGenerator().generate(plan) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - dispatcher = CBindingNames.overload_dispatch_function(overload) - - assert f'{{"convert", (PyCFunction){dispatcher}, METH_VARARGS | METH_KEYWORDS' in c_source - assert "int candidate_id = -1;" in c_source - assert "switch (candidate_id)" in c_source - assert "case 0: {" in c_source - assert "case 1: {" in c_source - assert "wrap__prik_overload_convert_0(self, candidate_args, candidate_kwargs)" in c_source - assert "wrap__prik_overload_convert_1(self, candidate_args, candidate_kwargs)" in c_source - assert "PyRun_String" not in c_source - - def test_binding_uses_numpy_bool_scalar_predicate_for_storage_specific_logicals(): + """Every logical storage width dispatches on `numpy.bool_`; only the default kind is built end to end.""" assert { name: CBindingGenerator._overload_numpy_scalar_kind(name) for name in ("Bool", "Bool8", "Bool16", "Bool32", "Bool64") - } == { - "Bool": "Bool", - "Bool8": "Bool", - "Bool16": "Bool", - "Bool32": "Bool", - "Bool64": "Bool", - } + } == dict.fromkeys(("Bool", "Bool8", "Bool16", "Bool32", "Bool64"), "Bool") def test_generator_rejects_ambiguous_edited_overload_plan_before_emission(): @@ -119,25 +95,21 @@ def test_generator_rejects_ambiguous_edited_overload_plan_before_emission(): WrapperGenerator().generate(invalid) -def test_generator_rejects_duplicate_candidate_ids_before_emission(): +@pytest.mark.parametrize( + ("candidate_ids", "code"), + [ + pytest.param((0, 0), "duplicate-overload-candidate-id", id="duplicate-ids"), + pytest.param((-1, 1), "invalid-overload-candidate-id", id="id-reserved-for-no-match"), + ], +) +def test_generator_rejects_invalid_candidate_ids_before_emission(candidate_ids, code: str): + """`-1` means no candidate matched, so every candidate needs its own non-negative id.""" plan = _plan() namespace = plan.namespaces[0] - overload = namespace.overloads[0] - duplicate_ids = replace(overload, candidate_ids=(0, 0)) - invalid = replace(plan, namespaces=(replace(namespace, overloads=(duplicate_ids,)),)) - - with pytest.raises(ValueError, match="duplicate-overload-candidate-id"): - WrapperGenerator().generate(invalid) - - -def test_generator_rejects_candidate_id_reserved_for_no_match(): - plan = _plan() - namespace = plan.namespaces[0] - overload = namespace.overloads[0] - invalid_ids = replace(overload, candidate_ids=(-1, 1)) - invalid = replace(plan, namespaces=(replace(namespace, overloads=(invalid_ids,)),)) + invalid_overload = replace(namespace.overloads[0], candidate_ids=candidate_ids) + invalid = replace(plan, namespaces=(replace(namespace, overloads=(invalid_overload,)),)) - with pytest.raises(ValueError, match="invalid-overload-candidate-id"): + with pytest.raises(ValueError, match=code): WrapperGenerator().generate(invalid) diff --git a/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py b/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py index 26db5bb1b..01e2d66b1 100644 --- a/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py +++ b/tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py @@ -88,37 +88,32 @@ def test_fortran_generic_interfaces_dispatch_in_generated_c_extension( value.add(np.complex128(1.0 + 0.0j)) -def test_public_generic_dispatches_to_private_inline_submodule_specifics(tmp_path: Path): +def test_a_public_generic_reaches_each_specific_by_the_route_its_accessibility_allows(tmp_path: Path): + """Private inline specifics go through the generic; public interface-body specifics are called by name. + + Both modules are built into one extension: a specific the module keeps + private is only reachable through the public generic's name, while a + public specific an interface body declares needs no route through it. + """ module, _payload = _build_sources_and_import( [ ("private_inline_generic.f90", PRIVATE_INLINE_GENERIC_MODULE), ("private_inline_generic_impl.f90", PRIVATE_INLINE_GENERIC_SUBMODULE), + ("interface_body_generic.f90", INTERFACE_BODY_GENERIC_MODULE), + ("interface_body_generic_impl.f90", INTERFACE_BODY_GENERIC_IMPL), ], tmp_path, ) assert module.private_inline_generic.shift(np.int32(4)) == np.int32(5) assert module.private_inline_generic.shift(np.float64(4.0)) == np.float64(4.5) - bridge = (tmp_path / "bind_c_private_inline_generic_wrapper.f90").read_text(encoding="utf-8").lower() + assert module.interface_body_generic.scale(np.int32(4)) == np.int32(8) + assert module.interface_body_generic.scale(np.float64(4.0)) == np.float64(10.0) + bridge = next(tmp_path.glob("bind_c_*_wrapper.f90")).read_text(encoding="utf-8").lower() assert "native__prik_overload_shift_0 => shift" in bridge assert "native__prik_overload_shift_1 => shift" in bridge assert "=> shift_integer" not in bridge assert "=> shift_real" not in bridge - - -def test_public_generic_calls_public_interface_body_specifics_by_their_own_names(tmp_path: Path): - """A public specific an interface body declares needs no route through the generic.""" - module, _payload = _build_sources_and_import( - [ - ("interface_body_generic.f90", INTERFACE_BODY_GENERIC_MODULE), - ("interface_body_generic_impl.f90", INTERFACE_BODY_GENERIC_IMPL), - ], - tmp_path, - ) - - assert module.interface_body_generic.scale(np.int32(4)) == np.int32(8) - assert module.interface_body_generic.scale(np.float64(4.0)) == np.float64(10.0) - bridge = (tmp_path / "bind_c_interface_body_generic_wrapper.f90").read_text(encoding="utf-8").lower() assert "native__prik_overload_scale_0 => scale_integer" in bridge assert "native__prik_overload_scale_1 => scale_real" in bridge diff --git a/tests/fortran/generic_interfaces/pipeline/test_generated_generic_contracts.py b/tests/fortran/generic_interfaces/pipeline/test_generated_generic_contracts.py index f6666d612..2a49d6711 100644 --- a/tests/fortran/generic_interfaces/pipeline/test_generated_generic_contracts.py +++ b/tests/fortran/generic_interfaces/pipeline/test_generated_generic_contracts.py @@ -17,22 +17,14 @@ FIXTURES = Path(__file__).parents[1] / "end_to_end" / "fixtures" CONTRACT_ROOT = FIXTURES / "contracts" +# `foverloads_f90` and `foperators_f90` are compared in their end-to-end +# generated-`.pyi` lanes; the fixed-form source is only generated, never built. CASES = ( - GeneratedContractCase( - "foverloads_f90", - (FIXTURES / "native" / "foverloads_f90.f90",), - CONTRACT_ROOT / "foverloads_f90", - ), GeneratedContractCase( "foverloads_fixed", (FIXTURES / "native" / "foverloads_fixed.f",), CONTRACT_ROOT / "foverloads_fixed", ), - GeneratedContractCase( - "foperators_f90", - (FIXTURES / "native" / "foperators_f90.f90",), - CONTRACT_ROOT / "foperators_f90", - ), ) diff --git a/tests/fortran/infrastructure/building/compiling/test_compiler_verbose.py b/tests/fortran/infrastructure/building/compiling/test_compiler_verbose.py index f16f4f527..432aef0f8 100644 --- a/tests/fortran/infrastructure/building/compiling/test_compiler_verbose.py +++ b/tests/fortran/infrastructure/building/compiling/test_compiler_verbose.py @@ -10,27 +10,6 @@ from prik.compiler.compiler_profiles import available_compilers, fortran_compiler_family, vendors -def test_record_only_compiler_keeps_object_command_without_executing(monkeypatch, tmp_path: Path): - compiler = Compiler("GNU", execute_commands=False) - monkeypatch.setattr(compiler, "_executable", lambda _language, _tools: "gcc") - monkeypatch.setattr( - Compiler, - "run_command", - lambda *_args, **_kwargs: (_ for _ in ()).throw(AssertionError("command executed")), - ) - object_file = ObjectFile( - source=tmp_path / "source.c", - object_path=tmp_path / "source.o", - language="c", - ) - - compiler.compile_object(object_file) - - command = compiler.command_log[0] - assert command[0] == "gcc" - assert command[-4:] == ("-c", str(object_file.source), "-o", str(object_file.object_path)) - - def test_user_compile_flags_follow_default_profile_flags(monkeypatch, tmp_path: Path): compiler = Compiler("GNU", debug=False, execute_commands=False) monkeypatch.setattr(compiler, "_executable", lambda _language, _tools: "gcc") @@ -249,26 +228,13 @@ def test_python_include_directories_add_existing_multiarch_root(monkeypatch, tmp ) -def test_supported_optional_profile_flags_are_used_when_executing(monkeypatch, tmp_path: Path): +@pytest.mark.parametrize("supported", [True, False], ids=["supported", "unsupported"]) +def test_optional_profile_flags_follow_the_selected_compiler_support(monkeypatch, tmp_path: Path, supported: bool): compiler = Compiler("GNU") monkeypatch.setattr(compiler, "_executable", lambda _language, _tools: "gfortran") - monkeypatch.setattr(compiler, "_supports_optional_flag", lambda _executable, flag: flag == "-ftrampoline-impl=heap") - monkeypatch.setattr(Compiler, "run_command", staticmethod(lambda command, _verbose=False: tuple(command))) - object_file = ObjectFile( - source=tmp_path / "bridge.f90", - object_path=tmp_path / "bridge.o", - language="fortran", + monkeypatch.setattr( + compiler, "_supports_optional_flag", lambda _executable, flag: supported and flag == "-ftrampoline-impl=heap" ) - - compiler.compile_object(object_file) - - assert "-ftrampoline-impl=heap" in compiler.command_log[0] - - -def test_unsupported_optional_profile_flags_are_omitted(monkeypatch, tmp_path: Path): - compiler = Compiler("GNU") - monkeypatch.setattr(compiler, "_executable", lambda _language, _tools: "gfortran") - monkeypatch.setattr(compiler, "_supports_optional_flag", lambda _executable, _flag: False) monkeypatch.setattr(Compiler, "run_command", staticmethod(lambda command, _verbose=False: tuple(command))) object_file = ObjectFile( source=tmp_path / "bridge.f90", @@ -278,7 +244,7 @@ def test_unsupported_optional_profile_flags_are_omitted(monkeypatch, tmp_path: P compiler.compile_object(object_file) - assert "-ftrampoline-impl=heap" not in compiler.command_log[0] + assert ("-ftrampoline-impl=heap" in compiler.command_log[0]) is supported def test_optional_profile_flag_probe_reads_the_selected_compiler_help(monkeypatch): diff --git a/tests/fortran/infrastructure/building/compiling/test_example_native_library.py b/tests/fortran/infrastructure/building/compiling/test_example_native_library.py index 1f3990b09..990550141 100644 --- a/tests/fortran/infrastructure/building/compiling/test_example_native_library.py +++ b/tests/fortran/infrastructure/building/compiling/test_example_native_library.py @@ -91,10 +91,8 @@ def fail_if_recompiled(*_args) -> None: @pytest.mark.parametrize( ("platform", "library", "expected_dependencies", "suffix"), ( - ("linux", "blas", (), ".so"), - ("linux", "lapack", ("-llapack", "-lblas"), ".so"), - ("darwin", "blas", (), ".dylib"), - ("darwin", "lapack", ("-llapack", "-lblas"), ".dylib"), + pytest.param("linux", "lapack", ("-llapack", "-lblas"), ".so", id="linux-lapack"), + pytest.param("darwin", "blas", (), ".dylib", id="darwin-blas"), ), ) def test_shared_example_library_links_its_native_dependencies( diff --git a/tests/fortran/infrastructure/building/compiling/test_logical_interop_flags.py b/tests/fortran/infrastructure/building/compiling/test_logical_interop_flags.py index 49df78047..d7d4fcb6b 100644 --- a/tests/fortran/infrastructure/building/compiling/test_logical_interop_flags.py +++ b/tests/fortran/infrastructure/building/compiling/test_logical_interop_flags.py @@ -6,6 +6,9 @@ from prik.compiler.objects import ObjectFile +LOGICAL_OPTIONS = ("-standard-semantics", "-Munixlogical") + + def _fortran_compile_command(vendor: str, *, standard_logicals: bool, tmp_path: Path) -> tuple[str, ...]: compiler = Compiler(vendor, execute_commands=False, standard_logicals=standard_logicals) compiler._executable = lambda _language, _tools: "fc" @@ -19,32 +22,6 @@ def _fortran_compile_command(vendor: str, *, standard_logicals: bool, tmp_path: return compiler.command_log[0] -@pytest.mark.parametrize( - ("vendor", "option"), - [("intel", "-standard-semantics"), ("PGI", "-Munixlogical"), ("nvidia", "-Munixlogical")], -) -def test_fortran_compilation_requests_the_interoperable_logical_by_default(vendor: str, option: str, tmp_path: Path): - """A logical must reach C as 0 or 1, so the vendor option is on without being asked for.""" - assert option in _fortran_compile_command(vendor, standard_logicals=True, tmp_path=tmp_path) - - -@pytest.mark.parametrize( - ("vendor", "option"), - [("intel", "-standard-semantics"), ("PGI", "-Munixlogical"), ("nvidia", "-Munixlogical")], -) -def test_standard_logicals_can_be_turned_off_for_prebuilt_objects(vendor: str, option: str, tmp_path: Path): - """Opting out is the only way to link objects built without the option, whose mangling differs.""" - assert option not in _fortran_compile_command(vendor, standard_logicals=False, tmp_path=tmp_path) - - -@pytest.mark.parametrize("vendor", ["GNU", "LLVM"]) -def test_compilers_that_already_interoperate_add_no_logical_option(vendor: str, tmp_path: Path): - """gfortran and Flang already store .true. as 1, so they must stay flag-free.""" - command = _fortran_compile_command(vendor, standard_logicals=True, tmp_path=tmp_path) - assert "-standard-semantics" not in command - assert "-Munixlogical" not in command - - @pytest.mark.parametrize( ("vendor", "expected"), [ @@ -52,16 +29,24 @@ def test_compilers_that_already_interoperate_add_no_logical_option(vendor: str, ("PGI", ("-Munixlogical",)), ("nvidia", ("-Munixlogical",)), ("GNU", ()), + ("LLVM", ()), ], ) -def test_compiler_exposes_required_logical_abi_flags_for_external_builds(vendor: str, expected: tuple[str, ...]): - compiler = Compiler(vendor, execute_commands=False) - - assert compiler.required_abi_flags("fortran") == expected - assert compiler.required_abi_flags("c") == () - - -def test_external_build_can_disable_required_logical_abi_flags(): - compiler = Compiler("intel", execute_commands=False, standard_logicals=False) - - assert compiler.required_abi_flags("fortran") == () +def test_vendor_logical_interop_option_is_on_by_default_and_can_be_turned_off( + vendor: str, expected: tuple[str, ...], tmp_path: Path +): + """A logical must reach C as 0 or 1, so the vendor option is on without being asked for. + + gfortran and Flang already store .true. as 1, so they stay flag-free. Opting + out is the only way to link objects built without the option, whose mangling + differs, and it also clears the flags external builds are told to use. + """ + default_command = _fortran_compile_command(vendor, standard_logicals=True, tmp_path=tmp_path) + assert tuple(option for option in LOGICAL_OPTIONS if option in default_command) == expected + default_compiler = Compiler(vendor, execute_commands=False) + assert default_compiler.required_abi_flags("fortran") == expected + assert default_compiler.required_abi_flags("c") == () + + disabled_command = _fortran_compile_command(vendor, standard_logicals=False, tmp_path=tmp_path) + assert not any(option in disabled_command for option in LOGICAL_OPTIONS) + assert Compiler(vendor, execute_commands=False, standard_logicals=False).required_abi_flags("fortran") == () diff --git a/tests/fortran/infrastructure/building/compiling/test_support_probe_artifacts.py b/tests/fortran/infrastructure/building/compiling/test_support_probe_artifacts.py deleted file mode 100644 index 4552e8f81..000000000 --- a/tests/fortran/infrastructure/building/compiling/test_support_probe_artifacts.py +++ /dev/null @@ -1,19 +0,0 @@ -"""Artifact-isolation checks for native capability probes used by tests.""" - -from pathlib import Path - -from tests.fortran._support import wrapper_build - - -def test_allocatable_result_probe_keeps_fortran_modules_out_of_invocation_directory( - tmp_path: Path, - monkeypatch, -) -> None: - invocation_dir = tmp_path / "invocation" - invocation_dir.mkdir() - monkeypatch.chdir(invocation_dir) - wrapper_build._supports_maybe_unallocated_function_result.cache_clear() - - wrapper_build._supports_maybe_unallocated_function_result() - - assert not tuple(invocation_dir.glob("*.mod")) diff --git a/tests/fortran/infrastructure/building/end_to_end/test_cmake_builds.py b/tests/fortran/infrastructure/building/end_to_end/test_cmake_builds.py index cddaedf81..38104dea2 100644 --- a/tests/fortran/infrastructure/building/end_to_end/test_cmake_builds.py +++ b/tests/fortran/infrastructure/building/end_to_end/test_cmake_builds.py @@ -6,10 +6,12 @@ import json import os from pathlib import Path +import re import shutil import subprocess import sys import tempfile +from types import SimpleNamespace import venv import numpy as np @@ -88,15 +90,24 @@ def _configure_and_build( _run(["cmake", "--build", str(build), "-j2"], environment=environment) +def _extension_artifact(module_name: str, build: Path) -> Path: + artifacts = tuple( + path + for path in build.rglob(f"{module_name}.*") + if path.suffix == ".so" and path.name.split(".")[0] == module_name + ) + assert len(artifacts) == 1, f"expected one {module_name} extension artifact in {build}: {artifacts}" + return artifacts[0] + + def _import_extension(module_name: str, build: Path): - artifacts = tuple(build.rglob(f"{module_name}*.so")) - assert artifacts, f"no CMake extension artifact in {build}" + directory = str(_extension_artifact(module_name, build).parent) sys.modules.pop(module_name, None) - sys.path.insert(0, str(artifacts[0].parent)) + sys.path.insert(0, directory) try: return importlib.import_module(module_name) finally: - sys.path.remove(str(artifacts[0].parent)) + sys.path.remove(directory) def _call_extension( @@ -108,10 +119,9 @@ def _call_extension( an extension module for the life of the interpreter, so a second import returns the shared library the first one loaded. """ - artifacts = tuple(build.rglob(f"{module_name}*.so")) - assert artifacts, f"no CMake extension artifact in {build}" environment = dict(environment or _environment()) - environment["PYTHONPATH"] = str(artifacts[0].parent) + os.pathsep + environment["PYTHONPATH"] + directory = str(_extension_artifact(module_name, build).parent) + environment["PYTHONPATH"] = directory + os.pathsep + environment["PYTHONPATH"] program = f"import numpy, {module_name}\nprint({expression})\n" return _run([sys.executable, "-c", program], environment=environment).stdout.strip() @@ -166,403 +176,467 @@ def _cmake_finds_blas() -> bool: return result.returncode == 0 -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_builds_source_first_fortran_module(tmp_path: Path): - project = tmp_path / "user project with spaces" - project.mkdir() - (project / "square.f90").write_text( - """real(8) function square(x) result(y) - real(8), intent(in) :: x - y = x * x -end function square -""", - encoding="utf-8", - ) - _write_project( - project, - """prik_add_module( - square - SOURCES square.f90 - FORTRAN_FLAGS -O0 +REQUIRES_CMAKE_FORTRAN = pytest.mark.skipif( + shutil.which("cmake") is None + or shutil.which("gfortran") is None + or shutil.which("gcc") is None + or shutil.which("ar") is None, + reason="CMake, gfortran, gcc, and ar are required", ) -""", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran", build_project=False) - assert not tuple((build / "prik" / "square").glob("*_wrapper.*")) - _run(["cmake", "--build", str(build), "-j2"]) - native_support_header = build / "prik" / "square" / "binding_support" / "prik_binding.h" - assert native_support_header.is_file() - native_support_header.unlink() - _run(["cmake", "--build", str(build), "-j2"]) - assert native_support_header.is_file() - module = _import_extension("square", build) - assert module.square(np.float64(3.0)) == np.float64(9.0) +GENERATION_COMMENT = re.compile(r"Generate PRIK wrapper sources for (\w+)") -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_preserves_per_source_compile_flags(tmp_path: Path): - project = tmp_path / "compile flag scopes" - project.mkdir() - (project / "native.f90").write_text( - "real(8) function native_value(x) result(y)\n real(8), intent(in) :: x\n y = x\nend function native_value\n", - encoding="utf-8", - ) - (project / "support.c").write_text("int prik_native_support(void) { return 0; }\n", encoding="utf-8") - _write_project( - project, - """set(CMAKE_EXPORT_COMPILE_COMMANDS ON) +def _regenerated(output: str) -> set[str]: + """Return the modules whose wrapper generation a build ran.""" + return set(GENERATION_COMMENT.findall(output)) + + +def _compile_archive(source: Path, archive: Path) -> Path: + native_object = archive.with_suffix(".o") + _run([shutil.which("gfortran"), "-fPIC", "-c", str(source), "-o", str(native_object)]) + _run([shutil.which("ar"), "rcs", str(archive), str(native_object)]) + native_object.unlink() + return archive + + +def _write_files(root: Path, files: dict[str, str]) -> None: + for relative, text in files.items(): + path = root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + + +SHARED_PROJECT_FILES = { + "square.f90": "real(8) function square(x) result(y)\n real(8), intent(in) :: x\n y = x * x\nend function square\n", + "math_mod.f90": ( + "module math_mod\ncontains\n real(8) function add(a, b) result(c)\n real(8), intent(in) :: a, b\n" + " c = a + b\n end function add\nend module math_mod\n" + ), + "flag_native.f90": ( + "real(8) function native_value(x) result(y)\n real(8), intent(in) :: x\n y = x\nend function native_value\n" + ), + "flag_support.c": "int prik_native_support(void) { return 0; }\n", + "common.f90": ( + "real(8) function common_value(value) result(result)\n real(8), intent(in) :: value\n" + " result = value\nend function common_value\n" + ), + "include files/cmath_api.h": "double c_add(double, double);\n", + "cexample.c": ( + '#include "cmath_api.h"\n#ifdef PRIK_CMAKE_TEST_FLAG\n' + "double c_add(double a, double b) { return a + b + 1.0; }\n#else\n" + "double c_add(double a, double b) { return a + b; }\n#endif\n" + ), + "dependency_interface.c": "double dependency_add(double value);\n", + "dependency_implementation.c": ( + "#ifndef PRIK_REQUIRED_DEFINE\n#error missing dependency compile definition\n#endif\n" + "double dependency_add(double value) { return value + 1.0; }\n" + ), + "external_math.f90": ( + "real(8) function native_add(x, y) result(z)\n real(8), intent(in) :: x, y\n z = x + y\n" + "end function native_add\n" + ), + "external_wrapper.f90": ( + "real(8) function call_native(x, y) result(z)\n real(8), intent(in) :: x, y\n interface\n" + " function native_add(a, b) result(c)\n real(8), intent(in) :: a, b\n real(8) :: c\n" + " end function native_add\n end interface\n z = native_add(x, y)\nend function call_native\n" + ), + "target_interface.f90": ( + "real(8) function target_square(value) result(result)\n real(8), intent(in) :: value\n" + " result = value * value\nend function target_square\n" + ), + "target_implementation.f90": ( + "real(8) function target_square(value) result(result)\n real(8), intent(in) :: value\n" + " result = value * value\nend function target_square\n" + ), + "raw_interface.f90": ( + "integer(c_int) function raw_add_two(value) bind(C, name='raw_add_two_symbol') result(output)\n" + " use iso_c_binding, only: c_int\n integer(c_int), value, intent(in) :: value\n" + " character(len=16) :: buffer\n write(buffer, '(I0)') value\n read(buffer, *) output\n" + " output = output + 2_c_int\nend function raw_add_two\n" + ), + "c_contract/api.pyi": "from prik.contracts import Float64\n\ndef add_one(value: Float64) -> Float64: ...\n", + "c_contract_implementation.f90": ( + "real(c_double) function add_one(value) bind(C, name='add_one') result(result)\n" + " use iso_c_binding, only: c_double\n real(c_double), value, intent(in) :: value\n" + " result = value + 1.0_c_double\nend function add_one\n" + ), + "c_contract_source/api.pyi": "from prik.contracts import Float64\n\ndef add_two(value: Float64) -> Float64: ...\n", + "c_contract_source_implementation.f90": ( + "real(c_double) function add_two(value) bind(C, name='add_two') result(result)\n" + " use iso_c_binding, only: c_double\n real(c_double), value, intent(in) :: value\n" + " result = value + 2.0_c_double\nend function add_two\n" + ), + "adapter_plain.c": "double capi_add(double value) { return value + 1.0; }\n", + "adapter_adapted.c": "double capi_add(double value) { return value + 1.0; }\n", +} + +SHARED_PROJECT_MODULES = f"""set(CMAKE_EXPORT_COMPILE_COMMANDS ON) +prik_add_module(square SOURCES square.f90 FORTRAN_FLAGS -O0) +prik_add_module(math_mod SOURCES math_mod.f90) prik_add_module( compile_flag_scopes - SOURCES native.f90 - C_SOURCES support.c + SOURCES flag_native.f90 + C_SOURCES flag_support.c FORTRAN_FLAGS -DPRIK_NATIVE_FORTRAN C_FLAGS -DPRIK_NATIVE_C WRAPPER_FORTRAN_FLAGS -DPRIK_WRAPPER_FORTRAN WRAPPER_C_FLAGS -DPRIK_WRAPPER_C ) -""", +prik_add_module(first SOURCES common.f90 FORTRAN_FLAGS -DFIRST_MODULE) +prik_add_module(second SOURCES common.f90 FORTRAN_FLAGS -DSECOND_MODULE) +prik_add_module( + free_external + CONTRACT "{BRIDGE_CONTRACT.as_posix()}" + FORTRAN_SOURCES "{BRIDGE_NATIVE.as_posix()}" +) +prik_add_module( + cexample + C_SOURCES cexample.c + INCLUDE_DIRS "${{CMAKE_CURRENT_SOURCE_DIR}}/include files" + C_FLAGS -DPRIK_CMAKE_TEST_FLAG + PRIK_ARGS --define PRIK_CMAKE_TEST_FLAG +) +add_library(native_dependency INTERFACE) +target_compile_definitions(native_dependency INTERFACE PRIK_REQUIRED_DEFINE) +prik_add_module( + dependency_usage + SOURCES dependency_interface.c + C_SOURCES dependency_implementation.c + LINK_LIBRARIES native_dependency +) +add_library(external_math STATIC external_math.f90) +prik_add_module(external_target SOURCES external_wrapper.f90 LINK_LIBRARIES external_math) +add_library(target_math STATIC target_implementation.f90) +prik_add_module( + target_only + SOURCES target_interface.f90 + NO_COMPILE_INPUT_SOURCES + LINK_LIBRARIES target_math +) +prik_add_module( + raw_archive + SOURCES raw_interface.f90 + NO_COMPILE_INPUT_SOURCES + LINKER_LANGUAGE Fortran + LINK_LIBRARIES "${{CMAKE_CURRENT_SOURCE_DIR}}/libraw_math.a" +) +prik_add_module( + c_contract + CONTRACT c_contract/api.pyi + NATIVE_LANGUAGE C + LINKER_LANGUAGE Fortran + LINK_LIBRARIES "${{CMAKE_CURRENT_SOURCE_DIR}}/libimplementation.a" +) +prik_add_module( + c_contract_source + CONTRACT c_contract_source/api.pyi + NATIVE_LANGUAGE C + FORTRAN_SOURCES c_contract_source_implementation.f90 +) +prik_add_module(adapter_plain C_SOURCES adapter_plain.c) +prik_add_module(adapter_adapted C_SOURCES adapter_adapted.c PRIK_ARGS --collision-adapter-all) +""" + + +@pytest.fixture(scope="module") +def shared_project(tmp_path_factory: pytest.TempPathFactory) -> SimpleNamespace: + """Configure and build one CMake project that finds PRIK and declares many independent modules. + + Every module here is read-only evidence for the tests below, so one + configure and one parallel build serve all of them. Tests that edit a + project after it is built keep their own project. + """ + if any(shutil.which(tool) is None for tool in ("cmake", "gfortran", "gcc", "ar")): + pytest.skip("CMake, gfortran, gcc, and ar are required") + project = tmp_path_factory.mktemp("use-prik") / "shared project with spaces" + _write_files(project, SHARED_PROJECT_FILES) + raw_source = project / "raw_implementation.f90" + raw_source.write_text(SHARED_PROJECT_FILES["raw_interface.f90"], encoding="utf-8") + _compile_archive(raw_source, project / "libraw_math.a") + _compile_archive(project / "c_contract_implementation.f90", project / "libimplementation.a") + (project / "CMakeLists.txt").write_text( + "cmake_minimum_required(VERSION 3.21)\n" + "project(cmake_test LANGUAGES C Fortran)\n" + "find_package(Python COMPONENTS Interpreter Development.Module REQUIRED)\n" + "find_package(PRIK CONFIG REQUIRED)\n" + SHARED_PROJECT_MODULES, + encoding="utf-8", ) build = project / "build" - _configure_and_build(project, build, language="fortran") - commands = json.loads((build / "compile_commands.json").read_text(encoding="utf-8")) + _configure_and_build( + project, + build, + language="fortran", + build_project=False, + defines=(f"PRIK_DIR={USE_PRIK_DIR.as_posix()}",), + ) + configured_wrappers = tuple((build / "prik").rglob("*_wrapper.*")) if (build / "prik").exists() else () + _run(["cmake", "--build", str(build), "-j2"]) + return SimpleNamespace(project=project, build=build, configured_wrappers=configured_wrappers) + + +@pytest.mark.fortran_end_to_end +def test_find_package_prik_builds_source_contract_and_c_modules_at_build_time(shared_project: SimpleNamespace): + """``find_package(PRIK CONFIG)`` provides the helper, and configure plans without generating.""" + build = shared_project.build + assert shared_project.configured_wrappers == () + + square = _import_extension("square", build) + assert square.square(np.float64(3.0)) == np.float64(9.0) + math_mod = _import_extension("math_mod", build) + assert math_mod.math_mod.add(np.float64(2.0), np.float64(3.0)) == np.float64(5.0) + + generated = build / "prik" / "free_external" + assert tuple(generated.glob("*.f90")), "the generated Fortran bridge is missing" + assert tuple(generated.glob("*.c")), "the generated C binding is missing" + free_external = _import_extension("free_external", build) + assert free_external.free_square(np.int32(6)) == np.int32(36) + + # The C flag reaches both native compilation and semantic preprocessing, + # and the include directory with a space in its path reaches both too. + cexample = _import_extension("cexample", build) + assert cexample.c_add(np.float64(2.0), np.float64(3.0)) == np.float64(6.0) + + # A C contract may be implemented by native Fortran sources. + c_contract_source = _import_extension("c_contract_source", build) + assert c_contract_source.add_two(np.float64(5.0)) == np.float64(7.0) + + +@pytest.mark.fortran_end_to_end +def test_use_prik_cmake_scopes_native_and_wrapper_flags_per_source_and_target(shared_project: SimpleNamespace): + commands = json.loads((shared_project.build / "compile_commands.json").read_text(encoding="utf-8")) by_name = {Path(record["file"]).name: record["command"] for record in commands} - assert "-DPRIK_NATIVE_FORTRAN" in by_name["native.f90"] - assert "-DPRIK_WRAPPER_FORTRAN" not in by_name["native.f90"] - assert "-DPRIK_NATIVE_C" in by_name["support.c"] - assert "-DPRIK_WRAPPER_C" not in by_name["support.c"] - bridge_command = next(command for name, command in by_name.items() if name.endswith("_wrapper.f90")) - binding_command = next(command for name, command in by_name.items() if name.endswith("_wrapper.c")) + assert "-DPRIK_NATIVE_FORTRAN" in by_name["flag_native.f90"] + assert "-DPRIK_WRAPPER_FORTRAN" not in by_name["flag_native.f90"] + assert "-DPRIK_NATIVE_C" in by_name["flag_support.c"] + assert "-DPRIK_WRAPPER_C" not in by_name["flag_support.c"] + bridge_command = by_name["bind_c_compile_flag_scopes_wrapper.f90"] + binding_command = by_name["compile_flag_scopes_wrapper.c"] assert "-DPRIK_WRAPPER_FORTRAN" in bridge_command assert "-DPRIK_NATIVE_FORTRAN" not in bridge_command assert "-DPRIK_WRAPPER_C" in binding_command assert "-DPRIK_NATIVE_C" not in binding_command - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_keeps_native_flags_target_local(tmp_path: Path): - project = tmp_path / "shared native source" - project.mkdir() - source = project / "common.f90" - source.write_text( - "real(8) function common_value(value) result(result)\n" - " real(8), intent(in) :: value\n" - " result = value\n" - "end function common_value\n", - encoding="utf-8", - ) - _write_project( - project, - """set(CMAKE_EXPORT_COMPILE_COMMANDS ON) -prik_add_module(first SOURCES common.f90 FORTRAN_FLAGS -DFIRST_MODULE) -prik_add_module(second SOURCES common.f90 FORTRAN_FLAGS -DSECOND_MODULE) -""", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - - commands = json.loads((build / "compile_commands.json").read_text(encoding="utf-8")) - native_commands = [record["command"] for record in commands if Path(record["file"]).resolve() == source.resolve()] + # One native source shared by two modules compiles once per target, each + # with only its own module's flags. + common = (shared_project.project / "common.f90").resolve() + native_commands = [record["command"] for record in commands if Path(record["file"]).resolve() == common] assert len(native_commands) == 2 assert any("-DFIRST_MODULE" in command and "-DSECOND_MODULE" not in command for command in native_commands) assert any("-DSECOND_MODULE" in command and "-DFIRST_MODULE" not in command for command in native_commands) @pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_maps_required_logical_abi_flags(tmp_path: Path): - project = tmp_path / "logical abi flags" - toolchain = project / "toolchain" - toolchain.mkdir(parents=True) - for name, compiler in (("ifort", shutil.which("gfortran")), ("icx", shutil.which("gcc"))): - executable = toolchain / name - executable.write_text(f'#!/bin/sh\nexec "{compiler}" "$@"\n', encoding="utf-8") - executable.chmod(0o755) - source_text = ( - "logical function logical_identity(value) result(output)\n" - " logical, intent(in) :: value\n" - " output = value\n" - "end function logical_identity\n" - ) - (project / "logical_default.f90").write_text(source_text, encoding="utf-8") - (project / "logical_disabled.f90").write_text(source_text, encoding="utf-8") - _write_project( - project, - """set(CMAKE_EXPORT_COMPILE_COMMANDS ON) -prik_add_module(abi_default SOURCES logical_default.f90) -prik_add_module(abi_disabled SOURCES logical_disabled.f90 NO_STANDARD_LOGICALS) -""", - ) - build = project / "build" - _run( - [ - "cmake", - "-S", - str(project), - "-B", - str(build), - "-G", - "Ninja" if shutil.which("ninja") else "Unix Makefiles", - f"-DCMAKE_C_COMPILER={shutil.which('gcc')}", - f"-DCMAKE_Fortran_COMPILER={toolchain / 'ifort'}", - ] - ) - commands = json.loads((build / "compile_commands.json").read_text(encoding="utf-8")) - default_commands = [record["command"] for record in commands if "abi_default" in record["command"]] - disabled_commands = [record["command"] for record in commands if "abi_disabled" in record["command"]] +def test_use_prik_cmake_links_native_implementations_from_targets_and_archives(shared_project: SimpleNamespace): + build = shared_project.build + + # An INTERFACE dependency's usage requirements reach the native C objects. + dependency_usage = _import_extension("dependency_usage", build) + assert dependency_usage.dependency_add(np.float64(2.0)) == np.float64(3.0) + # A normal CMake library target links beside compiled semantic sources. + external_target = _import_extension("external_target", build) + assert external_target.call_native(np.float64(2.0), np.float64(3.0)) == np.float64(5.0) + # With input compilation off, the target is the only native implementation. + target_only = _import_extension("target_only", build) + assert target_only.target_square(np.float64(4.0)) == np.float64(16.0) + # A raw Fortran archive that needs the Fortran runtime links with the Fortran driver. + raw_archive = _import_extension("raw_archive", build) + assert raw_archive.raw_add_two(np.int32(5)) == np.int32(7) + # A C contract keeps its contract language while Fortran links its archive. + c_contract = _import_extension("c_contract", build) + assert c_contract.add_one(np.float64(5.0)) == np.float64(6.0) - assert default_commands - assert any("-standard-semantics" in command for command in default_commands) - assert disabled_commands - assert all("-standard-semantics" not in command for command in disabled_commands) + +@pytest.mark.fortran_end_to_end +def test_use_prik_cmake_keeps_the_adapter_filename_fixed(shared_project: SimpleNamespace): + """The adapter unit is declared either way: a stub without adapters, real content with them.""" + results = {} + for label in ("plain", "adapted"): + module_name = f"adapter_{label}" + adapters = shared_project.build / "prik" / module_name / f"{module_name}_adapters.c" + assert adapters.is_file(), f"the deterministic adapter source is missing for {label}" + results[label] = adapters.read_text(encoding="utf-8") + module = _import_extension(module_name, shared_project.build) + assert module.capi_add(np.float64(2.0)) == np.float64(3.0) + + assert "prik_unused_adapter_stub" in results["plain"] + assert "prik_unused_adapter_stub" not in results["adapted"] + assert "capi_add" in results["adapted"] + + +ROUTING_DIRECT = """integer(c_int) function standalone_direct(value) bind(C, name="standalone_direct_symbol") result(output) + use iso_c_binding + integer(c_int), value, intent(in) :: value + + output = value + 2_c_int +end function standalone_direct +""" +ROUTING_MIXED = """integer(c_int) function standalone_direct(value) bind(C, name="standalone_mixed_direct") result(output) + use iso_c_binding + integer(c_int), value, intent(in) :: value + + output = value + 2_c_int +end function standalone_direct + +integer(c_int) function standalone_adapted(value) result(output) + use iso_c_binding + integer(c_int), intent(in) :: value + + output = value + 3_c_int +end function standalone_adapted +""" @pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_builds_a_fortran_module(tmp_path: Path): - project = tmp_path / "fortran module" - project.mkdir() - (project / "math_mod.f90").write_text( - """module math_mod -contains - real(8) function add(a, b) result(c) - real(8), intent(in) :: a, b - c = a + b - end function add -end module math_mod -""", +@REQUIRES_CMAKE_FORTRAN +def test_use_prik_cmake_rebuilds_regenerate_only_the_changed_module(tmp_path: Path): + """Each generation input is a build dependency of its own module, and nothing else is.""" + project = tmp_path / "incremental project" + native = project / "contract native.f90" + _write_files( + project, + { + "contract native.f90": ( + "real(8) function square(x) result(y)\n real(8), intent(in) :: x\n y = x * x\nend function square\n" + ), + "include/inner.h": "double c_square(double value);\n", + "include/api.h": '#include "inner.h"\n', + "c_header_module.c": '#include "api.h"\ndouble c_square(double value) { return value * value; }\n', + "declarations.inc": "implicit none\n real(8), intent(in) :: x\n", + "included.f90": ( + "real(8) function included_square(x) result(y)\n include 'declarations.inc'\n y = x * x\n" + "end function included_square\n" + ), + "routing.f90": ROUTING_DIRECT, + }, + ) + contracts = tmp_path / "contracts" + _run([sys.executable, "-m", "prik", "generate", "--pyi", str(native), "--out", str(contracts)]) + contract_dir = tmp_path / "contract_example" + contract_dir.mkdir() + contract_leaf = contract_dir / "marker.pyi" + (contract_dir / "__init__.pyi").write_text( + (contracts / "__init__.pyi").read_text(encoding="utf-8") + "\nfrom . import marker\n", encoding="utf-8", ) + contract_leaf.write_text("# included contract dependency\n", encoding="utf-8") _write_project( project, - """prik_add_module( - math_mod - SOURCES math_mod.f90 + f"""prik_add_module( + contract_example + CONTRACT "{(contract_dir / "__init__.pyi").as_posix()}" + FORTRAN_SOURCES "{native.as_posix()}" +) +prik_add_module( + c_header_dependency + C_SOURCES c_header_module.c + INCLUDE_DIRS "{(project / "include").as_posix()}" ) +prik_add_module(fortran_include_dependency FORTRAN_SOURCES included.f90) +prik_add_module(routing SOURCES routing.f90) """, ) build = project / "build" _configure_and_build(project, build, language="fortran") - module = _import_extension("math_mod", build) - assert module.math_mod.add(np.float64(2.0), np.float64(3.0)) == np.float64(5.0) + def rebuild(*arguments: str) -> str: + result = _run(["cmake", "--build", str(build), *arguments, "-j2"]) + return result.stdout + result.stderr -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_generate_cmake_builds_standalone_project_in_a_path_with_spaces(tmp_path: Path): - source = tmp_path / "standalone.f90" - source.write_text( - """real(8) function square(x) result(y) - real(8), intent(in) :: x - y = x * x -end function square -""", - encoding="utf-8", - ) - project = tmp_path / "generated project with spaces" - generated = _run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--cmake", - str(source), - "--module-name", - "generated_square", - "--out-dir", - str(project), - "--json", - ] - ) - assert Path(json.loads(generated.stdout)["cmake_project"]) == project / "CMakeLists.txt" - cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") - assert "include(UsePRIK)" in cmake_lists - assert "prik_add_module(\n generated_square" in cmake_lists - build = project / "cmake-build" - _configure_and_build(project, build, language="fortran", use_ninja=False) - module = _import_extension("generated_square", build) - assert module.square(np.float64(4.0)) == np.float64(16.0) + assert _regenerated(rebuild()) == set(), "an unchanged build reran PRIK generation" + native_support_header = build / "prik" / "routing" / "binding_support" / "prik_binding.h" + assert native_support_header.is_file() + native_support_header.unlink() + rebuild() + assert native_support_header.is_file() -@pytest.mark.fortran_end_to_end -def test_generate_cmake_preserves_ordered_native_link_items(tmp_path: Path): - source = tmp_path / "ordered.f90" - source.write_text("subroutine ordered()\nend subroutine ordered\n", encoding="utf-8") - archive = tmp_path / "libordered.a" - archive.touch() - project = tmp_path / "ordered project" - _run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--cmake", - str(source), - "--native-link-item", - "arg:-Wl,--start-group", - f"archive:{archive}", - "library:ordered", - "arg:-Wl,--end-group", - "--out-dir", - str(project), - ] - ) - cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") - ordered_items = ( - '"-Wl,--start-group"', - f'"${{CMAKE_CURRENT_LIST_DIR}}/{Path(os.path.relpath(archive, project)).as_posix()}"', - '"ordered"', - '"-Wl,--end-group"', - ) - positions = tuple(cmake_lists.index(item) for item in ordered_items) - assert positions == tuple(sorted(positions)) + contract_example = _import_extension("contract_example", build) + assert contract_example.square(np.float64(3.0)) == np.float64(9.0) + native.write_text(native.read_text(encoding="utf-8").replace("y = x * x", "y = x * x + 1.0"), encoding="utf-8") + native_output = rebuild("--verbose") + assert "contract_native.f90" in native_output + assert _regenerated(native_output) == set(), "a native-only edit reran contract generation" + contract_leaf.write_text(contract_leaf.read_text(encoding="utf-8") + "\n# contract changed\n", encoding="utf-8") + assert _regenerated(rebuild()) == {"contract_example"} -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_generate_cmake_links_prebuilt_object_and_archive_paths(tmp_path: Path): - prebuilt = tmp_path / "prebuilt native inputs" - prebuilt.mkdir() - (tmp_path / "scaled.f90").write_text( - """real(8) function scaled(x) result(y) - real(8), intent(in) :: x - y = x * 3.0d0 -end function scaled -""", - encoding="utf-8", - ) - (tmp_path / "shifted.f90").write_text( - """real(8) function shifted(x) result(y) - real(8), intent(in) :: x - y = x + 7.0d0 -end function shifted -""", - encoding="utf-8", - ) - archive_object = prebuilt / "scaled.o" - archive = prebuilt / "libscaled.a" - linked_object = prebuilt / "shifted.o" - _run(["gfortran", "-c", "-fPIC", "-o", str(archive_object), str(tmp_path / "scaled.f90")]) - _run(["ar", "rcs", str(archive), str(archive_object)]) - _run(["gfortran", "-c", "-fPIC", "-o", str(linked_object), str(tmp_path / "shifted.f90")]) - archive_object.unlink() - - project = tmp_path / "prebuilt inputs project" - project.mkdir() - (project / "interface.f90").write_text( - """real(8) function scaled(x) result(y) - real(8), intent(in) :: x - y = x -end function scaled - -real(8) function shifted(x) result(y) - real(8), intent(in) :: x - y = x -end function shifted -""", - encoding="utf-8", - ) - _run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--cmake", - str(project / "interface.f90"), - "--module-name", - "prebuilt_inputs", - "--no-compile-input-sources", - "--native-objects", - str(linked_object), - "--native-link-item", - f"archive:{archive}", - "--native-linker-language", - "fortran", - "--out-dir", - str(project), - ] - ) + (project / "include" / "inner.h").write_text("double c_square(double input);\n", encoding="utf-8") + assert _regenerated(rebuild()) == {"c_header_dependency"} - cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") - for prebuilt_path in (linked_object, archive): - relative = Path(os.path.relpath(prebuilt_path, project)).as_posix() - assert f'"${{CMAKE_CURRENT_LIST_DIR}}/{relative}"' in cmake_lists + (project / "declarations.inc").write_text("implicit none\n double precision, intent(in) :: x\n", encoding="utf-8") + assert _regenerated(rebuild()) == {"fortran_include_dependency"} - build = project / "build" - _configure_and_build(project, build, language="fortran") - module = _import_extension("prebuilt_inputs", build) - assert module.scaled(np.float64(4.0)) == np.float64(12.0) - assert module.shifted(np.float64(4.0)) == np.float64(11.0) + # A semantic edit changes the bridge file's contents, never the build graph. + # This module needs no bridge, but the declared source still exists so + # CMake's source list does not depend on semantic analysis. + bridge = build / "prik" / "routing" / "bind_c_routing_wrapper.f90" + assert "prik_unused_bridge_stub" in bridge.read_text(encoding="utf-8") + assert _call_extension("routing", build, "routing.standalone_direct(numpy.int32(4))") == "6" + # Adding a procedure that needs a bridge must not require a reconfigure. + (project / "routing.f90").write_text(ROUTING_MIXED, encoding="utf-8") + assert _regenerated(rebuild()) == {"routing"} + bridge_text = bridge.read_text(encoding="utf-8") + assert "prik_unused_bridge_stub" not in bridge_text, "the bridge stub was not replaced by real content" + assert "bind_c_routing_wrapper" in bridge_text + assert _call_extension("routing", build, "routing.standalone_adapted(numpy.int32(4))") == "7" + # ... and reverting is symmetric: real content becomes a stub again. + (project / "routing.f90").write_text(ROUTING_DIRECT, encoding="utf-8") + rebuild() + assert "prik_unused_bridge_stub" in bridge.read_text(encoding="utf-8") + assert _call_extension("routing", build, "routing.standalone_direct(numpy.int32(4))") == "6" @pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_generate_cmake_native_library_dir_reaches_the_runtime_search_path(tmp_path: Path): +@REQUIRES_CMAKE_FORTRAN +def test_generate_cmake_builds_a_standalone_project_from_every_native_input_kind(tmp_path: Path): + """A generated project links a native source, object, archive, and directory-found library. + + The semantic sources are not compiled, so every value below comes from the + separate native implementation, and the named library resolves at import + through the build's own runtime search path. + """ + sources = tmp_path / "sources" library_dir = tmp_path / "native runtime lib" library_dir.mkdir() - (tmp_path / "runtime_value.f90").write_text( - """real(8) function runtime_value(x) result(y) - real(8), intent(in) :: x - y = x * 5.0d0 -end function runtime_value -""", - encoding="utf-8", - ) + + def function(name: str, expression: str) -> str: + return f"real(8) function {name}(x) result(y)\n real(8), intent(in) :: x\n y = {expression}\nend function {name}\n" + + _write_files( + sources, + { + "scaled.f90": function("scaled", "x * 3.0d0"), + "shifted.f90": function("shifted", "x + 7.0d0"), + "runtime_value.f90": function("runtime_value", "x * 5.0d0"), + "implementation.f90": function("square", "x * x + 1.0d0"), + "interface.f90": "".join(function(name, "x") for name in ("scaled", "shifted", "runtime_value", "square")), + }, + ) + archive = _compile_archive(sources / "scaled.f90", sources / "libscaled.a") + linked_object = sources / "shifted.o" + _run(["gfortran", "-c", "-fPIC", "-o", str(linked_object), str(sources / "shifted.f90")]) native_library = library_dir / "libprikruntime.so" - _run(["gfortran", "-shared", "-fPIC", "-o", str(native_library), str(tmp_path / "runtime_value.f90")]) + _run(["gfortran", "-shared", "-fPIC", "-o", str(native_library), str(sources / "runtime_value.f90")]) - project = tmp_path / "runtime rpath project" - project.mkdir() - (project / "interface.f90").write_text( - """real(8) function runtime_value(x) result(y) - real(8), intent(in) :: x - y = x -end function runtime_value -""", - encoding="utf-8", - ) - _run( + project = tmp_path / "generated project with spaces" + generated = _run( [ sys.executable, "-m", "prik", "generate", "--cmake", - str(project / "interface.f90"), + str(sources / "interface.f90"), "--module-name", - "runtime_rpath", + "generated_inputs", "--no-compile-input-sources", + "--native-fortran-sources", + str(sources / "implementation.f90"), + "--native-objects", + str(linked_object), + "--native-link-item", + f"archive:{archive}", "--native-library", "prikruntime", "--native-library-dir", @@ -571,69 +645,47 @@ def test_generate_cmake_native_library_dir_reaches_the_runtime_search_path(tmp_p "fortran", "--out-dir", str(project), + "--json", ] ) + assert Path(json.loads(generated.stdout)["cmake_project"]) == project / "CMakeLists.txt" cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") + assert "include(UsePRIK)" in cmake_lists + assert "prik_add_module(\n generated_inputs" in cmake_lists + assert "NO_COMPILE_INPUT_SOURCES" in cmake_lists + for prebuilt_path in (linked_object, archive): + relative = Path(os.path.relpath(prebuilt_path, project)).as_posix() + assert f'"${{CMAKE_CURRENT_LIST_DIR}}/{relative}"' in cmake_lists relative_library_dir = Path(os.path.relpath(library_dir, project)).as_posix() assert f'LIBRARY_DIRS\n "{relative_library_dir}"' in cmake_lists # The directory carries CMake link and runtime meaning, so it is not also # repeated as a bare -L linker flag. assert "LINK_OPTIONS" not in cmake_lists - build = project / "build" - _configure_and_build(project, build, language="fortran") + build = project / "cmake-build" + _configure_and_build(project, build, language="fortran", use_ninja=False) called = _call_with_unassisted_loader( - "runtime_rpath", + "generated_inputs", build, - "runtime_rpath.runtime_value(numpy.float64(3.0))", + "[float(generated_inputs.scaled(numpy.float64(4.0))), float(generated_inputs.shifted(numpy.float64(4.0))), " + "float(generated_inputs.runtime_value(numpy.float64(3.0))), float(generated_inputs.square(numpy.float64(3.0)))]", native_library=native_library, ) - assert called == "15.0" + assert called == "[12.0, 11.0, 15.0, 10.0]" @pytest.mark.fortran_end_to_end -def test_generate_cmake_emits_contract_and_linker_languages_separately(tmp_path: Path): +def test_generate_cmake_emits_contract_source_and_linker_languages_separately(tmp_path: Path): contract = tmp_path / "api.pyi" contract.write_text( "from prik.contracts import Float64\ndef add(value: Float64) -> Float64: ...\n", encoding="utf-8" ) archive = tmp_path / "libimplementation.a" archive.touch() - project = tmp_path / "contract project" - _run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--cmake", - "--language", - "c", - str(contract), - "--native-objects", - str(archive), - "--native-linker-language", - "fortran", - "--out-dir", - str(project), - ] - ) - cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") - assert "CONTRACT" in cmake_lists - assert "NATIVE_LANGUAGE C" in cmake_lists - assert "LINKER_LANGUAGE Fortran" in cmake_lists - - -@pytest.mark.fortran_end_to_end -def test_generate_cmake_preserves_contract_language_with_different_source_language(tmp_path: Path): - contract = tmp_path / "api.pyi" - contract.write_text( - "from prik.contracts import Float64\ndef add(value: Float64) -> Float64: ...\n", encoding="utf-8" - ) implementation = tmp_path / "implementation.f90" implementation.write_text("subroutine implementation()\nend subroutine implementation\n", encoding="utf-8") - project = tmp_path / "mixed language contract project" + project = tmp_path / "contract project" _run( [ sys.executable, @@ -646,81 +698,25 @@ def test_generate_cmake_preserves_contract_language_with_different_source_langua str(contract), "--native-fortran-sources", str(implementation), - "--out-dir", - str(project), - ] - ) - cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") - assert "NATIVE_LANGUAGE C" in cmake_lists - assert "FORTRAN_SOURCES" in cmake_lists - - -@pytest.mark.fortran_end_to_end -def test_generate_cmake_keeps_compile_option_ownership_explicit(tmp_path: Path): - source = tmp_path / "interface.f90" - source.write_text("subroutine interface()\nend subroutine interface\n", encoding="utf-8") - archive = tmp_path / "libimplementation.a" - archive.touch() - project = tmp_path / "explicit options" - _run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--cmake", - str(source), - "--no-compile-input-sources", "--native-objects", str(archive), "--native-linker-language", "fortran", - "--native-compile-flags=-DPRIK_NATIVE", - "--wrapper-fortran-flags=-DPRIK_WRAPPER_FORTRAN", - "--wrapper-c-flags=-DPRIK_WRAPPER_C", - "--no-standard-logicals", - "--lto", "--out-dir", str(project), ] ) cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") - - assert 'FORTRAN_FLAGS\n "-DPRIK_NATIVE"' in cmake_lists - assert 'WRAPPER_FORTRAN_FLAGS\n "-DPRIK_WRAPPER_FORTRAN"' in cmake_lists - assert 'WRAPPER_C_FLAGS\n "-DPRIK_WRAPPER_C"' in cmake_lists + assert "CONTRACT" in cmake_lists + assert "NATIVE_LANGUAGE C" in cmake_lists + assert "FORTRAN_SOURCES" in cmake_lists assert "LINKER_LANGUAGE Fortran" in cmake_lists - assert "NO_STANDARD_LOGICALS" in cmake_lists - lto_initializer = "set(CMAKE_INTERPROCEDURAL_OPTIMIZATION TRUE)" - assert lto_initializer in cmake_lists - assert cmake_lists.index(lto_initializer) < cmake_lists.index("prik_add_module(") - assert "set_property(TARGET explicit_options PROPERTY INTERPROCEDURAL_OPTIMIZATION TRUE)" not in cmake_lists - assert "--compiler" not in cmake_lists - - -@pytest.mark.parametrize("option", ["--compiler=gfortran", "--wrapper-compiler-debug"]) -@pytest.mark.fortran_end_to_end -def test_generate_cmake_rejects_ambiguous_direct_compiler_options(tmp_path: Path, option: str): - source = tmp_path / "source.f90" - source.write_text("subroutine source()\nend subroutine source\n", encoding="utf-8") - - result = subprocess.run( - [sys.executable, "-m", "prik", "generate", "--cmake", str(source), option], - env=_environment(), - capture_output=True, - text=True, - ) - - assert result.returncode == 2 - assert "generate --cmake" in result.stderr @pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_generate_cmake_keeps_supplemental_c_sources_out_of_the_python_api(tmp_path: Path): +@REQUIRES_CMAKE_FORTRAN +def test_generate_cmake_lto_reaches_native_and_generated_compilation(tmp_path: Path): + """--lto must reach both target kinds, and a supplemental C source stays out of the Python API.""" source = tmp_path / "mixed.f90" source.write_text( """real(8) function add_one(value) result(result) @@ -740,7 +736,7 @@ def test_generate_cmake_keeps_supplemental_c_sources_out_of_the_python_api(tmp_p ) native_c = tmp_path / "native.c" native_c.write_text("double native_add_one(double value) { return value + 1.0; }\n", encoding="utf-8") - project = tmp_path / "generated mixed project" + project = tmp_path / "lto project" _run( [ sys.executable, @@ -749,163 +745,33 @@ def test_generate_cmake_keeps_supplemental_c_sources_out_of_the_python_api(tmp_p "generate", "--cmake", str(source), - "--module-name", - "mixed_extension", "--native-c-sources", str(native_c), + "--module-name", + "lto_module", + "--lto", "--out-dir", str(project), - ] - ) - cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") - assert "SOURCES" in cmake_lists - assert "C_SOURCES" in cmake_lists - build = project / "cmake-build" - _configure_and_build(project, build, language="fortran") - module = _import_extension("mixed_extension", build) - assert module.add_one(np.float64(4.0)) == np.float64(5.0) - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_contract_dependency_regenerates_wrapper(tmp_path: Path): - native = tmp_path / "contract native.f90" - native.write_text( - """real(8) function square(x) result(y) - real(8), intent(in) :: x - y = x * x -end function square -""", - encoding="utf-8", - ) - contracts = tmp_path / "contracts" - _run([sys.executable, "-m", "prik", "generate", "--pyi", str(native), "--out", str(contracts)]) - contract_dir = tmp_path / "contract_example" - contract_dir.mkdir() - contract = contract_dir / "__init__.pyi" - contract_leaf = contract_dir / "marker.pyi" - contract.write_text( - (contracts / "__init__.pyi").read_text(encoding="utf-8") + "\nfrom . import marker\n", - encoding="utf-8", - ) - contract_leaf.write_text("# included contract dependency\n", encoding="utf-8") - project = tmp_path / "contract project" - _write_project( - project, - f"""prik_add_module( - contract_example - CONTRACT "{contract.as_posix()}" - FORTRAN_SOURCES "{native.as_posix()}" -) -""", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - module = _import_extension("contract_example", build) - assert module.square(np.float64(3.0)) == np.float64(9.0) - native.write_text(native.read_text(encoding="utf-8").replace("y = x * x", "y = x * x + 1.0"), encoding="utf-8") - native_rebuild = _run(["cmake", "--build", str(build), "--verbose", "-j2"]) - native_output = native_rebuild.stdout + native_rebuild.stderr - assert "contract_native.f90" in native_output - assert "Generate PRIK wrapper sources for contract_example" not in native_output - contract_leaf.write_text(contract_leaf.read_text(encoding="utf-8") + "\n# contract changed\n", encoding="utf-8") - contract_rebuild = _run(["cmake", "--build", str(build), "-j2"]) - contract_output = contract_rebuild.stdout + contract_rebuild.stderr - assert "Generate PRIK wrapper sources for contract_example" in contract_output - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif(shutil.which("cmake") is None or shutil.which("gcc") is None, reason="CMake and gcc are required") -def test_use_prik_cmake_regenerates_after_nested_c_header_changes(tmp_path: Path): - project = tmp_path / "nested c dependency" - include_dir = project / "include" - include_dir.mkdir(parents=True) - inner_header = include_dir / "inner.h" - inner_header.write_text("double c_square(double value);\n", encoding="utf-8") - (include_dir / "api.h").write_text('#include "inner.h"\n', encoding="utf-8") - (project / "module.c").write_text( - '#include "api.h"\ndouble c_square(double value) { return value * value; }\n', - encoding="utf-8", - ) - _write_project( - project, - f"""prik_add_module( - c_header_dependency - C_SOURCES module.c - INCLUDE_DIRS "{include_dir.as_posix()}" -) -""", - languages="C", - ) - build = project / "build" - _configure_and_build(project, build, language="c") - - inner_header.write_text("double c_square(double input);\n", encoding="utf-8") - rebuilt = _run(["cmake", "--build", str(build), "-j2"]) - - assert "Generate PRIK wrapper sources for c_header_dependency" in rebuilt.stdout + rebuilt.stderr - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_regenerates_after_fortran_include_changes(tmp_path: Path): - project = tmp_path / "fortran include dependency" - project.mkdir() - include = project / "declarations.inc" - include.write_text("implicit none\n real(8), intent(in) :: x\n", encoding="utf-8") - (project / "included.f90").write_text( - "real(8) function included_square(x) result(y)\n" - " include 'declarations.inc'\n" - " y = x * x\n" - "end function included_square\n", - encoding="utf-8", - ) - _write_project( - project, - """prik_add_module( - fortran_include_dependency - FORTRAN_SOURCES included.f90 -) -""", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - - include.write_text("implicit none\n double precision, intent(in) :: x\n", encoding="utf-8") - rebuilt = _run(["cmake", "--build", str(build), "-j2"]) + ] + ) + assert "C_SOURCES" in (project / "CMakeLists.txt").read_text(encoding="utf-8") - assert "Generate PRIK wrapper sources for fortran_include_dependency" in rebuilt.stdout + rebuilt.stderr + build = project / "build" + _configure_and_build(project, build, language="fortran", defines=("CMAKE_EXPORT_COMPILE_COMMANDS=ON",)) + entries = json.loads((build / "compile_commands.json").read_text(encoding="utf-8")) + compiled = {Path(entry["file"]).name: entry["command"] for entry in entries} + # The caller's native sources and every generated unit must all carry it. + for name in ("mixed.f90", "native.c", "lto_module_wrapper.c", "bind_c_lto_module_wrapper.f90"): + assert name in compiled, f"{name} was not compiled" + assert "flto" in compiled[name], f"link-time optimization missing from {name}" -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_compiles_generated_fortran_bridge_and_binding(tmp_path: Path): - project = tmp_path / "bridge project" - _write_project( - project, - f"""prik_add_module( - free_external - CONTRACT "{BRIDGE_CONTRACT.as_posix()}" - FORTRAN_SOURCES "{BRIDGE_NATIVE.as_posix()}" -) -""", + called = _call_extension( + "lto_module", + build, + "float(lto_module.add_one(numpy.float64(4.0))), hasattr(lto_module, 'native_add_one')", ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - generated = build / "prik" / "free_external" - assert tuple(generated.glob("*.f90")), "the generated Fortran bridge is missing" - assert tuple(generated.glob("*.c")), "the generated C binding is missing" - module = _import_extension("free_external", build) - assert module.free_square(np.int32(6)) == np.int32(36) + assert called == "5.0 False" @pytest.mark.fortran_end_to_end @@ -913,108 +779,146 @@ def test_use_prik_cmake_compiles_generated_fortran_bridge_and_binding(tmp_path: shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, reason="CMake, gfortran, and gcc are required", ) -def test_use_prik_cmake_keeps_the_bridge_filename_fixed_across_semantic_edits(tmp_path: Path): - """A semantic edit changes the bridge file's contents, never the build graph.""" - project = tmp_path / "routing project" - project.mkdir() - source = project / "routing.f90" - source.write_text( - """integer(c_int) function standalone_direct(value) bind(C, name="standalone_direct_symbol") result(output) - use iso_c_binding - integer(c_int), value, intent(in) :: value - - output = value + 2_c_int -end function standalone_direct -""", - encoding="utf-8", +def test_use_prik_cmake_maps_required_logical_abi_flags(tmp_path: Path): + project = tmp_path / "logical abi flags" + toolchain = project / "toolchain" + toolchain.mkdir(parents=True) + for name, compiler in (("ifort", shutil.which("gfortran")), ("icx", shutil.which("gcc"))): + executable = toolchain / name + executable.write_text(f'#!/bin/sh\nexec "{compiler}" "$@"\n', encoding="utf-8") + executable.chmod(0o755) + source_text = ( + "logical function logical_identity(value) result(output)\n" + " logical, intent(in) :: value\n" + " output = value\n" + "end function logical_identity\n" ) + (project / "logical_default.f90").write_text(source_text, encoding="utf-8") + (project / "logical_disabled.f90").write_text(source_text, encoding="utf-8") _write_project( project, - """prik_add_module( - routing - SOURCES routing.f90 -) + """set(CMAKE_EXPORT_COMPILE_COMMANDS ON) +prik_add_module(abi_default SOURCES logical_default.f90) +prik_add_module(abi_disabled SOURCES logical_disabled.f90 NO_STANDARD_LOGICALS) """, ) build = project / "build" - _configure_and_build(project, build, language="fortran") - bridge = build / "prik" / "routing" / "bind_c_routing_wrapper.f90" - - # This module needs no bridge, but the declared source still exists so - # CMake's source list does not depend on semantic analysis. - assert bridge.is_file(), "the deterministic bridge source is missing" - assert "prik_unused_bridge_stub" in bridge.read_text(encoding="utf-8") - module = _import_extension("routing", build) - assert module.standalone_direct(np.int32(4)) == np.int32(6) - - # Adding a procedure that needs a bridge must not require a reconfigure. - source.write_text( - """integer(c_int) function standalone_direct(value) bind(C, name="standalone_mixed_direct") result(output) - use iso_c_binding - integer(c_int), value, intent(in) :: value + _run( + [ + "cmake", + "-S", + str(project), + "-B", + str(build), + "-G", + "Ninja" if shutil.which("ninja") else "Unix Makefiles", + f"-DCMAKE_C_COMPILER={shutil.which('gcc')}", + f"-DCMAKE_Fortran_COMPILER={toolchain / 'ifort'}", + ] + ) + commands = json.loads((build / "compile_commands.json").read_text(encoding="utf-8")) + default_commands = [record["command"] for record in commands if "abi_default" in record["command"]] + disabled_commands = [record["command"] for record in commands if "abi_disabled" in record["command"]] - output = value + 2_c_int -end function standalone_direct + assert default_commands + assert any("-standard-semantics" in command for command in default_commands) + assert disabled_commands + assert all("-standard-semantics" not in command for command in disabled_commands) -integer(c_int) function standalone_adapted(value) result(output) - use iso_c_binding - integer(c_int), intent(in) :: value - output = value + 3_c_int -end function standalone_adapted -""", - encoding="utf-8", +@pytest.mark.fortran_end_to_end +def test_generate_cmake_preserves_ordered_native_link_items(tmp_path: Path): + source = tmp_path / "ordered.f90" + source.write_text("subroutine ordered()\nend subroutine ordered\n", encoding="utf-8") + archive = tmp_path / "libordered.a" + archive.touch() + project = tmp_path / "ordered project" + _run( + [ + sys.executable, + "-m", + "prik", + "generate", + "--cmake", + str(source), + "--native-link-item", + "arg:-Wl,--start-group", + f"archive:{archive}", + "library:ordered", + "arg:-Wl,--end-group", + "--out-dir", + str(project), + ] ) - _run(["cmake", "--build", str(build), "-j2"]) - - bridge_text = bridge.read_text(encoding="utf-8") - assert "prik_unused_bridge_stub" not in bridge_text, "the bridge stub was not replaced by real content" - assert "bind_c_routing_wrapper" in bridge_text - assert _call_extension("routing", build, "routing.standalone_adapted(numpy.int32(4))") == "7" + cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") + ordered_items = ( + '"-Wl,--start-group"', + f'"${{CMAKE_CURRENT_LIST_DIR}}/{Path(os.path.relpath(archive, project)).as_posix()}"', + '"ordered"', + '"-Wl,--end-group"', + ) + positions = tuple(cmake_lists.index(item) for item in ordered_items) + assert positions == tuple(sorted(positions)) - # ... and reverting is symmetric: real content becomes a stub again. - source.write_text( - """integer(c_int) function standalone_direct(value) bind(C, name="standalone_direct_symbol") result(output) - use iso_c_binding - integer(c_int), value, intent(in) :: value - output = value + 2_c_int -end function standalone_direct -""", - encoding="utf-8", +@pytest.mark.fortran_end_to_end +def test_generate_cmake_keeps_compile_option_ownership_explicit(tmp_path: Path): + source = tmp_path / "interface.f90" + source.write_text("subroutine interface()\nend subroutine interface\n", encoding="utf-8") + archive = tmp_path / "libimplementation.a" + archive.touch() + project = tmp_path / "explicit options" + _run( + [ + sys.executable, + "-m", + "prik", + "generate", + "--cmake", + str(source), + "--no-compile-input-sources", + "--native-objects", + str(archive), + "--native-linker-language", + "fortran", + "--native-compile-flags=-DPRIK_NATIVE", + "--wrapper-fortran-flags=-DPRIK_WRAPPER_FORTRAN", + "--wrapper-c-flags=-DPRIK_WRAPPER_C", + "--no-standard-logicals", + "--lto", + "--out-dir", + str(project), + ] ) - _run(["cmake", "--build", str(build), "-j2"]) - assert "prik_unused_bridge_stub" in bridge.read_text(encoding="utf-8") - assert _call_extension("routing", build, "routing.standalone_direct(numpy.int32(4))") == "6" + cmake_lists = (project / "CMakeLists.txt").read_text(encoding="utf-8") + + assert 'FORTRAN_FLAGS\n "-DPRIK_NATIVE"' in cmake_lists + assert 'WRAPPER_FORTRAN_FLAGS\n "-DPRIK_WRAPPER_FORTRAN"' in cmake_lists + assert 'WRAPPER_C_FLAGS\n "-DPRIK_WRAPPER_C"' in cmake_lists + assert "LINKER_LANGUAGE Fortran" in cmake_lists + assert "NO_STANDARD_LOGICALS" in cmake_lists + lto_initializer = "set(CMAKE_INTERPROCEDURAL_OPTIMIZATION TRUE)" + assert lto_initializer in cmake_lists + assert cmake_lists.index(lto_initializer) < cmake_lists.index("prik_add_module(") + assert "set_property(TARGET explicit_options PROPERTY INTERPROCEDURAL_OPTIMIZATION TRUE)" not in cmake_lists + assert "--compiler" not in cmake_lists +@pytest.mark.parametrize("option", ["--compiler=gfortran", "--wrapper-compiler-debug"]) @pytest.mark.fortran_end_to_end -@pytest.mark.skipif(shutil.which("cmake") is None or shutil.which("gcc") is None, reason="CMake and gcc are required") -def test_use_prik_cmake_builds_c_source_with_include_directory_and_flag(tmp_path: Path): - project = tmp_path / "c project with spaces" - include_dir = project / "include files" - include_dir.mkdir(parents=True) - (include_dir / "cmath_api.h").write_text("double c_add(double, double);\n", encoding="utf-8") - (project / "cexample.c").write_text( - '#include "cmath_api.h"\n#ifdef PRIK_CMAKE_TEST_FLAG\ndouble c_add(double a, double b) { return a + b + 1.0; }\n#else\ndouble c_add(double a, double b) { return a + b; }\n#endif\n', - encoding="utf-8", - ) - _write_project( - project, - f"""prik_add_module( - cexample - C_SOURCES cexample.c - INCLUDE_DIRS "{include_dir.as_posix()}" - C_FLAGS -DPRIK_CMAKE_TEST_FLAG - PRIK_ARGS --define PRIK_CMAKE_TEST_FLAG -) -""", - languages="C", +def test_generate_cmake_rejects_ambiguous_direct_compiler_options(tmp_path: Path, option: str): + source = tmp_path / "source.f90" + source.write_text("subroutine source()\nend subroutine source\n", encoding="utf-8") + + result = subprocess.run( + [sys.executable, "-m", "prik", "generate", "--cmake", str(source), option], + env=_environment(), + capture_output=True, + text=True, ) - build = project / "build" - _configure_and_build(project, build, language="c") - module = _import_extension("cexample", build) - assert module.c_add(np.float64(2.0), np.float64(3.0)) == np.float64(6.0) + + assert result.returncode == 2 + assert "generate --cmake" in result.stderr @pytest.mark.fortran_end_to_end @@ -1198,81 +1102,23 @@ def test_use_prik_cmake_configure_does_not_run_the_semantic_pipeline(tmp_path: P build = project / "build" configure = subprocess.run( [ - "cmake", - "-S", - str(project), - "-B", - str(build), - f"-DCMAKE_C_COMPILER={shutil.which('gcc')}", - f"-DCMAKE_Fortran_COMPILER={shutil.which('gfortran')}", - ], - env=_environment(), - capture_output=True, - text=True, - ) - assert configure.returncode == 0, f"configure ran semantic analysis:\n{configure.stderr}" - assert not tuple((build / "prik" / "broken").glob("*.c")), "configure generated wrapper sources" - - built = subprocess.run(["cmake", "--build", str(build), "-j2"], env=_environment(), capture_output=True, text=True) - assert built.returncode != 0, "the unparsable source should fail during build-time generation" - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif(shutil.which("cmake") is None or shutil.which("gcc") is None, reason="CMake and gcc are required") -def test_use_prik_cmake_keeps_the_adapter_filename_fixed(tmp_path: Path): - """The adapter unit is declared either way: a stub without adapters, real content with them.""" - results = {} - for label, prik_args in (("plain", ""), ("adapted", " PRIK_ARGS --collision-adapter-all\n")): - project = tmp_path / f"adapter {label}" - project.mkdir() - (project / "capi.c").write_text("double capi_add(double value) { return value + 1.0; }\n", encoding="utf-8") - _write_project( - project, - f"""prik_add_module( - adapter_{label} - C_SOURCES capi.c -{prik_args}) -""", - languages="C", - ) - build = project / "build" - _configure_and_build(project, build, language="c") - adapters = build / "prik" / f"adapter_{label}" / f"adapter_{label}_adapters.c" - assert adapters.is_file(), f"the deterministic adapter source is missing for {label}" - results[label] = adapters.read_text(encoding="utf-8") - assert _call_extension(f"adapter_{label}", build, f"adapter_{label}.capi_add(numpy.float64(2.0))") == "3.0" - - assert "prik_unused_adapter_stub" in results["plain"] - assert "prik_unused_adapter_stub" not in results["adapted"] - assert "capi_add" in results["adapted"] - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_does_not_regenerate_an_unchanged_build(tmp_path: Path): - project = tmp_path / "idempotent project" - project.mkdir() - (project / "steady.f90").write_text( - "real(8) function steady(x) result(y)\n real(8), intent(in) :: x\n y = x\nend function steady\n", - encoding="utf-8", - ) - _write_project( - project, - """prik_add_module( - steady - SOURCES steady.f90 -) -""", + "cmake", + "-S", + str(project), + "-B", + str(build), + f"-DCMAKE_C_COMPILER={shutil.which('gcc')}", + f"-DCMAKE_Fortran_COMPILER={shutil.which('gfortran')}", + ], + env=_environment(), + capture_output=True, + text=True, ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - generation_comment = "Generate PRIK wrapper sources for steady" + assert configure.returncode == 0, f"configure ran semantic analysis:\n{configure.stderr}" + assert not tuple((build / "prik" / "broken").glob("*.c")), "configure generated wrapper sources" - rebuilt = _run(["cmake", "--build", str(build), "-j2"]) - assert generation_comment not in rebuilt.stdout + rebuilt.stderr, "an unchanged build reran PRIK generation" + built = subprocess.run(["cmake", "--build", str(build), "-j2"], env=_environment(), capture_output=True, text=True) + assert built.returncode != 0, "the unparsable source should fail during build-time generation" @pytest.mark.fortran_end_to_end @@ -1344,89 +1190,6 @@ def test_generated_placeholder_units_compile_under_strict_flags(tmp_path: Path): ) -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_generate_cmake_lto_reaches_native_and_generated_compilation(tmp_path: Path): - """--lto must reach both target kinds, not only the CMakeLists initializer.""" - (tmp_path / "solver.f90").write_text( - "real(8) function solve(x) result(y)\n real(8), intent(in) :: x\n y = x * 2.0d0\nend function solve\n", - encoding="utf-8", - ) - (tmp_path / "helper.c").write_text("double helper_value(void) { return 1.0; }\n", encoding="utf-8") - project = tmp_path / "lto project" - _run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--cmake", - str(tmp_path / "solver.f90"), - "--native-c-sources", - str(tmp_path / "helper.c"), - "--module-name", - "lto_module", - "--lto", - "--out-dir", - str(project), - ] - ) - - build = project / "build" - command = ["cmake", "-S", str(project), "-B", str(build), "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"] - if shutil.which("ninja"): - command.extend(("-G", "Ninja")) - command.append(f"-DCMAKE_C_COMPILER={shutil.which('gcc')}") - command.append(f"-DCMAKE_Fortran_COMPILER={shutil.which('gfortran')}") - _run(command) - _run(["cmake", "--build", str(build), "-j2"]) - - entries = json.loads((build / "compile_commands.json").read_text(encoding="utf-8")) - compiled = {Path(entry["file"]).name: entry["command"] for entry in entries} - # The caller's native sources and every generated unit must all carry it. - for source in ("solver.f90", "helper.c", "lto_module_wrapper.c", "bind_c_lto_module_wrapper.f90"): - assert source in compiled, f"{source} was not compiled" - assert "flto" in compiled[source], f"link-time optimization missing from {source}" - - assert _call_extension("lto_module", build, "lto_module.solve(numpy.float64(4.0))") == "8.0" - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif(shutil.which("cmake") is None or shutil.which("gcc") is None, reason="CMake and gcc are required") -def test_use_prik_cmake_propagates_dependency_usage_to_native_objects(tmp_path: Path): - project = tmp_path / "native dependency usage" - project.mkdir() - (project / "interface.c").write_text("double dependency_add(double value);\n", encoding="utf-8") - (project / "implementation.c").write_text( - "#ifndef PRIK_REQUIRED_DEFINE\n" - "#error missing dependency compile definition\n" - "#endif\n" - "double dependency_add(double value) { return value + 1.0; }\n", - encoding="utf-8", - ) - _write_project( - project, - """add_library(native_dependency INTERFACE) -target_compile_definitions(native_dependency INTERFACE PRIK_REQUIRED_DEFINE) -prik_add_module( - dependency_usage - SOURCES interface.c - C_SOURCES implementation.c - LINK_LIBRARIES native_dependency -) -""", - languages="C", - ) - build = project / "build" - _configure_and_build(project, build, language="c") - - module = _import_extension("dependency_usage", build) - assert module.dependency_add(np.float64(2.0)) == np.float64(3.0) - - @pytest.mark.fortran_end_to_end @pytest.mark.skipif( shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, @@ -1513,262 +1276,6 @@ def test_use_prik_cmake_requires_the_c_language(tmp_path: Path): assert "PRIK Python extensions require CMake's C language to be enabled" in result.stdout + result.stderr -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_links_a_normal_fortran_library_target(tmp_path: Path): - project = tmp_path / "external target" - project.mkdir() - (project / "native_math.f90").write_text( - "real(8) function native_add(x, y) result(z)\n" - " real(8), intent(in) :: x, y\n" - " z = x + y\n" - "end function native_add\n", - encoding="utf-8", - ) - (project / "wrapper.f90").write_text( - "real(8) function call_native(x, y) result(z)\n" - " real(8), intent(in) :: x, y\n" - " interface\n" - " function native_add(a, b) result(c)\n" - " real(8), intent(in) :: a, b\n" - " real(8) :: c\n" - " end function native_add\n" - " end interface\n" - " z = native_add(x, y)\n" - "end function call_native\n", - encoding="utf-8", - ) - _write_project( - project, - """add_library(native_math STATIC native_math.f90) -prik_add_module( - external_target - SOURCES wrapper.f90 - LINK_LIBRARIES native_math -) -""", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - module = _import_extension("external_target", build) - assert module.call_native(np.float64(2.0), np.float64(3.0)) == np.float64(5.0) - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_uses_target_as_the_only_native_implementation(tmp_path: Path): - project = tmp_path / "target only implementation" - project.mkdir() - interface = ( - "real(8) function target_square(value) result(result)\n" - " real(8), intent(in) :: value\n" - " result = value * value\n" - "end function target_square\n" - ) - (project / "interface.f90").write_text(interface, encoding="utf-8") - (project / "implementation.f90").write_text(interface, encoding="utf-8") - _write_project( - project, - """add_library(native_math STATIC implementation.f90) -prik_add_module( - target_only - SOURCES interface.f90 - NO_COMPILE_INPUT_SOURCES - LINK_LIBRARIES native_math -) -""", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - - module = _import_extension("target_only", build) - assert module.target_square(np.float64(4.0)) == np.float64(16.0) - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None - or shutil.which("gfortran") is None - or shutil.which("gcc") is None - or shutil.which("ar") is None, - reason="CMake, gfortran, gcc, and ar are required", -) -def test_use_prik_cmake_selects_fortran_linker_for_raw_archive(tmp_path: Path): - project = tmp_path / "raw fortran archive" - project.mkdir() - source_text = ( - "integer(c_int) function raw_add_two(value) bind(C, name='raw_add_two_symbol') result(output)\n" - " use iso_c_binding, only: c_int\n" - " integer(c_int), value, intent(in) :: value\n" - " character(len=16) :: buffer\n" - " write(buffer, '(I0)') value\n" - " read(buffer, *) output\n" - " output = output + 2_c_int\n" - "end function raw_add_two\n" - ) - interface = project / "interface.f90" - implementation = project / "implementation.f90" - interface.write_text(source_text, encoding="utf-8") - implementation.write_text(source_text, encoding="utf-8") - native_object = project / "implementation.o" - archive = project / "libraw_math.a" - _run([shutil.which("gfortran"), "-fPIC", "-c", str(implementation), "-o", str(native_object)]) - _run([shutil.which("ar"), "rcs", str(archive), str(native_object)]) - _write_project( - project, - f"""prik_add_module( - raw_archive - SOURCES interface.f90 - NO_COMPILE_INPUT_SOURCES - LINKER_LANGUAGE Fortran - LINK_LIBRARIES "{archive.as_posix()}" -) -""", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - - module = _import_extension("raw_archive", build) - assert module.raw_add_two(np.int32(5)) == np.int32(7) - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None - or shutil.which("gfortran") is None - or shutil.which("gcc") is None - or shutil.which("ar") is None, - reason="CMake, gfortran, gcc, and ar are required", -) -def test_use_prik_cmake_separates_c_contract_and_fortran_linker_languages(tmp_path: Path): - project = tmp_path / "c contract with fortran implementation" - project.mkdir() - (project / "api.pyi").write_text( - "from prik.contracts import Float64\n\ndef add_one(value: Float64) -> Float64: ...\n", - encoding="utf-8", - ) - implementation = project / "implementation.f90" - implementation.write_text( - "real(c_double) function add_one(value) bind(C, name='add_one') result(result)\n" - " use iso_c_binding, only: c_double\n" - " real(c_double), value, intent(in) :: value\n" - " result = value + 1.0_c_double\n" - "end function add_one\n", - encoding="utf-8", - ) - native_object = project / "implementation.o" - archive = project / "libimplementation.a" - _run([shutil.which("gfortran"), "-fPIC", "-c", str(implementation), "-o", str(native_object)]) - _run([shutil.which("ar"), "rcs", str(archive), str(native_object)]) - _write_project( - project, - """prik_add_module( - c_contract - CONTRACT api.pyi - NATIVE_LANGUAGE C - LINKER_LANGUAGE Fortran - LINK_LIBRARIES "${CMAKE_CURRENT_SOURCE_DIR}/libimplementation.a" -) -""", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - - module = _import_extension("c_contract", build) - assert module.add_one(np.float64(5.0)) == np.float64(6.0) - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_use_prik_cmake_allows_contract_language_to_differ_from_source_language(tmp_path: Path): - project = tmp_path / "c contract with fortran source" - project.mkdir() - (project / "api.pyi").write_text( - "from prik.contracts import Float64\n\ndef add_two(value: Float64) -> Float64: ...\n", - encoding="utf-8", - ) - (project / "implementation.f90").write_text( - "real(c_double) function add_two(value) bind(C, name='add_two') result(result)\n" - " use iso_c_binding, only: c_double\n" - " real(c_double), value, intent(in) :: value\n" - " result = value + 2.0_c_double\n" - "end function add_two\n", - encoding="utf-8", - ) - _write_project( - project, - """prik_add_module( - c_contract_source - CONTRACT api.pyi - NATIVE_LANGUAGE C - FORTRAN_SOURCES implementation.f90 -) -""", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran") - - module = _import_extension("c_contract_source", build) - assert module.add_two(np.float64(5.0)) == np.float64(7.0) - - -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_generate_cmake_can_keep_semantic_sources_out_of_native_compilation(tmp_path: Path): - project = tmp_path / "separate implementation" - project.mkdir() - (project / "interface.f90").write_text( - """real(8) function square(value) result(result) - real(8), intent(in) :: value - result = value * value -end function square -""", - encoding="utf-8", - ) - (project / "implementation.f90").write_text( - """real(8) function square(value) result(result) - real(8), intent(in) :: value - result = value * value + 1.0 -end function square -""", - encoding="utf-8", - ) - _run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--cmake", - "--module-name", - "separate_implementation", - "--no-compile-input-sources", - str(project / "interface.f90"), - "--native-fortran-sources", - str(project / "implementation.f90"), - "--out-dir", - str(project), - ] - ) - assert "NO_COMPILE_INPUT_SOURCES" in (project / "CMakeLists.txt").read_text(encoding="utf-8") - build = project / "build" - _configure_and_build(project, build, language="fortran") - module = _import_extension("separate_implementation", build) - assert module.square(np.float64(3.0)) == np.float64(10.0) - - @pytest.mark.fortran_end_to_end @pytest.mark.skipif(not _cmake_finds_blas(), reason="CMake cannot discover a Fortran BLAS implementation") def test_use_prik_cmake_links_a_cmake_discovered_blas_target(tmp_path: Path): @@ -1804,40 +1311,6 @@ def test_use_prik_cmake_links_a_cmake_discovered_blas_target(tmp_path: Path): assert module.blas_dot(np.array([1.0, 2.0]), np.array([3.0, 4.0])) == np.float64(11.0) -@pytest.mark.fortran_end_to_end -@pytest.mark.skipif( - shutil.which("cmake") is None or shutil.which("gfortran") is None or shutil.which("gcc") is None, - reason="CMake, gfortran, and gcc are required", -) -def test_find_package_prik_config_provides_the_module_helper(tmp_path: Path): - project = tmp_path / "found package" - project.mkdir() - (project / "square.f90").write_text( - """real(8) function config_square(x) result(y) - real(8), intent(in) :: x - y = x * x -end function config_square -""", - encoding="utf-8", - ) - (project / "CMakeLists.txt").write_text( - """cmake_minimum_required(VERSION 3.21) -project(cmake_test LANGUAGES C Fortran) -find_package(Python COMPONENTS Interpreter Development.Module REQUIRED) -find_package(PRIK CONFIG REQUIRED) -prik_add_module( - config_square - SOURCES square.f90 -) -""", - encoding="utf-8", - ) - build = project / "build" - _configure_and_build(project, build, language="fortran", defines=(f"PRIK_DIR={USE_PRIK_DIR.as_posix()}",)) - module = _import_extension("config_square", build) - assert module.config_square(np.float64(3.0)) == np.float64(9.0) - - @pytest.mark.fortran_end_to_end @pytest.mark.slow @pytest.mark.skipif( diff --git a/tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py b/tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py index 3659eea92..12d13ed85 100644 --- a/tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py +++ b/tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py @@ -6,6 +6,7 @@ import subprocess import sys from pathlib import Path +from types import SimpleNamespace import numpy as np import pytest @@ -154,6 +155,8 @@ def _assert_combined_runtime(module) -> None: def test_multi_file_modules_build_one_merged_extension(tmp_path: Path): + for name in ("first_api.first_api", "first_api.second_api"): + sys.modules.pop(name, None) module, payload = _build_sources_and_import( [ ("first_api.f90", _source_text(FIRST_API_SOURCE)), @@ -174,32 +177,19 @@ def test_multi_file_modules_build_one_merged_extension(tmp_path: Path): assert "use first_api" in bridge assert "use second_api" in bridge - -def test_generated_child_modules_are_importable_submodules(tmp_path: Path): - _module, payload = _build_sources_and_import( - [ - ("first_api.f90", _source_text(FIRST_API_SOURCE)), - ("second_api.f90", _source_text(SECOND_API_SOURCE)), - ], - tmp_path, - ) - - module_name = str(payload["module_name"]) - for name in (module_name, f"{module_name}.first_api", f"{module_name}.second_api"): - sys.modules.pop(name, None) + # Each generated child module is also an importable submodule of the extension. + assert sys.modules["first_api.first_api"] is module.first_api + assert sys.modules["first_api.second_api"] is module.second_api sys.path.insert(0, str(tmp_path)) try: from first_api.first_api import add_one from first_api.second_api import double_value - root = importlib.import_module("first_api") - assert sys.modules["first_api.first_api"] is root.first_api - assert sys.modules["first_api.second_api"] is root.second_api - assert add_one(np.int32(4)) == 5 - assert double_value(np.int32(4)) == 10 + assert add_one is module.first_api.add_one + assert double_value is module.second_api.double_value finally: sys.path.remove(str(tmp_path)) - for name in (f"{module_name}.second_api", f"{module_name}.first_api", module_name): + for name in ("first_api.second_api", "first_api.first_api", "first_api"): sys.modules.pop(name, None) @@ -247,35 +237,31 @@ def test_multi_file_build_resolves_reexported_intrinsic_kind_alias(tmp_path: Pat assert module.twice(np.float64(1.25)) == np.float64(2.5) -def test_multi_source_pyi_out_writes_one_flat_combined_package(tmp_path: Path): - sources = _write_combined_sources(tmp_path) - package = tmp_path / "contracts" - entry = _generate_combined_contract(sources, package) - - assert entry == package / "__init__.pyi" - assert sorted(path.relative_to(package).as_posix() for path in package.rglob("*.pyi")) == [ - "__init__.pyi", - "box_ops.pyi", - "first_math.pyi", - "second_math.pyi", - "shared_types.pyi", - ] - assert not (package / "first_api").exists() - assert not (package / "second_api").exists() - assert not (package / "combined_extensions").exists() - assert entry.read_text(encoding="utf-8") == ( - "from . import first_math\nfrom . import shared_types\nfrom . import second_math\nfrom . import box_ops\n\n" - '__all__ = ["first_math", "shared_types", "second_math", "box_ops"]\n' +@pytest.fixture(scope="module") +def combined(tmp_path_factory: pytest.TempPathFactory) -> SimpleNamespace: + """Build the combined sources once, and generate their contract and native objects once. + + Generating the contract compares the whole package, file set and text, with + its checked fixture, so every test using it also proves the flat layout. + """ + root = tmp_path_factory.mktemp("combined") + sources = _write_combined_sources(root) + source_module, source_payload = _build_sources(sources, root / "source_build") + return SimpleNamespace( + root=root, + sources=sources, + source_module=source_module, + source_payload=source_payload, + entry=_generate_combined_contract(sources, root / "contracts"), + native_objects=_compile_native_objects(sources, root / "native"), ) - assert "from .shared_types import Box\n" in (package / "box_ops.pyi").read_text(encoding="utf-8") - assert "from .first_math import add_one" in (package / "second_math.pyi").read_text(encoding="utf-8") -def test_multi_source_generated_contract_build_matches_source_runtime_and_link_order(tmp_path: Path): - sources = _write_combined_sources(tmp_path) - source_module, source_payload = _build_sources(sources, tmp_path / "source_build") - entry = _generate_combined_contract(sources, tmp_path / "contracts") - native_objects = _compile_native_objects(sources, tmp_path / "native") +def test_multi_source_generated_contract_build_matches_source_runtime_and_link_order( + combined: SimpleNamespace, tmp_path: Path +): + entry, native_objects = combined.entry, combined.native_objects + source_module, source_payload = combined.source_module, combined.source_payload generated_module, generated_payload = _build_contract( entry, native_objects, @@ -293,7 +279,7 @@ def test_multi_source_generated_contract_build_matches_source_runtime_and_link_o str(entry.parent / "shared_types.pyi"), ] assert [item["path"] for item in source_payload["native_build_plan"]["link_items"]] == [ - str(Path(source_payload["output_dir"]) / f"{source.stem}.o") for source in sources + str(Path(source_payload["output_dir"]) / f"{source.stem}.o") for source in combined.sources ] assert generated_payload["native_build_plan"]["link_items"] == [ {"kind": "object", "path": str(native_objects[0])}, @@ -309,14 +295,11 @@ def test_multi_source_generated_contract_build_matches_source_runtime_and_link_o assert generated_module.shared_types.Box is not None -def test_generated_module_leaf_loads_sibling_type_contract(tmp_path: Path): - sources = _write_combined_sources(tmp_path) - entry = _generate_combined_contract(sources, tmp_path / "contracts") - native_objects = _compile_native_objects(sources, tmp_path / "native") - +def test_generated_module_leaf_loads_sibling_type_contract(combined: SimpleNamespace, tmp_path: Path): + entry = combined.entry module, payload = _build_contract( entry.parent / "box_ops.pyi", - native_objects, + combined.native_objects, tmp_path / "leaf_build", output_name="box_leaf", ) @@ -335,13 +318,11 @@ def test_generated_module_leaf_loads_sibling_type_contract(tmp_path: Path): assert module.box_value(box) == np.int32(7) -def test_multi_source_modified_entry_preserves_modules_and_adds_documented_alias(tmp_path: Path): - sources = _write_combined_sources(tmp_path) - source_module, source_payload = _build_sources(sources, tmp_path / "source_build") - generated_entry = _generate_combined_contract(sources, tmp_path / "contracts") - native_objects = _compile_native_objects(sources, tmp_path / "native") +def test_multi_source_modified_entry_preserves_modules_and_adds_documented_alias( + combined: SimpleNamespace, tmp_path: Path +): modified_package = tmp_path / "modified_contracts" - shutil.copytree(generated_entry.parent, modified_package) + shutil.copytree(combined.entry.parent, modified_package) modified_entry = modified_package / "__init__.pyi" modified_entry.write_text( "# Intentional difference: preserve native module children and add a root alias.\n" @@ -356,17 +337,47 @@ def test_multi_source_modified_entry_preserves_modules_and_adds_documented_alias modified_module, modified_payload = _build_contract( modified_entry, - native_objects, + combined.native_objects, tmp_path / "modified_build", - output_name=str(source_payload["module_name"]), + output_name=str(combined.source_payload["module_name"]), ) - assert modified_payload["module_name"] == source_payload["module_name"] - assert not hasattr(source_module, "fused_value") + assert modified_payload["module_name"] == combined.source_payload["module_name"] + assert not hasattr(combined.source_module, "fused_value") assert modified_module.fused_value(np.int32(4)) == np.int32(10) _assert_combined_runtime(modified_module) +@pytest.mark.skipif(sys.platform == "win32", reason="shared-library loader behavior differs on Windows") +@pytest.mark.parametrize("artifact_kind", ["archive", "shared_library"]) +def test_imported_contracts_resolve_from_one_archive_or_shared_library( + combined: SimpleNamespace, + tmp_path: Path, + artifact_kind: str, +): + native_objects = combined.native_objects + if artifact_kind == "archive": + artifact = tmp_path / "libcombined.a" + subprocess.run(["ar", "rcs", str(artifact), *(str(obj) for obj in native_objects)], check=True) + else: + artifact = tmp_path / "libcombined.so" + subprocess.run([_compiler(), "-shared", "-o", str(artifact), *(str(obj) for obj in native_objects)], check=True) + + result = build_pyi_extension( + combined.entry, + native_objects=[artifact], + native_include_dirs=[native_objects[0].parent], + output_name="combined_from_single_artifact", + output_dir=tmp_path / "build", + ) + module = _import_extension(result.module_name, result.output_dir) + native_plan = result.native_build_plan.to_dict() + + assert native_plan["prebuilt_artifacts"] == [{"kind": artifact_kind, "path": str(artifact)}] + assert native_plan["link_items"] == [{"kind": artifact_kind, "path": str(artifact)}] + _assert_combined_runtime(module) + + @pytest.mark.skipif( sys.platform == "win32" or shutil.which("make") is None, reason="generated Makefile requires GNU Make and a POSIX shell", @@ -414,6 +425,7 @@ def test_makefile_mode_reproduces_multi_source_build(tmp_path: Path): "all", "PRIK_FFLAGS=-O3", "PRIK_CFLAGS=-O3", + "PRIK_LDFLAGS=-O3", ], capture_output=True, text=True, diff --git a/tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py b/tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py index bbac1a43c..be5c3ac4c 100644 --- a/tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py +++ b/tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py @@ -12,13 +12,6 @@ import pytest from prik import build_pyi_extension -from tests.fortran.infrastructure.building.end_to_end.test_multi_source_builds import ( - _assert_combined_runtime, - _compile_native_objects, - _generate_combined_contract, - _import_extension, - _write_combined_sources, -) CONTRACT_IMPORT = "from prik.contracts import Addr, Arg, Int32, native_call, standalone\n\n" @@ -136,55 +129,15 @@ def _direct_external_source(name: str, expression: str) -> str: """ -@pytest.mark.skipif(sys.platform == "win32", reason="shared-library loader behavior differs on Windows") -@pytest.mark.parametrize("artifact_kind", ["archive", "shared_library"]) -def test_imported_contracts_resolve_from_one_archive_or_shared_library( - tmp_path: Path, - artifact_kind: str, -): - source_dir = tmp_path / "sources" - source_dir.mkdir() - sources = _write_combined_sources(source_dir) - entry = _generate_combined_contract(sources, tmp_path / "contracts") - native_objects = _compile_native_objects(sources, tmp_path / "native") - artifact = ( - _archive(tmp_path / "native" / "libcombined.a", native_objects) - if artifact_kind == "archive" - else _shared_library(tmp_path / "native" / "libcombined.so", native_objects) - ) - - result = build_pyi_extension( - entry, - native_objects=[artifact], - native_include_dirs=[native_objects[0].parent], - output_name="combined_from_single_artifact", - output_dir=tmp_path / "build", - ) - module = _import_extension(result.module_name, result.output_dir) - native_plan = result.native_build_plan.to_dict() - - assert native_plan["prebuilt_artifacts"] == [{"kind": artifact_kind, "path": str(artifact)}] - assert native_plan["link_items"] == [{"kind": artifact_kind, "path": str(artifact)}] - _assert_combined_runtime(module) - - -@pytest.mark.skipif(sys.platform == "win32", reason="shared-library loader behavior differs on Windows") -@pytest.mark.parametrize("artifact_kind", ["archive", "shared_library"]) -def test_source_free_direct_entrypoint_resolves_from_external_fortran_library( - tmp_path: Path, - artifact_kind: str, -): +def test_source_free_direct_entrypoint_resolves_from_external_fortran_archive(tmp_path: Path): + """A bind(C) entrypoint needs no bridge, so the binding links the archive's own C symbol.""" source = _write_source( tmp_path / "sources", "external_direct.f90", _direct_external_source("external_direct", "value + 9_c_int"), ) native_object = _compile_source(source, tmp_path / "native" / "objects") - artifact = ( - _archive(tmp_path / "native" / "libexternal_direct.a", (native_object,)) - if artifact_kind == "archive" - else _shared_library(tmp_path / "native" / "libexternal_direct.so", (native_object,)) - ) + artifact = _archive(tmp_path / "native" / "libexternal_direct.a", (native_object,)) entry = _write_contract_package( tmp_path / "contracts" / "external_direct", entry=( @@ -209,7 +162,7 @@ def test_source_free_direct_entrypoint_resolves_from_external_fortran_library( "external_direct_wrapper.h", } assert result.native_generated_code_groups == () - assert result.native_build_plan.to_dict()["link_items"] == [{"kind": artifact_kind, "path": str(artifact)}] + assert result.native_build_plan.to_dict()["link_items"] == [{"kind": "archive", "path": str(artifact)}] @pytest.mark.skipif(sys.platform == "win32", reason="shared-library loader behavior differs on Windows") @@ -398,54 +351,6 @@ def test_static_archive_groups_resolve_cyclic_archive_dependencies(tmp_path: Pat assert module.cycle_entry(np.int32(5)) == np.int32(18) -@pytest.mark.skipif(sys.platform == "win32", reason="shared-library loader behavior differs on Windows") -def test_required_transitive_named_library_resolves_runtime_symbol(tmp_path: Path): - sources = tmp_path / "sources" - native = tmp_path / "native" - libs = tmp_path / "libs" - entry_source = _write_source( - sources, - "transitive_entry.f90", - """\ -integer function transitive_entry(value) result(out) - integer, intent(in) :: value - integer, external :: transitive_helper - out = transitive_helper(value) + 1 -end function transitive_entry -""", - ) - helper_source = _write_source( - sources, - "transitive_helper.f90", - _simple_external_source("transitive_helper", "value + 30"), - ) - entry_object = _compile_source(entry_source, native / "objects") - _shared_library( - libs / "libtransitive_bundle.so", - (_compile_source(helper_source, native / "helper"),), - ) - entry = _write_contract_package( - tmp_path / "contracts" / "transitive_native_bundle", - entry=_simple_external_contract("transitive_entry"), - ) - - result = build_pyi_extension( - entry, - native_objects=[entry_object], - native_libraries=["transitive_bundle"], - native_library_dirs=[libs], - output_name="transitive_native_bundle", - output_dir=tmp_path / "build", - ) - module = _import_from_build(result) - - assert result.native_build_plan.to_dict()["link_items"] == [ - {"kind": "object", "path": str(entry_object)}, - {"kind": "named_library", "name": "transitive_bundle"}, - ] - assert module.transitive_entry(np.int32(1)) == np.int32(32) - - def test_missing_symbol_reports_native_link_or_loader_error(tmp_path: Path): entry = _write_contract_package( tmp_path / "contracts" / "missing_symbol", diff --git a/tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py b/tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py index 65d43403c..837d8bb57 100644 --- a/tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py +++ b/tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py @@ -1,4 +1,4 @@ -"""Debug and optimized native wrapper ABI tests.""" +"""Native compile flags that change the wrapper's measured runtime ABI.""" import importlib import json @@ -10,7 +10,7 @@ import numpy as np import pytest -from tests.fortran._support.wrapper_build import _build_text_and_import, _sole_native_module +from tests.fortran._support.wrapper_build import _sole_native_module RUNTIME_ABI_SOURCE = Path(__file__).parent / "fixtures" / "native" / "fruntime_abi_f90.f90" pytestmark = pytest.mark.fortran_end_to_end @@ -45,72 +45,3 @@ def test_top_level_native_kind_flags_drive_internal_type_measurement(tmp_path: P assert module.scale(np.float64(4.0), np.float64(1.25)) == np.float64(5.0) finally: sys.path.remove(str(tmp_path)) - - -@pytest.mark.skipif( - sys.platform == "win32" or shutil.which("make") is None, - reason="optimized wrapper ABI smoke test requires GNU Make and a POSIX shell", -) -def test_debug_and_optimized_wrapper_builds_preserve_runtime_abi(tmp_path: Path): - source_text = RUNTIME_ABI_SOURCE.read_text(encoding="utf-8") - expected_generated_sources = { - "bind_c_fruntime_abi_f90_wrapper.f90", - "fruntime_abi_f90_wrapper.c", - "fruntime_abi_f90_wrapper.h", - } - debug_dir = tmp_path / "debug" - optimized_dir = tmp_path / "optimized" - debug_dir.mkdir() - optimized_dir.mkdir() - - debug_module = _build_text_and_import( - source_text, - "fruntime_abi_f90.f90", - debug_dir, - expected_generated_sources, - ) - assert debug_module.scale(np.float64(3.0), np.float64(2.5)) == np.float64(7.5) - - optimized_source = optimized_dir / "fruntime_abi_f90.f90" - optimized_source.write_text(source_text, encoding="utf-8") - generated = subprocess.run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--makefile", - str(optimized_source), - "--out-dir", - str(optimized_dir), - "--json", - ], - capture_output=True, - text=True, - check=True, - ) - payload = json.loads(generated.stdout) - makefile = Path(payload["build_makefile"]) - subprocess.run( - [ - "make", - "-j4", - "-f", - str(makefile), - "all", - "PRIK_FFLAGS=-O3", - "PRIK_CFLAGS=-O3", - "PRIK_LDFLAGS=-O3", - ], - capture_output=True, - text=True, - check=True, - ) - - sys.modules.pop("fruntime_abi_f90", None) - sys.path.insert(0, str(optimized_dir)) - try: - optimized_module = _sole_native_module(importlib.import_module("fruntime_abi_f90")) - assert optimized_module.scale(np.float64(4.0), np.float64(1.25)) == np.float64(5.0) - finally: - sys.path.remove(str(optimized_dir)) diff --git a/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py b/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py index 7f6d8ba71..07548d97e 100644 --- a/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py +++ b/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py @@ -334,43 +334,17 @@ def test_documented_readme_points_example_builds_and_imports(tmp_path: Path): sys.modules.pop("geometry", None) -def test_internal_preprocessing_mode_still_builds_importable_runtime_wrapper(tmp_path: Path): - source = tmp_path / SCALAR_SOURCE.name - build_dir = tmp_path / "build" - shutil.copyfile(SCALAR_SOURCE, source) - - result = build_fortran_extension( - source, - output_dir=build_dir, - preprocessing=PreprocessingConfig(), - ) - - assert result.compiled is True - assert result.build_makefile is None - assert any( - path.name == "prik_binding.h" and path.parent.name == "binding_support" for path in result.generated_files - ) - support_license = build_dir / "binding_support" / "LICENSE" - assert support_license in result.generated_files - assert "Copyright (c) 2026 Said Hadjout" in support_license.read_text(encoding="utf-8") - - sys.modules.pop(result.module_name, None) - sys.path.insert(0, str(build_dir)) - try: - module = _sole_native_module(importlib.import_module(result.module_name)) - finally: - sys.path.remove(str(build_dir)) - assert module.scale(np.float64(3.0), np.float64(2.5)) == np.float64(7.5) - - def test_source_build_result_records_structured_native_plan(tmp_path: Path): + """The internal preprocessor still builds an importable wrapper whose result records its native plan.""" source = tmp_path / SCALAR_SOURCE.name shutil.copyfile(SCALAR_SOURCE, source) - result = build_fortran_extension(source, output_dir=tmp_path) + result = build_fortran_extension(source, output_dir=tmp_path, preprocessing=PreprocessingConfig()) plan = result.native_build_plan object_path = tmp_path / "scale.o" + assert result.compiled is True + assert result.build_makefile is None assert isinstance(plan, NativeBuildPlan) assert result.to_dict()["native_build_plan"] == plan.to_dict() assert plan.compilation_units[0].source == source @@ -382,6 +356,20 @@ def test_source_build_result_records_structured_native_plan(tmp_path: Path): assert plan.include_dirs == (tmp_path,) assert plan.link_items == (NativeLinkItem("object", object_path),) assert "native_inputs" not in result.to_dict() + assert any( + path.name == "prik_binding.h" and path.parent.name == "binding_support" for path in result.generated_files + ) + support_license = tmp_path / "binding_support" / "LICENSE" + assert support_license in result.generated_files + assert "Copyright (c) 2026 Said Hadjout" in support_license.read_text(encoding="utf-8") + + sys.modules.pop(result.module_name, None) + sys.path.insert(0, str(tmp_path)) + try: + module = _sole_native_module(importlib.import_module(result.module_name)) + finally: + sys.path.remove(str(tmp_path)) + assert module.scale(np.float64(3.0), np.float64(2.5)) == np.float64(7.5) def test_source_build_reuses_native_plan_for_additional_compile_and_link_inputs(tmp_path: Path): @@ -530,41 +518,25 @@ def test_cli_builds_from_a_source_directory_and_prebuilt_object_only(tmp_path: P def test_native_link_plan_serializes_interleaved_item_kinds(): + """A serialized link plan keeps item order and every language fact used to select its driver.""" plan = NativeBuildPlan( link_items=( - NativeLinkItem("object", Path("objects/entry.o")), + NativeLinkItem("object", Path("objects/entry.o"), language="fortran"), NativeLinkItem("linker_argument", "-Wl,--start-group"), NativeLinkItem("archive", Path("lib/libsolver.a")), NativeLinkItem("shared_library", Path("lib/libsupport.so")), - NativeLinkItem("named_library", "lapack"), - NativeLinkItem("linker_argument", "-Wl,--end-group"), + NativeLinkItem("named_library", "lapack", language="fortran"), + NativeLinkItem("linker_argument", "-Wl,--end-group", language="c"), ) ) assert plan.to_dict()["link_items"] == [ - {"kind": "object", "path": "objects/entry.o"}, + {"kind": "object", "path": "objects/entry.o", "language": "fortran"}, {"kind": "linker_argument", "argument": "-Wl,--start-group"}, {"kind": "archive", "path": "lib/libsolver.a"}, {"kind": "shared_library", "path": "lib/libsupport.so"}, - {"kind": "named_library", "name": "lapack"}, - {"kind": "linker_argument", "argument": "-Wl,--end-group"}, - ] - - -def test_native_link_plan_preserves_language_requirements_for_every_item_kind(): - """A serialized link plan must retain every fact used to select its driver.""" - plan = NativeBuildPlan( - link_items=( - NativeLinkItem("object", Path("objects/entry.o"), language="fortran"), - NativeLinkItem("named_library", "runtime", language="fortran"), - NativeLinkItem("linker_argument", "-pthread", language="c"), - ) - ) - - assert plan.to_dict()["link_items"] == [ - {"kind": "object", "path": "objects/entry.o", "language": "fortran"}, - {"kind": "named_library", "name": "runtime", "language": "fortran"}, - {"kind": "linker_argument", "argument": "-pthread", "language": "c"}, + {"kind": "named_library", "name": "lapack", "language": "fortran"}, + {"kind": "linker_argument", "argument": "-Wl,--end-group", "language": "c"}, ] diff --git a/tests/fortran/infrastructure/building/pipeline/fixtures/generated_contracts/source_builds/fruntime_abi_f90/__init__.pyi b/tests/fortran/infrastructure/building/pipeline/fixtures/generated_contracts/source_builds/fruntime_abi_f90/__init__.pyi deleted file mode 100644 index ee22f99de..000000000 --- a/tests/fortran/infrastructure/building/pipeline/fixtures/generated_contracts/source_builds/fruntime_abi_f90/__init__.pyi +++ /dev/null @@ -1,3 +0,0 @@ -from . import fruntime_abi_f90 - -__all__ = ["fruntime_abi_f90"] diff --git a/tests/fortran/infrastructure/building/pipeline/fixtures/generated_contracts/source_builds/fruntime_abi_f90/fruntime_abi_f90.pyi b/tests/fortran/infrastructure/building/pipeline/fixtures/generated_contracts/source_builds/fruntime_abi_f90/fruntime_abi_f90.pyi deleted file mode 100644 index 48a2a532e..000000000 --- a/tests/fortran/infrastructure/building/pipeline/fixtures/generated_contracts/source_builds/fruntime_abi_f90/fruntime_abi_f90.pyi +++ /dev/null @@ -1,9 +0,0 @@ -from prik.contracts import Addr, Arg, Float64, native_call - -@native_call([Addr(Arg(0)), Addr(Arg(1))]) -def scale( - value: Float64, - factor: Float64 -) -> Float64: ... - -__all__ = ["scale"] diff --git a/tests/fortran/infrastructure/building/pipeline/test_cmake_packaging.py b/tests/fortran/infrastructure/building/pipeline/test_cmake_packaging.py index 048cf5941..753b37efa 100644 --- a/tests/fortran/infrastructure/building/pipeline/test_cmake_packaging.py +++ b/tests/fortran/infrastructure/building/pipeline/test_cmake_packaging.py @@ -15,70 +15,41 @@ @pytest.mark.slow -def test_installed_distribution_ships_prik_config_beside_use_prik() -> None: - module_dir, data_dir = ( - Path(line) - for line in installed_output( - "import sysconfig\n" +def test_installed_distribution_exposes_its_cmake_modules_through_every_discovery_route() -> None: + """The packaged directory, the data share, and both build-backend entry points name the modules. + + scikit-build-core reads ``cmake.module`` and sets ``_ROOT`` + from ``cmake.root``, so ``find_package(PRIK CONFIG REQUIRED)`` resolves with + no argument only because that root entry point is named ``PRIK``. + """ + module_dir, data_dir, module_entry_dir, root_name, root_dir = ( + installed_output( + "import os, sysconfig\n" + "from importlib import metadata, resources\n" "from prik.cmake import cmake_module_dir\n" + "entries = metadata.distribution('prik').entry_points\n" + "modules = [entry for entry in entries if entry.group == 'cmake.module']\n" + "roots = [entry for entry in entries if entry.group == 'cmake.root']\n" + "assert len(modules) == 1, modules\n" + "assert len(roots) == 1, roots\n" "print(cmake_module_dir())\n" "print(sysconfig.get_path('data'))\n" - ).splitlines() + "print(os.path.realpath(str(resources.files(modules[0].load()))))\n" + "print(roots[0].name)\n" + "print(os.path.realpath(str(resources.files(roots[0].load()))))\n" + ) + .strip() + .splitlines() ) + module_dir, data_dir, module_entry_dir = Path(module_dir), Path(data_dir), Path(module_entry_dir) assert REPO_ROOT not in module_dir.parents for directory in (module_dir, data_dir / "share" / "prik" / "cmake"): assert (directory / "UsePRIK.cmake").is_file(), directory assert (directory / "PRIKConfig.cmake").is_file(), directory - - -@pytest.mark.slow -def test_installed_distribution_exposes_the_cmake_module_entry_point() -> None: - """The entry point resolves the way a scikit-build-core build reads it.""" - entry_point_dir = Path( - installed_output( - "import os\n" - "from importlib import metadata, resources\n" - "from prik.cmake import cmake_module_dir\n" - "modules = [\n" - " entry\n" - " for entry in metadata.distribution('prik').entry_points\n" - " if entry.group == 'cmake.module'\n" - "]\n" - "assert len(modules) == 1, modules\n" - "directory = os.path.realpath(str(resources.files(modules[0].load())))\n" - "assert directory == os.path.realpath(str(cmake_module_dir())), directory\n" - "print(directory)\n" - ).strip() - ) - - assert REPO_ROOT not in entry_point_dir.parents - assert (entry_point_dir / "UsePRIK.cmake").is_file() - - -@pytest.mark.slow -def test_installed_distribution_exposes_the_cmake_root_entry_point_as_prik() -> None: - """scikit-build-core sets ``_ROOT``, so the name is the contract. - - ``find_package(PRIK CONFIG REQUIRED)`` resolves with no argument only - because that variable comes out as ``PRIK_ROOT``, which makes the entry - point's name load-bearing rather than decorative. - """ - name, directory = installed_output( - "import os\n" - "from importlib import metadata, resources\n" - "roots = [\n" - " entry\n" - " for entry in metadata.distribution('prik').entry_points\n" - " if entry.group == 'cmake.root'\n" - "]\n" - "assert len(roots) == 1, roots\n" - "print(roots[0].name)\n" - "print(os.path.realpath(str(resources.files(roots[0].load()))))\n" - ).splitlines() - - assert name == "PRIK" - assert (Path(directory) / "PRIKConfig.cmake").is_file() + assert module_entry_dir == module_dir.resolve() + assert root_name == "PRIK" + assert (Path(root_dir) / "PRIKConfig.cmake").is_file() @pytest.mark.slow diff --git a/tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py b/tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py index 097b953d7..c7a930d08 100644 --- a/tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py +++ b/tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py @@ -111,20 +111,6 @@ def _assert_scale_runtime_contract(module) -> None: assert module.scale(np.float64(2.0), np.float64(4.0)) == np.float64(8.0) -def test_wrapper_build_result_import_module_loads_and_caches_a_built_extension(tmp_path: Path): - result = build_fortran_extension(SOURCE, output_dir=tmp_path / "source_build") - - sys.modules.pop(result.module_name, None) - try: - module = result.import_module() - assert module.__file__ == str(result.shared_library) - native_module = _sole_native_module(module) - assert native_module.scale(np.float64(3.0), np.float64(2.5)) == np.float64(7.5) - assert result.import_module() is module - finally: - sys.modules.pop(result.module_name, None) - - def test_wrapper_build_result_import_module_requires_a_built_artifact(tmp_path: Path): result = WrapperBuildResult( sources=(), @@ -141,18 +127,6 @@ def test_wrapper_build_result_import_module_requires_a_built_artifact(tmp_path: result.import_module() -@pytest.fixture -def scale_runtime_module(pyi_parity_build_mode: str, tmp_path: Path): - if pyi_parity_build_mode == "source": - result = build_fortran_extension(SOURCE, output_dir=tmp_path / "source_build") - return _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) - - generated_pyi = _generate_pyi(SOURCE, tmp_path / "contracts", RUNTIME_ABI_GENERATED) - native_object = _compile_native_object(SOURCE, tmp_path / "native") - module, _payload = _build_pyi_cli(generated_pyi, native_object, tmp_path / "pyi_build") - return _sole_native_module(module) - - def test_pyi_cli_requires_a_native_link_input(tmp_path: Path): result = subprocess.run( [sys.executable, "-m", "prik", str(PYI_FIXTURE), "--out-dir", str(tmp_path)], @@ -398,13 +372,31 @@ def test_pyi_python_api_accepts_exactly_one_entry_contract(tmp_path: Path): build_pyi_extension([PYI_FIXTURE], native_objects=[tmp_path / "unused.o"]) -def test_generated_pyi_fixture_builds_from_native_object_without_source_reparse(tmp_path: Path): - native_object = _compile_native_object(SOURCE, tmp_path / "native") - module, payload = _build_pyi_cli(PYI_FIXTURE, native_object, tmp_path / "pyi_build") +def test_generated_pyi_replay_matches_source_build_without_source_reparse(tmp_path: Path): + """A source build and a replay of its generated contract over the native object agree. + + The generated contract is compared with its checked fixture, and the replay + compiles nothing from source: its only native input is the prebuilt object. + """ + source_result = build_fortran_extension(SOURCE, output_dir=tmp_path / "source_build") + sys.modules.pop(source_result.module_name, None) + try: + source_module = source_result.import_module() + assert source_module.__file__ == str(source_result.shared_library) + assert source_result.import_module() is source_module + source_native = _sole_native_module(source_module) + + generated_pyi = _generate_pyi(SOURCE, tmp_path / "contracts", RUNTIME_ABI_GENERATED) + native_object = _compile_native_object(SOURCE, tmp_path / "native") + replay_module, payload = _build_pyi_cli(generated_pyi, native_object, tmp_path / "pyi_build") + replay_native = _sole_native_module(replay_module) + finally: + sys.modules.pop(source_result.module_name, None) native_plan = payload["native_build_plan"] + assert payload["module_name"] == source_result.module_name assert Path(payload["shared_library"]).is_file() - assert payload["sources"] == [str(PYI_FIXTURE)] + assert payload["sources"] == [str(generated_pyi), str(generated_pyi.parent / PYI_FIXTURE.name)] assert "native_inputs" not in payload assert native_plan["compilation_units"] == [] assert native_plan["produced_objects"] == [] @@ -412,7 +404,9 @@ def test_generated_pyi_fixture_builds_from_native_object_without_source_reparse( assert native_plan["module_dirs"] == [str(native_object.parent)] assert native_plan["include_dirs"] == [str(native_object.parent)] assert native_plan["link_items"] == [{"kind": "object", "path": str(native_object)}] - assert module.scale(np.float64(2.0), np.float64(4.0)) == np.float64(8.0) + for arguments, expected in (((2.0, 4.0), 8.0), ((3.0, 2.5), 7.5)): + values = tuple(np.float64(value) for value in arguments) + assert source_native.scale(*values) == replay_native.scale(*values) == np.float64(expected) def test_pyi_cli_preserves_explicit_ordered_link_items(tmp_path: Path): @@ -458,11 +452,3 @@ def test_pyi_cli_preserves_explicit_ordered_link_items(tmp_path: Path): assert manifest_link_items[1]["kind"] == "object" assert manifest_link_items[1]["path"].endswith(native_object.name) assert manifest_link_items[2] == {"argument": "-Wl,--end-group", "kind": "linker_argument"} - - -def test_generated_pyi_matches_checked_in_fixture(tmp_path: Path): - _generate_pyi(SOURCE, tmp_path / "contracts", RUNTIME_ABI_GENERATED) - - -def test_scale_runtime_contract(scale_runtime_module): - _assert_scale_runtime_contract(scale_runtime_module) diff --git a/tests/fortran/infrastructure/building/pipeline/test_root_build_api.py b/tests/fortran/infrastructure/building/pipeline/test_root_build_api.py index 38c309190..5ce33ee08 100644 --- a/tests/fortran/infrastructure/building/pipeline/test_root_build_api.py +++ b/tests/fortran/infrastructure/building/pipeline/test_root_build_api.py @@ -21,16 +21,3 @@ def test_root_facade_exposes_only_version_and_build_entrypoints(): assert prik.build_fortran_extension is build_fortran_extension assert prik.build_pyi_extension is build_pyi_extension assert prik.build_pyi_extension_from_manifest is build_pyi_extension_from_manifest - - -def test_root_build_entrypoints_support_direct_imports(): - """Normal users can import the documented build functions from ``prik``.""" - from prik import ( - build_fortran_extension, - build_pyi_extension, - build_pyi_extension_from_manifest, - ) - - assert build_fortran_extension is prik.build_fortran_extension - assert build_pyi_extension is prik.build_pyi_extension - assert build_pyi_extension_from_manifest is prik.build_pyi_extension_from_manifest diff --git a/tests/fortran/infrastructure/building/pipeline/test_source_generated_contracts.py b/tests/fortran/infrastructure/building/pipeline/test_source_generated_contracts.py index 72d258194..0a2d91d62 100644 --- a/tests/fortran/infrastructure/building/pipeline/test_source_generated_contracts.py +++ b/tests/fortran/infrastructure/building/pipeline/test_source_generated_contracts.py @@ -17,11 +17,6 @@ CONTRACT_ROOT = Path(__file__).parent / "fixtures" / "generated_contracts" / "source_builds" CASES = ( GeneratedContractCase("fdefault_output", (NATIVE_ROOT / "fdefault_output.f",), CONTRACT_ROOT / "fdefault_output"), - GeneratedContractCase( - "fruntime_abi_f90", - (NATIVE_ROOT / "fruntime_abi_f90.f90",), - CONTRACT_ROOT / "fruntime_abi_f90", - ), GeneratedContractCase("verbose_api", (NATIVE_ROOT / "verbose_api.f90",), CONTRACT_ROOT / "verbose_api"), ) diff --git a/tests/fortran/infrastructure/cli/pipeline/_support.py b/tests/fortran/infrastructure/cli/pipeline/_support.py deleted file mode 100644 index c4902d63e..000000000 --- a/tests/fortran/infrastructure/cli/pipeline/_support.py +++ /dev/null @@ -1,114 +0,0 @@ -import sys -import types - -import prik.cli as prik_cli -from tests.fortran._support.paths import GENERAL_FORTRAN_DIR - -TEST_FILE = GENERAL_FORTRAN_DIR / "basic_subroutine.f90" - - -class _MainParserError(Exception): - pass - - -def _main_args(**overrides): - values = { - "paths": ["input.f90"], - "command": "build", - "language": "fortran", - "parse": False, - "preprocessor_adapter": "auto", - "compiler": None, - "compile_commands": None, - "preprocess_template": None, - "include_dirs": [], - "defines": [], - "undefs": [], - "std": None, - "compiler_args": [], - "include_exposure": "reachable-project", - "public_includes": [], - "private_includes": [], - "show_vars": False, - "print_limit": None, - "vars_limit": None, - "makefile": False, - "generate_sources": False, - "build_manifest": None, - "no_compile_input_sources": False, - "native_fortran_sources": None, - "native_compile_flags": None, - "native_objects": None, - "native_libraries": None, - "native_link_items": None, - "native_library_dirs": None, - "strict_wrapper_names": False, - "wrapper_compiler_debug": False, - "wrapper_fortran_flags": None, - "wrapper_c_flags": None, - "semantics": False, - "pyi": False, - "json": False, - "out": None, - "out_dir": None, - "verbose": False, - "no_color": False, - "debug": False, - } - values.update(overrides) - if "command" not in overrides: - if values["parse"]: - values["command"] = "parse" - elif values["semantics"]: - values["command"] = "semantics" - elif values["pyi"] or values["generate_sources"] or values["makefile"]: - values["command"] = "generate" - return types.SimpleNamespace(**values) - - -def _install_main_parser(monkeypatch, args): - class FakeParser: - def add_argument(self, *_args, **_kwargs): - pass - - def add_argument_group(self, *_args, **_kwargs): - return self - - def parse_args(self, _argv=None): - return args - - def error(self, message): - raise _MainParserError(message) - - parser = FakeParser() - monkeypatch.setattr(prik_cli, "_parser_for_argv", lambda argv: (parser, argv)) - # main() falls back to sys.argv when called without an argv, and an empty - # command line prints help instead of dispatching. - monkeypatch.setattr(sys, "argv", ["prik", "input.f90"]) - return parser - - -def _patch_main_report_payloads( - monkeypatch, - *, - language="fortran", - parse_payload=None, - semantic_payload=None, -): - preprocessing = object() - calls = [] - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, _active_language, parser: language) - monkeypatch.setattr(prik_cli, "_build_preprocessing_config", lambda args, parser: preprocessing) - monkeypatch.setattr( - prik_cli, - "_parse_report", - lambda paths, active_preprocessing: calls.append(("parse", paths, active_preprocessing)) or parse_payload, - ) - monkeypatch.setattr( - prik_cli, - "_semantic_report", - lambda paths, active_preprocessing, *, language: ( - calls.append(("semantic", paths, active_preprocessing, language)) or semantic_payload - ), - ) - return preprocessing, calls diff --git a/tests/fortran/infrastructure/cli/pipeline/test_argument_contract.py b/tests/fortran/infrastructure/cli/pipeline/test_argument_contract.py index 405ebc92b..f16c9ec45 100644 --- a/tests/fortran/infrastructure/cli/pipeline/test_argument_contract.py +++ b/tests/fortran/infrastructure/cli/pipeline/test_argument_contract.py @@ -1,205 +1,322 @@ -"""Tests split by stable CLI argument-contract ownership.""" +"""CLI argument contracts: validation diagnostics, routing to the pipeline owners, and help. + +The CLI and the Python build API are two entry points with one owner, so these +tests run real command lines through ``prik.cli.main`` and assert what reaches +the public ``prik.pipeline.build`` functions instead of re-proving the build. +""" import argparse import json from pathlib import Path -import subprocess -import sys import types import pytest import prik.cli as prik_cli -from prik.preprocessing import PreprocessingError -from tests.fortran.infrastructure.cli.pipeline._support import ( - TEST_FILE, - _MainParserError, - _install_main_parser, - _main_args, -) - - -def test_cli_pyi_out_rejects_ambiguous_single_file_contract_package(tmp_path: Path): - source = tmp_path / "combined.f90" - source.write_text( - """module first_mod -contains - subroutine first() - end subroutine first -end module first_mod +from prik.pipeline import build as pipeline_build -module second_mod +MODULE_SOURCE = """module m contains - subroutine second() - end subroutine second -end module second_mod -""", - encoding="utf-8", - ) - output = tmp_path / "combined.pyi" - - cmd = [sys.executable, "-m", "prik", "generate", "--pyi", str(source), "--out", str(output)] - result = subprocess.run(cmd, capture_output=True, text=True) - - assert result.returncode != 0 - assert "generated contracts use one file per module" in result.stderr - assert not output.exists() - - -def test_cli_rejects_conflicting_json_and_pyi_out_from_inline_code(tmp_path: Path): - f90 = tmp_path / "conflict.f90" - f90.write_text( - """module conflict_mod -contains - subroutine ping() - end subroutine ping -end module conflict_mod -""", - encoding="utf-8", - ) - - cmd = [ - sys.executable, - "-m", - "prik", - "generate", - "--pyi", - str(f90), - "--json", - "--out", - str(tmp_path / "out"), - ] - res = subprocess.run(cmd, capture_output=True, text=True) - - assert res.returncode == 2 - assert "--out cannot be used with both --json and --pyi" in res.stderr - + subroutine add1(x) + integer, intent(inout) :: x + end subroutine add1 +end module m +""" + + +def _invoke(argv: list[str], capsys) -> tuple[int, str, str]: + """Run one command line in-process and return its exit code and output.""" + try: + code = prik_cli.main(argv) + except SystemExit as exc: + code = exc.code + captured = capsys.readouterr() + return code, captured.out, captured.err + + +@pytest.fixture +def cli_inputs(tmp_path: Path, monkeypatch) -> Path: + """Write one input of each kind the argument rows name, relative to the working directory.""" + monkeypatch.chdir(tmp_path) + (tmp_path / "src.f90").write_text(MODULE_SOURCE, encoding="utf-8") + (tmp_path / "solver.F90").write_text(MODULE_SOURCE, encoding="utf-8") + (tmp_path / "solver.source").write_text("subroutine solve()\nend subroutine solve\n", encoding="utf-8") + (tmp_path / "contract.pyi").write_text("def add1(x: int) -> int: ...\n", encoding="utf-8") + (tmp_path / "iface.PYI").write_text("def add1(x: int) -> int: ...\n", encoding="utf-8") + (tmp_path / "sources").mkdir() + (tmp_path / "sources" / "src.f90").write_text(MODULE_SOURCE, encoding="utf-8") + for language in ("fortran", "c"): + (tmp_path / f"{language}-build.json").write_text( + json.dumps({"schema_version": 5, "build_kind": "pyi-wrapper", "extension": {"native_language": language}}), + encoding="utf-8", + ) + return tmp_path -def test_prik_pyi_report_formats_and_rejects_conflicting_dependency_stubs(): - report = { - "first.f90": { - "pyi": "def first() -> None: ...", - "pyi_dependencies": {"shared": "class shared(Opaque):\n pass"}, - }, - "second.f90": { - "pyi": "def second() -> None: ...", - "pyi_dependencies": { - "shared": "class shared(Opaque):\n pass", - "extra": "class extra(Opaque):\n pass", - }, - }, - "empty.f90": {}, - } - text = prik_cli._format_pyi_report(report) +_WRAPPER_NATIVE_INPUTS = ( + "--native-fortran-sources, --native-c-sources, --native-objects, --native-library, or --native-link-item" +) - assert ( - text - == """File: first.f90 -def first() -> None: ... -Dependency stub: shared.pyi -class shared(Opaque): - pass +@pytest.mark.parametrize( + ("argv", "exit_code", "message"), + [ + pytest.param(["src.f90", "--out", ""], 2, "--out for wrapper builds requires an output name", id="out-empty"), + pytest.param(["src.f90", "--out", "module.txt"], 2, "expects NAME or NAME.so", id="out-suffix"), + pytest.param(["src.f90", "--out", "bad-name"], 2, "expects a valid Python module name", id="out-identifier"), + pytest.param(["src.f90", "--out"], 2, "argument --out: expected one argument", id="out-without-name"), + pytest.param( + ["generate", "--makefile", "src.f90", "--out", "module"], + 2, + "generate --sources/--makefile/--cmake uses --out-dir, not --out", + id="generate-makefile-out", + ), + pytest.param( + ["generate", "--pyi", "src.f90", "--json", "--out", "reports"], + 2, + "--out cannot be used with both --json and --pyi", + id="generate-json-and-pyi-out", + ), + pytest.param(["generate", "src.f90"], 2, "one of the arguments --pyi", id="generate-without-mode"), + pytest.param( + ["generate", "--pyi", "--sources", "src.f90"], 2, "not allowed with argument --pyi", id="generate-two-modes" + ), + pytest.param( + ["generate", "--pyi", "contract.pyi"], + 2, + "generate --pyi expects recognized fortran source suffixes; unsupported input: contract.pyi", + id="source-stage-given-contract", + ), + pytest.param(["parse", "src.f90", "--print-limit", "-1"], 2, "--print-limit must be >= 0", id="print-limit"), + pytest.param( + ["parse", "sources"], 2, "Input directory sources requires an explicit frontend", id="directory-language" + ), + pytest.param( + ["parse", "solver.source"], + 2, + "Cannot determine the input language for solver.source; pass --language fortran or --language c", + id="unknown-suffix-language", + ), + pytest.param( + ["parse", "src.f90", "--language", "c"], + 2, + "Fortran input src.f90 is incompatible with --language c; pass --language fortran", + id="fortran-input-with-c-frontend", + ), + pytest.param( + ["parse", "src.f90", "-D", "=bad"], 2, "--define/-D requires a macro name before '='", id="define-name" + ), + pytest.param( + ["parse", "src.f90", "--compiler", "cc", "--preprocess-template", "{source}"], + 2, + "--preprocess-template requires --preprocessor-adapter command-template", + id="template-without-adapter", + ), + pytest.param( + ["contract.pyi"], 2, f"A .pyi wrapper build requires {_WRAPPER_NATIVE_INPUTS}", id="contract-without-native" + ), + pytest.param( + ["solver.source", "--language", "fortran"], + 2, + "A wrapper build expects recognized Fortran source suffixes or one semantic .pyi contract; " + "unsupported input: solver.source", + id="wrapper-unknown-suffix", + ), + pytest.param( + ["src.f90", "--no-compile-input-sources"], + 2, + f"--no-compile-input-sources requires {_WRAPPER_NATIVE_INPUTS}", + id="no-compile-without-native", + ), + pytest.param( + ["contract.pyi", "--no-compile-input-sources", "--native-objects", "impl.o"], + 2, + "--no-compile-input-sources applies only to source-driven wrapper builds", + id="no-compile-with-contract", + ), + pytest.param( + ["contract.pyi", "--native-fortran-sources", "src.f90", "--assume-intent-in-scalars"], + 2, + "a semantic .pyi contract already states its own results", + id="contract-with-assumed-intent", + ), + pytest.param(["src.f90", "--jobs", "0"], 2, "jobs must be a positive integer", id="jobs-zero"), + pytest.param(["src.f90", "--jobs", "many"], 2, "jobs must be a positive integer", id="jobs-non-integer"), + pytest.param( + ["src.f90", "--native-compile-flags='-O2"], + 1, + "prik: error: Invalid --native-compile-flags value", + id="unbalanced-native-flags", + ), + pytest.param( + ["src.f90", "--wrapper-c-flags='-O0"], + 1, + "prik: error: Invalid --wrapper-c-flags value", + id="unbalanced-wrapper-flags", + ), + pytest.param( + ["probe", "--language", "fortran", "--compiler", "gfortran", "-I", "inc", "--std", "f2018"], + 2, + "add --expr to probe preprocessed expressions", + id="probe-mapping-with-preprocessing-options", + ), + pytest.param( + ["probe", "--language", "c", "--compiler", "cc", "--expr", "kind(1.0)"], + 2, + "--expr is supported only for --language fortran", + id="probe-expressions-for-c", + ), + pytest.param( + ["--build-manifest", "fortran-build.json", "--out-dir", "elsewhere"], + 2, + "replays its saved output directory", + id="manifest-out-dir", + ), + pytest.param( + ["--build-manifest", "fortran-build.json", "--language", "fortran"], + 2, + "replays its saved input language", + id="manifest-language", + ), + pytest.param( + ["--build-manifest", "fortran-build.json", "-D", "USE_FAST=1"], + 2, + "replays its saved preprocessing recipe", + id="manifest-preprocessing", + ), + pytest.param( + ["--build-manifest", "fortran-build.json", "--strict-wrapper-names"], + 2, + "replays saved wrapper behavior", + id="manifest-wrapper-behavior", + ), + pytest.param( + ["--build-manifest", "fortran-build.json", "--native-library", "openblas"], + 2, + "replays saved native inputs", + id="manifest-native-inputs", + ), + ], +) +@pytest.mark.usefixtures("cli_inputs") +def test_cli_rejects_invalid_invocation_with_its_documented_diagnostic( + capsys, argv: list[str], exit_code: int, message: str +): + code, out, err = _invoke(argv, capsys) -File: second.f90 -def second() -> None: ... + assert code == exit_code + assert out == "" + assert message in " ".join(err.split()) -Dependency stub: extra.pyi -class extra(Opaque): - pass -File: empty.f90 -""" - ) - with pytest.raises(ValueError, match="Conflicting generated dependency stub"): - prik_cli._write_pyi_dependencies( - { - "first.f90": {"pyi_dependencies": {"shared": "class shared:\n pass"}}, - "second.f90": {"pyi_dependencies": {"shared": "class shared:\n value: int"}}, - } +@pytest.fixture +def pipeline_calls(monkeypatch) -> list[tuple[str, tuple, dict]]: + """Record the public pipeline owner each wrapper build reaches instead of compiling.""" + calls: list[tuple[str, tuple, dict]] = [] + result = types.SimpleNamespace( + compiled=False, + to_dict=lambda: {"module_name": "demo", "compiled": False, "output_dir": "out"}, + ) + for owner in ("build_fortran_extension", "build_pyi_extension", "build_pyi_extension_from_manifest"): + monkeypatch.setattr( + pipeline_build, + owner, + lambda *args, _owner=owner, **kwargs: calls.append((_owner, args, kwargs)) or result, ) + return calls @pytest.mark.parametrize( - ("overrides", "expected"), + ("argv", "owner", "positional", "expected"), [ - ( - {"out": ""}, - "--out for wrapper builds requires an output name", + pytest.param( + ["src.f90"], + "build_fortran_extension", + (["src.f90"],), + {"compile_input_sources": True, "standard_logicals": True, "assume_intent_in_scalars": False}, + id="source-defaults", ), - ( - {"out": "module.txt"}, - "--out for wrapper builds expects NAME or NAME.so", + pytest.param( + ["solver.F90", "--no-standard-logicals", "--assume-intent-in-scalars"], + "build_fortran_extension", + (["solver.F90"],), + {"standard_logicals": False, "assume_intent_in_scalars": True}, + id="uppercase-source-interpretation-choices", ), - ( - {"out": "bad-name"}, - "--out for wrapper builds expects a valid Python module name", + pytest.param( + ["sources", "--language", "fortran", "--no-compile-input-sources", "--native-objects", "libnative.so"], + "build_fortran_extension", + (["sources"],), + {"compile_input_sources": False, "native_objects": ["libnative.so"]}, + id="source-directory-with-prebuilt-implementation", ), - ( - {"out": "module", "makefile": True}, - "generate --sources/--makefile/--cmake uses --out-dir, not --out", + pytest.param( + ["contract.pyi", "--native-objects", "native.o", "--compiler", "selected-ifx"], + "build_pyi_extension", + ("contract.pyi",), + {"input_compiler": "selected-ifx", "input_c_compiler": None, "standard_logicals": True}, + id="contract-defers-c-driver-to-compiler-pair", ), - ({"parse": True, "print_limit": -1}, "--print-limit must be >= 0"), - ( - {"paths": ["input.pyi"]}, - "A .pyi wrapper build requires --native-fortran-sources, --native-c-sources, --native-objects, " - "--native-library, or --native-link-item", + pytest.param( + ["iface.PYI", "--native-objects", "native.o", "--no-standard-logicals"], + "build_pyi_extension", + ("iface.PYI",), + {"native_language": "fortran", "standard_logicals": False}, + id="uppercase-contract-logical-choice", ), - ( - {"paths": ["input.unknown"]}, - "A wrapper build expects recognized Fortran source suffixes or one semantic .pyi contract; " - "unsupported input: input.unknown", - ), - ( - {"no_compile_input_sources": True}, - "--no-compile-input-sources requires --native-fortran-sources, --native-c-sources, --native-objects, " - "--native-library, or --native-link-item", - ), - ( + pytest.param( + [ + "--build-manifest", + "fortran-build.json", + "--out", + "REPLAYED", + "--compiler", + "selected-driver", + "-I", + "include", + "--json", + "--verbose", + "--no-color", + "--debug", + ], + "build_pyi_extension_from_manifest", + ("fortran-build.json",), { - "paths": ["input.pyi"], - "no_compile_input_sources": True, - "native_objects": ["implementation.o"], + "output_name": "REPLAYED", + "input_compiler": "selected-driver", + "input_c_compiler": None, + "include_dirs": ["include"], + "verbose": 1, }, - "--no-compile-input-sources applies only to source-driven wrapper builds", + id="manifest-accepts-documented-overrides", + ), + pytest.param( + ["--build-manifest", "c-build.json", "--compiler", "selected-driver"], + "build_pyi_extension_from_manifest", + ("c-build.json",), + {"input_compiler": None, "input_c_compiler": "selected-driver"}, + id="manifest-compiler-targets-recorded-c-language", ), ], ) -def test_prik_main_preserves_validation_diagnostics(monkeypatch, overrides, expected): - args = _main_args(**overrides) - _install_main_parser(monkeypatch, args) - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: language) - monkeypatch.setattr(prik_cli, "_build_preprocessing_config", lambda active_args, parser: object()) - - with pytest.raises(_MainParserError) as exc_info: - prik_cli.main() - - assert str(exc_info.value) == expected +@pytest.mark.usefixtures("cli_inputs") +def test_cli_routes_wrapper_builds_to_the_public_pipeline_owner( + pipeline_calls, capsys, argv, owner, positional, expected +): + code, _out, _err = _invoke(argv, capsys) + assert code == 0 + assert [(name, args) for name, args, _kwargs in pipeline_calls] == [(owner, positional)] + kwargs = pipeline_calls[0][2] + assert {name: kwargs[name] for name in expected} == expected -def test_prik_main_collects_many_native_inputs_from_one_option_group( - monkeypatch, - tmp_path: Path, - capsys, -): - contract = tmp_path / "module.pyi" - contract.write_text("def scale(x: float) -> float: ...\n", encoding="utf-8") - build_dir = tmp_path / "build" - calls = [] - result = types.SimpleNamespace( - to_dict=lambda: { - "module_name": "module", - "shared_library": str(build_dir / "module.so"), - } - ) - monkeypatch.setattr( - sys, - "argv", +@pytest.mark.usefixtures("cli_inputs") +def test_cli_forwards_grouped_and_repeated_build_options_to_the_pipeline(pipeline_calls, capsys): + """Multi-value options collect every value and grouped shell words split as a shell would.""" + code, out, _err = _invoke( [ - "prik", - str(contract), + "contract.pyi", "--compiler", "selected-gfortran", "-I", @@ -209,7 +326,7 @@ def test_prik_main_collects_many_native_inputs_from_one_option_group( "--native-fortran-sources", "source_one.f90", "source_two.f90", - "--native-compile-flags=-O2 -g0", + "--native-compile-flags=-O2 -DNAME='value with spaces'", "--jobs", "3", "--native-objects", @@ -218,7 +335,7 @@ def test_prik_main_collects_many_native_inputs_from_one_option_group( "libsolver.so", "--native-library", "blas", - "lapack", + "-llapack -lscalapack", "--native-link-item", "arg:-Wl,--start-group", "object:one.o", @@ -228,715 +345,71 @@ def test_prik_main_collects_many_native_inputs_from_one_option_group( "vendor/lib", "-I", "mods", - "-I", - "vendor/mods", "--wrapper-compiler-debug", "--wrapper-fortran-flags=-fno-range-check -g0", "--wrapper-c-flags=-O0 -g0", - "--out-dir", - str(build_dir), "--json", ], - ) - monkeypatch.setattr( - prik_cli, - "_run_wrap_build_with_diagnostics", - lambda active_args, active_preprocessing: calls.append((active_args, active_preprocessing)) or result, - ) - - assert prik_cli.main() == 0 - - assert len(calls) == 1 - active_args, _preprocessing = calls[0] - assert active_args.paths == [str(contract)] - assert active_args.compiler == "selected-gfortran" - assert active_args.include_dirs == ["include", "vendor/include", "mods", "vendor/mods"] - assert active_args.native_fortran_sources == ["source_one.f90", "source_two.f90"] - assert active_args.native_compile_flags == ["-O2 -g0"] - assert active_args.jobs == 3 - assert active_args.native_objects == ["one.o", "two.a", "libsolver.so"] - assert active_args.native_libraries == ["blas", "lapack"] - assert active_args.native_link_items == [ - "arg:-Wl,--start-group", - "object:one.o", - "arg:-Wl,--end-group", - ] - assert active_args.native_library_dirs == ["lib", "vendor/lib"] - assert prik_cli._cli_build_include_dirs(active_args) == ( - "include", - "vendor/include", - "mods", - "vendor/mods", - ) - assert active_args.wrapper_compiler_debug is True - assert active_args.wrapper_fortran_flags == ["-fno-range-check -g0"] - assert active_args.wrapper_c_flags == ["-O0 -g0"] - payload = json.loads(capsys.readouterr().out) - assert payload["module_name"] == "module" - - -@pytest.mark.parametrize( - "overrides", - [ - {"paths": ["input.f90"]}, - {"paths": ["contract.pyi"], "native_objects": ["native.o"]}, - {"paths": ["input.f90"], "makefile": True}, - {"paths": [], "build_manifest": "build/prik-build.json"}, - ], -) -def test_wrapper_inputs_select_the_default_build_stage(overrides): - assert prik_cli._is_wrapper_build(_main_args(**overrides)) - - -def test_explicit_inspection_stage_prevents_default_wrapper_selection(): - assert not prik_cli._is_wrapper_build(_main_args(parse=True)) - - -def test_cli_native_compile_flags_split_grouped_shell_words(): - assert prik_cli._cli_native_compile_flags(["-O2 -g0", "-DNAME='value with spaces'"]) == ( - "-O2", - "-g0", - "-DNAME=value with spaces", - ) - - -def test_source_build_routes_disabled_input_compilation_to_the_pipeline(monkeypatch): - from prik.pipeline import build as pipeline_build - - calls = [] - result = types.SimpleNamespace(compiled=False) - monkeypatch.setattr( - pipeline_build, - "build_fortran_extension", - lambda *args, **kwargs: calls.append((args, kwargs)) or result, - ) - args = _main_args( - paths=["native"], - no_compile_input_sources=True, - native_objects=["libnative.so"], - ) - - assert prik_cli._run_wrap_build(args, types.SimpleNamespace(compiler="gfortran")) is result - assert calls[0][0] == (["native"],) - assert calls[0][1]["compile_input_sources"] is False - assert calls[0][1]["native_objects"] == ["libnative.so"] - - -@pytest.mark.parametrize( - ("cli_arguments", "forwarded"), - [({}, True), ({"standard_logicals": False}, False)], -) -def test_source_build_routes_the_logical_interop_choice_to_the_pipeline( - monkeypatch, cli_arguments: dict[str, bool], forwarded: bool -): - """--no-standard-logicals is the only opt-out, so it must reach the build unchanged.""" - from prik.pipeline import build as pipeline_build - - calls = [] - result = types.SimpleNamespace(compiled=False) - monkeypatch.setattr( - pipeline_build, - "build_fortran_extension", - lambda *args, **kwargs: calls.append((args, kwargs)) or result, - ) - args = _main_args(paths=[str(TEST_FILE)], **cli_arguments) - - assert prik_cli._run_wrap_build(args, types.SimpleNamespace(compiler="gfortran")) is result - assert calls[0][1]["standard_logicals"] is forwarded - - -def test_fortran_pyi_build_defers_c_driver_selection_to_the_compiler_pair(monkeypatch): - from prik.pipeline import build as pipeline_build - - calls = [] - result = types.SimpleNamespace(compiled=False) - monkeypatch.setattr( - pipeline_build, - "build_pyi_extension", - lambda *args, **kwargs: calls.append((args, kwargs)) or result, - ) - args = _main_args( - paths=["contract.pyi"], - language="fortran", - native_objects=["native.o"], - ) - - assert prik_cli._run_wrap_build(args, types.SimpleNamespace(compiler="selected-ifx")) is result - assert calls[0][1]["input_compiler"] == "selected-ifx" - assert calls[0][1]["input_c_compiler"] is None - - -@pytest.mark.parametrize( - ("cli_arguments", "forwarded"), - [({}, True), ({"standard_logicals": False}, False)], -) -def test_pyi_contract_build_routes_the_logical_interop_choice_to_the_pipeline( - monkeypatch, cli_arguments: dict[str, bool], forwarded: bool -): - """A contract build compiles the same native Fortran, so it must carry the same choice.""" - from prik.pipeline import build as pipeline_build - - calls = [] - result = types.SimpleNamespace(compiled=False) - monkeypatch.setattr( - pipeline_build, - "build_pyi_extension", - lambda *args, **kwargs: calls.append((args, kwargs)) or result, - ) - args = _main_args(paths=["contract.pyi"], language="fortran", **cli_arguments) - - assert prik_cli._run_wrap_build(args, types.SimpleNamespace(compiler="gfortran")) is result - assert calls[0][1]["standard_logicals"] is forwarded - - -@pytest.mark.parametrize( - ("native_language", "expected_compilers"), - [ - ("fortran", ("selected-driver", None)), - ("c", (None, "selected-driver")), - ], -) -def test_manifest_compiler_override_targets_only_its_recorded_native_language( - tmp_path: Path, - monkeypatch, - native_language: str, - expected_compilers: tuple[str | None, str | None], -) -> None: - from prik.pipeline import build as pipeline_build - - manifest = tmp_path / "prik-build.json" - manifest.write_text( - json.dumps( - { - "schema_version": 5, - "build_kind": "pyi-wrapper", - "extension": {"native_language": native_language}, - } - ), - encoding="utf-8", - ) - calls = [] - result = types.SimpleNamespace(compiled=False) - monkeypatch.setattr( - pipeline_build, - "build_pyi_extension_from_manifest", - lambda *args, **kwargs: calls.append((args, kwargs)) or result, - ) - args = _main_args( - paths=[], - build_manifest=str(manifest), - compiler="selected-driver", - ) - - assert prik_cli._run_wrap_build(args, types.SimpleNamespace(compiler=None)) is result - assert (calls[0][1]["input_compiler"], calls[0][1]["input_c_compiler"]) == expected_compilers - - -@pytest.mark.parametrize("jobs", ("0", "many")) -def test_cli_compile_jobs_rejects_non_positive_or_non_integer_values(jobs: str, capsys) -> None: - with pytest.raises(SystemExit) as exc_info: - prik_cli.main([str(TEST_FILE), "--jobs", jobs]) - - assert exc_info.value.code == 2 - assert "jobs must be a positive integer" in capsys.readouterr().err - - -def test_cli_wrapper_flags_split_grouped_shell_words(): - assert prik_cli._cli_wrapper_fortran_flags(["-O0 -g", "-DNAME='value with spaces'"]) == ( - "-O0", - "-g", - "-DNAME=value with spaces", - ) - assert prik_cli._cli_wrapper_c_flags(["-O1 -g0"]) == ("-O1", "-g0") - - -def test_cli_native_compile_flags_reject_malformed_grouped_value(): - with pytest.raises(ValueError, match="Invalid --native-compile-flags value"): - prik_cli._cli_native_compile_flags(["'-O2"]) - - -def test_cli_wrapper_flags_reject_malformed_grouped_value(): - with pytest.raises(ValueError, match="Invalid --wrapper-c-flags value"): - prik_cli._cli_wrapper_c_flags(["'-O0"]) - - -def test_prik_build_preprocessing_config_preserves_macro_validation_errors(monkeypatch): - class Parser: - def error(self, message): - raise ValueError(message) - - def args(**overrides): - values = { - "defines": [], - "undefs": [], - "compiler": None, - "compile_commands": None, - "preprocessor_adapter": "auto", - "preprocess_template": None, - "include_dirs": [], - "std": None, - "compiler_args": [], - "include_exposure": "reachable-project", - "public_includes": [], - "private_includes": [], - "language": "fortran", - } - values.update(overrides) - return types.SimpleNamespace(**values) - - def reject(value, option): - raise PreprocessingError(f"{option}: invalid {value}", category="INVALID_MACRO_NAME") - - monkeypatch.setattr(prik_cli, "validate_macro_name", reject) - - with pytest.raises(ValueError) as define_error: - prik_cli._build_preprocessing_config(args(defines=["=bad"]), Parser()) - assert str(define_error.value) == "--define/-D: invalid =bad" - - with pytest.raises(ValueError) as undef_error: - prik_cli._build_preprocessing_config(args(undefs=["=bad"]), Parser()) - assert str(undef_error.value) == "--undef/-U: invalid =bad" - - -@pytest.mark.parametrize( - ("overrides", "message"), - [ - ( - {"compiler": "cc", "preprocess_template": "{source}"}, - "--preprocess-template requires --preprocessor-adapter command-template", - ), - ], -) -def test_prik_build_preprocessing_config_preserves_validation_diagnostics(overrides, message): - values = { - "defines": [], - "undefs": [], - "compiler": None, - "compile_commands": None, - "preprocessor_adapter": "auto", - "preprocess_template": None, - "include_dirs": [], - "std": None, - "compiler_args": [], - "include_exposure": "reachable-project", - "public_includes": [], - "private_includes": [], - "language": "fortran", - } - values.update(overrides) - - class Parser: - def error(self, received): - raise ValueError(received) - - with pytest.raises(ValueError) as error: - prik_cli._build_preprocessing_config(types.SimpleNamespace(**values), Parser()) - assert str(error.value) == message - - -def test_prik_resolve_language_handles_fortran_and_ambiguous_input_edges(tmp_path: Path): - class ErrorParser: - def error(self, message): - raise ValueError(message) - - parser = ErrorParser() - input_dir = tmp_path / "inputs" - input_dir.mkdir() - f_source = tmp_path / "solver.F90" - f_source.write_text("subroutine solve()\nend subroutine solve\n", encoding="utf-8") - stub = tmp_path / "iface.PYI" - stub.write_text("def solve() -> None: ...\n", encoding="utf-8") - unknown = tmp_path / "notes.txt" - unknown.write_text("notes\n", encoding="utf-8") - - with pytest.raises(ValueError) as directory_error: - prik_cli._resolve_language([str(input_dir)], None, parser) - assert str(directory_error.value) == ( - f"Input directory {input_dir} requires an explicit frontend; " - "pass --language fortran or --language c. Use --help for examples." - ) - - assert prik_cli._resolve_language([str(f_source)], None, parser) == "fortran" - assert prik_cli._resolve_language([str(stub)], None, parser) == "fortran" - - with pytest.raises(ValueError) as unknown_error: - prik_cli._resolve_language([str(unknown)], None, parser) - assert str(unknown_error.value) == ( - f"Cannot determine the input language for {unknown}; " - "pass --language fortran or --language c. Use --help for examples." - ) - - -def test_cli_without_language_keeps_fortran_default_behavior(): - cmd = [sys.executable, "-m", "prik", "parse", str(TEST_FILE)] - - result = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert "subroutine add1" in result.stdout - assert "Language: c" not in result.stdout - - -def test_cli_wrapper_out_requires_name_for_default_wrap(): - cmd = [sys.executable, "-m", "prik", str(TEST_FILE), "--out"] - res = subprocess.run(cmd, capture_output=True, text=True) - assert res.returncode == 2 - assert "argument --out: expected one argument" in res.stderr - - -def test_prik_command_parsers_group_options_by_user_intent(): - top_help = prik_cli._top_level_parser(["--help"]).format_help() - build_help = prik_cli._build_parser(["input.f90", "--help"]).format_help() - parse_help = prik_cli._parse_parser(["--help"]).format_help() - semantics_help = prik_cli._semantics_parser(["--help"]).format_help() - generate_help = prik_cli._generate_parser(["--help"]).format_help() - probe_help = prik_cli._probe_parser(["--help"]).format_help() - normalized_top_help = " ".join(top_help.split()) - normalized_build_help = " ".join(build_help.split()) - normalized_parse_help = " ".join(parse_help.split()) - normalized_semantics_help = " ".join(semantics_help.split()) - normalized_generate_help = " ".join(generate_help.split()) - normalized_probe_help = " ".join(probe_help.split()) - - def assert_group_order(help_text, *headings): - positions = [help_text.index(heading) for heading in headings] - assert positions == sorted(positions) - - assert "commands:" in top_help - assert all(command in top_help for command in ("parse", "semantics", "generate", "probe")) - assert top_help.startswith("usage: python3 -m prik INPUT [INPUT ...] [BUILD OPTIONS]") - assert "python3 -m prik {parse,semantics,generate,probe} [OPTIONS] ..." in top_help - assert "python3 -m prik --version" in top_help - assert ( - "Build Python extensions from Fortran or supported C APIs and inspect native interface artifacts." in top_help - ) - assert prik_cli._HELP_DIVIDER in top_help - assert "Basic wrapper build:" in top_help - assert "Name the Python extension:" in top_help - assert "Generate an editable semantic contract:" in top_help - assert "python3 -m prik points.f90" in top_help - assert "python3 -m prik points.f90 --out geometry" in top_help - assert "python3 -m prik generate --pyi points.f90 --out contracts" in top_help - assert "See the PRIK homepage for the points.f90 source and generated Python API:" in top_help - assert "https://pynumlab.github.io/prik/#see-it-in-action" in top_help - assert "points.f90" in build_help - assert "points.f90" in parse_help - assert "points.f90" in semantics_help - assert "points.f90" in generate_help - for help_text in (top_help, build_help, parse_help, semantics_help, generate_help, probe_help): - assert "scale.f90" not in help_text - assert "Run `python3 -m prik --help-build` for the full list of build options." in top_help + capsys, + ) + + assert code == 0 + assert json.loads(out)["module_name"] == "demo" + ((owner, _args, kwargs),) = pipeline_calls + assert owner == "build_pyi_extension" + assert kwargs["input_compiler"] == "selected-gfortran" + assert kwargs["native_include_dirs"] == ("include", "vendor/include", "mods") + assert kwargs["native_fortran_sources"] == ["source_one.f90", "source_two.f90"] + assert kwargs["native_fortran_flags"] == ("-O2", "-DNAME=value with spaces") + assert kwargs["jobs"] == 3 + assert kwargs["native_objects"] == ["one.o", "two.a", "libsolver.so"] + assert kwargs["native_libraries"] == ("blas", "-llapack", "-lscalapack") + assert kwargs["native_link_items"] == ( + {"kind": "linker_argument", "argument": "-Wl,--start-group"}, + {"kind": "object", "path": "one.o"}, + {"kind": "linker_argument", "argument": "-Wl,--end-group"}, + ) + assert kwargs["native_library_dirs"] == ["lib", "vendor/lib"] + assert kwargs["wrapper_compiler_debug"] is True + assert kwargs["wrapper_fortran_flags"] == ("-fno-range-check", "-g0") + assert kwargs["wrapper_c_flags"] == ("-O0", "-g0") + + +def test_top_level_help_is_concise_and_routes_to_detailed_help(capsys): + code, top_help, _err = _invoke(["--help"], capsys) + assert code == 0 + normalized = " ".join(top_help.split()) for command in ("parse", "semantics", "generate", "probe"): assert f"python3 -m prik {command} --help" in top_help - for heading in ("positional arguments:", "build options:"): - assert heading in top_help - for common_option in ( - "--out", - "--out-dir", - "--language", - "--compiler", - "--include-dir", - "--native-compile-flags", - "--native-c-compile-flags", - "--jobs", - "--native-library", - "--verbose", - "--help-build", - "--version", - ): - assert common_option in top_help - assert "Input-language compiler used throughout the extension" in top_help - assert "Input language (default: fortran; use c for direct C wrappers)" in normalized_top_help - assert "default: gfortran" in normalized_top_help - assert "Add an include directory used throughout the extension" in top_help - assert "Compiler used for preprocessing and internal datatype measurement" not in top_help - assert "--native-library openblas passes -lopenblas to the linker" in normalized_top_help - assert "--native-link-item" not in top_help - assert "--wrapper-c-flags" not in top_help - assert build_help.startswith("usage: python3 -m prik INPUT [INPUT ...]\n") - assert "[OUTPUT OPTIONS] [COMPILER OPTIONS] [WRAPPER OPTIONS]" in build_help - assert "[NATIVE OPTIONS] [DIAGNOSTIC OPTIONS]" in build_help - assert "python3 -m prik --build-manifest PATH [MANIFEST OVERRIDES]" in build_help - assert "positional arguments:" in build_help - for heading in ( - "input selection:", - "output options:", - "compiler options:", - "wrapper options:", - "native options:", - "diagnostic options:", - ): - assert heading in build_help - assert "--native-link-item" in build_help - assert "--jobs" in build_help - assert "--wrapper-c-flags" in build_help - assert "compiler used throughout the extension build" in normalized_build_help - assert "Add a compiler include search directory" in build_help - assert "default: gfortran" in normalized_build_help - assert "default: ./__prik__" in normalized_build_help - assert ( - "Fortran or C source file(s), one source directory, or exactly one semantic .pyi contract" - in normalized_build_help - ) - assert "--no-compile-input-sources" in build_help - assert "Input language (default: fortran; use c for direct C wrappers)" in normalized_build_help - assert "Rebuild the extension from an existing prik-build.json" in normalized_build_help - assert "Name the Python extension and stable NAME.so library" in normalized_build_help - assert "Print build paths and metadata as JSON" in normalized_build_help - assert 'Native compiler flags (for example, "-O3 -fopenmp")' in normalized_build_help - assert "docs/user/reference/cli-commands.md" in build_help - assert "See docs/user/reference/cli-commands.md for all build options." in build_help - assert "Build from a semantic contract:" in build_help - assert "Replay a build manifest:" in build_help - assert "Manifest overrides: --out, --compiler, -I/--include-dir, --jobs" in normalized_build_help - assert "--language {fortran,c}" in build_help - assert parse_help.startswith("usage: python3 -m prik parse INPUT [INPUT ...] [OPTIONS]") - for heading in ( - "positional arguments:", - "input options:", - "preprocessing options:", - "C include options:", - "report options:", - "output options:", - "diagnostic options:", - ): - assert heading in parse_help - assert_group_order( - parse_help, - "options:", - "positional arguments:", - "input options:", - "preprocessing options:", - "C include options:", - "report options:", - "output options:", - "diagnostic options:", - ) - assert "--language {fortran,c}" in parse_help - assert "--compile-commands" in parse_help - assert "Compiler used for preprocessing" in normalized_parse_help - assert "Add a preprocessing include search directory" in normalized_parse_help - assert "datatype measurement" not in normalized_parse_help - assert "native and bridge compilation" not in normalized_parse_help - assert "default: gfortran; cc with --language c" in normalized_parse_help - assert semantics_help.startswith("usage: python3 -m prik semantics INPUT [INPUT ...] [OPTIONS]") - assert "--json" in semantics_help - assert "--print-limit" in semantics_help - assert "Write the report to PATH" in semantics_help - assert "Define a preprocessing macro" in semantics_help - for heading in ( - "positional arguments:", - "input options:", - "preprocessing options:", - "C include options:", - "report options:", - "output options:", - "diagnostic options:", - ): - assert heading in semantics_help - assert_group_order( - semantics_help, - "options:", - "positional arguments:", - "input options:", - "preprocessing options:", - "C include options:", - "report options:", - "output options:", - "diagnostic options:", - ) - assert "preprocessing and datatype measurement" in normalized_semantics_help - assert "native and bridge compilation" not in normalized_semantics_help - assert "default: gfortran; cc with --language c" in normalized_semantics_help - assert "(--pyi | --sources | --makefile | --cmake)" in generate_help - assert "INPUT [INPUT ...] [OPTIONS]" in generate_help - assert "--build-manifest PATH [OVERRIDES]" in generate_help - for heading in ( - "generation modes:", - "positional arguments:", - "input options:", - "compiler and preprocessing options:", - "C include options:", - "wrapper options:", - "native options:", - "output options:", - "diagnostic options:", - ): - assert heading in generate_help - assert_group_order( - generate_help, - "options:", - "generation modes:", - "positional arguments:", - "input options:", - "compiler and preprocessing options:", - "C include options:", - "wrapper options:", - "native options:", - "output options:", - "diagnostic options:", - ) - assert "--pyi" in generate_help - assert "--sources" in generate_help - assert "--makefile" in generate_help - assert "--cmake" in generate_help - assert "Read an existing prik-build.json and regenerate wrapper artifacts" in normalized_generate_help - assert "Compiler used for source analysis and wrapper build files" in normalized_generate_help - assert "default: gfortran; cc with --language c" in normalized_generate_help - assert "native compiler options:" not in generate_help - assert "link options:" not in generate_help - assert probe_help.startswith("usage: python3 -m prik probe --language {fortran,c} --compiler COMPILER [OPTIONS]\n") - for heading in ("probe options:", "execution options:", "output options:", "diagnostic options:"): - assert heading in probe_help - assert_group_order( - probe_help, - "options:", - "probe options:", - "execution options:", - "output options:", - "diagnostic options:", - ) - assert "--json" in probe_help - assert "--format" not in probe_help - assert "Probe compiler-target datatype sizes, alignment, and ABI facts." in probe_help - assert "Probe flags that change default kinds:" in probe_help - assert "--compiler-arg=-fdefault-real-8 --compiler-arg=-fdefault-integer-8" in probe_help - assert "Cross-target probe:" in probe_help - assert "--runner qemu-aarch64" in probe_help - assert "Native or cross compiler used to build the probe" in normalized_probe_help - assert "Add a probe include search directory" in normalized_probe_help - assert "default: gfortran" not in normalized_probe_help - assert "--sources" not in parse_help - assert "--show-vars" not in semantics_help - assert "--native-library-dir" not in probe_help - - -@pytest.mark.parametrize( - ("parser_factory", "purpose"), - [ - ( - prik_cli._top_level_parser, - "Build Python extensions from Fortran or supported C APIs and inspect native interface artifacts.", - ), - ( - prik_cli._build_parser, - "Build a Python extension from Fortran source, a supported C API, or a semantic .pyi contract.", - ), - ( - prik_cli._parse_parser, - "Inspect Fortran or C source declarations before semantic conversion.", - ), - ( - prik_cli._semantics_parser, - "Convert Fortran or C source code into language-neutral semantic IR models.", - ), - ( - prik_cli._generate_parser, - "Generate semantic .pyi contracts or wrapper build artifacts without compiling.", - ), - ( - prik_cli._probe_parser, - "Probe compiler-target datatype sizes, alignment, and ABI facts.", - ), - ], -) -def test_cli_help_places_a_clear_purpose_below_usage(parser_factory, purpose): - help_text = parser_factory(["--help", "--no-color"]).format_help() - - assert f"\n\n{purpose}\n\n" in help_text - - -@pytest.mark.parametrize( - ("parser_factory", "example_headings"), - [ - ( - prik_cli._top_level_parser, - ("Basic wrapper build:", "Name the Python extension:", "More help:"), - ), - ( - prik_cli._build_parser, - ("Basic wrapper build:", "Build from a semantic contract:", "Replay a build manifest:"), - ), - ( - prik_cli._parse_parser, - ( - "Basic Fortran inspection:", - "Detailed Fortran report:", - "C header as JSON:", - "--json picks the format, --out picks the destination:", - ), - ), - ( - prik_cli._semantics_parser, - ( - "Basic Fortran conversion:", - "C header:", - "Complete semantic IR as JSON on standard output:", - "--json picks the format, --out picks the destination:", - ), - ), - ( - prik_cli._generate_parser, - ("Editable semantic contract:", "Wrapper sources only:", "Reproducible Makefile build:"), - ), - ( - prik_cli._probe_parser, - ( - "Target datatype mapping table:", - "Complete measured report as JSON:", - "--json picks the format, --out picks the destination:", - "Probe flags that change default kinds:", - "Cross-target probe:", - ), - ), - ], -) -def test_cli_help_groups_examples_by_task(parser_factory, example_headings): - help_text = parser_factory(["--help", "--no-color"]).format_help() - - assert prik_cli._HELP_DIVIDER in help_text - assert all(heading in help_text for heading in example_headings) - - -def test_help_build_routes_to_the_full_default_build_help(): - result = subprocess.run( - [sys.executable, "-m", "prik", "--help-build", "--no-color"], - capture_output=True, - text=True, - ) - - assert result.returncode == 0 - assert "[OUTPUT OPTIONS] [COMPILER OPTIONS] [WRAPPER OPTIONS]" in result.stdout - assert "[NATIVE OPTIONS] [DIAGNOSTIC OPTIONS]" in result.stdout - assert "compiler options:" in result.stdout - assert "native options:" in result.stdout - assert "diagnostic options:" in result.stdout - assert "--native-link-item" in result.stdout - assert "--wrapper-c-flags" in result.stdout - normalized_help = " ".join(result.stdout.split()) - assert "Link NAME as -lNAME; for example, openblas adds -lopenblas" in normalized_help - - -def test_help_build_exposes_every_supported_build_option(): - parser = prik_cli._build_parser(["--help"]) - help_text = parser.format_help() - option_strings = { - option for action in parser._actions if action.help != argparse.SUPPRESS for option in action.option_strings - } + assert "python3 -m prik --help-build" in top_help + assert prik_cli._HELP_DIVIDER in top_help + # The first screen carries the options a first build needs, including one + # that changes the default Python surface, and leaves advanced linking out. + for option in ("--out", "--out-dir", "--compiler", "--native-library", "--jobs", "--assume-intent-in-scalars"): + assert option in top_help + assert "--native-library openblas passes -lopenblas to the linker" in normalized + for advanced in ("--native-link-item", "--wrapper-c-flags"): + assert advanced not in top_help - assert option_strings - assert all(option in help_text for option in option_strings) + code, build_help, _err = _invoke(["--help-build", "--no-color"], capsys) + assert code == 0 + for advanced in ("--native-link-item", "--wrapper-c-flags"): + assert advanced in build_help + assert "Link NAME as -lNAME; for example, openblas adds -lopenblas" in " ".join(build_help.split()) @pytest.mark.parametrize( "parser_factory", - ( + [ + prik_cli._build_parser, prik_cli._parse_parser, prik_cli._semantics_parser, prik_cli._generate_parser, prik_cli._probe_parser, - ), + ], + ids=["build", "parse", "semantics", "generate", "probe"], ) -def test_subcommand_help_exposes_every_supported_option(parser_factory): +def test_command_help_lists_every_supported_option(parser_factory): parser = parser_factory(["--help"]) help_text = parser.format_help() option_strings = { @@ -944,189 +417,24 @@ def test_subcommand_help_exposes_every_supported_option(parser_factory): } assert option_strings - assert all(option in help_text for option in option_strings) - - -@pytest.mark.parametrize( - ("extra_args", "message"), - [ - (["--out-dir", "elsewhere"], "replays its saved output directory"), - (["--language", "fortran"], "replays its saved input language"), - (["--preprocessor-adapter", "auto"], "replays its saved preprocessing recipe"), - (["-D", "USE_FAST=1"], "replays its saved preprocessing recipe"), - (["--strict-wrapper-names"], "replays saved wrapper behavior"), - (["--assume-intent-in-scalars"], "replays saved wrapper behavior"), - (["--native-library", "openblas"], "replays saved native inputs"), - ], -) -def test_manifest_replay_rejects_saved_settings_instead_of_ignoring_them(extra_args, message, capsys): - with pytest.raises(SystemExit) as exc_info: - prik_cli.main(["--build-manifest", "build/prik-build.json", *extra_args]) - - assert exc_info.value.code == 2 - assert message in capsys.readouterr().err - - -def test_manifest_replay_accepts_documented_overrides(monkeypatch): - captured = {} - - def run_build(args, preprocessing): - captured["args"] = args - captured["preprocessing"] = preprocessing - return object() - - monkeypatch.setattr(prik_cli, "_run_wrap_build_with_diagnostics", run_build) - monkeypatch.setattr(prik_cli, "_print_wrap_build_output", lambda _args, _result: None) - - result = prik_cli.main( - [ - "--build-manifest", - "build/prik-build.json", - "--out", - "REPLAYED", - "--compiler", - "selected-gfortran", - "-I", - "include", - "--json", - "--verbose", - "--no-color", - "--debug", - ] - ) - - assert result == 0 - assert captured["args"].out == "REPLAYED" - assert captured["args"].json is True - assert captured["args"].verbose is True - assert captured["args"].no_color is True - assert captured["args"].debug is True - assert captured["preprocessing"].compiler == "selected-gfortran" - assert captured["preprocessing"].include_dirs == ["include"] + assert sorted(option for option in option_strings if option not in help_text) == [] @pytest.mark.parametrize( - "argv", + ("command", "expected", "excluded"), [ - ["generate", "input.f90"], - ["generate", "--pyi", "--sources", "input.f90"], + ("parse", "Compiler used for preprocessing", "datatype measurement"), + ("semantics", "preprocessing and datatype measurement", "wrapper build files"), + ("generate", "source analysis and wrapper build files", "used to build the probe"), + ("probe", "used to build the probe", "source preprocessing"), ], ) -def test_generate_requires_exactly_one_output_mode(argv): - with pytest.raises(SystemExit) as exc_info: - prik_cli.main(argv) - - assert exc_info.value.code == 2 - - -def test_cli_requires_explicit_language_for_directory_and_unknown_suffix(tmp_path: Path): - source = tmp_path / "solver.source" - source.write_text("subroutine solve()\nend subroutine solve\n", encoding="utf-8") - - unknown = subprocess.run( - [sys.executable, "-m", "prik", "parse", str(source)], - capture_output=True, - text=True, - ) - assert unknown.returncode == 2 - assert "Cannot determine the input language" in unknown.stderr - assert "--language fortran or --language c" in unknown.stderr - - directory = subprocess.run( - [sys.executable, "-m", "prik", "parse", str(tmp_path)], - capture_output=True, - text=True, - ) - assert directory.returncode == 2 - assert "requires an explicit frontend" in directory.stderr - - explicit = subprocess.run( - [sys.executable, "-m", "prik", "parse", str(source), "--language", "fortran"], - capture_output=True, - text=True, - check=True, - ) - assert "subroutine solve" in explicit.stdout - - -def test_cli_rejects_fortran_file_with_explicit_c_frontend(tmp_path: Path): - source = tmp_path / "solver.f90" - source.write_text("subroutine solve()\nend subroutine solve\n", encoding="utf-8") - - result = subprocess.run( - [sys.executable, "-m", "prik", "parse", str(source), "--language", "c"], - capture_output=True, - text=True, - ) - - assert result.returncode == 2 - assert "incompatible with --language c" in result.stderr - assert "pass --language fortran" in result.stderr - - -def test_cli_fortran_rejects_embedded_c_declaration_outside_execution_body(tmp_path: Path): - source = tmp_path / "solver.f90" - source.write_text( - "subroutine solve()\n int add(int a, int b);\nend subroutine solve\n", - encoding="utf-8", - ) - result = subprocess.run( - [sys.executable, "-m", "prik", "generate", "--pyi", str(source)], - capture_output=True, - text=True, - ) - - assert result.returncode == 1 - assert "PARSE_UNSUPPORTED_DECLARATION" in result.stderr - assert "Unknown or unsupported datatype declaration" in result.stderr - - -def test_prik_cli_defaults_pyi_to_wrapper_and_requires_native_implementation(tmp_path: Path): - pyi = tmp_path / "module.pyi" - pyi.write_text("def f() -> None: ...\n", encoding="utf-8") - cmd = [sys.executable, "-m", "prik", str(pyi)] - res = subprocess.run(cmd, capture_output=True, text=True) - assert res.returncode == 2 - assert "A .pyi wrapper build requires --native-fortran-sources" in res.stderr - - -@pytest.mark.parametrize("macro_flag", ["-D", "-U"]) -def test_prik_main_rejects_invalid_macro_names(macro_flag: str, monkeypatch): - monkeypatch.setattr(sys, "argv", ["prik", "parse", str(TEST_FILE), macro_flag, "=invalid"]) - with pytest.raises(SystemExit): - prik_cli.main() - - -def test_assume_intent_in_scalars_is_discoverable_from_the_first_help_screen(): - """The option changes the default Python surface, so it is not hidden behind --help-build.""" - top_help = prik_cli._top_level_parser(["--help"]).format_help() - build_help = prik_cli._build_parser(["input.f90", "--help"]).format_help() - generate_help = prik_cli._generate_parser(["--help"]).format_help() +def test_command_help_tailors_the_shared_compiler_option(capsys, command, expected, excluded): + code, help_text, _err = _invoke([command, "--help", "--no-color"], capsys) + normalized = " ".join(help_text.split()) - semantics_help = prik_cli._semantics_parser(["--help"]).format_help() - - assert "--assume-intent-in-scalars" in top_help - assert "--assume-intent-in-scalars" in build_help - assert "--assume-intent-in-scalars" in generate_help - assert "--assume-intent-in-scalars" in semantics_help - - -def test_pyi_wrapper_build_rejects_assume_intent_in_scalars(tmp_path: Path, capsys): - """A contract states its own results, so the option has no missing intent to interpret.""" - contract = tmp_path / "api.pyi" - contract.write_text("from prik.contracts import Float64\n", encoding="utf-8") - source = tmp_path / "api.f90" - source.write_text("subroutine noop()\nend subroutine noop\n", encoding="utf-8") - - with pytest.raises(SystemExit) as exc_info: - prik_cli.main( - [ - str(contract), - "--native-fortran-sources", - str(source), - "--assume-intent-in-scalars", - ] - ) - - assert exc_info.value.code == 2 - assert "already states its own results" in capsys.readouterr().err + assert code == 0 + assert prik_cli._HELP_DIVIDER in help_text + assert expected in normalized + assert excluded not in normalized + assert ("default: gfortran; cc with --language c" in normalized) is (command != "probe") diff --git a/tests/fortran/infrastructure/cli/pipeline/test_output_contract.py b/tests/fortran/infrastructure/cli/pipeline/test_output_contract.py index 70da16fb8..433ccadb8 100644 --- a/tests/fortran/infrastructure/cli/pipeline/test_output_contract.py +++ b/tests/fortran/infrastructure/cli/pipeline/test_output_contract.py @@ -1,10 +1,8 @@ -"""Tests split by stable CLI output-contract ownership.""" +"""CLI output contracts: report formats, destinations, and generated contract packages.""" import builtins -from dataclasses import dataclass from importlib import metadata import json -import os from pathlib import Path import shutil import subprocess @@ -14,24 +12,18 @@ import prik import pytest -from prik.preprocessing import PreprocessResult -from prik.preprocessing import source as preprocessing_source - import prik.cli as prik_cli from prik.parsers.fortran import cli as fortran_parser_cli -from prik.preprocessing import ( - PreprocessingConfig, - PreprocessingDiagnostic, - PreprocessingError, -) from tests.fortran._support.paths import GENERAL_FORTRAN_DIR -from tests.fortran.infrastructure.cli.pipeline._support import ( - TEST_FILE, - _MainParserError, - _install_main_parser, - _main_args, - _patch_main_report_payloads, -) + + +def _invoke(argv: list[str], capsys) -> tuple[int, str, str]: + try: + code = prik_cli.main(argv) + except SystemExit as exc: + code = exc.code + captured = capsys.readouterr() + return code, captured.out, captured.err def test_cli_and_python_api_report_installed_distribution_version(): @@ -51,724 +43,94 @@ def test_cli_and_python_api_report_installed_distribution_version(): assert result.stderr == "" -def test_cli_readable_output(): - cmd = [sys.executable, "-m", "prik", "parse", str(TEST_FILE)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - assert f"File: {TEST_FILE}" in res.stdout - assert "subroutine add1" in res.stdout - assert "Variables:" not in res.stdout - assert "Derived types: 0" not in res.stdout - print(res.stdout) - assert "Wrappable:" not in res.stdout - +SCOPES_SOURCE = """subroutine work(n) + integer, intent(in) :: n +end subroutine work -def test_cli_parse_show_vars_prints_scope_variables(tmp_path: Path): - f90 = tmp_path / "module_vars.f90" - f90.write_text( - """ -module module_vars +module m integer :: n real(kind=8), dimension(3) :: x contains - subroutine work() + subroutine work(n) + integer, intent(in) :: n end subroutine work -end module module_vars -""".strip(), - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik", "parse", str(f90), "--show-vars"] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert " - module module_vars (vars=2, uses=0)" in res.stdout - assert " Variables: 2" in res.stdout - assert " - n:integer[0]" in res.stdout - assert " - x:real(8)[1]" in res.stdout - - -def test_cli_parse_print_limit_limits_scope_variables_when_shown(tmp_path: Path): - f90 = tmp_path / "module_vars.f90" - f90.write_text( - """ -module module_vars - integer :: n - real(kind=8), dimension(3) :: x -contains - subroutine work() - end subroutine work -end module module_vars -""".strip(), - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik", "parse", str(f90), "--show-vars", "--print-limit", "1"] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert " Variables: 2" in res.stdout - assert " - n:integer[0]" in res.stdout - assert " - x:real(8)[1]" not in res.stdout - assert " ... 1 more variables" in res.stdout - - -def test_cli_parse_print_limit_limits_procedures(tmp_path: Path): - f90 = tmp_path / "many_procs.f90" - f90.write_text( - """ -module many_procs -contains - subroutine first() - end subroutine first subroutine second() end subroutine second -end module many_procs -""".strip(), - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik", "parse", str(f90), "--print-limit", "1"] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert " Procedures: 2" in res.stdout - assert " - subroutine first()" in res.stdout - assert " - subroutine second()" not in res.stdout - assert " ... 1 more procedures" in res.stdout - assert "Variables:" not in res.stdout - - -def test_cli_json_out(tmp_path: Path): - out = tmp_path / "report.json" - cmd = [ - sys.executable, - "-m", - "prik", - "parse", - str(TEST_FILE), - "--json", - "--out", - str(out), - ] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - assert res.stdout == "" - assert out.exists() - file_payload = json.loads(out.read_text()) - assert str(TEST_FILE) in file_payload - - -def test_cli_out_without_filename_uses_source_basename_json(tmp_path: Path): - """--out with no path writes one sibling file per source in the selected format.""" - f90 = tmp_path / "mini.f90" - f90.write_text("subroutine work(n)\n integer, intent(in) :: n\nend subroutine work\n", encoding="utf-8") - cmd = [sys.executable, "-m", "prik", "parse", str(f90), "--json", "--out"] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - assert res.stdout == "" - out = tmp_path / "mini.json" - assert out.exists() - file_payload = json.loads(out.read_text()) - assert str(f90) in file_payload - - -def test_cli_out_without_json_writes_the_human_report_beside_each_source(tmp_path: Path): - """--out selects only the destination, so without --json it writes the report text.""" - f90 = tmp_path / "mini.f90" - f90.write_text("subroutine work(n)\n integer, intent(in) :: n\nend subroutine work\n", encoding="utf-8") - cmd = [sys.executable, "-m", "prik", "parse", str(f90), "--out"] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - assert res.stdout == "" - assert not (tmp_path / "mini.json").exists() - assert f"File: {f90}" in (tmp_path / "mini.txt").read_text(encoding="utf-8") - - -def test_cli_json_output_without_out(): - cmd = [sys.executable, "-m", "prik", "parse", str(TEST_FILE), "--json"] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - payload = json.loads(res.stdout) - assert str(TEST_FILE) in payload - - -def test_cli_pyi_output_without_out(): - cmd = [sys.executable, "-m", "prik", "generate", "--pyi", str(TEST_FILE)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - assert f"File: {TEST_FILE}" in res.stdout - assert "def add1(" in res.stdout - - -def test_cli_formats_parse_error_with_ansi_by_default(tmp_path: Path): - f90 = tmp_path / "bad.f90" - f90.write_text( - """subroutine bad(x) - weirdtype :: x -end subroutine bad -""", - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik", "parse", str(f90)] - env = {k: v for k, v in os.environ.items() if k != "NO_COLOR"} - res = subprocess.run(cmd, capture_output=True, text=True, env=env) - - assert res.returncode == 1 - assert "\033[" in res.stderr - assert "error" in res.stderr - - -def test_cli_semantics_out_writes_json_without_stdout(tmp_path: Path): - out = tmp_path / "prik.semantics.json" - cmd = [sys.executable, "-m", "prik", "semantics", str(TEST_FILE), "--json", "--out", str(out)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert res.stdout == "" - assert out.exists() - payload = json.loads(out.read_text(encoding="utf-8")) - assert str(TEST_FILE) in payload - assert "semantic_modules" in payload[str(TEST_FILE)] - - -def test_cli_semantics_without_json_output(): - """semantics prints the human summary by default and the record under --json.""" - cmd = [sys.executable, "-m", "prik", "semantics", str(TEST_FILE)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - assert res.stdout.startswith(f"File: {TEST_FILE}") - assert "Semantic modules:" in res.stdout +end module m +""" - res = subprocess.run([*cmd, "--json"], capture_output=True, text=True, check=True) - payload = json.loads(res.stdout) - assert str(TEST_FILE) in payload - assert "semantic_modules" in payload[str(TEST_FILE)] +MODERN_DERIVED_BLOCK = """ Derived types: 3 + - type particle (fields=3, methods=0) + Fields: 3 + - id:integer[0] + - mass:real(8)[0] + - position:real(8)[1] + - type vector3 (fields=1, methods=0) + Fields: 1 + - values:real(8)[1] + - type hidden_state (fields=1, methods=0) + Fields: 1 + - code:integer[0] +""" @pytest.mark.parametrize( - ("command", "description"), + ("source", "options", "present", "absent"), [ - (("semantics",), "semantics"), - (("generate", "--pyi"), "generate --pyi"), - ], -) -def test_cli_source_stage_rejects_pyi_contract_instead_of_printing_empty_output( - tmp_path: Path, - command: tuple[str, ...], - description: str, -): - contract = tmp_path / "contract.pyi" - contract.write_text("def add1(value: int) -> int: ...\n", encoding="utf-8") - - result = subprocess.run( - [sys.executable, "-m", "prik", *command, str(contract)], - capture_output=True, - text=True, - check=False, - ) - - assert result.returncode == 2 - assert result.stdout == "" - assert f"{description} expects recognized fortran source suffixes" in result.stderr - assert str(contract) in result.stderr - - -def test_cli_pyi_output(): - cmd = [sys.executable, "-m", "prik", "generate", "--pyi", str(TEST_FILE)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - assert f"File: {TEST_FILE}" in res.stdout - assert "def add1(" in res.stdout - - -def test_cli_pyi_out_writes_adjacent_contract_package(tmp_path: Path): - f90 = tmp_path / "mini.f90" - f90.write_text( - """module m -contains - subroutine add1(x) - integer, intent(inout) :: x - x = x + 1 - end subroutine add1 -end module m -""", - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik", "generate", "--pyi", str(f90), "--out"] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert res.stdout == "" - package = tmp_path / "mini" - assert (package / "mini.pyi").read_text(encoding="utf-8") == 'from . import m\n\n__all__ = ["m"]\n' - assert "def add1" in (package / "m.pyi").read_text(encoding="utf-8") - - -def test_cli_pyi_out_writes_modules_inside_source_contract_package(tmp_path: Path): - source = tmp_path / "combined.f90" - source.write_text( - """module first_mod -contains - subroutine first() - end subroutine first -end module first_mod - -module second_mod -contains - subroutine second() - end subroutine second -end module second_mod -""", - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik", "generate", "--pyi", str(source), "--out"] - result = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert result.stdout == "" - package = tmp_path / "combined" - assert (package / "combined.pyi").read_text(encoding="utf-8") == ( - 'from . import first_mod\nfrom . import second_mod\n\n__all__ = ["first_mod", "second_mod"]\n' - ) - assert "def first(" in (package / "first_mod.pyi").read_text(encoding="utf-8") - assert "def second(" in (package / "second_mod.pyi").read_text(encoding="utf-8") - - -def test_cli_pyi_out_uses_explicit_contract_package_from_inline_code(tmp_path: Path): - f90 = tmp_path / "explicit.f90" - f90.write_text( - """module explicit_mod -contains - subroutine set_value(x) - real(8), intent(out) :: x - end subroutine set_value -end module explicit_mod -""", - encoding="utf-8", - ) - out = tmp_path / "contracts" - - cmd = [sys.executable, "-m", "prik", "generate", "--pyi", str(f90), "--out", str(out)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert res.stdout == "" - text = (out / "__init__.pyi").read_text(encoding="utf-8") - assert text == 'from . import explicit_mod\n\n__all__ = ["explicit_mod"]\n' - leaf_text = (out / "explicit_mod.pyi").read_text(encoding="utf-8") - assert "@native_call([Return('x', 0)])" in leaf_text - assert "def set_value(" in leaf_text - assert "-> Float64: ..." in leaf_text - - -def test_cli_pyi_out_directory_resolves_renamed_project_kind(tmp_path: Path): - (tmp_path / "precision.f90").write_text( - """module precision_mod - integer, parameter :: word = 4 - integer, parameter :: wp = word * 2 -end module precision_mod -""", - encoding="utf-8", - ) - (tmp_path / "solver.f90").write_text( - """subroutine consume(x) - use precision_mod, only: local_wp => wp - real(kind=local_wp), intent(inout) :: x(*) -end subroutine consume -""", - encoding="utf-8", - ) - out = tmp_path / "contracts" - - cmd = [ - sys.executable, - "-m", - "prik", - "generate", - "--pyi", - str(tmp_path), - "--language", - "fortran", - "--out", - str(out), - ] - result = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert result.stdout == "" - text = (out / "__init__.pyi").read_text(encoding="utf-8") - assert "def consume(" in text - assert "x: Float64[Flat]" in text - assert "local_wp" not in text - - -def test_prik_write_pyi_dependencies_handles_nested_modules_and_empty_payloads(tmp_path: Path): - output_dir = tmp_path / "out" - text = "class Shared:\n pass" - - prik_cli._write_pyi_dependencies( - { - str(tmp_path / "nodeps.f90"): {}, - str(tmp_path / "first.f90"): {"pyi_dependencies": {"pkg.sub.shared": text}}, - str(tmp_path / "second.f90"): {"pyi_dependencies": {"pkg.sub.shared": text}}, - }, - output_dir=output_dir, - ) - - assert (output_dir / "pkg" / "sub" / "shared.pyi").read_text(encoding="utf-8") == text + "\n" - assert not (output_dir / "pkg.sub.shared.pyi").exists() - - -def test_prik_write_pyi_dependencies_uses_explicit_utf8(tmp_path: Path, monkeypatch): - writes = [] - - def write_text(path, data, *args, **kwargs): - assert not args - assert kwargs.get("encoding") is not None - assert kwargs["encoding"].lower() == "utf-8" - writes.append((path, data)) - return len(data) - - monkeypatch.setattr(Path, "write_text", write_text) - - prik_cli._write_pyi_dependencies( - {str(tmp_path / "first.f90"): {"pyi_dependencies": {"shared": "class Shared:\n pass"}}}, - output_dir=tmp_path, - ) - - assert writes == [(tmp_path / "shared.pyi", "class Shared:\n pass\n")] - - -def test_prik_main_formats_preprocessing_errors_with_and_without_diagnostics(monkeypatch, capsys): - monkeypatch.setattr(sys, "argv", ["prik", "parse", str(TEST_FILE)]) - - def fail_with_diagnostic(_paths, _preprocessing): - raise PreprocessingError( - "compiler failed", - category="PREPROCESSOR_FAILED", - diagnostics=[ - PreprocessingDiagnostic( - category="PREPROCESSOR_FAILED", - message="bad include", - path="source.F90", - line=9, - ) + pytest.param( + "scopes.f90", + [], + [ + " Procedures: 1\n - subroutine work(n:integer[0])", + " - module m (vars=2, uses=0)\n Procedures: 2", + " - subroutine second()", ], - ) - - monkeypatch.setattr(prik_cli, "_parse_report", fail_with_diagnostic) - assert prik_cli.main() == 1 - assert "source.F90:9: error[PREPROCESSOR_FAILED]: bad include" in capsys.readouterr().err - - def fail_without_diagnostic(_paths, _preprocessing): - raise PreprocessingError("plain failure", category="PREPROCESSOR_FAILED") - - monkeypatch.setattr(prik_cli, "_parse_report", fail_without_diagnostic) - assert prik_cli.main() == 1 - assert "prik: error[PREPROCESSOR_FAILED]: plain failure" in capsys.readouterr().err - - -def test_prik_main_preserves_zero_print_limit_and_legacy_vars_limit_contract(monkeypatch, capsys): - args = _main_args(parse=True, print_limit=0, vars_limit=7) - _install_main_parser(monkeypatch, args) - preprocessing = object() - parse_payload = {"parse": "payload"} - format_calls = [] - - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: language) - monkeypatch.setattr(prik_cli, "_build_preprocessing_config", lambda active_args, parser: preprocessing) - monkeypatch.setattr(prik_cli, "_parse_report", lambda paths, active_preprocessing: parse_payload) - monkeypatch.setattr( - prik_cli, - "_format_report", - lambda payload, **kwargs: format_calls.append((payload, kwargs)) or "formatted", - ) - - assert prik_cli.main() == 0 - assert capsys.readouterr().out == "formatted\n" - assert format_calls == [(parse_payload, {"show_vars": True, "print_limit": 0})] - - -def test_prik_main_preserves_conflicting_json_and_pyi_out_diagnostic(monkeypatch): - args = _main_args(pyi=True, json=True, out="/tmp/conflict.pyi") - _install_main_parser(monkeypatch, args) - _patch_main_report_payloads(monkeypatch, semantic_payload={"input.f90": {"pyi": "def work() -> None: ..."}}) - - with pytest.raises(_MainParserError) as exc_info: - prik_cli.main() - - assert str(exc_info.value) == "--out cannot be used with both --json and --pyi" - - -def test_prik_main_preserves_explicit_and_adjacent_json_write_contracts(monkeypatch): - writes = [] - monkeypatch.setattr( - Path, - "write_text", - lambda path, data, **kwargs: writes.append((path, data, kwargs)) or len(data), - ) - - explicit_payload = {"input.f90": {"node": 1}} - explicit_args = _main_args(parse=True, json=True, out="/tmp/report.json") - _install_main_parser(monkeypatch, explicit_args) - _patch_main_report_payloads(monkeypatch, parse_payload=explicit_payload) - assert prik_cli.main() == 0 - - adjacent_payload = { - "/tmp/first.f90": {"node": 1}, - "/tmp/empty.f90": {}, - } - adjacent_args = _main_args(parse=True, json=True, out="") - _install_main_parser(monkeypatch, adjacent_args) - _patch_main_report_payloads(monkeypatch, parse_payload=adjacent_payload) - assert prik_cli.main() == 0 - - assert writes == [ - (Path("/tmp/report.json"), json.dumps(explicit_payload, indent=2), {"encoding": "utf-8"}), - ( - Path("/tmp/first.json"), - json.dumps({"/tmp/first.f90": {"node": 1}}, indent=2), - {"encoding": "utf-8"}, + ["Variables:"], + id="free-and-module-procedure-share-a-name", ), - (Path("/tmp/empty.json"), json.dumps({"/tmp/empty.f90": {}}, indent=2), {"encoding": "utf-8"}), - ] - - -def test_prik_main_preserves_stdout_mode_matrix(monkeypatch, capsys): - parse_payload = {"parse": {"node": 1}} - semantic_payload = {"semantic": {"node": 2}} - scenarios = [ - ({"semantics": True}, "SEMANTIC\n", [("semantic-format", semantic_payload, {"print_limit": None})]), - ({"parse": True, "json": True}, json.dumps(parse_payload, indent=2) + "\n", []), - ({"pyi": True}, "", [("pyi-format", semantic_payload), ("pyi-output", "PYI")]), - ( - {"parse": True}, - "PARSE\n", - [("parse-format", parse_payload, {"show_vars": False, "print_limit": None})], + pytest.param( + "scopes.f90", + ["--show-vars"], + [" Variables: 2\n - n:integer[0]\n - x:real(8)[1]"], + [], + id="show-vars", + ), + pytest.param( + "scopes.f90", + ["--show-vars", "--print-limit", "1"], + [" - n:integer[0]\n ... 1 more variables", " ... 1 more procedures"], + ["x:real(8)[1]", "subroutine second()"], + id="print-limit-truncates-variables-and-procedures", + ), + pytest.param( + "scopes.f90", + ["--print-limit", "0"], + [" ... 1 more procedures", " ... 1 more modules"], + ["subroutine work"], + id="zero-print-limit-is-honoured", + ), + pytest.param( + str(GENERAL_FORTRAN_DIR / "modern_pyi_example.f90"), + [], + [MODERN_DERIVED_BLOCK, "init_particle(p:type(particle)[0]"], + [], + id="module-derived-types-and-derived-arguments", ), - ] - - for overrides, expected_stdout, expected_formats in scenarios: - args = _main_args(**overrides) - _install_main_parser(monkeypatch, args) - _patch_main_report_payloads( - monkeypatch, - parse_payload=parse_payload, - semantic_payload=semantic_payload, - ) - formats = [] - monkeypatch.setattr( - prik_cli, - "_format_report", - lambda payload, _formats=formats, **kwargs: _formats.append(("parse-format", payload, kwargs)) or "PARSE", - ) - monkeypatch.setattr( - prik_cli, - "_format_semantic_report", - lambda payload, _formats=formats, **kwargs: ( - _formats.append(("semantic-format", payload, kwargs)) or "SEMANTIC" - ), - ) - monkeypatch.setattr( - prik_cli, - "_format_pyi_report", - lambda payload, _formats=formats: _formats.append(("pyi-format", payload)) or "PYI", - ) - monkeypatch.setattr( - prik_cli, - "print_pyi_output", - lambda text, _formats=formats: _formats.append(("pyi-output", text)), - ) - - assert prik_cli.main() == 0 - assert capsys.readouterr().out == expected_stdout - assert formats == expected_formats - - -def test_prik_cli_helpers_cover_language_and_preprocessing_edges(tmp_path: Path, monkeypatch): - class ErrorParser: - def error(self, message): - raise ValueError(message) - - def args(**overrides): - values = { - "defines": [], - "undefs": [], - "compiler": None, - "compile_commands": None, - "preprocessor_adapter": "auto", - "preprocess_template": None, - "include_dirs": [], - "std": None, - "compiler_args": [], - "include_exposure": "reachable-project", - "public_includes": [], - "private_includes": [], - "language": "fortran", - } - values.update(overrides) - return types.SimpleNamespace(**values) - - parser = ErrorParser() - stub = tmp_path / "api.pyi" - stub.write_text("def add(x: Int32) -> Int32: ...\n", encoding="utf-8") - upper_stub = tmp_path / "upper.PYI" - upper_stub.write_text("def upper() -> None: ...\n", encoding="utf-8") - (tmp_path / "notes.txt").write_text("ignore", encoding="utf-8") - - # A directory yields every contract it holds, whatever the suffix's case, and each once. - assert prik_cli._expand_pyi_paths([str(tmp_path), str(stub)]) == [stub, upper_stub] - assert prik_cli._expand_pyi_paths([str(stub)]) == [stub] - assert prik_cli._expand_pyi_paths([str(upper_stub)]) == [upper_stub] - assert prik_cli._expand_pyi_paths([str(tmp_path / "notes.txt")]) == [] - with pytest.raises(ValueError, match="Cannot determine"): - prik_cli._resolve_language([str(tmp_path / "notes.txt")], None, parser) - - with pytest.raises(ValueError, match="--preprocess-template requires"): - prik_cli._build_preprocessing_config( - args( - compiler="cc", - preprocess_template="{compiler} -E {source}", - ), - parser, - ) - - def preprocess(path, *, language, config): - assert path == source - assert language == "fortran" - assert config.compiler == "gfortran" - return PreprocessResult( - source="subroutine work()\nend subroutine work\n", - recipe={"language": "fortran", "mode": "compiler"}, - ) - - # A parse report reads each file through the shared reader, recipe included. - source = tmp_path / "api.f90" - source.write_text("subroutine ignored()\nend subroutine ignored\n", encoding="utf-8") - monkeypatch.setattr(preprocessing_source, "preprocess_source", preprocess) - report = prik_cli._parse_report( - [str(source)], - PreprocessingConfig(mode="compiler", compiler="gfortran"), - ) - assert report[str(source)]["signatures"][0]["name"] == "work" - assert report[str(source)]["preprocessing_recipe"]["mode"] == "compiler" - - -def test_cli_help_is_concise_and_points_to_detailed_help(): - cmd = [sys.executable, "-m", "prik", "--help"] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - normalized_help = " ".join(res.stdout.split()) - assert "INPUT [INPUT ...] [BUILD OPTIONS]" in res.stdout - assert "positional arguments:" in res.stdout.lower() - assert "build options:" in res.stdout - assert "--native-compile-flags" in res.stdout - assert "--jobs" in res.stdout - assert "Input-language compiler used throughout the extension" in res.stdout - assert "default: gfortran" in normalized_help - assert "Add an include directory used throughout the extension" in res.stdout - assert "--native-library openblas passes -lopenblas to the linker" in normalized_help - assert "------------------------------ EXAMPLES ------------------------------" in res.stdout - assert "Basic wrapper build:" in res.stdout - assert "Name the Python extension:" in res.stdout - assert "Generate an editable semantic contract:" in res.stdout - assert "python3 -m prik points.f90" in res.stdout - assert "python3 -m prik points.f90 --out geometry" in res.stdout - assert "python3 -m prik generate --pyi points.f90 --out contracts" in res.stdout - assert "See the PRIK homepage for the points.f90 source and generated Python API:" in res.stdout - assert "https://pynumlab.github.io/prik/#see-it-in-action" in res.stdout - assert "python3 -m prik --help-build" in res.stdout - assert "python3 -m prik parse --help" in res.stdout - assert "python3 -m prik semantics --help" in res.stdout - assert "python3 -m prik generate --help" in res.stdout - assert "python3 -m prik probe --help" in res.stdout - - -@pytest.mark.parametrize( - ("command", "expected", "excluded"), - [ - ("parse", "Compiler used for preprocessing", "datatype measurement"), - ("semantics", "preprocessing and datatype measurement", "wrapper build files"), - ("generate", "source analysis and wrapper build files", "used to build the probe"), - ("probe", "used to build the probe", "source preprocessing"), ], ) -def test_subcommand_help_tailors_shared_compiler_options(command, expected, excluded): - result = subprocess.run( - [sys.executable, "-m", "prik", command, "--help", "--no-color"], - capture_output=True, - text=True, - check=True, - ) - normalized_help = " ".join(result.stdout.split()) - - assert "options:" in result.stdout - assert "------------------------------ EXAMPLES ------------------------------" in result.stdout - if command != "probe": - assert "positional arguments:" in result.stdout - assert "default: gfortran; cc with --language c" in normalized_help - else: - assert "default: gfortran" not in normalized_help - assert expected in normalized_help - assert excluded not in normalized_help - - -def test_cli_parse_shows_module_derived_types_and_derived_arg_kinds(): - fixture = GENERAL_FORTRAN_DIR / "modern_pyi_example.f90" - cmd = [sys.executable, "-m", "prik", "parse", str(fixture)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert " Derived types: 3" in res.stdout - assert "type particle" in res.stdout - assert "Fields: 3" in res.stdout - assert "- id:integer[0]" in res.stdout - assert "- mass:real(8)[0]" in res.stdout - assert "- position:real(8)[1]" in res.stdout - assert "init_particle(p:type(particle)[0]" in res.stdout - - -def test_fortran_parser_cli_helper_branches(tmp_path: Path, monkeypatch): - @dataclass - class Node: - name: str - parent: object = None - - monkeypatch.setenv("FORTRAN_PARSER_TEST_FLAG", " yes ") - assert fortran_parser_cli._env_flag("FORTRAN_PARSER_TEST_FLAG") is True - monkeypatch.delenv("FORTRAN_PARSER_TEST_FLAG") - assert fortran_parser_cli._env_flag("FORTRAN_PARSER_TEST_FLAG") is False - - assert fortran_parser_cli._diagnostic_color_enabled(disabled=True) is False - monkeypatch.setenv("NO_COLOR", "1") - assert fortran_parser_cli._diagnostic_color_enabled(disabled=False) is False - monkeypatch.delenv("NO_COLOR") - assert fortran_parser_cli._diagnostic_color_enabled(disabled=False) is True - - parent = Node("root") - assert fortran_parser_cli._to_dict_no_parent(Node("child", parent=parent)) == {"name": "child"} - assert fortran_parser_cli._to_dict_no_parent(Node("child", parent="root")) == { - "name": "child", - "parent": "root", - } +def test_parse_report_lists_each_scope_and_honours_report_options( + tmp_path: Path, monkeypatch, capsys, source, options, present, absent +): + monkeypatch.chdir(tmp_path) + (tmp_path / "scopes.f90").write_text(SCOPES_SOURCE, encoding="utf-8") - source = tmp_path / "nested" / "mini.f90" - source.parent.mkdir() - source.write_text("subroutine work(n)\n integer, intent(in) :: n\nend subroutine work\n", encoding="utf-8") - (source.parent / "notes.txt").write_text("ignore", encoding="utf-8") + code, out, _err = _invoke(["parse", source, *options], capsys) - report = fortran_parser_cli._parse_paths([str(tmp_path)]) - assert list(report) == [str(source)] - assert report[str(source)]["signatures"][0]["name"] == "work" - - -def test_fortran_parser_cli_formatting_branches(): - report = fortran_parser_cli._format_report( - { - "types.f90": { - "signatures": [], - "types": [{"name": "particle", "fields": [], "methods": []}], - "modules": [], - "submodules": [], - "programs": [], - "block_data": [], - } - } - ) - assert "Derived types: 1" in report - assert "- type particle (fields=0, methods=0)" in report - assert ( - fortran_parser_cli._format_var_type({"base_type": "derived", "kind": "particle", "rank": 0}) - == "type(particle)[0]" - ) - assert fortran_parser_cli._format_var_type({"base_type": "real", "kind": "4", "rank": 2}) == "real(4)[2]" + assert code == 0 + assert out.startswith(f"File: {source}\n") + assert [text for text in present if text not in out] == [] + assert [text for text in absent if text in out] == [] def test_fortran_parser_cli_format_report_print_limit_covers_sections(): @@ -859,135 +221,219 @@ def test_fortran_parser_cli_format_report_print_limit_covers_sections(): assert " - block data (vars=2)" in report assert " ... 1 more block data units" in report - assert fortran_parser_cli._format_variable_lines([], indent=" ", print_limit=1) == [] +MINI_SOURCE = """module m +contains + subroutine work(n) + integer, intent(in) :: n + end subroutine work +end module m +""" -def test_fortran_parser_cli_json_and_parse_errors(tmp_path: Path): - good = tmp_path / "good.f90" - good.write_text("subroutine work(n)\n integer, intent(in) :: n\nend subroutine work\n", encoding="utf-8") - - json_cmd = [sys.executable, "-m", "prik.parsers.fortran", str(good), "--json"] - json_res = subprocess.run(json_cmd, capture_output=True, text=True, check=True) - assert str(good) in json.loads(json_res.stdout) - bad = tmp_path / "bad.f90" - bad.write_text("subroutine bad(x)\n weirdtype :: x\nend subroutine bad\n", encoding="utf-8") - bad_cmd = [sys.executable, "-m", "prik.parsers.fortran", str(bad), "--no-color"] - bad_res = subprocess.run(bad_cmd, capture_output=True, text=True) - assert bad_res.returncode == 1 - assert bad_res.stdout == "" - assert "Traceback" not in bad_res.stderr - assert "Unknown or unsupported datatype" in bad_res.stderr +@pytest.mark.parametrize( + ("argv", "report_format", "written", "marker"), + [ + pytest.param(["parse", "mini.f90"], "text", None, "subroutine work", id="parse-text-stdout"), + pytest.param(["parse", "mini.f90", "--json"], "json", None, "modules", id="parse-json-stdout"), + pytest.param( + ["parse", "mini.f90", "--json", "--out", "report.json"], + "json", + "report.json", + "modules", + id="parse-json-file", + ), + pytest.param( + ["parse", "mini.f90", "--json", "--out"], "json", "mini.json", "modules", id="parse-json-adjacent" + ), + pytest.param(["parse", "mini.f90", "--out"], "text", "mini.txt", "subroutine work", id="parse-text-adjacent"), + pytest.param(["semantics", "mini.f90"], "text", None, "Semantic modules:", id="semantics-text-stdout"), + pytest.param(["semantics", "mini.f90", "--json"], "json", None, "semantic_modules", id="semantics-json-stdout"), + pytest.param( + ["semantics", "mini.f90", "--json", "--out", "semantics.json"], + "json", + "semantics.json", + "semantic_modules", + id="semantics-json-file", + ), + pytest.param(["generate", "--pyi", "mini.f90"], "text", None, "def work(", id="generate-pyi-stdout"), + ], +) +def test_report_format_and_destination_are_chosen_independently( + tmp_path: Path, monkeypatch, capsys, argv, report_format, written, marker +): + """--json picks the format and --out picks the destination; neither changes the report.""" + monkeypatch.chdir(tmp_path) + (tmp_path / "mini.f90").write_text(MINI_SOURCE, encoding="utf-8") + code, out, _err = _invoke(argv, capsys) -def test_prik_cli_helper_branches(tmp_path: Path, monkeypatch, capsys): - @dataclass - class Node: - name: str - parent: object = None + assert code == 0 + if written is None: + report = out + else: + assert out == "" + report = (tmp_path / written).read_text(encoding="utf-8") + if report_format == "json": + assert marker in json.loads(report)["mini.f90"] + else: + assert report.startswith("File: mini.f90\n") + assert marker in report - @dataclass - class ParentFirstNode: - parent: object - name: str - monkeypatch.setenv("PRIK_TEST_FLAG", "ON") - assert prik_cli._env_flag("PRIK_TEST_FLAG") is True - monkeypatch.delenv("PRIK_TEST_FLAG") - assert prik_cli._env_flag("PRIK_TEST_FLAG") is False +TWO_MODULES_SOURCE = """module first_mod +contains + subroutine first() + end subroutine first +end module first_mod - assert prik_cli._diagnostic_color_enabled(disabled=True) is False - monkeypatch.setenv("NO_COLOR", "1") - assert prik_cli._diagnostic_color_enabled(disabled=False) is False - monkeypatch.delenv("NO_COLOR") +module second_mod +contains + subroutine second() + end subroutine second +end module second_mod +""" - assert prik_cli._to_dict_no_parent(Node("child", parent=Node("root"))) == {"name": "child"} - assert prik_cli._to_dict_no_parent(ParentFirstNode(parent=Node("root"), name="child")) == {"name": "child"} - assert prik_cli._to_dict_no_parent({"node": Node("child", parent=Node("root"))}) == {"node": {"name": "child"}} +EXTERNAL_TYPE_SOURCE = """module physics + use types_mod, only: particle +contains + function create_particle() result(p) + type(particle) :: p + end function create_particle +end module physics +""" - source = tmp_path / "mini.f90" - source.write_text("subroutine work(n)\n integer, intent(in) :: n\nend subroutine work\n", encoding="utf-8") - assert prik_cli._expand_paths([str(tmp_path)]) == [source] +OPAQUE_PARTICLE_STUB = ( + 'from prik.contracts import Opaque\n\nclass particle(Opaque):\n pass\n\n__all__ = ["particle"]\n' +) - monkeypatch.setattr(sys.stdout, "isatty", lambda: True) - real_import = builtins.__import__ - def fail_rich_import(name, *args, **kwargs): - if name.startswith("rich"): - raise ImportError("rich disabled for test") - return real_import(name, *args, **kwargs) +@pytest.mark.parametrize( + ("sources", "argv", "expected"), + [ + pytest.param( + {"mini.f90": MINI_SOURCE}, + ["mini.f90", "--out"], + {"mini/mini.pyi": 'from . import m\n\n__all__ = ["m"]\n', "mini/m.pyi": ("def work(",)}, + id="adjacent-package-for-one-module", + ), + pytest.param( + {"combined.f90": TWO_MODULES_SOURCE}, + ["combined.f90", "--out"], + { + "combined/combined.pyi": ( + 'from . import first_mod\nfrom . import second_mod\n\n__all__ = ["first_mod", "second_mod"]\n' + ), + "combined/first_mod.pyi": ("def first(",), + "combined/second_mod.pyi": ("def second(",), + }, + id="adjacent-package-for-two-modules", + ), + pytest.param( + { + "explicit.f90": "module explicit_mod\ncontains\n subroutine set_value(x)\n" + " real(8), intent(out) :: x\n end subroutine set_value\nend module explicit_mod\n" + }, + ["explicit.f90", "--out", "contracts"], + { + "contracts/__init__.pyi": 'from . import explicit_mod\n\n__all__ = ["explicit_mod"]\n', + "contracts/explicit_mod.pyi": ("@native_call([Return('x', 0)])", "def set_value(", "-> Float64: ..."), + }, + id="explicit-package-directory", + ), + pytest.param( + {"physics.f90": EXTERNAL_TYPE_SOURCE}, + ["physics.f90", "--out"], + { + "physics/__init__.pyi": 'from . import physics\n\n__all__ = ["physics"]\n', + "physics/types_mod.pyi": OPAQUE_PARTICLE_STUB, + }, + id="opaque-dependency-stub-for-external-type", + ), + pytest.param( + { + "project/precision.f90": "module precision_mod\n integer, parameter :: word = 4\n" + " integer, parameter :: wp = word * 2\nend module precision_mod\n", + "project/solver.f90": "subroutine consume(x)\n use precision_mod, only: local_wp => wp\n" + " real(kind=local_wp), intent(inout) :: x(*)\nend subroutine consume\n", + }, + ["project", "--language", "fortran", "--out", "contracts"], + {"contracts/__init__.pyi": ("def consume(", "x: Float64[Flat]")}, + id="directory-input-resolves-renamed-project-kind", + ), + ], +) +def test_generate_pyi_writes_one_contract_per_module(tmp_path: Path, monkeypatch, capsys, sources, argv, expected): + monkeypatch.chdir(tmp_path) + for name, text in sources.items(): + (tmp_path / name).parent.mkdir(parents=True, exist_ok=True) + (tmp_path / name).write_text(text, encoding="utf-8") - monkeypatch.setattr(builtins, "__import__", fail_rich_import) - prik_cli.print_pyi_output("def f() -> None: ...") - assert "def f() -> None: ..." in capsys.readouterr().out + code, out, _err = _invoke(["generate", "--pyi", *argv], capsys) + assert code == 0 + assert out == "" + for name, content in expected.items(): + text = (tmp_path / name).read_text(encoding="utf-8") + if isinstance(content, str): + assert text == content + else: + assert [fragment for fragment in content if fragment not in text] == [] -def test_prik_print_pyi_output_uses_rich_and_falls_back(monkeypatch, capsys): - calls = [] - class FakeSyntax: - def __init__(self, code, lexer, **options): - self.code = code - self.lexer = lexer - self.options = options +def test_generate_pyi_rejects_a_single_file_destination(tmp_path: Path, monkeypatch): + monkeypatch.chdir(tmp_path) + (tmp_path / "combined.f90").write_text(TWO_MODULES_SOURCE, encoding="utf-8") - class FakeConsole: - def __init__(self, **options): - self.options = options + with pytest.raises(ValueError, match="generated contracts use one file per module"): + prik_cli.main(["generate", "--pyi", "combined.f90", "--out", "combined.pyi"]) - def print(self, syntax): - calls.append((syntax.code, syntax.lexer, syntax.options, self.options)) + assert not (tmp_path / "combined.pyi").exists() - rich_module = types.ModuleType("rich") - console_module = types.ModuleType("rich.console") - syntax_module = types.ModuleType("rich.syntax") - console_module.Console = FakeConsole - syntax_module.Syntax = FakeSyntax - monkeypatch.setitem(sys.modules, "rich", rich_module) - monkeypatch.setitem(sys.modules, "rich.console", console_module) - monkeypatch.setitem(sys.modules, "rich.syntax", syntax_module) - monkeypatch.setattr(sys.stdout, "isatty", lambda: True) - prik_cli.print_pyi_output("def f() -> None: ...") - assert calls == [ - ( - "def f() -> None: ...", - "python", - { - "theme": "ansi_dark", - "background_color": "default", - "line_numbers": False, - "word_wrap": False, +def test_pyi_dependency_stubs_are_shared_once_and_conflicts_rejected(tmp_path: Path): + report = { + "first.f90": { + "pyi": "def first() -> None: ...", + "pyi_dependencies": {"shared": "class shared(Opaque):\n pass"}, + }, + "second.f90": { + "pyi": "def second() -> None: ...", + "pyi_dependencies": { + "shared": "class shared(Opaque):\n pass", + "extra": "class extra(Opaque):\n pass", }, - {"force_terminal": True, "color_system": "auto"}, - ) - ] - assert capsys.readouterr().out == "" - - class RaisingConsole(FakeConsole): - def print(self, syntax): - raise RuntimeError("terminal failed") - - console_module.Console = RaisingConsole - prik_cli.print_pyi_output("def g() -> None: ...") - assert "def g() -> None: ..." in capsys.readouterr().out - - -def test_prik_main_formats_value_errors_or_reraises_for_debug(tmp_path: Path, monkeypatch, capsys): - source = tmp_path / "input.f90" - source.write_text("module input\nend module input\n", encoding="utf-8") + }, + "empty.f90": {}, + } - def fail_parse(_paths, _preprocessing): - raise ValueError("invalid generated interface") + assert prik_cli._format_pyi_report(report) == ( + "File: first.f90\ndef first() -> None: ...\n\n" + "Dependency stub: shared.pyi\nclass shared(Opaque):\n pass\n\n" + "File: second.f90\ndef second() -> None: ...\n\n" + "Dependency stub: extra.pyi\nclass extra(Opaque):\n pass\n\n" + "File: empty.f90\n" + ) - monkeypatch.setattr(prik_cli, "_parse_report", fail_parse) - monkeypatch.setattr(sys, "argv", ["prik", "parse", str(source)]) - assert prik_cli.main() == 1 - assert "prik: error: invalid generated interface" in capsys.readouterr().err + stub = "class Shared:\n pass" + prik_cli._write_pyi_dependencies( + { + "first.f90": {"pyi_dependencies": {"pkg.sub.shared": stub}}, + "second.f90": {"pyi_dependencies": {"pkg.sub.shared": stub}}, + }, + output_dir=tmp_path, + ) + assert (tmp_path / "pkg" / "sub" / "shared.pyi").read_text(encoding="utf-8") == stub + "\n" + assert not (tmp_path / "pkg.sub.shared.pyi").exists() - monkeypatch.setattr(sys, "argv", ["prik", "parse", str(source), "--debug"]) - with pytest.raises(ValueError, match="invalid generated interface"): - prik_cli.main() + with pytest.raises(ValueError, match="Conflicting generated dependency stub"): + prik_cli._write_pyi_dependencies( + { + "first.f90": {"pyi_dependencies": {"shared": "class shared:\n pass"}}, + "second.f90": {"pyi_dependencies": {"shared": "class shared:\n value: int"}}, + }, + output_dir=tmp_path, + ) ASSUMED_INTENT_SOURCE = """module legacy_mod @@ -1001,37 +447,31 @@ def fail_parse(_paths, _preprocessing): """ -def _generated_legacy_contract(tmp_path: Path, *extra_options: str) -> str: - source = tmp_path / f"legacy{len(extra_options)}.f90" - source.write_text(ASSUMED_INTENT_SOURCE, encoding="utf-8") - out = tmp_path / f"contracts{len(extra_options)}" - - cmd = [sys.executable, "-m", "prik", "generate", "--pyi", str(source), "--out", str(out), *extra_options] - subprocess.run(cmd, capture_output=True, text=True, check=True) - return (out / "legacy_mod.pyi").read_text(encoding="utf-8") +def test_assume_intent_in_scalars_reaches_the_generated_contract(tmp_path: Path, monkeypatch, capsys): + monkeypatch.chdir(tmp_path) + (tmp_path / "legacy.f90").write_text(ASSUMED_INTENT_SOURCE, encoding="utf-8") + assert _invoke(["generate", "--pyi", "legacy.f90", "--out", "default"], capsys)[0] == 0 + assert ( + _invoke(["generate", "--pyi", "legacy.f90", "--out", "assumed", "--assume-intent-in-scalars"], capsys)[0] == 0 + ) -def test_generated_contract_projects_undeclared_scalars_by_default(tmp_path: Path): - text = _generated_legacy_contract(tmp_path) - - assert 'Returns["count", Int32]' in text - assert 'Returns["factor", Float64]' in text - - -def test_assume_intent_in_scalars_removes_them_from_the_generated_contract(tmp_path: Path): - text = _generated_legacy_contract(tmp_path, "--assume-intent-in-scalars") - - assert "Returns" not in text - assert "-> Float64: ..." in text + default = (tmp_path / "default" / "legacy_mod.pyi").read_text(encoding="utf-8") + assumed = (tmp_path / "assumed" / "legacy_mod.pyi").read_text(encoding="utf-8") + assert 'Returns["count", Int32]' in default + assert 'Returns["factor", Float64]' in default + assert "Returns" not in assumed + assert "-> Float64: ..." in assumed -def test_fortran_parser_cli_pyi_is_the_contract_generate_writes(tmp_path: Path): +def test_fortran_parser_cli_pyi_is_the_contract_generate_writes(tmp_path: Path, monkeypatch, capsys): """The parser CLI shows the generated contract, not an unplanned rendering of its own. Its report converted and printed each module alone, so a module importing from another file lost the import completion plans and the spelling completion gives each name. """ + monkeypatch.chdir(tmp_path) helpers = tmp_path / "helpers.f90" helpers.write_text( "module helpers\ncontains\n" @@ -1046,15 +486,46 @@ def test_fortran_parser_cli_pyi_is_the_contract_generate_writes(tmp_path: Path): "end module user_mod\n", encoding="utf-8", ) - contracts = tmp_path / "contracts" - subprocess.run( - [sys.executable, "-m", "prik", "generate", "--pyi", str(helpers), str(user), "--out", str(contracts)], - check=True, - capture_output=True, - ) + assert _invoke(["generate", "--pyi", str(helpers), str(user), "--out", "contracts"], capsys)[0] == 0 report = fortran_parser_cli._semantic_report([str(helpers), str(user)]) + contracts = tmp_path / "contracts" assert report[str(helpers)]["pyi"] == (contracts / "helpers.pyi").read_text(encoding="utf-8").strip() assert report[str(user)]["pyi"] == (contracts / "user_mod.pyi").read_text(encoding="utf-8").strip() assert "from .helpers import lambda_" in report[str(user)]["pyi"] + + +@pytest.mark.parametrize("failure", ["rich-unavailable", "terminal-print-fails"]) +def test_pyi_terminal_highlighting_falls_back_to_plain_text(monkeypatch, capsys, failure): + """Syntax highlighting is optional; the contract text must still reach the terminal.""" + monkeypatch.setattr(sys.stdout, "isatty", lambda: True) + if failure == "rich-unavailable": + real_import = builtins.__import__ + + def fail_rich_import(name, *args, **kwargs): + if name.startswith("rich"): + raise ImportError("rich disabled for test") + return real_import(name, *args, **kwargs) + + monkeypatch.setattr(builtins, "__import__", fail_rich_import) + else: + + class RaisingConsole: + def __init__(self, **_options): + pass + + def print(self, _syntax): + raise RuntimeError("terminal failed") + + console_module = types.ModuleType("rich.console") + syntax_module = types.ModuleType("rich.syntax") + console_module.Console = RaisingConsole + syntax_module.Syntax = lambda code, *_args, **_options: code + monkeypatch.setitem(sys.modules, "rich", types.ModuleType("rich")) + monkeypatch.setitem(sys.modules, "rich.console", console_module) + monkeypatch.setitem(sys.modules, "rich.syntax", syntax_module) + + prik_cli.print_pyi_output("def f() -> None: ...") + + assert "def f() -> None: ..." in capsys.readouterr().out diff --git a/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py b/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py index 704d8d948..5f958122b 100644 --- a/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py +++ b/tests/fortran/infrastructure/cli/pipeline/test_stage_dispatch.py @@ -1,18 +1,14 @@ -"""Tests split by stable CLI stage-dispatch ownership.""" +"""CLI stage dispatch: report launchers, error presentation, probes, and path commands.""" import json -import os from pathlib import Path -import runpy import subprocess import sys -import types import pytest from prik.cmake import cmake_module_dir from prik.parsers.fortran import FortranParseError -from prik.pipeline.sources import fortran_sources_to_semantic_modules from prik.preprocessing import read_fortran_source import prik.cli as prik_cli from prik.parsers.fortran import cli as fortran_parser_cli @@ -21,121 +17,10 @@ PreprocessingDiagnostic, PreprocessingError, ) -from prik.semantics.fortran2ir import collect_semantic_compile_time_requirements -from tests.fortran._support.paths import GENERAL_FORTRAN_DIR -from tests.fortran.infrastructure.cli.pipeline._support import ( - TEST_FILE, - _install_main_parser, - _main_args, - _patch_main_report_payloads, -) NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" - -def test_cli_keeps_free_procedure_when_module_has_same_name(tmp_path: Path): - f90 = tmp_path / "same_name_scopes.f90" - f90.write_text( - """ -subroutine work(n) - integer, intent(in) :: n -end subroutine work - -module m -contains - subroutine work(n) - integer, intent(in) :: n - end subroutine work -end module m -""".strip() - ) - - cmd = [sys.executable, "-m", "prik", "parse", str(f90)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert " Procedures: 1" in res.stdout - assert " - subroutine work(n:integer[0])" in res.stdout - assert " Modules: 1" in res.stdout - assert " Procedures: 1" in res.stdout - - -def test_cli_debug_traceback_env_reraises_parse_errors(tmp_path: Path): - f90 = tmp_path / "bad.f90" - f90.write_text( - """subroutine bad(x) - weirdtype :: x -end subroutine bad -""", - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik", "parse", str(f90)] - res = subprocess.run( - cmd, - capture_output=True, - text=True, - env={**os.environ, "FORTRAN_PARSER_DEBUG": "1"}, - ) - - assert res.returncode == 1 - assert "Traceback" in res.stderr - assert "note: parser raised at" in res.stderr - - -def test_cli_no_color_env_disables_default_ansi(tmp_path: Path): - f90 = tmp_path / "bad.f90" - f90.write_text( - """subroutine bad(x) - weirdtype :: x -end subroutine bad -""", - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik", "parse", str(f90)] - res = subprocess.run( - cmd, - capture_output=True, - text=True, - env={**os.environ, "NO_COLOR": "1"}, - ) - - assert res.returncode == 1 - assert "\033[" not in res.stderr - assert f"{f90}:" in res.stderr - assert "error[PARSE_UNSUPPORTED_DECLARATION]:" in res.stderr - - -def test_fortran_parser_cli_reports_full_source_tree_from_source_file(tmp_path: Path): - f90 = tmp_path / "full_tree.f90" - f90.write_text( - (NATIVE_FIXTURES / "fortran_parser_cli_full_source_tree.f90").read_text(encoding="utf-8"), - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik.parsers.fortran", str(f90)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - assert f"File: {f90}" in res.stdout - assert "Modules: 1" in res.stdout - assert "- module parent_mod (vars=1, uses=0)" in res.stdout - assert "Derived types: 1" in res.stdout - assert "- type particle (fields=2, methods=1)" in res.stdout - assert "Fields: 2" in res.stdout - assert "- x:real(8)[1]" in res.stdout - assert "Submodules: 1" in res.stdout - assert "- submodule child_mod (parent=parent_mod, vars=0, uses=0)" in res.stdout - assert "Programs: 1" in res.stdout - assert "- program driver (vars=1, uses=1)" in res.stdout - assert "Block data: 1" in res.stdout - assert "- block data init_block (vars=1)" in res.stdout - - -def test_fortran_parser_cli_semantics_pyi_and_empty_module_report_from_inline_code(tmp_path: Path): - module_source = tmp_path / "prik.semantics.f90" - module_source.write_text( - """ -module solver_mod +SOLVER_SOURCE = """module solver_mod contains subroutine solve(a, x, b) real(8), intent(in) :: a @@ -143,683 +28,259 @@ def test_fortran_parser_cli_semantics_pyi_and_empty_module_report_from_inline_co real(8), intent(in) :: b end subroutine solve end module solver_mod -""", - encoding="utf-8", - ) - program_source = tmp_path / "driver.f90" - program_source.write_text( - """ -program driver - integer :: n -end program driver -""", - encoding="utf-8", - ) - json_out = tmp_path / "prik.semantics.json" - - semantics_cmd = [ - sys.executable, - "-m", - "prik.parsers.fortran", - str(module_source), - "--semantics", - "--json-out", - str(json_out), - ] - semantics_res = subprocess.run(semantics_cmd, capture_output=True, text=True, check=True) - payload = json.loads(json_out.read_text(encoding="utf-8")) - - assert "solver_mod" in semantics_res.stdout - assert str(module_source) in payload - assert payload[str(module_source)]["semantic_modules"][0]["functions"][0]["name"] == "solve" - - pyi_cmd = [sys.executable, "-m", "prik.parsers.fortran", str(module_source), "--pyi"] - pyi_res = subprocess.run(pyi_cmd, capture_output=True, text=True, check=True) - assert "@native_call([Addr(Arg(0)), Return('x', 0), Addr(Arg(1))])" in pyi_res.stdout - assert "x: Addr(Float64)" not in pyi_res.stdout - assert "def solve(" in pyi_res.stdout - - empty_pyi_cmd = [sys.executable, "-m", "prik.parsers.fortran", str(program_source), "--pyi"] - empty_pyi_res = subprocess.run(empty_pyi_cmd, capture_output=True, text=True, check=True) - assert "" in empty_pyi_res.stdout - - -def test_prik_semantics_marks_explicit_cross_file_derived_type_as_wrapped(tmp_path: Path): - types_mod = tmp_path / "types_mod.f90" - physics = tmp_path / "physics.f90" - types_mod.write_text( - """ -module types_mod - type :: particle - real :: mass - end type particle -end module types_mod -""", - encoding="utf-8", - ) - physics.write_text( - """ -module physics - use types_mod, only: particle -contains - subroutine move(p) - type(particle), intent(inout) :: p - end subroutine move -end module physics -""", - encoding="utf-8", - ) +""" - payload = prik_cli._semantic_report([str(types_mod), str(physics)]) - semantic_type = payload[str(physics)]["semantic_modules"][0]["functions"][0]["arguments"][0]["semantic_type"] +BAD_SOURCE = """subroutine bad(x) + weirdtype :: x +end subroutine bad +""" - assert semantic_type["metadata"]["external_type_ref"]["wrapped"] is True - assert "class particle" not in payload[str(physics)]["pyi"] +def _parser_module_main(monkeypatch, *argv: str) -> int: + monkeypatch.setattr(sys, "argv", ["prik.parsers.fortran", *argv]) + return fortran_parser_cli.main() -def test_single_file_cli_resolves_direct_intrinsic_kind_rename_before_probing(tmp_path: Path): - source = tmp_path / "direct_intrinsic_kind.f90" - source.write_text( - """ -module direct_intrinsic_kind - use iso_fortran_env, only: wp => real64 - real(wp), parameter :: scale = 2.0_wp -contains - real(wp) function twice(value) result(output) - real(wp), intent(in) :: value - output = scale*value - end function twice -end module direct_intrinsic_kind -""", - encoding="utf-8", - ) - project = fortran_sources_to_semantic_modules([source], PreprocessingConfig()).project - parsed = project.files[0] - module = parsed.modules[0] - - assert module.variables[0].kind == "real64" - assert module.procedures[0].arguments[0].kind == "real64" - assert module.procedures[0].result.kind == "real64" - assert collect_semantic_compile_time_requirements(parsed) == [] - - -def test_cli_cross_file_resolution_reaches_imported_derived_field_kinds(tmp_path: Path): - precision = tmp_path / "precision.f90" - records = tmp_path / "records.f90" - precision.write_text( - """ -module precision - integer, parameter :: rk = 8 -end module precision -""", - encoding="utf-8", - ) - records.write_text( - """ -module records - use precision, only: wp => rk - type :: sample - real(kind=wp) :: value - end type sample -end module records -""", - encoding="utf-8", +def test_fortran_parser_module_launcher_reports_every_mode(tmp_path: Path, monkeypatch, capsys): + """``python -m prik.parsers.fortran`` prints the parse report and each explicit inspection mode.""" + full_tree = NATIVE_FIXTURES / "fortran_parser_cli_full_source_tree.f90" + launched = subprocess.run( + [sys.executable, "-m", "prik.parsers.fortran", str(full_tree)], + capture_output=True, + text=True, + check=True, ) + for line in ( + f"File: {full_tree}", + "- module parent_mod (vars=1, uses=0)", + "- type particle (fields=2, methods=1)", + "- x:real(8)[1]", + "- submodule child_mod (parent=parent_mod, vars=0, uses=0)", + "- program driver (vars=1, uses=1)", + "- block data init_block (vars=1)", + ): + assert line in launched.stdout - project = fortran_sources_to_semantic_modules([precision, records], PreprocessingConfig()).project - record_file = next(parsed for parsed in project.files if parsed.filename == str(records)) - - assert record_file.modules[0].derived_types[0].fields[0].kind == "8" - + solver = tmp_path / "solver.f90" + solver.write_text(SOLVER_SOURCE, encoding="utf-8") + driver = tmp_path / "driver.f90" + driver.write_text("program driver\n integer :: n\nend program driver\n", encoding="utf-8") -def test_prik_pyi_report_writes_opaque_dependency_stub_for_external_type(tmp_path: Path, monkeypatch): - physics = tmp_path / "physics.f90" - physics.write_text( - """ -module physics - use types_mod, only: particle -contains - function create_particle() result(p) - type(particle) :: p - end function create_particle -end module physics -""", - encoding="utf-8", - ) + parse_json = tmp_path / "parse.json" + assert _parser_module_main(monkeypatch, str(solver), "--json", "--json-out", str(parse_json)) == 0 + assert str(solver) in json.loads(capsys.readouterr().out) + assert str(solver) in json.loads(parse_json.read_text(encoding="utf-8")) - payload = prik_cli._semantic_report([str(physics)]) + semantics_json = tmp_path / "semantics.json" + assert _parser_module_main(monkeypatch, str(solver), "--semantics", "--json-out", str(semantics_json)) == 0 + assert "solver_mod" in capsys.readouterr().out + payload = json.loads(semantics_json.read_text(encoding="utf-8")) + assert payload[str(solver)]["semantic_modules"][0]["functions"][0]["name"] == "solve" - assert payload[str(physics)]["pyi_dependencies"] == { - "types_mod": 'from prik.contracts import Opaque\n\nclass particle(Opaque):\n pass\n\n__all__ = ["particle"]' - } - monkeypatch.setattr(sys, "argv", ["prik", "generate", "--pyi", str(physics), "--out"]) - assert prik_cli.main() == 0 + assert _parser_module_main(monkeypatch, str(solver), "--pyi") == 0 + contract = capsys.readouterr().out + assert "@native_call([Addr(Arg(0)), Return('x', 0), Addr(Arg(1))])" in contract + assert "def solve(" in contract + assert "x: Addr(Float64)" not in contract - package = tmp_path / "physics" - assert (package / "__init__.pyi").read_text(encoding="utf-8") == ( - 'from . import physics\n\n__all__ = ["physics"]\n' - ) - assert (package / "types_mod.pyi").read_text( - encoding="utf-8" - ) == 'from prik.contracts import Opaque\n\nclass particle(Opaque):\n pass\n\n__all__ = ["particle"]\n' + assert _parser_module_main(monkeypatch, str(driver), "--pyi") == 0 + assert "" in capsys.readouterr().out @pytest.mark.parametrize( - ("overrides", "expected_stage_calls"), + ("entrypoint", "options", "environment", "concise"), [ - ({"parse": True}, [("parse",)]), - ({"semantics": True}, [("semantic",)]), - ({"pyi": True}, [("semantic",)]), + pytest.param("prik", [], {}, ("\033[", "error"), id="prik-colored-by-default"), + pytest.param( + "prik", + [], + {"NO_COLOR": "1"}, + ("bad.f90:", "error[PARSE_UNSUPPORTED_DECLARATION]:"), + id="prik-no-color-environment", + ), + pytest.param("prik", [], {"FORTRAN_PARSER_DEBUG": "1"}, None, id="prik-debug-environment"), + pytest.param( + "parser-module", + ["--no-color"], + {}, + ("bad.f90:2:1: error[PARSE_UNSUPPORTED_DECLARATION]:", "Unknown or unsupported datatype"), + id="parser-module-concise", + ), + pytest.param("parser-module", ["--debug"], {}, None, id="parser-module-debug-flag"), + pytest.param("parser-module", [], {"FORTRAN_PARSER_DEBUG": "1"}, None, id="parser-module-debug-environment"), ], ) -def test_prik_main_preserves_fortran_stage_dispatch_contract(monkeypatch, overrides, expected_stage_calls): - class StopAfterDispatch(Exception): - pass - - args = _main_args(**overrides) - parser = _install_main_parser(monkeypatch, args) - preprocessing = object() - parse_payload = {"parse": "payload"} - semantic_payload = {"semantic": "payload"} - calls = [] - - def resolve_language(paths, language, active_parser): - calls.append(("resolve", paths, language, active_parser)) - return "fortran" - - def build_preprocessing_config(active_args, active_parser): - calls.append(("config", active_args, active_parser)) - return preprocessing - - def parse_report(paths, active_preprocessing): - calls.append(("parse", paths, active_preprocessing)) - return parse_payload - - def semantic_report(paths, active_preprocessing, *, language): - calls.append(("semantic", paths, active_preprocessing, language)) - return semantic_payload - - def select_main_payload(*_args): - raise StopAfterDispatch - - monkeypatch.setattr(prik_cli, "_resolve_language", resolve_language) - monkeypatch.setattr(prik_cli, "_build_preprocessing_config", build_preprocessing_config) - monkeypatch.setattr(prik_cli, "_parse_report", parse_report) - monkeypatch.setattr(prik_cli, "_semantic_report", semantic_report) - monkeypatch.setattr(prik_cli, "_select_main_payload", select_main_payload) - - with pytest.raises(StopAfterDispatch): - prik_cli.main() - - expected_calls = [ - ("resolve", args.paths, "fortran", parser), - ("config", args, parser), - ] - for (stage_name,) in expected_stage_calls: - if stage_name == "parse": - expected_calls.append(("parse", args.paths, preprocessing)) - elif stage_name == "semantic": - expected_calls.append(("semantic", args.paths, preprocessing, "fortran")) - assert calls == expected_calls - - -def test_prik_main_runs_default_wrapper_build(monkeypatch, tmp_path: Path, capsys): - source = tmp_path / "fmath.f" - source.write_text(" real function square(x)\n real x\n square = x*x\n end\n", encoding="utf-8") - args = _main_args(paths=[str(source)], out_dir=str(tmp_path), json=True) - _install_main_parser(monkeypatch, args) - preprocessing = object() - calls = [] - result = types.SimpleNamespace( - to_dict=lambda: { - "source": str(source), - "module_name": "fmath", - "shared_library": str(tmp_path / "fmath.so"), - "generated_sources": [str(tmp_path / "fmath_wrapper.c")], - } - ) +def test_parse_errors_are_concise_unless_debugging( + tmp_path: Path, monkeypatch, capsys, entrypoint, options, environment, concise +): + monkeypatch.chdir(tmp_path) + (tmp_path / "bad.f90").write_text(BAD_SOURCE, encoding="utf-8") + for name in ("NO_COLOR", "FORTRAN_PARSER_DEBUG", "PRIK_DEBUG"): + monkeypatch.delenv(name, raising=False) + for name, value in environment.items(): + monkeypatch.setenv(name, value) - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: "fortran") - monkeypatch.setattr(prik_cli, "_build_preprocessing_config", lambda active_args, parser: preprocessing) - monkeypatch.setattr( - prik_cli, - "_run_wrap_build_with_diagnostics", - lambda active_args, active_preprocessing: calls.append((active_args, active_preprocessing)) or result, - ) + def run() -> int: + if entrypoint == "prik": + return prik_cli.main(["parse", "bad.f90", *options]) + return _parser_module_main(monkeypatch, "bad.f90", *options) - assert prik_cli.main() == 0 + if concise is None: + with pytest.raises(FortranParseError) as raised: + run() + if "FORTRAN_PARSER_DEBUG" in environment: + assert "note: parser raised at" in str(raised.value) + return - assert calls == [(args, preprocessing)] - payload = json.loads(capsys.readouterr().out) - assert payload["module_name"] == "fmath" + assert run() == 1 + captured = capsys.readouterr() + assert captured.out == "" + assert [fragment for fragment in concise if fragment not in captured.err] == [] + assert ("\033[" in captured.err) is ("NO_COLOR" not in environment and "--no-color" not in options) -def test_cli_native_libraries_split_grouped_prefixed_names(): - assert prik_cli._cli_native_libraries(["blas", "-llapack -lscalapack"]) == ( - "blas", - "-llapack", - "-lscalapack", +def _located_preprocessing_error(): + return PreprocessingError( + "compiler failed", + category="PREPROCESSOR_FAILED", + diagnostics=[ + PreprocessingDiagnostic(category="PREPROCESSOR_FAILED", message="bad include", path="source.F90", line=9) + ], ) -def test_prik_main_preserves_pathless_preprocessing_diagnostic_contract(monkeypatch, capsys): - args = _main_args(parse=True) - _install_main_parser(monkeypatch, args) - calls = [] - - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: language) - monkeypatch.setattr(prik_cli, "_build_preprocessing_config", lambda active_args, parser: object()) - monkeypatch.setattr(prik_cli, "_env_flag", lambda name: calls.append(name) or False) - monkeypatch.setattr( - prik_cli, - "_parse_report", - lambda paths, preprocessing: (_ for _ in ()).throw( - PreprocessingError( - "compiler failed", - diagnostics=[PreprocessingDiagnostic(category="PREPROCESSOR_FAILED", message="bad include")], - ) - ), - ) - - assert prik_cli.main() == 1 - assert capsys.readouterr().err == ": error[PREPROCESSOR_FAILED]: bad include\n" - assert calls == ["PRIK_DEBUG"] - - -def test_prik_main_reraises_value_errors_for_debug_environment(monkeypatch): - args = _main_args(parse=True) - _install_main_parser(monkeypatch, args) - calls = [] - - monkeypatch.setattr(prik_cli, "_resolve_language", lambda paths, language, parser: language) - monkeypatch.setattr(prik_cli, "_build_preprocessing_config", lambda active_args, parser: object()) - monkeypatch.setattr(prik_cli, "_env_flag", lambda name: calls.append(name) or name == "PRIK_DEBUG") - monkeypatch.setattr( - prik_cli, - "_parse_report", - lambda paths, preprocessing: (_ for _ in ()).throw(ValueError("invalid generated interface")), +def _pathless_preprocessing_error(): + return PreprocessingError( + "compiler failed", + diagnostics=[PreprocessingDiagnostic(category="PREPROCESSOR_FAILED", message="bad include")], ) - with pytest.raises(ValueError, match="invalid generated interface"): - prik_cli.main() - assert calls == ["PRIK_DEBUG"] +def _plain_preprocessing_error(): + return PreprocessingError("plain failure", category="PREPROCESSOR_FAILED") -def test_prik_main_preserves_explicit_pyi_write_contract(monkeypatch): - semantic_payload = { - "first.f90": {"pyi": "def first() -> None: ..."}, - "empty.f90": {}, - "second.f90": {"pyi": "def second() -> None: ..."}, - } - args = _main_args(pyi=True, out="/tmp/api.pyi") - _install_main_parser(monkeypatch, args) - _patch_main_report_payloads(monkeypatch, semantic_payload=semantic_payload) - writes = [] - dependencies = [] +def _value_error(): + return ValueError("invalid generated interface") - monkeypatch.setattr( - Path, - "write_text", - lambda path, data, **kwargs: writes.append((path, data, kwargs)) or len(data), - ) - monkeypatch.setattr( - prik_cli, - "_write_pyi_dependencies", - lambda payload, **kwargs: dependencies.append((payload, kwargs)), - ) - - assert prik_cli.main() == 0 - assert writes == [ - (Path("/tmp/api.pyi"), "def first() -> None: ...\n\n\n\ndef second() -> None: ...\n", {"encoding": "utf-8"}) - ] - assert dependencies == [(semantic_payload, {"output_dir": Path("/tmp")})] - - -def test_prik_main_preserves_adjacent_pyi_write_contract(monkeypatch): - semantic_payload = { - "/tmp/first.f90": { - "pyi": "def first() -> None: ...", - "pyi_modules": {"first_mod": "def first() -> None: ..."}, - }, - "/tmp/empty.f90": {"pyi_modules": {}}, - } - args = _main_args(pyi=True, out="") - _install_main_parser(monkeypatch, args) - _patch_main_report_payloads(monkeypatch, semantic_payload=semantic_payload) - writes = [] - dependencies = [] - - monkeypatch.setattr( - Path, - "write_text", - lambda path, data, **kwargs: writes.append((path, data, kwargs)) or len(data), - ) - monkeypatch.setattr( - prik_cli, - "_write_pyi_dependencies", - lambda payload, **kwargs: dependencies.append((payload, kwargs)), - ) - - assert prik_cli.main() == 0 - assert writes == [ - (Path("/tmp/first_mod.pyi"), "def first() -> None: ...\n", {"encoding": "utf-8"}), - ] - assert dependencies == [(semantic_payload, {})] - - -def test_prik_and_fortran_module_entrypoints_and_debug_errors(monkeypatch, capsys): - original_fortran_main = fortran_parser_cli.main - monkeypatch.setattr(prik_cli, "main", lambda: 0) - with pytest.raises(SystemExit) as prik_exit: - runpy.run_module("prik.__main__", run_name="__main__") - assert prik_exit.value.code == 0 - - monkeypatch.setattr(fortran_parser_cli, "main", lambda: 0) - with pytest.raises(SystemExit) as fortran_exit: - runpy.run_module("prik.parsers.fortran.__main__", run_name="__main__") - assert fortran_exit.value.code == 0 - monkeypatch.setattr(fortran_parser_cli, "main", original_fortran_main) - - def fail_parse(_paths): - raise FortranParseError("bad", filename="bad.f90", line_number=1, source_line="bad") - - monkeypatch.setattr(fortran_parser_cli, "_parse_paths", fail_parse) - monkeypatch.setattr(sys, "argv", ["prik.parsers.fortran", "bad.f90", "--no-color"]) - assert fortran_parser_cli.main() == 1 - assert "bad.f90:1:1: error[PARSE_ERROR]: bad" in capsys.readouterr().err - monkeypatch.setenv("FORTRAN_PARSER_DEBUG", "1") - with pytest.raises(FortranParseError): - fortran_parser_cli.main() - - -def test_prik_main_debug_reraises_preprocessing_errors(monkeypatch): - monkeypatch.setattr(sys, "argv", ["prik", "parse", str(TEST_FILE)]) - monkeypatch.setenv("PRIK_DEBUG", "1") - - def fail_parse(_paths, _preprocessing): - raise PreprocessingError("plain failure", category="PREPROCESSOR_FAILED") - - monkeypatch.setattr(prik_cli, "_parse_report", fail_parse) - with pytest.raises(PreprocessingError): - prik_cli.main() - - -def test_cli_parse_modern_fixture_prints_derived_block_verbatim(): - fixture = GENERAL_FORTRAN_DIR / "modern_pyi_example.f90" - cmd = [sys.executable, "-m", "prik", "parse", str(fixture)] - res = subprocess.run(cmd, capture_output=True, text=True, check=True) - - expected_block = """ Derived types: 3 - - type particle (fields=3, methods=0) - Fields: 3 - - id:integer[0] - - mass:real(8)[0] - - position:real(8)[1] - - type vector3 (fields=1, methods=0) - Fields: 1 - - values:real(8)[1] - - type hidden_state (fields=1, methods=0) - Fields: 1 - - code:integer[0] -""" - assert expected_block in res.stdout +@pytest.mark.parametrize( + ("error", "options", "environment", "stderr"), + [ + pytest.param( + _located_preprocessing_error, + [], + {}, + "source.F90:9: error[PREPROCESSOR_FAILED]: bad include\n", + id="located-preprocessing-diagnostic", + ), + pytest.param( + _pathless_preprocessing_error, + [], + {}, + ": error[PREPROCESSOR_FAILED]: bad include\n", + id="pathless-preprocessing-diagnostic", + ), + pytest.param( + _plain_preprocessing_error, + [], + {}, + "prik: error[PREPROCESSOR_FAILED]: plain failure\n", + id="preprocessing-error-without-diagnostics", + ), + pytest.param(_value_error, [], {}, "prik: error: invalid generated interface\n", id="value-error"), + pytest.param(_plain_preprocessing_error, [], {"PRIK_DEBUG": "1"}, None, id="preprocessing-debug-environment"), + pytest.param(_value_error, ["--debug"], {}, None, id="value-error-debug-flag"), + pytest.param(_value_error, [], {"PRIK_DEBUG": "1"}, None, id="value-error-debug-environment"), + ], +) +def test_stage_errors_are_reported_concisely_unless_debugging( + tmp_path: Path, monkeypatch, capsys, error, options, environment, stderr +): + source = tmp_path / "input.f90" + source.write_text("module input\nend module input\n", encoding="utf-8") + monkeypatch.delenv("PRIK_DEBUG", raising=False) + for name, value in environment.items(): + monkeypatch.setenv(name, value) + raised = error() -def test_fortran_parser_cli_debug_flag_reraises_parse_errors(tmp_path: Path): - f90 = tmp_path / "bad.f90" - f90.write_text( - """subroutine bad(x) - weirdtype :: x -end subroutine bad -""", - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik.parsers.fortran", str(f90), "--debug"] - res = subprocess.run(cmd, capture_output=True, text=True) - - assert res.returncode == 1 - assert "Traceback" in res.stderr - assert "FortranParseError" in res.stderr - - -def test_fortran_parser_cli_debug_traceback_env_reraises_parse_errors(tmp_path: Path): - f90 = tmp_path / "bad.f90" - f90.write_text( - """subroutine bad(x) - weirdtype :: x -end subroutine bad -""", - encoding="utf-8", - ) - - cmd = [sys.executable, "-m", "prik.parsers.fortran", str(f90)] - res = subprocess.run( - cmd, - capture_output=True, - text=True, - env={**os.environ, "FORTRAN_PARSER_DEBUG": "1"}, - ) - - assert res.returncode == 1 - assert "Traceback" in res.stderr - assert "note: parser raised at" in res.stderr - - -def test_fortran_parser_main_public_api_modes_from_inline_source(tmp_path: Path, monkeypatch, capsys): - f90 = tmp_path / "mini.f90" - f90.write_text( - """module m -contains - subroutine work(n) - integer, intent(in) :: n - end subroutine work -end module m -""", - encoding="utf-8", - ) - json_out = tmp_path / "report.json" - - monkeypatch.setattr(sys, "argv", ["prik.parsers.fortran", str(f90), "--json-out", str(json_out), "--json"]) - assert fortran_parser_cli.main() == 0 - stdout_payload = json.loads(capsys.readouterr().out) - assert str(f90) in stdout_payload - assert json_out.exists() - - monkeypatch.setattr(sys, "argv", ["prik.parsers.fortran", str(f90), "--pyi"]) - assert fortran_parser_cli.main() == 0 - pyi_out = capsys.readouterr().out - assert "File:" in pyi_out - assert "def work(" in pyi_out - - monkeypatch.setattr(sys, "argv", ["prik.parsers.fortran", str(f90)]) - assert fortran_parser_cli.main() == 0 - readable = capsys.readouterr().out - assert "module m" in readable - - -def test_prik_main_public_api_modes_from_inline_source(tmp_path: Path, monkeypatch, capsys): - f90 = tmp_path / "mini.f90" - f90.write_text( - """module m -contains - subroutine work(n) - integer, intent(in) :: n - end subroutine work -end module m -""", - encoding="utf-8", - ) - json_out = tmp_path / "parse.json" - - monkeypatch.setattr(sys, "argv", ["prik", "parse", str(f90), "--json", "--out", str(json_out)]) - assert prik_cli.main() == 0 - assert capsys.readouterr().out == "" - assert json.loads(json_out.read_text(encoding="utf-8")).get(str(f90)) is not None + def fail(_paths, _preprocessing): + raise raised - monkeypatch.setattr(sys, "argv", ["prik", "generate", "--pyi", str(f90)]) - assert prik_cli.main() == 0 - assert "def work(" in capsys.readouterr().out + monkeypatch.setattr(prik_cli, "_parse_report", fail) - monkeypatch.setattr(sys, "argv", ["prik", "parse", str(f90)]) - assert prik_cli.main() == 0 - assert "module m" in capsys.readouterr().out + if stderr is None: + with pytest.raises(type(raised)): + prik_cli.main(["parse", str(source), *options]) + return + assert prik_cli.main(["parse", str(source), *options]) == 1 + assert capsys.readouterr().err == stderr -def test_a_fortran_source_read_as_written_carries_its_internal_recipe(tmp_path: Path): - """Without compiler preprocessing the text is read as UTF-8 and the macros are recorded.""" - path = tmp_path / "raw.f90" - path.write_text("subroutine raw()\n ! \u00e9\nend subroutine raw\n", encoding="utf-8") - text = read_fortran_source(path, PreprocessingConfig(defines=["FLAG=1"])) - - assert text.source == path.read_text(encoding="utf-8") - assert text.recipe is not None and text.recipe["mode"] == "internal" - assert text.included_files == () +MEASURED_MAPPING = {"report": "type_mapping", "target_profile": "t", "types": []} -def test_prik_probe_subcommand_dispatches_one_flag_driven_probe(monkeypatch, capsys): - calls = [] +@pytest.mark.parametrize( + ("argv", "expected"), + [ + pytest.param( + ["--language", "c", "--compiler", "cc", "--json"], + json.dumps({**MEASURED_MAPPING, "language": "c"}, indent=2), + id="c-mapping-json", + ), + pytest.param( + ["--language", "fortran", "--compiler", "gfortran", "--json"], + json.dumps({**MEASURED_MAPPING, "language": "fortran"}, indent=2), + id="fortran-mapping-json", + ), + pytest.param(["--language", "fortran", "--compiler", "gfortran"], "MD:fortran", id="mapping-markdown"), + pytest.param( + ["--language", "fortran", "--compiler", "gfortran-13", "--expr", "storage_size(0)"], + "EXPR:gfortran-13:storage_size(0)", + id="expressions-markdown", + ), + ], +) +def test_probe_selects_its_report_by_expressions_and_its_rendering_by_json(monkeypatch, capsys, argv, expected): + """--expr selects the measured report; --json only selects how it is rendered.""" + monkeypatch.setattr(prik_cli, "c_type_mapping_report", lambda **options: {**MEASURED_MAPPING, "language": "c"}) monkeypatch.setattr( - prik_cli, - "_probe_output", - lambda args: calls.append(args) or '{"target": "fortran"}', - ) - - assert ( - prik_cli.main( - [ - "probe", - "--language", - "fortran", - "--compiler", - "gfortran-13", - "--expr", - "storage_size(0)", - ] - ) - == 0 + prik_cli, "fortran_type_mapping_report", lambda **options: {**MEASURED_MAPPING, "language": "fortran"} ) - - assert capsys.readouterr().out == '{"target": "fortran"}\n' - assert len(calls) == 1 - assert calls[0].language == "fortran" - assert calls[0].compiler == "gfortran-13" - assert calls[0].expressions == ["storage_size(0)"] - - -def _probe_args(**overrides): - defaults = { - "language": "fortran", - "compiler": "gfortran", - "json": False, - "expressions": [], - "include_dirs": [], - "defines": [], - "undefs": [], - "std": None, - "compiler_args": [], - "runner": [], - "cache_dir": None, - "refresh": False, - } - return types.SimpleNamespace(**{**defaults, **overrides}) - - -@pytest.mark.parametrize("language", ["c", "fortran"]) -def test_probe_without_expressions_reports_the_measured_type_mapping(monkeypatch, language): - """Omitting --expr selects the mapping report rather than an empty measurement.""" - measured = {"report": "type_mapping", "language": language, "target_profile": "t", "types": []} - monkeypatch.setattr(prik_cli, "c_type_mapping_report", lambda **options: measured) - monkeypatch.setattr(prik_cli, "fortran_type_mapping_report", lambda **options: measured) - - assert json.loads(prik_cli._probe_output(_probe_args(language=language, json=True))) == measured - - -@pytest.mark.parametrize("as_json", [False, True]) -def test_probe_renders_each_report_in_both_formats(monkeypatch, as_json): - """--json selects a rendering; it must not select a different report.""" - measured = {"report": "type_mapping", "language": "fortran", "target_profile": "t", "types": []} - monkeypatch.setattr(prik_cli, "fortran_type_mapping_report", lambda **options: measured) monkeypatch.setattr(prik_cli, "type_mapping_markdown", lambda report: f"MD:{report['language']}") - - output = prik_cli._probe_output(_probe_args(json=as_json)) - - assert output == (json.dumps(measured, indent=2) if as_json else "MD:fortran") - - -def test_probe_expressions_render_as_markdown(monkeypatch): - """--expr is a report selector, so its table is the default human rendering.""" - measured = object() - monkeypatch.setattr(prik_cli, "probe_fortran_type_expressions_cached", lambda *args, **options: measured) - monkeypatch.setattr(prik_cli, "expression_probe_markdown", lambda report: "EXPR-TABLE") - - output = prik_cli._probe_output(_probe_args(expressions=["kind(1.0d0)"])) - - assert output == "EXPR-TABLE" - - -@pytest.mark.parametrize( - "option", [{"include_dirs": ["inc"]}, {"defines": ["A=1"]}, {"undefs": ["A"]}, {"std": "f2018"}] -) -def test_probe_mapping_report_rejects_preprocessing_options(option): - """The mapping inventory is fixed, so preprocessing options cannot affect it.""" - with pytest.raises(ValueError, match="add --expr to probe preprocessed expressions"): - prik_cli._probe_output(_probe_args(**option)) - - -def test_probe_expressions_are_fortran_only(): - with pytest.raises(ValueError, match="--expr is supported only for --language fortran"): - prik_cli._probe_output(_probe_args(language="c", expressions=["kind(1.0)"])) - - -def _cli_path(command: str) -> Path: - """Return the single path one path-printing command prints.""" - printed = subprocess.run( - [sys.executable, "-m", "prik", command], - capture_output=True, - text=True, - check=True, + monkeypatch.setattr( + prik_cli, + "probe_fortran_type_expressions_cached", + lambda config, expressions, **options: (config.compiler, expressions), ) - return Path(printed.stdout.strip()) + monkeypatch.setattr(prik_cli, "expression_probe_markdown", lambda report: f"EXPR:{report[0]}:{','.join(report[1])}") + assert prik_cli.main(["probe", *argv]) == 0 + assert capsys.readouterr().out == expected + "\n" -def test_cmake_dir_prints_the_packaged_cmake_module_directory(): - """A CMake build substitutes this into PRIK_DIR or CMAKE_MODULE_PATH.""" - module_dir = _cli_path("cmake-dir") +def test_path_commands_report_the_packaged_cmake_module(monkeypatch, capsys): + """``cmake-dir``, ``install-dir``, and ``doctor cmake`` print the facts a CMake build would use.""" + assert prik_cli.main(["cmake-dir"]) == 0 + module_dir = Path(capsys.readouterr().out.strip()) assert module_dir == cmake_module_dir() assert (module_dir / "UsePRIK.cmake").is_file() assert (module_dir / "PRIKConfig.cmake").is_file() - -def test_install_dir_prints_a_prefix_that_holds_installed_data_or_reports_none(): - """The prefix is only useful when PRIK's data is under it, so nothing else is printed.""" - printed = subprocess.run( - [sys.executable, "-m", "prik", "install-dir"], - capture_output=True, - text=True, - ) - - if printed.returncode == 0: - assert (Path(printed.stdout.strip()) / "share" / "prik").is_dir() + try: + code = prik_cli.main(["install-dir"]) + except SystemExit as exc: + code = exc.code + printed = capsys.readouterr() + if code == 0: + assert (Path(printed.out.strip()) / "share" / "prik").is_dir() else: - assert "prik" in printed.stderr.lower() - assert not printed.stdout.strip() - - -def test_doctor_cmake_reports_the_discovery_facts_a_build_would_use(): - """The report is observed facts, so a confusing environment shows up in it.""" - printed = subprocess.run( - [sys.executable, "-m", "prik", "doctor", "cmake"], - capture_output=True, - text=True, - check=True, - ) - report = dict(line.split(": ", 1) for line in printed.stdout.splitlines()) + assert "prik" in printed.err.lower() + assert not printed.out.strip() + assert prik_cli.main(["doctor", "cmake"]) == 0 + report = dict(line.split(": ", 1) for line in capsys.readouterr().out.splitlines()) assert report["cmake-dir"] == str(cmake_module_dir()) assert report["imported package"] == str(cmake_module_dir().parent) assert report["python executable"] == sys.executable @@ -846,3 +307,15 @@ def test_parse_reports_resolve_kinds_one_input_file_declares_for_another(tmp_pat ): procedure = report[str(user)]["modules"][0]["procedures"][0] assert procedure["arguments"][0]["kind"] == "8" + + +def test_a_fortran_source_read_as_written_carries_its_internal_recipe(tmp_path: Path): + """Without compiler preprocessing the text is read as UTF-8 and the macros are recorded.""" + path = tmp_path / "raw.f90" + path.write_text("subroutine raw()\n ! é\nend subroutine raw\n", encoding="utf-8") + + text = read_fortran_source(path, PreprocessingConfig(defines=["FLAG=1"])) + + assert text.source == path.read_text(encoding="utf-8") + assert text.recipe is not None and text.recipe["mode"] == "internal" + assert text.included_files == () diff --git a/tests/fortran/infrastructure/codegen/test_binding.py b/tests/fortran/infrastructure/codegen/test_binding.py deleted file mode 100644 index 3a46ee2ac..000000000 --- a/tests/fortran/infrastructure/codegen/test_binding.py +++ /dev/null @@ -1,23 +0,0 @@ -"""Internal CPython binding lowering contracts.""" - -from __future__ import annotations - - -import pytest - -from prik.codegen import ( - BackendScalarType, -) -from prik.codegen.c.binding import CBindingGenerator - - -def test_c_binding_rejects_an_unprefixed_numpy_scalar_macro(): - scalar = BackendScalarType( - semantic_name="Invalid", - c_spelling="double", - fortran_spelling="real(c_double)", - numpy_type_macro="FLOAT64", - ) - - with pytest.raises(ValueError, match="Unsupported NumPy scalar type macro 'FLOAT64'"): - CBindingGenerator()._scalar_helper_suffix(scalar) diff --git a/tests/fortran/infrastructure/codegen/test_checks.py b/tests/fortran/infrastructure/codegen/test_checks.py index 5ac8d21bb..2094aa33b 100644 --- a/tests/fortran/infrastructure/codegen/test_checks.py +++ b/tests/fortran/infrastructure/codegen/test_checks.py @@ -4,44 +4,14 @@ from pathlib import Path +import pytest + from prik.codegen.checks import ( WrapperCodegenCheckConfig, check_codegen_paths, ) - -def _write_module(root: Path, relative_path: str, source: str) -> Path: - path = root / relative_path - path.parent.mkdir(parents=True, exist_ok=True) - path.write_text(source, encoding="utf-8") - return path - - -def _check_source(tmp_path: Path, source: str, *, filename: str = "bad.py") -> set[str]: - path = _write_module(tmp_path, filename, source) - violations = check_codegen_paths( - [path], - config=WrapperCodegenCheckConfig(max_complexity=3, max_statements=4, max_nesting=2), - ) - return {violation.code for violation in violations} - - -def test_reviewer_reports_module_level_production_functions(tmp_path: Path): - codes = _check_source(tmp_path, "def build_plan():\n return None\n") - - assert "module-function" in codes - - -def test_reviewer_recommends_visitor_based_production_classes(tmp_path: Path): - codes = _check_source(tmp_path, "class WrapperPlanner:\n pass\n") - - assert "visitor-class" in codes - - -def test_reviewer_reports_complexity_statement_and_nesting_limits(tmp_path: Path): - codes = _check_source( - tmp_path, - """ +_OVERSIZED_FUNCTION = """ def oversized(value): first = value + 1 second = first + 1 @@ -58,17 +28,29 @@ def oversized(value): if value == 3: return third return fourth -""", - ) +""" - assert {"complexity", "statement-count", "nesting-depth"} <= codes +_EMITTER_WITH_MISSING_HANDLERS = """ +from prik.codegen import ClassVisitor +class DemoEmitter(ClassVisitor): + PRIMARY_REGISTRY = {"item": "_emit_item"} + SECONDARY_DISPATCHER = {"item": {"value": "_emit_item_value"}} +""" -def test_reviewer_uses_stricter_recommendations_for_emitter_handlers(tmp_path: Path): - path = _write_module( - tmp_path, - "strict.py", - """ +_EMITTER_CALLING_THE_PRINTER = """ +from prik.codegen import ClassVisitor + +class DemoEmitter(ClassVisitor): + HANDLER_REGISTRY = {"item": "_emit_item"} + + def _emit_item(self, node): + return self.printer.doprint(node) +""" + +# Within the default limits for an ordinary function, but over the stricter +# recommendation for an emitter's `_convert_*` handler. +_BRANCHY_EMITTER_HANDLER = """ from prik.codegen import ClassVisitor class DemoEmitter(ClassVisitor): @@ -84,41 +66,40 @@ def _convert_item(self, value): if value == 5: return 5 return 6 -""", - ) - - violations = check_codegen_paths([path]) - - assert "complexity" in {violation.code for violation in violations} - - -def test_reviewer_reports_missing_primary_and_secondary_registry_handlers(tmp_path: Path): - codes = _check_source( - tmp_path, - """ -from prik.codegen import ClassVisitor - -class DemoEmitter(ClassVisitor): - PRIMARY_REGISTRY = {"item": "_emit_item"} - SECONDARY_DISPATCHER = {"item": {"value": "_emit_item_value"}} -""", - ) - - assert "registry-missing-handler" in codes - - -def test_reviewer_reports_printer_calls_from_handlers(tmp_path: Path): - codes = _check_source( - tmp_path, - """ -from prik.codegen import ClassVisitor - -class DemoEmitter(ClassVisitor): - HANDLER_REGISTRY = {"item": "_emit_item"} +""" + +_SMALL_LIMITS = WrapperCodegenCheckConfig(max_complexity=3, max_statements=4, max_nesting=2) + + +@pytest.mark.parametrize( + ("source", "config", "expected_codes"), + [ + pytest.param("def build_plan():\n return None\n", _SMALL_LIMITS, {"module-function"}, id="module_function"), + pytest.param("class WrapperPlanner:\n pass\n", _SMALL_LIMITS, {"visitor-class"}, id="non_visitor_class"), + pytest.param( + _OVERSIZED_FUNCTION, + _SMALL_LIMITS, + {"complexity", "statement-count", "nesting-depth"}, + id="complexity_statement_and_nesting_limits", + ), + pytest.param( + _EMITTER_WITH_MISSING_HANDLERS, + _SMALL_LIMITS, + {"registry-missing-handler"}, + id="missing_primary_and_secondary_registry_handlers", + ), + pytest.param( + _EMITTER_CALLING_THE_PRINTER, _SMALL_LIMITS, {"handler-printer-call"}, id="printer_call_from_handler" + ), + pytest.param(_BRANCHY_EMITTER_HANDLER, None, {"complexity"}, id="stricter_default_for_emitter_handlers"), + ], +) +def test_reviewer_reports_advisory_violations( + tmp_path: Path, source: str, config: WrapperCodegenCheckConfig | None, expected_codes: set[str] +): + path = tmp_path / "reviewed.py" + path.write_text(source, encoding="utf-8") - def _emit_item(self, node): - return self.printer.doprint(node) -""", - ) + violations = check_codegen_paths([path]) if config is None else check_codegen_paths([path], config=config) - assert "handler-printer-call" in codes + assert expected_codes <= {violation.code for violation in violations} diff --git a/tests/fortran/infrastructure/codegen/test_native_entrypoint_routing.py b/tests/fortran/infrastructure/codegen/test_native_entrypoint_routing.py index 1d958ee5b..2b76c4760 100644 --- a/tests/fortran/infrastructure/codegen/test_native_entrypoint_routing.py +++ b/tests/fortran/infrastructure/codegen/test_native_entrypoint_routing.py @@ -20,7 +20,7 @@ def _plan(source: str): return WrapperPlanner().build(module) -def test_direct_plan_keeps_one_projected_sequence_and_no_adapter_facets(): +def test_direct_plan_keeps_one_projected_sequence_and_emits_no_fortran_source(): plan = _plan( """ module direct_projection @@ -48,6 +48,13 @@ def test_direct_plan_keeps_one_projected_sequence_and_no_adapter_facets(): assert slot.projection_action is EntrypointProjectionAction.ARGUMENT_DEFAULT assert slot.passing is EntrypointPassingConvention.C_VALUE + # An all-direct module assembles only the binding and its header; the user's + # own Fortran objects still need the Fortran runtime at link time. + generated = WrapperGenerator().generate(plan) + assert generated.bridge_sources == () + assert generated.required_link_languages == ("fortran",) + assert [path.suffix for path in generated.source_paths] == [".c", ".h"] + def test_bind_c_descriptor_arrays_call_the_user_symbol_without_an_adapter(): plan = _plan( @@ -167,24 +174,3 @@ def test_direct_user_operation_and_fortran_support_own_separate_group_membership assert "bind_c_get_counter" in bridge assert "direct_value" not in bridge.casefold() - - -def test_all_direct_lowering_assembles_binding_and_header_without_fortran_source(): - generated = WrapperGenerator().generate( - _plan( - """ -module all_direct_lowering - use iso_c_binding -contains - integer(c_int) function value(input) bind(C) result(output) - integer(c_int), value, intent(in) :: input - output = input - end function value -end module all_direct_lowering -""" - ) - ) - - assert generated.bridge_sources == () - assert generated.required_link_languages == ("fortran",) - assert [path.suffix for path in generated.source_paths] == [".c", ".h"] diff --git a/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py b/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py deleted file mode 100644 index 37c16437e..000000000 --- a/tests/fortran/infrastructure/codegen/test_ordinary_fortran_codegen_baseline.py +++ /dev/null @@ -1,42 +0,0 @@ -"""Exact generated-output baseline for an ordinary Fortran procedure.""" - -from __future__ import annotations - -from hashlib import sha256 - -from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.pipeline.wrapper import WrapperGenerator -from prik.planning import WrapperPlanner -from prik.policy.completion import complete_semantic_policies - - -def test_ordinary_fortran_wrapper_preserves_exact_generated_bytes(): - module = parse_pyi_text( - "def scale(value: Float64) -> Float64: ...\n", - module_name="ordinary_entrypoint_baseline", - ) - complete_semantic_policies(module) - - generated = WrapperGenerator().generate(WrapperPlanner().build(module)) - - expected = { - "bind_c_ordinary_entrypoint_baseline_wrapper.f90": ( - 843, - "01c092ac9eaa0d90b58f0289a49ba0c71c967510e60a384602fe2e6e1e9b035f", - ), - "ordinary_entrypoint_baseline_wrapper.c": ( - 1823, - "ccefd674a96a84969abe54dbc9f599f9f004c2deefd1c3edc37117e34b9a1151", - ), - "ordinary_entrypoint_baseline_wrapper.h": ( - 248, - "6b29d016c71f463b5395d8875c1e1f04b98a625241baaf8f85685eee4fe2ce63", - ), - } - actual = { - source.path.name: (len(payload), sha256(payload).hexdigest()) - for source in generated.sources - for payload in (source.text.encode("utf-8"),) - } - - assert actual == expected diff --git a/tests/fortran/infrastructure/codegen/test_package.py b/tests/fortran/infrastructure/codegen/test_package.py index 685681504..d6b61af32 100644 --- a/tests/fortran/infrastructure/codegen/test_package.py +++ b/tests/fortran/infrastructure/codegen/test_package.py @@ -62,20 +62,14 @@ def _package_imports(root: Path) -> set[str]: return set().union(*(_imported_modules(path) for path in root.rglob("*.py"))) -def test_backend_generators_do_not_import_each_other(): - binding_imports = _imported_modules(CODEGEN_ROOT / "c" / "binding.py") - bridge_imports = _imported_modules(CODEGEN_ROOT / "fortran" / "bridge.py") - - assert not _imports_under(binding_imports, "prik.codegen.fortran") - assert not _imports_under(bridge_imports, "prik.codegen.c") - - def test_wrapper_stage_packages_follow_the_documented_dependency_direction(): semantic_imports = _package_imports(SEMANTICS_ROOT) policy_imports = _package_imports(POLICY_ROOT) planning_imports = _package_imports(PLANNING_ROOT) codegen_imports = _package_imports(CODEGEN_ROOT) printer_imports = _package_imports(PRINTERS_ROOT) + binding_imports = _imported_modules(CODEGEN_ROOT / "c" / "binding.py") + bridge_imports = _imported_modules(CODEGEN_ROOT / "fortran" / "bridge.py") assert not _imports_under(semantic_imports, "prik.policy") assert not _imports_under(semantic_imports, "prik.planning") @@ -88,3 +82,6 @@ def test_wrapper_stage_packages_follow_the_documented_dependency_direction(): assert not _imports_under(printer_imports, "prik.policy") assert not _imports_under(printer_imports, "prik.planning") assert not _imports_under(printer_imports, "prik.pipeline") + # The two backend generators lower the same plan independently. + assert not _imports_under(binding_imports, "prik.codegen.fortran") + assert not _imports_under(bridge_imports, "prik.codegen.c") diff --git a/tests/fortran/infrastructure/codegen/test_planner.py b/tests/fortran/infrastructure/codegen/test_planner.py index 4d4fb541b..a1a69e670 100644 --- a/tests/fortran/infrastructure/codegen/test_planner.py +++ b/tests/fortran/infrastructure/codegen/test_planner.py @@ -3,15 +3,11 @@ from __future__ import annotations -from dataclasses import replace - import pytest from tests.fortran._support.ownership_policy import parse_pyi_text from prik.semantics.models import PYTHON_EXPORTS_METADATA from prik.policy.completion import complete_semantic_policies -from prik.planning.planner import _ClassPolicyCatalog -from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner @@ -21,19 +17,6 @@ def _plan(source: str, *, module_name: str = "fmath"): return WrapperPlanner().build(module) -def _scalar_plan(): - return _plan( - """ -@nogil -@bind("SWAP_ARGS") -@standalone -@native_call([Addr(Arg(1)), Addr(Arg(0))]) -def swap_args(x: Float64, y: Float64) -> Float64: ... -""", - module_name="runtime_policy", - ) - - def _hidden_result_plan(): return _plan( """ @@ -44,12 +27,6 @@ def scale(x: Float64) -> Float64: ... ) -def _edit_first_function(plan, edit): - root = plan.namespaces[0] - functions = (edit(root.functions[0]), *root.functions[1:]) - return replace(plan, namespaces=(replace(root, functions=functions), *plan.namespaces[1:])) - - def test_planner_groups_completed_exports_into_explicit_namespace_nodes(): module = parse_pyi_text( """ @@ -72,31 +49,6 @@ def right_value(x: Int32) -> Int32: ... assert plan.namespaces[2].functions[0].symbol_name == "right_shared_value" -def test_planner_keeps_one_module_variable_plan_for_multiple_publications(): - """Namespace publications reference one plan that owns native access.""" - module = parse_pyi_text("counter: Int32\n", module_name="state") - module.variables[0].metadata[PYTHON_EXPORTS_METADATA] = [ - {"namespace": (), "name": "counter"}, - {"namespace": ("facade",), "name": "counter"}, - ] - complete_semantic_policies(module) - - plan = WrapperPlanner().build(module) - - variables = list(plan.variables) - publications = [ - (namespace.python_path, publication.variable, publication.python_names) - for namespace in plan.namespaces - for publication in namespace.variable_publications - ] - assert len(variables) == 1 - assert [(path, names) for path, _variable, names in publications] == [ - ((), ("counter",)), - (("facade",), ("counter",)), - ] - assert all(variable is variables[0] for _path, variable, _names in publications) - - def test_module_variable_owner_is_its_native_identity_not_a_publication_path(): """Adding a facade changes publications without moving native ownership.""" @@ -116,6 +68,12 @@ def planned_owner(*namespaces: str): assert [variable.owner_path for variable in facade_only.variables] == ["home.values"] assert [variable.owner_path for variable in facade_and_api.variables] == ["home.values"] + # Every publication references the one plan that owns native access. + assert all( + publication.variable is facade_and_api.variables[0] + for namespace in facade_and_api.namespaces + for publication in namespace.variable_publications + ) assert [variable.binding.support_namespace for variable in facade_only.variables] == [()] assert [variable.binding.support_namespace for variable in facade_and_api.variables] == [()] assert facade_only.entrypoint.support_procedures @@ -157,117 +115,6 @@ def right_value(x: Int32) -> Int32: ... assert len({stem.casefold() for stem in stems}) == len(stems) -def test_binding_registers_child_namespaces_as_importable_submodules(): - module = parse_pyi_text( - """ -def left_value(x: Int32) -> Int32: ... -def right_value(x: Int32) -> Int32: ... -""", - module_name="namespaced", - ) - module.functions[0].metadata[PYTHON_EXPORTS_METADATA] = [{"namespace": ("left",), "name": "shared_value"}] - module.functions[1].metadata[PYTHON_EXPORTS_METADATA] = [{"namespace": ("right",), "name": "shared_value"}] - complete_semantic_policies(module) - artifacts = WrapperGenerator().generate(WrapperPlanner().build(module)) - c_source = next(source.text for source in artifacts.sources if source.path.name.endswith(".c")) - - assert "PyModule_Create(&namespaced_left_module)" in c_source - assert "PyModule_Create(&namespaced_right_module)" in c_source - left_registration = 'PyDict_SetItemString(PyImport_GetModuleDict(), "namespaced.left", namespace_left) < 0' - assert left_registration in c_source - assert 'PyDict_SetItemString(PyImport_GetModuleDict(), "namespaced.right", namespace_right) < 0' in c_source - assert c_source.index(left_registration) > c_source.index("PyModule_Create(&namespaced_left_module)") - - -def test_post_ir_export_policy_fixes_names_within_each_namespace(): - module = parse_pyi_text( - """ -def first(x: Int32) -> Int32: ... -def second(x: Int32) -> Int32: ... -""", - module_name="namespaced_fixes", - ) - module.functions[0].name = "lambda" - module.functions[0].native_name = "first" - module.functions[0].metadata[PYTHON_EXPORTS_METADATA] = [{"namespace": ("child",), "name": None}] - module.functions[1].name = "lambda_" - module.functions[1].native_name = "second" - module.functions[1].metadata[PYTHON_EXPORTS_METADATA] = [{"namespace": ("child",), "name": None}] - - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - child = next(namespace for namespace in plan.namespaces if namespace.python_path == ("child",)) - - assert [function.binding.python_name for function in child.functions] == ["lambda_", "lambda__2"] - - -def test_planner_omits_private_functions_from_public_namespaces(): - module = parse_pyi_text( - """ -def visible(x: Int32) -> Int32: ... -def hidden(x: Int32) -> Int32: ... -""", - module_name="visibility", - ) - module.functions[1].visibility = "private" - complete_semantic_policies(module) - - plan = WrapperPlanner().build(module) - - assert [function.binding.python_name for function in plan.namespaces[0].functions] == ["visible"] - - -def test_class_policy_catalog_organizes_nested_classes_and_callable_owner_paths(): - module = parse_pyi_text( - """ -class outer: - class inner: - @native_call([Pass(), Addr(Arg(0))]) - def shift(self, dx: Float64) -> None: ... - - @overload("shift") - def move(self, dx: Float64) -> None: ... -""", - module_name="nested_catalog", - ) - complete_semantic_policies(module) - - catalog = _ClassPolicyCatalog.from_module(module) - outer, inner = catalog.entries - - assert tuple(entry.semantic_class.name for entry in catalog.entries) == ("outer", "inner") - assert inner.methods_by_owner_path["nested_catalog.outer.inner.shift"].name == "shift" - assert inner.method_policies_by_owner_path["nested_catalog.outer.inner.shift"].python_name == "shift" - assert inner.overload_functions_by_owner_path["nested_catalog.outer.inner.move.shift"].name == "shift" - - with pytest.raises(TypeError): - inner.methods_by_owner_path["nested_catalog.outer.inner.shift"] = outer.semantic_class - - plan = WrapperPlanner().build(module) - generated = WrapperGenerator().generate(plan) - - planned_outer, planned_inner = plan.namespaces[0].derived_types - assert (planned_outer.native_type_name, planned_inner.native_type_name) == ("outer", "inner") - # The nested type is defined beside its parent and bound on it, not here. - assert planned_outer.python_names == ("outer",) - assert planned_inner.python_names == () - assert planned_inner.nested_in == planned_outer.type_identity - assert planned_inner.contract_name == "inner" - assert {source.path.suffix for source in generated.sources} == {".c", ".h", ".f90"} - - -def test_planner_projects_required_array_buffer_policy(): - module = parse_pyi_text( - """ -def sum_values(values: Float64[:]) -> Float64: ... -""", - module_name="array_argument", - ) - complete_semantic_policies(module) - - assert WrapperPlanner().build(module).namespaces[0].functions[0].arguments[0].array is not None - - def test_planner_directly_projects_three_facets_and_distinct_call_orders(): function = _hidden_result_plan().namespaces[0].functions[0] argument = function.arguments[0] diff --git a/tests/fortran/infrastructure/codegen/test_visitor.py b/tests/fortran/infrastructure/codegen/test_visitor.py index 38b0f2341..545ac38b1 100644 --- a/tests/fortran/infrastructure/codegen/test_visitor.py +++ b/tests/fortran/infrastructure/codegen/test_visitor.py @@ -15,42 +15,23 @@ class ChildNode(BaseNode): """Child node that should use the most specific available handler.""" -class UnsupportedNode: - """Node with no matching handler.""" +class GrandchildNode(ChildNode): + """Node whose nearest handler is its parent's.""" -def test_class_visitor_uses_mro_specific_handler(): +def test_class_visitor_dispatches_by_mro_and_rejects_unhandled_nodes(): class Visitor(ClassVisitor): - def _visit_BaseNode(self, node): + def _render_BaseNode(self, node): return ("base", type(node).__name__) - def _visit_ChildNode(self, node): + def _render_ChildNode(self, node): return ("child", type(node).__name__) - assert Visitor().visit(ChildNode()) == ("child", "ChildNode") - - -def test_class_visitor_falls_back_to_base_handler(): - class Visitor(ClassVisitor): - def _visit_BaseNode(self, node): - return ("base", type(node).__name__) - - assert Visitor().visit(ChildNode()) == ("base", "ChildNode") - - -def test_class_visitor_supports_configurable_prefix(): - class Visitor(ClassVisitor): - def _render_BaseNode(self, node): - return ("rendered", type(node).__name__) - - assert Visitor(method_prefix="_render").visit(BaseNode()) == ("rendered", "BaseNode") - - -def test_class_visitor_reports_unsupported_nodes(): - visitor = ClassVisitor() - - with pytest.raises(UnsupportedWrapperCodegenNodeError) as exc_info: - visitor.visit(UnsupportedNode()) + visitor = Visitor(method_prefix="_render") - assert "UnsupportedNode" in str(exc_info.value) - assert "_visit" in str(exc_info.value) + assert visitor.visit(BaseNode()) == ("base", "BaseNode") + assert visitor.visit(ChildNode()) == ("child", "ChildNode") + assert visitor.visit(GrandchildNode()) == ("child", "GrandchildNode") + with pytest.raises(UnsupportedWrapperCodegenNodeError, match="_render") as exc_info: + visitor.visit(object()) + assert "object" in str(exc_info.value) diff --git a/tests/fortran/infrastructure/jupyter/end_to_end/test_fortran_magic_runtime.py b/tests/fortran/infrastructure/jupyter/end_to_end/test_fortran_magic_runtime.py index 26073a378..49bc78c44 100644 --- a/tests/fortran/infrastructure/jupyter/end_to_end/test_fortran_magic_runtime.py +++ b/tests/fortran/infrastructure/jupyter/end_to_end/test_fortran_magic_runtime.py @@ -19,7 +19,14 @@ @pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") -def test_fortran_cell_compiles_once_and_publishes_its_declared_module(tmp_path: Path, monkeypatch): +def test_fortran_cell_compiles_once_and_publishes_its_module_under_the_bound_name(tmp_path: Path, monkeypatch): + """A cell's private cache module name must not reach anything the user sees. + + The extension is imported under a cache name derived from the cell digest. + Every published object is renamed to what the session actually binds, + including the generated heap type that carries module variables, and + re-running the unchanged cell reuses that build. + """ build_calls = 0 build_fortran_extension = magic_module.build_fortran_extension @@ -33,44 +40,7 @@ def counting_build(*args, **kwargs): monkeypatch.setenv("IPYTHONDIR", str(tmp_path / "ipython")) shell = InteractiveShell() load_ipython_extension(shell) - cell = """module maths -contains - real(8) function square(x) - real(8), intent(in) :: x - square = x*x - end function -end module -""" - - shell.run_cell_magic("fortran", "", cell) - first_namespace = shell.user_ns["maths"] - assert first_namespace.square(np.float64(4.0)) == np.float64(16.0) - assert first_namespace.__name__ == "maths" - assert first_namespace.square.__module__ == "maths" - assert pretty(first_namespace.square) == "" - - shell.run_cell_magic("fortran", "", cell) - assert build_calls == 1 - assert shell.user_ns["maths"] is first_namespace - - -@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") -def test_published_names_carry_no_private_cache_module_identity(tmp_path: Path, monkeypatch): - """A cell's private cache module name must not reach anything the user sees. - - The extension is imported under a cache name derived from the cell digest. - Every published object is renamed to what the session actually binds, - including the generated heap type that carries module variables. - """ - monkeypatch.setenv("PRIK_CACHE_DIR", str(tmp_path / "cache")) - monkeypatch.setenv("IPYTHONDIR", str(tmp_path / "ipython")) - shell = InteractiveShell() - load_ipython_extension(shell) - - shell.run_cell_magic( - "fortran", - "", - """module cfg + cell = """module cfg real(8) :: gain = 2.0d0 contains real(8) function scaled(x) @@ -78,14 +48,16 @@ def test_published_names_carry_no_private_cache_module_identity(tmp_path: Path, scaled = gain*x end function end module -""", - ) +""" + + shell.run_cell_magic("fortran", "", cell) namespace = shell.user_ns["cfg"] assert namespace.scaled(np.float64(3.0)) == np.float64(6.0) assert namespace.gain == np.float64(2.0) assert namespace.__name__ == "cfg" assert namespace.scaled.__module__ == "cfg" + assert pretty(namespace.scaled) == "" # The module-variable namespace is an instance of a generated heap type, # whose own name embeds the private root until it is restated too. assert type(namespace).__module__ == "cfg" @@ -102,6 +74,10 @@ def test_published_names_carry_no_private_cache_module_identity(tmp_path: Path, # The private name still owns the import registration. assert any(name.startswith("_prik_f_") for name in sys.modules) + shell.run_cell_magic("fortran", "", cell) + assert build_calls == 1 + assert shell.user_ns["cfg"] is namespace + @pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") def test_generated_fortran_contract_can_be_edited_then_compiled_once(tmp_path: Path, monkeypatch): diff --git a/tests/fortran/infrastructure/jupyter/test_fortran_magic.py b/tests/fortran/infrastructure/jupyter/test_fortran_magic.py index f24cc9d66..2dc343b40 100644 --- a/tests/fortran/infrastructure/jupyter/test_fortran_magic.py +++ b/tests/fortran/infrastructure/jupyter/test_fortran_magic.py @@ -378,59 +378,6 @@ def build(contract: Path, **kwargs) -> WrapperBuildResult: assert shell.user_ns["maths"] is not first_namespace -def test_generated_standalone_contract_publishes_direct_declarations(tmp_path: Path, monkeypatch): - modules: dict[str, ModuleType] = {} - calls: list[Path] = [] - - def generate(path: Path, *, source_digest: str, options) -> contract_cells.GeneratedContracts: - return contract_cells.GeneratedContracts( - language="fortran", - source_digest=source_digest, - module_contracts={}, - direct_contract="@standalone\ndef square() -> None: ...", - dependency_contracts={}, - ) - - def build(contract: Path, **kwargs) -> WrapperBuildResult: - calls.append(contract) - output_dir = Path(kwargs["output_dir"]) - output_dir.mkdir(parents=True, exist_ok=True) - module_name = str(kwargs["output_name"]) - shared_library = output_dir / f"{module_name}.so" - shared_library.write_bytes(b"mock extension") - extension = ModuleType(module_name) - extension.square = lambda: 4 - modules[module_name] = extension - return WrapperBuildResult( - sources=(contract,), - module_name=module_name, - output_dir=output_dir, - shared_library=shared_library, - build_makefile=None, - compiled=True, - generated_sources=(), - generated_files=(), - ) - - monkeypatch.setattr(contract_cells, "generate_contracts_from_source", generate) - monkeypatch.setattr(magic_module, "build_pyi_extension", build) - monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) - shell = _Shell() - magic = PrikMagics(shell, cache_dir=tmp_path / "cache") - - magic.fortran("--pyi", "subroutine square()\nend subroutine\n") - inserted = shell.next_inputs[0][0] - magic_line, editable_cell = inserted.split("\n", 1) - assert magic_line == "%%pyi" - assert " file=" not in editable_cell - - magic.pyi(magic_line.removeprefix("%%pyi").strip(), editable_cell) - - assert calls[0].read_text(encoding="utf-8").endswith("@standalone\ndef square() -> None: ...\n") - assert shell.user_ns["square"]() == 4 - assert "cell" not in shell.user_ns - - def test_multiple_generated_contracts_use_distinct_jupyter_payloads(): writes: list[tuple[dict[str, object], bool]] = [] @@ -454,103 +401,110 @@ def set_next_input(self, text: str, *, replace: bool = False) -> None: ] -def test_editable_contract_requires_its_exact_cached_source(tmp_path: Path): - magic = PrikMagics(_Shell(), cache_dir=tmp_path / "cache") - digest = "a" * 64 - cell = f"# prik: file=maths.pyi source-sha256={digest}\n\ndef square(): ...\n" +def test_ipython_extension_refuses_to_replace_an_existing_cell_magic(): + class _ConflictingShell: + def find_cell_magic(self, name: str): + return (lambda: None) if name == "c" else None - with pytest.raises(UsageError, match="execute its %%fortran --pyi or %%c --pyi source cell again"): - magic.pyi("", cell) + def register_magics(self, _magic_class) -> None: + raise AssertionError("conflicting magics must be reported before registration") + with pytest.raises(UsageError, match=r"already registered: %%c"): + load_ipython_extension(_ConflictingShell()) -def test_magic_reports_usage_without_terminating_ipython(tmp_path: Path, capsys): - magic = PrikMagics(_Shell(), cache_dir=tmp_path / "cache") - magic.fortran("--help", "") - assert "usage: %%fortran" in capsys.readouterr().out +_DIGEST = "a" * 64 - with pytest.raises(UsageError, match="non-empty"): - magic.fortran("", "\n") - with pytest.raises(UsageError, match="only generates editable cells"): - magic.fortran("--pyi --force", "source") - with pytest.raises(UsageError, match="generated source metadata or explicit"): - magic.pyi("", "def square(): ...\n") - with pytest.raises(UsageError, match="cannot mix"): - magic.pyi( + +@pytest.mark.parametrize( + ("magic_name", "line", "cell", "message"), + [ + pytest.param("fortran", "", "\n", "non-empty", id="empty-cell"), + pytest.param("fortran", "--pyi --force", "source", "only generates editable cells", id="pyi-with-force"), + pytest.param( + "fortran", + "--native-compile-flags -O3", + "source", + r'--native-compile-flags="-O3 -march=native"', + id="dash-prefixed-flag-group", + ), + # --compiler-arg carries exactly one argument, so it is not told to pass a quoted group. + pytest.param( + "fortran", + "--compiler-arg -fopenmp", + "source", + r"--compiler-arg=-fopenmp(?![\s\S]*quoted group)", + id="dash-prefixed-single-flag", + ), + pytest.param( + "fortran", "--compiler", "source", r"^argument --compiler: expected one argument$", id="missing-plain-value" + ), + pytest.param( + "pyi", "", "def square(): ...\n", "generated source metadata or explicit", id="contract-without-source" + ), + pytest.param( + "pyi", "--native-fortran-sources one.f90 --native-c-sources one.c", "def square(): ...\n", - ) - source = tmp_path / "native.f90" - source.write_text("subroutine native()\nend subroutine\n", encoding="utf-8") - with pytest.raises(UsageError, match="cannot combine generated source-sha256 metadata"): - magic.pyi( - f"--native-fortran-sources {source}", - f"# prik: source-sha256={'a' * 64}\n\ndef native(): ...\n", - ) - with pytest.raises(UsageError, match="full lowercase source-sha256"): - magic.pyi( + "cannot mix", + id="mixed-native-languages", + ), + pytest.param( + "pyi", + "--native-fortran-sources {native}", + f"# prik: source-sha256={_DIGEST}\n\ndef native(): ...\n", + "cannot combine generated source-sha256 metadata", + id="generated-metadata-with-explicit-sources", + ), + pytest.param( + "pyi", "", "# prik: file=maths.pyi source-sha256=short\ndef square(): ...\n", - ) - - -def test_editable_contract_metadata_errors_name_what_the_cell_got_wrong(tmp_path: Path): - """An edited contract cell must say which metadata a user broke. - - These are the guards an ordinary edit reaches: duplicating the reserved - line, or renaming the contract to something that is not a module path. - """ - magic = PrikMagics(_Shell(), cache_dir=tmp_path / "cache") - digest = "a" * 64 - - with pytest.raises(UsageError, match="exactly one PRIK metadata line"): - magic.pyi("", f"# prik: source-sha256={digest}\n# prik: file=maths.pyi\n\ndef square(): ...\n") - for filename in ("../escape.pyi", "maths.txt", "not-an-identifier.pyi", "__init__.pyi"): - with pytest.raises(UsageError, match=r"Invalid editable \.pyi filename"): - magic.pyi("", f"# prik: file={filename} source-sha256={digest}\n\ndef square(): ...\n") - - -def test_dash_prefixed_flag_value_usage_names_the_equals_form(tmp_path: Path): - """A flag value argparse read as an option must say how to write it.""" + "full lowercase source-sha256", + id="short-digest", + ), + pytest.param( + "pyi", + "", + f"# prik: file=maths.pyi source-sha256={_DIGEST}\n\ndef square(): ...\n", + "execute its %%fortran --pyi or %%c --pyi source cell again", + id="uncached-generated-source", + ), + pytest.param( + "pyi", + "", + f"# prik: source-sha256={_DIGEST}\n# prik: file=maths.pyi\n\ndef square(): ...\n", + "exactly one PRIK metadata line", + id="duplicated-metadata-line", + ), + *( + pytest.param( + "pyi", + "", + f"# prik: file={filename} source-sha256={_DIGEST}\n\ndef square(): ...\n", + r"Invalid editable \.pyi filename", + id=f"invalid-filename-{label}", + ) + for label, filename in ( + ("escape", "../escape.pyi"), + ("suffix", "maths.txt"), + ("identifier", "not-an-identifier.pyi"), + ("package-root", "__init__.pyi"), + ) + ), + ], +) +def test_magic_usage_errors_name_what_the_cell_got_wrong(tmp_path: Path, magic_name, line, cell, message): + """Invalid cells raise IPython usage errors, never SystemExit, and say what to change.""" + native = tmp_path / "native.f90" + native.write_text("subroutine native()\nend subroutine\n", encoding="utf-8") magic = PrikMagics(_Shell(), cache_dir=tmp_path / "cache") - with pytest.raises(UsageError, match=r'--native-compile-flags="-O3 -march=native"'): - magic.fortran("--native-compile-flags -O3", "source") - # ``--compiler-arg`` carries exactly one argument, so it must not be told - # to pass a quoted group of several flags. - with pytest.raises(UsageError, match=r"--compiler-arg=-fopenmp"): - magic.fortran("--compiler-arg -fopenmp", "source") - assert "quoted group" not in _compiler_arg_usage_message(magic) - # An option whose value is never dash-prefixed keeps the plain message. - with pytest.raises(UsageError, match=r"^argument --compiler: expected one argument$"): - magic.fortran("--compiler", "source") + with pytest.raises(UsageError, match=message): + getattr(magic, magic_name)(line.format(native=native), cell) -def _compiler_arg_usage_message(magic: PrikMagics) -> str: - with pytest.raises(UsageError) as raised: - magic.fortran("--compiler-arg -fopenmp", "source") - return str(raised.value) +def test_magic_help_prints_usage_without_terminating_ipython(tmp_path: Path, capsys): + PrikMagics(_Shell(), cache_dir=tmp_path / "cache").fortran("--help", "") - -def test_ipython_extension_hook_registers_the_magic_class(): - registered = [] - - class _RegistrationShell: - def register_magics(self, magic_class) -> None: - registered.append(magic_class) - - load_ipython_extension(_RegistrationShell()) - - assert registered == [PrikMagics] - - -def test_ipython_extension_refuses_to_replace_an_existing_cell_magic(): - class _ConflictingShell: - def find_cell_magic(self, name: str): - return (lambda: None) if name == "c" else None - - def register_magics(self, magic_class) -> None: - raise AssertionError("conflicting magics must be reported before registration") - - with pytest.raises(UsageError, match=r"already registered: %%c"): - load_ipython_extension(_ConflictingShell()) + assert "usage: %%fortran" in capsys.readouterr().out diff --git a/tests/fortran/infrastructure/naming/test_policy.py b/tests/fortran/infrastructure/naming/test_policy.py index db655f640..9b72c757e 100644 --- a/tests/fortran/infrastructure/naming/test_policy.py +++ b/tests/fortran/infrastructure/naming/test_policy.py @@ -36,30 +36,28 @@ def test_two_spellings_collide_only_where_the_source_folds_them(): assert preserving.reserve_public_name((), "foo", category="function") == "foo" -def test_a_wrapped_type_is_named_as_a_python_class(): - """A derived type reaches Python as a class, so PRIK spells it like one.""" - assert normalize_public_name("point_t", category="class").name == "Point_T" - assert normalize_public_name("my_particle_type", category="class").name == "My_Particle_Type" - assert normalize_public_name("accumulator", category="class").name == "Accumulator" - # Fortran writes one type under many spellings, so the style does not - # depend on which one the source happened to use. - assert normalize_public_name("POINT_T", category="class").name == "Point_T" +def test_class_names_take_python_class_style_only_where_the_source_folds_case(): + cases = { + # A derived type reaches Python as a class, so PRIK spells it like one. + ("point_t", False): "Point_T", + ("my_particle_type", False): "My_Particle_Type", + ("accumulator", False): "Accumulator", + # Fortran writes one type under many spellings; the style ignores which. + ("POINT_T", False): "Point_T", + # C names each declaration exactly, so PRIK has no spelling to choose. + ("point", True): "point", + ("Point", True): "Point", + } + for (name, preserve_case), expected in cases.items(): + normalized = normalize_public_name(name, preserve_case=preserve_case, category="class") + assert (normalized.name, normalized.needs_fix) == (expected, False), name + + # Strict naming rejects what Python cannot spell, not how PRIK cases it. + assert normalize_public_name("point t", category="class").needs_fix is True # Every other declaration keeps the lower-case form. assert normalize_public_name("point_t").name == "point_t" -def test_a_source_that_spells_its_own_types_keeps_that_spelling(): - """C names each declaration exactly, so PRIK has no spelling to choose.""" - assert normalize_public_name("point", preserve_case=True, category="class").name == "point" - assert normalize_public_name("Point", preserve_case=True, category="class").name == "Point" - - -def test_a_chosen_class_style_is_not_a_name_python_forced(): - """Strict naming rejects what Python cannot spell, not how PRIK cases it.""" - assert normalize_public_name("point_t", category="class").needs_fix is False - assert normalize_public_name("point t", category="class").needs_fix is True - - def test_public_python_names_escape_keywords_and_collisions(): policy = NamingPolicy() @@ -157,17 +155,17 @@ def test_generated_symbols_reserve_c_entry_point_and_rewrite_special_methods(): ) -def test_generated_symbols_number_after_an_escaped_native_name(): +def test_generated_symbols_number_after_an_escaped_native_name_past_every_taken_spelling(): policy = NamingPolicy() assert ( policy.generated_symbol( "module", - {"MODULE_PRIK"}, + {"MODULE_PRIK", "module_prik_2"}, language="fortran", prefix="owner__", context="function", parent_context="module", ) - == "module_prik_2" + == "module_prik_3" ) diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_argument_name.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_argument_name.f90 deleted file mode 100644 index cfdc3e927..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_argument_name.f90 +++ /dev/null @@ -1,4 +0,0 @@ -subroutine dup(x, y, x) - integer, intent(in) :: x - real, intent(in) :: y -end subroutine dup diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_argument_name.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_argument_name.json deleted file mode 100644 index ec6dbdc14..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_argument_name.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Duplicate argument name 'x' in procedure 'dup'." - ], - "diagnostic_contains": [ - "error[PARSE_DUPLICATE_ARGUMENT]", - "Duplicate argument name 'x' in procedure 'dup'.", - "subroutine dup(x, y, x)" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_declaration_procedure.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_declaration_procedure.f90 deleted file mode 100644 index 5b16abab3..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_declaration_procedure.f90 +++ /dev/null @@ -1,4 +0,0 @@ -subroutine dup(x) - real :: x - integer :: x -end subroutine dup diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_declaration_procedure.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_declaration_procedure.json deleted file mode 100644 index 73cb9eee5..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_declaration_procedure.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Duplicate declaration of symbol 'x' in procedure 'dup'." - ], - "diagnostic_contains": [ - "error[PARSE_DUPLICATE_DECLARATION]", - "Duplicate declaration of symbol 'x' in procedure 'dup'.", - "integer :: x" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_field_derived_type.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_field_derived_type.f90 deleted file mode 100644 index 4e2a7742e..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_field_derived_type.f90 +++ /dev/null @@ -1,6 +0,0 @@ -module m - type :: point - real :: x - integer :: x - end type point -end module m diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_field_derived_type.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_field_derived_type.json deleted file mode 100644 index d005c62a3..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_field_derived_type.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_types", - "error_type": "FortranParseError", - "message_contains": [ - "Duplicate field 'x' in derived type 'point'." - ], - "diagnostic_contains": [ - "error[PARSE_DUPLICATE_FIELD]", - "Duplicate field 'x' in derived type 'point'.", - "" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_parameter.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_parameter.f90 deleted file mode 100644 index fbd7056fa..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_parameter.f90 +++ /dev/null @@ -1,4 +0,0 @@ -subroutine dup_param() - integer, parameter :: n = 5 - integer, parameter :: n = 10 -end subroutine dup_param diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_parameter.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_parameter.json deleted file mode 100644 index 3a49cbc84..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_parameter.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Duplicate PARAMETER declaration of symbol 'n' in procedure 'dup_param'." - ], - "diagnostic_contains": [ - "error[PARSE_DUPLICATE_PARAMETER]", - "Duplicate PARAMETER declaration of symbol 'n' in procedure 'dup_param'.", - "integer, parameter :: n = 10" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_global.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_global.f90 deleted file mode 100644 index 0247eb5fe..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_global.f90 +++ /dev/null @@ -1,7 +0,0 @@ -subroutine work(n) - integer, intent(in) :: n -end subroutine work - -subroutine work(n) - integer, intent(in) :: n -end subroutine work diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_global.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_global.json deleted file mode 100644 index 9f036d8b8..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_global.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Duplicate procedure name 'work' in global scope." - ], - "diagnostic_contains": [ - "error[PARSE_DUPLICATE_PROCEDURE]", - "Duplicate procedure name 'work' in global scope.", - "subroutine work(n)" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_module.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_module.f90 deleted file mode 100644 index 4ece1fe33..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_module.f90 +++ /dev/null @@ -1,9 +0,0 @@ -module m -contains - subroutine work(n) - integer, intent(in) :: n - end subroutine work - subroutine work(n) - integer, intent(in) :: n - end subroutine work -end module m diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_module.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_module.json deleted file mode 100644 index c2cc637c7..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_procedure_module.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Duplicate procedure name 'work' in module 'm'." - ], - "diagnostic_contains": [ - "error[PARSE_DUPLICATE_PROCEDURE]", - "Duplicate procedure name 'work' in module 'm'.", - "subroutine work(n)" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_variable_module.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_variable_module.f90 deleted file mode 100644 index 85411df17..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_variable_module.f90 +++ /dev/null @@ -1,4 +0,0 @@ -module m - integer :: n - real :: n -end module m diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_variable_module.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_variable_module.json deleted file mode 100644 index 4096848cc..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_duplicate_variable_module.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_modules", - "error_type": "FortranParseError", - "message_contains": [ - "Duplicate variable 'n' in module 'm'." - ], - "diagnostic_contains": [ - "error[PARSE_DUPLICATE_VARIABLE]", - "Duplicate variable 'n' in module 'm'.", - "" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_arg.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_arg.f90 deleted file mode 100644 index ebd99414c..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_arg.f90 +++ /dev/null @@ -1,4 +0,0 @@ -subroutine foo(x, y) - implicit none - integer, intent(in) :: x -end subroutine foo diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_arg.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_arg.json deleted file mode 100644 index e6fac08b3..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_arg.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Argument 'y' in procedure 'foo' has no type declaration (implicit none is active)." - ], - "diagnostic_contains": [ - "error[PARSE_IMPLICIT_NONE_UNDECLARED_SYMBOL]", - "Argument 'y' in procedure 'foo' has no type declaration (implicit none is active).", - "" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_result.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_result.f90 deleted file mode 100644 index 310580d16..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_result.f90 +++ /dev/null @@ -1,4 +0,0 @@ -function f(x) - implicit none - integer, intent(in) :: x -end function f diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_result.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_result.json deleted file mode 100644 index 4a55be7c6..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_implicit_none_undeclared_result.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Function result 'f' in procedure 'f' has no type declaration (implicit none is active)." - ], - "diagnostic_contains": [ - "error[PARSE_IMPLICIT_NONE_UNDECLARED_SYMBOL]", - "Function result 'f' in procedure 'f' has no type declaration (implicit none is active).", - "" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_parameter_without_type_implicit_none.f b/tests/fortran/infrastructure/parsing/fixtures/errors/err_parameter_without_type_implicit_none.f deleted file mode 100644 index 459b48a7f..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_parameter_without_type_implicit_none.f +++ /dev/null @@ -1,5 +0,0 @@ - subroutine cst(a) - implicit none - real a - parameter ( zero = 0.0e+0 ) - end diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_parameter_without_type_implicit_none.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_parameter_without_type_implicit_none.json deleted file mode 100644 index 341e701cc..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_parameter_without_type_implicit_none.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Unknown datatype for PARAMETER symbol 'zero' in procedure 'cst'." - ], - "diagnostic_contains": [ - "error[PARSE_UNKNOWN_PARAMETER_TYPE]", - "Unknown datatype for PARAMETER symbol 'zero' in procedure 'cst'.", - "parameter ( zero = 0.0e+0 )" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_result_shadows_argument.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_result_shadows_argument.f90 deleted file mode 100644 index 532c4cda2..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_result_shadows_argument.f90 +++ /dev/null @@ -1,3 +0,0 @@ -function f(res) result(res) - integer, intent(in) :: res -end function f diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_result_shadows_argument.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_result_shadows_argument.json deleted file mode 100644 index c3fabbb38..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_result_shadows_argument.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Function result variable 'res' in function 'f' shadows an argument name." - ], - "diagnostic_contains": [ - "error[PARSE_RESULT_SHADOWS_ARGUMENT]", - "Function result variable 'res' in function 'f' shadows an argument name.", - "" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_function_result.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_function_result.f90 deleted file mode 100644 index cb7efaa6f..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_function_result.f90 +++ /dev/null @@ -1,4 +0,0 @@ -function f(x) result(res) - implicit none - real :: x -end function f diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_function_result.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_function_result.json deleted file mode 100644 index 1d087b74c..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_function_result.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Unknown datatype for function result 'res' in procedure 'f'." - ], - "diagnostic_contains": [ - "error[PARSE_UNKNOWN_FUNCTION_RESULT_TYPE]", - "Unknown datatype for function result 'res' in procedure 'f'.", - "" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_derived_type.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_derived_type.f90 deleted file mode 100644 index da7c5b88d..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_derived_type.f90 +++ /dev/null @@ -1,5 +0,0 @@ -module m - type :: t - weirdtype :: x - end type t -end module m diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_derived_type.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_derived_type.json deleted file mode 100644 index b35d27028..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_derived_type.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_types", - "error_type": "FortranParseError", - "message_contains": [ - "Unknown or unsupported datatype declaration in type 't': weirdtype :: x" - ], - "diagnostic_contains": [ - "error[PARSE_UNSUPPORTED_DECLARATION]", - "Unknown or unsupported datatype declaration in type 't': weirdtype :: x", - "weirdtype :: x" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_module.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_module.f90 deleted file mode 100644 index 5b830dde1..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_module.f90 +++ /dev/null @@ -1,3 +0,0 @@ -module m - weirdtype :: x -end module m diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_module.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_module.json deleted file mode 100644 index ec47a6201..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_module.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_modules", - "error_type": "FortranParseError", - "message_contains": [ - "Unknown or unsupported datatype declaration in module 'm': weirdtype :: x" - ], - "diagnostic_contains": [ - "error[PARSE_UNSUPPORTED_DECLARATION]", - "Unknown or unsupported datatype declaration in module 'm': weirdtype :: x", - "weirdtype :: x" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_procedure.f90 b/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_procedure.f90 deleted file mode 100644 index ac31ca6e1..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_procedure.f90 +++ /dev/null @@ -1,3 +0,0 @@ -subroutine bad(x) - weirdtype :: x -end subroutine bad diff --git a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_procedure.json b/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_procedure.json deleted file mode 100644 index 6dccf0f9e..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/errors/err_unknown_type_procedure.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "parser": "parse_fortran_procedures", - "error_type": "FortranParseError", - "message_contains": [ - "Unknown or unsupported datatype declaration for procedure 'bad': weirdtype :: x" - ], - "diagnostic_contains": [ - "error[PARSE_UNSUPPORTED_DECLARATION]", - "Unknown or unsupported datatype declaration for procedure 'bad': weirdtype :: x", - "weirdtype :: x" - ] -} diff --git a/tests/fortran/infrastructure/parsing/fixtures/json_sanity_allowlist.json b/tests/fortran/infrastructure/parsing/fixtures/json_sanity_allowlist.json deleted file mode 100644 index 1f9024b7e..000000000 --- a/tests/fortran/infrastructure/parsing/fixtures/json_sanity_allowlist.json +++ /dev/null @@ -1,3 +0,0 @@ -{ - "allowed_unknown_base_types": [] -} diff --git a/tests/fortran/infrastructure/parsing/generate_error_goldens.py b/tests/fortran/infrastructure/parsing/generate_error_goldens.py deleted file mode 100644 index fde55268c..000000000 --- a/tests/fortran/infrastructure/parsing/generate_error_goldens.py +++ /dev/null @@ -1,103 +0,0 @@ -"""Generate/update golden files for Fortran parser error fixtures.""" - -from __future__ import annotations - -import json -import sys -from pathlib import Path - -_REPO_ROOT = Path(__file__).resolve().parents[4] -if str(_REPO_ROOT) not in sys.path: - sys.path.insert(0, str(_REPO_ROOT)) - - -def parse_fortran_procedures(source, filename=None): - return _parse_fortran_file(source, filename=filename).procedures - - -def parse_fortran_types(source, filename=None): - return _parse_fortran_file(source, filename=filename).derived_types - - -def parse_fortran_modules(source, filename=None): - return _parse_fortran_file(source, filename=filename).modules - - -_ERRORS_DIR = Path(__file__).parent / "fixtures" / "errors" -_EXPECTED_ERRORS_DIR = _ERRORS_DIR -_PARSER_MAP = { - "parse_fortran_procedures": parse_fortran_procedures, - "parse_fortran_types": parse_fortran_types, - "parse_fortran_modules": parse_fortran_modules, -} -_DEFAULT_PARSER = "parse_fortran_procedures" - - -def _parse_fortran_file(source, filename=None): - from prik.parsers.fortran import parse_fortran_file - - return parse_fortran_file(source, filename=filename) - - -def _get_parser_for_fixture(fixture: Path) -> str: - json_path = _EXPECTED_ERRORS_DIR / f"{fixture.stem}.json" - if json_path.exists(): - try: - data = json.loads(json_path.read_text(encoding="utf-8")) - return data.get("parser", _DEFAULT_PARSER) - except (json.JSONDecodeError, KeyError): - pass - return _DEFAULT_PARSER - - -def _serialize_error_fixture(fixture: Path) -> dict: - from prik.parsers.fortran import FortranParseError - - source = fixture.read_text(encoding="utf-8") - parser_name = _get_parser_for_fixture(fixture) - parser_fn = _PARSER_MAP[parser_name] - try: - parser_fn(source, filename=fixture.name) - raise SystemExit( - f"ERROR: {fixture.name} did not raise FortranParseError — error fixture files must trigger a parse error." - ) - except FortranParseError as exc: - return { - "parser": parser_name, - "error_type": "FortranParseError", - "message_contains": [exc.base_message], - "diagnostic_contains": [ - f"error[{exc.code}]", - exc.base_message, - exc.source_line.strip() if exc.source_line else "", - ], - } - - -def main() -> None: - _EXPECTED_ERRORS_DIR.mkdir(parents=True, exist_ok=True) - requested = sys.argv[1:] - if requested: - fixtures = [] - for item in requested: - p = Path(item) - if not p.is_absolute(): - p = _ERRORS_DIR / p - fixtures.append(p) - else: - extensions = {".f", ".for", ".ftn", ".f77", ".f90", ".f95", ".f03", ".f08"} - fixtures = sorted(f for f in _ERRORS_DIR.glob("*") if f.is_file() and f.suffix.lower() in extensions) - if not fixtures: - raise SystemExit("No Fortran error fixtures found") - - for fixture in fixtures: - if not fixture.exists(): - raise SystemExit(f"Fixture does not exist: {fixture}") - payload = _serialize_error_fixture(fixture) - out = _EXPECTED_ERRORS_DIR / f"{fixture.stem}.json" - out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") - print(f"updated {out}") - - -if __name__ == "__main__": - main() diff --git a/tests/fortran/infrastructure/parsing/generate_parser_goldens.py b/tests/fortran/infrastructure/parsing/generate_parser_goldens.py deleted file mode 100644 index 24ed3604a..000000000 --- a/tests/fortran/infrastructure/parsing/generate_parser_goldens.py +++ /dev/null @@ -1,82 +0,0 @@ -"""Generate/update golden files for Fortran parser fixtures.""" - -from __future__ import annotations - -import json -import sys -from dataclasses import asdict -from pathlib import Path - -_REPO_ROOT = Path(__file__).resolve().parents[4] -if str(_REPO_ROOT) not in sys.path: - sys.path.insert(0, str(_REPO_ROOT)) - -_FIXTURES_DIR = Path(__file__).parent / "fixtures" -_FORTRAN_DIR = _FIXTURES_DIR - - -def _strip_parent_fields(value): - if isinstance(value, dict): - return {k: _strip_parent_fields(v) for k, v in value.items() if k != "parent"} - if isinstance(value, list): - return [_strip_parent_fields(v) for v in value] - return value - - -def _parser_filename_for_fixture(fixture: Path) -> str: - relpath = fixture.relative_to(_FORTRAN_DIR).as_posix() - if relpath.startswith("general/"): - return fixture.name - return relpath - - -def _serialize_fixture(fixture: Path) -> dict: - from prik.parsers.fortran import parse_fortran_file - - source = fixture.read_text(encoding="utf-8") - parsed = parse_fortran_file(source, filename=_parser_filename_for_fixture(fixture)) - return _strip_parent_fields(asdict(parsed)) - - -def _output_path_for_fixture(fixture: Path) -> Path: - return (_FIXTURES_DIR / fixture.relative_to(_FORTRAN_DIR)).with_suffix(".json") - - -def main() -> None: - _FIXTURES_DIR.mkdir(parents=True, exist_ok=True) - requested = sys.argv[1:] - if requested: - fixtures = [] - for item in requested: - p = Path(item) - if not p.is_absolute(): - p = _FORTRAN_DIR / p - if p.suffix.lower() not in (".f", ".for", ".ftn", ".f90", ".f95", ".f03", ".f08"): - p = p.with_suffix(".f90") - fixtures.append(p) - else: - fixtures = sorted(_FORTRAN_DIR.rglob("*.f*")) - if not fixtures: - raise SystemExit("No Fortran fixtures found") - - failures = [] - for fixture in fixtures: - if not fixture.exists(): - raise SystemExit(f"Fixture does not exist: {fixture}") - try: - payload = _serialize_fixture(fixture) - out = _output_path_for_fixture(fixture) - out.parent.mkdir(parents=True, exist_ok=True) - out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") - print(f"updated {out}") - except Exception as exc: - failures.append((fixture, exc)) - - if failures: - for fixture, exc in failures: - print(f"failed {fixture}: {exc}", file=sys.stderr) - raise SystemExit(1) - - -if __name__ == "__main__": - main() diff --git a/tests/fortran/infrastructure/parsing/test_declaration_and_interface_edges.py b/tests/fortran/infrastructure/parsing/test_declaration_and_interface_edges.py index aa8c7199a..fa4a5fdcc 100644 --- a/tests/fortran/infrastructure/parsing/test_declaration_and_interface_edges.py +++ b/tests/fortran/infrastructure/parsing/test_declaration_and_interface_edges.py @@ -1,11 +1,7 @@ -"""Declaration parsing, interfaces, and less common scope edges.""" +"""Declaration forms, compile-time kind and shape resolution, and type definitions.""" -import pytest - -from prik.parsers.fortran.models import FortranModule +from prik.parsers.fortran import parse_fortran_file, parse_fortran_project from prik.parsers.fortran.scope import ScopeUses -from prik.parsers.fortran.parser import FortranParser, _ParserScope -from prik.parsers.fortran import FortranParseError, parse_fortran_file, parse_fortran_project def test_legacy_star_kind_and_declarations_without_double_colon_are_resolved(): @@ -20,18 +16,15 @@ def test_legacy_star_kind_and_declarations_without_double_colon_are_resolved(): """ sig = parse_fortran_file(code, filename="legacy_decl.f").procedures[0] - args = {arg.name: arg for arg in sig.arguments} + args = {arg.name: (arg.base_type, arg.kind) for arg in sig.arguments} - assert args["x"].base_type == "real" - assert args["x"].kind == "8" - assert args["z"].base_type == "complex" - assert args["z"].kind == "16" - assert args["p"].base_type == "derived" - assert args["p"].kind == "point" - assert args["c"].base_type == "derived" - assert args["c"].kind == "point" - assert args["f"].base_type == "procedure" - assert args["f"].kind == "cb" + assert args == { + "x": ("real", "8"), + "z": ("complex", "16"), + "p": ("derived", "point"), + "c": ("derived", "point"), + "f": ("procedure", "cb"), + } def test_bind_c_and_dummy_argument_attributes_from_inline_fortran(): @@ -61,118 +54,101 @@ def test_bind_c_and_dummy_argument_attributes_from_inline_fortran(): assert args["cb"].base_type == "real" -def test_project_duplicate_registries_raise_for_public_scopes(): - with pytest.raises(FortranParseError, match="Duplicate symbol 'work' in project procedure scope"): - parse_fortran_project( - { - "a.f90": """ -subroutine work() -end subroutine work -""", - "b.f90": """ -subroutine work() -end subroutine work -""", - } - ) - - with pytest.raises(FortranParseError, match="Duplicate symbol 'driver' in project program scope"): - parse_fortran_project( - { - "a.f90": """ -program driver -end program driver -""", - "b.f90": """ -program driver -end program driver -""", - } - ) - - -def test_module_scope_ignores_non_variable_spec_lines(): - code = """ +def test_module_specification_keeps_variables_and_skips_other_statements(): + """Statements that declare no variable are skipped; every declared entity is kept once.""" + parsed = parse_fortran_file( + """ module spec_mod public :: private :: import :: external_symbol implicit none save - integer :: kept + module procedure :: ignored_impl + integer, parameter :: rk = 8, n = 3 + real(kind=rk), parameter, private, dimension(0:n) :: weights = 1.0_rk + logical, parameter :: flag = .true. + integer, parameter :: pair(2) = (/ 1, 2 /) + integer kept, , also_kept + real values(2) + real*8 :: wide + character(len=default_len) :: label*(name_len), other + type :: state + sequence + private + real :: x, , y + end type state contains subroutine worker() end subroutine worker end module spec_mod -""" - - module = parse_fortran_file(code).modules[0] - - assert module.default_visibility == "private" - assert [var.name for var in module.variables] == ["kept"] - -def test_module_declarations_without_double_colon_are_parsed_not_dropped(): - code = """ -module legacy_module_decls - integer kept - real values(2) -end module legacy_module_decls -""" - - module = parse_fortran_file(code).modules[0] +program type_stmt_program + type :: local_state + integer :: marker + end type local_state + integer :: kept +end program type_stmt_program +""", + filename="module_spec.f90", + ) + module = parsed.modules[0] variables = {var.name: var for var in module.variables} - assert variables["kept"].base_type == "integer" - assert variables["values"].base_type == "real" - assert variables["values"].shape == ["2"] + assert module.default_visibility == "private" + assert list(variables) == [ + "rk", + "n", + "weights", + "flag", + "pair", + "kept", + "also_kept", + "values", + "wide", + "label", + "other", + ] + weights = variables["weights"] + assert (weights.kind, weights.shape, weights.lbound, weights.ubound) == ("8", ["0:3"], ["0"], ["3"]) + assert (weights.is_parameter, weights.value, weights.symbolic_value) == (True, "1", "1.0_rk") + assert [(variables[name].value, variables[name].symbolic_value) for name in ("flag", "pair")] == [ + ("1", ".true."), + ("(/ 1, 2 /)", "(/ 1, 2 /)"), + ] + assert module.private_symbols == ["weights"] + assert (variables["kept"].base_type, variables["values"].shape) == ("integer", ["2"]) + assert (variables["wide"].base_type, variables["wide"].kind) == ("real", "8") + # An entity's own character length does not leak into the next entity. + assert (variables["other"].kind, variables["other"].shape) == ("len=default_len", []) + dtype = module.derived_types[0] + assert [field.name for field in dtype.fields] == ["x", "y"] + assert dtype.attributes == ["sequence"] + assert [var.name for var in parsed.programs[0].variables] == ["kept"] -def test_use_rename_and_intrinsic_forms_are_recorded(): +def test_use_rename_intrinsic_and_empty_only_items_are_recorded(): code = """ module use_forms use list_input, delete_input => delete_input_list use, intrinsic :: iso_c_binding, only: c_int, c_double + use constants_mod, only: rk, , ik end module use_forms """ - module = parse_fortran_file(code).modules[0] + scope = ScopeUses(parse_fortran_file(code).modules[0].uses) - scope = ScopeUses(module.uses) # A rename without `only` binds the new name and still imports the rest. assert scope.imports_all("list_input") is True - assert list(scope.mappings("list_input")) == ["delete_input"] - assert (scope.mappings("list_input")[0].source, scope.mappings("list_input")[0].target) == ( - "delete_input_list", - "delete_input", - ) - + assert [(item.source, item.target) for item in scope.mappings("list_input")] == [ + ("delete_input_list", "delete_input") + ] assert scope.imports_all("iso_c_binding") is False - assert list(scope.mappings("iso_c_binding")) == ["c_int", "c_double"] assert [(item.source, item.target) for item in scope.mappings("iso_c_binding")] == [ ("c_int", None), ("c_double", None), ] - - -def test_unknown_no_colon_declarations_raise_in_metadata_scopes(): - module_code = """ -module bad_mod - weirdtype value -end module bad_mod -""" - type_code = """ -module bad_type_mod - type :: bad_type - weirdtype value - end type bad_type -end module bad_type_mod -""" - - with pytest.raises(FortranParseError, match="Unknown or unsupported datatype declaration in module"): - parse_fortran_file(module_code, filename="bad_mod.f90") - with pytest.raises(FortranParseError, match="Unknown or unsupported datatype declaration in type"): - parse_fortran_file(type_code, filename="bad_type.f90") + assert [item.local_name for item in scope.mappings("constants_mod")] == ["rk", "ik"] def test_declaration_and_execution_edge_branches_from_inline_fortran(): @@ -223,15 +199,12 @@ def test_cross_file_kind_resolution_for_arguments_results_and_local_parameters() } ) - result_proc = project.procedures["solver_mod.make_value"] - local_proc = project.procedures["solver_mod.use_local"] + assert project.procedures["solver_mod.make_value"].result.kind == "8" + assert project.procedures["solver_mod.use_local"].arguments[0].shape == ["1:4"] - assert result_proc.result.kind == "8" - assert local_proc.arguments[0].shape == ["1:4"] - -def test_local_kind_parameter_chain_resolves_to_final_integer_kind(): - parsed = parse_fortran_file( +def test_local_parameter_chains_resolve_kinds_and_shapes(): + modern = parse_fortran_file( """ subroutine consume(x, y) integer, parameter :: word = 4 @@ -243,11 +216,26 @@ def test_local_kind_parameter_chain_resolves_to_final_integer_kind(): end subroutine consume """, filename="local_kind_chain.f90", - ) - args = {arg.name: arg for arg in parsed.procedures[0].arguments} + ).procedures[0] + dependent = parse_fortran_file( + """ +subroutine sized(c, d, x) + character(len=4) :: c*(8), d + integer, parameter :: m = 3, k = m + 1 + real :: x(k) +end subroutine sized +""", + filename="dependent_parameters.f90", + ).procedures[0] + legacy = parse_fortran_file( + " subroutine loose(x, y)\n parameter (ival = 2, alpha = 1.0)\n real x(ival), y(ival+1)\n end\n", + filename="loose.f", + ).procedures[0] - assert args["x"].kind == "8" - assert args["y"].kind == "16" + assert [arg.kind for arg in modern.arguments] == ["8", "16"] + assert [arg.shape for arg in legacy.arguments] == [["2"], ["3"]] + # A parameter defined through another resolves; an entity length stays on its entity. + assert [(arg.kind, arg.shape) for arg in dependent.arguments] == [("len=4", ["8"]), ("len=4", []), ("", ["4"])] def test_local_compile_time_arithmetic_is_folded_for_shapes_and_parameters(): @@ -277,135 +265,133 @@ def test_local_compile_time_arithmetic_is_folded_for_shapes_and_parameters(): ] -def test_type_contains_accepts_bindings_and_rejects_other_lines(): - valid_code = """ -module type_contains_valid_mod - type :: state - contains - procedure :: update - final :: destroy - end type state -end module type_contains_valid_mod -""" - - parsed = parse_fortran_file(valid_code, filename="type_contains_valid.f90") - dtype = parsed.modules[0].derived_types[0] - assert dtype.methods == ["update"] - assert dtype.final_procedures == ["destroy"] - - for invalid_line in ("call ignored_statement()", "!$omp declare target", "integer, public :: bad_binding"): - code = f""" -module type_contains_bad_mod - type :: state - contains - {invalid_line} - end type state -end module type_contains_bad_mod -""" - with pytest.raises(FortranParseError, match="Unsupported or malformed type-bound declaration"): - parse_fortran_file(code, filename="type_contains_bad.f90") - - -def test_contains_alternative_line_validation_accepts_spec_lines_without_mutating_scope(): - parser = FortranParser() - module = FortranModule("alternative_mod") - scope = _ParserScope(kind="module", name=module.name, model=module, module_owner=module.name) - - assert parser._helper_is_valid_contains_alternative_line(scope, "integer :: fallback") is True - assert parser._helper_is_valid_contains_alternative_line(scope, "call fallback()") is False - assert parser._helper_is_valid_contains_alternative_line(scope, "@@@") is False - assert module.variables == [] - - -def test_use_statement_empty_only_items_are_ignored(): - code = """ -module use_empty_items_mod - use constants_mod, only: rk, , ik - integer :: value -end module use_empty_items_mod -""" - - module = parse_fortran_file(code, filename="use_empty_items.f90").modules[0] - - assert [item.local_name for item in ScopeUses(module.uses).mappings("constants_mod")] == ["rk", "ik"] - +def test_local_parameters_of_one_procedure_do_not_leak_into_a_sibling(): + project = parse_fortran_project( + { + "dims.f90": """ +module dims_mod + integer, parameter :: n = 3 +contains + subroutine a() + integer, parameter :: n = 9 + end subroutine a -def test_type_field_spec_variants_and_empty_entities_from_public_source(): - code = """ -module type_field_edges_mod - type :: state - sequence - private - integer :: first, , second - end type state -end module type_field_edges_mod + subroutine b(x) + real, intent(inout) :: x(1:n) + end subroutine b +end module dims_mod """ + } + ) - dtype = parse_fortran_file(code, filename="type_field_edges.f90").modules[0].derived_types[0] - - assert [field.name for field in dtype.fields] == ["first", "second"] - assert dtype.attributes == ["sequence"] + assert project.procedures["dims_mod.b"].arguments[0].shape[0] in {"1:n", "1:3"} -@pytest.mark.parametrize("invalid_line", ["type :: nested_marker", "call invalid_in_type_spec()"]) -def test_type_field_specification_rejects_invalid_nested_syntax(invalid_line): - code = f""" -module type_field_invalid_mod - type :: state - {invalid_line} +def test_type_bound_bindings_and_final_procedures_are_recorded(): + dtype = ( + parse_fortran_file( + """ +module type_contains_valid_mod + type :: parent + end type parent + type, extends(parent), public :: state + contains + procedure, pass(self), public :: update, reset + generic, public :: assignment(=) => assign_child, assign_other + FINAL :: cleanup, destroy end type state -end module type_field_invalid_mod -""" +end module type_contains_valid_mod +""", + filename="type_contains_valid.f90", + ) + .modules[0] + .derived_types[1] + ) - with pytest.raises(FortranParseError): - parse_fortran_file(code, filename="type_field_invalid.f90") + assert dtype.extends.name == "parent" + assert dtype.attributes == ["public"] + assert dtype.methods == ["update", "reset"] + assert dtype.procedure_bindings == [ + {"name": "update", "attrs": ["pass(self)", "public"], "visibility": "public"}, + {"name": "reset", "attrs": ["pass(self)", "public"], "visibility": "public"}, + ] + assert dtype.generic_bindings == [ + { + "name": "assignment(=)", + "targets": ["assign_child", "assign_other"], + "attrs": ["public"], + "visibility": "public", + } + ] + assert dtype.final_procedures == ["cleanup", "destroy"] -def test_module_like_declaration_edges_from_program_and_module_sources(): - module_code = """ -module module_spec_edges_mod - module procedure :: ignored_impl - integer :: first, , second -end module module_spec_edges_mod -""" +def test_type_accessibility_statements_set_component_and_binding_defaults(): + """A type's `private` statement is a default, not an unsupported declaration. - program_code = """ -program type_stmt_program - type :: local_state - integer :: marker - end type local_state - integer :: kept -end program type_stmt_program + The statement before `contains` sets component accessibility; the statement + after it sets type-bound accessibility. Each declaration that states its own + accessibility keeps it. + """ + module = parse_fortran_file( + """ +module access_mod + implicit none + type,public :: t + private + integer :: hidden = 0 + integer,public :: shown = 0 + contains + private + procedure :: internal_step + procedure,public :: step => internal_step + end type t +contains + subroutine internal_step(self) + class(t),intent(inout) :: self + end subroutine internal_step +end module access_mod """ - - module = FortranParser().parse_module( - module_code, - filename="module_like_edges.f90", - ) - program = parse_fortran_file(program_code, filename="module_like_edges.f90").programs[0] - - assert [var.name for var in module.variables] == ["first", "second"] - assert [var.name for var in program.variables] == ["kept"] + ).modules[0] + + dtype = module.derived_types[0] + assert dtype.component_visibility == "private" + assert dtype.binding_visibility == "private" + assert {field.name: field.visibility for field in dtype.fields} == { + "hidden": "private", + "shown": "public", + } + assert [(binding["name"], binding["visibility"]) for binding in dtype.procedure_bindings] == [ + ("internal_step", "private"), + ("step => internal_step", "public"), + ] -def test_public_parse_paths_ignore_empty_declaration_entities(): - parsed = parse_fortran_file( +def test_deferred_type_bound_binding_records_its_declaring_interface(): + """A deferred binding parses; whether it can be wrapped belongs to policy.""" + module = parse_fortran_file( """ -module empty_entity_mod - integer :: kept, , also_kept - - type :: state - real :: x, , y - end type state -end module empty_entity_mod -""", - filename="empty_entities.f90", - ) - - module = parsed.modules[0] +module deferred_mod + implicit none + type,public,abstract :: base + contains + procedure(size_func),deferred,public :: size_of + end type base + abstract interface + pure function size_func(self) result(s) + import :: base + class(base),intent(in) :: self + integer :: s + end function size_func + end interface +end module deferred_mod +""" + ).modules[0] - assert [var.name for var in module.variables] == ["kept", "also_kept"] - assert [field.name for field in module.derived_types[0].fields] == ["x", "y"] + binding = module.derived_types[0].procedure_bindings[0] + assert binding["name"] == "size_of" + assert binding["interface"] == "size_func" + assert "deferred" in binding["attrs"] def test_nested_interface_procedure_without_matching_dummy_stays_publicly_parseable(): diff --git a/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py b/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py index b19e37626..a7e586502 100644 --- a/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py +++ b/tests/fortran/infrastructure/parsing/test_declaration_and_scope_regressions.py @@ -1,306 +1,10 @@ -"""Tests split by stable ownership concept from `test_source_form_and_diagnostics_regressions.py`.""" +"""Project assembly: file discovery, dependency order, registries, and scope rules.""" -import pytest from pathlib import Path -from prik.parsers.fortran import FortranParseError, parse_fortran_file -from prik.parsers.fortran.models import ( - FortranArgument, - FortranDerivedType, - FortranModule, - FortranProcedureSignature, -) -from prik.parsers.fortran.parser import ( - _Declaration, - FortranParser, - _ParserScope, - parse_fortran_project, -) - - -def test_compile_time_resolution_helpers_preserve_kind_shape_values_and_literal_policy(): - parser = FortranParser() - signature = FortranProcedureSignature( - "consume", - "subroutine", - module="api_mod", - arguments=[FortranArgument("values", base_type="real", kind="rk_alias", shape=["n + 1"])], - ) - signature.variables["local_n"] = FortranArgument( - "local_n", - base_type="integer", - value="n + 2", - symbolic_value="n + 2", - value_type="expression", - is_parameter=True, - ) - - symbols = parser._resolve_compile_time_symbols( - { - "api_mod": {"rk": "4 + 4", "rk_alias": "rk", "n": "3"}, - } - ) - parser._resolve_procedure_compile_time_facts( - signature, - symbols, - resolve_shapes=True, - ) - - assert signature.arguments[0].kind == "8" - assert signature.arguments[0].shape == ["4"] - assert signature.variables["local_n"].value == "5" - assert signature.variables["local_n"].symbolic_value == "n + 2" - - module = FortranModule("api_mod") - module.variables.append( - FortranArgument( - "values", - base_type="real", - kind="rk_alias", - shape=["0:n"], - value="n + 2", - symbolic_value="n + 2", - value_type="expression", - is_parameter=True, - ) - ) - parser._resolve_module_like_compile_time_facts(module, symbols) - - assert module.variables[0].kind == "8" - assert module.variables[0].shape == ["0:3"] - assert module.variables[0].lbound == ["0"] - assert module.variables[0].ubound == ["3"] - assert module.variables[0].value == "5" - - assert parser._resolve_kind_expression("len=n + 1", {"n": "3"}) == "len=4" - assert parser._resolve_symbol_reference("alias", {"alias": "target", "target": "8"}) == "8" - resolved = parser._resolve_compile_time_symbols( - {"M": {"a": "4", "b": "a + 2", "rk": "selected_real_kind(12)", "dp": "rk"}} - ) - assert dict(resolved.in_module("m")) == { - "a": "4", - "b": "6", - "rk": "selected_real_kind(12)", - "dp": "selected_real_kind(12)", - } - assert parser._collect_relevant_local_params( - FortranProcedureSignature( - "shape", - "subroutine", - arguments=[FortranArgument("values", kind="rk", shape=["n"])], - ), - {"rk": "base", "base": "4", "n": "m + 1", "m": "3", "unused": "10"}, - ) == {"rk": "base", "base": "4", "n": "m + 1", "m": "3"} - assert parser._extract_symbol_names("n + max(m, 2) .and. flag") == {"n", "max", "m", "flag"} - assert parser._normalize_parameter_value("2.0d+0") == "2" - assert parser._normalize_parameter_value("selected_real_kind(12)") is None - assert parser._is_literal_parameter_value("(/ 1, 2, .true. /)") is True - assert parser._safe_eval_int_expr("max(3, 7) + len_trim('abc ')") == 10 - assert parser._infer_implicit_base_type("index") == "integer" - assert parser._infer_implicit_base_type("alpha") == "real" - - -def test_declaration_storage_preserves_module_variables_parameters_visibility_and_bounds(): - parser = FortranParser() - module = FortranModule("owner_mod") - scope = _ParserScope(kind="module", name=module.name, model=module, module_owner=module.name) - declaration = parser._new_declaration("real", "rk") - parser._apply_declaration_attributes( - declaration, - ["parameter", "private", "dimension(0:n)"], - ) - - parser._store_declaration( - scope, - declaration=declaration, - right="weights = 1.0_rk", - role="module_variable", - filename="declarations.f90", - lineno=4, - source_line="real(kind=rk), parameter, dimension(0:n) :: weights = 1.0_rk", - ) - - assert [(var.name, var.base_type, var.kind, var.shape, var.lbound, var.ubound) for var in module.variables] == [ - ("weights", "real", "rk", ["0:n"], ["0"], ["n"]) - ] - assert module.variables[0].is_parameter is True - assert module.variables[0].value == "1" - assert module.variables[0].symbolic_value == "1.0_rk" - assert module.variables[0].value_type == "expression" - assert module.private_symbols == ["weights"] - - -def test_procedure_declaration_storage_updates_dummy_or_records_local_type_and_duplicate_metadata(): - parser = FortranParser() - signature = FortranProcedureSignature( - "apply", - "subroutine", - arguments=[FortranArgument("callback"), FortranArgument("value")], - ) - state = parser._new_procedure_scope_state( - signature, - symbols={argument.name.lower(): argument for argument in signature.arguments}, - ) - scope = _ParserScope(kind="procedure", name=signature.name, model=signature, state=state) - procedure_declaration = parser._new_declaration("procedure", "callback_iface") - procedure_declaration.external = True - - parser._store_declaration( - scope, - declaration=procedure_declaration, - right="callback", - role="procedure_symbol", - filename="declarations.f90", - lineno=9, - source_line="procedure(callback_iface), external :: callback", - ) - local_declaration = parser._new_declaration("real", "rk") - parser._store_declaration( - scope, - declaration=local_declaration, - right="scratch", - role="procedure_symbol", - filename="declarations.f90", - lineno=10, - source_line="real(kind=rk) :: scratch", - ) - - assert signature.arguments[0].base_type == "procedure" - assert signature.arguments[0].kind == "callback_iface" - assert state.external_symbols == {"callback"} - assert state.declared_local_types == {"scratch": _Declaration(base_type="real", kind="rk")} - - with pytest.raises(FortranParseError) as error: - parser._store_declaration( - scope, - declaration=parser._new_declaration("integer", ""), - right="callback", - role="procedure_symbol", - filename="declarations.f90", - lineno=11, - source_line="integer :: callback", - ) - - assert error.value.base_message == "Duplicate declaration of symbol 'callback' in procedure 'apply'." - assert error.value.filename == "declarations.f90" - assert error.value.line_number == 11 - assert error.value.source_line == "integer :: callback" - assert error.value.code == "PARSE_DUPLICATE_DECLARATION" - - -def test_entity_character_length_uses_an_independent_typed_declaration(): - parser = FortranParser() - declaration = parser._new_declaration("character", "default_len") - - entity_declaration = parser._entity_declaration("label*(name_len)", declaration) - - assert entity_declaration is not declaration - assert entity_declaration.kind == "name_len" - assert entity_declaration.character_length_syntax is True - assert declaration.kind == "default_len" - assert declaration.character_length_syntax is False - - -def test_procedure_finalization_consumes_typed_local_declarations(): - parser = FortranParser() - signature = FortranProcedureSignature( - "consume", - "subroutine", - arguments=[FortranArgument("value")], - ) - state = parser._new_procedure_scope_state( - signature, - symbols={"value": signature.arguments[0]}, - ) - state.declared_local_types["value"] = _Declaration( - base_type="real", - kind="rk", - declared_storage_bits=64, - ) - state.declared_local_types["n"] = _Declaration( - base_type="integer", - kind="i4", - target_kind_expression="kind(1)", - ) - - parser._reconcile_procedure_local_declarations(signature, state) - parser._materialize_procedure_parameters(signature, state, {"n": "4"}) - assert signature.arguments[0].base_type == "real" - assert signature.arguments[0].kind == "rk" - assert signature.arguments[0].declared_storage_bits == 64 - assert signature.variables["n"].base_type == "integer" - assert signature.variables["n"].kind == "i4" - assert signature.variables["n"].target_kind_expression == "kind(1)" +import pytest - -def test_procedure_parameter_lines_preserve_local_parameter_state_and_duplicate_metadata(): - parser = FortranParser() - signature = FortranProcedureSignature("shape", "subroutine", arguments=[FortranArgument("values")]) - state = parser._new_procedure_scope_state( - signature, - symbols={"values": signature.arguments[0]}, - ) - - assert parser._handle_proc_parameter_line( - "integer, parameter :: n = 4, m = n + 2", - state, - filename="parameters.f90", - lineno=5, - source_line="integer, parameter :: n = 4, m = n + 2", - ) - assert state.local_params == {"n": "4", "m": "n + 2"} - assert state.legacy_local_params == set() - assert state.implicit_typed_symbols == {} - - with pytest.raises(FortranParseError) as error: - parser._handle_proc_parameter_line( - "integer, parameter :: n = 8", - state, - filename="parameters.f90", - lineno=6, - source_line="integer, parameter :: n = 8", - ) - - assert error.value.base_message == "Duplicate PARAMETER declaration of symbol 'n' in procedure 'shape'." - assert error.value.filename == "parameters.f90" - assert error.value.line_number == 6 - assert error.value.source_line == "integer, parameter :: n = 8" - assert error.value.code == "PARSE_DUPLICATE_PARAMETER" - - -def test_legacy_parameter_lines_respect_implicit_none_and_implicit_typing_contracts(): - parser = FortranParser() - strict_signature = FortranProcedureSignature("strict", "subroutine") - strict_state = parser._new_procedure_scope_state(strict_signature, symbols={}) - strict_state.implicit_none = True - - with pytest.raises(FortranParseError) as error: - parser._handle_proc_parameter_line( - "parameter (zero = 0.0e+0)", - strict_state, - filename="parameters.f90", - lineno=8, - source_line="parameter (zero = 0.0e+0)", - ) - - assert error.value.base_message == "Unknown datatype for PARAMETER symbol 'zero' in procedure 'strict'." - assert error.value.filename == "parameters.f90" - assert error.value.line_number == 8 - assert error.value.source_line == "parameter (zero = 0.0e+0)" - assert error.value.code == "PARSE_UNKNOWN_PARAMETER_TYPE" - - loose_signature = FortranProcedureSignature("loose", "subroutine") - loose_state = parser._new_procedure_scope_state(loose_signature, symbols={}) - assert parser._handle_proc_parameter_line( - "parameter (ival = 2, alpha = 1.0)", - loose_state, - filename="parameters.f90", - lineno=9, - source_line="parameter (ival = 2, alpha = 1.0)", - ) - assert loose_state.local_params == {"ival": "2", "alpha": "1.0"} - assert loose_state.implicit_typed_symbols == {"ival": "integer", "alpha": "real"} - assert loose_state.legacy_local_params == set() +from prik.parsers.fortran import FortranParseError, FortranParser, parse_fortran_file, parse_fortran_project def test_directory_project_parses_once_and_assembles_dependency_ordered_models(tmp_path: Path, monkeypatch): @@ -531,213 +235,47 @@ def test_parse_file_preserves_top_level_models_but_limits_file_symbol_registry() assert parsed.symbols["global_step"] is parsed.procedures[0] -def test_parse_project_resolves_cross_file_used_module_parameters_once(): - project = parse_fortran_project( - { - "precision.f90": """ -module precision_mod - integer, parameter :: rk = 8 -end module precision_mod -""", - "api.f90": """ -module api_mod - use precision_mod -contains - subroutine consume(value) - real(kind=rk), intent(in) :: value - end subroutine consume -end module api_mod -""", - } - ) - - procedure = project.procedures["api_mod.consume"] - assert procedure is project.procedures["consume"] - assert procedure.arguments[0].kind == "8" - assert project.dependencies == { - "api_mod": {"precision_mod"}, - "precision_mod": set(), - } - - -def test_parse_project_rejects_duplicate_modules_across_files_with_project_scope_metadata(): +@pytest.mark.parametrize( + ("unit", "scope"), + [ + pytest.param("module shared\nend module shared\n", "module", id="module"), + pytest.param("subroutine shared()\nend subroutine shared\n", "procedure", id="procedure"), + pytest.param("program shared\nend program shared\n", "program", id="program"), + ], +) +def test_parse_project_rejects_duplicate_units_across_files_with_project_scope_metadata(unit, scope): with pytest.raises(FortranParseError) as duplicate: - parse_fortran_project( - { - "first.f90": "module shared_mod\nend module shared_mod\n", - "second.f90": "module shared_mod\nend module shared_mod\n", - } - ) - - assert duplicate.value.base_message == "Duplicate symbol 'shared_mod' in project module scope." - assert duplicate.value.filename is None - assert duplicate.value.line_number is None - assert duplicate.value.source_line is None + parse_fortran_project({"first.f90": unit, "second.f90": unit}) + + assert duplicate.value.base_message == f"Duplicate symbol 'shared' in project {scope} scope." + assert (duplicate.value.filename, duplicate.value.line_number, duplicate.value.source_line) == (None, None, None) assert duplicate.value.code == "PARSE_DUPLICATE_SYMBOL" -def test_project_topological_files_are_dependency_first_sorted_and_cycle_tolerant(): - ordered = FortranParser._topological_files( +def test_project_with_cyclic_module_uses_still_parses_every_file(): + project = parse_fortran_project( { - "consumer.f90": {"module_a.f90", "module_b.f90"}, - "module_b.f90": set(), - "module_a.f90": set(), - "cycle_left.f90": {"cycle_right.f90"}, - "cycle_right.f90": {"cycle_left.f90"}, + "left.f90": "module left_mod\n use right_mod\nend module left_mod\n", + "right.f90": "module right_mod\n use left_mod\nend module right_mod\n", } ) - assert ordered[:3] == ["module_a.f90", "module_b.f90", "consumer.f90"] - assert ordered[3:] == ["cycle_left.f90", "cycle_right.f90"] + assert [parsed.filename for parsed in project.files] == ["left.f90", "right.f90"] + assert project.dependencies == {"left_mod": {"right_mod"}, "right_mod": {"left_mod"}} -def test_project_encoding_is_forwarded_to_explicit_path_inputs(tmp_path: Path): +@pytest.mark.parametrize("as_directory", [False, True], ids=["path-list", "directory"]) +def test_project_encoding_is_forwarded_to_file_parsing(tmp_path: Path, as_directory: bool): source = tmp_path / "latin1.f90" source.write_bytes("! caf\xe9\nmodule encoded_mod\nend module encoded_mod\n".encode("latin-1")) - project = parse_fortran_project([source], encoding="latin-1") - - assert project.files[0].encoding == "latin-1" - assert project.files[0].source.startswith("! caf\xe9") - - -def test_project_encoding_is_forwarded_to_directory_file_parsing(tmp_path: Path): - source = tmp_path / "latin1.f90" - source.write_bytes("! caf\xe9\nmodule encoded_mod\nend module encoded_mod\n".encode("latin-1")) - - project = parse_fortran_project(tmp_path, encoding="latin-1") + project = parse_fortran_project(tmp_path if as_directory else [source], encoding="latin-1") assert set(project.modules) == {"encoded_mod"} assert project.files[0].encoding == "latin-1" assert project.files[0].source.startswith("! caf\xe9") -def test_scope_include_import_and_derived_type_binding_contracts(): - parser = FortranParser() - state = parser._new_procedure_scope_state( - FortranProcedureSignature("scope_contract", "subroutine"), - symbols={}, - ) - parser._proc_scope_add_include(state, "shared.inc") - parser._proc_scope_add_imports(state, ["State_T", " ", "Callback"]) - - assert state.includes == ["shared.inc"] - assert state.imports == {"state_t", "callback"} - - dtype = parser._init_derived_type( - "type, extends(parent(kind)), public :: child", - current_module="owner_mod", - ) - assert dtype == FortranDerivedType( - name="child", - module="owner_mod", - extends="parent(kind)", - attributes=["public"], - ) - malformed = parser._init_derived_type("type, extends(parent :: child", current_module="owner_mod") - assert malformed == FortranDerivedType( - name="child", - module="owner_mod", - attributes=["extends(parent"], - ) - - parser._parse_derived_type_contains_line("procedure, pass(self), public :: update, reset", dtype) - parser._parse_derived_type_contains_line("generic, public :: assignment(=) => assign_child, assign_other", dtype) - parser._parse_derived_type_contains_line("FINAL :: cleanup, destroy", dtype) - - assert dtype.methods == ["update", "reset"] - assert dtype.procedure_bindings == [ - {"name": "update", "attrs": ["pass(self)", "public"], "visibility": "public"}, - {"name": "reset", "attrs": ["pass(self)", "public"], "visibility": "public"}, - ] - assert dtype.generic_bindings == [ - { - "name": "assignment(=)", - "targets": ["assign_child", "assign_other"], - "attrs": ["public"], - "visibility": "public", - } - ] - - -def test_unknown_procedure_declaration_kind_preserves_declaration_and_invalid_syntax_split(): - parser = FortranParser() - state = parser._new_procedure_scope_state( - FortranProcedureSignature(name="work", kind="subroutine"), - symbols={}, - ) - - with pytest.raises(FortranParseError) as declaration_error: - parser._handle_unknown_proc_declaration( - "vector(kind=4) :: value", - state, - filename="procedure_contract.f90", - lineno=9, - source_line="vector(kind=4) :: value", - ) - - assert ( - declaration_error.value.base_message - == "Unknown or unsupported datatype declaration for procedure 'work': vector(kind=4) :: value" - ) - assert declaration_error.value.filename == "procedure_contract.f90" - assert declaration_error.value.line_number == 9 - assert declaration_error.value.source_line == "vector(kind=4) :: value" - assert declaration_error.value.code == "PARSE_UNSUPPORTED_DECLARATION" - - with pytest.raises(FortranParseError) as syntax_error: - parser._handle_unknown_proc_declaration( - "call work()", - state, - filename="procedure_contract.f90", - lineno=10, - source_line="call work()", - ) - - assert ( - syntax_error.value.base_message == "Invalid Fortran syntax in procedure 'work' specification part: call work()" - ) - assert syntax_error.value.filename == "procedure_contract.f90" - assert syntax_error.value.line_number == 10 - assert syntax_error.value.source_line == "call work()" - assert syntax_error.value.code == "PARSE_INVALID_SYNTAX" - - -def test_derived_type_collection_retains_sibling_and_nested_scope_contexts(): - parser = FortranParser() - types = parser._collect_derived_type_source_units( - """ -type :: global_state -end type global_state -module owner_mod - type :: first_state - end type first_state - type :: second_state - end type second_state -contains - subroutine work() - type :: local_state - end type local_state - end subroutine work -end module owner_mod -""", - filename="nested_types.f90", - ) - - assert [(unit.name, scope.kind, scope.name, scope.module_owner) for unit, scope in types] == [ - ("global_state", "file", None, None), - ("first_state", "module", "owner_mod", "owner_mod"), - ("second_state", "module", "owner_mod", "owner_mod"), - ("local_state", "procedure", "work", "owner_mod"), - ] - assert [(scope.parent.kind, scope.parent.name) if scope.parent else None for _unit, scope in types] == [ - None, - ("file", None), - ("file", None), - ("module", "owner_mod"), - ] - - @pytest.mark.parametrize( ("statement", "forbids_typing"), [ diff --git a/tests/fortran/infrastructure/parsing/test_derived_types_and_program_units.py b/tests/fortran/infrastructure/parsing/test_derived_types_and_program_units.py deleted file mode 100644 index 1c6a5bfe1..000000000 --- a/tests/fortran/infrastructure/parsing/test_derived_types_and_program_units.py +++ /dev/null @@ -1,350 +0,0 @@ -"""Tests split by stable ownership concept from `test_procedures_and_interfaces.py`.""" - -import pytest -from prik.parsers.fortran import FortranParseError, parse_fortran_file, parse_fortran_project -from tests.fortran._support.parser_procedures import ( - parse_fortran_block_data_unit, - parse_fortran_module, - parse_fortran_modules, - parse_fortran_program, - parse_fortran_submodule, -) - - -def test_duplicate_symbols_are_reported_from_inline_fortran(): - with pytest.raises(FortranParseError, match="Duplicate variable 'x' in module 'dup_mod'"): - parse_fortran_file( - """ -module dup_mod - integer :: x - real :: x -end module dup_mod -""" - ) - - with pytest.raises(FortranParseError, match="Duplicate field 'id' in derived type 'particle'"): - parse_fortran_file( - """ -module dup_type_mod - type :: particle - integer :: id - real :: id - end type particle -end module dup_type_mod -""" - ) - - with pytest.raises(FortranParseError, match="Duplicate argument name 'x' in procedure 'dup_arg'"): - parse_fortran_file( - """ -subroutine dup_arg(x, x) - integer, intent(in) :: x -end subroutine dup_arg -""" - ) - - -def test_fixed_form_fortran77_continuation(): - code = """ - subroutine saxpy(n,x,y,a) - integer n - real x(n),y(n),a - do 10 i=1,n - 1y(i)=y(i)+a*x(i) - 10 continue - end -""" - sigs = parse_fortran_file(code, filename="legacy.f").procedures - assert len(sigs) == 1 - assert sigs[0].name == "saxpy" - assert sigs[0].arguments[0].base_type == "integer" - assert sigs[0].arguments[1].base_type == "real" - assert sigs[0].arguments[1].rank == 1 - - -def test_unknown_datatype_for_argument_crashes_parser(): - code = """ -subroutine bad(x) - weirdtype :: x -end subroutine bad -""" - with pytest.raises(ValueError, match="Unknown or unsupported datatype"): - _ = parse_fortran_file(code, filename="bad.f90").procedures - - -def test_parse_fortran_file_returns_file_model_for_source_string(): - code = """ -module file_mod -contains -subroutine ping(x) - integer, intent(in) :: x -end subroutine ping -end module file_mod -""" - parsed = parse_fortran_file(code) - assert parsed.filename is None - assert [m.name for m in parsed.modules] == ["file_mod"] - assert [p.name for p in parsed.modules[0].procedures] == ["ping"] - - -def test_parse_fortran_project_returns_project_registry(): - project = parse_fortran_project( - { - "a.f90": """ -module a_mod -contains -subroutine step(x) - integer, intent(in) :: x -end subroutine step -end module a_mod -""", - "b.f90": """ -subroutine free_proc(y) - real, intent(in) :: y -end subroutine free_proc -""", - } - ) - assert [f.filename for f in project.files] == ["a.f90", "b.f90"] - assert "a_mod" in project.modules - assert "a_mod.step" in project.procedures - assert "free_proc" in project.procedures - - -def test_parse_fortran_project_accepts_directory_and_orders_dependencies(tmp_path): - (tmp_path / "10_solver.f90").write_text( - """ -module solver_mod - use kinds_mod, only: rk -contains - subroutine step(x) - real(kind=rk), intent(inout) :: x(:) - end subroutine step -end module solver_mod -""", - encoding="utf-8", - ) - (tmp_path / "00_kinds.f90").write_text( - """ -module kinds_mod - integer, parameter :: rk = selected_real_kind(15, 307) -end module kinds_mod -""", - encoding="utf-8", - ) - (tmp_path / "20_driver.f90").write_text( - """ -program driver - use solver_mod - real :: x(4) -end program driver -""", - encoding="utf-8", - ) - (tmp_path / "30_init.f90").write_text( - """ -block data init_data - integer :: seed -end block data init_data -""", - encoding="utf-8", - ) - - project = parse_fortran_project(tmp_path) - - assert len(project.files) == 4 - assert "kinds_mod" in project.modules - assert "solver_mod" in project.modules - assert "driver" in project.programs - assert "solver_mod.step" in project.procedures - solver = project.procedures["solver_mod.step"] - assert solver.arguments[0].kind == "selected_real_kind(15, 307)" - - -def test_singular_parse_entrypoints_return_single_models(): - assert ( - parse_fortran_file(""" -subroutine one(x) - integer, intent(in) :: x -end subroutine one -""") - .procedures[0] - .name - == "one" - ) - - assert ( - parse_fortran_module(""" -module single_mod -end module single_mod -""").name - == "single_mod" - ) - - assert ( - parse_fortran_file(""" -module type_mod - type :: particle - integer :: id - end type particle -end module type_mod -""") - .modules[0] - .derived_types[0] - .name - == "particle" - ) - - assert ( - parse_fortran_file(""" -module iface_mod - interface apply - subroutine do_apply(x) - integer, intent(in) :: x - end subroutine do_apply - end interface -end module iface_mod -""") - .modules[0] - .interfaces[0] - .name - == "apply" - ) - - assert ( - parse_fortran_program(""" -program driver - integer :: ierr -end program driver -""").name - == "driver" - ) - - assert ( - parse_fortran_block_data_unit(""" -block data init_data - integer :: seed -end block data init_data -""").name - == "init_data" - ) - - assert ( - parse_fortran_submodule(""" -submodule (parent_mod) child_impl -end submodule child_impl -""").name - == "child_impl" - ) - - -def test_singular_parse_entrypoint_rejects_ambiguous_sources(): - assert ( - len( - parse_fortran_modules(""" -module first_mod -end module first_mod -module second_mod -end module second_mod -""") - ) - == 2 - ) - - parsed = parse_fortran_file(""" -subroutine first() -end subroutine first -subroutine second() -end subroutine second -""") - assert len(parsed.procedures) == 2 - - -def test_type_accessibility_statements_set_component_and_binding_defaults(): - """A type's `private` statement is a default, not an unsupported declaration. - - The statement before `contains` sets component accessibility; the statement - after it sets type-bound accessibility. Each declaration that states its own - accessibility keeps it. - """ - module = parse_fortran_module( - """ -module access_mod - implicit none - type,public :: t - private - integer :: hidden = 0 - integer,public :: shown = 0 - contains - private - procedure :: internal_step - procedure,public :: step => internal_step - end type t -contains - subroutine internal_step(self) - class(t),intent(inout) :: self - end subroutine internal_step -end module access_mod -""" - ) - - dtype = module.derived_types[0] - assert dtype.component_visibility == "private" - assert dtype.binding_visibility == "private" - assert {field.name: field.visibility for field in dtype.fields} == { - "hidden": "private", - "shown": "public", - } - assert [(binding["name"], binding["visibility"]) for binding in dtype.procedure_bindings] == [ - ("internal_step", "private"), - ("step => internal_step", "public"), - ] - - -def test_deferred_type_bound_binding_records_its_declaring_interface(): - """A deferred binding parses; whether it can be wrapped belongs to policy.""" - module = parse_fortran_module( - """ -module deferred_mod - implicit none - type,public,abstract :: base - contains - procedure(size_func),deferred,public :: size_of - end type base - abstract interface - pure function size_func(self) result(s) - import :: base - class(base),intent(in) :: self - integer :: s - end function size_func - end interface -end module deferred_mod -""" - ) - - binding = module.derived_types[0].procedure_bindings[0] - assert binding["name"] == "size_of" - assert binding["interface"] == "size_func" - assert "deferred" in binding["attrs"] - - -def test_named_block_construct_starts_the_execution_part(): - """`name: block` is an executable construct, not a declaration.""" - module = parse_fortran_module( - """ -module block_mod - implicit none -contains - subroutine scale_value(x) - real(8),intent(inout) :: x - main: block - real(8) :: factor - factor = 2.0d0 - x = x * factor - end block main - end subroutine scale_value -end module block_mod -""" - ) - - assert [procedure.name for procedure in module.procedures] == ["scale_value"] diff --git a/tests/fortran/infrastructure/parsing/test_developer_tutorial.py b/tests/fortran/infrastructure/parsing/test_developer_tutorial.py deleted file mode 100644 index 835f7bb4a..000000000 --- a/tests/fortran/infrastructure/parsing/test_developer_tutorial.py +++ /dev/null @@ -1,86 +0,0 @@ -"""Executable developer tutorial for the grammar-style parser internals. - -This test is intentionally written as a small walkthrough rather than as a -black-box public API test. It shows the private visitor/scanner sequence that -maintainers should follow when changing `prik/parsers/fortran/parser.py`: - -1. preprocess, then scan fully classified file-level source units, -2. inspect the scanner-owned grammar regions and direct children, -3. visit the unit with a scope, -4. inspect a retained child without rescanning its parent's source. -""" - -from prik.parsers.fortran.parser import FortranParser, _SourceUnitScanner - - -def test_developer_tutorial_recursive_unit_visitors_and_helpers(): - source = "\n".join( - [ - "module dims_mod", - " implicit none", - " integer, parameter :: n = 8", - "contains", - " function total(values) result(out)", - " implicit none", - " real, intent(in) :: values(n)", - " real :: out", - " end function total", - "end module dims_mod", - "", - ] - ) - - parser = FortranParser() - scanner = _SourceUnitScanner() - - lines, root_scope, top_units = parser._helper_prepare_source_units( - source, - filename="developer_tutorial.f90", - ) - assert [line[1] for line in lines[:3]] == [1, 2, 3] - assert [(unit.kind, unit.name, unit.start_line, unit.end_line) for unit in top_units] == [ - ("module", "dims_mod", 1, 10) - ] - - module_unit = top_units[0] - module_grammar = scanner.grammar("module") - assert module_grammar.has_contains_part is True - assert module_unit.header == module_unit.lines[0] - assert [line[0].strip() for line in module_unit.specification] == [ - "implicit none", - "integer, parameter :: n = 8", - ] - assert module_unit.contains == [] - - module = parser._visit( - module_unit, - parent_scope=root_scope, - filename="developer_tutorial.f90", - ) - assert module.name == "dims_mod" - assert module.variables[0].name == "n" - assert module.variables[0].value == "8" - assert module.variables[0].symbolic_value == "8" - - child_units = module_unit.children - assert [(unit.kind, unit.name, unit.start_line, unit.end_line) for unit in child_units] == [ - ("procedure", "total", 5, 9) - ] - assert child_units[0].parent_region == "contains" - - procedure_unit = child_units[0] - assert [line[0].strip() for line in procedure_unit.specification] == [ - "implicit none", - "real, intent(in) :: values(n)", - "real :: out", - ] - assert procedure_unit.execution == [] - assert procedure_unit.contains == [] - - proc = module.procedures[0] - assert proc.name == "total" - assert proc.arguments[0].name == "values" - assert proc.arguments[0].shape == ["n"] - assert proc.arguments[0].base_type == "real" - assert proc.result.name == "out" - assert proc.result.base_type == "real" diff --git a/tests/fortran/infrastructure/parsing/test_error_fixture_suite.py b/tests/fortran/infrastructure/parsing/test_error_fixture_suite.py deleted file mode 100644 index d161f5b73..000000000 --- a/tests/fortran/infrastructure/parsing/test_error_fixture_suite.py +++ /dev/null @@ -1,109 +0,0 @@ -import json -import os -from pathlib import Path - -import pytest - -from prik.parsers.fortran import FortranParseError, parse_fortran_file - - -def parse_fortran_procedures(source, filename=None): - return parse_fortran_file(source, filename=filename).procedures - - -def parse_fortran_types(source, filename=None): - return parse_fortran_file(source, filename=filename).derived_types - - -def parse_fortran_modules(source, filename=None): - return parse_fortran_file(source, filename=filename).modules - - -_ERRORS_DIR = Path(__file__).parent / "fixtures" / "errors" -_EXPECTED_ERRORS_DIR = _ERRORS_DIR -_PARSER_MAP = { - "parse_fortran_procedures": parse_fortran_procedures, - "parse_fortran_types": parse_fortran_types, - "parse_fortran_modules": parse_fortran_modules, -} -_ERROR_FIXTURES = sorted( - f - for f in _ERRORS_DIR.glob("*") - if f.is_file() and f.suffix.lower() in {".f", ".f90", ".f95", ".f03", ".f08", ".f77", ".for", ".ftn"} -) - - -def _load_expected_error(expected_path: Path) -> dict: - with expected_path.open("r", encoding="utf-8") as fh: - return json.load(fh) - - -def _dump_expected_error(path: Path, error_type: str, exc: FortranParseError, parser: str) -> None: - payload = { - "parser": parser, - "error_type": error_type, - "message_contains": [exc.base_message], - "diagnostic_contains": [ - f"error[{exc.code}]", - exc.base_message, - exc.source_line.strip() if exc.source_line else "", - ], - } - path.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") - - -def _run_error_fixture(fixture: Path, *, filename_for_parser: str, expected_path: Path) -> None: - source = fixture.read_text(encoding="utf-8") - assert source.strip(), f"Error fixture is empty: {filename_for_parser}" - - update_mode = os.getenv("FORTRAN_PARSER_UPDATE_GOLDENS", "0") == "1" - - if update_mode: - expected = _load_expected_error(expected_path) - parser_name = expected.get("parser", "parse_fortran_procedures") - parser_fn = _PARSER_MAP[parser_name] - try: - parser_fn(source, filename=filename_for_parser) - raise AssertionError(f"Expected FortranParseError from {fixture.name} but no error was raised") - except FortranParseError as exc: - _dump_expected_error(expected_path, "FortranParseError", exc, parser_name) - return - - expected = _load_expected_error(expected_path) - parser_name = expected["parser"] - parser_fn = _PARSER_MAP[parser_name] - error_type = expected["error_type"] - message_contains = expected["message_contains"] - diagnostic_contains = expected.get("diagnostic_contains", []) - - assert error_type == "FortranParseError", f"Unknown error_type '{error_type}' in {expected_path.name}" - - with pytest.raises(FortranParseError) as exc_info: - parser_fn(source, filename=filename_for_parser) - - err_msg = str(exc_info.value) - for fragment in message_contains: - assert fragment in err_msg, ( - f"Expected fragment {fragment!r} not found in error message for {fixture.name}.\nGot: {err_msg!r}" - ) - - diagnostic = exc_info.value.format_diagnostic(color=False) - for fragment in diagnostic_contains: - if not fragment: - continue - assert fragment in diagnostic, ( - f"Expected diagnostic fragment {fragment!r} not found for {fixture.name}.\nGot: {diagnostic!r}" - ) - - -def test_fortran_error_fixture_suite_has_fixtures(): - assert _ERROR_FIXTURES, f"No parser error fixtures found in {_ERRORS_DIR}" - - -@pytest.mark.parametrize("fixture", _ERROR_FIXTURES, ids=lambda f: f.name) -def test_fortran_error_fixture_suite(fixture): - _run_error_fixture( - fixture, - filename_for_parser=fixture.name, - expected_path=_EXPECTED_ERRORS_DIR / f"{fixture.stem}.json", - ) diff --git a/tests/fortran/infrastructure/parsing/test_error_handling.py b/tests/fortran/infrastructure/parsing/test_error_handling.py index 14f835d38..27d8ca026 100644 --- a/tests/fortran/infrastructure/parsing/test_error_handling.py +++ b/tests/fortran/infrastructure/parsing/test_error_handling.py @@ -1,696 +1,479 @@ -import pytest - -from prik.parsers.fortran import FortranParseError, parse_fortran_file - - -# --------------------------------------------------------------------------- -# FortranParseError attributes -# --------------------------------------------------------------------------- - - -# --------------------------------------------------------------------------- -# Duplicate declaration errors -# --------------------------------------------------------------------------- - - -def test_duplicate_declaration_raises_parse_error(): - code = """ -subroutine dup(x) - real :: x - integer :: x -end subroutine dup -""" - with pytest.raises(FortranParseError, match="Duplicate declaration"): - parse_fortran_file(code, filename="dup.f90") - - -def test_duplicate_function_result_raises_parse_error(): - code = """ -real function f(x) - real :: x - real :: f -end function f -""" - with pytest.raises(FortranParseError, match="Duplicate declaration"): - parse_fortran_file(code, filename="dup_result.f90") - - -def test_duplicate_function_result_with_result_keyword_raises_parse_error(): - code = """ -real function f(x) result(res) - real :: x - real :: res -end function f -""" - with pytest.raises(FortranParseError, match="Duplicate declaration"): - parse_fortran_file(code, filename="dup_result_kw.f90") - - -def test_duplicate_initialized_declaration_raises_parse_error(): - code = """ -subroutine dup_init() - integer :: x = 1 - integer :: x = 2 -end subroutine dup_init -""" - with pytest.raises(FortranParseError, match="Duplicate declaration"): - parse_fortran_file(code, filename="dup_init.f90") - - -# --------------------------------------------------------------------------- -# Duplicate procedure name errors -# --------------------------------------------------------------------------- - - -def test_duplicate_procedure_name_global_scope_raises_parse_error(): - code = """ -subroutine work(n) - integer, intent(in) :: n -end subroutine work - -function work(n) result(out) - integer, intent(in) :: n - integer :: out -end function work -""" - with pytest.raises(FortranParseError, match="Duplicate procedure name"): - parse_fortran_file(code, filename="dup_proc.f90") - - -def test_duplicate_procedure_name_in_module_raises_parse_error(): - code = """ -module m -contains - subroutine work(n) - integer, intent(in) :: n - end subroutine work - function work(n) result(out) - integer, intent(in) :: n - integer :: out - end function work -end module m -""" - with pytest.raises(FortranParseError, match="Duplicate procedure name"): - parse_fortran_file(code, filename="dup_mod_proc.f90") - - -def test_contained_procedures_with_same_name_in_different_hosts_are_allowed(): - code = """ -module m -contains - subroutine host_a() - contains - subroutine swap_order() - end subroutine swap_order - end subroutine host_a - - subroutine host_b() - contains - subroutine swap_order() - end subroutine swap_order - end subroutine host_b -end module m -""" - parsed = parse_fortran_file(code, filename="contained_scope_ok.f90") - assert [sig.name.lower() for sig in parsed.modules[0].procedures] == ["host_a", "host_b"] - - -def test_duplicate_procedure_name_error_carries_location(): - code = """ -subroutine work(n) - integer, intent(in) :: n -end subroutine work - -subroutine work(n) - integer, intent(in) :: n -end subroutine work -""" - with pytest.raises(FortranParseError) as exc_info: - parse_fortran_file(code, filename="dup.f90") - err = exc_info.value - assert err.filename == "dup.f90" - assert err.line_number is not None - - -# --------------------------------------------------------------------------- -# Star-kind declarations -# --------------------------------------------------------------------------- - - -def test_star_kind_in_modern_source_is_parsed(): - code = """ -subroutine bad(x) - real*8 :: x -end subroutine bad -""" - proc = parse_fortran_file(code, filename="bad.f90").procedures[0] - assert proc.arguments[0].base_type == "real" - assert proc.arguments[0].kind == "8" - - -def test_star_kind_in_module_variable_is_parsed(): - code = """ -module m - real*8 :: x -end module m -""" - var = parse_fortran_file(code, filename="bad.f90").modules[0].variables[0] - assert var.base_type == "real" - assert var.kind == "8" - - -# --------------------------------------------------------------------------- -# Unknown/unsupported type declaration errors -# --------------------------------------------------------------------------- - - -def test_unknown_type_in_subroutine_raises_parse_error(): - code = """ -subroutine bad(x) - weirdtype :: x -end subroutine bad -""" - with pytest.raises(FortranParseError, match="Unknown or unsupported datatype"): - parse_fortran_file(code, filename="bad.f90") - - -def test_unknown_type_in_module_variable_raises_parse_error(): - code = """ -module m - weirdtype :: x -end module m -""" - with pytest.raises(FortranParseError, match="Unknown or unsupported datatype"): - parse_fortran_file(code, filename="bad.f90") - - -def test_unknown_type_in_interface_raises_parse_error(): - code = """ -module m - interface foo - subroutine bar(x) - weirdtype :: x - end subroutine bar - end interface -end module m -""" - with pytest.raises(FortranParseError, match="Unknown or unsupported datatype"): - parse_fortran_file(code, filename="bad.f90") - - -# --------------------------------------------------------------------------- -# Module variable type validation -# --------------------------------------------------------------------------- - - -def test_module_variable_with_unknown_type_raises_parse_error(): - code = """ -module m - integer :: n -end module m -""" - parsed = parse_fortran_file(code) - assert parsed.modules[0].variables[0].base_type == "integer" - - -def test_module_variable_parsed_correctly_no_error(): - code = """ -module cfg - real(kind=8) :: tolerance - integer :: max_iter - logical :: verbose -end module cfg -""" - parsed = parse_fortran_file(code) - assert len(parsed.modules[0].variables) == 3 - assert parsed.modules[0].variables[0].base_type == "real" - assert parsed.modules[0].variables[1].base_type == "integer" - assert parsed.modules[0].variables[2].base_type == "logical" - - -# --------------------------------------------------------------------------- -# Derived type field type validation -# --------------------------------------------------------------------------- - - -# --------------------------------------------------------------------------- -# Parameter symbol errors -# --------------------------------------------------------------------------- - - -def test_parameter_without_type_in_implicit_none_scope_raises_parse_error(): - code = """ - subroutine cst(a) - implicit none - real a - parameter ( zero = 0.0e+0 ) - end -""" - with pytest.raises(FortranParseError, match="Unknown datatype for PARAMETER symbol"): - parse_fortran_file(code, filename="legacy.f") - - -def test_duplicate_parameter_declaration_raises_parse_error(): - code = """ -subroutine dup_param() - integer, parameter :: n = 5 - integer, parameter :: n = 10 -end subroutine dup_param -""" - with pytest.raises(FortranParseError, match="Duplicate PARAMETER declaration"): - parse_fortran_file(code, filename="dup_param.f90") - - -# --------------------------------------------------------------------------- -# Mixed-era source forms -# --------------------------------------------------------------------------- - - -def test_f77_source_with_module_keyword_is_parsed(): - code = """ - module bad_module - end module bad_module -""" - parsed = parse_fortran_file(code, filename="legacy.f77") - assert parsed.format == "fixed" - assert parsed.modules[0].name == "bad_module" - - -def test_f77_source_file_metadata_is_preserved(): - code = """ - module bad_module - end module bad_module -""" - parsed = parse_fortran_file(code, filename="legacy.f77") - assert parsed.filename == "legacy.f77" - - -# --------------------------------------------------------------------------- -# Function result type errors -# --------------------------------------------------------------------------- - - -def test_function_with_implicit_none_and_missing_result_type_raises(): - code = """ -function f(x) result(res) - implicit none - real :: x -end function f -""" - with pytest.raises(FortranParseError, match=r"has no type declaration|Unknown datatype for function result"): - parse_fortran_file(code, filename="bad.f90") - - -# --------------------------------------------------------------------------- -# Error location accuracy -# --------------------------------------------------------------------------- - - -def test_error_reports_correct_line_number(): - code = "subroutine foo(x)\n integer :: x\n weirdtype :: y\nend subroutine foo\n" - with pytest.raises(FortranParseError) as exc_info: - parse_fortran_file(code, filename="foo.f90") - err = exc_info.value - assert err.line_number == 3 - - -def test_error_reports_source_line_content(): - code = "subroutine foo(x)\n integer :: x\n weirdtype :: y\nend subroutine foo\n" - with pytest.raises(FortranParseError) as exc_info: - parse_fortran_file(code, filename="foo.f90") - err = exc_info.value - assert "weirdtype" in (err.source_line or "") - - -# --------------------------------------------------------------------------- -# Duplicate argument name errors -# --------------------------------------------------------------------------- - - -def test_duplicate_argument_name_in_subroutine_raises_parse_error(): - code = """ -subroutine dup(x, y, x) - integer, intent(in) :: x - real, intent(in) :: y -end subroutine dup -""" - with pytest.raises(FortranParseError, match="Duplicate argument name"): - parse_fortran_file(code, filename="dup_arg.f90") - - -def test_duplicate_argument_name_in_function_raises_parse_error(): - code = """ -function f(a, b, a) result(res) - integer, intent(in) :: a - real, intent(in) :: b - integer :: res -end function f -""" - with pytest.raises(FortranParseError, match="Duplicate argument name"): - parse_fortran_file(code, filename="dup_arg_func.f90") - - -# --------------------------------------------------------------------------- -# Implicit none: undeclared arguments -# --------------------------------------------------------------------------- - - -def test_implicit_none_undeclared_arg_raises_parse_error(): - code = """ -subroutine foo(x, y) - implicit none - integer, intent(in) :: x -end subroutine foo -""" - with pytest.raises(FortranParseError, match="has no type declaration"): - parse_fortran_file(code, filename="implicit_none.f90") - - -def test_implicit_none_all_args_declared_no_error(): - code = """ -subroutine foo(x, y) - implicit none - integer, intent(in) :: x - real, intent(out) :: y -end subroutine foo -""" - parsed = parse_fortran_file(code, filename="ok.f90") - assert len(parsed.procedures) == 1 - assert all(a.base_type != "unknown" for a in parsed.procedures[0].arguments) - - -def test_implicit_none_undeclared_function_result_raises_parse_error(): - code = """ -function f(x) - implicit none - integer, intent(in) :: x -end function f -""" - with pytest.raises(FortranParseError, match=r"has no type declaration|Unknown datatype for function result"): - parse_fortran_file(code, filename="implicit_none_func.f90") +"""Parser diagnostics and the boundary where the parser stops reading a unit. - -# --------------------------------------------------------------------------- -# Function result validation -# --------------------------------------------------------------------------- - - -def test_function_result_shadowing_arg_name_raises_parse_error(): - code = """ -function f(res) result(res) - integer, intent(in) :: res -end function f -""" - with pytest.raises(FortranParseError, match="shadows an argument name"): - parse_fortran_file(code, filename="shadow.f90") - - -def test_function_with_explicit_result_clause_no_error(): - code = """ -function f(x) result(out) - implicit none - integer, intent(in) :: x - integer :: out -end function f +Every rejected source reports a stable error code, a message naming the owning +scope, and, when the parser knows it, the line the user wrote. Content the +parser does not model (execution parts, internal procedure bodies) is skipped +rather than validated. """ - parsed = parse_fortran_file(code, filename="ok.f90") - assert len(parsed.procedures) == 1 - assert parsed.procedures[0].result is not None - assert parsed.procedures[0].result.name == "out" - assert parsed.procedures[0].result.base_type == "integer" - - -# --------------------------------------------------------------------------- -# Derived type duplicate field names -# --------------------------------------------------------------------------- - - -# --------------------------------------------------------------------------- -# Module duplicate variable names -# --------------------------------------------------------------------------- - - -def test_duplicate_variable_in_module_raises_parse_error(): - code = """ -module m - integer :: n - real :: n -end module m -""" - with pytest.raises(FortranParseError, match="Duplicate variable"): - parse_fortran_file(code, filename="dup_var.f90") - - -def test_module_unique_variables_no_error(): - code = """ -module m - integer :: n - real :: x - logical :: flag -end module m -""" - parsed = parse_fortran_file(code, filename="ok.f90") - assert len(parsed.modules[0].variables) == 3 - - -def test_duplicate_variable_in_program_raises_parse_error(): - code = """ -program main - integer n - real n -end program main -""" - with pytest.raises(FortranParseError, match=r"Duplicate variable.*program"): - parse_fortran_file(code, filename="dup_program_var.f90") - - -def test_duplicate_variable_in_block_data_raises_parse_error(): - code = """ - block data init_data - integer n - real n - end -""" - with pytest.raises(FortranParseError, match=r"Duplicate variable.*block data"): - parse_fortran_file(code, filename="dup_block_data_var.f") - - -def test_same_level_duplicate_modules_raise_parse_error(): - code = """ -module same_name -end module same_name - -module same_name -end module same_name -""" - with pytest.raises(FortranParseError, match="Duplicate module name 'same_name' in file scope"): - parse_fortran_file(code, filename="dup_modules.f90") - - -def test_slicer_reports_mismatched_end_unit_name(): - code = """ -module expected_name -end module wrong_name -""" - with pytest.raises(FortranParseError, match="Mismatched end module name 'wrong_name' for module 'expected_name'"): - parse_fortran_file(code, filename="mismatch_module.f90") - - -def test_slicer_accepts_mismatched_procedure_end_name_without_preferred_alternative(): - parsed = parse_fortran_file( - """ -subroutine expected_name() -end subroutine alternate_name -""", - filename="mismatch_procedure_raw_alternative.f90", - ) - - assert parsed.procedures[0].name == "expected_name" +import pytest -def test_slicer_reports_missing_end_unit(): - code = """ -module missing_end - integer :: n -""" - with pytest.raises(FortranParseError, match="Missing end module for module 'missing_end'"): - parse_fortran_file(code, filename="missing_end_module.f90") +from prik.parsers.fortran import FortranParseError, parse_fortran_file -@pytest.mark.parametrize( - "code", - [ +def _error(filename, source, code, message, line): + return pytest.param(filename, source, code, message, line) + + +_DIAGNOSTICS = { + # Duplicate names, compared case-insensitively within one scope. + "duplicate-declaration-in-procedure": _error( + "dup.f90", + "subroutine dup(x)\n real :: x\n integer :: x\nend subroutine dup\n", + "PARSE_DUPLICATE_DECLARATION", + "Duplicate declaration of symbol 'x' in procedure 'dup'.", + 3, + ), + "duplicate-function-name-result": _error( + "dup_result.f90", + "real function f(x)\n real :: x\n real :: f\nend function f\n", + "PARSE_DUPLICATE_DECLARATION", + "Duplicate declaration of symbol 'f' in procedure 'f'.", + 3, + ), + "duplicate-result-clause-variable": _error( + "dup_result_kw.f90", + "real function f(x) result(res)\n real :: x\n real :: res\nend function f\n", + "PARSE_DUPLICATE_DECLARATION", + "Duplicate declaration of symbol 'res' in procedure 'f'.", + 3, + ), + "procedure-dummy-retyped": _error( + "declarations.f90", + "subroutine apply(callback)\n" + " procedure(callback_iface), external :: callback\n" + " integer :: callback\n" + "end subroutine apply\n", + "PARSE_DUPLICATE_DECLARATION", + "Duplicate declaration of symbol 'callback' in procedure 'apply'.", + 3, + ), + "duplicate-parameter": _error( + "parameters.f90", + "subroutine shape()\n integer, parameter :: n = 4, m = n + 2\n integer, parameter :: n = 8\nend subroutine shape\n", + "PARSE_DUPLICATE_PARAMETER", + "Duplicate PARAMETER declaration of symbol 'n' in procedure 'shape'.", + 3, + ), + "duplicate-argument-case-insensitive": _error( + "dup_arg.f90", + "subroutine step(value, VALUE)\nend subroutine step\n", + "PARSE_DUPLICATE_ARGUMENT", + "Duplicate argument name 'VALUE' in procedure 'step'.", + 1, + ), + "duplicate-procedure-global-scope": _error( + "dup_proc.f90", + "subroutine work(n)\n integer, intent(in) :: n\nend subroutine work\n\n" + "function work(n) result(out)\n integer, intent(in) :: n\n integer :: out\nend function work\n", + "PARSE_DUPLICATE_PROCEDURE", + "Duplicate procedure name 'work' in global scope.", + 5, + ), + "duplicate-procedure-in-module-case-insensitive": _error( + "dup_mod_proc.f90", + "module m\ncontains\n subroutine step()\n end subroutine step\n" + " subroutine STEP()\n end subroutine STEP\nend module m\n", + "PARSE_DUPLICATE_PROCEDURE", + "Duplicate procedure name 'STEP' in module 'm'.", + 5, + ), + "duplicate-module-in-file-case-insensitive": _error( + "dup_modules.f90", + "module same_name\nend module same_name\n\nmodule Same_Name\nend module Same_Name\n", + "PARSE_DUPLICATE_UNIT", + "Duplicate module name 'Same_Name' in file scope.", + 4, + ), + "duplicate-field-case-insensitive": _error( + "dup_field.f90", + "module m\n type :: state_t\n integer, pointer :: ids(:), IDs(:)\n end type state_t\nend module m\n", + "PARSE_DUPLICATE_FIELD", + "Duplicate field 'IDs' in derived type 'state_t'.", + None, + ), + "duplicate-module-variable": _error( + "dup_var.f90", + "module m\n integer :: n\n real :: n\nend module m\n", + "PARSE_DUPLICATE_VARIABLE", + "Duplicate variable 'n' in module 'm'.", + None, + ), + "duplicate-program-variable": _error( + "dup_program_var.f90", + "program main\n integer n\n real n\nend program main\n", + "PARSE_DUPLICATE_VARIABLE", + "Duplicate variable 'n' in program 'main'.", + None, + ), + "duplicate-block-data-variable-fixed-form": _error( + "dup_block_data_var.f", + " block data init_data\n integer n\n real n\n end\n", + "PARSE_DUPLICATE_VARIABLE", + "Duplicate variable 'n' in block data 'init_data'.", + None, + ), + # Unknown datatypes in every metadata scope. + "unknown-type-with-kind-in-procedure": _error( + "procedure_contract.f90", + "subroutine work()\n vector(kind=4) :: value\nend subroutine work\n", + "PARSE_UNSUPPORTED_DECLARATION", + "Unknown or unsupported datatype declaration for procedure 'work': vector(kind=4) :: value", + 2, + ), + "unknown-type-in-interface-body": _error( + "bad_iface.f90", + "module m\n interface foo\n subroutine bar(x)\n weirdtype :: x\n" + " end subroutine bar\n end interface\nend module m\n", + "PARSE_UNSUPPORTED_DECLARATION", + "Unknown or unsupported datatype declaration for procedure 'bar': weirdtype :: x", + 4, + ), + "unknown-type-in-module": _error( + "bad_mod.f90", + "module m\n weirdtype :: x\nend module m\n", + "PARSE_UNSUPPORTED_DECLARATION", + "Unknown or unsupported datatype declaration in module 'm': weirdtype :: x", + 2, + ), + "unknown-type-in-module-without-double-colon": _error( + "bad_mod.f90", + "module owner_mod\n weirdtype value\nend module owner_mod\n", + "PARSE_UNSUPPORTED_DECLARATION", + "Unknown or unsupported datatype declaration in module 'owner_mod': weirdtype value", + 2, + ), + "unknown-type-in-derived-type-without-double-colon": _error( + "bad_type.f90", + "module m\n type :: state_t\n weirdtype value\n end type state_t\nend module m\n", + "PARSE_UNSUPPORTED_DECLARATION", + "Unknown or unsupported datatype declaration in type 'state_t': weirdtype value", + 3, + ), + "c-prototype-in-procedure-specification": _error( + "mixed.f90", + "subroutine mixed_spec()\n api_size count(void);\nend subroutine mixed_spec\n", + "PARSE_UNSUPPORTED_DECLARATION", + "Unknown or unsupported datatype declaration for procedure 'mixed_spec': api_size count(void);", + 2, + ), + # Implicit typing and function results. + "implicit-none-undeclared-argument": _error( + "implicit_none.f90", + "subroutine foo(x, y)\n implicit none\n integer, intent(in) :: x\nend subroutine foo\n", + "PARSE_IMPLICIT_NONE_UNDECLARED_SYMBOL", + "Argument 'y' in procedure 'foo' has no type declaration (implicit none is active).", + None, + ), + "implicit-none-undeclared-function-name-result": _error( + "implicit_none_func.f90", + "function f(x)\n implicit none\n integer, intent(in) :: x\nend function f\n", + "PARSE_IMPLICIT_NONE_UNDECLARED_SYMBOL", + "Function result 'f' in procedure 'f' has no type declaration (implicit none is active).", + None, + ), + "implicit-none-undeclared-result-clause": _error( + "bad_result.f90", + "function f(x) result(res)\n implicit none\n real :: x\nend function f\n", + "PARSE_UNKNOWN_FUNCTION_RESULT_TYPE", + "Unknown datatype for function result 'res' in procedure 'f'.", + None, + ), + "legacy-parameter-without-type-under-implicit-none": _error( + "legacy.f", + " subroutine cst(a)\n implicit none\n real a\n parameter ( zero = 0.0e+0 )\n end\n", + "PARSE_UNKNOWN_PARAMETER_TYPE", + "Unknown datatype for PARAMETER symbol 'zero' in procedure 'cst'.", + 4, + ), + "result-shadows-argument": _error( + "shadow.f90", + "function f(res) result(res)\n integer, intent(in) :: res\nend function f\n", + "PARSE_RESULT_SHADOWS_ARGUMENT", + "Function result variable 'res' in function 'f' shadows an argument name.", + None, + ), + # Unit boundaries and headers. + "mismatched-end-module-name": _error( + "mismatch_module.f90", + "module expected_name\nend module wrong_name\n", + "PARSE_MISMATCHED_UNIT_END", + "Mismatched end module name 'wrong_name' for module 'expected_name'.", + 2, + ), + "missing-end-module": _error( + "missing_end_module.f90", + "module missing_end\n integer :: n\n", + "PARSE_MISSING_UNIT_END", + "Missing end module for module 'missing_end'.", + 1, + ), + "unterminated-internal-procedure": _error( + "unterminated_internal_unit.f90", + "subroutine host()\ncontains\n subroutine nested()\nend subroutine host\n", + "PARSE_MISSING_UNIT_END", + "Missing end procedure for procedure 'host'.", + 1, + ), + "missing-end-derived-type": _error( + "module_contract.f90", + "module owner_mod\n type :: missing_end\nend module owner_mod\n", + "PARSE_MISSING_DERIVED_TYPE_END", + "Missing end derived type for derived type 'missing_end'.", + 2, + ), + "missing-end-type-nested-in-derived-type": _error( + "type_field_invalid.f90", + "module m\n type :: state\n type :: nested_marker\n end type state\nend module m\n", + "PARSE_MISSING_DERIVED_TYPE_END", + "Missing end derived type for derived type 'nested_marker'.", + 3, + ), + "malformed-module-header": _error( + "headers.f90", + "module bad-name\nend module bad-name\n", + "PARSE_MALFORMED_HEADER", + "Unsupported or malformed module header: module bad-name", + 1, + ), + "malformed-separate-module-procedure-header": _error( + "headers.f90", + "submodule (p) c\ncontains\n module procedure bad(x)\n end procedure bad\nend submodule c\n", + "PARSE_MALFORMED_HEADER", + "Unsupported or malformed module procedure header: module procedure bad(x)", + 3, + ), + "stray-end-statement-in-file-scope": _error( + "stray_ends.f90", + "end module stray_mod\nsubroutine kept()\nend subroutine kept\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in file scope: end module stray_mod", + 1, + ), + # Invalid syntax, reported by the scope that owns the line. + "invalid-syntax-in-file-scope": _error( + "invalid_syntax.f90", + "@@@\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in file scope: @@@", + 1, + ), + "c-prototype-in-file-scope": _error( + "mixed.f90", "int add(int a, int b);\n", - "api_size count(void);\n", - """ -subroutine mixed_spec() - api_size count(void); -end subroutine mixed_spec -""", - ], -) -def test_fortran_parser_rejects_invalid_non_fortran_syntax_outside_execution_bodies(code): - with pytest.raises( - FortranParseError, - match=r"Invalid Fortran syntax|Unknown or unsupported datatype declaration", - ) as exc_info: - parse_fortran_file(code, filename="mixed.f90") - - assert exc_info.value.code in {"PARSE_INVALID_SYNTAX", "PARSE_UNSUPPORTED_DECLARATION"} - - -def test_fortran_parser_ignores_non_fortran_syntax_after_execution_boundary(): - parsed = parse_fortran_file( - """ -subroutine mixed_body() - call noop() - api_size count(void); -end subroutine mixed_body -""", - filename="mixed_body.f90", - ) - - assert parsed.procedures[0].name == "mixed_body" + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in file scope: int add(int a, int b);", + 1, + ), + "invalid-syntax-in-module-specification": _error( + "invalid_syntax.f90", + "module bad_spec\n @@@\nend module bad_spec\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in module 'bad_spec' specification part: @@@", + 2, + ), + "invalid-syntax-in-procedure-specification": _error( + "invalid_syntax.f90", + "subroutine work()\n @@@\nend subroutine work\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in procedure 'work' specification part: @@@", + 2, + ), + "invalid-syntax-in-derived-type-specification": _error( + "type_contract.f90", + "module m\n type :: state_t\n call invalid_in_type_spec()\n end type state_t\nend module m\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in type 'state_t' specification part: call invalid_in_type_spec()", + 3, + ), + "invalid-syntax-in-module-contains-part": _error( + "contains_contract.f90", + "module owner_mod\ncontains\n @@@\nend module owner_mod\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in module 'owner_mod' contains part: @@@", + 3, + ), + "invalid-syntax-in-interface-after-valid-lines": _error( + "interface_contract.f90", + "module m\n interface Callbacks\n MODULE PROCEDURE :: First, Second\n" + " PROCEDURE(Callback) :: Handler\n @@@\n end interface Callbacks\nend module m\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in interface 'Callbacks': @@@", + 5, + ), + "nested-type-in-interface": _error( + "nested_contract.f90", + "interface callbacks\n type :: nested\n end type nested\nend interface\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in interface 'callbacks': type :: nested", + 2, + ), + "nested-type-in-derived-type": _error( + "nested_contract.f90", + "type :: outer\n type :: nested\n end type nested\nend type outer\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in derived type 'outer' specification part: type :: nested", + 2, + ), + "nested-type-in-block-data": _error( + "nested_contract.f90", + "block data init_data\n type :: nested\n end type nested\nend block data init_data\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in block data 'init_data' specification part: type :: nested", + 2, + ), + "interface-in-block-data": _error( + "invalid_block_data.f90", + "block data invalid_block\n interface\n end interface\nend block data invalid_block\n", + "PARSE_INVALID_SYNTAX", + "Invalid Fortran syntax in block data 'invalid_block' specification part: interface", + 2, + ), + "malformed-type-bound-declaration": _error( + "type_contains_bad.f90", + "module m\n type :: state\n contains\n integer, public :: bad_binding\n end type state\nend module m\n", + "PARSE_UNSUPPORTED_TYPE_BOUND_DECLARATION", + "Unsupported or malformed type-bound declaration in type 'state': integer, public :: bad_binding", + 4, + ), + # Executable statements and OpenMP directives outside an execution part. + "executable-statement-in-module-specification": _error( + "module_contract.f90", + "module owner_mod\n call work()\nend module owner_mod\n", + "PARSE_EXECUTABLE_IN_SPECIFICATION", + "Executable statement is not allowed in module specification part 'owner_mod': call work()", + 2, + ), + "openmp-executable-directive-in-module": _error( + "bad_omp_mod.f90", + "module bad_omp_mod\n!$omp parallel\nend module bad_omp_mod\n", + "PARSE_EXECUTABLE_IN_SPECIFICATION", + "Executable statement is not allowed in module specification part 'bad_omp_mod': !$omp parallel", + 2, + ), + "openmp-declarative-directive-in-module": _error( + "omp_mod.f90", + "module owner_mod\n !$omp threadprivate(counter)\nend module owner_mod\n", + "PARSE_UNSUPPORTED_OPENMP_DIRECTIVE", + "Unsupported OpenMP declarative directive in module 'owner_mod': !$omp threadprivate(counter)", + 2, + ), + "openmp-declarative-directive-in-procedure": _error( + "omp_decl.f90", + "subroutine omp_decl(x)\n!$omp declare simd\n integer, intent(inout) :: x\nend subroutine omp_decl\n", + "PARSE_UNSUPPORTED_OPENMP_DIRECTIVE", + "Unsupported OpenMP declarative directive in procedure 'omp_decl': !$omp declare simd", + 2, + ), + "openmp-declarative-directive-in-derived-type": _error( + "omp_type.f90", + "module m\n type :: state\n!$omp declare target\n integer :: value\n end type state\nend module m\n", + "PARSE_UNSUPPORTED_OPENMP_DIRECTIVE", + "Unsupported OpenMP declarative directive in type 'state': !$omp declare target", + 3, + ), +} @pytest.mark.parametrize( - "code", - [ - "@@@\n", - """ -module bad_spec - @@@ -end module bad_spec -""", - """ -subroutine bad_spec() - @@@ -end subroutine bad_spec -""", - ], + ("filename", "source", "code", "message", "line"), + list(_DIAGNOSTICS.values()), + ids=list(_DIAGNOSTICS), ) -def test_fortran_parser_rejects_invalid_syntax_outside_execution_bodies(code): - with pytest.raises(FortranParseError, match="Invalid Fortran syntax") as exc_info: - parse_fortran_file(code, filename="invalid_syntax.f90") - - assert exc_info.value.code == "PARSE_INVALID_SYNTAX" - - -def test_fortran_parser_ignores_invalid_syntax_after_execution_boundary(): - parsed = parse_fortran_file( - """ -subroutine ignored_body() - call noop() - @@@ -end subroutine ignored_body -""", - filename="invalid_body.f90", - ) - - assert parsed.procedures[0].name == "ignored_body" - - -def test_fortran_parser_skips_standalone_include_fragment_after_execution_boundary(): - parsed = parse_fortran_file( - """ -include 'fragment.inc' -if (enabled) then - @@@ -else - @@@ -endif -""", - filename="fragment.inc", - ) - - assert parsed.procedures == [] - - -def test_fortran_parser_skips_balanced_internal_procedure_contents(): - parsed = parse_fortran_file( - """ -subroutine host() -contains - subroutine nested() - @@@ - end subroutine nested -end subroutine host -""", - filename="ignored_internal_body.f90", - ) - - assert parsed.procedures[0].name == "host" - - -def test_fortran_parser_rejects_unterminated_internal_procedure_unit(): - with pytest.raises(FortranParseError, match="Missing end procedure"): - parse_fortran_file( - """ -subroutine host() -contains - subroutine nested() -end subroutine host -""", - filename="unterminated_internal_unit.f90", - ) - - -def test_fortran_parser_skips_nested_unit_like_lines_after_execution_boundary(): - parsed = parse_fortran_file( - """ -subroutine host() - call begin_work() - interface - subroutine ignored() - @@@ - end subroutine ignored - end interface -end subroutine host -""", - filename="ignored_nested_execution.f90", - ) - - assert parsed.procedures[0].name == "host" - - -def test_fortran_parser_skips_unterminated_unit_like_lines_after_execution_boundary(): - parsed = parse_fortran_file( - """ -subroutine host() - call begin_work() - subroutine ignored() -end subroutine host -""", - filename="ignored_unterminated_nested_execution.f90", - ) - - assert parsed.procedures[0].name == "host" - - -def test_fortran_parser_rejects_subunit_inside_block_data(): - with pytest.raises(FortranParseError, match="Invalid Fortran syntax") as exc_info: - parse_fortran_file( - """ -block data invalid_block - interface - end interface -end block data invalid_block -""", - filename="invalid_block_data.f90", - ) - - assert exc_info.value.code == "PARSE_INVALID_SYNTAX" - - -def test_invalid_syntax_guard_preserves_valid_semicolon_separated_fortran_statements(): - parsed = parse_fortran_file( - """ -subroutine valid_body(x) - real :: x - call update(x); write(*,*) x -end subroutine valid_body -""", - filename="valid_body.f90", - ) - - assert parsed.procedures[0].name == "valid_body" +def test_parse_error_reports_code_message_and_location(filename, source, code, message, line): + with pytest.raises(FortranParseError) as exc_info: + parse_fortran_file(source, filename=filename) + + error = exc_info.value + assert isinstance(error, ValueError) + assert (error.code, error.base_message, error.filename, error.line_number) == (code, message, filename, line) + if line is not None: + assert error.source_line.strip() == source.splitlines()[line - 1].strip() + diagnostic = error.format_diagnostic(color=False) + assert f"error[{code}]" in diagnostic + assert message in diagnostic + + +_ACCEPTED = { + "non-fortran-after-execution-boundary": ( + "mixed_body.f90", + "subroutine mixed_body()\n call noop()\n api_size count(void);\nend subroutine mixed_body\n", + ["mixed_body"], + ), + "invalid-syntax-after-execution-boundary": ( + "accepted.f90", + "subroutine ignored_body()\n call noop()\n @@@\nend subroutine ignored_body\n", + ["ignored_body"], + ), + "semicolon-separated-statements": ( + "accepted.f90", + "subroutine valid_body(x)\n real :: x\n call update(x); write(*,*) x\nend subroutine valid_body\n", + ["valid_body"], + ), + "nested-unit-lines-after-execution-boundary": ( + "accepted.f90", + "subroutine host()\n call begin_work()\n interface\n subroutine ignored()\n @@@\n" + " end subroutine ignored\n end interface\nend subroutine host\n", + ["host"], + ), + "unterminated-unit-line-after-execution-boundary": ( + "accepted.f90", + "subroutine host()\n call begin_work()\n subroutine ignored()\nend subroutine host\n", + ["host"], + ), + "internal-procedure-body-is-not-validated": ( + "accepted.f90", + "subroutine host()\ncontains\n subroutine nested()\n @@@\n end subroutine nested\nend subroutine host\n", + ["host"], + ), + "same-internal-procedure-name-in-different-hosts": ( + "accepted.f90", + "module m\ncontains\n subroutine host_a()\n contains\n subroutine swap_order()\n" + " end subroutine swap_order\n end subroutine host_a\n\n subroutine host_b()\n contains\n" + " subroutine swap_order()\n end subroutine swap_order\n end subroutine host_b\nend module m\n", + ["host_a", "host_b"], + ), + "procedure-end-name-mismatch-is-tolerated": ( + "accepted.f90", + "subroutine expected_name()\nend subroutine alternate_name\n", + ["expected_name"], + ), + "named-block-construct-starts-execution": ( + "accepted.f90", + "module block_mod\n implicit none\ncontains\n subroutine scale_value(x)\n real(8),intent(inout) :: x\n" + " main: block\n real(8) :: factor\n factor = 2.0d0\n x = x * factor\n" + " end block main\n end subroutine scale_value\nend module block_mod\n", + ["scale_value"], + ), + "openmp-executable-directive-in-body": ( + "accepted.f90", + "subroutine omp_body(x)\n integer, intent(inout) :: x\n!$omp parallel do\n do i = 1, x\n" + " x = x + i\n end do\nend subroutine omp_body\n", + ["omp_body"], + ), + "fixed-form-openmp-sentinel-in-body": ( + "fixed_omp.f", + " subroutine fixed_omp(n)\n integer n\nC$OMP PARALLEL DO\n do 10 i = 1, n\n10 continue\n end\n", + ["fixed_omp"], + ), + "include-and-declaration-in-module-contains-part": ( + "accepted.f90", + "module owner_mod\ncontains\n include 'shape.inc'\n integer :: macro_decl\n" + " subroutine s()\n end subroutine s\nend module owner_mod\n", + ["s"], + ), + "standalone-include-fragment": ( + "fragment.inc", + "include 'fragment.inc'\nif (enabled) then\n @@@\nelse\n @@@\nendif\n", + [], + ), +} + + +@pytest.mark.parametrize(("filename", "source", "procedures"), list(_ACCEPTED.values()), ids=list(_ACCEPTED)) +def test_parser_skips_content_it_does_not_model(filename, source, procedures): + parsed = parse_fortran_file(source, filename=filename) + + names = [procedure.name for procedure in parsed.procedures] + names += [procedure.name for module in parsed.modules for procedure in module.procedures] + assert names == procedures diff --git a/tests/fortran/infrastructure/parsing/test_fortran_fixture_suite.py b/tests/fortran/infrastructure/parsing/test_fortran_fixture_suite.py index f3567aed7..779b04c43 100644 --- a/tests/fortran/infrastructure/parsing/test_fortran_fixture_suite.py +++ b/tests/fortran/infrastructure/parsing/test_fortran_fixture_suite.py @@ -1,3 +1,9 @@ +"""Every general Fortran fixture parses to its reviewed JSON golden. + +Set ``FORTRAN_PARSER_UPDATE_GOLDENS=1`` to rewrite the goldens from the +current parser, then review the diff. +""" + import json import os from dataclasses import asdict @@ -7,53 +13,15 @@ from prik.parsers.fortran import parse_fortran_file - -def parse_fortran_modules(source, filename=None): - return parse_fortran_file(source, filename=filename).modules - - -_TESTS_DIR = Path(__file__).parent / "fixtures" -_FIXTURES_DIR = _TESTS_DIR +_GENERAL_DIR = Path(__file__).parent / "fixtures" / "general" _SOURCE_SUFFIXES = {".f", ".f90", ".f95", ".f03", ".f08", ".for", ".f77", ".ftn"} _UPDATE_GOLDENS = os.getenv("FORTRAN_PARSER_UPDATE_GOLDENS", "0") == "1" - -def _has_direct_expected_json(fixture: Path) -> bool: - return (_FIXTURES_DIR / fixture.relative_to(_TESTS_DIR)).with_suffix(".json").exists() - - -def _source_json_relpaths(root: Path) -> set[Path]: - return { - path.relative_to(root).with_suffix(".json") - for path in root.rglob("*") - if path.is_file() and path.suffix.lower() in _SOURCE_SUFFIXES - } - - -def _fixture_json_relpaths(root: Path) -> set[Path]: - return {path.relative_to(root) for path in root.rglob("*.json") if path.is_file()} - - _GOLDEN_FIXTURES = sorted( - f - for f in (_TESTS_DIR / "general").glob("*") - if f.is_file() and f.suffix.lower() in _SOURCE_SUFFIXES and (_UPDATE_GOLDENS or _has_direct_expected_json(f)) + path for path in _GENERAL_DIR.iterdir() if path.is_file() and path.suffix.lower() in _SOURCE_SUFFIXES ) -def _expected_json_for_fixture(fixture: Path) -> Path: - rel = fixture.relative_to(_TESTS_DIR) - direct = (_FIXTURES_DIR / rel).with_suffix(".json") - if direct.exists(): - return direct - return _FIXTURES_DIR / "general" / (fixture.stem + ".json") - - -def _load_expected(expected_path: Path): - with expected_path.open("r", encoding="utf-8") as f: - return json.load(f) - - def _strip_parent_fields(value): if isinstance(value, dict): return {k: _strip_parent_fields(v) for k, v in value.items() if k != "parent"} @@ -71,54 +39,13 @@ def _to_dict(value): return json.loads(json.dumps(_strip_parent_fields(asdict(value)))) -def _dump_expected(path: Path, parsed: dict) -> None: - payload = parsed - path.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") - - -def _run_fixture_comparison(fixture: Path, *, filename_for_parser: str, expected_path: Path) -> None: - source = fixture.read_text(encoding="utf-8") - assert source.strip(), f"Fixture is empty: {filename_for_parser}" - - parsed = _to_dict(parse_fortran_file(source, filename=filename_for_parser)) +@pytest.mark.parametrize("fixture", _GOLDEN_FIXTURES, ids=lambda f: f.name) +def test_fortran_fixture_golden_suite(fixture): + parsed = _to_dict(parse_fortran_file(fixture.read_text(encoding="utf-8"), filename=fixture.name)) + expected_path = fixture.with_suffix(".json") if _UPDATE_GOLDENS: - _dump_expected(expected_path, parsed) + expected_path.write_text(json.dumps(parsed, indent=2) + "\n", encoding="utf-8") return - expected = _load_expected(expected_path) - assert parsed == expected, f"FortranFile mismatch for {fixture.name}" - - -def test_fortran_fixture_golden_suite_has_fixtures(): - assert _GOLDEN_FIXTURES, f"No fixtures found in {_TESTS_DIR}" - - -@pytest.mark.parametrize( - ("data_root", "fixture_subdir"), - [ - pytest.param(_TESTS_DIR / "general", "general", id="general-general"), - pytest.param(_TESTS_DIR / "errors", "errors", id="errors/parser-errors"), - ], -) -def test_fortran_parser_fixtures_match_data_files_one_to_one(data_root, fixture_subdir): - fixture_root = _FIXTURES_DIR / fixture_subdir - - expected = _source_json_relpaths(data_root) - actual = _fixture_json_relpaths(fixture_root) - - missing = sorted(expected - actual) - extra = sorted(actual - expected) - - if not _UPDATE_GOLDENS: - assert not missing, f"Missing parser JSON fixtures for {fixture_subdir}: {missing[:20]}" - assert not extra, f"Parser JSON fixtures without matching data files in {fixture_subdir}: {extra[:20]}" - - -@pytest.mark.parametrize("fixture", _GOLDEN_FIXTURES, ids=lambda f: f.name) -def test_fortran_fixture_golden_suite(fixture): - _run_fixture_comparison( - fixture, - filename_for_parser=fixture.name, - expected_path=_expected_json_for_fixture(fixture), - ) + assert parsed == json.loads(expected_path.read_text(encoding="utf-8")), f"FortranFile mismatch for {fixture.name}" diff --git a/tests/fortran/infrastructure/parsing/test_fortran_parser_procedures_and_interfaces.py b/tests/fortran/infrastructure/parsing/test_fortran_parser_procedures_and_interfaces.py index 403769005..58596a042 100644 --- a/tests/fortran/infrastructure/parsing/test_fortran_parser_procedures_and_interfaces.py +++ b/tests/fortran/infrastructure/parsing/test_fortran_parser_procedures_and_interfaces.py @@ -1,70 +1,10 @@ -"""Tests split by stable ownership concept from `test_procedures_and_interfaces.py`.""" +"""Procedure headers, dummy procedures, source forms, and program units.""" import pytest -from prik.parsers.fortran import parse_fortran_file, parse_fortran_project -from prik.parsers.fortran.scope import ScopeUses -from prik.parsers.fortran.models import ( - FortranFunctionCall, - FortranSlice, - FortranVariable, -) -from tests.fortran._support.parser_procedures import ( - collect_project_procedure_signatures, - parse_fortran_block_data, - parse_fortran_modules, - parse_fortran_programs, - parse_fortran_submodules, -) - -def test_subroutine_signature_with_intent_and_dimension(): - code = """ -module linalg_mod -contains -pure subroutine axpy(n, a, x, y) - integer, intent(in) :: n - real(kind=8), intent(in) :: a - real(kind=8), intent(in), dimension(:) :: x - real(kind=8), intent(inout) :: y(:) -end subroutine axpy -end module linalg_mod -""" - parsed = parse_fortran_file(code) - signatures = parsed.modules[0].procedures - assert len(signatures) == 1 - - sig = signatures[0] - assert sig.module == "linalg_mod" - assert sig.name == "axpy" - assert sig.kind == "subroutine" - assert "pure" in sig.attributes - - n, a, x, y = sig.arguments - assert n.base_type == "integer" - assert a.base_type == "real" - assert a.kind == "8" - assert x.rank == 1 - assert x.shape == [":"] - assert y.rank == 1 - assert y.shape == [":"] - - -def test_function_result_and_use_statement(): - code = """ -function norm2(x) result(res) - use iso_c_binding, only: c_double - real(kind=c_double), intent(in) :: x(:) - real(kind=c_double) :: res -end function norm2 -""" - signatures = parse_fortran_file(code).procedures - sig = signatures[0] - assert sig.kind == "function" - assert sig.result is not None - assert sig.result.name == "res" - assert sig.result.base_type == "real" - assert list(ScopeUses(sig.uses).mappings("iso_c_binding")) == ["c_double"] - assert sig.arguments[0].shape == [":"] +from prik.parsers.fortran import parse_fortran_file +from prik.parsers.fortran.models import FortranFunctionCall, FortranSlice +from prik.parsers.fortran.scope import ScopeUses def test_fixed_form_and_interface_detection(): @@ -80,240 +20,62 @@ def test_fixed_form_and_interface_detection(): end subroutine cb end interface """ - parsed = parse_fortran_file(code, filename="legacy.f") - assert len(parsed.procedures) == 1 - assert parsed.procedures[0].name == "saxpy" - assert parsed.procedures[0].arguments[1].shape == ["n"] - assert len(parsed.interfaces) == 1 - assert parsed.interfaces[0].procedures[0].in_interface is True - - parsed = parse_fortran_file(code, filename="legacy.f77") - assert len(parsed.procedures) == 1 - assert parsed.procedures[0].name == "saxpy" - assert len(parsed.interfaces) == 1 - - parsed = parse_fortran_file(code, filename="legacy.f90") - assert len(parsed.procedures) == 1 - assert parsed.procedures[0].name == "saxpy" - assert parsed.procedures[0].arguments[1].shape == ["n"] - assert len(parsed.interfaces) == 1 - assert parsed.interfaces[0].procedures[0].in_interface is True - - -def test_module_contains_procedure_and_type_children(): - code = """ -module m1 - integer :: cfg - type :: particle - integer :: id - end type particle -contains - subroutine add1(n, x) - integer, intent(in) :: n - real, intent(inout) :: x(:) - end subroutine add1 -end module m1 -""" - modules = parse_fortran_modules(code) - assert len(modules) == 1 - mod = modules[0] - assert mod.name == "m1" - assert mod.filename is None - assert [p.name for p in mod.procedures] == ["add1"] - assert mod.procedures[0].arguments[0].procedure == "add1" - assert [t.name for t in mod.derived_types] == ["particle"] - - -def test_module_contains_interfaces(): - code = """ -module m1 - interface apply - subroutine do_apply(n) - integer, intent(in) :: n - end subroutine do_apply - end interface -end module m1 -""" - mod = parse_fortran_modules(code)[0] - assert len(mod.interfaces) == 1 - iface = mod.interfaces[0] - assert iface.name == "apply" - assert iface.module == "m1" - assert [p.name for p in iface.procedures] == ["do_apply"] - - -def test_duplicate_function_result_declaration_raises_error(): - code = """ -real function f(x) - real :: x - real :: f -end function f -""" - with pytest.raises(ValueError, match="Duplicate declaration"): - _ = parse_fortran_file(code, filename="dup_result.f90").procedures - - -def test_duplicate_function_result_with_result_keyword_raises_error(): - code = """ -real function f(x) result(res) - real :: x - real :: res -end function f -""" - with pytest.raises(ValueError, match="Duplicate declaration"): - _ = parse_fortran_file(code, filename="dup_result_kw.f90").procedures - - -def test_duplicate_procedure_name_same_scope_raises_error(): - code = """ -subroutine work(n) - integer, intent(in) :: n -end subroutine work - -function work(n) result(out) - integer, intent(in) :: n - integer :: out -end function work -""" - with pytest.raises(ValueError, match="Duplicate procedure name"): - _ = parse_fortran_file(code, filename="dup_proc.f90").procedures - - -def test_duplicate_procedure_name_same_module_scope_raises_error(): - code = """ -module m -contains - subroutine work(n) - integer, intent(in) :: n - end subroutine work - function work(n) result(out) - integer, intent(in) :: n - integer :: out - end function work -end module m -""" - with pytest.raises(ValueError, match="Duplicate procedure name"): - _ = parse_fortran_file(code, filename="dup_mod_proc.f90").procedures - - -def test_implicit_none_allows_external_dummy_procedure_argument(): - code = """ -subroutine fdjac1(fcn, n) - implicit none - integer, intent(in) :: n - external :: fcn -end subroutine fdjac1 -""" - sig = parse_fortran_file(code, filename="fdjac1.f90").procedures[0] - assert sig.arguments[0].name == "fcn" - assert sig.arguments[0].base_type == "procedure" - - -def test_external_attribute_and_later_type_declaration_do_not_conflict(): - code = """ -subroutine lapack_style(slamch) - implicit none - external slamch - real slamch -end subroutine lapack_style -""" - sig = parse_fortran_file(code, filename="lapack_style.f90").procedures[0] - assert sig.arguments[0].name == "slamch" - assert sig.arguments[0].base_type == "real" - - -def test_external_dummy_keeps_recursive_attribute_metadata(): - code = """ -recursive function apply_once(f, x) result(y) - implicit none - real, external :: f - real, intent(in) :: x - real :: y - y = f(x) -end function apply_once -""" - sig = parse_fortran_file(code, filename="apply_once.f90").procedures[0] - assert sig.kind == "function" - assert "recursive" in sig.attributes - assert sig.arguments[0].name == "f" - assert sig.arguments[0].base_type == "real" - - -def test_external_dummy_keeps_interface_context_metadata(): - code = """ -interface - subroutine driver(fcn, x) - implicit none - external :: fcn - real, intent(in) :: x - end subroutine driver -end interface -""" - parsed = parse_fortran_file(code, filename="iface_external.f90") - sig = parsed.interfaces[0].procedures[0] - assert sig.in_interface is True - assert sig.arguments[0].name == "fcn" - assert sig.arguments[1].name == "x" - assert sig.arguments[1].base_type == "real" - - -def test_external_function_dummy_with_explicit_result_type_is_parsed(): - code = """ -subroutine apply_cb(f, x, y) - implicit none - real, intent(in) :: x - real, intent(out) :: y - real, external :: f - y = f(x) -end subroutine apply_cb -""" - sig = parse_fortran_file(code, filename="apply_cb.f90").procedures[0] - f_arg = sig.arguments[0] - assert f_arg.name == "f" - assert f_arg.base_type == "real" - - -def test_ignore_local_variables_in_signatures(): - code = """ -subroutine update(n, x) - integer, intent(in) :: n - real, intent(inout) :: x(n) - integer :: i - real :: tmp -end subroutine update -""" - sig = parse_fortran_file(code).procedures[0] - assert [a.name for a in sig.arguments] == ["n", "x"] - assert sig.arguments[0].base_type == "integer" - assert sig.arguments[1].base_type == "real" - + for filename, source_form in (("legacy.f", "fixed"), ("legacy.f77", "fixed"), ("legacy.f90", "free")): + parsed = parse_fortran_file(code, filename=filename) + assert parsed.format == source_form + assert [procedure.name for procedure in parsed.procedures] == ["saxpy"] + assert parsed.procedures[0].arguments[1].shape == ["n"] + assert len(parsed.interfaces) == 1 + assert parsed.interfaces[0].procedures[0].in_interface is True + + +@pytest.mark.parametrize( + ("declarations", "expected"), + [ + pytest.param(" implicit none\n external :: arg\n", ("procedure", ""), id="external-statement-only"), + pytest.param(" implicit none\n external arg\n real arg\n", ("real", ""), id="external-then-type"), + pytest.param(" implicit none\n real, external :: arg\n", ("real", ""), id="typed-external-attribute"), + pytest.param(" implicit none\n real :: arg\n external :: arg\n", ("real", ""), id="type-then-external"), + pytest.param(" procedure(local_cb) :: arg\n", ("procedure", "local_cb"), id="procedure-interface"), + pytest.param( + " procedure(callback_iface), external :: arg\n", + ("procedure", "callback_iface"), + id="procedure-interface-external", + ), + pytest.param( + " import :: ext_cb\n procedure(ext_cb) :: arg\n", + ("procedure", ""), + id="imported-interface-stays-unresolved", + ), + ], +) +def test_dummy_procedure_declarations_record_type_and_interface(declarations, expected): + """EXTERNAL and PROCEDURE() declarations in either order never conflict with a type.""" + sig = parse_fortran_file(f"subroutine caller(arg)\n{declarations}end subroutine caller\n").procedures[0] -def test_ignore_internal_procedures_in_contains_block(): - code = """ -subroutine outer(x) - real, intent(inout) :: x -contains - subroutine inner(y) - real, intent(inout) :: y - end subroutine inner -end subroutine outer -""" - sigs = parse_fortran_file(code).procedures - assert len(sigs) == 1 - assert sigs[0].name == "outer" + assert [(arg.name, arg.base_type, arg.kind) for arg in sig.arguments] == [("arg", *expected)] -def test_recursive_function_and_result_keyword_variants(): +def test_procedure_prefixes_result_keyword_variants_and_local_use(): code = """ recursive function fact(n) results(res) integer, intent(in) :: n integer :: res end function fact + +pure subroutine axpy(a, x) + use iso_c_binding, only: c_double + real(kind=c_double), intent(in) :: a + real(kind=c_double), intent(in), dimension(:) :: x +end subroutine axpy """ - sig = parse_fortran_file(code).procedures[0] - assert "recursive" in sig.attributes - assert sig.result is not None - assert sig.result.name == "res" + fact, axpy = parse_fortran_file(code).procedures + assert "recursive" in fact.attributes + assert fact.result is not None + assert fact.result.name == "res" + assert "pure" in axpy.attributes + assert list(ScopeUses(axpy.uses).mappings("iso_c_binding")) == ["c_double"] + assert [(arg.base_type, arg.shape) for arg in axpy.arguments] == [("real", []), ("real", [":"])] def test_structured_shape_preserves_slices_and_function_calls(): @@ -345,86 +107,6 @@ def test_structured_shape_preserves_slices_and_function_calls(): assert (y_dim.lower, y_dim.upper, y_dim.stride) == ("1", "n", "2") -def test_fortran_variable_spec_expressions_parse_function_calls(): - var = FortranVariable( - name="work", - kind="selected_real_kind(15)", - value="size(work, 1)", - ) - - assert isinstance(var.kind_expression, FortranFunctionCall) - assert var.kind_expression.name == "selected_real_kind" - assert var.kind_expression.arguments == ["15"] - assert isinstance(var.value_expression, FortranFunctionCall) - assert var.value_expression.name == "size" - assert var.value_expression.arguments == ["work", "1"] - - -def test_local_parameters_in_contained_procedures_do_not_leak_across_signatures(): - files = { - "dims.f90": """ -module dims_mod - integer, parameter :: n = 3 -contains - subroutine a() - integer, parameter :: n = 9 - end subroutine a - - subroutine b(x) - real, intent(inout) :: x(1:n) - end subroutine b -end module dims_mod -""" - } - - sig = collect_project_procedure_signatures(files)[0] - assert sig.name == "a" - sig_b = collect_project_procedure_signatures(files)[1] - assert sig_b.name in {"a", "b"} - if sig_b.name == "b": - assert sig_b.arguments[0].shape == ["1:n"] - - -def test_submodule_procedures_and_namespace_dependencies(tmp_path): - parent = tmp_path / "parent.f90" - child = tmp_path / "child.f90" - parent.write_text( - """ -module parent_mod - integer, parameter :: rk = 8 - interface - module subroutine scale(x) - real(kind=rk), intent(inout) :: x(:) - end subroutine scale - end interface -end module parent_mod -""", - encoding="utf-8", - ) - child.write_text( - """ -submodule (parent_mod) child_impl -contains - module subroutine scale(x) - real(kind=8), intent(inout) :: x(:) - end subroutine scale -end submodule child_impl -""", - encoding="utf-8", - ) - - namespace = parse_fortran_project({str(p.name): p.read_text(encoding="utf-8") for p in tmp_path.glob("*.f90")}) - assert len(namespace.files) == 2 - assert len(namespace.submodules) == 1 - submodule = namespace.submodules["parent_mod:child_impl"] - assert submodule.name == "child_impl" - assert submodule.parent == "parent_mod" - assert submodule.ancestor is None - assert [p.name for p in submodule.procedures] == ["scale"] - # A submodule name is local to its ancestor, so its procedures are owned by ``ancestor:name``. - assert submodule.procedures[0].module == "parent_mod:child_impl" - - def test_submodule_module_procedure_stub_and_additional_program_units(): code = """ submodule (ancestor_mod:parent_impl) child_impl @@ -444,72 +126,19 @@ def test_submodule_module_procedure_stub_and_additional_program_units(): integer :: seed end block data init_data """ - submodules = parse_fortran_submodules(code) - assert len(submodules) == 1 - submodule = submodules[0] + parsed = parse_fortran_file(code) + (submodule,) = parsed.submodules assert submodule.parent == "parent_impl" assert submodule.ancestor == "ancestor_mod" assert list(ScopeUses(submodule.uses).mappings("iso_c_binding")) == ["c_int"] assert [v.name for v in submodule.variables] == ["counter"] assert [(p.name, p.kind) for p in submodule.procedures] == [("reset_counter", "module procedure")] - programs = parse_fortran_programs(code) - assert len(programs) == 1 - assert programs[0].name == "driver" - assert ScopeUses(programs[0].uses).imports_all("ancestor_mod") is True - assert [v.name for v in programs[0].variables] == ["ierr"] - - block_data = parse_fortran_block_data(code) - assert len(block_data) == 1 - assert block_data[0].name == "init_data" - assert [v.name for v in block_data[0].variables] == ["seed"] - - -def test_procedure_dummy_declaration_tracks_local_interface_kind(): - code = """ -subroutine caller(cb) - procedure(local_cb) :: cb -end subroutine caller -""" - sig = parse_fortran_file(code).procedures[0] - cb = next(a for a in sig.arguments if a.name == "cb") - assert cb.base_type == "procedure" - assert cb.kind == "local_cb" - - -def test_procedure_dummy_declaration_with_imported_interface_keeps_kind_unresolved(): - code = """ -subroutine caller(cb) - import :: ext_cb - procedure(ext_cb) :: cb -end subroutine caller -""" - sig = parse_fortran_file(code).procedures[0] - cb = next(a for a in sig.arguments if a.name == "cb") - assert cb.base_type == "procedure" - assert cb.kind == "" - assert "import(ext_cb)" in sig.attributes - + (program,) = parsed.programs + assert program.name == "driver" + assert ScopeUses(program.uses).imports_all("ancestor_mod") is True + assert [v.name for v in program.variables] == ["ierr"] -def test_parse_fortran_modules_rejects_standalone_procedure_entrypoint(): - code = """ -subroutine lonely(x) - integer, intent(in) :: x -end subroutine lonely -""" - assert parse_fortran_modules(code) == [] - - -def test_module_parser_ignores_procedure_local_variables(): - code = """ -module no_leak - integer :: module_value -contains - subroutine worker(x) - integer, intent(in) :: x - real :: local_value - end subroutine worker -end module no_leak -""" - mod = parse_fortran_modules(code)[0] - assert [v.name for v in mod.variables] == ["module_value"] + (block_data,) = parsed.block_data_units + assert block_data.name == "init_data" + assert [v.name for v in block_data.variables] == ["seed"] diff --git a/tests/fortran/infrastructure/parsing/test_fortran_parser_properties.py b/tests/fortran/infrastructure/parsing/test_fortran_parser_properties.py index 246860810..3dcb7799c 100644 --- a/tests/fortran/infrastructure/parsing/test_fortran_parser_properties.py +++ b/tests/fortran/infrastructure/parsing/test_fortran_parser_properties.py @@ -1,4 +1,4 @@ -"""Tests split by stable ownership concept from `test_properties.py`.""" +"""Property and fuzz checks for the Fortran parser.""" import pytest from contextlib import suppress @@ -17,30 +17,16 @@ @pytest.mark.property @given(fortran_subroutines()) -def test_generated_fortran_subroutines_preserve_argument_order(case): +def test_generated_fortran_subroutines_preserve_argument_order_in_any_case(case): proc_name, arg_names, source = case - parsed = parse_fortran_file(source, filename=f"{proc_name}.f90") - - assert parsed.diagnostics == [] - assert len(parsed.procedures) == 1 - procedure = parsed.procedures[0] - assert procedure.name == proc_name - assert [arg.name for arg in procedure.arguments] == arg_names - + for text in (source, source.upper()): + parsed = parse_fortran_file(text, filename=f"{proc_name}.f90") -@pytest.mark.property -@given(fortran_subroutines()) -def test_generated_fortran_subroutines_survive_case_changes(case): - proc_name, arg_names, source = case - - parsed = parse_fortran_file(source.upper(), filename=f"{proc_name}.f90") - - assert parsed.diagnostics == [] - assert len(parsed.procedures) == 1 - procedure = parsed.procedures[0] - assert procedure.name.lower() == proc_name - assert [arg.name.lower() for arg in procedure.arguments] == arg_names + assert parsed.diagnostics == [] + (procedure,) = parsed.procedures + assert procedure.name.lower() == proc_name + assert [arg.name.lower() for arg in procedure.arguments] == arg_names @pytest.mark.property diff --git a/tests/fortran/infrastructure/parsing/test_json_sanity.py b/tests/fortran/infrastructure/parsing/test_json_sanity.py deleted file mode 100644 index 7025114e6..000000000 --- a/tests/fortran/infrastructure/parsing/test_json_sanity.py +++ /dev/null @@ -1,315 +0,0 @@ -import json -import re -from pathlib import Path - -_PARSER_DATA = Path(__file__).parent -_FCODE_DIR = _PARSER_DATA / "fixtures" -_ALLOWLIST_PATH = _FCODE_DIR / "json_sanity_allowlist.json" - - -def _is_external_argument(arg: dict) -> bool: - """Return True for procedure/external arguments where base_type may be unresolved.""" - attrs = {str(a).lower() for a in (arg.get("attributes") or [])} - return arg.get("base_type") == "procedure" or "external" in attrs - - -def _has_known_base_type(entry: dict) -> bool: - return entry.get("base_type") not in (None, "", "unknown") - - -def _is_valid_rank(entry: dict) -> bool: - rank = entry.get("rank") - return isinstance(rank, int) and rank >= 0 - - -def _has_non_empty_name(entry: dict) -> bool: - name = entry.get("name") - return isinstance(name, str) and bool(name.strip()) - - -def _split_dim_bounds(dim: str) -> tuple[str | None, str | None]: - """Return the parser-normalized lower/upper bounds for a shape token.""" - part = dim.strip() - if not part: - return None, None - if ":" not in part: - return "1", part - lower, upper = part.split(":", 1) - return lower.strip() or None, upper.strip() or None - - -def _bounds_from_shape(shape: list[str]) -> tuple[list[str | None], list[str | None]]: - lower_bounds = [] - upper_bounds = [] - for dim in shape: - lower, upper = _split_dim_bounds(dim) - lower_bounds.append(lower) - upper_bounds.append(upper) - return lower_bounds, upper_bounds - - -def _has_valid_bounds(entry: dict) -> bool: - """Validate optional lower/upper bound metadata against rank and shape.""" - rank = entry.get("rank") - shape = entry.get("shape") - lbound = entry.get("lbound") - ubound = entry.get("ubound") - - if not isinstance(rank, int) or rank < 0: - return False - - if rank == 0: - return lbound in (None, []) and ubound in (None, []) - - if lbound is not None and (not isinstance(lbound, list) or len(lbound) != rank): - return False - if ubound is not None and (not isinstance(ubound, list) or len(ubound) != rank): - return False - - if isinstance(shape, list) and lbound is not None and ubound is not None: - expected_lbound, expected_ubound = _bounds_from_shape(shape) - return lbound == expected_lbound and ubound == expected_ubound - - return True - - -def _split_top_level_csv(text: str) -> list[str]: - parts = [] - current = [] - depth = 0 - quote: str | None = None - for char in text: - if quote: - current.append(char) - if char == quote: - quote = None - continue - if char in {"'", '"'}: - quote = char - current.append(char) - continue - if char in "([": - depth += 1 - elif char in ")]" and depth: - depth -= 1 - if char == "," and depth == 0: - parts.append("".join(current).strip()) - current = [] - continue - current.append(char) - parts.append("".join(current).strip()) - return [part for part in parts if part] - - -def _is_literal_json_parameter_value(value) -> bool: - """Return True only for evaluated/literal parameter values in JSON output.""" - if isinstance(value, bool): - return True - if isinstance(value, int | float): - return True - if value is None: - return False - - text = str(value).strip() - if not text: - return False - if re.fullmatch(r"[+-]?\d+(?:\.\d*)?(?:[deDE][+-]?\d+)?", text): - return True - if re.fullmatch(r"\.(?:true|false)\.", text, re.IGNORECASE): - return True - if re.fullmatch(r"(['\"]).*\1", text): - return True - if text.startswith("[") and text.endswith("]"): - return all(_is_literal_json_parameter_value(part) for part in _split_top_level_csv(text[1:-1])) - if text.startswith("(/") and text.endswith("/)"): - return all(_is_literal_json_parameter_value(part) for part in _split_top_level_csv(text[2:-2])) - if text.startswith("(") and text.endswith(")"): - parts = _split_top_level_csv(text[1:-1]) - return len(parts) == 2 and all(_is_literal_json_parameter_value(part) for part in parts) - return False - - -def _is_valid_shape_info(entry: dict) -> bool: - """Enforce non-ambiguous shape metadata with clear scalar/array exception rules. - - Rule: rank must match the shape metadata. - - scalar (rank == 0): `shape`, `dimensions`, `lbound`, and `ubound` may be - omitted/None/[] only. - - array (rank > 0): `shape`, `dimensions`, `lbound`, and `ubound` can be - omitted for legacy fixtures, but if present they must be lists. Present - bounds must match rank, and when `shape` is available they must match the - parser-normalized lower/upper bounds for each dimension. - """ - rank = entry.get("rank") - shape = entry.get("shape") - dimensions = entry.get("dimensions") - - if not isinstance(rank, int) or rank < 0: - return False - - if rank == 0: - return shape in (None, []) and dimensions in (None, []) and _has_valid_bounds(entry) - - return ( - (shape is None or isinstance(shape, list)) - and (dimensions is None or isinstance(dimensions, list)) - and _has_valid_bounds(entry) - ) - - -def _load_fixture_payload(path: Path): - with path.open("r", encoding="utf-8") as f: - data = json.load(f) - if isinstance(data, list): - return {"signatures": data, "types": []} - return {"signatures": data.get("signatures", []), "types": data.get("types", [])} - - -def _iter_typed_entries(node): - if isinstance(node, dict): - if "base_type" in node: - yield node - for value in node.values(): - yield from _iter_typed_entries(value) - elif isinstance(node, list): - for item in node: - yield from _iter_typed_entries(item) - - -def test_fortran_json_fixtures_are_valid_json(): - for path in _FCODE_DIR.rglob("*.json"): - if path.name.endswith("_errors.json"): - continue - with path.open("r", encoding="utf-8") as f: - json.load(f) - - -def test_fortran_json_fixtures_have_sane_types(): - with _ALLOWLIST_PATH.open("r", encoding="utf-8") as f: - allowlist = { - (item["file"], item["kind"], item["owner"], item["name"]) - for item in json.load(f)["allowed_unknown_base_types"] - } - - unknown_entries = [] - invalid_rank_entries = [] - duplicate_argument_entries = [] - invalid_name_entries = [] - invalid_shape_entries = [] - invalid_kind_entries = [] - invalid_parameter_value_entries = [] - - for path in _FCODE_DIR.rglob("*.json"): - if path.name.endswith("_errors.json") or path.name == _ALLOWLIST_PATH.name: - continue - - with path.open("r", encoding="utf-8") as f: - raw_payload = json.load(f) - payload = _load_fixture_payload(path) - relpath = str(path.relative_to(_FCODE_DIR)) - if relpath.startswith("general/"): - relpath = relpath[len("general/") :] - for entry in _iter_typed_entries(raw_payload): - if "kind" not in entry or entry.get("kind") is None: - invalid_kind_entries.append((relpath, entry.get("base_type"), entry.get("name"))) - if entry.get("is_parameter"): - value = entry.get("value") - symbolic_value = entry.get("symbolic_value") - if value is None and symbolic_value in (None, ""): - invalid_parameter_value_entries.append( - (relpath, entry.get("name"), "missing value and symbolic_value") - ) - if value is not None and not _is_literal_json_parameter_value(value): - invalid_parameter_value_entries.append((relpath, entry.get("name"), value)) - - for sig in payload["signatures"]: - arg_names = [] - for arg in sig.get("arguments", []): - arg_name = arg.get("name") - arg_names.append(arg_name) - - if not _has_non_empty_name(arg): - invalid_name_entries.append((relpath, "argument", sig.get("name"), arg_name)) - - if not _is_valid_rank(arg): - invalid_rank_entries.append((relpath, "argument", sig.get("name"), arg_name, arg.get("rank"))) - - if not _is_valid_shape_info(arg): - invalid_shape_entries.append( - ( - relpath, - "argument", - sig.get("name"), - arg_name, - arg.get("rank"), - arg.get("shape"), - arg.get("dimensions"), - ) - ) - - if not _has_known_base_type(arg) and not _is_external_argument(arg): - unknown_entries.append((relpath, "argument", sig.get("name"), arg.get("name"))) - - if len(arg_names) != len(set(arg_names)): - duplicate_argument_entries.append((relpath, sig.get("name"))) - - for var_name, var in sig.get("variables", {}).items(): - if not _has_non_empty_name(var): - invalid_name_entries.append((relpath, "variable", sig.get("name"), var_name)) - - if not _is_valid_rank(var): - invalid_rank_entries.append((relpath, "variable", sig.get("name"), var_name, var.get("rank"))) - - if not _is_valid_shape_info(var): - invalid_shape_entries.append( - ( - relpath, - "variable", - sig.get("name"), - var_name, - var.get("rank"), - var.get("shape"), - var.get("dimensions"), - ) - ) - - if not _has_known_base_type(var): - unknown_entries.append((relpath, "variable", sig.get("name"), var_name)) - - for dtype in payload["types"]: - for field in dtype.get("fields", []): - if not _has_non_empty_name(field): - invalid_name_entries.append((relpath, "field", dtype.get("name"), field.get("name"))) - - if not _is_valid_rank(field): - invalid_rank_entries.append( - (relpath, "field", dtype.get("name"), field.get("name"), field.get("rank")) - ) - - if not _is_valid_shape_info(field): - invalid_shape_entries.append( - ( - relpath, - "field", - dtype.get("name"), - field.get("name"), - field.get("rank"), - field.get("shape"), - field.get("dimensions"), - ) - ) - - if not _has_known_base_type(field): - unknown_entries.append((relpath, "field", dtype.get("name"), field.get("name"))) - - unexpected = sorted(e for e in unknown_entries if e not in allowlist) - stale_allowlist = sorted(e for e in allowlist if e not in set(unknown_entries)) - - assert not unexpected, f"Unexpected unknown base_type entries: {unexpected[:20]}" - assert not stale_allowlist, f"Stale allowlist entries: {stale_allowlist[:20]}" - assert not invalid_rank_entries, f"Invalid rank entries: {invalid_rank_entries[:20]}" - assert not invalid_name_entries, f"Invalid empty/missing names: {invalid_name_entries[:20]}" - assert not invalid_shape_entries, f"Invalid shape/dimensions metadata: {invalid_shape_entries[:20]}" - assert not invalid_kind_entries, f"Missing/null kind entries: {invalid_kind_entries[:20]}" - assert not invalid_parameter_value_entries, f"Invalid parameter values: {invalid_parameter_value_entries[:20]}" - assert not duplicate_argument_entries, f"Duplicate argument names in signatures: {duplicate_argument_entries[:20]}" diff --git a/tests/fortran/infrastructure/parsing/test_parser_benchmarks.py b/tests/fortran/infrastructure/parsing/test_parser_benchmarks.py deleted file mode 100644 index 31432fe50..000000000 --- a/tests/fortran/infrastructure/parsing/test_parser_benchmarks.py +++ /dev/null @@ -1,46 +0,0 @@ -"""Representative parser and code-generation performance benchmarks.""" - -from __future__ import annotations - - -import pytest - -from prik.semantics.fortran2ir import fortran_file_to_semantic_modules -from prik.pipeline.pyi import emit_module_stubs -from prik.parsers.fortran import parse_fortran_file -from tests.fortran._support.paths import REPO_ROOT - -pytestmark = pytest.mark.skip(reason="Benchmarks are parked until benchmark adoption resumes.") - - -_FORTRAN_MODULE = ( - "module generated\n" - "contains\n" - + "".join( - f"subroutine step_{index}(x)\n integer, intent(in) :: x\nend subroutine step_{index}\n" for index in range(50) - ) - + "end module generated\n" -) - - -def _parse_convert_emit_fortran(source: str) -> dict[str, str]: - parsed = parse_fortran_file(source, filename="benchmark.f90") - return emit_module_stubs(fortran_file_to_semantic_modules(parsed)) - - -@pytest.mark.benchmark -def test_parse_convert_emit_representative_fortran_module(benchmark): - stubs = benchmark(_parse_convert_emit_fortran, _FORTRAN_MODULE) - - assert stubs["generated"].count("def step_") == 50 - - -@pytest.mark.benchmark -def test_parse_real_lapack_dgesv(benchmark): - source = (REPO_ROOT / "examples" / "fortran" / "lapack" / "native" / "dgesv.f").read_text( - encoding="utf-8", - ) - parsed = benchmark(parse_fortran_file, source, filename="lapack/dgesv.f") - - assert [procedure.name for procedure in parsed.procedures] == ["DGESV"] - assert parsed.diagnostics == [] diff --git a/tests/fortran/infrastructure/parsing/test_public_entrypoints.py b/tests/fortran/infrastructure/parsing/test_public_entrypoints.py index 4b1ef48ab..d6ad15863 100644 --- a/tests/fortran/infrastructure/parsing/test_public_entrypoints.py +++ b/tests/fortran/infrastructure/parsing/test_public_entrypoints.py @@ -3,7 +3,6 @@ import pytest from prik.parsers.fortran import FortranParseError, FortranParser, parse_fortran_file, parse_fortran_project -from prik.semantics.fortran2ir import fortran_file_to_semantic_modules def test_parser_public_entrypoint_aliases_and_singular_contracts_use_inline_sources(): @@ -44,6 +43,9 @@ def test_parser_public_entrypoint_aliases_and_singular_contracts_use_inline_sour with pytest.raises(FortranParseError, match="none were found"): parser.parse_module("program not_a_module\nend program not_a_module\n") + with pytest.raises(FortranParseError, match="only standalone procedures were found"): + parser.parse_module("subroutine lone_proc()\nend subroutine lone_proc\n") + with pytest.raises(FortranParseError, match="found 2"): parser.parse_module( """ @@ -55,18 +57,6 @@ def test_parser_public_entrypoint_aliases_and_singular_contracts_use_inline_sour ) -def test_fortran_parser_package_exports_the_supported_parser_api(): - parsed_fortran = FortranParser().parse_file( - """ -subroutine work(n) - integer, intent(in) :: n -end subroutine work -""" - ) - - assert fortran_file_to_semantic_modules(parsed_fortran)[0].functions[0].name == "work" - - def test_file_path_and_unknown_filename_public_parse_paths(tmp_path): source_path = tmp_path / "path_input.f90" source_path.write_text( @@ -97,41 +87,6 @@ def test_file_path_and_unknown_filename_public_parse_paths(tmp_path): assert parsed_unknown_suffix.format == "free" -def test_public_instance_visitor_entrypoints_use_source_strings(): - parser = FortranParser() - - assert ( - parser.parse_file( - """ -subroutine alias_proc() -end subroutine alias_proc -""" - ) - .procedures[0] - .name - == "alias_proc" - ) - assert ( - "alias_mod" - in parser.parse_project( - { - "alias_mod.f90": """ -module alias_mod -end module alias_mod -""" - } - ).modules - ) - - with pytest.raises(FortranParseError, match="only standalone procedures were found"): - parser.parse_module( - """ -subroutine lone_proc() -end subroutine lone_proc -""" - ) - - @pytest.mark.parametrize( ("source", "expected_module"), [ diff --git a/tests/fortran/infrastructure/parsing/test_real_world_interaction_regressions.py b/tests/fortran/infrastructure/parsing/test_real_world_interaction_regressions.py index 3566fa167..585f62568 100644 --- a/tests/fortran/infrastructure/parsing/test_real_world_interaction_regressions.py +++ b/tests/fortran/infrastructure/parsing/test_real_world_interaction_regressions.py @@ -3,23 +3,10 @@ from pathlib import Path from prik.parsers.fortran import parse_fortran_file -from prik.parsers.fortran.lexer import preprocess_lines, strip_comment -from prik.parsers.fortran.models import FortranProcedureSignature -from prik.parsers.fortran.parser import FortranParser, _SourceUnitScanner -from prik.parsers.fortran.utils import split_csv NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "support" -def test_free_form_lexing_preserves_mixed_quotes_and_folds_leading_ampersands(): - assert strip_comment('print *, "don\'t remove ! here" ! remove me', "free") == ('print *, "don\'t remove ! here" ') - assert preprocess_lines( - "value = first &\n & + second &\n & + third\n", - filename="continuation.f90", - ) == [("value = first+ second+ third", 1, "value = first &")] - assert split_csv("left,") == ["left"] - - def test_legacy_and_extended_types_keep_initializers_and_declaration_attributes(): source = NATIVE_FIXTURES / "declaration_interactions.f90" parsed = parse_fortran_file(source.read_text(encoding="utf-8"), filename=source.name) @@ -99,15 +86,3 @@ def test_legacy_procedure_specifications_preserve_wrapper_relevant_facts(): assert signature.result is not None assert signature.result.kind == "selected_real_kind(12)" assert signature.common_variables == ["cache"] - assert _SourceUnitScanner.is_executable_statement_start("square(value) = value * value") is False - - -def test_procedure_include_is_recorded_before_signature_finalization(): - parser = FortranParser() - state = parser._new_procedure_scope_state( - FortranProcedureSignature("include_contract", "subroutine"), - symbols={}, - ) - - assert parser._handle_proc_include_or_import_line("include 'constants.inc'", state) is True - assert state.includes == ["'constants.inc'"] diff --git a/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py b/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py index 15b281a68..4830d0ada 100644 --- a/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py +++ b/tests/fortran/infrastructure/parsing/test_source_form_and_diagnostics_regressions.py @@ -1,555 +1,8 @@ -"""Tests split by stable ownership concept from `test_source_form_and_diagnostics_regressions.py`.""" +"""Singular-unit entrypoint diagnostics, unit filenames, and line numbers after compiler preprocessing.""" import pytest -from pathlib import Path -from prik.parsers.fortran import FortranParseError, parse_fortran_file -from prik.parsers.fortran.models import ( - FortranArgument, - FortranDerivedType, - FortranModule, - FortranProcedureSignature, -) -from prik.parsers.fortran.parser import ( - FortranParser, - _ParserScope, - _SourceUnitScanner, -) -from tests.fortran._support.parser_regressions import ( - _lines, - _unit, -) - - -def test_source_unit_classification_preserves_child_regions_and_direct_ownership(): - scanner = _SourceUnitScanner() - unit = scanner.scan_file_units( - _lines( - "module owner", - "integer :: value", - "interface callbacks", - " subroutine callback()", - " end subroutine callback", - "end interface callbacks", - "interface generic_work", - " module procedure work", - "end interface generic_work", - "contains", - "# generated marker", - "subroutine work()", - "end subroutine work", - "end module owner", - ), - filename="regions.f90", - )[0] - - assert unit.header == unit.lines[0] - assert unit.footer == unit.lines[-1] - assert [line for line, _lineno, _source in unit.specification] == ["integer :: value"] - assert unit.execution == [] - assert [line.strip() for line, _lineno, _source in unit.contains] == ["# generated marker"] - assert [(child.kind, child.name, child.parent_region) for child in unit.children] == [ - ("interface", "callbacks", "specification"), - ("interface", "generic_work", "specification"), - ("procedure", "work", "contains"), - ] - assert [(child.kind, child.name, child.parent_region) for child in unit.children[0].children] == [ - ("procedure", "callback", "specification") - ] - assert [line.strip() for line, _lineno, _source in unit.children[1].specification] == ["module procedure work"] - - -def test_procedure_classification_keeps_local_interfaces_and_omits_internal_procedures(): - scanner = _SourceUnitScanner() - unit = scanner.scan_file_units( - _lines( - "subroutine work(callback)", - "integer :: value", - "interface", - " subroutine callback()", - " end subroutine callback", - "end interface", - "value = 1", - "contains", - "subroutine inner()", - "end subroutine inner", - "end subroutine work", - ), - filename="regions.f90", - )[0] - - assert [line for line, _lineno, _source in unit.specification] == ["integer :: value"] - assert [line for line, _lineno, _source in unit.execution] == ["value = 1"] - assert unit.contains == [] - assert [(child.kind, child.name, child.parent_region) for child in unit.children] == [ - ("interface", None, "specification") - ] - - assert scanner.has_preferred_unit_end_ahead(unit.lines, 0, "procedure", "work") is True - assert scanner.has_preferred_unit_end_ahead(unit.lines, 0, "procedure", "missing") is False - assert scanner.has_preferred_unit_end_ahead(unit.lines[:-1], 0, "procedure", "work") is False - immediate_type = _lines("type :: immediate", "end type immediate") - assert scanner.has_preferred_unit_end_ahead(immediate_type, 0, "derived_type", "immediate") is True - assert scanner.has_unit_end_ahead(immediate_type, 0, "derived_type") is True - assert scanner.has_unit_end_ahead(unit.lines, 0, "procedure") is True - - -def test_unit_end_search_tracks_nested_specification_and_contains_units(): - scanner = _SourceUnitScanner() - lines = _lines( - "subroutine work()", - "type :: local_state", - "end type local_state", - "contains", - "subroutine inner()", - "end subroutine inner", - "end subroutine work", - ) - - assert scanner.find_unit_end(lines, 0, "procedure", filename="regions.f90") == 6 - assert scanner.has_unit_end_ahead(lines, 1, "derived_type") is True - assert scanner.has_unit_end_ahead(lines, 4, "procedure") is True - assert scanner.has_unit_end_ahead(lines[:-1], 0, "procedure") is True - - -def test_source_preparation_rejects_raw_cpp_and_preserves_root_units_and_source_form(tmp_path: Path): - parser = FortranParser() - lines, root_scope, units = parser._helper_prepare_source_units( - """ -module owner_mod -end module owner_mod -subroutine global_step() -end subroutine global_step -""", - filename="prepare_contract.f90", - ) - - assert root_scope == _ParserScope(kind="file", name=None) - assert [(unit.kind, unit.name, unit.start_line, unit.end_line) for unit in units] == [ - ("module", "owner_mod", 2, 3), - ("procedure", "global_step", 4, 5), - ] - assert [line for line, _lineno, _source in lines if line.strip()] == [ - "module owner_mod", - "end module owner_mod", - "subroutine global_step()", - "end subroutine global_step", - ] - - source_path = tmp_path / "path_input.f90" - source_path.write_text("module from_path\nend module from_path\n", encoding="utf-8") - assert parser._looks_like_existing_source_path(source_path) is True - assert parser._looks_like_existing_source_path("module inline\nend module inline\n") is False - assert parser._looks_like_existing_source_path(object()) is False - - with pytest.raises(FortranParseError) as error: - parser._helper_prepare_source_units("#define VALUE 1\nmodule bad\nend module bad\n", filename="raw_cpp.f90") - - assert error.value.base_message == "Fortran CPP directives require compiler preprocessing before parsing." - assert error.value.filename == "raw_cpp.f90" - assert error.value.line_number == 1 - assert error.value.source_line == "#define VALUE 1" - assert error.value.code == "PARSE_PREPROCESSING_REQUIRED" - - -def test_file_unit_scanning_skips_preprocessed_linemarkers_and_blank_unit_starts(): - scanner = _SourceUnitScanner() - lines = _lines( - '# 4 "generated.f90"', - "", - "module owner", - "end module owner", - ) - - units = scanner.scan_file_units( - lines, - filename="generated.f90", - ) - - assert [(unit.kind, unit.name) for unit in units] == [("module", "owner")] - assert scanner.classify_unit_start(" ") is None - - -def test_unit_end_and_header_validation_preserve_public_diagnostics(): - parser = FortranParser() - scanner = _SourceUnitScanner() - - assert scanner.parse_unit_end("module", "end module owner_mod") == (True, "owner_mod") - assert scanner.parse_unit_end("block_data", "end") == (True, None) - assert scanner.parse_unit_end("procedure", "end function value") == (True, "value") - assert scanner.unit_end_matches("enum", "end enum") is True - assert scanner.unit_label("block_data") == "block data" - assert parser._parse_submodule_header("submodule (ancestor_mod:parent_mod) child_mod", "headers.f90").parent == ( - "parent_mod" - ) - assert parser._split_submodule_parent("ancestor_mod:parent_mod") == ("parent_mod", "ancestor_mod") - assert parser._split_submodule_parent("parent_mod") == ("parent_mod", None) - assert scanner.parse_interface_header("abstract interface") == (True, None) - assert scanner.parse_interface_header("interface callbacks") == (True, "callbacks") - - with pytest.raises(FortranParseError) as module_error: - parser._parse_module_header( - "module bad-name", - filename="headers.f90", - lineno=3, - source_line="module bad-name", - ) - assert module_error.value.base_message == "Unsupported or malformed module header: module bad-name" - assert module_error.value.filename == "headers.f90" - assert module_error.value.line_number == 3 - assert module_error.value.source_line == "module bad-name" - assert module_error.value.code == "PARSE_MALFORMED_HEADER" - - with pytest.raises(FortranParseError) as procedure_error: - parser._helper_validate_possible_unit_header( - "module procedure bad(x)", - filename="headers.f90", - lineno=4, - source_line="module procedure bad(x)", - ) - assert ( - procedure_error.value.base_message - == "Unsupported or malformed module procedure header: module procedure bad(x)" - ) - assert procedure_error.value.filename == "headers.f90" - assert procedure_error.value.line_number == 4 - assert procedure_error.value.source_line == "module procedure bad(x)" - assert procedure_error.value.code == "PARSE_MALFORMED_HEADER" - - -def test_classified_unit_regions_skip_nested_units_and_preserve_executable_boundary(): - unit = _unit( - "procedure", - "work", - "subroutine work()", - "integer :: counter", - "", - "interface", - " subroutine callback()", - " end subroutine callback", - "end interface", - "counter = counter + 1", - "contains", - "subroutine inner()", - "end subroutine inner", - "end subroutine work", - ) - - assert [line for line, _lineno, _source in unit.specification] == ["integer :: counter"] - assert [line for line, _lineno, _source in unit.execution] == ["counter = counter + 1"] - assert unit.contains == [] - assert unit.header == unit.lines[0] - assert unit.footer == unit.lines[-1] - - -def test_sibling_unit_validation_ignores_unnamed_units_and_preserves_duplicate_diagnostics(): - parser = FortranParser() - parser._helper_validate_sibling_units( - [ - _unit("enum", None, "enum, bind(c)", "end enum"), - _unit("enum", None, "enum, bind(c)", "end enum"), - ], - parent_scope=_ParserScope(kind="module", name="owner_mod"), - filename="siblings.f90", - ) - - with pytest.raises(FortranParseError) as duplicate_module: - parser._helper_validate_sibling_units( - [ - _unit("module", "owner_mod", "module owner_mod", "end module owner_mod"), - _unit("module", "Owner_Mod", "module Owner_Mod", "end module Owner_Mod"), - ], - parent_scope=_ParserScope(kind="file", name=None), - filename="siblings.f90", - ) - - assert duplicate_module.value.base_message == "Duplicate module name 'Owner_Mod' in file scope." - assert duplicate_module.value.filename == "siblings.f90" - assert duplicate_module.value.line_number == 1 - assert duplicate_module.value.source_line == "module Owner_Mod" - assert duplicate_module.value.code == "PARSE_DUPLICATE_UNIT" - - with pytest.raises(FortranParseError) as duplicate_procedure: - parser._helper_validate_sibling_units( - [ - _unit("procedure", "step", "subroutine step()", "end subroutine step"), - _unit("procedure", "STEP", "subroutine STEP()", "end subroutine STEP"), - ], - parent_scope=_ParserScope(kind="module", name="owner_mod"), - filename="siblings.f90", - ) - - assert duplicate_procedure.value.base_message == "Duplicate procedure name 'STEP' in module 'owner_mod'." - assert duplicate_procedure.value.filename == "siblings.f90" - assert duplicate_procedure.value.line_number == 1 - assert duplicate_procedure.value.source_line == "subroutine STEP()" - assert duplicate_procedure.value.code == "PARSE_DUPLICATE_PROCEDURE" - - -def test_finalize_proc_duplicate_argument_diagnostic_preserves_header_metadata(): - parser = FortranParser() - signature = FortranProcedureSignature( - "step", - "subroutine", - arguments=[FortranArgument("value"), FortranArgument("VALUE")], - ) - - state = parser._new_procedure_scope_state(signature, symbols={}) - state.filename = "finalize_contract.f90" - state.header_lineno = 12 - state.header_source_line = "subroutine step(value, VALUE)" - - with pytest.raises(FortranParseError) as error: - parser._finalize_proc(state) - - assert error.value.base_message == "Duplicate argument name 'VALUE' in procedure 'step'." - assert error.value.filename == "finalize_contract.f90" - assert error.value.line_number == 12 - assert error.value.source_line == "subroutine step(value, VALUE)" - assert error.value.code == "PARSE_DUPLICATE_ARGUMENT" - - -def test_declaration_storage_preserves_type_field_metadata_and_duplicate_field_diagnostic(): - parser = FortranParser() - dtype = FortranDerivedType("state_t") - scope = _ParserScope(kind="derived_type", name=dtype.name, model=dtype) - declaration = parser._new_declaration("integer", "i4") - parser._apply_declaration_attributes(declaration, ["pointer", "dimension(:)"]) - - parser._store_declaration( - scope, - declaration=declaration, - right="ids, IDs", - role="type_field", - filename="declarations.f90", - lineno=7, - source_line="integer(kind=i4), pointer, dimension(:) :: ids, IDs", - ) - - assert [(field.name, field.base_type, field.kind, field.pointer, field.shape) for field in dtype.fields] == [ - ("ids", "integer", "i4", True, [":"]), - ("IDs", "integer", "i4", True, [":"]), - ] - with pytest.raises(FortranParseError) as error: - parser._validate_derived_type_fields(dtype, filename="declarations.f90") - - assert error.value.base_message == "Duplicate field 'IDs' in derived type 'state_t'." - assert error.value.filename == "declarations.f90" - assert error.value.code == "PARSE_DUPLICATE_FIELD" - - -def test_unknown_procedure_declaration_diagnostic_preserves_public_metadata(): - parser = FortranParser() - state = parser._new_procedure_scope_state( - FortranProcedureSignature(name="work", kind="subroutine"), - symbols={}, - ) - - with pytest.raises(FortranParseError) as error: - parser._handle_unknown_proc_declaration( - "@@@", - state, - filename="procedure_contract.f90", - lineno=8, - source_line="@@@", - ) - - assert error.value.base_message == "Invalid Fortran syntax in procedure 'work' specification part: @@@" - assert error.value.filename == "procedure_contract.f90" - assert error.value.line_number == 8 - assert error.value.source_line == "@@@" - assert error.value.code == "PARSE_INVALID_SYNTAX" - - -def test_contains_line_validation_accepts_spec_alternatives_without_mutating_scope_and_reports_invalid_lines(): - parser = FortranParser() - module = FortranModule("owner_mod") - scope = _ParserScope(kind="module", name=module.name, model=module, module_owner=module.name) - - parser._helper_validate_contains_lines( - scope, - _lines("", "# marker", "include 'shape.inc'", "integer :: macro_decl"), - filename="contains_contract.f90", - ) - - assert module.variables == [] - - with pytest.raises(FortranParseError) as error: - parser._helper_validate_contains_lines( - scope, - _lines("@@@"), - filename="contains_contract.f90", - ) - - assert error.value.base_message == "Invalid Fortran syntax in module 'owner_mod' contains part: @@@" - assert error.value.filename == "contains_contract.f90" - assert error.value.line_number == 1 - assert error.value.source_line == "@@@" - assert error.value.code == "PARSE_INVALID_SYNTAX" - - -def test_interface_validation_keeps_scanning_after_valid_lines(): - parser = FortranParser() - scope = _ParserScope(kind="interface", name="Callbacks") - - with pytest.raises(FortranParseError) as interface_error: - parser._helper_validate_interface_lines( - scope, - _lines("", "# marker", "MODULE PROCEDURE :: First, Second", "PROCEDURE(Callback) :: Handler", "@@@"), - filename="interface_contract.f90", - ) - assert interface_error.value.base_message == "Invalid Fortran syntax in interface 'Callbacks': @@@" - assert interface_error.value.filename == "interface_contract.f90" - assert interface_error.value.line_number == 5 - assert interface_error.value.source_line == "@@@" - assert interface_error.value.code == "PARSE_INVALID_SYNTAX" - - -@pytest.mark.parametrize( - ("use_enum_validator", "unit", "expected_message"), - [ - ( - False, - _unit( - "interface", "callbacks", "interface callbacks", "type :: nested", "end type nested", "end interface" - ), - "Invalid Fortran syntax in interface 'callbacks': type :: nested", - ), - ( - False, - _unit("derived_type", "outer", "type :: outer", "type :: nested", "end type nested", "end type outer"), - "Invalid Fortran syntax in derived type 'outer' specification part: type :: nested", - ), - ( - False, - _unit( - "block_data", - "init_data", - "block data init_data", - "type :: nested", - "end type nested", - "end block data init_data", - ), - "Invalid Fortran syntax in block data 'init_data' specification part: type :: nested", - ), - ], -) -def test_nested_units_rejected_by_restricted_scopes_preserve_public_metadata( - use_enum_validator, unit, expected_message -): - parser = FortranParser() - - with pytest.raises(FortranParseError) as error: - if use_enum_validator: - parser._helper_validate_enum_unit(unit, filename="nested_contract.f90") - else: - parser._visit( - unit, - parent_scope=_ParserScope(kind="file", name=None), - filename="nested_contract.f90", - ) - - assert error.value.base_message == expected_message - assert error.value.filename == "nested_contract.f90" - assert error.value.line_number == 2 - assert error.value.source_line == "type :: nested" - assert error.value.code == "PARSE_INVALID_SYNTAX" - - -@pytest.mark.parametrize( - ("line", "expected_message", "expected_code"), - [ - ( - "!$omp threadprivate(counter)", - "Unsupported OpenMP declarative directive in module 'owner_mod': !$omp threadprivate(counter)", - "PARSE_UNSUPPORTED_OPENMP_DIRECTIVE", - ), - ( - "type :: missing_end", - "Missing end derived type for derived type 'missing_end'.", - "PARSE_MISSING_DERIVED_TYPE_END", - ), - ( - "call work()", - "Executable statement is not allowed in module specification part 'owner_mod': call work()", - "PARSE_EXECUTABLE_IN_SPECIFICATION", - ), - ( - "@@@", - "Invalid Fortran syntax in module 'owner_mod' specification part: @@@", - "PARSE_INVALID_SYNTAX", - ), - ( - "weirdtype value", - "Unknown or unsupported datatype declaration in module 'owner_mod': weirdtype value", - "PARSE_UNSUPPORTED_DECLARATION", - ), - ], -) -def test_module_like_spec_diagnostics_preserve_public_metadata(line, expected_message, expected_code): - parser = FortranParser() - module = FortranModule("owner_mod") - scope = _ParserScope(kind="module", name=module.name, model=module, module_owner=module.name) - with pytest.raises(FortranParseError) as error: - parser._parse_module_like_spec_line( - scope, - line, - filename="module_contract.f90", - lineno=7, - source_line=line, - ) - - assert error.value.base_message == expected_message - assert error.value.filename == "module_contract.f90" - assert error.value.line_number == 7 - assert error.value.source_line == line - assert error.value.code == expected_code - - -@pytest.mark.parametrize( - ("line", "expected_message", "expected_code"), - [ - ( - "type :: missing_end", - "Missing end derived type for derived type 'missing_end'.", - "PARSE_MISSING_DERIVED_TYPE_END", - ), - ( - "!$omp threadprivate(counter)", - "Unsupported OpenMP declarative directive in type 'state_t': !$omp threadprivate(counter)", - "PARSE_UNSUPPORTED_OPENMP_DIRECTIVE", - ), - ( - "@@@", - "Invalid Fortran syntax in type 'state_t' specification part: @@@", - "PARSE_INVALID_SYNTAX", - ), - ( - "weirdtype value", - "Unknown or unsupported datatype declaration in type 'state_t': weirdtype value", - "PARSE_UNSUPPORTED_DECLARATION", - ), - ], -) -def test_type_spec_diagnostics_preserve_public_metadata(line, expected_message, expected_code): - parser = FortranParser() - dtype = FortranDerivedType("state_t") - scope = _ParserScope(kind="derived_type", name=dtype.name, model=dtype) - - with pytest.raises(FortranParseError) as error: - parser._parse_type_spec_line( - line, - scope, - filename="type_contract.f90", - lineno=7, - source_line=line, - ) - - assert error.value.base_message == expected_message - assert error.value.filename == "type_contract.f90" - assert error.value.line_number == 7 - assert error.value.source_line == line - assert error.value.code == expected_code +from prik.parsers.fortran import FortranParseError, FortranParser, parse_fortran_file @pytest.mark.parametrize( @@ -574,58 +27,6 @@ def test_singular_parser_entrypoint_diagnostics_preserve_names_entities_and_file assert error.value.code == "PARSE_WRONG_ENTRYPOINT" -@pytest.mark.parametrize( - ("header_parser", "header_result", "unit_kind", "entity_name"), - [ - ("_parse_module_header", None, "module", "module"), - ("_parse_submodule_header", None, "submodule", "submodule"), - ("_parse_program_header", None, "program", "program"), - ("_parse_block_data_header", None, "block_data", "block data"), - ("_init_derived_type", None, "derived_type", "derived-type"), - ], -) -def test_source_unit_visitor_defensive_diagnostics_preserve_public_metadata( - monkeypatch, - header_parser, - header_result, - unit_kind, - entity_name, -): - parser = FortranParser() - monkeypatch.setattr(parser, header_parser, lambda *args, **kwargs: header_result) - - with pytest.raises(FortranParseError) as error: - parser._visit( - _unit(unit_kind, "broken", "broken header", "broken footer"), - parent_scope=_ParserScope(kind="file", name=None), - filename="visitor_contract.f90", - ) - - assert error.value.base_message == f"Expected {entity_name} unit." - assert error.value.filename == "visitor_contract.f90" - assert error.value.line_number == 1 - assert error.value.source_line == "broken header" - assert error.value.code == "PARSE_EXPECTED_UNIT" - - -def test_interface_unit_defensive_diagnostic_uses_scanner_header_recognition(monkeypatch): - parser = FortranParser() - monkeypatch.setattr(parser._source_unit_scanner, "parse_interface_header", lambda _line: (False, None)) - - with pytest.raises(FortranParseError) as error: - parser._visit( - _unit("interface", "broken", "broken header", "broken footer"), - parent_scope=_ParserScope(kind="file", name=None), - filename="visitor_contract.f90", - ) - - assert error.value.base_message == "Expected interface unit." - assert error.value.filename == "visitor_contract.f90" - assert error.value.line_number == 1 - assert error.value.source_line == "broken header" - assert error.value.code == "PARSE_EXPECTED_UNIT" - - def test_unit_models_preserve_filename_propagation(): parsed = parse_fortran_file( """ diff --git a/tests/fortran/infrastructure/pipeline/test_wrapper.py b/tests/fortran/infrastructure/pipeline/test_wrapper.py deleted file mode 100644 index 3dc5b9e18..000000000 --- a/tests/fortran/infrastructure/pipeline/test_wrapper.py +++ /dev/null @@ -1,28 +0,0 @@ -"""Internal generated-wrapper handoff contracts.""" - -from __future__ import annotations - -from pathlib import Path - -from prik.pipeline.wrapper import GeneratedSource, GeneratedWrapper - - -def test_generated_wrapper_keeps_compile_and_link_ownership_out_of_the_handoff(): - wrapper = GeneratedWrapper( - module_name="demo", - sources=( - GeneratedSource(Path("bind_c_demo.f90"), "module bind_c_demo\nend module bind_c_demo\n"), - GeneratedSource(Path("demo.c"), "PyObject *demo;\n"), - GeneratedSource(Path("demo.h"), "#pragma once\n"), - ), - bridge_sources=(Path("bind_c_demo.f90"),), - binding_sources=(Path("demo.c"),), - headers=(Path("demo.h"),), - native_support_keys=("binding_support",), - required_headers=(), - extension_init_name="PyInit_demo", - ) - - assert wrapper.compile_sources == (Path("bind_c_demo.f90"), Path("demo.c")) - assert wrapper.generated_files == (Path("bind_c_demo.f90"), Path("demo.c"), Path("demo.h")) - assert wrapper.source_paths == wrapper.generated_files diff --git a/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py b/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py index 8d4b1a3fe..91c0f2d8f 100644 --- a/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py +++ b/tests/fortran/infrastructure/pipeline/test_wrapper_generator.py @@ -12,14 +12,8 @@ from prik.policy.ownership import CodegenAction, NativeBarrierAction, ObjectKind from prik.policy.models import ArgumentHandoffMode, BridgeDataAction from prik.utilities.stage_values import FrozenStageRecordError -from prik.codegen import ( - CBindingGenerator, - FortranBridgeGenerator, -) -from prik.codegen.docstrings import WrapperDocstringBuilder from prik.pipeline.wrapper import WrapperGenerator from prik.planning import NamespacePlan, WrapperPlanner -from prik.printers import CSourcePrinter, FortranSourcePrinter def _rendered_source(generated_wrapper, suffix: str) -> str: @@ -99,6 +93,9 @@ def swap_args(x: Float64, y: Float64) -> Float64: ... Path("render_demo_wrapper.c"), Path("render_demo_wrapper.h"), ) + # The handoff names what the build compiles; headers are generated but never compiled. + assert generated_wrapper.compile_sources == (Path("bind_c_render_demo_wrapper.f90"), Path("render_demo_wrapper.c")) + assert generated_wrapper.generated_files == generated_wrapper.source_paths assert generated_wrapper.extension_init_name == "PyInit_render_demo" assert "double bind_c_swap_args(double * y, double * x);" in c_source assert 'static char * kwlist[] = {"x", "y", NULL};' in c_source @@ -115,24 +112,6 @@ def swap_args(x: Float64, y: Float64) -> Float64: ... assert "result = SWAP_ARGS(y, x)" in fortran_source -def test_public_generator_reports_each_rendering_operation_in_execution_order(): - plan = _plan("def value(x: Float64) -> Float64: ...", module_name="render_progress") - progress = [] - - WrapperGenerator().generate(plan, progress=lambda label, elapsed: progress.append((label, elapsed))) - - assert [label for label, _ in progress] == [ - "Generate binding source", - "Generate binding source", - "Generate bridge source", - "Generate bridge source", - "Generate binding header", - "Generate binding header", - ] - assert [elapsed is None for _, elapsed in progress] == [True, False, True, False, True, False] - assert all(elapsed >= 0.0 for _, elapsed in progress if elapsed is not None) - - def test_procedure_only_binding_stays_one_compile_unit_at_any_size(): declarations = "\n".join(f"def value_{index:03d}(x: Float64) -> Float64: ..." for index in range(128)) @@ -153,44 +132,6 @@ def test_procedure_only_binding_stays_one_compile_unit_at_any_size(): ) -@pytest.mark.parametrize( - ("source", "c_fragment", "fortran_fragment"), - [ - ( - "def required_value(x: Float64) -> Float64: ...", - "PyObject * bound_x_obj;", - "result = native_required_value(x)", - ), - ( - "def optional_value(x: Int32 = ...) -> Int32: ...", - "PyObject * bound_x_obj = Py_None;", - "if (c_associated(bound_x)) then", - ), - ( - """ -@native_call([Allocatable(Arg(0))]) -def descriptor_value(value: Annotated[Float64, Immutable] | None = ...) -> Int32: ... -""", - "PyObject * bound_value_obj = NULL;", - "type(c_ptr), value :: bound_value_present", - ), - ( - """ -@native_call([Addr(Arg(0)), Return("result", 0)]) -def hidden_value(x: Float64) -> Float64: ... -""", - "void bind_c_hidden_value(double * x, double * result);", - "subroutine bind_c_hidden_value(x, result)", - ), - ], -) -def test_supported_function_actions_select_their_backend_behavior(source, c_fragment, fortran_fragment): - generated_wrapper = WrapperGenerator().generate(_plan(source, module_name="action_dispatch")) - - assert c_fragment in _rendered_source(generated_wrapper, ".c") - assert fortran_fragment in _rendered_source(generated_wrapper, ".f90") - - def test_direct_plan_edits_change_binding_and_bridge_generation_then_freeze_plan(): plan = _plan( """ @@ -249,90 +190,37 @@ def test_entrypoint_symbol_edit_changes_both_sides_of_shared_c_abi(): assert 'bind(c, name="custom_scale_entrypoint")' in fortran_source -def test_backend_visitors_return_complete_nodes_and_printers_freeze_them(): - plan = _plan( - """ -@bind("SCALE") -@native_call([Int32(1), Arg(0), Bool(False)]) -def scale(x: Float64) -> Float64: ... -""", - module_name="backend_nodes", - ) - c_generator = CBindingGenerator() - fortran_generator = FortranBridgeGenerator() - WrapperDocstringBuilder().render(plan) - c_generator.require_supported(plan) - fortran_generator.require_supported(plan) - - c_module, c_header = c_generator.visit(plan) - fortran_module = fortran_generator.visit(plan) - - assert [function.name for function in c_module.functions] == ["wrap_scale", "PyInit_backend_nodes"] - assert [prototype.name for prototype in c_header.prototypes] == ["wrap_scale"] - assert [procedure.name for procedure in fortran_module.procedures] == ["bind_c_scale"] - assert "result = native_scale(literal_0, x, literal_2)" in FortranSourcePrinter().doprint(fortran_module) - CSourcePrinter().doprint(c_module) - with pytest.raises(FrozenStageRecordError): - c_module.name = "later" - with pytest.raises(FrozenStageRecordError): - fortran_module.name = "later" - - -def test_generator_rejects_unregistered_typed_lowering_combination(): - plan = _plan( - """ -def scale(x: Float64) -> Float64: ... -""", - module_name="unsupported_lowering", - ) +def _unregistered_optional_mode(plan): function = plan.namespaces[0].functions[0] + argument = function.arguments[0] invalid_argument = replace( - function.arguments[0], - binding=replace(function.arguments[0].binding, optional_mode="x"), - entrypoint=replace(function.arguments[0].entrypoint, optional_mode="x"), - ) - root = plan.namespaces[0] - invalid = replace( - plan, - namespaces=(replace(root, functions=(replace(function, arguments=(invalid_argument,)),)),), + argument, + binding=replace(argument.binding, optional_mode="x"), + entrypoint=replace(argument.entrypoint, optional_mode="x"), ) + return _edit_first_function(plan, lambda item: replace(item, arguments=(invalid_argument, *item.arguments[1:]))) - with pytest.raises(ValueError, match="Unsupported C argument optional mode"): - WrapperGenerator().generate(invalid) - -def test_generator_rejects_hidden_result_native_action_disagreement(): - plan = _hidden_result_plan() - function = plan.namespaces[0].functions[0] - result = function.results[0] +def _hidden_result_native_action(plan): + result = plan.namespaces[0].functions[0].results[0] replacement = ( NativeBarrierAction.PASS_VALUE if result.bridge.native_action is not NativeBarrierAction.PASS_VALUE else NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS ) - invalid = _edit_first_function( - plan, - lambda item: replace( - item, - results=(replace(result, bridge=replace(result.bridge, native_action=replacement)),), - ), - ) + edited = replace(result, bridge=replace(result.bridge, native_action=replacement)) + return _edit_first_function(plan, lambda item: replace(item, results=(edited,))) - with pytest.raises(ValueError, match="inconsistent-result-native-action"): - WrapperGenerator().generate(invalid) - -def test_generator_rejects_hidden_result_slot_codegen_action_disagreement(): - plan = _hidden_result_plan() +def _hidden_result_slot_codegen_action(plan): function = plan.namespaces[0].functions[0] result = function.results[0] original_slot = result.projected_call_slot - edited_adapter = replace(original_slot.adapter, codegen_action=CodegenAction.COPY_OUT) - edited_slot = replace(original_slot, adapter=edited_adapter) + edited_slot = replace(original_slot, adapter=replace(original_slot.adapter, codegen_action=CodegenAction.COPY_OUT)) projected_slots = tuple( edited_slot if slot is original_slot else slot for slot in function.entrypoint.projected_slots ) - invalid = _edit_first_function( + return _edit_first_function( plan, lambda item: replace( item, @@ -341,127 +229,105 @@ def test_generator_rejects_hidden_result_slot_codegen_action_disagreement(): ), ) - with pytest.raises(ValueError, match="inconsistent-result-slot-codegen-action"): - WrapperGenerator().generate(invalid) - - -def test_generator_rejects_argument_native_slot_object_kind_disagreement(): - plan = _scalar_plan() - argument = plan.namespaces[0].functions[0].arguments[0] - argument.projected_call_slot.object_kind = ObjectKind.STRING - - with pytest.raises(ValueError, match="inconsistent-argument-object-kind"): - WrapperGenerator().generate(plan) +def _argument_object_kind(plan): + plan.namespaces[0].functions[0].arguments[0].projected_call_slot.object_kind = ObjectKind.STRING + return plan -def test_generator_rejects_result_native_slot_object_kind_disagreement(): - plan = _hidden_result_plan() - result = plan.namespaces[0].functions[0].results[0] - result.projected_call_slot.object_kind = ObjectKind.STRING - with pytest.raises(ValueError, match="inconsistent-result-object-kind"): - WrapperGenerator().generate(plan) +def _result_object_kind(plan): + plan.namespaces[0].functions[0].results[0].projected_call_slot.object_kind = ObjectKind.STRING + return plan -def test_generator_rejects_advertised_role_without_a_plan_producer(): - invalid = _edit_first_function( - _scalar_plan(), - lambda function: replace(function, available_roles=(*function.available_roles, "invented:role")), +def _invented_role(plan): + return _edit_first_function( + plan, lambda function: replace(function, available_roles=(*function.available_roles, "invented:role")) ) - with pytest.raises(ValueError, match="inconsistent-available-roles"): - WrapperGenerator().generate(invalid) - -def test_generator_rejects_duplicate_python_exports_before_lowering(): - plan = _scalar_plan() +def _duplicate_python_export(plan): root = plan.namespaces[0] function = root.functions[0] duplicate = replace(function, symbol_name="other_symbol") - invalid = replace(plan, namespaces=(replace(root, functions=(function, duplicate)),)) + return replace(plan, namespaces=(replace(root, functions=(function, duplicate)),)) - with pytest.raises(ValueError, match="duplicate-python-export"): - WrapperGenerator().generate(invalid) - -def test_generator_rejects_duplicate_generated_symbols_before_lowering(): - plan = _scalar_plan() +def _duplicate_generated_symbol(plan): root = plan.namespaces[0] function = root.functions[0] duplicate = replace( - function, - owner_path="runtime_policy.other", - binding=replace(function.binding, python_name="other"), + function, owner_path="runtime_policy.other", binding=replace(function.binding, python_name="other") ) - invalid = replace(plan, namespaces=(replace(root, functions=(function, duplicate)),)) - - with pytest.raises(ValueError, match="duplicate-generated-symbol"): - WrapperGenerator().generate(invalid) + return replace(plan, namespaces=(replace(root, functions=(function, duplicate)),)) -def test_generator_rejects_colliding_generated_namespace_symbols(): - plan = _scalar_plan() - invalid = replace( +def _colliding_namespace(plan): + return replace( plan, - namespaces=( - *plan.namespaces, - NamespacePlan(owner_path="runtime_policy.root", python_path=("root",)), - ), + namespaces=(*plan.namespaces, NamespacePlan(owner_path="runtime_policy.root", python_path=("root",))), ) - with pytest.raises(ValueError, match="duplicate-generated-namespace-symbol"): - WrapperGenerator().generate(invalid) + +def _foreign_binding_owner(plan): + return replace(plan, binding=replace(plan.binding, owner_path="other")) + + +def _edit_first_argument(plan, edit): + return _edit_first_function( + plan, lambda function: replace(function, arguments=(edit(function.arguments[0]), *function.arguments[1:])) + ) + + +def _out_of_range_python_position(plan): + return _edit_first_argument(plan, lambda argument: replace(argument, python_position=99)) + + +def _inconsistent_native_handoff(plan): + return _edit_first_argument( + plan, lambda argument: replace(argument, entrypoint=replace(argument.entrypoint, handoff_role="other:role")) + ) @pytest.mark.parametrize( - ("mutate", "expected_code"), + ("plan_factory", "edit", "diagnostic"), [ - ( - lambda plan: replace( - plan, - binding=replace(plan.binding, owner_path="other"), - ), - "binding-module-owner", + pytest.param( + _scalar_plan, _unregistered_optional_mode, "Unsupported C argument optional mode", id="optional-mode" + ), + pytest.param( + _hidden_result_plan, + _hidden_result_native_action, + "inconsistent-result-native-action", + id="result-native-action", ), - ( - lambda plan: _edit_first_function( - plan, - lambda function: replace( - function, - arguments=( - replace(function.arguments[0], python_position=99), - function.arguments[1], - ), - ), - ), - "out-of-range-python-position", + pytest.param( + _hidden_result_plan, + _hidden_result_slot_codegen_action, + "inconsistent-result-slot-codegen-action", + id="result-slot-codegen-action", ), - ( - lambda plan: _edit_first_function( - plan, - lambda function: replace( - function, - arguments=( - replace( - function.arguments[0], - entrypoint=replace( - function.arguments[0].entrypoint, - handoff_role="other:role", - ), - ), - function.arguments[1], - ), - ), - ), - "inconsistent-native-handoff", + pytest.param( + _scalar_plan, _argument_object_kind, "inconsistent-argument-object-kind", id="argument-object-kind" ), + pytest.param( + _hidden_result_plan, _result_object_kind, "inconsistent-result-object-kind", id="result-object-kind" + ), + pytest.param(_scalar_plan, _invented_role, "inconsistent-available-roles", id="role-without-producer"), + pytest.param(_scalar_plan, _duplicate_python_export, "duplicate-python-export", id="duplicate-python-export"), + pytest.param(_scalar_plan, _duplicate_generated_symbol, "duplicate-generated-symbol", id="duplicate-symbol"), + pytest.param( + _scalar_plan, _colliding_namespace, "duplicate-generated-namespace-symbol", id="colliding-namespace-symbol" + ), + pytest.param(_scalar_plan, _foreign_binding_owner, "binding-module-owner", id="binding-module-owner"), + pytest.param(_scalar_plan, _out_of_range_python_position, "out-of-range-python-position", id="python-position"), + pytest.param(_scalar_plan, _inconsistent_native_handoff, "inconsistent-native-handoff", id="native-handoff"), ], ) -def test_generator_revalidates_direct_plan_edits(mutate, expected_code): - invalid = mutate(_scalar_plan()) - - with pytest.raises(ValueError, match=expected_code): - WrapperGenerator().generate(invalid) +def test_generator_revalidates_edited_plans_before_lowering(plan_factory, edit, diagnostic): + """A plan stays editable until generation, so every cross-stage agreement is checked again there.""" + with pytest.raises(ValueError, match=diagnostic): + WrapperGenerator().generate(edit(plan_factory())) @pytest.mark.parametrize( @@ -534,9 +400,6 @@ def test_scalar_copy_in_out_reuses_one_binding_local_without_bridge_copy(): assert c_source.count("int32_t bound_value;") == 1 assert "prik_int32_or_storage(bound_value_obj, NPY_INT32, " in c_source - assert "bind_c_bump(&bound_value);" in c_source - assert "PyObject * result_obj = NULL;" in c_source - assert "result_obj = prik_int32_to_numpy(&bound_value);" in c_source assert "integer(c_int32_t) :: value" in bridge_source assert "call native_bump(value)" in bridge_source assert "value =" not in bridge_source diff --git a/tests/fortran/infrastructure/policy/test_native_array_handles.py b/tests/fortran/infrastructure/policy/test_native_array_handles.py deleted file mode 100644 index b071a068f..000000000 --- a/tests/fortran/infrastructure/policy/test_native_array_handles.py +++ /dev/null @@ -1,45 +0,0 @@ -"""Internal native-array handle policy dispatch contracts.""" - -from prik.policy.native_array_handles import ( - ArrayInteropPolicy, - ArrayInteropPolicyDispatcher, -) - - -def test_array_interop_dispatcher_routes_completed_abi_selector_to_named_method(): - class Subject: - name = "values" - - class Target: - def data_buffer(self, subject, policy, marker): - return marker, subject.name, policy.abi - - def descriptor(self, subject, policy, marker): - return marker, subject.name, policy.abi, policy.descriptor_kind - - dispatcher = ArrayInteropPolicyDispatcher( - { - ("argument", "data_buffer"): "data_buffer", - ("argument", "descriptor"): "descriptor", - }, - ) - - assert dispatcher.dispatch( - Target(), - Subject(), - ArrayInteropPolicy(abi="data_buffer", owner="argument values"), - "argument", - "seen", - ) == ("seen", "values", "data_buffer") - assert dispatcher.dispatch( - Target(), - Subject(), - ArrayInteropPolicy( - abi="descriptor", - owner="argument values", - descriptor_kind="allocatable", - handle_kind="argument_descriptor", - ), - "argument", - "seen", - ) == ("seen", "values", "descriptor", "allocatable") diff --git a/tests/fortran/infrastructure/policy/test_ownership.py b/tests/fortran/infrastructure/policy/test_ownership.py index 2fdbe2a29..2f763f2a7 100644 --- a/tests/fortran/infrastructure/policy/test_ownership.py +++ b/tests/fortran/infrastructure/policy/test_ownership.py @@ -11,7 +11,6 @@ OwnershipContext, OwnershipDecision, OwnershipOwner, - OwnershipPolicyResolver, PolicyActionDispatcher, PythonBarrierAction, PythonBarrierDispatcher, @@ -210,27 +209,6 @@ def test_default_policy_completes_non_raw_python_and_native_barrier_actions(): assert decision.native_barrier_action is native_action, label -def test_policy_handler_dictionary_changes_one_object_kind(): - def native_scalar_handler(_facts, _context): - return OwnershipDecision( - ObjectKind.SCALAR, - OwnershipOwner.NATIVE, - TransferMode.BORROWED_VIEW, - DestructionPolicy.NATIVE_OWNER, - borrowed=True, - ) - - resolver = OwnershipPolicyResolver({ObjectKind.SCALAR: native_scalar_handler}) - - scalar = resolver.decide_semantic_type(_scalar_type(), OwnershipContext.result()) - array = resolver.decide_semantic_type(_array_type(allocatable=True), OwnershipContext.result()) - - assert scalar.owner is OwnershipOwner.NATIVE - assert scalar.transfer is TransferMode.BORROWED_VIEW - assert array.owner is OwnershipOwner.WRAPPER - assert array.transfer is TransferMode.WRAPPER_INSTANCE - - def test_explicit_ownership_override_preserves_normalized_fields_and_storage_invariants(): metadata: dict[str, object] = {} set_ownership_metadata( @@ -282,85 +260,20 @@ def test_borrowed_pointer_override_blocks_before_unrelated_destruction_validatio assert decision.blocker == ("borrowed pointer views need native-owner retention and stale-view invalidation") -def test_codegen_action_dispatcher_routes_policy_actions_to_named_methods(): - class FakeVar: - rank = 1 - ownership_decision = OwnershipDecision( - ObjectKind.NUMPY_ARRAY, - OwnershipOwner.PYTHON, - TransferMode.SNAPSHOT_COPY, - DestructionPolicy.PYTHON_REFCOUNT, - storage_mode=StorageMode.ALIAS, - codegen_action=CodegenAction.SNAPSHOT_COPY, - ) - - class Target: - def snapshot(self, var, decision, marker): - return marker, var.rank, decision.codegen_action - - dispatcher = PolicyActionDispatcher( - {(ObjectKind.NUMPY_ARRAY, CodegenAction.SNAPSHOT_COPY): "snapshot"}, - ) - - assert dispatcher.dispatch(Target(), FakeVar(), "seen") == ( - "seen", - 1, - CodegenAction.SNAPSHOT_COPY, +def test_policy_dispatchers_reject_a_missing_completed_action_instead_of_falling_back(): + """Lowering dispatches completed actions; an unhandled one is a policy bug, not a default.""" + string_input = OwnershipDecision( + ObjectKind.STRING, + OwnershipOwner.TEMPORARY, + TransferMode.CALL_LOCAL, + DestructionPolicy.CALL_LOCAL, + codegen_action=CodegenAction.CALL_LOCAL_INPUT, ) - -def test_codegen_action_dispatcher_rejects_missing_policy_pairs(): class FakeVar: - ownership_decision = OwnershipDecision( - ObjectKind.STRING, - OwnershipOwner.TEMPORARY, - TransferMode.CALL_LOCAL, - DestructionPolicy.CALL_LOCAL, - codegen_action=CodegenAction.CALL_LOCAL_INPUT, - ) - - dispatcher = PolicyActionDispatcher({}) - - with pytest.raises(ValueError, match="string/call_local_input"): - dispatcher.handler_name(FakeVar()) - + ownership_decision = string_input -def test_barrier_dispatchers_route_completed_actions_to_named_methods(): - class FakeVar: - ownership_decision = OwnershipDecision( - ObjectKind.SCALAR, - OwnershipOwner.CALLER, - TransferMode.CALL_LOCAL, - DestructionPolicy.NONE, - codegen_action=CodegenAction.CALL_LOCAL_INPUT, - python_barrier_action=PythonBarrierAction.SCALAR_VALUE, - native_barrier_action=NativeBarrierAction.PASS_VALUE, - ) - - class Target: - def python_scalar(self, var, decision, marker): - return marker, var.ownership_decision.python_barrier_action, decision.python_barrier_action - - def native_value(self, var, decision, marker): - return marker, var.ownership_decision.native_barrier_action, decision.native_barrier_action - - python_dispatcher = PythonBarrierDispatcher({PythonBarrierAction.SCALAR_VALUE: "python_scalar"}) - native_dispatcher = NativeBarrierDispatcher({NativeBarrierAction.PASS_VALUE: "native_value"}) - - assert python_dispatcher.dispatch(Target(), FakeVar(), "py") == ( - "py", - PythonBarrierAction.SCALAR_VALUE, - PythonBarrierAction.SCALAR_VALUE, - ) - assert native_dispatcher.dispatch(Target(), FakeVar(), "native") == ( - "native", - NativeBarrierAction.PASS_VALUE, - NativeBarrierAction.PASS_VALUE, - ) - - -def test_barrier_dispatchers_reject_missing_completed_actions(): - decision = OwnershipDecision( + raw_address = OwnershipDecision( ObjectKind.SCALAR, OwnershipOwner.CALLER, TransferMode.CALL_LOCAL, @@ -370,7 +283,9 @@ def test_barrier_dispatchers_reject_missing_completed_actions(): native_barrier_action=NativeBarrierAction.PASS_RAW_ADDRESS, ) + with pytest.raises(ValueError, match="string/call_local_input"): + PolicyActionDispatcher({}).handler_name(FakeVar()) with pytest.raises(ValueError, match="Python-barrier handler"): - PythonBarrierDispatcher({}).handler_name_for_decision(decision, "x") + PythonBarrierDispatcher({}).handler_name_for_decision(raw_address, "x") with pytest.raises(ValueError, match="native-barrier handler"): - NativeBarrierDispatcher({}).handler_name_for_decision(decision, "x") + NativeBarrierDispatcher({}).handler_name_for_decision(raw_address, "x") diff --git a/tests/fortran/infrastructure/policy/test_wrapper_policy.py b/tests/fortran/infrastructure/policy/test_wrapper_policy.py index f0e6a5b91..fe312da78 100644 --- a/tests/fortran/infrastructure/policy/test_wrapper_policy.py +++ b/tests/fortran/infrastructure/policy/test_wrapper_policy.py @@ -13,8 +13,6 @@ from prik.semantics.models import ( RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, RESOLVED_RUNTIME_STATUS_ERROR_POLICY_METADATA, - SemanticFunction, - SemanticType, ) from prik.policy.ownership import ( CodegenAction, @@ -26,15 +24,13 @@ from prik.policy.completion import complete_semantic_policies from prik.policy.models import ( ArgumentConversionPhase, - ArgumentHandoffMode, BridgeDataAction, ExternalDeclarationMode, - FunctionWrapperPolicy, NativeStatusErrorPolicy, OptionalMode, PythonExceptionKind, ) -from prik.policy.construction import build_function_wrapper_policy, completed_function_wrapper_policy +from prik.policy.construction import completed_function_wrapper_policy FMATH_CONTRACT = Path("tests/fortran/data_types/end_to_end/fixtures/contracts/fmath/__init__.pyi") @@ -49,24 +45,84 @@ def _source_semantic_module(filename: str, *, module_name: str): return module -def test_fmath_fixture_gets_completed_function_wrapper_policy(): - module = pyi_file_to_semantic_module(FMATH_CONTRACT, module_name="fmath") - - complete_semantic_policies(module) - - policies = [function.metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] for function in module.functions] - assert policies - assert all(isinstance(policy, FunctionWrapperPolicy) for policy in policies) - assert all(policy.supported for policy in policies) - assert all(policy.blockers == () for policy in policies) - assert all(policy.writeback_actions for policy in policies) - assert all( - argument.conversion_phase is ArgumentConversionPhase.IMMEDIATE - for policy in policies +@pytest.mark.parametrize( + ("lane", "native_name", "slot_value_kind"), + [ + pytest.param("source", "ADD_R8", "arg", id="fortran_source"), + # The contract states no separate native name: `add_r8` reaches + # Fortran's `ADD_R8`, which is named without regard to case. + pytest.param("contract", "add_r8", "addr", id="pyi_contract"), + ], +) +def test_fmath_scalar_replacements_complete_one_policy_from_source_and_contract(lane, native_name, slot_value_kind): + if lane == "source": + module = _source_semantic_module("fmath.f", module_name="fmath") + else: + module = pyi_file_to_semantic_module(FMATH_CONTRACT, module_name="fmath") + complete_semantic_policies(module) + policies = {function.name.casefold(): completed_function_wrapper_policy(function) for function in module.functions} + + # Every conservative scalar replacement is supported and needs no cleanup. + for policy in policies.values(): + assert policy.supported is True + assert policy.blockers == () + assert policy.writeback_actions + assert policy.cleanup_actions == () + assert policy.release_actions == () + assert all(argument.conversion_phase is ArgumentConversionPhase.IMMEDIATE for argument in policy.arguments) + + policy = policies["add_r8"] + assert policy.owner_path.casefold() == "fmath.add_r8" + assert [(export.namespace, export.name) for export in policy.python_exports] == [((), "add_r8")] + assert policy.native_name == native_name + assert policy.standalone is True + assert [ + ( + argument.name, + argument.python_position, + argument.native_position, + argument.codegen_action, + argument.python_barrier_action, + argument.native_barrier_action, + argument.storage_mode, + ) for argument in policy.arguments - ) - assert all(policy.cleanup_actions == () for policy in policies) - assert all(policy.release_actions == () for policy in policies) + ] == [ + ( + name, + position, + position, + CodegenAction.COPY_IN_OUT, + PythonBarrierAction.SCALAR_VALUE, + NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS, + StorageMode.STACK, + ) + for position, name in enumerate(("X", "Y")) + ] + assert [ + ( + slot.source_kind, + slot.value_kind, + slot.native_position, + slot.python_position, + slot.native_barrier_action, + slot.codegen_action, + ) + for slot in policy.native_call_slots + ] == [ + ( + "projection", + slot_value_kind, + position, + position, + NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS, + CodegenAction.COPY_IN_OUT, + ) + for position in (0, 1) + ] + [result] = policy.results + assert result.codegen_action is CodegenAction.DIRECT_VALUE + assert result.storage_mode is StorageMode.STACK def test_rank_zero_scalar_storage_results_complete_as_numpy_array_policies(): @@ -105,30 +161,6 @@ def hidden_storage_result() -> Float64[()]: ... assert hidden_policy.native_call_slots[0].native_barrier_action is hidden.native_barrier_action -def test_hidden_result_policy_reports_a_missing_return_projection_after_selection(): - module = parse_pyi_text( - """ -@native_call([Return("status", 0)]) -def hidden_status() -> Int32: ... -""", - module_name="missing_hidden_projection", - ) - complete_semantic_policies(module) - function = module.functions[0] - - # Preserve the completed hidden-output ownership decision while removing - # its result mapping to characterize the candidate builder's fail-closed path. - function.projection = [] - policy = build_function_wrapper_policy( - function, - owner_path="missing_hidden_projection.hidden_status", - module_export=True, - ) - - assert policy.results == () - assert "hidden result 'status' has no completed return projection" in policy.blockers - - def test_hidden_result_policy_keeps_blocked_bridge_action_on_the_candidate(): module = parse_pyi_text( """ @@ -164,56 +196,6 @@ def optional(value: Annotated[Float64, Immutable] | None = ...) -> None: ... assert optional.external_declaration is ExternalDeclarationMode.EXPLICIT_INTERFACE -def test_source_fmath_scalar_policy_projects_conservative_replacements(): - module = _source_semantic_module("fmath.f", module_name="fmath") - function = next(item for item in module.functions if item.name == "ADD_R8") - policies = [item.metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] for item in module.functions] - - policy = function.metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert isinstance(policy, FunctionWrapperPolicy) - assert policy.supported is True - assert policy.blockers == () - assert [(export.namespace, export.name) for export in policy.python_exports] == [((), "add_r8")] - assert policy.native_name == "ADD_R8" - assert policy.standalone is True - assert [argument.name for argument in policy.arguments] == ["X", "Y"] - assert [argument.codegen_action for argument in policy.arguments] == [ - CodegenAction.COPY_IN_OUT, - CodegenAction.COPY_IN_OUT, - ] - assert all(argument.conversion_phase is ArgumentConversionPhase.IMMEDIATE for argument in policy.arguments) - assert [argument.python_barrier_action for argument in policy.arguments] == [ - PythonBarrierAction.SCALAR_VALUE, - PythonBarrierAction.SCALAR_VALUE, - ] - assert [argument.native_barrier_action for argument in policy.arguments] == [ - NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS, - NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS, - ] - assert [argument.storage_mode for argument in policy.arguments] == [StorageMode.STACK, StorageMode.STACK] - assert all(policy.writeback_actions for policy in policies) - assert all(policy.cleanup_actions == () for policy in policies) - assert all(policy.release_actions == () for policy in policies) - assert [ - (slot.source_kind, slot.value_kind, slot.native_barrier_action, slot.codegen_action) - for slot in policy.native_call_slots - ] == [ - ( - "projection", - "arg", - NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS, - CodegenAction.COPY_IN_OUT, - ), - ( - "projection", - "arg", - NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS, - CodegenAction.COPY_IN_OUT, - ), - ] - - def test_source_export_policy_resolves_names_inside_each_namespace(): module = _source_semantic_module("fnaming_f90.f90", module_name="fnaming_f90") policies = { @@ -235,61 +217,6 @@ def test_source_export_policy_resolves_names_inside_each_namespace(): ) -def test_fmath_scalar_policy_records_address_projected_call_slots(): - module = pyi_file_to_semantic_module(FMATH_CONTRACT, module_name="fmath") - complete_semantic_policies(module) - function = next(item for item in module.functions if item.name == "add_r8") - - policy = completed_function_wrapper_policy(function) - - assert policy.owner_path == "fmath.add_r8" - assert [(export.namespace, export.name) for export in policy.python_exports] == [((), "add_r8")] - # The contract states no separate native name: `add_r8` reaches Fortran's - # `ADD_R8`, which is named without regard to case. - assert policy.native_name == "add_r8" - assert policy.standalone is True - - assert [argument.name for argument in policy.arguments] == ["X", "Y"] - assert [argument.python_position for argument in policy.arguments] == [0, 1] - assert [argument.native_position for argument in policy.arguments] == [0, 1] - for argument in policy.arguments: - assert argument.semantic_type_name == "Float64" - assert argument.rank == 0 - assert argument.optional is False - assert argument.ownership.kind is ObjectKind.SCALAR - assert argument.codegen_action is CodegenAction.COPY_IN_OUT - assert argument.conversion_phase is ArgumentConversionPhase.IMMEDIATE - assert argument.python_barrier_action is PythonBarrierAction.SCALAR_VALUE - assert argument.native_barrier_action is NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS - assert argument.storage_mode is StorageMode.STACK - assert argument.boundary_storage_mode is StorageMode.STACK - assert argument.projects_result is True - assert argument.python_visible is True - - assert [(slot.native_position, slot.python_position) for slot in policy.native_call_slots] == [ - (0, 0), - (1, 1), - ] - assert [slot.source_kind for slot in policy.native_call_slots] == ["projection", "projection"] - assert [slot.value_kind for slot in policy.native_call_slots] == ["addr", "addr"] - assert [slot.native_name for slot in policy.native_call_slots] == ["X", "Y"] - assert all( - slot.native_barrier_action is NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS for slot in policy.native_call_slots - ) - - assert len(policy.results) == 1 - result = policy.results[0] - assert result.owner_path == "fmath.add_r8.return" - assert result.semantic_type_name == "Float64" - assert result.rank == 0 - assert result.ownership.kind is ObjectKind.SCALAR - assert result.codegen_action is CodegenAction.DIRECT_VALUE - assert result.python_barrier_action is PythonBarrierAction.NONE - assert result.native_barrier_action is NativeBarrierAction.NONE - assert result.storage_mode is StorageMode.STACK - assert result.boundary_storage_mode is StorageMode.STACK - - def test_wrapper_policy_records_runtime_and_native_order_metadata(): module = parse_pyi_text( """ @@ -298,12 +225,14 @@ def test_wrapper_policy_records_runtime_and_native_order_metadata(): @standalone @native_call([Addr(Arg(1)), Addr(Arg(0))]) def swap_args(x: Float64, y: Float64) -> Float64: ... + +def add(x: Float64, y: Float64) -> Float64: ... """, module_name="runtime_policy", ) complete_semantic_policies(module) - policy = completed_function_wrapper_policy(module.functions[0]) + policy, implicit = (completed_function_wrapper_policy(function) for function in module.functions) assert policy.release_gil is True assert policy.standalone is True @@ -313,6 +242,13 @@ def swap_args(x: Float64, y: Float64) -> Float64: ... (0, 1, "addr"), (1, 0, "addr"), ] + # Without a native_call, Python order is the native order. + assert implicit.release_gil is False + assert [argument.native_position for argument in implicit.arguments] == [0, 1] + assert [(slot.source_kind, slot.native_position, slot.python_position) for slot in implicit.native_call_slots] == [ + ("implicit", 0, 0), + ("implicit", 1, 1), + ] def test_runtime_status_policy_is_completed_before_wrapper_planning(): @@ -347,25 +283,6 @@ def solve(value: Int32) -> None: ... assert [slot.character_length for slot in policy.native_call_slots] == [None, None, 32] -def test_wrapper_policy_records_implicit_native_order(): - module = parse_pyi_text( - """ -def add(x: Float64, y: Float64) -> Float64: ... -""", - module_name="implicit_order", - ) - complete_semantic_policies(module) - - policy = completed_function_wrapper_policy(module.functions[0]) - - assert policy.release_gil is False - assert [argument.native_position for argument in policy.arguments] == [0, 1] - assert [(slot.source_kind, slot.native_position, slot.python_position) for slot in policy.native_call_slots] == [ - ("implicit", 0, 0), - ("implicit", 1, 1), - ] - - def test_wrapper_policy_records_primitive_hidden_literals(): module = parse_pyi_text( """ @@ -440,65 +357,33 @@ def scale(x: Float64) -> Float64: ... ] -def test_wrapper_policy_completes_a_computed_projection_with_its_stated_type(): - module = parse_pyi_text( - """ -@native_call([Int32(Arg(0).shape[0]), Arg(0)]) -def scale(values: Float64[:]) -> None: ... -""", - module_name="typed_extent", - ) - complete_semantic_policies(module) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert policy.supported is True - assert policy.native_call_slots[0].semantic_type_name == "Int32" - - -def test_wrapper_policy_defaults_an_unstated_computed_projection_to_size_t(): - module = parse_pyi_text( - """ -@native_call([Arg(0).shape[0], Arg(0)]) -def scale(values: Float64[:]) -> None: ... -""", - module_name="default_extent", - ) - complete_semantic_policies(module) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert policy.supported is True - assert policy.native_call_slots[0].semantic_type_name == "SizeT" - - -def test_wrapper_policy_blocks_a_non_integer_computed_projection_type(): - module = parse_pyi_text( - """ -@native_call([Float64(Arg(0).shape[0]), Arg(0)]) -def scale(values: Float64[:]) -> None: ... -""", - module_name="real_extent", - ) - complete_semantic_policies(module) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert policy.supported is False - assert "native-call shape slot 0 cannot be materialized as 'Float64'" in policy.blockers - - -@pytest.mark.parametrize("cast_type", ["Int", "UInt"]) -def test_wrapper_policy_blocks_an_unresolved_integer_computed_projection_type(cast_type): +@pytest.mark.parametrize( + ("slot_spelling", "slot_type", "blocker"), + [ + pytest.param("Int32(Arg(0).shape[0])", "Int32", None, id="stated_integer_type"), + pytest.param("Arg(0).shape[0]", "SizeT", None, id="unstated_type_defaults_to_size_t"), + pytest.param("Float64(Arg(0).shape[0])", None, "cannot be materialized as 'Float64'", id="non_integer_type"), + pytest.param("Int(Arg(0).shape[0])", None, "cannot be materialized as 'Int'", id="unresolved_int"), + pytest.param("UInt(Arg(0).shape[0])", None, "cannot be materialized as 'UInt'", id="unresolved_uint"), + ], +) +def test_wrapper_policy_types_or_blocks_a_computed_shape_projection(slot_spelling, slot_type, blocker): module = parse_pyi_text( f""" -@native_call([{cast_type}(Arg(0).shape[0]), Arg(0)]) +@native_call([{slot_spelling}, Arg(0)]) def scale(values: Float64[:]) -> None: ... """, - module_name="unresolved_integer_extent", + module_name="computed_extent", ) complete_semantic_policies(module) policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - assert policy.supported is False - assert f"native-call shape slot 0 cannot be materialized as '{cast_type}'" in policy.blockers + if blocker is None: + assert policy.supported is True + assert policy.native_call_slots[0].semantic_type_name == slot_type + else: + assert policy.supported is False + assert f"native-call shape slot 0 {blocker}" in policy.blockers def test_wrapper_policy_completes_a_one_character_hidden_literal(): @@ -519,31 +404,23 @@ def tagged(x: Float64) -> Float64: ... assert slot.object_kind is None -def test_wrapper_policy_blocks_a_multi_character_hidden_literal(): - module = parse_pyi_text( - """ -@native_call([Arg(0), String[4]("NOPE")]) -def tagged(x: Float64) -> Float64: ... -""", - module_name="long_string_literal", - ) - complete_semantic_policies(module) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert policy.supported is False - assert "native-call literal slot 1 uses unsupported first-lane literal type 'String[4]'" in policy.blockers - - @pytest.mark.parametrize( ("literal", "blocker"), [ - ('String[1]("")', "its value must contain exactly one character"), - ('String[1]("NO")', "its value must contain exactly one character"), - ("String[1](1)", "its value is not a string"), - ('String[1]("🎉")', "its value is not representable as one C char byte"), + pytest.param('String[4]("NOPE")', "uses unsupported first-lane literal type 'String[4]'", id="multi_character"), + pytest.param( + 'String[1]("")', "declares String[1] but its value must contain exactly one character", id="empty" + ), + pytest.param( + 'String[1]("NO")', "declares String[1] but its value must contain exactly one character", id="two" + ), + pytest.param("String[1](1)", "declares String[1] but its value is not a string", id="not_a_string"), + pytest.param( + 'String[1]("🎉")', "declares String[1] but its value is not representable as one C char byte", id="non_byte" + ), ], ) -def test_wrapper_policy_blocks_an_invalid_one_character_literal_value(literal, blocker): +def test_wrapper_policy_blocks_an_invalid_character_hidden_literal(literal, blocker): module = parse_pyi_text( f""" @native_call([Arg(0), {literal}]) @@ -555,104 +432,7 @@ def tagged(x: Float64) -> Float64: ... policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] assert policy.supported is False - assert f"native-call literal slot 1 declares String[1] but {blocker}" in policy.blockers - - -def test_wrapper_policy_completes_required_rank_one_array_buffer_handoff(): - module = parse_pyi_text( - """ -def sum_values(values: Float64[:]) -> Float64: ... -""", - module_name="array_argument", - ) - complete_semantic_policies(module) - function = module.functions[0] - policy = function.metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert isinstance(policy, FunctionWrapperPolicy) - assert policy.supported is True - assert policy.blockers == () - argument = policy.arguments[0] - assert argument.ownership.kind is ObjectKind.NUMPY_ARRAY - assert argument.python_barrier_action is PythonBarrierAction.ARRAY_STORAGE - assert argument.native_barrier_action is NativeBarrierAction.PASS_ARRAY_BUFFER - assert argument.bridge_data_action is BridgeDataAction.ASSOCIATE_VIEW - assert argument.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER - assert argument.array is not None - assert argument.array.rank == 1 - assert argument.array.shape == (":",) - assert argument.array.axes == ("dense",) - assert argument.array.contiguous is True - assert policy.native_call_slots[0].array == argument.array - - -def test_wrapper_policy_flattens_python_rank_for_rank_one_assumed_size_storage(): - module = parse_pyi_text( - """ -def sum_flat(n: Int32, values: Float64[Flat]) -> Float64: ... -""", - module_name="flat_array_argument", - ) - complete_semantic_policies(module) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - argument = policy.arguments[1] - assert argument.array is not None - assert argument.array.rank == 1 - assert argument.array.shape == (":",) - assert argument.array.category == "assumed_size" - assert argument.array.flatten_python_storage is True - assert argument.array.flat_axis == 0 - assert argument.native_array_actual is not None - assert argument.native_array_actual.rank == 1 - assert argument.native_array_actual.shape == (":",) - assert argument.native_array_actual.flatten_storage is True - assert argument.native_array_actual.flat_axis == 0 - assert policy.native_call_slots[1].array == argument.array - - -def test_wrapper_policy_flattens_remaining_axes_for_multidimensional_assumed_size_storage(): - module = parse_pyi_text( - """ -from prik.contracts import Annotated, Flat, Float64, Int32, ORDER_C - -def sum_fortran(rows: Int32, values: Float64[rows, Flat]) -> Float64: ... -def sum_c(columns: Int32, values: Annotated[Float64[Flat, columns], ORDER_C]) -> Float64: ... -""", - module_name="flat_matrix_argument", - ) - complete_semantic_policies(module) - policies = { - function.name: function.metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] for function in module.functions - } - - fortran_argument = policies["sum_fortran"].arguments[1] - assert fortran_argument.array is not None - assert fortran_argument.array.rank == 2 - assert fortran_argument.array.shape == ("rows", ":") - assert fortran_argument.array.order == "ORDER_F" - assert fortran_argument.array.category == "assumed_size" - assert fortran_argument.array.flatten_python_storage is True - assert fortran_argument.array.flat_axis == 1 - assert fortran_argument.native_array_actual is not None - assert fortran_argument.native_array_actual.rank == 2 - assert fortran_argument.native_array_actual.shape == ("rows", ":") - assert fortran_argument.native_array_actual.flatten_storage is True - assert fortran_argument.native_array_actual.flat_axis == 1 - - c_argument = policies["sum_c"].arguments[1] - assert c_argument.array is not None - assert c_argument.array.rank == 2 - assert c_argument.array.shape == (":", "columns") - assert c_argument.array.order == "ORDER_C" - assert c_argument.array.category == "assumed_size" - assert c_argument.array.flatten_python_storage is True - assert c_argument.array.flat_axis == 0 - assert c_argument.native_array_actual is not None - assert c_argument.native_array_actual.rank == 2 - assert c_argument.native_array_actual.shape == (":", "columns") - assert c_argument.native_array_actual.flatten_storage is True - assert c_argument.native_array_actual.flat_axis == 0 + assert f"native-call literal slot 1 {blocker}" in policy.blockers def test_wrapper_policy_completes_assumed_optional_replacements_and_blocks_unreleased_status_cleanup(): @@ -715,17 +495,6 @@ def projected_raw(label: Addr(String[8])) -> Returns["label", String[8]]: ... assert "raw string address unexpectedly projects a result" in "; ".join(policies[3].blockers) -def test_missing_wrapper_policy_fails_before_planning(): - function = SemanticFunction( - name="add", - arguments=[], - return_type=SemanticType(name="Float64", dtype="Float64"), - ) - - with pytest.raises(ValueError, match="missing completed wrapper policy"): - completed_function_wrapper_policy(function) - - def test_completed_function_policy_rejects_unimplemented_runtime_constraints(): module = parse_pyi_text( "def solve(value: Annotated[Int32, Bounded(1, 8), Finite]) -> Int32: ...\n", diff --git a/tests/fortran/infrastructure/preprocessing/_support.py b/tests/fortran/infrastructure/preprocessing/_support.py index cf8bc81fd..e77f1a5e5 100644 --- a/tests/fortran/infrastructure/preprocessing/_support.py +++ b/tests/fortran/infrastructure/preprocessing/_support.py @@ -32,21 +32,6 @@ def _fake_compiler(tmp_path: Path, output: str) -> tuple[Path, Path, dict[str, s return script, args_file, env -def _failing_compiler(tmp_path: Path, stderr: str) -> Path: - script = tmp_path / "failing-cc" - script.write_text( - f"""#!{sys.executable} -import sys - -sys.stderr.write({stderr!r}) -sys.exit(1) -""", - encoding="utf-8", - ) - script.chmod(0o755) - return script - - def _assert_preprocessing_error( exc_info: pytest.ExceptionInfo[PreprocessingError], *, diff --git a/tests/fortran/infrastructure/preprocessing/test_cli.py b/tests/fortran/infrastructure/preprocessing/test_cli.py index 16abb1eef..b329193bd 100644 --- a/tests/fortran/infrastructure/preprocessing/test_cli.py +++ b/tests/fortran/infrastructure/preprocessing/test_cli.py @@ -1,4 +1,4 @@ -"""Tests split by stable ownership concept from `test_cli.py`.""" +"""The ``prik parse`` CLI runs Fortran compiler preprocessing and reports its recipe.""" import json from pathlib import Path @@ -8,22 +8,6 @@ from tests.fortran.infrastructure.preprocessing._support import _fake_compiler -def test_cli_help_documents_exact_compiler_and_preprocessing_examples(): - res = subprocess.run( - [sys.executable, "-m", "prik", "parse", "--help"], - capture_output=True, - text=True, - check=True, - ) - - assert "--compiler COMPILER" in res.stdout - assert "Compiler used for preprocessing" in res.stdout - assert "default: gfortran; cc with --language c" in " ".join(res.stdout.split()) - assert "--compile-commands PATH" in res.stdout - assert "-D" in res.stdout - assert "--define NAME[=VALUE]" in res.stdout - - def test_cli_accepts_compile_database_for_fortran_compiler_mode(tmp_path: Path): source = tmp_path / "solver.F90" source.write_text("subroutine solve()\nend subroutine solve\n", encoding="utf-8") @@ -67,34 +51,20 @@ def test_cli_accepts_compile_database_for_fortran_compiler_mode(tmp_path: Path): assert payload["preprocessing_recipe"]["compile_commands"] == str(database) -def test_cli_fortran_default_compiler_json_records_preprocessing_recipe(tmp_path: Path): - source = tmp_path / "branch.F90" - source.write_text( - "subroutine selected()\nend subroutine selected\n", - encoding="utf-8", - ) - - res = subprocess.run( - [sys.executable, "-m", "prik", "parse", str(source), "--json"], - capture_output=True, - text=True, - check=True, - ) - - assert json.loads(res.stdout)[str(source)]["preprocessing_recipe"]["compiler"] == "gfortran" - - -def test_cli_fortran_default_compiler_mode_accepts_include_dirs(tmp_path: Path): +def test_cli_fortran_default_compiler_mode_accepts_include_dirs_and_records_recipe(tmp_path: Path): source = tmp_path / "mini.F90" source.write_text("subroutine work()\nend subroutine work\n", encoding="utf-8") res = subprocess.run( - [sys.executable, "-m", "prik", "parse", str(source), "-I", "include"], + [sys.executable, "-m", "prik", "parse", str(source), "--json", "-I", "include"], capture_output=True, text=True, + check=True, ) - assert res.returncode == 0 + recipe = json.loads(res.stdout)[str(source)]["preprocessing_recipe"] + assert recipe["compiler"] == "gfortran" + assert recipe["include_dirs"] == ["include"] def test_cli_fortran_compiler_mode_runs_exact_compiler_and_parses_stdout(tmp_path: Path): diff --git a/tests/fortran/infrastructure/preprocessing/test_configuration_and_adapters.py b/tests/fortran/infrastructure/preprocessing/test_configuration_and_adapters.py index 99448152f..2a9bef80b 100644 --- a/tests/fortran/infrastructure/preprocessing/test_configuration_and_adapters.py +++ b/tests/fortran/infrastructure/preprocessing/test_configuration_and_adapters.py @@ -11,7 +11,6 @@ PreprocessingError, build_direct_preprocess_invocation, build_preprocess_invocation, - expand_native_fortran_includes, run_compiler_preprocessor_with_recipe, validate_macro_name, ) @@ -120,20 +119,6 @@ def test_preprocessing_config_internal_macros_recipe_and_validation(tmp_path: Pa ) -def test_preprocessing_error_default_category_and_diagnostics(): - diagnostic = preprocessing.PreprocessingDiagnostic(category="PREPROCESSOR_FAILED", message="bad") - - default_error = PreprocessingError("default") - detailed_error = PreprocessingError("detailed", diagnostics=[diagnostic]) - - assert default_error.category == "PREPROCESSOR_FAILED" - assert default_error.diagnostics == [] - assert str(default_error) == "default" - assert detailed_error.category == "PREPROCESSOR_FAILED" - assert detailed_error.diagnostics == [diagnostic] - assert str(detailed_error) == "detailed" - - def test_recipe_round_trip_preserves_all_preprocessing_metadata(monkeypatch, tmp_path: Path): source = tmp_path / "solver.F90" included = preprocessing.IncludedFile( @@ -280,29 +265,3 @@ def test_build_preprocess_invocation_supports_fortran_compile_database(tmp_path: }, capabilities={"dependency_output": True, "macro_dump": True, "linemarkers": True}, ) - - -def test_native_fortran_missing_include_does_not_drop_following_source(tmp_path: Path): - root = tmp_path / "root.F90" - - expanded, included_files, mappings, diagnostics = expand_native_fortran_includes( - 'include "missing.inc"\ninteger :: retained\n', - root_path=root, - include_dirs=[], - ) - - assert expanded == "integer :: retained\n" - assert included_files == [] - assert [(mapping.generated_line, mapping.original_path, mapping.original_line) for mapping in mappings] == [ - (1, str(root), 2) - ] - assert [diagnostic.to_dict() for diagnostic in diagnostics] == [ - { - "category": "INCLUDE_NOT_FOUND", - "message": 'Fortran INCLUDE file "missing.inc" was not found', - "severity": "error", - "path": str(root), - "line": 1, - "command": [], - } - ] diff --git a/tests/fortran/infrastructure/preprocessing/test_dependencies_and_includes.py b/tests/fortran/infrastructure/preprocessing/test_dependencies_and_includes.py index c94e21002..0f72fe85a 100644 --- a/tests/fortran/infrastructure/preprocessing/test_dependencies_and_includes.py +++ b/tests/fortran/infrastructure/preprocessing/test_dependencies_and_includes.py @@ -1,9 +1,8 @@ -"""Tests split by stable ownership concept from `test_cli.py`.""" +"""Compiler linemarker provenance and native Fortran INCLUDE expansion.""" from pathlib import Path import prik.preprocessing.source as preprocessing -import prik.preprocessing.fortran as fortran_preprocessing from prik.preprocessing import PreprocessingConfig, expand_native_fortran_includes @@ -106,21 +105,6 @@ def test_linemarker_nested_returns_restore_parent_stack(tmp_path: Path): assert direct_include.include_line == 1 -def test_native_include_line_marker_and_mapping_helpers_preserve_provenance(tmp_path: Path): - root = tmp_path / "root.F90" - mappings = preprocessing.parse_linemarker_mappings('# 7 "api.inc"\ninteger :: value\n', filename=str(root)) - - assert fortran_preprocessing._line_marker(3, 'dir\\api".inc') == '# 3 "dir\\\\api\\".inc"' - assert fortran_preprocessing._line_marker(3, "api.inc", 1) == '# 3 "api.inc" 1' - assert fortran_preprocessing._mapping_for_generated_line(mappings, mappings[0].generated_line, root) == mappings[0] - - fallback = fortran_preprocessing._mapping_for_generated_line([], 99, root) - assert fallback.generated_line == 99 - assert fallback.original_path == str(root) - assert fallback.original_line == 99 - assert fallback.include_stack == [str(root)] - - def test_native_fortran_include_expansion_is_recursive_and_preserves_duplicates(tmp_path: Path): root = tmp_path / "src" / "root.F90" include = root.parent / "decls.inc" @@ -274,44 +258,6 @@ def test_native_fortran_include_lookup_order_missing_and_cycle_diagnostics(tmp_p ] -def test_native_fortran_include_reports_files_that_disappear_before_read(monkeypatch, tmp_path: Path): - root = tmp_path / "root.F90" - include = tmp_path / "vanished.inc" - root.write_text('include "vanished.inc"\n', encoding="utf-8") - include.write_text("integer :: vanished\n", encoding="utf-8") - original_read_text = Path.read_text - - seen_encodings = [] - - def fail_for_include(path: Path, *args, **kwargs): - if path == include: - seen_encodings.append(kwargs["encoding"]) - raise OSError("disappeared") - return original_read_text(path, *args, **kwargs) - - monkeypatch.setattr(Path, "read_text", fail_for_include) - - expanded, included_files, _mappings, diagnostics = expand_native_fortran_includes( - root.read_text(encoding="utf-8"), - root_path=root, - include_dirs=[], - ) - - assert expanded.startswith(f'# 1 "{include}" 1') - assert [Path(item.path) for item in included_files] == [include] - assert seen_encodings == ["utf-8"] - assert [diagnostic.to_dict() for diagnostic in diagnostics] == [ - { - "category": "INCLUDE_NOT_FOUND", - "message": 'Fortran INCLUDE file "vanished.inc" could not be read: disappeared', - "severity": "error", - "path": str(root.resolve()), - "line": 1, - "command": [], - } - ] - - def test_native_fortran_include_resolution_continues_after_oserror(monkeypatch, tmp_path: Path): root = tmp_path / "src" / "root.F90" include_dir = tmp_path / "include" @@ -340,54 +286,45 @@ def fail_first_candidate(path: Path): assert [Path(item.path) for item in included_files] == [include] -def test_native_fortran_include_uses_absolute_fallback_when_resolve_fails(monkeypatch, tmp_path: Path): - root = tmp_path / "root.F90" - include = tmp_path / "decls.inc" - include.write_text("integer :: value\n", encoding="utf-8") - original_resolve = Path.resolve - - def fail_include_resolve(path: Path): - if path == include: - raise OSError("cannot resolve include") - return original_resolve(path) - - monkeypatch.setattr(Path, "resolve", fail_include_resolve) - - expanded, included_files, _mappings, diagnostics = expand_native_fortran_includes( - 'include "decls.inc"\n', - root_path=root, - include_dirs=[], - ) - - assert diagnostics == [] - assert f'# 1 "{include.absolute()}" 1' in expanded - assert [item.path for item in included_files] == [str(include.absolute())] - - def test_native_fortran_include_diagnostics_do_not_drop_following_lines(monkeypatch, tmp_path: Path): + """A missing, cyclic, or unreadable INCLUDE reports a diagnostic and expansion continues.""" root = tmp_path / "root.F90" cycle = tmp_path / "cycle.inc" vanished = tmp_path / "vanished.inc" cycle.write_text('include "cycle.inc"\ninteger :: after_cycle\n', encoding="utf-8") vanished.write_text("integer :: vanished\n", encoding="utf-8") original_read_text = Path.read_text + seen_encodings = [] def fail_for_vanished(path: Path, *args, **kwargs): if path == vanished: + seen_encodings.append(kwargs["encoding"]) raise OSError("disappeared") return original_read_text(path, *args, **kwargs) monkeypatch.setattr(Path, "read_text", fail_for_vanished) - expanded, _included_files, _mappings, diagnostics = expand_native_fortran_includes( + expanded, included_files, _mappings, diagnostics = expand_native_fortran_includes( + 'include "missing.inc"\ninteger :: after_missing\n' 'include "cycle.inc"\ninclude "vanished.inc"\ninteger :: after_read_error\n', root_path=root, include_dirs=[], ) + assert "after_missing" in expanded assert "after_cycle" in expanded assert "after_read_error" in expanded - assert [diagnostic.category for diagnostic in diagnostics] == ["INCLUDE_CYCLE", "INCLUDE_NOT_FOUND"] + assert f'# 1 "{vanished}" 1' in expanded + assert vanished in [Path(item.path) for item in included_files] + assert seen_encodings == ["utf-8"] + assert [(diagnostic.category, diagnostic.line) for diagnostic in diagnostics] == [ + ("INCLUDE_NOT_FOUND", 1), + ("INCLUDE_CYCLE", 1), + ("INCLUDE_NOT_FOUND", 4), + ] + assert diagnostics[0].message == 'Fortran INCLUDE file "missing.inc" was not found' + assert diagnostics[2].message == 'Fortran INCLUDE file "vanished.inc" could not be read: disappeared' + assert diagnostics[2].path == str(root.resolve()) def test_native_fortran_include_expansion_preserves_input_linemarkers_and_private_exposure(tmp_path: Path): diff --git a/tests/fortran/infrastructure/preprocessing/test_execution.py b/tests/fortran/infrastructure/preprocessing/test_execution.py index c0e6d1121..e40546263 100644 --- a/tests/fortran/infrastructure/preprocessing/test_execution.py +++ b/tests/fortran/infrastructure/preprocessing/test_execution.py @@ -4,7 +4,6 @@ import pytest -import prik.preprocessing.fortran as fortran_preprocessing import prik.preprocessing.source as preprocessing from prik.preprocessing import ( PreprocessingConfig, @@ -12,36 +11,6 @@ ) -def test_preprocess_source_reparses_fortran_mapping_when_native_expansion_returns_none(monkeypatch, tmp_path: Path): - source = tmp_path / "solver.F90" - source.write_text("integer :: value\n", encoding="utf-8") - monkeypatch.setattr( - preprocessing.subprocess, - "run", - lambda *_args, **_kwargs: type("Done", (), {"returncode": 0, "stdout": "ignored\n", "stderr": ""})(), - ) - monkeypatch.setattr( - fortran_preprocessing, - "expand_native_fortran_includes", - lambda *_args, **_kwargs: ("integer :: value\n", [], [], []), - ) - - result = preprocessing.preprocess_source( - source, - language="fortran", - config=PreprocessingConfig(mode="compiler", compiler=str(tmp_path / "gfortran")), - ) - - assert result.source_mappings == [ - preprocessing.SourceMapping( - generated_line=1, - original_path=str(source), - original_line=1, - include_stack=[str(source)], - ) - ] - - def test_preprocess_source_preserves_fortran_native_metadata(monkeypatch, tmp_path: Path): fortran_source = tmp_path / "solver.F90" private_include = tmp_path / "private.inc" diff --git a/tests/fortran/infrastructure/preprocessing/test_parser_boundaries.py b/tests/fortran/infrastructure/preprocessing/test_parser_boundaries.py index 671ac5392..1774a3263 100644 --- a/tests/fortran/infrastructure/preprocessing/test_parser_boundaries.py +++ b/tests/fortran/infrastructure/preprocessing/test_parser_boundaries.py @@ -1,9 +1,5 @@ """Preprocessor selection and declaration/execution boundary handling.""" -import ast -from dataclasses import replace -import re - import pytest from prik.parsers.fortran import FortranParseError, parse_fortran_file @@ -58,105 +54,45 @@ def test_fortran_lexer_preprocess_lines_folds_free_and_fixed_continuations(): ] -def collect_signature_shape_symbols(signature): - symbols = set() - for arg in signature.arguments: - for dim in arg.shape: - symbols.update(re.findall(r"[A-Za-z_]\w*", dim)) - return symbols - - -def evaluate_signature_shapes(signature, symbol_values=None): - symbol_values = symbol_values or {} - out = replace(signature) - out.arguments = [replace(a) for a in signature.arguments] - - def fold_integer_expr(text): - try: - tree = ast.parse(text, mode="eval") - except SyntaxError: - return text - allowed = ( - ast.Expression, - ast.BinOp, - ast.UnaryOp, - ast.Constant, - ast.Add, - ast.Sub, - ast.Mult, - ast.Div, - ast.FloorDiv, - ast.Mod, - ast.Pow, - ast.USub, - ast.UAdd, - ) - if any(not isinstance(node, allowed) for node in ast.walk(tree)): - return text - value = eval(compile(tree, "", "eval"), {"__builtins__": {}}, {}) - return str(int(value)) if isinstance(value, int | float) and value == int(value) else text - - for arg in out.arguments: - arg.shape = list(arg.shape) - for index, dim in enumerate(arg.shape): - for key, value in symbol_values.items(): - dim = re.sub(rf"\b{re.escape(str(key))}\b", str(value), dim, flags=re.IGNORECASE) - if ":" in dim: - dim = ":".join(fold_integer_expr(part) if part.strip() else part for part in dim.split(":")) - else: - dim = fold_integer_expr(dim) - arg.shape[index] = dim - return out - - -def test_signature_shape_helpers_evaluate_publicly_parsed_signature(): - code = """ -subroutine fill(a) - real, intent(inout) :: a(0:nx-1, 1:ny) -end subroutine fill -""" - - sig = parse_fortran_file(code).procedures[0] - - assert collect_signature_shape_symbols(sig) == {"nx", "ny"} - evaluated = evaluate_signature_shapes(sig, {"NX": 4, "ny": 3}) - assert evaluated.arguments[0].shape == ["0:3", "1:3"] - assert sig.arguments[0].shape == ["0:nx-1", "1:ny"] - - -@pytest.mark.parametrize("directive", ["#if USE_FAST", "#ifdef USE_FAST", "#define USE_FAST 1", '#include "api.inc"']) -def test_cpp_directives_require_compiler_preprocessing(directive): - code = f"{directive}\nsubroutine selected()\nend subroutine selected\n" - - with pytest.raises(FortranParseError, match="require compiler preprocessing") as exc_info: - parse_fortran_file(code, filename="raw_cpp.F90") - - assert exc_info.value.code == "PARSE_PREPROCESSING_REQUIRED" - assert exc_info.value.line_number == 1 - - -def test_compiler_linemarkers_remain_parseable_for_provenance(): - code = '# 40 "include/api.inc" 1\nsubroutine selected()\nend subroutine selected\n' - - parsed = parse_fortran_file(code, filename="preprocessed.F90") - - assert [procedure.name for procedure in parsed.procedures] == ["selected"] - - -def test_fixed_form_cpp_directives_are_rejected_before_comment_handling(): - code = "#ifdef USE_FAST\n subroutine selected()\n end\n#endif\n" - +@pytest.mark.parametrize( + ("filename", "code"), + [ + pytest.param("raw_cpp.F90", "#if USE_FAST\nsubroutine selected()\nend subroutine selected\n", id="if"), + pytest.param("raw_cpp.F90", "#ifdef USE_FAST\nsubroutine selected()\nend subroutine selected\n", id="ifdef"), + pytest.param( + "raw_cpp.F90", "#define USE_FAST 1\nsubroutine selected()\nend subroutine selected\n", id="define" + ), + pytest.param( + "raw_cpp.F90", '#include "api.inc"\nsubroutine selected()\nend subroutine selected\n', id="include" + ), + pytest.param( + "raw_cpp.F", + "#ifdef USE_FAST\n subroutine selected()\n end\n#endif\n", + id="fixed-form-before-comment-handling", + ), + ], +) +def test_cpp_directives_require_compiler_preprocessing(filename, code): with pytest.raises(FortranParseError, match="require compiler preprocessing") as exc_info: - parse_fortran_file(code, filename="raw_cpp.F") + parse_fortran_file(code, filename=filename) assert exc_info.value.code == "PARSE_PREPROCESSING_REQUIRED" assert exc_info.value.line_number == 1 -def test_fixed_form_compiler_linemarkers_are_removed_before_lexing(): - code = '# 1 "api.F"\n subroutine selected()\n end\n' - - parsed = parse_fortran_file(code, filename="preprocessed.F") +@pytest.mark.parametrize( + ("filename", "code"), + [ + pytest.param( + "preprocessed.F90", + '# 40 "include/api.inc" 1\nsubroutine selected()\nend subroutine selected\n', + id="free-form", + ), + pytest.param("preprocessed.F", '# 1 "api.F"\n subroutine selected()\n end\n', id="fixed-form"), + ], +) +def test_compiler_linemarkers_remain_parseable_for_provenance(filename, code): + parsed = parse_fortran_file(code, filename=filename) assert [procedure.name for procedure in parsed.procedures] == ["selected"] @@ -257,73 +193,6 @@ def test_program_execution_part_is_ignored_after_first_executable_statement(): assert [var.name for var in program.variables] == ["ierr"] -def test_executable_statement_in_module_spec_part_raises(): - code = """ -module bad_exec_mod - write(*,*) "not allowed" -end module bad_exec_mod -""" - - with pytest.raises(FortranParseError, match="Executable statement is not allowed"): - parse_fortran_file(code, filename="bad_exec_mod.f90") - - -def test_openmp_declarative_directives_raise_but_executable_directives_are_body_lines(): - declarative = """ -module omp_mod - integer :: state -!$omp threadprivate(state) -end module omp_mod -""" - executable = """ -subroutine omp_body(x) - integer, intent(inout) :: x -!$omp parallel do - do i = 1, x - x = x + i - end do -end subroutine omp_body -""" - proc_declarative = """ -subroutine omp_decl(x) -!$omp declare simd - integer, intent(inout) :: x -end subroutine omp_decl -""" - type_declarative = """ -module omp_type_mod - type :: state -!$omp declare target - integer :: value - end type state -end module omp_type_mod -""" - module_executable = """ -module bad_omp_mod -!$omp parallel -end module bad_omp_mod -""" - fixed_form_executable = """ - subroutine fixed_omp(n) - integer n -C$OMP PARALLEL DO - do 10 i = 1, n -10 continue - end -""" - - with pytest.raises(FortranParseError, match="Unsupported OpenMP declarative directive"): - parse_fortran_file(declarative, filename="omp_mod.f90") - with pytest.raises(FortranParseError, match="Unsupported OpenMP declarative directive"): - parse_fortran_file(proc_declarative, filename="omp_decl.f90") - with pytest.raises(FortranParseError, match="Unsupported OpenMP declarative directive"): - parse_fortran_file(type_declarative, filename="omp_type.f90") - with pytest.raises(FortranParseError, match="Executable statement is not allowed"): - parse_fortran_file(module_executable, filename="bad_omp_mod.f90") - assert parse_fortran_file(executable, filename="omp_body.f90").procedures[0].name == "omp_body" - assert parse_fortran_file(fixed_form_executable, filename="fixed_omp.f").procedures[0].name == "fixed_omp" - - def test_statement_function_and_numeric_label_before_execution_part(): code = """ subroutine old_style(x) @@ -354,21 +223,3 @@ def test_implicit_mapping_parameter_noise_and_assignment_lines_do_not_break_proc assert sig.arguments[0].name == "x" assert sig.arguments[0].base_type == "real" - - -def test_stray_end_unit_lines_are_rejected_by_public_file_parse(): - with pytest.raises(FortranParseError, match="Invalid Fortran syntax") as exc_info: - parse_fortran_file( - """ -end module stray_mod -end submodule stray_submod -end program stray_program -end interface - -subroutine kept() -end subroutine kept -""", - filename="stray_ends.f90", - ) - - assert exc_info.value.code == "PARSE_INVALID_SYNTAX" diff --git a/tests/fortran/infrastructure/preprocessing/test_preprocessing_properties.py b/tests/fortran/infrastructure/preprocessing/test_preprocessing_properties.py deleted file mode 100644 index be29896c1..000000000 --- a/tests/fortran/infrastructure/preprocessing/test_preprocessing_properties.py +++ /dev/null @@ -1,91 +0,0 @@ -"""Tests split by stable ownership concept from `test_properties.py`.""" - -import prik.preprocessing.source as preprocessing -import pytest -import sys -from hypothesis import ( - given, - strategies as st, -) -from pathlib import Path -from prik.parsers.fortran import FortranParseError, parse_fortran_file -from prik.preprocessing import ( - PreprocessingConfig, - preprocess_source, -) -from tempfile import TemporaryDirectory -from unittest.mock import patch -from tests.fortran._support.parser_properties import _FORTRAN_IDENTIFIER_STEMS - - -@pytest.mark.property -@given(include_stem=_FORTRAN_IDENTIFIER_STEMS) -def test_generated_fortran_native_includes_do_not_change_public_signature(include_stem): - target = f"{include_stem}.inc" - baseline = "subroutine generated_include(value)\n integer, intent(in) :: value\nend subroutine generated_include\n" - with_include = ( - f"subroutine generated_include(value)\n" - f" include '{target}'\n" - " integer, intent(in) :: value\n" - "end subroutine generated_include\n" - ) - - baseline_parsed = parse_fortran_file(baseline, filename="generated_include.f90") - included_parsed = parse_fortran_file(with_include, filename="generated_include.f90") - - assert included_parsed.diagnostics == [] - assert included_parsed.procedures == baseline_parsed.procedures - - -@pytest.mark.property -@given(feature_stem=_FORTRAN_IDENTIFIER_STEMS) -def test_generated_fortran_raw_preprocessor_directives_require_preprocessing(feature_stem): - feature = f"feature_{feature_stem}" - source = f"#ifdef {feature}\nsubroutine generated_conditional()\nend subroutine generated_conditional\n#endif\n" - - with pytest.raises(FortranParseError, match="require compiler preprocessing") as exc_info: - parse_fortran_file(source, filename="generated_conditional.F90") - - assert exc_info.value.code == "PARSE_PREPROCESSING_REQUIRED" - - -@pytest.mark.property -@given(feature_stem=_FORTRAN_IDENTIFIER_STEMS, select_feature=st.booleans()) -def test_generated_fortran_compiler_preprocessing_selects_macro_branch(feature_stem, select_feature): - feature = f"feature_{feature_stem}" - with TemporaryDirectory() as tmp_dir: - source_path = Path(tmp_dir) / "generated_conditional.F90" - source_path.write_text( - f"#ifdef {feature}\n" - "subroutine selected_path()\n" - "end subroutine selected_path\n" - "#else\n" - "subroutine fallback_path()\n" - "end subroutine fallback_path\n" - "#endif\n", - encoding="utf-8", - ) - captured_argv = [] - - def run_compiler(argv, **_kwargs): - captured_argv.extend(argv) - selected = f"-D{feature}" in argv - procedure_name = "selected_path" if selected else "fallback_path" - stdout = f"subroutine {procedure_name}()\nend subroutine {procedure_name}\n" - return type("Done", (), {"returncode": 0, "stdout": stdout, "stderr": ""})() - - defines = [feature] if select_feature else [] - with patch.object(preprocessing.subprocess, "run", run_compiler): - result = preprocess_source( - source_path, - language="fortran", - config=PreprocessingConfig(mode="compiler", compiler=sys.executable, defines=defines), - ) - parsed = parse_fortran_file(result.source, filename=str(source_path)) - - assert "-cpp" in captured_argv - assert (f"-D{feature}" in captured_argv) is select_feature - assert result.recipe["defines"] == defines - assert [procedure.name for procedure in parsed.procedures] == [ - "selected_path" if select_feature else "fallback_path" - ] diff --git a/tests/fortran/infrastructure/printers/test_source_printers.py b/tests/fortran/infrastructure/printers/test_source_printers.py index 71584fd12..9314e90dc 100644 --- a/tests/fortran/infrastructure/printers/test_source_printers.py +++ b/tests/fortran/infrastructure/printers/test_source_printers.py @@ -1,237 +1,114 @@ -"""Internal C and Fortran source-printer contracts.""" +"""Fortran source-printer line-wrapping contracts. -from __future__ import annotations +Generated Fortran must stay within the free-form 132-column limit, and every +continuation must be placed where it cannot change what the statement means. +""" +from __future__ import annotations -import ast import re import pytest -from tests.fortran._support.wrapper_build import REPO_ROOT from prik.codegen import ( - BackendScalarType, - CDeclaration, - CExpressionStatement, - CFunction, - CFunctionPrototype, - CGoto, - CHeader, - CInclude, - CLabel, - CModule, - CParameter, - CReturn, CodeExpression, FortranAssignment, FortranCall, FortranFunction, FortranIf, FortranModule, - FortranParameter, FortranPointerAssignment, - FortranUse, - UnsupportedWrapperCodegenNodeError, -) -from prik.planning import ( - BindingModulePlan, - BridgeModulePlan, - ModulePlan, - NamespacePlan, - NativeEntrypointModulePlan, ) -from prik.printers import CSourcePrinter, FortranSourcePrinter +from prik.printers import FortranSourcePrinter +_SLICES = ", ".join(f"1:values_upper_bound_{axis} + 1:values_stride_{axis}" for axis in range(4)) +_LONG_SUM = " + ".join(f"value_{index}" for index in range(20)) +_PADDING = "x" * 40 -def test_source_printers_render_complete_c_header_and_fortran_modules(): - float64 = BackendScalarType( - semantic_name="Float64", - c_spelling="double", - fortran_spelling="real(c_double)", - python_parse_unit="d", - numpy_type_macro="NPY_FLOAT64", - ) - parameters = (CParameter("self", "PyObject *"), CParameter("args", "PyObject *")) - c_function = CFunction( - name="wrap_add_r8", - return_type="PyObject *", - parameters=parameters, - storage="static", - body=( - CDeclaration("x", float64.c_spelling, CodeExpression("0.0")), - CExpressionStatement(CodeExpression("import_array()")), - CReturn(CodeExpression("NULL")), - ), - ) - c_header = CHeader( - guard="FMATH_WRAPPER_H", - includes=(CInclude("Python.h"),), - prototypes=(CFunctionPrototype("wrap_add_r8", "PyObject *", parameters),), - ) - c_module = CModule( - name="fmath_wrapper", - includes=( - CInclude("Python.h"), - CInclude("numpy/arrayobject.h"), - CInclude("fmath_wrapper.h", system=False), - ), - functions=(c_function,), - ) - fortran_module = FortranModule( - name="bind_c_fmath_wrapper", - uses=(FortranUse("iso_c_binding", ("c_double",)),), + +def _nested_assignment(expression: str) -> FortranModule: + return FortranModule( + "nested_lines", procedures=( FortranFunction( - name="bind_c_add_r8", - parameters=(FortranParameter("x", float64.fortran_spelling, ("value",)),), - result_name="result", - result_type=float64.fortran_spelling, - bind_name="ADD_R8", - body=(FortranAssignment("result", CodeExpression("x")),), + "nested", + body=( + FortranIf( + CodeExpression("outer"), + body=( + FortranIf( + CodeExpression("inner"), + body=(FortranAssignment("result", CodeExpression(expression)),), + ), + ), + ), + ), + is_subroutine=True, ), ), ) - c_header_source = CSourcePrinter().doprint(c_header) - c_source = CSourcePrinter().doprint(c_module) - fortran_source = FortranSourcePrinter().doprint(fortran_module) - - assert "#ifndef FMATH_WRAPPER_H" in c_header_source - assert "PyObject * wrap_add_r8(PyObject * self, PyObject * args);" in c_header_source - assert '#include "fmath_wrapper.h"' in c_source - assert "static PyObject * wrap_add_r8(PyObject * self, PyObject * args)" in c_source - assert "double x = 0.0;" in c_source - assert "import_array();" in c_source - assert "use iso_c_binding, only: c_double" in fortran_source - assert 'function bind_c_add_r8(x) result(result) bind(c, name="ADD_R8")' in fortran_source - assert "real(c_double), value :: x" in fortran_source -def test_c_source_printer_renders_function_local_cleanup_jumps(): - function = CFunction( - name="wrap_outputs", - return_type="PyObject *", - body=( - CGoto("prik_output_cleanup_1"), - CLabel("prik_output_cleanup_1"), - CReturn(CodeExpression("NULL")), +@pytest.mark.parametrize( + ("node", "present", "absent"), + [ + pytest.param( + FortranCall( + "native_scale", + (CodeExpression(f"values_base({_SLICES})"), CodeExpression(f"out_base({_SLICES})")), + ), + ("& values_base(&", "& 1:values_upper_bound_3 + 1:values_stride_3), &", "& out_base(&"), + (), + id="parenthesized_call_arguments", ), - ) - - source = CSourcePrinter().doprint(function) - - assert "goto prik_output_cleanup_1;" in source - assert "prik_output_cleanup_1:" in source - - -def test_source_printers_reject_wrapper_plan_models(): - plan = ModulePlan( - owner_path="demo", - binding=BindingModulePlan("demo", "demo"), - entrypoint=NativeEntrypointModulePlan("demo"), - bridge=BridgeModulePlan("demo"), - variables=(), - namespaces=(NamespacePlan(owner_path="demo", python_path=()),), - ) - - with pytest.raises(UnsupportedWrapperCodegenNodeError): - CSourcePrinter().doprint(plan) - with pytest.raises(UnsupportedWrapperCodegenNodeError): - FortranSourcePrinter().doprint(plan) - - -def test_fortran_source_printer_wraps_long_parenthesized_call_arguments(): - slices = ", ".join(f"1:values_upper_bound_{axis} + 1:values_stride_{axis}" for axis in range(4)) - source = FortranSourcePrinter().doprint( - FortranCall( - "native_scale", - ( - CodeExpression(f"values_base({slices})"), - CodeExpression(f"out_base({slices})"), + pytest.param( + FortranPointerAssignment("values", CodeExpression(f"values_base({_SLICES})")), + ("values => values_base(&", "& 1:values_upper_bound_3 + 1:values_stride_3)"), + (), + id="pointer_array_section", + ), + pytest.param( + FortranAssignment("result", CodeExpression(_LONG_SUM)), + (" &\n & ",), + (), + id="unstructured_statement", + ), + pytest.param( + _nested_assignment(_LONG_SUM), + (" & ",), + (), + id="after_nested_indentation", + ), + # A nested call's commas belong to it, so the outer break skips them. + pytest.param( + FortranAssignment( + "destination", + CodeExpression(f"compute_total(first_{_PADDING}, max(second_term, third_term), fourth_{_PADDING})"), ), - ) - ) - - assert "& values_base(&" in source - assert "& 1:values_upper_bound_3 + 1:values_stride_3), &" in source - assert "& out_base(&" in source - assert max(map(len, source.splitlines())) <= 124 - - -def test_fortran_source_printer_never_continues_inside_a_character_literal(): - """A literal's commas are its characters, so no continuation may split it. - - Fortran resumes a continued literal after the next line's `&`, so a break - placed at a comma inside quotes changes the characters the literal states - while still compiling. - """ - padding = "x" * 40 - expression = f"build_message(prefix_{padding}, 'alpha, beta', suffix_{padding})" - - source = FortranSourcePrinter().doprint(FortranAssignment("destination", CodeExpression(expression))) - - assert "'alpha, beta'" in source - assert "'alpha, &" not in source - assert max(map(len, source.splitlines())) <= 132 - - -def test_fortran_source_printer_breaks_a_call_at_its_own_arguments(): - """A nested call's commas belong to it, so the outer break skips them.""" - padding = "y" * 40 - expression = f"compute_total(first_{padding}, max(second_term, third_term), fourth_{padding})" - - source = FortranSourcePrinter().doprint(FortranAssignment("destination", CodeExpression(expression))) - - assert "& max(second_term, third_term), &" in source - assert max(map(len, source.splitlines())) <= 132 - - -def test_fortran_source_printer_wraps_long_pointer_array_sections(): - slices = ", ".join(f"1:values_upper_bound_{axis} + 1:values_stride_{axis}" for axis in range(4)) - - source = FortranSourcePrinter().doprint( - FortranPointerAssignment("values", CodeExpression(f"values_base({slices})")) - ) - - assert source.startswith("values => values_base(&") - assert "& 1:values_upper_bound_3 + 1:values_stride_3)" in source - assert max(map(len, source.splitlines())) <= 132 - - -def test_fortran_source_printer_formats_unstructured_long_statements_automatically(): - expression = " + ".join(f"value_{index}" for index in range(20)) - - source = FortranSourcePrinter().doprint(FortranAssignment("result", CodeExpression(expression))) - - assert " &\n & " in source - assert max(map(len, source.splitlines())) <= 132 - - -def test_fortran_source_printer_formats_after_nested_indentation_is_complete(): - expression = " + ".join(f"value_{index}" for index in range(20)) - source = FortranSourcePrinter().doprint( - FortranModule( - "nested_lines", - procedures=( - FortranFunction( - "nested", - body=( - FortranIf( - CodeExpression("outer"), - body=( - FortranIf( - CodeExpression("inner"), - body=(FortranAssignment("result", CodeExpression(expression)),), - ), - ), - ), - ), - is_subroutine=True, - ), + ("& max(second_term, third_term), &",), + (), + id="call_breaks_at_its_own_arguments", + ), + # Fortran resumes a continued literal after the next line's `&`, so a + # break at a comma inside quotes would change the literal's characters. + pytest.param( + FortranAssignment( + "destination", + CodeExpression(f"build_message(prefix_{_PADDING}, 'alpha, beta', suffix_{_PADDING})"), ), - ) - ) + ("'alpha, beta'",), + ("'alpha, &",), + id="never_inside_a_character_literal", + ), + ], +) +def test_fortran_source_printer_continues_long_statements_at_safe_points(node, present, absent): + source = FortranSourcePrinter().doprint(node) - assert " & " in source + for fragment in present: + assert fragment in source + for fragment in absent: + assert fragment not in source assert max(map(len, source.splitlines())) <= 132 @@ -256,16 +133,3 @@ def test_fortran_source_printer_continues_long_literals_without_changing_their_v def test_fortran_source_printer_rejects_an_overlong_token_without_a_safe_break(): with pytest.raises(ValueError, match=r"free-form limit is 132"): FortranSourcePrinter().doprint(FortranAssignment("result", CodeExpression("x" * 134))) - - -def test_source_printers_do_not_import_wrapper_plan_models(): - imports = set() - for filename in ("c.py", "fortran.py"): - path = REPO_ROOT / "prik" / "printers" / filename - imports.update( - node.module - for node in ast.walk(ast.parse(path.read_text(encoding="utf-8"))) - if isinstance(node, ast.ImportFrom) and node.module is not None - ) - - assert "prik.planning.models" not in imports diff --git a/tests/fortran/infrastructure/runtime/test_native_support.py b/tests/fortran/infrastructure/runtime/test_native_support.py index 722199502..cb4915f49 100644 --- a/tests/fortran/infrastructure/runtime/test_native_support.py +++ b/tests/fortran/infrastructure/runtime/test_native_support.py @@ -1,4 +1,4 @@ -"""Public native-binding support surface checks.""" +"""ABI and lifetime mechanisms of the bundled native-array backend record.""" import re @@ -6,38 +6,6 @@ SUPPORT_HEADER = REPO_ROOT / "prik" / "runtime" / "native_support" / "prik_binding.h" -SUPPORT_SOURCE = REPO_ROOT / "prik" / "runtime" / "native_support" / "prik_binding.c" - - -def test_native_binding_support_is_header_only_and_exposes_the_small_prik_api(): - header = SUPPORT_HEADER.read_text(encoding="utf-8") - assert not SUPPORT_SOURCE.exists() - - assert "static inline int prik_array_validate(" in header - assert "static inline int prik_array_validate_ndarray(" in header - assert "PyArrayObject *array," in header - assert "PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F" in header - assert "prik_array_actual" in header - assert "prik_release_owned_memory" in header - assert "prik_capture_address" in header - - scalar_suffixes = ( - "bool", - "int8", - "int16", - "int32", - "int64", - "float32", - "float64", - "complex64", - "complex128", - ) - for suffix in scalar_suffixes: - assert f"prik_{suffix}_unpack_exact" in header - assert f"PRIK_DEFINE_SCALAR_OR_STORAGE({suffix}, " in header - assert f"prik_{suffix}_unpack" in header - assert f"prik_{suffix}_to_python" in header - assert f"prik_{suffix}_to_numpy" in header BACKEND_RECORD = ( @@ -98,33 +66,6 @@ def test_the_capsule_name_covers_semantic_version_and_the_whole_record(): assert "for (character = layout[index].name; *character != '\\0'; ++character)" in header -def test_native_array_backend_capsule_exposes_one_entry_point_and_its_readers(): - """One entry point reaches the descriptor; the readers validate a producer. - - The context is what that entry point needs to get there, and a release - marks that context as this extension's to free. - """ - header = SUPPORT_HEADER.read_text(encoding="utf-8") - - for name in ( - "prik_native_array_backend_capsule_new", - "prik_native_array_backend_capsule_destructor", - "prik_native_array_backend_from_capsule", - "prik_native_array_backend_for_descriptor", - "prik_native_array_backend_for_actual", - "prik_native_array_backend_owned_descriptor", - "prik_native_array_backend_layout_tag", - "prik_native_array_backend_capsule_name", - "prik_native_array_backend_release", - "prik_native_array_owned_with_descriptor", - ): - assert name in header - - assert "invalid prik native array descriptor attribute" in header - assert "backend->descriptor_attribute != expected_descriptor_attribute" in header - assert "does not expose the descriptor attribute required by the dummy argument" in header - - def test_native_array_backend_release_is_idempotent_and_defers_active_storage(): """Owned storage is cleared once and remains alive through active calls.""" header = SUPPORT_HEADER.read_text(encoding="utf-8") @@ -139,8 +80,9 @@ def test_native_array_backend_release_is_idempotent_and_defers_active_storage(): assert "prik_native_array_backend_release_now(backend);" in body assert "backend->context = NULL;" in release_now assert "backend->release(context);" in release_now - assert "context_kind == PRIK_NATIVE_ARRAY_CONTEXT_DESCRIPTOR" in release_now - assert "free(context);" in release_now + # Release frees C descriptor storage only; a Fortran owner belongs to its + # runtime, so freeing it here would be undefined. + assert "if (context_kind == PRIK_NATIVE_ARRAY_CONTEXT_DESCRIPTOR) {\n free(context);" in release_now # A released owned backend reports itself closed rather than handing over # storage that is gone. reader = header[header.index("static inline prik_native_array_backend *prik_native_array_backend_from_capsule(") :] @@ -164,18 +106,6 @@ def test_address_capture_primitive_has_external_linkage_behind_one_opt_in(): assert "static inline void *prik_capture_address" not in header -def test_a_fortran_owner_context_is_never_freed_as_c_storage(): - """Release frees C storage only; a Fortran owner belongs to its runtime. - - v1 called ``free()`` on every released context because an owned context was - always ``malloc``'d descriptor storage. An owner is allocated by the Fortran - runtime instead, so freeing it here as well would be undefined. - """ - header = SUPPORT_HEADER.read_text(encoding="utf-8") - - assert "if (context_kind == PRIK_NATIVE_ARRAY_CONTEXT_DESCRIPTOR) {\n free(context);" in header - - def test_owning_storage_is_no_longer_the_same_fact_as_owning_a_descriptor(): """The owned-descriptor accessor refuses a context that is not a descriptor. @@ -190,15 +120,21 @@ def test_owning_storage_is_no_longer_the_same_fact_as_owning_a_descriptor(): assert "owns a Fortran entity" in body -def test_a_fortran_owner_cannot_be_published_without_an_identity(): - """An owner is the one context a reader cannot check by inspection. +def test_a_published_context_agrees_with_its_ownership_and_identity(): + """A backend cannot publish owned storage as borrowed, or the reverse. - Nothing about the address says which compiler laid out the bytes behind it - or which entity it was generated for, so publishing one without both values - would leave a foreign reader nothing to compare. + A Fortran owner is also the one context a reader cannot check by + inspection: nothing about the address says which compiler laid out the + bytes behind it or which entity it was generated for, so publishing one + without both identity values would leave a foreign reader nothing to + compare. """ header = SUPPORT_HEADER.read_text(encoding="utf-8") + assert "context == NULL && context_kind != PRIK_NATIVE_ARRAY_CONTEXT_NONE" in header + assert "prik native array backend cannot release borrowed context storage" in header + assert "prik native array owned context needs a release entry point" in header + assert "PRIK_NATIVE_ARRAY_CONTEXT_FORTRAN_OWNER && (owner_abi == 0 || owner_signature == 0)" in header assert "prik native array Fortran owner needs an owner ABI identity" in header # And the identity is meaningless on any other kind. @@ -208,15 +144,6 @@ def test_a_fortran_owner_cannot_be_published_without_an_identity(): assert "#define PRIK_FORTRAN_OWNER_ABI UINT64_C(0)" in header -def test_context_kind_agrees_with_storage_ownership(): - """A backend cannot publish owned storage as borrowed, or the reverse.""" - header = SUPPORT_HEADER.read_text(encoding="utf-8") - - assert "context == NULL && context_kind != PRIK_NATIVE_ARRAY_CONTEXT_NONE" in header - assert "prik native array backend cannot release borrowed context storage" in header - assert "prik native array owned context needs a release entry point" in header - - def test_owner_identity_is_compared_before_any_dereference(): """Both halves of the identity gate the read, and each reports its own cause.""" header = SUPPORT_HEADER.read_text(encoding="utf-8") diff --git a/tests/fortran/infrastructure/semantic_ir/semantics/test_compile_time_values.py b/tests/fortran/infrastructure/semantic_ir/semantics/test_compile_time_values.py index 167b6824e..be8f94620 100644 --- a/tests/fortran/infrastructure/semantic_ir/semantics/test_compile_time_values.py +++ b/tests/fortran/infrastructure/semantic_ir/semantics/test_compile_time_values.py @@ -1,5 +1,7 @@ """Tests split by stable ownership concept from `test_compile_time_values.py`.""" +from dataclasses import asdict + import pytest from prik.parsers.fortran.models import ( FortranArgument, @@ -26,6 +28,7 @@ ProjectionMapping, SemanticArgument, SemanticClass, + SemanticConstraint, SemanticFunction, SemanticMethod, SemanticModule, @@ -396,67 +399,6 @@ def test_semantic_compile_time_requirements_cover_all_parser_contexts(): assert _compile_time_requirement_message("other", "n", "n + 1") == "Compile-time value required for 'n'." -def test_resolve_semantic_compile_time_values_leaves_recorded_decisions_opaque(): - """A metadata tag is a decision already taken, not text awaiting a value. - - `fortran_pointer_association="runtime"` records how a pointer is - associated. A module that also declares `runtime` as a parameter must not - turn that recorded choice into the parameter's value. - """ - module = SemanticModule( - name="tagged_mod", - variables=[ - SemanticArgument( - name="view", - semantic_type=SemanticType( - name="Float64", - dtype="Float64", - rank=1, - shape=["runtime"], - metadata={"fortran_pointer_association": "runtime"}, - ), - ) - ], - ) - - resolved = resolve_semantic_compile_time_values(module, {"runtime": 4}) - - semantic_type = resolved.variables[0].semantic_type - assert semantic_type.shape == ["4"] - assert semantic_type.metadata == {"fortran_pointer_association": "runtime"} - - -def test_resolve_semantic_compile_time_values_rewrites_shapes(): - module = SemanticModule( - name="shape_mod", - variables=[ - SemanticArgument( - name="values", - semantic_type=SemanticType( - name="Float64", - dtype="Float64", - rank=1, - shape=["1:n"], - storage=semantic_models.SemanticStorageContract( - kind="array", - array=semantic_models.SemanticArrayContract( - rank=1, - shape=["1:n"], - source_shape=["1:n"], - ), - ), - ), - ) - ], - ) - - resolved = resolve_semantic_compile_time_values(module, {"n": 8}) - - assert module.variables[0].semantic_type.shape == ["1:n"] - assert resolved.variables[0].semantic_type.shape == ["1:8"] - assert resolved.variables[0].semantic_type.storage.array.shape == ["1:8"] - - def test_resolve_semantic_compile_time_values_handles_nested_modules(): """Specialization reaches every nesting level and touches only expression fields. @@ -473,14 +415,17 @@ def test_resolve_semantic_compile_time_values_handles_nested_modules(): dtype="Float64", rank=1, shape=["n"], + constraints=[SemanticConstraint("Extent", ["n", {"upper": "m"}])], storage=semantic_models.SemanticStorageContract( kind="array", + metadata={"address_role": "n"}, array=semantic_models.SemanticArrayContract( rank=1, shape=["n"], source_shape=["1:n"], lower_bounds=["n"], upper_bounds=["n"], + metadata={"representation": "m"}, ), ), metadata={"fortran_character_length": "n", "fortran_pointer_association": "n"}, @@ -531,10 +476,14 @@ def test_resolve_semantic_compile_time_values_handles_nested_modules(): metadata={"fortran_bind_c": "n"}, ) + original = asdict(module) + resolved = resolve_semantic_compile_time_values([module], {"n": 4, "m": 2}) - assert module.variables[0].semantic_type.shape == ["n"] + # Specialization returns new modules and is idempotent. + assert asdict(module) == original resolved_module = resolved[0] + assert asdict(resolve_semantic_compile_time_values(resolved_module, {"n": 4, "m": 2})) == asdict(resolved_module) # Every level's declaration expressions are specialized. assert resolved_module.variables[0].semantic_type.shape == ["4"] @@ -553,6 +502,9 @@ def test_resolve_semantic_compile_time_values_handles_nested_modules(): # Recorded decisions are opaque at every level, however they are spelled. assert resolved_module.variables[0].semantic_type.metadata["fortran_pointer_association"] == "n" + assert resolved_module.variables[0].semantic_type.constraints[0].arguments == ["n", {"upper": "m"}] + assert resolved_module.variables[0].semantic_type.storage.metadata == {"address_role": "n"} + assert resolved_module.variables[0].semantic_type.storage.array.metadata == {"representation": "m"} assert resolved_module.variables[0].metadata["address_role"] == "m" assert resolved_module.functions[0].arguments[0].metadata == {"native_callback_kind": "n"} assert resolved_module.functions[0].projection[0].value == {"kind": "return", "name": "n", "position": 0} @@ -563,18 +515,6 @@ def test_resolve_semantic_compile_time_values_handles_nested_modules(): assert resolved_module.metadata == {"fortran_bind_c": "n"} -def test_module_parameters_preserve_literal_values_in_semantic_ir(): - source = """ -module constants_mod - integer, parameter :: nmax = 12 -end module constants_mod -""" - - module = fortran_module_to_semantic_module(parse_fortran_source(source)) - - assert module.variables[0].default_value == "12" - - def test_fortran_file_and_project_helpers_forward_compile_time_values(): proc = FortranProcedureSignature( name="scale", diff --git a/tests/fortran/infrastructure/semantic_ir/semantics/test_semantic_conversion_smoke.py b/tests/fortran/infrastructure/semantic_ir/semantics/test_semantic_conversion_smoke.py index 9c2843871..2220a0ebc 100644 --- a/tests/fortran/infrastructure/semantic_ir/semantics/test_semantic_conversion_smoke.py +++ b/tests/fortran/infrastructure/semantic_ir/semantics/test_semantic_conversion_smoke.py @@ -7,24 +7,11 @@ from tests.fortran._support.paths import PARSER_FIXTURE_ROOT as TESTS_DIR from tests.fortran._support.fixture_conversion import FORTRAN_FIXTURES from tests.fortran._support.fixture_outputs import ( - SEMANTICS_FIXTURE_DIR, semantic_payload_for_fixture, semantics_fixture_path, ) -def test_semantics_fixture_suite_has_fixtures(): - assert FORTRAN_FIXTURES, "No final general Fortran parser fixtures found" - - -def test_semantic_fixtures_match_fortran_data_one_to_one(): - expected = {path.with_suffix(".json").name for path in FORTRAN_FIXTURES} - actual = {path.name for path in SEMANTICS_FIXTURE_DIR.glob("*.json")} - - assert not sorted(expected - actual) - assert not sorted(actual - expected) - - def _iter_semantic_types(node): if isinstance(node, dict): if "semantic_type" in node: @@ -48,15 +35,12 @@ def test_semantic_model_fixture_suite(fixture: Path): expected_path = semantics_fixture_path(fixture) expected = json.loads(expected_path.read_text(encoding="utf-8")) - assert semantic_payload_for_fixture(fixture) == expected - - -def test_semantic_fixtures_do_not_contain_unknown_types(): - unknown_types = [] - for path in SEMANTICS_FIXTURE_DIR.glob("*.json"): - payload = json.loads(path.read_text(encoding="utf-8")) - for semantic_type in _iter_semantic_types(payload): - if semantic_type.get("name") == "Unknown" or semantic_type.get("dtype") == "Unknown": - unknown_types.append(path.name) + payload = semantic_payload_for_fixture(fixture) - assert not unknown_types, f"Unknown semantic types in fixtures: {unknown_types[:20]}" + assert payload == expected + unknown = [ + semantic_type + for semantic_type in _iter_semantic_types(payload) + if semantic_type.get("name") == "Unknown" or semantic_type.get("dtype") == "Unknown" + ] + assert not unknown, f"Unknown semantic types: {unknown[:20]}" diff --git a/tests/fortran/infrastructure/semantic_ir/semantics/test_semantic_specialization_properties.py b/tests/fortran/infrastructure/semantic_ir/semantics/test_semantic_specialization_properties.py deleted file mode 100644 index f3d2221a8..000000000 --- a/tests/fortran/infrastructure/semantic_ir/semantics/test_semantic_specialization_properties.py +++ /dev/null @@ -1,104 +0,0 @@ -"""Tests split by stable ownership concept from `test_c_conversion_properties.py`.""" - -import pytest -from dataclasses import asdict -from hypothesis import ( - given, - strategies as st, -) -from prik.parsers.fortran import parse_fortran_file -from prik.semantics.fortran2ir import ( - fortran_file_to_semantic_modules, - resolve_semantic_compile_time_values, -) -from prik.semantics.models import ( - SemanticArgument, - SemanticArrayContract, - SemanticConstraint, - SemanticModule, - SemanticStorageContract, - SemanticType, -) -from tests.fortran._support.semantic_properties import _FORTRAN_VALUE_TYPES - - -@pytest.mark.property -@given(_FORTRAN_VALUE_TYPES) -def test_generated_fortran_value_arguments_have_the_expected_semantic_contract(case): - fortran_type, semantic_type = case - fortran_module = fortran_file_to_semantic_modules( - parse_fortran_file( - f"subroutine consume(value)\n {fortran_type}, intent(in), value :: value\nend subroutine consume\n", - filename="generated.f90", - ), - standalone_module_name="generated", - )[0] - - fortran_argument = fortran_module.functions[0].arguments[0] - assert fortran_argument.semantic_type.name == semantic_type - assert fortran_argument.semantic_type.origin.source_language == "fortran" - - -@pytest.mark.property -@given(n=st.integers(min_value=1, max_value=999), m=st.integers(min_value=1, max_value=999)) -def test_generated_semantic_specialization_is_non_mutating_and_idempotent(n, m): - module = SemanticModule( - name="generated", - variables=[ - SemanticArgument( - name="values", - semantic_type=SemanticType( - name="Float64", - dtype="Float64", - rank=2, - shape=["1:n", "m + 1"], - constraints=[SemanticConstraint("Extent", ["n", {"upper": "m"}])], - metadata={"fortran_character_length": "n", "enum_name": "n"}, - storage=SemanticStorageContract( - kind="array", - metadata={"address_role": "n"}, - array=SemanticArrayContract( - rank=2, - shape=["1:n", "m + 1"], - lower_bounds=["1", "0"], - upper_bounds=["n", "m"], - source_shape=["1:n", "0:m"], - metadata={"representation": "m"}, - ), - ), - ), - default_value="n + m", - metadata={"fortran_initializer": "m", "fortran_pointer_association": "n"}, - ) - ], - metadata={"import_scope": "n"}, - ) - original = asdict(module) - - resolved = resolve_semantic_compile_time_values(module, {"n": n, "m": m}) - - assert asdict(module) == original - variable = resolved.variables[0] - semantic_type = variable.semantic_type - assert semantic_type.shape == [f"1:{n}", f"{m} + 1"] - assert semantic_type.storage is not None - assert semantic_type.storage.array is not None - assert semantic_type.storage.array.shape == [f"1:{n}", f"{m} + 1"] - assert semantic_type.storage.array.lower_bounds == ["1", "0"] - assert semantic_type.storage.array.upper_bounds == [str(n), str(m)] - assert semantic_type.storage.array.source_shape == [f"1:{n}", f"0:{m}"] - - # Declared expression fields are specialized; the default value is one too. - assert semantic_type.metadata["fortran_character_length"] == str(n) - assert variable.metadata["fortran_initializer"] == str(m) - assert variable.default_value == f"{n} + {m}" - - # Everything else is an opaque recorded decision, whatever it spells. - assert semantic_type.metadata["enum_name"] == "n" - assert semantic_type.constraints[0].arguments == ["n", {"upper": "m"}] - assert semantic_type.storage.metadata == {"address_role": "n"} - assert semantic_type.storage.array.metadata == {"representation": "m"} - assert variable.metadata["fortran_pointer_association"] == "n" - assert resolved.metadata == {"import_scope": "n"} - - assert asdict(resolve_semantic_compile_time_values(resolved, {"n": n, "m": m})) == asdict(resolved) diff --git a/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/pipeline/test_declaring_namespace_publication.py b/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/pipeline/test_declaring_namespace_publication.py index f9c36826c..81f7d18bf 100644 --- a/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/pipeline/test_declaring_namespace_publication.py +++ b/tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/pipeline/test_declaring_namespace_publication.py @@ -81,14 +81,15 @@ def _plan(entry: Path, tmp_path: Path, name: str): ALL_NAMES = ["counter", "area_i", "area_r", "scale_value", "area"] -@pytest.mark.parametrize("name", ["counter", "area"]) -def test_the_declaring_namespace_may_publish_either_kind(name: str, tmp_path: Path): - """Publishing one where it is declared is what a source build already does.""" +def test_the_declaring_namespace_may_publish_every_kind(tmp_path: Path): + """Publishing a variable, procedure, and generic where declared is what a source build already does.""" entry = _package(tmp_path, home_exports=ALL_NAMES, facade="__all__ = []\n") - result = _plan(entry, tmp_path, f"declaring_only_{name}") + result = _plan(entry, tmp_path, "declaring_only") - assert result.output_dir.is_dir() + generated = (result.output_dir / "declaring_only_wrapper.c").read_text(encoding="utf-8") + assert generated.count("static PyObject * module_get_counter(void) {") == 1 + assert '"area"' in generated def test_a_facade_may_publish_the_declaring_namespaces_variable(tmp_path: Path): @@ -146,4 +147,5 @@ def test_a_procedure_still_reaches_python_through_a_facade(tmp_path: Path): result = _plan(entry, tmp_path, "procedure_facade") - assert result.output_dir.is_dir() + generated = (result.output_dir / "procedure_facade_wrapper.c").read_text(encoding="utf-8") + assert '"scale_value"' in generated diff --git a/tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py b/tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py index d42048615..03f575e78 100644 --- a/tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py +++ b/tests/fortran/infrastructure/semantic_pyi/contracts/functions_and_classes/semantics/test_method_and_constructor_contracts.py @@ -2,7 +2,6 @@ import pytest import re -from dataclasses import asdict from prik.printers import emit_module from prik.semantics.metadata import ( BIND_TARGET_METADATA, @@ -28,17 +27,9 @@ def reset(self) -> Int32: ... assert method.visibility == "private" assert method.origin.metadata[USER_PRIVATE_METADATA] is True assert [arg.name for arg in method.arguments] == ["self"] - assert asdict(method.projection[0]) == { - "python_name": "self", - "native_name": "self", - "native_position": 0, - "python_position": 0, - "result_position": None, - "value_kind": None, - "value": None, - "value_cast": None, - "native_c_identity": None, - } + assert [(item.native_position, item.python_position, item.python_name) for item in method.projection] == [ + (0, 0, "self") + ] emitted = emit_module(module) assert " @private\n def reset(self) -> Int32: ..." in emitted reparsed = parse_pyi_text(emitted, module_name="edited") diff --git a/tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py b/tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py index b18c66a3b..e462ae0fb 100644 --- a/tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py +++ b/tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py @@ -1,6 +1,7 @@ """An unedited semantic contract is authoritative runtime build input.""" import importlib +import json import subprocess import sys from pathlib import Path @@ -102,8 +103,8 @@ def test_generated_contract_rebuilds_without_native_source_fallback(compiled_con WILDCARD_SOURCE = (NATIVE_FIXTURES / "wildcard_home.f90").read_text(encoding="utf-8") -def _wildcard_contracts(tmp_path: Path, consumer: str) -> Path: - """Generate contracts, withhold `two` from the home surface, add a consumer.""" +def _wildcard_contracts(tmp_path: Path, readers: dict[str, str]) -> Path: + """Generate contracts, withhold `two` from the home surface, add consumer contracts.""" source = tmp_path / "wild.f90" source.write_text(WILDCARD_SOURCE, encoding="utf-8") package = tmp_path / "contracts" @@ -115,48 +116,40 @@ def _wildcard_contracts(tmp_path: Path, consumer: str) -> Path: ) home = package / "wild_home.pyi" home.write_text(home.read_text(encoding="utf-8").replace('["one", "two"]', '["one"]'), encoding="utf-8") - package.joinpath("wild_reader.pyi").write_text(consumer, encoding="utf-8") - package.joinpath("__init__.pyi").write_text( - 'from . import wild_home\nfrom . import wild_reader\n\n__all__ = ["wild_home", "wild_reader"]\n', - encoding="utf-8", - ) + for name, text in readers.items(): + package.joinpath(f"{name}.pyi").write_text(text, encoding="utf-8") + names = ["wild_home", *readers] + imports = "".join(f"from . import {name}\n" for name in names) + package.joinpath("__init__.pyi").write_text(f"{imports}\n__all__ = {json.dumps(names)}\n", encoding="utf-8") return package / "__init__.pyi" -def _build_wildcard(entry: Path, tmp_path: Path, name: str): +def test_withheld_name_is_skipped_by_a_wildcard_but_reachable_by_name(tmp_path: Path): + """A wildcard takes what a contract publishes; naming a withheld name still reaches it. + + The dependency stated its surface, and a name left off it is not part of + what writing `*` asks for. A contract may still need the withheld name, to + express a declaration or to publish it again, and naming it in an import + asks for exactly that. + """ + entry = _wildcard_contracts( + tmp_path, + { + "wild_star": "from .wild_home import *\n", + "wild_named": 'from .wild_home import two\n\n__all__ = ["two"]\n', + }, + ) result = build_pyi_extension( entry, input_compiler=_compiler(), native_fortran_sources=[str(tmp_path / "wild.f90")], - output_dir=tmp_path / name, - output_name=name, + output_dir=tmp_path / "wildcard", + output_name="wildcard", ) - return _import_from_build_dir(result.module_name, result.output_dir) - - -def test_wildcard_import_reads_only_the_surface_its_dependency_publishes(tmp_path: Path): - """A wildcard takes what a contract publishes, not everything it holds. - - The dependency stated its surface, and a name left off it is not part of - what writing `*` asks for. - """ - entry = _wildcard_contracts(tmp_path, "from .wild_home import *\n") - module = _build_wildcard(entry, tmp_path, "wildcard_star") + module = _import_from_build_dir(result.module_name, result.output_dir) assert hasattr(module.wild_home, "one") assert not hasattr(module.wild_home, "two") - assert hasattr(module.wild_reader, "one") - assert not hasattr(module.wild_reader, "two") - - -def test_explicit_import_reaches_and_can_republish_a_withheld_name(tmp_path: Path): - """A withheld name stays reachable, because a contract may still need it. - - Expressing a declaration or publishing the name again both require asking - for it, which is exactly what naming it in an import does. - """ - entry = _wildcard_contracts(tmp_path, 'from .wild_home import two\n\n__all__ = ["two"]\n') - module = _build_wildcard(entry, tmp_path, "wildcard_named") - - assert not hasattr(module.wild_home, "two") - assert module.wild_reader.two(np.int32(5)) == np.int32(7) + assert hasattr(module.wild_star, "one") + assert not hasattr(module.wild_star, "two") + assert module.wild_named.two(np.int32(5)) == np.int32(7) diff --git a/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py b/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py index 9752812ff..f67cb3456 100644 --- a/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py +++ b/tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py @@ -3,16 +3,11 @@ import ast import pytest from prik.printers import emit_module -from prik.contracts import CONTRACT_SYMBOLS from prik.semantics.models import ( - ProjectionMapping, - SemanticArgument, SemanticConstraint, SemanticModule, - SemanticType, ) from prik.semantics.pyi2ir import ( - _PyiAstParser, convert_pyi_to_ir, ) from tests.fortran._support.pyi_conversion import ( @@ -23,21 +18,13 @@ def test_pyi_parser_returns_python_ast_only(): - tree = parse_pyi_ast_text("def scale(value: Float64) -> Float64: ...\n", filename="scale.pyi") + """The parser owns syntax only; semantic conversion accepts nothing but its AST.""" + source = f"{CONTRACT_IMPORT}def scale(value: Float64) -> Float64: ...\n" + tree = parse_pyi_ast_text(source, filename="scale.pyi") assert isinstance(tree, ast.Module) - assert isinstance(tree.body[0], ast.FunctionDef) - assert tree.body[0].name == "scale" - - -def test_convert_pyi_to_ir_accepts_parsed_pyi_ast_only(): - source = f"{CONTRACT_IMPORT}value: Int32\n" - tree = parse_pyi_ast_text(source, filename="contract.pyi") - - module = convert_pyi_to_ir(tree, module_name="parsed", source=source) - - assert module.name == "parsed" - assert module.variables[0].name == "value" + assert isinstance(tree.body[-1], ast.FunctionDef) + assert convert_pyi_to_ir(tree, module_name="parsed", source=source).functions[0].name == "scale" with pytest.raises(TypeError, match=r"expects a Python ast\.Module"): convert_pyi_to_ir(source) @@ -75,12 +62,6 @@ def test_pyi_parser_reports_unsupported_lines_and_invalid_helpers(): with pytest.raises(ValueError, match="Expected imported prik contract helper"): parse_pyi_text("@native_call([Unknown(0)])\ndef f(x: Int32) -> None: ...\n", module_name="edited") - with pytest.raises(ValueError, match="Expected imported prik contract helper"): - parse_pyi_text("@native_call([Len(Unknown(0))])\ndef f(x: Int32) -> None: ...\n", module_name="edited") - - with pytest.raises(ValueError, match="Unknown semantic type is not allowed"): - parse_pyi_text("x: Unknown\n", module_name="edited") - def test_pyi_parser_preserves_generic_constraints_as_annotation_metadata(): module = parse_pyi_text( @@ -102,30 +83,3 @@ def test_pyi_parser_preserves_generic_constraints_as_annotation_metadata(): emitted = emit_module(SemanticModule(name="constraints", variables=[module.variables[0]])) assert "value: Annotated[Int32, Bounded(1, 8), Finite]" in emitted assert parse_pyi_text(emitted, module_name="constraints").variables[0] == module.variables[0] - - -def test_pyi_parser_internal_projection_helpers_preserve_native_names(): - parser = _PyiAstParser(module_name="internal") - parser._contract_bindings.update({name: name for name in CONTRACT_SYMBOLS}) - return_type, returned_values = parser.return_projection( - ast.parse("tuple[Float64, Returns['extra', Int32] | None, Returns['other', Float64]]", mode="eval").body - ) - pointer = parser.semantic_type(ast.parse("Addr(Float64)", mode="eval").body) - returned = SemanticArgument("result", SemanticType("Float64"), metadata={"return_position": 1}) - mapping = ProjectionMapping(native_name="native_result", result_position=1) - _, values = parser._apply_native_call_returns(None, [returned], [mapping]) - native_arg = SemanticArgument("python_name", SemanticType("Int32")) - arg_mapping = ProjectionMapping(native_name="native_name", python_position=0) - parser._apply_native_call_argument_names([native_arg], {}, [arg_mapping]) - - assert return_type.name == "Float64" - assert returned_values[0].name == "extra" - assert returned_values[0].optional is True - assert returned_values[0].metadata == {"return_position": 1} - assert returned_values[0].semantic_type.ownership.mutable is True - assert returned_values[1].name == "other" - assert returned_values[1].metadata == {"return_position": 2} - assert pointer.storage.mutable is True - assert pointer.ownership.mutable is True - assert values[0].name == "native_result" - assert arg_mapping.native_name == "native_name" diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/fixtures/native/emit_complex_fem_module.f90 b/tests/fortran/infrastructure/semantic_pyi/pipeline/fixtures/native/emit_complex_fem_module.f90 deleted file mode 100644 index de6266a3b..000000000 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/fixtures/native/emit_complex_fem_module.f90 +++ /dev/null @@ -1,30 +0,0 @@ -module fem_mod - -type :: mesh - - integer :: nelements - integer :: nnodes - -end type - -contains - -subroutine assemble(K, coords, connectivity) - - real(8), intent(out) :: K(:, :) - - real(8), intent(in) :: coords(:, :) - - integer, intent(in) :: connectivity(:, :) - -end subroutine - -function compute_norm(x) result(r) - - real(8), intent(in) :: x(:) - - real(8) :: r - -end function - -end module diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_calls_and_policy_metadata.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_calls_and_policy_metadata.py deleted file mode 100644 index 636569d3f..000000000 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_calls_and_policy_metadata.py +++ /dev/null @@ -1,505 +0,0 @@ -"""Tests split by stable ownership concept from `test_imports_and_packages.py`.""" - -import pytest -from prik.parsers.fortran import parse_fortran_file as parse_fortran_source -from prik.printers import ( - PyiPrinter, - emit_module, -) -from prik.semantics.fortran2ir import fortran_module_to_semantic_module -from prik.semantics.models import ( - ProjectionMapping, - SemanticArgument, - SemanticArrayContract, - SemanticConstraint, - SemanticFunction, - SemanticModule, - SemanticStorageContract, - SemanticType, - SemanticVariable, -) -from tests.fortran._support.printer_models import ( - generate_pyi, - generate_wrapper, - normalize, - parse_pyi_text, - rendered_source, -) -from prik.semantics.metadata import MAYBE_UNALLOCATED_METADATA - - -def test_emit_optional_scalar_output_as_visible_scalar_storage(): - source = """ -module opt_out_mod -contains -subroutine maybe_status(status) - integer(4), intent(out), optional :: status -end subroutine maybe_status -end module opt_out_mod -""" - - code = generate_pyi(source) - - assert "Return('status'" not in code - assert "status: Int32[()] = ..." in code - assert ') -> Returns["status", Int32[()]] | None: ...' in code - - -def test_emit_scalar_character_inout_as_replacement_return(): - source = """ -module m -contains -subroutine normalize(name) - character(len=8), intent(inout) :: name -end subroutine -end module -""" - - code = generate_pyi(source) - - annotation = "String[8]" - assert "@native_call([Arg(0)])" not in code - assert f"name: {annotation}" in code - assert f') -> Returns["name", {annotation}]: ...' in code - - -def test_emit_primitive_scalar_inout_as_visible_replacement_return(): - source = """ -module outputs -contains -subroutine scale_in_place(value, factor) - real(8), intent(inout) :: value - real(8), intent(in) :: factor -end subroutine scale_in_place -end module outputs -""" - - code = generate_pyi(source) - - assert "@native_call([Addr(Arg(0)), Addr(Arg(1))])" in code - assert "value: Float64" in code - assert "factor: Float64" in code - assert ') -> Returns["value", Float64]: ...' in code - - -def test_emit_exact_output(): - source = """ -module simple_mod - -contains - -subroutine scale(x) - - real(8), intent(inout) :: x(:) - -end subroutine - -end module -""" - - code = normalize(generate_pyi(source)) - - expected = normalize( - """ -def scale( - x: Float64[::] -) -> None: ... -""" - ) - - assert expected in code - - -def test_output_argument_uses_plain_return_annotation(): - source = """ -module output_name_mod - -contains - -subroutine add(a, b, c) - - real(8), intent(in) :: a - real(8), intent(in) :: b - real(8), intent(out) :: c - -end subroutine - -end module -""" - - fmod = parse_fortran_source(source) - smod = fortran_module_to_semantic_module(fmod) - - code = PyiPrinter().emit(smod) - - assert "c: Annotated[Addr(Float64)" not in code - assert 'Returns["c"' not in code - assert ") -> Float64: ..." in code - - -def test_emit_module_with_projection_helpers_and_private_function(): - module = SemanticModule( - name="projection_mod", - functions=[ - SemanticFunction( - name="wrapper", - native_name="wrapper", - arguments=[ - SemanticArgument("x", SemanticType("Float64", dtype="Float64")), - SemanticArgument("b", SemanticType("Int32", dtype="Int32"), optional=True), - ], - projection=[ - ProjectionMapping(native_position=0, python_position=0), - ProjectionMapping( - native_position=1, - value_kind="literal", - value={"type": "Int32", "value": 1}, - ), - ProjectionMapping( - native_position=2, - value_kind="len", - value={"kind": "arg", "position": 0}, - ), - ProjectionMapping( - native_position=3, - value_kind="shape", - value={"value": {"kind": "arg", "position": 0}, "dim": 0}, - ), - ProjectionMapping( - native_position=4, - value_kind="is_present", - value={"kind": "arg", "position": 1}, - ), - ProjectionMapping(native_position=5, value_kind="work", value="tmp"), - ], - ) - ], - ) - - code = emit_module(module) - - assert "@native_call([Arg(0), Int32(1), Len(Arg(0)), Arg(0).shape[0], IsPresent(Arg(1)), Work('tmp')])" in code - - -def test_emit_native_call_supports_return_and_work_value_references(): - module = SemanticModule( - name="projection_refs_mod", - functions=[ - SemanticFunction( - name="wrapper", - native_name="wrapper", - return_type=SemanticType("Float64", dtype="Float64"), - projection=[ - ProjectionMapping( - native_position=0, - value_kind="len", - value={"kind": "return", "position": 0}, - ), - ProjectionMapping( - native_position=1, - value_kind="shape", - value={"value": {"kind": "work", "name": "tmp"}, "dim": 1}, - ), - ], - ) - ], - ) - - code = emit_module(module) - - assert "@native_call([Len(Return(0)), Work('tmp').shape[1]])" in code - assert "def wrapper() -> Float64: ..." in code - - -@pytest.mark.parametrize( - "projection, message", - [ - ( - [ProjectionMapping(native_position=0)], - "native-only projection entry", - ), - ( - [ProjectionMapping(native_position=0, value_kind="unknown", value=None)], - "Unsupported native_call projection entry", - ), - ( - [ - ProjectionMapping( - native_position=0, - value_kind="len", - value={"kind": "unknown"}, - ) - ], - "Unsupported native_call value reference", - ), - ], -) -def test_emit_native_call_rejects_unrepresentable_projection_entries(projection, message): - module = SemanticModule( - name="bad_projection_mod", - functions=[ - SemanticFunction( - name="wrapper", - native_name="wrapper", - projection=projection, - ) - ], - ) - - with pytest.raises(ValueError, match=message): - emit_module(module) - - -def test_printer_rejects_optional_absent_array_handles_outside_callable_arguments(): - variable = SemanticVariable( - "values", - SemanticType( - "Float64", - rank=1, - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract(rank=1, shape=[":"], allocatable=True), - ), - ), - ) - variable.optional = True - module = SemanticModule( - name="bad_optional_handle_mod", - variables=[variable], - ) - - with pytest.raises(ValueError, match="only be emitted for callable arguments"): - emit_module(module) - - -def test_printer_emits_extended_storage_and_callable_forms(): - printer = PyiPrinter() - readonly_value = SemanticType( - "Int32", - storage=SemanticStorageContract(kind="value", read_only=True), - ) - mutable_value = SemanticType("Int32", storage=SemanticStorageContract(kind="value")) - deep_pointer = SemanticType( - "Float64", - storage=SemanticStorageContract(kind="pointer", read_only=True, pointer_depth=3), - ) - double_pointer = SemanticType( - "Float64", - storage=SemanticStorageContract(kind="pointer", pointer_depth=2), - ) - unspecified_storage = SemanticType("Int32", storage=SemanticStorageContract(kind="custom")) - inferred_array = SemanticType( - "Float64", - rank=2, - storage=SemanticStorageContract(kind="array"), - ) - allocatable_handle = SemanticType( - "Float64", - rank=2, - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract( - rank=2, - shape=[":", ":"], - order="ORDER_F", - allocatable=True, - ), - ), - ) - constrained_allocatable_handle = SemanticType( - "Bool", - rank=1, - constraints=[SemanticConstraint("Finite")], - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract(rank=1, shape=["1"], allocatable=True), - ), - ) - maybe_unallocated_handle = SemanticType( - "Float64", - rank=1, - metadata={MAYBE_UNALLOCATED_METADATA: True}, - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract(rank=1, shape=[":"], allocatable=True), - ), - ) - pointer_handle = SemanticType( - "Float64", - rank=1, - metadata={"fortran_pointer_association": "runtime"}, - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract(rank=1, shape=[":"], pointer=True), - ), - ) - string_pointer_handle = SemanticType( - "String", - rank=1, - metadata={"fortran_character_length": "8"}, - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract(rank=1, shape=[":"], pointer=True), - ), - ) - annotated_array = SemanticType( - "Float64", - constraints=[SemanticConstraint("Finite"), SemanticConstraint("Range", [1, 3])], - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract( - rank=2, - shape=[":", ":"], - order="ORDER_ANY", - ), - ), - ) - character = SemanticType( - "String", - metadata={"fortran_character_length": "16"}, - storage=SemanticStorageContract(kind="reference", mutable=True, pointer_depth=1), - ) - allocatable_character = SemanticType( - "String", - metadata={"fortran_character_length": ":", "fortran_allocatable": True}, - ) - pointer_scalar = SemanticType( - "Int32", - metadata={"fortran_pointer": True, "fortran_pointer_association": "runtime"}, - storage=SemanticStorageContract(kind="reference", mutable=True, pointer_depth=1), - ) - - canonical_constant = SemanticArgument( - "answer", - SemanticType("Int32", constraints=[SemanticConstraint("Constant")]), - ) - assert printer.emit(canonical_constant) == "answer: Final[Int32]" - with pytest.raises(ValueError, match=r"Final\[\.\.\.\]"): - printer.emit(canonical_constant.semantic_type) - assert printer.emit(readonly_value) == "Int32" - assert printer.emit(mutable_value) == "Int32" - assert printer.emit(deep_pointer) == "Addr[3](Float64)" - assert printer.emit(double_pointer) == "Addr[2](Float64)" - assert printer.emit(unspecified_storage) == "Int32" - assert printer.emit(inferred_array) == "Float64[:, :]" - assert printer.emit(allocatable_handle) == "Allocatable[Annotated[Float64[:, :], ORDER_F]]" - assert printer.emit(constrained_allocatable_handle) == "Allocatable[Annotated[Bool[1], Finite]]" - assert printer.emit(maybe_unallocated_handle) == "Annotated[Allocatable[Float64[:]], MaybeUnallocated]" - assert printer.emit(pointer_handle) == 'Annotated[Pointer[Float64[:]], PointerAssociation("runtime")]' - assert printer.emit(string_pointer_handle) == "Pointer[String[8][:]]" - assert printer.emit(annotated_array) == "Annotated[Float64[:, :], ORDER_ANY, Finite, Range(1, 3)]" - assert printer.emit(character) == "String[16]" - assert printer.emit(allocatable_character) == "Allocatable[String[:]]" - assert printer.emit(pointer_scalar) == "Pointer[Int32]" - - -@pytest.mark.parametrize( - ("argument_type", "projection"), - [ - ( - SemanticType( - "Float64", - metadata={"fortran_pointer": True, "fortran_pointer_association": "runtime"}, - storage=SemanticStorageContract(kind="reference", mutable=True, pointer_depth=1), - ), - "Pointer", - ), - ], - ids=["argument_type1-Pointer"], -) -def test_printer_emits_nullable_scalar_descriptor_boundary_projections(argument_type, projection): - module = SemanticModule( - name="pointer_descriptor_mod", - functions=[ - SemanticFunction( - name="read_pointer", - native_name="read_pointer", - arguments=[SemanticArgument("value", argument_type)], - return_type=argument_type, - projection=[ProjectionMapping(native_position=0, python_position=0)], - ) - ], - ) - - code = emit_module(module) - - assert f"@native_call([{projection}(Arg(0))], result={projection}(Return(0)))" in code - assert "value: Float64 | None" in code - assert ") -> Float64 | None: ..." in code - - -def test_defaulted_scalar_descriptors_preserve_omitted_vs_none_in_generated_wrappers(): - loaded = parse_pyi_text( - """ -@native_call([Allocatable(Arg(0)), Pointer(Arg(1))]) -def update(scale: Float64 | None = ..., target: Float64 | None = ...) -> None: ... -""", - module_name="optional_scalar_descriptors", - ) - artifacts = generate_wrapper(loaded) - bridge_source = rendered_source(artifacts, ".f90") - c_wrapper = rendered_source(artifacts, ".c") - - assert "bound_scale_present" in bridge_source - assert "bound_target_present" in bridge_source - assert "if (c_associated(bound_scale_present)) then" in bridge_source - assert "if (c_associated(bound_target_present)) then" in bridge_source - assert "scale=prik_optional_scale" in bridge_source - assert "target=prik_optional_target" in bridge_source - assert bridge_source.count("call native_update(") == 1 - - assert "bound_scale_obj = NULL;" in c_wrapper - assert "bound_target_obj = NULL;" in c_wrapper - assert "if (bound_scale_obj != NULL)" in c_wrapper - assert "bound_scale_present = &bound_scale;" in c_wrapper - assert "if ((bound_scale_obj != NULL) && (bound_scale_obj != Py_None))" in c_wrapper - assert "bound_scale_nullable = &bound_scale;" in c_wrapper - assert ( - "bind_c_update(bound_scale_nullable, bound_scale_present, bound_target_nullable, bound_target_present);" - in c_wrapper - ) - assert "Omit to make the native optional dummy absent." in c_wrapper - assert "Pass None for a present unallocated or unassociated descriptor." in c_wrapper - assert "Default is None." not in c_wrapper - - -def test_printer_projection_return_helpers_and_keyword_data_members(): - printer = PyiPrinter() - argument = SemanticArgument( - "x", - SemanticType( - "Float64", - storage=SemanticStorageContract(kind="reference", mutable=True, pointer_depth=1), - ), - optional=True, - ) - plain = SemanticArgument("value", SemanticType("Int32")) - module = SemanticModule( - name="returns", - variables=[SemanticArgument("class", SemanticType("Int32"))], - functions=[ - SemanticFunction( - name="created", - projection=[ProjectionMapping(native_position=0, result_position=0)], - ) - ], - ) - context = printer._emission_context(module) - - assert printer._projected_argument_return(argument, context, visible=True) == 'Returns["x", Addr(Float64)] | None' - assert printer._named_return(plain, context) == 'Returns["value", Int32]' - assert printer._projected_argument_return(argument, context, visible=False) == "Float64 | None" - assert printer._projected_argument_return(plain, context, visible=False) == "Int32" - assert "var['class']: Int32" in emit_module(module) - assert "@native_call([Return(0)])" in emit_module(module) - - -def test_native_call_sorts_synthetic_entries_before_native_positions(): - printer = PyiPrinter() - projection = [ - ProjectionMapping(native_position=0, value_kind="literal", value={"type": "Int32", "value": 1}), - ProjectionMapping(result_position=0), - ] - - context = printer._emission_context(SemanticModule(name="native_call")) - assert printer._native_call(projection, context) == "@native_call([Return(0), Int32(1)])" diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_classes_and_methods.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_classes_and_methods.py index a7ab6efb7..c9aee6345 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_classes_and_methods.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_classes_and_methods.py @@ -1,26 +1,15 @@ -"""Tests split by stable ownership concept from `test_imports_and_packages.py`.""" +"""Generated overload sets, generics, and defined operators reload from their contracts.""" from pathlib import Path -import pytest from prik.parsers.fortran import parse_fortran_file as parse_fortran_source -from prik.pipeline.pyi import emit_module_stubs, pyi_text_to_semantic_module -from prik.printers import PyiPrinter, emit_module +from prik.pipeline.pyi import pyi_text_to_semantic_module +from prik.printers import emit_module from prik.semantics.fortran2ir import fortran_module_to_semantic_module -from prik.semantics.models import ( - ProjectionMapping, - SemanticArgument, - SemanticClass, - SemanticFunction, - SemanticMethod, - SemanticModule, - SemanticType, -) from tests.fortran._support.printer_models import ( OPERATOR_F90_SOURCE, generate_pyi, generate_wrapper, - normalize, parse_pyi_text, rendered_source, ) @@ -28,176 +17,6 @@ NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" -def test_emit_class(): - source = """ -module sparse_mod - -type :: sparse_matrix - - integer :: nrows - integer :: ncols - -end type - -end module -""" - - code = generate_pyi(source) - - assert "class sparse_matrix" in code - - -def test_emit_inheritance(): - source = """ -module inheritance_mod - -type :: base_matrix -end type - -type, extends(base_matrix) :: sparse_matrix -end type - -end module -""" - - code = generate_pyi(source) - - assert "class sparse_matrix(base_matrix)" in code - - -def test_printer_class_entrypoint(): - source = """ -module class_print_mod - -contains - -subroutine touch(x) - - integer, intent(inout) :: x - -end subroutine - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - code = PyiPrinter().emit(smod) - - assert "def touch(" in code - assert "@native_call([Addr(Arg(0))])" in code - assert "x: Int32" in code - - -def test_emit_class_method_keeps_method_indentation(): - module = SemanticModule( - name="method_mod", - classes=[ - SemanticClass( - name="thing", - methods=[SemanticMethod(name="reset")], - ) - ], - ) - - code = emit_module(module) - - assert "class thing:\n def reset(self) -> None: ..." in code - - -def test_emit_type_bound_procedure_as_python_method_without_duplicate_self(): - source = """ -module vector_mod - private - public :: vector - - type :: vector - real(8), allocatable :: values(:) - contains - procedure :: scale - end type vector - -contains - subroutine scale(self, alpha) - type(vector), intent(inout) :: self - real(8), intent(in) :: alpha - end subroutine scale -end module vector_mod -""" - - code = generate_pyi(source) - - assert "class vector:" in code - assert "values: Allocatable[Float64[:]]" in code - assert " @native_call([Pass(), Addr(Arg(0))])" in code - assert " def scale(\n self,\n alpha: Float64\n ) -> None: ..." in code - assert " self: vector" not in code - - -def test_emit_fortran_type_default_constructor_and_field_values(): - source = """ -module constructor_mod - type :: state - integer :: id = 7 - real(8) :: scale = 2.5 - logical :: enabled = .true. - end type state -end module constructor_mod -""" - - code = generate_pyi(source) - - assert normalize( - """ -class state: - def __init__( - self, - *, - id: Int32 = 7, - scale: Float64 = 2.5, - enabled: Bool = True - ) -> None: ... -""" - ) in normalize(code) - assert " id: Int32 = 7" in code - assert " scale: Float64 = 2.5" in code - assert " enabled: Bool = True" in code - - -def test_emit_explicit_pass_name_and_nopass_methods(): - source = """ -module pass_mod - type :: vector - contains - procedure, pass(owner) :: shift => shift_vector - procedure, nopass :: make => make_vector - end type vector -contains - subroutine shift_vector(dx, owner, dy) - real(8), intent(in) :: dx - class(vector), intent(inout) :: owner - real(8), intent(in) :: dy - end subroutine shift_vector - function make_vector(value) result(created) - real(8), intent(in) :: value - type(vector) :: created - end function make_vector -end module pass_mod -""" - - code = generate_pyi(source) - - assert " def shift(\n self,\n dx: Float64,\n dy: Float64" in code - assert " owner: Addr(vector)" not in code - assert "@native_call([Addr(Arg(0)), Pass(), Addr(Arg(1))])" in code - assert "owner: Annotated[vector, Polymorphic]" in code - assert ' @staticmethod\n @bind("make_vector")' in code - assert "value: Float64" in code - assert "-> vector: ..." in code - - def test_emit_and_load_module_and_type_bound_overload_sets(): source = (NATIVE_FIXTURES / "emit_and_load_module_and_type_bound_overload_sets.f90").read_text(encoding="utf-8") code = generate_pyi(source) @@ -291,45 +110,8 @@ def test_public_type_bound_generic_specifics_do_not_emit_bind(): ] -def test_emit_and_load_aliased_derived_module_variable_declaration(): - source = """ -module derived_module_state - type :: box - real(8), allocatable :: values(:) - end type box - type(box), target :: current -end module derived_module_state -""" - code = generate_pyi(source) - - assert "current: Annotated[box, Aliased]" in code - - loaded = parse_pyi_text(code, module_name="derived_module_state") - assert [variable.name for variable in loaded.variables] == ["current"] - assert loaded.variables[0].semantic_type.name == "box" - assert loaded.variables[0].semantic_type.metadata["aliased"] is True - - -def test_emit_module_stubs_print_plain_derived_module_variable_as_live_object(): - source = """ -module derived_module_snapshot - type :: box - real(8) :: value - end type box - type(box) :: current -end module derived_module_snapshot -""" - parsed = parse_fortran_source(source) - semantic_module = fortran_module_to_semantic_module(parsed) - - code = emit_module_stubs(semantic_module)["derived_module_snapshot"] - - assert "current: box" in code - loaded = parse_pyi_text(code, module_name="derived_module_snapshot") - assert loaded.variables[0].semantic_type.name == "box" - - def test_defined_operator_pyi_round_trip_preserves_native_links_without_fortran_source(): + """Operators and assignment survive the contract and still reach their native specifics.""" semantic_module = fortran_module_to_semantic_module( parse_fortran_source(OPERATOR_F90_SOURCE.read_text(), filename=str(OPERATOR_F90_SOURCE)) ) @@ -352,20 +134,7 @@ def test_defined_operator_pyi_round_trip_preserves_native_links_without_fortran_ loaded = parse_pyi_text(code, module_name=semantic_module.name) assert emit_module(loaded) == code - -def test_defined_operator_pyi_generates_wrapper_sources_without_fortran_source(): - semantic_module = fortran_module_to_semantic_module( - parse_fortran_source(OPERATOR_F90_SOURCE.read_text(), filename=str(OPERATOR_F90_SOURCE)) - ) - pyi = emit_module(semantic_module) - loaded = parse_pyi_text(pyi, module_name=semantic_module.name) generated = generate_wrapper(loaded) - - assert [path.name for path in generated.source_paths] == [ - "bind_c_foperators_f90_wrapper.f90", - "foperators_f90_wrapper.c", - "foperators_f90_wrapper.h", - ] fortran_wrapper = rendered_source(generated, ".f90") c_wrapper = rendered_source(generated, ".c") assert "left + right" in fortran_wrapper @@ -376,94 +145,6 @@ def test_defined_operator_pyi_generates_wrapper_sources_without_fortran_source() assert "def __ne__(self, *args, **kwargs):" in c_wrapper -def test_emit_module_variables_with_visibility(): - source = """ -module state_mod - implicit none - private - public :: counter - integer, parameter :: answer = 42 - integer :: counter - real(8) :: hidden_scale -contains - subroutine ping(x) - integer, intent(in) :: x - end subroutine -end module -""" - code = generate_pyi(source) - assert "answer:" not in code - assert "counter: Int32" in code - assert "hidden_scale" not in code - assert "ping" not in code - - -def test_emit_omits_fortran_source_private_methods_and_fields(): - source = (NATIVE_FIXTURES / "emit_omits_fortran_source_private_methods_and_fields.f90").read_text(encoding="utf-8") - - code = generate_pyi(source) - - assert "class box:" in code - assert " id: Int32" in code - assert "secret" not in code - assert "hidden" not in code - assert "hidden_impl" not in code - assert '@bind("visible_impl")' in code - assert " def visible(self) -> None: ..." in code - - -def test_printer_rejects_each_unresolved_semantic_type_field(): - printer = PyiPrinter() - message = "Cannot emit .pyi with unresolved semantic type 'Unknown'" - - with pytest.raises(ValueError) as unknown_name: - printer.emit(SemanticType("Unknown", dtype="Int32")) - with pytest.raises(ValueError) as unknown_dtype: - printer.emit(SemanticType("Int32", dtype="Unknown")) - - assert str(unknown_name.value) == message - assert str(unknown_dtype.value) == message - - -def test_printer_preserves_structured_class_and_decorator_layout(): - printer = PyiPrinter() - decorated_method = SemanticMethod( - name="lookup", - return_type=SemanticType("Float64"), - visibility="private", - projection=[ProjectionMapping(native_position=0, result_position=0)], - ) - cls = SemanticClass( - name="thing", - base_classes=["Opaque", "Protocol"], - fields=[SemanticArgument("value", SemanticType("Int32"))], - methods=[decorated_method, SemanticMethod(name="reset")], - ) - decorated_function = SemanticFunction( - name="wrapper", - visibility="private", - projection=[ProjectionMapping(native_position=0, result_position=0)], - ) - - assert ( - printer.emit(cls) - == """class thing(Opaque, Protocol): - value: Int32 - - @private - @native_call([Return(0)]) - def lookup(self) -> Float64: ... - - def reset(self) -> None: ...""" - ) - assert ( - printer.emit(decorated_function) - == """@private -@native_call([Return(0)]) -def wrapper() -> None: ...""" - ) - - def test_generic_specifics_with_projected_outputs_round_trip(): """A generic whose specifics project an `intent(out)` reloads from its contract. diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py index bd5ef3be4..86d0b32ea 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py @@ -17,19 +17,6 @@ CHECKED_CONTRACTS = sorted(CONTRACT_FIXTURES.rglob("*.pyi")) -def test_checked_contract_package_has_reviewed_files(): - assert [str(path.relative_to(CONTRACT_FIXTURES)) for path in CHECKED_CONTRACTS] == [ - "contract_import_graph/generated/__init__.pyi", - "contract_import_graph/generated/deep.pyi", - "contract_import_graph/generated/m1.pyi", - "contract_mixed_module_external/generated/__init__.pyi", - "contract_mixed_module_external/generated/contract_math_mod.pyi", - "contract_same_name/generated/__init__.pyi", - "contract_same_name/generated/contract_same_name.pyi", - "contract_standalone_only/generated/__init__.pyi", - ] - - @pytest.mark.parametrize( "fixture", CHECKED_CONTRACTS, diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py index df755ee5a..682614a8a 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py @@ -6,6 +6,8 @@ import sys from pathlib import Path +import pytest + from tests.fortran._support.pyi_fixtures import assert_generated_pyi_package_matches_fixture FIXTURES = Path(__file__).parent / "fixtures" @@ -18,88 +20,22 @@ MULTI_MODULE = NATIVE_FIXTURES / "contract_multi_module.f90" -def _generate_contract_package(source: Path, package: Path) -> Path: +@pytest.mark.parametrize( + "source", + [STANDALONE_ONLY, SOURCE_NAMESPACE, SAME_NAME_MIXED, TRANSITIVE_NATIVE], + ids=lambda path: path.stem, +) +def test_generated_contract_package_matches_reviewed_layout(source: Path, tmp_path: Path): + """The CLI writes an explicit `__init__.pyi` entry plus one leaf per native module, as reviewed.""" + package = tmp_path / "contracts" / source.stem subprocess.run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--pyi", - str(source), - "--out", - str(package), - ], + [sys.executable, "-m", "prik", "generate", "--pyi", str(source), "--out", str(package)], capture_output=True, text=True, check=True, ) - assert_generated_pyi_package_matches_fixture( - package, - CONTRACT_FIXTURES / source.stem / "generated", - ) - return package / "__init__.pyi" - - -def test_standalone_generation_writes_explicit_package_entry(tmp_path: Path): - entry = _generate_contract_package( - STANDALONE_ONLY, - tmp_path / "contracts" / "contract_standalone_only", - ) - - assert entry == tmp_path / "contracts" / "contract_standalone_only" / "__init__.pyi" - assert {path.name for path in entry.parent.iterdir()} == {"__init__.pyi"} - text = entry.read_text(encoding="utf-8") - assert text.count("@standalone") == 2 - assert "def standalone_ping() -> None: ..." in text - assert "def standalone_double(" in text - - -def test_module_generation_writes_explicit_package_entry_and_native_leaf(tmp_path: Path): - entry = _generate_contract_package( - SOURCE_NAMESPACE, - tmp_path / "contracts" / "contract_mixed_module_external", - ) - - assert entry == tmp_path / "contracts" / "contract_mixed_module_external" / "__init__.pyi" - assert {path.name for path in entry.parent.iterdir()} == { - "__init__.pyi", - "contract_math_mod.pyi", - } - assert entry.read_text(encoding="utf-8").startswith( - "from prik.contracts import Addr, Arg, Int32, native_call, standalone\n" - "from . import contract_math_mod\n\n" - "@standalone\n" - ) - - -def test_same_named_module_uses_init_entry_and_keeps_externals_at_root(tmp_path: Path): - entry = _generate_contract_package( - SAME_NAME_MIXED, - tmp_path / "contracts" / "contract_same_name", - ) - - assert entry == tmp_path / "contracts" / "contract_same_name" / "__init__.pyi" - assert {path.name for path in entry.parent.iterdir()} == {"__init__.pyi", "contract_same_name.pyi"} - assert entry.read_text(encoding="utf-8") == ( - "from prik.contracts import standalone\n" - "from . import contract_same_name\n\n" - "@standalone\n" - "def external_ping() -> None: ...\n\n" - '__all__ = ["contract_same_name", "external_ping"]\n' - ) - assert "def module_ping() -> None: ..." in (entry.parent / "contract_same_name.pyi").read_text(encoding="utf-8") - - -def test_import_graph_generation_writes_entry_and_native_leaves(tmp_path: Path): - entry = _generate_contract_package( - TRANSITIVE_NATIVE, - tmp_path / "contracts" / "contract_import_graph", - ) - assert entry == tmp_path / "contracts" / "contract_import_graph" / "__init__.pyi" - assert {path.name for path in entry.parent.iterdir()} == {"__init__.pyi", "deep.pyi", "m1.pyi"} - assert entry.read_text(encoding="utf-8") == ('from . import m1\nfrom . import deep\n\n__all__ = ["m1", "deep"]\n') + assert_generated_pyi_package_matches_fixture(package, CONTRACT_FIXTURES / source.stem / "generated") def test_multi_module_generation_keeps_each_native_namespace(tmp_path: Path): diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_modern_example.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_modern_example.py index b59d2214d..4da202c25 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_modern_example.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_modern_example.py @@ -18,15 +18,3 @@ def test_modern_fortran_example_pyi_snapshot(): pyi = "\n\n".join(emit_module(m) for m in modules).strip() assert pyi == expected_fixture.read_text(encoding="utf-8").strip() - - -def test_pyi_visibility_private_public_markers(): - source = (NATIVE_FIXTURES / "pyi_visibility_private_public_markers.f90").read_text(encoding="utf-8") - parsed = parse_fortran_file(source, filename="visibility_mod.f90") - pyi = emit_module(fortran_module_to_semantic_module(parsed.modules[0])).strip() - - assert "a: Int32" in pyi - assert "b: Int32" in pyi - assert "class hidden_t:" not in pyi - assert "def pub_proc(" in pyi - assert "def hidden_proc(" not in pyi diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_printer_diagnostics.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_printer_diagnostics.py new file mode 100644 index 000000000..992444f21 --- /dev/null +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_printer_diagnostics.py @@ -0,0 +1,88 @@ +"""The `.pyi` printer refuses semantic IR it cannot write as a loadable contract.""" + +import pytest +from prik.printers import PyiPrinter, emit_module +from prik.semantics.models import ( + ProjectionMapping, + SemanticArrayContract, + SemanticFunction, + SemanticModule, + SemanticStorageContract, + SemanticType, + SemanticVariable, +) + + +def test_printer_rejects_each_unresolved_semantic_type_field(): + printer = PyiPrinter() + message = "Cannot emit .pyi with unresolved semantic type 'Unknown'" + + with pytest.raises(ValueError) as unknown_name: + printer.emit(SemanticType("Unknown", dtype="Int32")) + with pytest.raises(ValueError) as unknown_dtype: + printer.emit(SemanticType("Int32", dtype="Unknown")) + + assert str(unknown_name.value) == message + assert str(unknown_dtype.value) == message + + +@pytest.mark.parametrize( + "projection, message", + [ + ( + [ProjectionMapping(native_position=0)], + "native-only projection entry", + ), + ( + [ProjectionMapping(native_position=0, value_kind="unknown", value=None)], + "Unsupported native_call projection entry", + ), + ( + [ + ProjectionMapping( + native_position=0, + value_kind="len", + value={"kind": "unknown"}, + ) + ], + "Unsupported native_call value reference", + ), + ], + ids=["native-only-entry", "unknown-entry-kind", "unknown-value-reference"], +) +def test_emit_native_call_rejects_unrepresentable_projection_entries(projection, message): + module = SemanticModule( + name="bad_projection_mod", + functions=[ + SemanticFunction( + name="wrapper", + native_name="wrapper", + projection=projection, + ) + ], + ) + + with pytest.raises(ValueError, match=message): + emit_module(module) + + +def test_printer_rejects_optional_absent_array_handles_outside_callable_arguments(): + variable = SemanticVariable( + "values", + SemanticType( + "Float64", + rank=1, + storage=SemanticStorageContract( + kind="array", + array=SemanticArrayContract(rank=1, shape=[":"], allocatable=True), + ), + ), + ) + variable.optional = True + module = SemanticModule( + name="bad_optional_handle_mod", + variables=[variable], + ) + + with pytest.raises(ValueError, match="only be emitted for callable arguments"): + emit_module(module) diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_conversion_smoke.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_conversion_smoke.py index d41226138..088d65973 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_conversion_smoke.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_conversion_smoke.py @@ -1,18 +1,16 @@ +"""Every parser fixture emits a contract that reloads and prints back unchanged.""" + from pathlib import Path import pytest -from prik.semantics.fortran2ir import fortran_module_to_semantic_module +from prik.pipeline.pyi import pyi_text_to_semantic_module from prik.policy.exports import complete_python_export_policy from prik.printers import emit_module - +from prik.semantics.fortran2ir import fortran_module_to_semantic_module +from tests.fortran._support.fixture_conversion import FORTRAN_FIXTURES from tests.fortran._support.fixture_outputs import parse_fixture from tests.fortran._support.paths import PARSER_FIXTURE_ROOT as TESTS_DIR -from tests.fortran._support.fixture_conversion import FORTRAN_FIXTURES - - -def test_pyi_printer_fixture_suite_has_fixtures(): - assert FORTRAN_FIXTURES, "No final general Fortran parser fixtures found" @pytest.mark.parametrize( @@ -26,4 +24,5 @@ def test_pyi_printer_conversion_smoke(fixture: Path): for module in parsed.modules: semantic_module = fortran_module_to_semantic_module(module) complete_python_export_policy(semantic_module) - emit_module(semantic_module) + code = emit_module(semantic_module) + assert emit_module(pyi_text_to_semantic_module(code, module_name=semantic_module.name)) == code diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py index 1615d81ab..7c8f761ca 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py @@ -1,4 +1,4 @@ -"""Tests split by stable ownership concept from `test_imports_and_packages.py`.""" +"""Generated contract imports, packages, public-name spelling, and `__all__`.""" import json import pytest @@ -7,7 +7,6 @@ from prik.policy.contract_imports import complete_contract_imports from prik.policy.exports import contract_name_for_source from prik.printers import ( - PyiPrinter, emit_module, ) from prik.pipeline.pyi import ( @@ -36,26 +35,6 @@ from tests.fortran._support.printer_models import generate_pyi -def test_pyi_pipeline_exports_module_stub_emitter(): - assert "emit_module_stubs" in pyi_pipeline.__all__ - assert pyi_pipeline.emit_module_stubs is emit_module_stubs - - -def test_generated_pyi_separates_top_level_functions_with_a_blank_line(): - int_type = SemanticType("Int") - code = emit_module( - SemanticModule( - name="readable", - functions=[ - SemanticFunction("first", return_type=int_type), - SemanticFunction("second", return_type=int_type), - ], - ) - ) - - assert "def first() -> Int: ...\n\ndef second() -> Int: ..." in code - - def test_fortran_generated_contracts_reserve_colliding_public_names_by_namespace(): int32_type = SemanticType("Int32") origin = SemanticOrigin(source_language="fortran", native_scope="naming_mod") @@ -88,22 +67,6 @@ def test_fortran_generated_contracts_reserve_colliding_public_names_by_namespace assert "def lambda__3" not in code -def test_pyi_emission_context_isolates_modules_and_shares_nested_imports(): - printer = PyiPrinter(normalize_public_names=True) - first = printer._emission_context(SemanticModule(name="first")) - second = printer._emission_context(SemanticModule(name="second")) - nested = first.inside_class("record_t") - - first.contract("Addr") - nested.contract("Pointer") - - assert first.contract_import() == "from prik.contracts import Addr, Pointer" - assert nested.contract_import() == first.contract_import() - assert nested.public_namespace == ("record_t",) - assert first.public_namespace == () - assert second.contract_import() == "" - - def test_printing_loaded_contract_preserves_absolute_support_imports(): module = _parse_pyi_text( "from typing import Any\nfrom prik.contracts import Int32\n\ndef identity(value: Int32) -> Int32: ...\n", @@ -113,16 +76,7 @@ def test_printing_loaded_contract_preserves_absolute_support_imports(): assert "from typing import Any" in emit_module(module) -def test_printer_validation_and_opaque_dependency_edge_cases(): - printer = PyiPrinter() - - with pytest.raises(ValueError, match="Shape constraints are not canonical"): - printer.emit(SemanticConstraint("Shape")) - - plain_type = SemanticType("Float64", dtype="Float64") - context = printer._emission_context(SemanticModule(name="edge_cases")) - assert printer._emit_storage_type(plain_type, context) == "Float64" - +def test_contract_imports_skip_malformed_refs_and_stub_emission_rejects_duplicate_modules(): malformed_import = SemanticType( "external_type", dtype="external_type", @@ -139,42 +93,6 @@ def test_printer_validation_and_opaque_dependency_edge_cases(): complete_contract_imports([malformed_module]) assert malformed_module.imports == [] - invalid_opaque_ref = SemanticType( - "external_type", - dtype="external_type", - metadata={ - "external_type_ref": { - "representation": "opaque", - "origin_module": "types", - "name": 42, - } - }, - ) - known_opaque_ref = SemanticType( - "external_type", - dtype="external_type", - metadata={ - "external_type_ref": { - "representation": "opaque", - "origin_module": "types", - "name": "external_type", - } - }, - ) - assert ( - opaque_dependency_modules( - SemanticModule( - name="api", - variables=[ - SemanticArgument("invalid", invalid_opaque_ref), - SemanticArgument("known", known_opaque_ref), - ], - ), - available_modules=[SemanticModule(name="types", classes=[SemanticClass(name="external_type")])], - ) - == [] - ) - with pytest.raises(ValueError, match="duplicate semantic module"): emit_module_stubs([SemanticModule(name="duplicate"), SemanticModule(name="duplicate")]) @@ -241,28 +159,11 @@ def test_opaque_dependency_modules_scan_all_references_and_preserve_metadata(): ], ) ] - - -def test_emit_module_stubs_honors_available_opaque_dependency_modules(): - known_opaque_ref = SemanticType( - "known_type", - dtype="known_type", - metadata={ - "external_type_ref": { - "representation": "opaque", - "origin_module": "types", - "name": "known_type", - } - }, - ) + # A dependency the caller already has adds no opaque stub module. stubs = emit_module_stubs( - SemanticModule( - name="api", - variables=[SemanticArgument("known", known_opaque_ref)], - ), + SemanticModule(name="api", variables=[SemanticArgument("known", known_opaque_ref)]), available_modules=[SemanticModule(name="types", classes=[SemanticClass(name="known_type")])], ) - assert set(stubs) == {"api"} @@ -712,36 +613,19 @@ def test_non_fortran_declaration_compares_its_native_spelling_exactly(): assert '@bind("ScaleValue")' in code -def test_generated_contract_binds_a_class_whose_python_name_renames_its_type(): - """A renamed class states its native type so the contract reads back.""" - origin = SemanticOrigin(source_language="fortran", native_scope="shapes_mod") - module = SemanticModule( - name="shapes_mod", - classes=[ - SemanticClass( - name="PointType", - native_name="POINT_T", - fields=[SemanticField("x", SemanticType("Float64"))], - origin=origin, - ) - ], - origin=origin, - ) - complete_python_export_policy(module) - - code = emit_module(module, normalize_public_names=True) - - assert '@bind("POINT_T")\nclass Pointtype:' in code - - -def test_generated_contract_omits_a_class_bind_for_a_case_only_python_name(): - """A class named without regard to case states no separate native type.""" +@pytest.mark.parametrize( + ("python_name", "expected", "binds"), + [("PointType", '@bind("POINT_T")\nclass Pointtype:', True), ("point_t", "class Point_T:", False)], + ids=["renamed-type-binds", "case-only-name-binds-nothing"], +) +def test_generated_contract_binds_a_class_only_when_its_python_name_renames_its_type(python_name, expected, binds): + """A renamed class states its native type so the contract reads back; a case-only rename does not.""" origin = SemanticOrigin(source_language="fortran", native_scope="shapes_mod") module = SemanticModule( name="shapes_mod", classes=[ SemanticClass( - name="point_t", + name=python_name, native_name="POINT_T", fields=[SemanticField("x", SemanticType("Float64"))], origin=origin, @@ -753,8 +637,8 @@ def test_generated_contract_omits_a_class_bind_for_a_case_only_python_name(): code = emit_module(module, normalize_public_names=True) - assert "class Point_T:" in code - assert "@bind(" not in code + assert expected in code + assert ("@bind(" in code) is binds def test_prototype_spelling_is_kept_only_for_the_module_that_declares_one(): diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_types_and_declarations.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_types_and_declarations.py index ac7a0ddff..71da8f35f 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_types_and_declarations.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_types_and_declarations.py @@ -1,529 +1,84 @@ -"""Tests split by stable ownership concept from `test_imports_and_packages.py`.""" +"""Fortran declarations emit their `.pyi` spelling, and that contract reloads unchanged.""" from pathlib import Path import pytest from prik.parsers.fortran import parse_fortran_file as parse_fortran_source +from prik.policy.contract_imports import complete_contract_imports from prik.policy.exports import complete_python_export_policy -from prik.printers import ( - PyiPrinter, - emit_module, -) +from prik.printers import emit_module from prik.semantics.fortran2ir import fortran_module_to_semantic_module -from prik.semantics.models import ( - ProjectionMapping, - SemanticArgument, - SemanticArrayContract, - SemanticClass, - SemanticConstraint, - SemanticFunction, - SemanticMethod, - SemanticModule, - SemanticOrigin, - SemanticStorageContract, - SemanticType, -) -from tests.fortran._support.printer_models import ( - generate_pyi, - parse_pyi_text, -) +from tests.fortran._support.printer_models import parse_pyi_text NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" - -def test_emit_basic_scalar_function(): - source = """ -module math_mod - +SCALAR_ARGUMENTS_AND_RESULTS = """ +module scalars contains - subroutine add(a, b, c) - real(8), intent(in) :: a real(8), intent(in) :: b real(8), intent(out) :: c - end subroutine - -end module -""" - - code = generate_pyi(source) - - assert "def add(" in code - - assert "@native_call([Addr(Arg(0)), Addr(Arg(1)), Return('c', 0)])" in code - assert "a: Float64" in code - assert "b: Float64" in code - assert "c: Addr(Float64)" not in code - assert 'Returns["c"' not in code - assert ") -> Float64: ..." in code - - -def test_fortran_generated_contracts_emit_python_name_without_binding_the_same_name(): - """A capitalized Fortran procedure is written lower case and binds nothing. - - Fortran reaches a procedure without regard to case, so the lower-case - Python name already names it and no original spelling has to be recorded. - """ - module = SemanticModule( - name="math_mod", - functions=[ - SemanticFunction( - "SQUARE_R4", - native_name="SQUARE_R4", - arguments=[SemanticArgument("X", SemanticType("Float32"))], - return_type=SemanticType("Float32"), - origin=SemanticOrigin(source_language="fortran", native_name="SQUARE_R4", native_scope="math_mod"), - ) - ], - origin=SemanticOrigin(source_language="fortran", source_kind="module"), - ) - complete_python_export_policy(module) - - code = emit_module(module, normalize_public_names=True) - - assert "def square_r4(" in code - assert "@bind(" not in code - - -def test_emit_rejects_unknown_semantic_type(): - module = SemanticModule( - name="bad", - variables=[ - SemanticArgument( - name="x", - semantic_type=SemanticType("Unknown", dtype="Unknown"), - ) - ], - ) - - with pytest.raises(ValueError, match="unresolved semantic type 'Unknown'"): - emit_module(module) - - -def test_emit_no_argument_subroutine_is_single_line_signature(): - source = """ -module no_arg_mod - -contains - subroutine ping() end subroutine - -end module -""" - - code = generate_pyi(source) - - assert "def ping() -> None: ..." in code - assert "def ping(\n \n)" not in code - - -def test_emit_array_constraints(): - source = """ -module array_mod - -contains - -subroutine scale(x) - - real(8), intent(inout) :: x(:) - +function norm2(x) result(r) + real(8), intent(in) :: x(:) + real(8) :: r +end function +subroutine solve(tol) + real(8), intent(in), optional :: tol end subroutine - +subroutine maybe_status(status) + integer(4), intent(out), optional :: status +end subroutine maybe_status +subroutine normalize(name) + character(len=8), intent(inout) :: name +end subroutine +subroutine scale_in_place(value, factor) + real(8), intent(inout) :: value + real(8), intent(in) :: factor +end subroutine scale_in_place end module """ - code = generate_pyi(source) - - assert "def scale(" in code - - assert "Float64[" in code - - assert "Shape" not in code - assert "Float64[::]" in code - - -def test_emit_matrix_shapes(): - source = """ -module matrix_mod - +ARRAY_SHAPES = """ +module arrays contains - subroutine matvec(A, x, y) - real(8), intent(in) :: A(:, :) real(8), intent(in) :: x(:) real(8), intent(out) :: y(:) - end subroutine - -end module -""" - - code = generate_pyi(source) - - assert "A: Float64[::, ::]" in code - assert "Shape" not in code - assert "x: Float64[::]" in code - assert "y: Float64[::]" in code - assert "Annotated[Float64[::]" not in code - assert "y: Float64[::]\n) -> None: ..." in code - - -def test_emit_explicit_bound_ranges_as_extents_without_source_dimension_metadata(): - source = """ -module bound_mod -contains -subroutine bounded(n, default_bound, zero_bound, shifted_bound) +subroutine scale(x) + real(8), intent(inout) :: x(:) +end subroutine +subroutine explicit(n, fixed, default_bound, zero_bound, shifted_bound) integer, intent(in) :: n + real(8), intent(in) :: fixed(10, 20) real(8), intent(inout) :: default_bound(1:n) real(8), intent(inout) :: zero_bound(0:n-1) real(8), intent(inout) :: shifted_bound(2:n+1) -end subroutine bounded -end module bound_mod -""" - code = generate_pyi(source) - - assert "default_bound: Float64[n]" in code - assert "zero_bound: Float64[n]" in code - assert "shifted_bound: Float64[n]" in code - - -def test_emit_optional_argument(): - source = """ -module opt_mod - -contains - -subroutine solve(A, tol) - - real(8), intent(in) :: A(:, :) - real(8), intent(in), optional :: tol - -end subroutine - -end module -""" - - code = generate_pyi(source) - - assert "tol:" in code - - assert "= ..." in code - - -def test_emit_function_result(): - source = """ -module func_mod - -contains - -function norm2(x) result(r) - - real(8), intent(in) :: x(:) - - real(8) :: r - -end function - +end subroutine explicit +subroutine assumed_size(ldb, columns, flat, bounded) + integer, intent(in) :: ldb + real(8), intent(inout) :: columns(3, *) + real(8), intent(inout) :: flat(0:*) + real(8), intent(inout) :: bounded(0:ldb-1, 0:*) +end subroutine assumed_size +subroutine use_labels(labels) + character(len=4), intent(in) :: labels(:) +end subroutine use_labels +subroutine replace_names(names) + character(len=:), allocatable, intent(inout) :: names(:) +end subroutine replace_names end module """ - code = generate_pyi(source) - - assert "def norm2(" in code - - assert "-> Float64" in code - - -def test_emit_explicit_shape(): - source = """ -module shape_mod - -contains - -subroutine foo(A) - - real(8), intent(in) :: A(10, 20) - -end subroutine - -end module -""" - - code = generate_pyi(source) - - assert "A: Float64[10, 20]" in code - - -def test_parameter_target_sanitizes_non_identifier_names(): - assert PyiPrinter._parameter_target("has-dash") == "has_dash" - assert PyiPrinter._parameter_target("1value") == "arg_1value" - assert PyiPrinter._parameter_target("class!") == "class_" - assert PyiPrinter._parameter_target("!!!") == "arg" - assert PyiPrinter._requires_explicit_projection_mapping(ProjectionMapping(native_position=1, result_position=0)) - - -def test_emit_argument_escapes_original_name_metadata(): - emitted = PyiPrinter().emit(SemanticArgument('quote"name', SemanticType("Int32"))) - reparsed = parse_pyi_text(f"def consume({emitted}) -> None: ...\n", module_name="quoted") - - assert emitted == 'quote_name: Annotated[Int32, SourceName("quote\\"name")]' - assert reparsed.functions[0].arguments[0].name == 'quote"name' - - -def test_emit_multiple_functions(): - source = """ -module multi_mod - -contains - -subroutine foo(x) - - integer, intent(in) :: x - -end subroutine - -subroutine bar(y) - - real(8), intent(in) :: y - -end subroutine - -end module -""" - - code = generate_pyi(source) - - assert "def foo(" in code - - assert "def bar(" in code - - -def test_emit_complex_fem_module(): - source = (NATIVE_FIXTURES / "emit_complex_fem_module.f90").read_text(encoding="utf-8") - - code = generate_pyi(source) - - # -------------------------------------------------------- - # Class - # -------------------------------------------------------- - - assert "class mesh" in code - - # -------------------------------------------------------- - # Procedures - # -------------------------------------------------------- - - assert "def assemble(" in code - - assert "def compute_norm(" in code - - # -------------------------------------------------------- - # Matrix annotations - # -------------------------------------------------------- - - assert "K: Float64[::, ::]" in code - assert "connectivity: Int32[::, ::]\n) -> None: ..." in code - - assert "coords: Float64[::, ::]" in code - - assert "connectivity: Int32[::, ::]" in code - - # -------------------------------------------------------- - # Return type - # -------------------------------------------------------- - - assert "def compute_norm(" in code - assert ") -> Float64: ..." in code - - -def test_emit_empty_module(): - source = """ -module empty_mod -end module -""" - - code = generate_pyi(source) - - assert isinstance(code, str) - - -def test_emit_is_deterministic(): - source = """ -module stable_mod - -contains - -subroutine foo(x) - - integer, intent(in) :: x - -end subroutine - -end module -""" - - code1 = generate_pyi(source) - - code2 = generate_pyi(source) - - assert code1 == code2 - - -def test_printer_emit_visitor_dispatches_semantic_models(): - printer = PyiPrinter() - constraint = SemanticConstraint("Finite") - semantic_type = SemanticType( - "Float64", - dtype="Float64", - rank=1, - shape=[":"], - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract(rank=1, shape=[":"], source_shape=[":"]), - ), - ) - argument = SemanticArgument("class", semantic_type, optional=True) - method = SemanticMethod(name="reset") - cls = SemanticClass( - name="thing", - fields=[SemanticArgument("bad-name", semantic_type)], - methods=[method], - visibility="private", - ) - func = SemanticFunction(name="wrap", arguments=[argument]) - module = SemanticModule(name="visitor_mod", classes=[cls], functions=[func]) - - assert printer.emit(constraint) == "Finite" - assert printer.emit(semantic_type) == "Float64[:]" - assert printer.emit(argument) == 'class_: Annotated[Float64[:], SourceName("class")] = ...' - assert "def reset(self) -> None: ..." in printer.emit(method) - assert "@private\nclass thing:" in printer.emit(cls) - assert "var['bad-name']: Float64[:]" in printer.emit(cls) - assert "def wrap(" in printer.emit(func) - assert "class thing:" in printer.emit(module) - - with pytest.raises(TypeError) as unsupported: - printer.emit(object()) - assert str(unsupported.value) == "Unsupported semantic model for .pyi emission: " - - -def test_printer_emits_flat_dimension_for_assumed_size_arrays(): - fortran_type = SemanticType( - "Float64", - dtype="Float64", - rank=2, - shape=["3", ":"], - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract( - rank=2, - shape=["3", ":"], - category="assumed_size", - source_shape=["3", "*"], - order="ORDER_F", - contiguous=True, - ), - ), - ) - c_type = SemanticType( - "Float64", - dtype="Float64", - rank=2, - shape=[":", "3"], - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract( - rank=2, - shape=[":", "3"], - category="assumed_size", - source_shape=["*", "3"], - order="ORDER_C", - contiguous=True, - ), - ), - ) - - assert PyiPrinter().emit(fortran_type) == "Float64[3, Flat]" - assert PyiPrinter().emit(c_type) == "Annotated[Float64[Flat, 3], ORDER_C]" - - nondefault_c_type = SemanticType( - "Float64", - dtype="Float64", - rank=2, - shape=[":", ":"], - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract( - rank=2, - shape=[":", ":"], - source_shape=[":", ":"], - order="ORDER_C", - contiguous=True, - ), - ), - ) - assert PyiPrinter().emit(nondefault_c_type) == "Annotated[Float64[:, :], ORDER_C]" - - lower_bound_assumed_size = SemanticType( - "Float64", - dtype="Float64", - rank=1, - shape=[":"], - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract( - rank=1, - shape=[":"], - category="assumed_size", - source_shape=["0:*"], - order="ORDER_F", - contiguous=True, - ), - ), - ) - assert PyiPrinter().emit(lower_bound_assumed_size) == "Float64[Flat]" - - bounded_assumed_size = SemanticType( - "Float64", - dtype="Float64", - rank=2, - shape=["LDB", ":"], - storage=SemanticStorageContract( - kind="array", - array=SemanticArrayContract( - rank=2, - shape=["LDB", ":"], - category="assumed_size", - source_shape=["0:LDB-1", "0:*"], - order="ORDER_F", - contiguous=True, - ), - ), - ) - assert PyiPrinter().emit(bounded_assumed_size) == "Float64[LDB, Flat]" - - -def test_emit_fortran_parameter_defaults_only_when_resolved_to_literals(): - source = """ -module trig_constants +PARAMETERS_AND_VALUE_DUMMIES = """ +module value_contract real, parameter :: c = cos(0.0) integer, parameter :: n = 3 + 4 -end module -""" - code = generate_pyi(source) - - assert "c: Final[Float32]\n" in code - assert "c: Final[Float32] = cos(0.0)" not in code - assert "n: Final[Int32] = 7" in code - - -def test_fortran_derived_value_dummy_emits_value_native_projection(): - source = """ -module value_contract type :: item real(8) :: x end type item @@ -536,35 +91,154 @@ def test_fortran_derived_value_dummy_emits_value_native_projection(): end module value_contract """ - code = generate_pyi(source) - - assert "@native_call([Value(Arg(0))])" in code - assert "value: item" in code - +DERIVED_TYPES_AND_METHODS = """ +module shapes + type :: base_matrix + end type + type, extends(base_matrix) :: sparse_matrix + integer :: nrows + end type + type :: state + integer :: id = 7 + real(8) :: scale = 2.5 + logical :: enabled = .true. + end type state + type :: vector + real(8), allocatable :: values(:) + contains + procedure :: scale + procedure, pass(owner) :: shift => shift_vector + procedure, nopass :: make => make_vector + end type vector + type(vector), target :: current + type(state) :: snapshot +contains + subroutine scale(self, alpha) + class(vector), intent(inout) :: self + real(8), intent(in) :: alpha + end subroutine scale + subroutine shift_vector(dx, owner, dy) + real(8), intent(in) :: dx + class(vector), intent(inout) :: owner + real(8), intent(in) :: dy + end subroutine shift_vector + function make_vector(value) result(created) + real(8), intent(in) :: value + type(vector) :: created + end function make_vector +end module shapes +""" -def test_character_array_pyi_spelling_round_trips_fixed_and_deferred_lengths(): - source = """ -module char_array_mod +DEFAULT_PRIVATE_MODULE = """ +module state_mod + implicit none + private + public :: counter + integer, parameter :: answer = 42 + integer :: counter + real(8) :: hidden_scale contains - subroutine use_labels(labels) - character(len=4), intent(in) :: labels(:) - end subroutine use_labels - subroutine replace_names(names) - character(len=:), allocatable, intent(inout) :: names(:) - end subroutine replace_names -end module char_array_mod + subroutine ping(x) + integer, intent(in) :: x + end subroutine +end module """ - semantic_module = fortran_module_to_semantic_module(parse_fortran_source(source)) - emitted = emit_module(semantic_module) - assert "String[4][::]" in emitted - assert "Allocatable[String[:][:]]" in emitted - parsed = parse_pyi_text(emitted, module_name="char_array_mod") - use_labels = next(func for func in parsed.functions if func.name == "use_labels") - assert use_labels.arguments[0].semantic_type.metadata["fortran_character_length"] == "4" +def _fixture(name: str) -> str: + return (NATIVE_FIXTURES / name).read_text(encoding="utf-8") + + +DECLARATION_CASES = { + "scalar-arguments-and-results": ( + SCALAR_ARGUMENTS_AND_RESULTS, + [ + "@native_call([Addr(Arg(0)), Addr(Arg(1)), Return('c', 0)])\n" + "def add(\n a: Float64,\n b: Float64\n) -> Float64: ...", + "def ping() -> None: ...", + "def norm2(\n x: Float64[::]\n) -> Float64: ...", + "tol: Float64 = ...", + 'status: Int32[()] = ...\n) -> Returns["status", Int32[()]] | None: ...', + 'def normalize(\n name: String[8]\n) -> Returns["name", String[8]]: ...', + "@native_call([Addr(Arg(0)), Addr(Arg(1))])\ndef scale_in_place(\n" + ' value: Float64,\n factor: Float64\n) -> Returns["value", Float64]: ...', + ], + ['Returns["c"', "c: Addr(Float64)", "Return('status'"], + ), + "array-shapes": ( + ARRAY_SHAPES, + [ + "def matvec(\n A: Float64[::, ::],\n x: Float64[::],\n y: Float64[::]\n) -> None: ...", + "def scale(\n x: Float64[::]\n) -> None: ...", + "fixed: Float64[10, 20]", + "default_bound: Float64[n]", + "zero_bound: Float64[n]", + "shifted_bound: Float64[n]", + "columns: Float64[3, Flat]", + "flat: Float64[Flat]", + "bounded: Float64[ldb, Flat]", + "labels: String[4][::]", + "names: Allocatable[String[:][:]]", + ], + ["Shape", "Annotated[Float64"], + ), + "parameters-and-value-dummies": ( + PARAMETERS_AND_VALUE_DUMMIES, + [ + "c: Final[Float32]\n", + "n: Final[Int32] = 7", + "@native_call([Value(Arg(0))])\ndef score(\n value: item\n) -> Float64: ...", + ], + ["cos(0.0)"], + ), + "derived-types-and-methods": ( + DERIVED_TYPES_AND_METHODS, + [ + "class sparse_matrix(base_matrix):", + "class state:\n def __init__(\n self,\n *,\n id: Int32 = 7,\n" + " scale: Float64 = 2.5,\n enabled: Bool = True\n ) -> None: ...\n\n" + " id: Int32 = 7\n scale: Float64 = 2.5\n enabled: Bool = True\n", + " values: Allocatable[Float64[:]]", + " @native_call([Pass(), Addr(Arg(0))])\n def scale(\n self,\n alpha: Float64\n" + " ) -> None: ...", + ' @bind("shift_vector")\n @native_call([Addr(Arg(0)), Pass(), Addr(Arg(1))])\n def shift(\n' + " self,\n dx: Float64,\n dy: Float64\n ) -> None: ...", + ' @staticmethod\n @bind("make_vector")', + "owner: Annotated[vector, Polymorphic]", + "current: Annotated[vector, Aliased]", + "snapshot: state\n", + ], + [" self: vector", "owner: Addr(vector)"], + ), + "default-private-module": ( + DEFAULT_PRIVATE_MODULE, + ["counter: Int32"], + ["answer", "hidden_scale", "ping"], + ), + "public-private-markers": ( + _fixture("pyi_visibility_private_public_markers.f90"), + ["a: Int32", "b: Int32", "def pub_proc("], + ["class hidden_t:", "def hidden_proc("], + ), + "private-type-members": ( + _fixture("emit_omits_fortran_source_private_methods_and_fields.f90"), + ["class box:", " id: Int32", '@bind("visible_impl")', " def visible(self) -> None: ..."], + ["secret", "hidden"], + ), +} + + +@pytest.mark.parametrize( + ("source", "expected", "absent"), + list(DECLARATION_CASES.values()), + ids=list(DECLARATION_CASES), +) +def test_fortran_declarations_emit_contract_spelling_that_reloads(source: str, expected: list, absent: list): + module = fortran_module_to_semantic_module(parse_fortran_source(source, filename="declarations.f90")) + complete_python_export_policy(module) + complete_contract_imports([module]) + code = emit_module(module) - replace_names = next(func for func in parsed.functions if func.name == "replace_names") - names_type = replace_names.arguments[0].semantic_type - assert names_type.metadata["fortran_character_length"] == ":" - assert names_type.storage.array.allocatable is True + assert [fragment for fragment in expected if fragment not in code] == [] + assert [fragment for fragment in absent if fragment in code] == [] + assert emit_module(parse_pyi_text(code, module_name=module.name)) == code diff --git a/tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py b/tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py index 74b3b17fa..f10af6ef2 100644 --- a/tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py +++ b/tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py @@ -1,58 +1,16 @@ -"""Tests split by stable ownership concept from `test_python_ast_contracts.py`.""" +"""Semantic meaning of `@native_call` projections, returns, and callable shapes.""" -import ast import pytest -from dataclasses import asdict -from prik.parsers.fortran import parse_fortran_file from prik.printers import emit_module -from prik.contracts import CONTRACT_SYMBOLS -from prik.semantics.fortran2ir import fortran_file_to_semantic_modules from prik.semantics.metadata import ( ADDRESS_ROLE_METADATA, ADDRESS_ROLE_PROJECTION, PROJECTED_OUTPUT_METADATA, ) -from prik.semantics.models import ( - ProjectionMapping, - SemanticArgument, - SemanticConstraint, - SemanticFunction, - SemanticModule, - SemanticType, -) from prik.policy.completion import complete_semantic_policies -from prik.semantics.pyi2ir import _PyiAstParser from tests.fortran._support.pyi_conversion import parse_pyi_text -def test_convert_pyi_to_ir_accepts_ast_only_projection_value_refs(): - module = parse_pyi_text( - """ -@native_call([Return(0), Len(Return(0)), Work("tmp").shape[0]]) -def f() -> Float64: ... -""", - module_name="edited", - ) - - projection = module.functions[0].projection - assert projection[1].value == {"kind": "return", "position": 0} - assert projection[2].value == {"value": {"kind": "work", "name": "tmp"}, "dim": 0} - - -def test_convert_pyi_to_ir_accepts_plain_return_type(): - pyi = """ -def make_value( - x: Float64 -) -> Float64: ... -""" - - module = parse_pyi_text(pyi, module_name="edited") - - func = module.functions[0] - assert func.return_type is not None - assert func.return_type.name == "Float64" - - def test_native_call_address_argument_projection_records_native_address_storage(): module = parse_pyi_text( """ @@ -65,7 +23,6 @@ def add_one(value: Int32) -> Int32: ... function = module.functions[0] value = function.arguments[0] - assert value.semantic_type.name == "Int32" assert value.semantic_type.storage is None assert function.projection[0].value_kind == "addr" assert function.projection[0].value == {"kind": "arg", "position": 0} @@ -83,102 +40,46 @@ def add_one(value: Int32) -> Int32: ... ) -def test_function_equality_treats_argument_names_as_placeholders(): - left = parse_pyi_text( - """ -def resize( - n: Int32, - x: Float64[1:n] -) -> None: ... -""", - module_name="edited", - ) +def test_ir_equality_ignores_only_callable_argument_names(): + """Callable argument names are placeholders; declaration and field names are identity.""" + left = parse_pyi_text("def resize(\n n: Int32,\n x: Float64[1:n]\n) -> None: ...\n", module_name="edited") right = parse_pyi_text( - """ -def resize( - extent: Int32, - values: Float64[1:extent] -) -> None: ... -""", - module_name="edited", + "def resize(\n extent: Int32,\n values: Float64[1:extent]\n) -> None: ...\n", module_name="edited" ) assert left == right assert left.functions[0].arguments[0] != right.functions[0].arguments[0] - - -def test_plain_return_type_represents_direct_return_not_output_argument(): - from_pyi = parse_pyi_text( - """ -def add( - a: Float64, - b: Float64 -) -> Float64: ... -""", - module_name="edited", - ) - func = from_pyi.functions[0] - assert func.return_type.name == "Float64" - assert [arg.name for arg in func.arguments] == ["a", "b"] - - -def test_native_call_preserves_unnamed_output_argument_position(): - from_pyi = parse_pyi_text( - """ -@native_call([Arg(0), Arg(1), Return(0)]) -def add( - a: Float64, - b: Float64 -) -> Float64: ... -""", - module_name="edited", + assert parse_pyi_text("value: Int32\n", module_name="edited") != parse_pyi_text( + "other: Int32\n", module_name="edited" ) - from_ir = SemanticModule( - name="edited", - functions=[ - SemanticFunction( - name="add", - native_name="add", - arguments=[ - SemanticArgument("a", SemanticType("Float64", dtype="Float64")), - SemanticArgument("b", SemanticType("Float64", dtype="Float64")), - SemanticArgument( - "c", - SemanticType("Float64", dtype="Float64"), - ), - ], - projection=[ - ProjectionMapping( - native_name="c", - native_position=2, - result_position=0, - ) - ], - ) - ], + assert parse_pyi_text("class vector:\n x: Float64\n", module_name="edited") != parse_pyi_text( + "class vector:\n y: Float64\n", module_name="edited" ) - assert from_pyi != from_ir - assert from_pyi.functions[0].projection[2].native_position == 2 - -def test_native_call_return_entry_can_preserve_output_name(): - from_pyi = parse_pyi_text( - """ -@native_call([Arg(0), Arg(1), Return("c", 0)]) +@pytest.mark.parametrize( + ("entry", "output_name"), + [("Return(0)", "__return_0"), ('Return("c", 0)', "c")], + ids=["unnamed-output", "named-output"], +) +def test_native_call_return_entry_keeps_the_native_output_slot(entry: str, output_name: str): + function = parse_pyi_text( + f""" +@native_call([Arg(0), Arg(1), {entry}]) def add( a: Float64, b: Float64 ) -> Float64: ... """, module_name="edited", - ) - func = from_pyi.functions[0] + ).functions[0] - assert [arg.name for arg in func.arguments] == ["a", "b", "c"] - assert func.projection[2].native_name == "c" - assert func.projection[2].python_name == "c" - assert func.projection[2].result_position == 0 + assert [arg.name for arg in function.arguments][:2] == ["a", "b"] + assert function.projection[2].native_position == 2 + assert function.projection[2].result_position == 0 + if output_name == "c": + assert function.arguments[2].name == "c" + assert function.projection[2].native_name == function.projection[2].python_name == "c" def test_projected_replacement_without_native_call_keeps_writable_argument_storage(): @@ -199,22 +100,6 @@ def fixed_inout( assert func.projection[0].result_position == 0 -def test_native_call_projected_output_keeps_visible_storage_writable(): - from_pyi = parse_pyi_text( - """ -@native_call([Arg(0), Arg(1)]) -def fill( - n: Addr(Int32), - values: Float64[n] -) -> Returns["values", Float64[n]]: ... -""", - module_name="edited", - ) - func = from_pyi.functions[0] - - assert func.projection[1].result_position == 0 - - def test_native_call_compact_array_output_marks_projection_without_direction_label(): from_pyi = parse_pyi_text( """ @@ -307,236 +192,68 @@ def wrapper( x: Float64[n], b: Vector | None = None ) -> None: ... + +@native_call([Return(0), Len(Return(0)), Work("tmp").shape[0]]) +def f() -> Float64: ... """, module_name="edited", ) - projection = module.functions[0].projection - - # The stated materialization type and exact C scalar identity are orthogonal - # to these producer facts; they have their own focused evidence above. - orthogonal = {"native_c_identity", "value_cast"} + wrapper, returned = module.functions assert [ - {name: value for name, value in asdict(mapping).items() if name not in orthogonal} for mapping in projection + (item.native_position, item.python_position, item.value_kind, item.value) for item in wrapper.projection ] == [ - { - "python_name": "x", - "native_name": "x", - "native_position": 0, - "python_position": 0, - "result_position": None, - "value_kind": "", - "value": None, - }, - { - "python_name": None, - "native_name": "", - "native_position": 1, - "python_position": None, - "result_position": None, - "value_kind": "literal", - "value": {"type": "Int32", "value": 1}, - }, - { - "python_name": None, - "native_name": "", - "native_position": 2, - "python_position": None, - "result_position": None, - "value_kind": "literal", - "value": {"type": "Float64", "value": 0.5}, - }, - { - "python_name": None, - "native_name": "", - "native_position": 3, - "python_position": None, - "result_position": None, - "value_kind": "literal", - "value": {"type": "Bool", "value": False}, - }, - { - "python_name": None, - "native_name": "", - "native_position": 4, - "python_position": None, - "result_position": None, - "value_kind": "literal", - "value": {"type": "String[1]", "value": "N"}, - }, - { - "python_name": None, - "native_name": "", - "native_position": 5, - "python_position": None, - "result_position": None, - "value_kind": "len", - "value": {"kind": "arg", "position": 0}, - }, - { - "python_name": None, - "native_name": "", - "native_position": 6, - "python_position": None, - "result_position": None, - "value_kind": "shape", - "value": {"value": {"kind": "arg", "position": 0}, "dim": 0}, - }, - { - "python_name": None, - "native_name": "", - "native_position": 7, - "python_position": None, - "result_position": None, - "value_kind": "is_present", - "value": {"kind": "arg", "position": 1}, - }, - { - "python_name": None, - "native_name": "", - "native_position": 8, - "python_position": None, - "result_position": None, - "value_kind": "work", - "value": "tmp", - }, + (0, 0, "", None), + (1, None, "literal", {"type": "Int32", "value": 1}), + (2, None, "literal", {"type": "Float64", "value": 0.5}), + (3, None, "literal", {"type": "Bool", "value": False}), + (4, None, "literal", {"type": "String[1]", "value": "N"}), + (5, None, "len", {"kind": "arg", "position": 0}), + (6, None, "shape", {"value": {"kind": "arg", "position": 0}, "dim": 0}), + (7, None, "is_present", {"kind": "arg", "position": 1}), + (8, None, "work", "tmp"), ] - assert projection[1].value_kind == "literal" - assert projection[1].value == {"type": "Int32", "value": 1} - assert projection[4].value == {"type": "String[1]", "value": "N"} - assert projection[5].value_kind == "len" - assert projection[5].value == {"kind": "arg", "position": 0} - assert projection[6].value_kind == "shape" - assert projection[6].value == {"value": {"kind": "arg", "position": 0}, "dim": 0} - assert projection[7].value_kind == "is_present" - assert projection[7].value == {"kind": "arg", "position": 1} - assert projection[8].value_kind == "work" - assert projection[8].value == "tmp" - assert module.functions[0].arguments[1].optional - - -def test_emit_native_call_hidden_native_values(): - module = SemanticModule( - name="edited", - functions=[ - SemanticFunction( - name="wrapper", - native_name="wrapper", - arguments=[ - SemanticArgument("x", SemanticType("Float64", dtype="Float64")), - SemanticArgument("b", SemanticType("Vector", dtype="Vector"), optional=True), - ], - projection=[ - ProjectionMapping(native_position=0, python_position=0), - ProjectionMapping( - native_position=1, - value_kind="literal", - value={"type": "Int32", "value": 1}, - ), - ProjectionMapping( - native_position=2, - value_kind="len", - value={"kind": "arg", "position": 0}, - ), - ProjectionMapping( - native_position=3, - value_kind="shape", - value={"value": {"kind": "arg", "position": 0}, "dim": 0}, - ), - ProjectionMapping( - native_position=4, - value_kind="is_present", - value={"kind": "arg", "position": 1}, - ), - ProjectionMapping(native_position=5, value_kind="work", value="tmp"), - ], - ) - ], - ) - - pyi = emit_module(module) - - assert "@native_call([Arg(0), Int32(1), Len(Arg(0)), Arg(0).shape[0], IsPresent(Arg(1)), Work('tmp')])" in pyi + assert wrapper.arguments[1].optional + assert returned.projection[1].value == {"kind": "return", "position": 0} + assert returned.projection[2].value == {"value": {"kind": "work", "name": "tmp"}, "dim": 0} -def test_typed_computed_projection_records_its_producer_and_requested_type(): +def test_typed_projection_entries_record_producer_and_requested_type(): + """A typed wrapper around a size/shape/stride/length producer is a cast; around a literal, a constant.""" module = parse_pyi_text( - """from prik.contracts import Arg, Float64, Int32, Int64, Len, String, native_call - -@native_call([Int32(Arg(0).shape[0]), Int64(Arg(0).strides[0]), Arg(0), Int32(Len(Arg(1))), Arg(1)]) + """ +@native_call([ + Int32(Arg(0).shape[0]), + Int64(Arg(0).strides[0]), + Arg(0), + Int32(Len(Arg(1))), + Arg(1), + Arg(0).size, + Int32(Arg(0).size), + Int32(1), + Int32(-1), + Float64(-0.5), + Complex64(1+2j), +]) def scale(values: Float64[::], label: String[8]) -> None: ... """, module_name="typed_projection", ) - projection = module.functions[0].projection - - assert [(item.value_kind, item.value_cast) for item in projection] == [ - ("shape", "Int32"), - ("stride", "Int64"), - ("", None), - ("len", "Int32"), - ("", None), - ] - assert "@native_call([Int32(Arg(0).shape[0]), Int64(Arg(0).strides[0]), Arg(0), Int32(Len(Arg(1))), Arg(1)])" in ( - emit_module(module) - ) - - -def test_total_size_projection_round_trips_with_default_and_typed_integer_storage(): - module = parse_pyi_text( - """from prik.contracts import Arg, Float64, Int32, native_call - -@native_call([Arg(0).size, Int32(Arg(0).size), Arg(0)]) -def scale(values: Float64[:]) -> None: ... -""", - module_name="total_size_projection", - ) - - projection = module.functions[0].projection - - assert [(item.value_kind, item.value, item.value_cast) for item in projection] == [ - ("size", {"kind": "arg", "position": 0}, None), - ("size", {"kind": "arg", "position": 0}, "Int32"), + arg0 = {"kind": "arg", "position": 0} + assert [(item.value_kind, item.value, item.value_cast) for item in module.functions[0].projection] == [ + ("shape", {"value": arg0, "dim": 0}, "Int32"), + ("stride", {"value": arg0, "dim": 0}, "Int64"), ("", None, None), + ("len", {"kind": "arg", "position": 1}, "Int32"), + ("", None, None), + ("size", arg0, None), + ("size", arg0, "Int32"), + ("literal", {"type": "Int32", "value": 1}, None), + ("literal", {"type": "Int32", "value": -1}, None), + ("literal", {"type": "Float64", "value": -0.5}, None), + ("literal", {"type": "Complex64", "value": 1 + 2j}, None), ] - assert "@native_call([Arg(0).size, Int32(Arg(0).size), Arg(0)])" in emit_module(module) - - -def test_typed_literal_keeps_its_constant_form_beside_typed_projections(): - module = parse_pyi_text( - """from prik.contracts import Arg, Float64, Int32, native_call - -@native_call([Int32(Arg(0).shape[0]), Arg(0), Int32(1)]) -def scale(values: Float64[:]) -> None: ... -""", - module_name="typed_literal_and_projection", - ) - - projection = module.functions[0].projection - - assert [item.value_kind for item in projection] == ["shape", "", "literal"] - assert projection[2].value == {"type": "Int32", "value": 1} - assert projection[2].value_cast is None - - -def test_typed_literal_uses_literal_evaluation_before_projection_parsing(): - module = parse_pyi_text( - """ -@native_call([Int32(-1), Float64(-0.5), Complex64(1+2j), Arg(0)]) -def scale(value: Float64) -> None: ... -""", - module_name="literal_expressions", - ) - - projection = module.functions[0].projection - - assert [item.value for item in projection[:3]] == [ - {"type": "Int32", "value": -1}, - {"type": "Float64", "value": -0.5}, - {"type": "Complex64", "value": 1 + 2j}, - ] - assert parse_pyi_text(emit_module(module), module_name="literal_expressions") == module def test_typed_scalar_constructor_rejects_a_visible_argument_reference(): @@ -554,101 +271,23 @@ def scale(count: Int32) -> None: ... ) -def test_plain_return_without_native_call_does_not_preserve_native_output_position(): - from_pyi = parse_pyi_text( - """ -def add( - a: Float64, - b: Float64 -) -> Float64: ... -""", - module_name="edited", - ) - with_native_call = SemanticModule( - name="edited", - functions=[ - SemanticFunction( - name="add", - native_name="add", - arguments=[ - SemanticArgument("a", SemanticType("Float64", dtype="Float64")), - SemanticArgument("b", SemanticType("Float64", dtype="Float64")), - SemanticArgument( - "c", - SemanticType("Float64", dtype="Float64"), - ), - ], - projection=[ - ProjectionMapping( - native_name="c", - native_position=2, - result_position=0, - ) - ], - ) - ], - ) - - assert from_pyi != with_native_call - - def test_plain_tuple_return_types_parse_component_returns(): - from_pyi = parse_pyi_text( + func = parse_pyi_text( """ def split( x: Float64 -) -> tuple[Float64, Int32]: ... +) -> tuple[Float64, Int32, Logical]: ... """, module_name="edited", - ) - func = from_pyi.functions[0] - assert func.return_type.name == "Float64" - assert [arg.name for arg in func.arguments] == ["x", "__return_1"] - - -def test_return_projection_preserves_multiple_plain_output_components(): - parser = _PyiAstParser(module_name="internal") - parser._contract_bindings.update({name: name for name in CONTRACT_SYMBOLS}) - - return_type, returned = parser.return_projection(ast.parse("tuple[Float64, Int32, Logical]", mode="eval").body) + ).functions[0] - assert return_type.name == "Float64" - assert [asdict(arg) for arg in returned] == [ - asdict( - SemanticArgument( - "__return_1", - SemanticType("Int32", dtype="Int32"), - metadata={"return_position": 1}, - ) - ), - asdict( - SemanticArgument( - "__return_2", - SemanticType("Logical", dtype="Logical"), - metadata={"return_position": 2}, - ) - ), + assert func.return_type.name == "Float64" + assert [(arg.name, arg.semantic_type.name) for arg in func.arguments] == [ + ("x", "Float64"), + ("__return_1", "Int32"), + ("__return_2", "Logical"), ] - - -def test_non_callable_argument_names_remain_significant(): - assert parse_pyi_text("value: Int32\n", module_name="edited") != parse_pyi_text( - "other: Int32\n", - module_name="edited", - ) - assert parse_pyi_text( - """ -class vector: - x: Float64 -""", - module_name="edited", - ) != parse_pyi_text( - """ -class vector: - y: Float64 -""", - module_name="edited", - ) + assert [(item.native_position, item.result_position) for item in func.projection] == [(0, None), (1, 1), (2, 2)] @pytest.mark.parametrize( @@ -726,10 +365,6 @@ class vector: "def f(x: Int32) -> Returns['x']: ...\n", "Returns expects a name and type; use '| None' for nullable returns: \"Returns['x']\"", ), - ( - "def f(x: Int32) -> Returns['x', Int32, Optional]: ...\n", - "Returns expects a name and type; use '| None' for nullable returns: \"Returns['x', Int32, Optional]\"", - ), ("value: Final[Int32, Float64]\n", "Final expects exactly one type: 'Final[Int32, Float64]'"), ("value: Unknown\n", "Unknown semantic type is not allowed in .pyi annotations"), ("value: Annotated[()]\n", "Annotated type is empty: 'Annotated[()]'"), @@ -741,70 +376,6 @@ def test_convert_pyi_to_ir_rejects_invalid_projection_and_type_forms(source: str assert str(error.value) == message -def test_fortran_to_pyi_and_back_preserves_mixed_input_output_projection(): - source = """ -module solver_mod -contains - subroutine solve(a, x, b) - real(8), intent(in) :: a - real(8), intent(out) :: x - real(8), intent(in) :: b - end subroutine solve -end module solver_mod -""" - - parsed = parse_fortran_file(source) - modules = fortran_file_to_semantic_modules(parsed) - pyi = "\n\n".join(emit_module(module) for module in modules) - reparsed = parse_pyi_text(pyi, module_name="solver_mod") - - assert "@native_call([Addr(Arg(0)), Return('x', 0), Addr(Arg(1))])" in pyi - func = reparsed.functions[0] - assert func.name == "solve" - assert [arg.name for arg in func.arguments] == ["a", "x", "b"] - - -def test_convert_pyi_to_ir_accepts_scalar_descriptor_state_and_callable_projections(): - module = parse_pyi_text( - """ -scratch: Allocatable[Float64] -current: Pointer[Int32] -maybe_value: Float64 | None - -@native_call( - [Allocatable(Arg(0)), Pointer(Arg(1))], - result=Pointer(Return(0)), -) -def combine(scale: Float64 | None, value: Int32 | None) -> Float64 | None: ... -""", - module_name="scalar_descriptors", - ) - - scratch, current, maybe_value = [variable.semantic_type for variable in module.variables] - assert scratch.name == "Float64" - assert scratch.rank == 0 - assert scratch.storage is None - assert scratch.metadata["fortran_allocatable"] is True - - assert current.name == "Int32" - assert current.rank == 0 - assert current.metadata["fortran_pointer"] is True - assert current.metadata["fortran_pointer_association"] == "runtime" - assert current.storage.kind == "reference" - assert current.storage.pointer_depth == 1 - - assert maybe_value.name == "Float64 | None" - assert maybe_value.metadata.get("fortran_allocatable") is None - assert maybe_value.metadata.get("fortran_pointer") is None - - scale, value = [argument.semantic_type for argument in module.functions[0].arguments] - result = module.functions[0].return_type - assert scale.metadata["fortran_allocatable"] is True - assert value.metadata["fortran_pointer"] is True - assert result.metadata["fortran_pointer"] is True - assert [mapping.value_kind for mapping in module.functions[0].projection] == ["allocatable", "pointer"] - - def test_convert_pyi_to_ir_accepts_nullable_descriptor_output_and_inout_projections(): module = parse_pyi_text( """ @@ -831,23 +402,6 @@ def update( assert selected.optional is False -def test_convert_pyi_to_ir_resolves_aliased_scalar_descriptor_projection_helpers(): - module = parse_pyi_text( - """ -from prik.contracts import Allocatable as A, Arg as Input, Float64 as F64, Pointer as P, Return as Output, native_call as call - -@call([A(Input(0))], result=P(Output(0))) -def convert(value: F64 | None) -> F64 | None: ... -""", - module_name="aliased_descriptor_projection", - ) - - function = module.functions[0] - assert function.projection[0].value_kind == "allocatable" - assert function.arguments[0].semantic_type.metadata["fortran_allocatable"] is True - assert function.return_type.metadata["fortran_pointer"] is True - - def test_convert_pyi_to_ir_handles_pointer_and_array_storage_variants(): module = parse_pyi_text( """ @@ -855,14 +409,11 @@ def test_convert_pyi_to_ir_handles_pointer_and_array_storage_variants(): deep: Addr[3](Float64) rank_any: Float64[...] strided: Float64[0:n:] -computed: Float64[xl.size] -bounded_answer: Final[Annotated[Int32, Bounded(1, 8)]] -nested_answer: Final[Final[Int32]] """, module_name="storage", ) - constant, deep, rank_any, strided, computed, bounded, nested = [var.semantic_type for var in module.variables] + constant, deep, rank_any, strided = [var.semantic_type for var in module.variables] assert constant.storage is None assert deep.storage.kind == "pointer" assert deep.storage.pointer_depth == 3 @@ -874,23 +425,3 @@ def test_convert_pyi_to_ir_handles_pointer_and_array_storage_variants(): assert rank_any.rank == 1 assert strided.shape == ["0:n:"] assert strided.storage.array.contiguous is False - assert computed.shape == ["xl.size"] - assert bounded.constraints == [ - SemanticConstraint("Bounded", [1, 8]), - SemanticConstraint("Constant"), - ] - assert nested.constraints == [SemanticConstraint("Constant")] - - -def test_convert_pyi_to_ir_preserves_module_fields_and_private_function_arguments(): - module = parse_pyi_text( - """ -output: Float64[:] = ... - -def consume(value: private[Int32]) -> None: ... -""", - module_name="fields", - ) - - assert module.variables[0].optional is True - assert module.functions[0].arguments[0].visibility == "private" diff --git a/tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py b/tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py index 04e35b422..57dfc3c71 100644 --- a/tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py +++ b/tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py @@ -19,7 +19,6 @@ CONTRACT_IMPORT, parse_pyi_text, ) -import prik.pipeline.pyi as pyi_pipeline def test_convert_pyi_to_ir_requires_imported_contract_types(): @@ -40,9 +39,12 @@ class Float64: assert module.variables[0].semantic_type.name == "Float64" -def test_convert_pyi_to_ir_accepts_import_aliases(): +def test_convert_pyi_to_ir_records_absolute_relative_and_aliased_imports(): module = parse_pyi_text( - "from list_input import delete_input_list as delete_input\n", + "from list_input import delete_input_list as delete_input\n" + "from m import a, b as c\n" + "from ..types_mod import particle\n" + "from . import local_particle\n", module_name="edited", ) @@ -50,22 +52,13 @@ def test_convert_pyi_to_ir_accepts_import_aliases(): SemanticImport( module="list_input", items=[SemanticImportItem(source="delete_input_list", target="delete_input")], - ) - ] - - -def test_convert_pyi_to_ir_accepts_relative_imports(): - module = parse_pyi_text("from ..types_mod import particle\nfrom . import local_particle\n", module_name="edited") - - assert module.imports == [ - SemanticImport( - module="..types_mod", - items=[SemanticImportItem(source="particle")], ), SemanticImport( - module=".", - items=[SemanticImportItem(source="local_particle")], + module="m", + items=[SemanticImportItem(source="a"), SemanticImportItem(source="b", target="c")], ), + SemanticImport(module="..types_mod", items=[SemanticImportItem(source="particle")]), + SemanticImport(module=".", items=[SemanticImportItem(source="local_particle")]), ] @@ -165,70 +158,41 @@ class particle: def test_pyi_paths_to_semantic_modules_reconciles_relative_namespace_type_refs(tmp_path: Path): - physics = tmp_path / "physics.pyi" - a_types = tmp_path / "a_types.pyi" - physics.write_text( + """`from . import a_types [as at]` qualifies a wrapped type by its declaring module.""" + (tmp_path / "physics.pyi").write_text( """ from . import a_types +from . import a_types as at def move(p: a_types.state) -> None: ... -""", - encoding="utf-8", - ) - a_types.write_text( - """ -class state: - pass -""", - encoding="utf-8", - ) - modules = {module.name: module for module in pyi_paths_to_semantic_modules(tmp_path)} - state_ref = modules["physics"].functions[0].arguments[0].semantic_type.metadata["external_type_ref"] - - assert state_ref == { - "name": "state", - "local_name": "a_types.state", - "origin_module": "a_types", - "wrapped": True, - "representation": "wrapped", - } - - -def test_convert_pyi_to_ir_accepts_relative_namespace_alias_type_refs(tmp_path: Path): - physics = tmp_path / "physics.pyi" - a_types = tmp_path / "a_types.pyi" - physics.write_text( - """ -from . import a_types as at - -def move(p: at.state) -> None: ... -""", - encoding="utf-8", - ) - a_types.write_text( - """ -class state: - pass +def move_alias(p: at.state) -> None: ... """, encoding="utf-8", ) + (tmp_path / "a_types.pyi").write_text("class state:\n pass\n", encoding="utf-8") modules = {module.name: module for module in pyi_paths_to_semantic_modules(tmp_path)} - state_ref = modules["physics"].functions[0].arguments[0].semantic_type.metadata["external_type_ref"] - - assert state_ref == { - "name": "state", - "local_name": "at.state", - "origin_module": "a_types", - "wrapped": True, - "representation": "wrapped", - } + refs = [ + function.arguments[0].semantic_type.metadata["external_type_ref"] for function in modules["physics"].functions + ] + + assert refs == [ + { + "name": "state", + "local_name": local_name, + "origin_module": "a_types", + "wrapped": True, + "representation": "wrapped", + } + for local_name in ("a_types.state", "at.state") + ] def test_pyi_paths_to_semantic_modules_preserves_dotted_module_names_from_directory(tmp_path: Path): package = tmp_path / "shared" package.mkdir() + (tmp_path / "ignored.pyi").mkdir() (tmp_path / "physics.pyi").write_text( """ from shared.types_mod import particle @@ -248,7 +212,8 @@ class particle(Opaque): modules = {module.name: module for module in pyi_paths_to_semantic_modules(tmp_path)} particle_ref = modules["physics"].functions[0].arguments[0].semantic_type.metadata["external_type_ref"] - assert "shared.types_mod" in modules + # A directory named like a contract is not one. + assert set(modules) == {"physics", "shared.types_mod"} assert particle_ref["origin_module"] == "shared.types_mod" assert particle_ref["representation"] == "opaque" @@ -265,13 +230,6 @@ def test_pyi_paths_to_semantic_modules_handles_duplicate_roots_and_ambiguous_mod assert str(error.value) == f"Ambiguous module name for {pyi_path}: 'shared.types_mod' or 'types_mod'" -def test_pyi_paths_to_semantic_modules_ignores_directories_with_pyi_suffix(tmp_path: Path): - (tmp_path / "ignored.pyi").mkdir() - (tmp_path / "types_mod.pyi").write_text("class particle:\n pass\n", encoding="utf-8") - - assert [module.name for module in pyi_paths_to_semantic_modules(tmp_path)] == ["types_mod"] - - def test_pyi_file_to_semantic_module_and_modules_forward_module_name_encoding_and_filename(tmp_path: Path): pyi_path = tmp_path / "types_mod.pyi" pyi_path.write_bytes("# caf\xe9\nclass particle:\n pass\n".encode("latin-1")) @@ -296,49 +254,11 @@ def test_pyi_file_to_semantic_module_and_modules_forward_module_name_encoding_an assert "Expected typed argument: 'x'" in message -def test_pyi_conversion_cache_reuses_file_parse_for_same_module_key(monkeypatch, tmp_path: Path): - pyi_path = tmp_path / "types_mod.pyi" - pyi_path.write_text(f"{CONTRACT_IMPORT}value: Int32\n", encoding="utf-8") - - original_parse = pyi_pipeline.parse_pyi_text - parsed_filenames: list[str] = [] - - def parse_once(source: str, *, filename: str = ""): - parsed_filenames.append(filename) - return original_parse(source, filename=filename) - - monkeypatch.setattr(pyi_pipeline, "parse_pyi_text", parse_once) - cache = pyi_pipeline._PyiSemanticModuleCache() - - first = cache.file_to_semantic_module(pyi_path) - second = cache.file_to_semantic_module(pyi_path, module_name="types_mod") - renamed = cache.file_to_semantic_module(pyi_path, module_name="renamed_types") - - assert first is second - assert renamed is not first - assert [Path(filename) for filename in parsed_filenames] == [pyi_path, pyi_path] - - -def test_convert_pyi_to_ir_and_import_parser_edge_cases(): - module = pyi_text_to_semantic_module("from m import a, b as c\n", module_name="edited") - assert module.name == "edited" - assert module.imports == [ - SemanticImport( - module="m", - items=[ - SemanticImportItem(source="a"), - SemanticImportItem(source="b", target="c"), - ], - ), - ] - - with pytest.raises(SyntaxError): - pyi_text_to_semantic_module("from m import\n", module_name="edited") - - def test_generated_native_scope_comes_from_contract_filename(): - parsed = parse_fortran_file( - """ + """A renamed module contract names its native module; a standalone contract names none.""" + module_source = fortran_file_to_semantic_modules( + parse_fortran_file( + """ module solver_mod contains subroutine solve(value) @@ -346,30 +266,30 @@ def test_generated_native_scope_comes_from_contract_filename(): end subroutine solve end module solver_mod """ - ) - module = fortran_file_to_semantic_modules(parsed)[0] - loaded = parse_pyi_text(emit_module(module), module_name="renamed_contract") - - assert loaded.name == "renamed_contract" - assert native_contract_issues(loaded) == [] - assert loaded.origin.native_name == "renamed_contract" - assert loaded.functions[0].origin.native_scope == "renamed_contract" - - -def test_generated_standalone_contract_retains_standalone_native_placement(): - parsed = parse_fortran_file( - """ + ) + )[0] + standalone_source = fortran_file_to_semantic_modules( + parse_fortran_file( + """ subroutine solve(value) real(8), intent(in) :: value end subroutine solve """ + ), + standalone_module_name="root_contract", + )[0] + standalone_text = emit_module(standalone_source) + + loaded = parse_pyi_text(emit_module(module_source), module_name="renamed_contract") + standalone = parse_pyi_text(standalone_text, module_name="renamed_root_contract") + + assert "@standalone" in standalone_text + assert native_contract_issues(loaded) == native_contract_issues(standalone) == [] + assert (loaded.origin.native_name, loaded.functions[0].origin.native_scope) == ( + "renamed_contract", + "renamed_contract", + ) + assert (standalone.origin.native_name, standalone.functions[0].origin.native_scope) == ( + "renamed_root_contract", + None, ) - module = fortran_file_to_semantic_modules(parsed, standalone_module_name="root_contract")[0] - generated = emit_module(module) - loaded = parse_pyi_text(generated, module_name="renamed_root_contract") - - assert "@standalone" in generated - assert loaded.functions[0].origin.native_scope is None - assert native_contract_issues(loaded) == [] - assert loaded.origin.native_name == "renamed_root_contract" - assert loaded.functions[0].origin.native_scope is None diff --git a/tests/fortran/infrastructure/semantic_pyi/semantics/test_native_abi.py b/tests/fortran/infrastructure/semantic_pyi/semantics/test_native_abi.py index d41364ee2..b758cdc72 100644 --- a/tests/fortran/infrastructure/semantic_pyi/semantics/test_native_abi.py +++ b/tests/fortran/infrastructure/semantic_pyi/semantics/test_native_abi.py @@ -91,29 +91,6 @@ def test_native_abi_rejects_contradictory_or_misplaced_annotations( parse_pyi_text(source, module_name="invalid_native_abi", native_language=native_language) -def test_native_abi_round_trip_keeps_marker_symbol_and_projection(): - original = parse_pyi_text( - """ -@native_abi("c") -@bind("renamed_entry") -@native_call([Addr(Arg(0)), Arg(0).shape[0], Return("result", 0)]) -def transform(values: Float64[:]) -> Float64: ... -""", - module_name="round_trip_native_abi", - ) - - rendered = emit_module(original) - loaded = parse_pyi_text(rendered, module_name=original.name) - function = loaded.functions[0] - - assert '@native_abi("c")' in rendered - assert '@bind("renamed_entry")' in rendered - assert function.origin.native_abi == "c" - assert function.origin.native_symbol == "renamed_entry" - assert function.origin.source_language == "fortran" - assert function.projection == original.functions[0].projection - - def test_source_free_native_abi_selects_the_preserved_symbol_and_direct_route(): module = parse_pyi_text( """ diff --git a/tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py b/tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py index 3cf50ff57..be817a5d1 100644 --- a/tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py +++ b/tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py @@ -1,4 +1,4 @@ -"""Tests split by stable ownership concept from `test_c_conversion_properties.py`.""" +"""Printer and loader agreement: generated and canonical contracts round-trip through semantic IR.""" import pytest from hypothesis import ( @@ -17,6 +17,7 @@ SemanticModule, SemanticType, ) +from tests.fortran._support.pyi_conversion import parse_pyi_text as parse_pyi_contract from tests.fortran._support.semantic_properties import ( _NATIVE_NAMES, _PYI_IDENTIFIER_STEMS, @@ -113,3 +114,143 @@ def test_generated_semantic_ir_round_trips_through_pyi(arguments): assert emit_module(reparsed) == emitted assert parse_pyi_text(emit_module(reparsed), module_name="generated") == reparsed + + +CONTRACT_SPELLINGS = { + "pointer-depths": """ +deep: Addr[3](Float64) +shallow: Addr[2](Float64) +raw: Addr(Float64) +""", + "array-descriptor-handles": """ +grid: Allocatable[Float64[:, :]] +mask: Allocatable[Annotated[Bool[1], Finite]] +maybe: Annotated[Allocatable[Float64[:]], MaybeUnallocated] +target: Annotated[Pointer[Float64[:]], PointerAssociation("runtime")] +labels: Pointer[String[8][:]] +""", + "array-layout-and-constraints": """ +flat: Float64[Flat] +matrix: Float64[3, Flat] +c_matrix: Annotated[Float64[Flat, 3], ORDER_C] +c_dense: Annotated[Float64[:, :], ORDER_C] +any_order: Annotated[Float64[:, :], ORDER_ANY, Finite, Range(1, 3)] +bounded: Annotated[Int32, Bounded(1, 8), Finite] +""", + "array-dimensions": """ +def apply( + A: Float64[LDA, N], + work: Float64[::], + bounded: Float64[0:n:], + scratch: Float64[:], + rank_any: Float64[...], + computed: Float64[xl.size], + scalar: Float64[()] +) -> None: ... +""", + "copy-order": """ +def consume( + values: Annotated[Float64[:, :], ORDER_C, COPY_F] +) -> None: ... +""", + "scalar-descriptors": """ +scratch: Allocatable[Float64] +current: Pointer[Int32] +maybe_value: Float64 | None +label: String[16] +names: Allocatable[String[:]] + +@native_call([Allocatable(Arg(0)), Pointer(Arg(1))], result=Pointer(Return(0))) +def combine( + scale: Float64 | None, + value: Int32 | None +) -> Float64 | None: ... +""", + "optional-descriptor-handles": """ +def maybe_consume( + values: Allocatable[Float64[:]] | None = ..., + target: Pointer[Float64[:]] | None = ... +) -> None: ... +""", + "boolean-widths": """ +def inspect( + default: Bool, + byte: Bool8[:], + short: Bool16[:], + word: Bool32[:], + wide: Bool64[:] +) -> None: ... +""", + "hidden-native-values": """ +@native_call([Arg(0), Int32(1), Float64(0.5), Bool(False), String[1]("N"), Len(Arg(0)), Arg(0).shape[0], IsPresent(Arg(1)), Work('tmp')]) +def wrapper( + x: Float64[n], + b: Int32 | None = ... +) -> None: ... +""", + "return-and-work-references": """ +@native_call([Len(Return(0)), Work('tmp').shape[1]]) +def wrapper() -> Float64: ... +""", + "typed-projections": """ +@native_call([Int32(Arg(0).shape[0]), Int64(Arg(0).strides[0]), Arg(0), Int32(Len(Arg(1))), Arg(1), Arg(0).size, Int32(Arg(0).size), Int32(-1), Float64(-0.5), Complex64((1+2j))]) +def scale( + values: Float64[::], + label: String[8] +) -> None: ... +""", + "named-output-return": """ +@native_call([Arg(0), Arg(1), Return('c', 0)]) +def add( + a: Float64, + b: Float64 +) -> Float64: ... +""", + "value-transport-and-native-abi": """ +@native_abi("c") +class point: + x: Float64 + +@native_call([Value(Arg(0))]) +def score( + value: point +) -> Float64: ... + +@native_abi("c") +@bind("renamed_entry") +@native_call([Addr(Arg(0)), Arg(0).shape[0], Return('result', 0)]) +def transform( + values: Float64[:] +) -> Float64: ... +""", + "visibility-and-module-state": """ +import iso_c_binding + +class particle: + id: Int32 + +scale: private[Float64] +answer: Final[Int32] +hidden_answer: private[Final[Int32]] +literal_answer: Final[Int32] = 42 +output: Float64[:] = ... +var['class']: Int32 + +@private +@bind("native_helper") +def helper( + value: private[Int32] +) -> None: ... +""", +} + + +@pytest.mark.parametrize("body", list(CONTRACT_SPELLINGS.values()), ids=list(CONTRACT_SPELLINGS)) +def test_contract_spellings_round_trip_through_semantic_ir(body: str): + """Each canonical spelling loads, prints back as written, and reloads to the same IR.""" + module = parse_pyi_contract(body, module_name="spellings") + emitted = emit_module(module) + + emitted_lines = set(emitted.splitlines()) + assert [line for line in body.strip().splitlines() if line and line not in emitted_lines] == [] + assert parse_pyi_contract(emitted, module_name="spellings") == module diff --git a/tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py b/tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py index 951e98e23..98f2a6e7e 100644 --- a/tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py +++ b/tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py @@ -1,6 +1,5 @@ """Tests split by stable ownership concept from `test_python_ast_contracts.py`.""" -import ast import pytest from prik.parsers.fortran import parse_fortran_file from prik.printers import emit_module @@ -9,7 +8,6 @@ from prik.semantics.metadata import ( ADDRESS_ROLE_METADATA, ADDRESS_ROLE_RAW, - NATIVE_ARRAY_DESCRIPTOR_METADATA, OPTIONAL_ABSENT_HANDLE_METADATA, USER_PRIVATE_METADATA, ) @@ -17,8 +15,6 @@ PYTHON_VALUE_IMMUTABLE, PYTHON_VALUE_MUTABILITY_METADATA, SemanticConstraint, - SemanticField, - SemanticVariable, ) from prik.semantics.native_array_handles import ( is_native_array_handle, @@ -27,98 +23,60 @@ native_array_handle_facts, ) from prik.semantics.native_contract import native_contract_issues -from prik.semantics.pyi2ir import _node_text from tests.fortran._support.pyi_conversion import parse_pyi_text def test_convert_pyi_to_ir_dispatches_nested_and_qualified_semantic_types(): + """A declared `Addr(T)` is raw native address storage, unlike an `Addr(Arg)` projection.""" module = parse_pyi_text( """ public_value: Int32 bounded: Final[Annotated[Int32, Bounded(1, 8)]] pointer: Addr(Float64) -raw_pointer: Addr(Float64) """, module_name="dispatch", ) - public_value, bounded, pointer, raw_pointer = module.variables - assert isinstance(public_value, SemanticVariable) + public_value, bounded, pointer = module.variables assert public_value.visibility == "public" + assert public_value.semantic_type.storage is None assert bounded.semantic_type.constraints == [ SemanticConstraint("Bounded", [1, 8]), SemanticConstraint("Constant"), ] assert pointer.semantic_type.storage.kind == "address" assert pointer.semantic_type.storage.metadata[ADDRESS_ROLE_METADATA] == ADDRESS_ROLE_RAW - assert raw_pointer.semantic_type.storage.read_only is False - - -def test_boolean_storage_widths_round_trip_as_one_semantic_type_family(): - module = pyi_text_to_semantic_module( - """ -from prik.contracts import Bool, Bool8, Bool16, Bool32, Bool64 - -def inspect( - default: Bool, - byte: Bool8[:], - short: Bool16[:], - word: Bool32[:], - wide: Bool64[:], -) -> None: ... -""", - module_name="boolean_widths", - ) - - assert [argument.semantic_type.name for argument in module.functions[0].arguments] == [ - "Bool", - "Bool8", - "Bool16", - "Bool32", - "Bool64", - ] - emitted = emit_module(module) - assert "from prik.contracts import Bool, Bool16, Bool32, Bool64, Bool8" in emitted - - -def test_value_projection_round_trips_as_argument_specific_native_transport(): - module = parse_pyi_text( - """ -from prik.contracts import Arg, Float64, Value, native_abi, native_call - -@native_abi("c") -class point: - x: Float64 - -@native_call([Value(Arg(0))]) -def score(value: point) -> Float64: ... -""", - module_name="value_contract", - ) - - value = module.functions[0].arguments[0] - assert value.metadata["native_by_value"] is True - assert "@native_call([Value(Arg(0))])" in emit_module(module) - assert "value: point" in emit_module(module) + assert pointer.semantic_type.storage.read_only is False def test_convert_pyi_to_ir_follows_arbitrary_contract_aliases(): + """Every contract helper, wrapper, and projection name resolves through its import alias.""" module = pyi_text_to_semantic_module( """ from prik.contracts import Addr as AddressOf, Arg as PythonArg, Final as Frozen from prik.contracts import Flat as Layout, Float64 as F64, Int32 as I32, native_call as call +from prik.contracts import Annotated as Metadata, SourceName as NativeName, Returns as Gives Flat: Frozen[I32] = 10 +alias: Metadata[F64[1:n], NativeName("native_alias")] @call([AddressOf(PythonArg(0))]) def inspect(values: F64[Layout], dense: F64[Flat]) -> None: ... + +def f() -> tuple[F64, Gives["y", F64]]: ... """, module_name="aliases", ) - assert module.variables[0].name == "Flat" - assert module.functions[0].arguments[0].semantic_type.storage.array.category == "assumed_size" - assert module.functions[0].arguments[1].semantic_type.shape == ["Flat"] + assert [variable.name for variable in module.variables] == ["Flat", "alias"] + assert module.variables[1].origin.native_name == "native_alias" + assert module.variables[1].semantic_type.shape == ["1:n"] + inspect, returns = module.functions + assert inspect.projection[0].value_kind == "addr" + assert inspect.arguments[0].semantic_type.storage.array.category == "assumed_size" + assert inspect.arguments[1].semantic_type.shape == ["Flat"] + assert returns.return_type.name == "Float64" + assert returns.arguments[0].name == "y" def test_convert_pyi_to_ir_preserves_immutable_python_value_metadata(): @@ -141,100 +99,30 @@ def scale( assert reparsed_values.metadata[PYTHON_VALUE_MUTABILITY_METADATA] == PYTHON_VALUE_IMMUTABLE -def test_convert_pyi_to_ir_allows_user_modified_stub(): - pyi = """ -import iso_c_binding - -class particle: - id: Int32 - -scale: private[Float64] -answer: Final[Int32] -hidden_answer: private[Final[Int32]] -literal_answer: Final[Int32] = 42 - -def touch( - p: particle -) -> Returns["p", particle]: ... -""" - - module = parse_pyi_text(pyi, module_name="edited") - - assert module.name == "edited" - assert module.imports == ["iso_c_binding"] - assert module.classes[0].name == "particle" - assert isinstance(module.classes[0].fields[0], SemanticField) - assert module.variables[0].name == "scale" - assert module.variables[0].visibility == "private" - assert module.variables[1].name == "answer" - assert [c.name for c in module.variables[1].semantic_type.constraints] == ["Constant"] - assert module.variables[2].name == "hidden_answer" - assert module.variables[2].visibility == "private" - assert [c.name for c in module.variables[2].semantic_type.constraints] == ["Constant"] - assert module.variables[3].name == "literal_answer" - assert module.variables[3].default_value == "42" - - -def test_convert_pyi_to_ir_forwards_filename_to_syntax_errors(): - with pytest.raises(SyntaxError) as error: - parse_pyi_text("from broken import\n", filename="custom.pyi") - assert error.value.filename == "custom.pyi" - - -def test_convert_pyi_to_ir_accepts_aliased_contract_wrapper_names(): - module = pyi_text_to_semantic_module( - """ -from prik.contracts import Annotated as Metadata, Float64 as F64, SourceName as NativeName -from prik.contracts import Returns as Gives - -alias: Metadata[F64[1:n], NativeName("native_alias")] - -def f() -> tuple[F64, Gives["y", F64]]: ... -""", - module_name="edited", - ) - - # The declared name stays the Python name; SourceName states the native - # entity it reaches, as bind does for a callable. - assert module.variables[0].name == "alias" - assert module.variables[0].origin.native_name == "native_alias" - assert module.variables[0].semantic_type.shape == ["1:n"] - assert module.functions[0].return_type is not None - assert module.functions[0].return_type.name == "Float64" - assert module.functions[0].arguments[0].name == "y" - - -def test_rank_zero_scalar_storage_round_trips_as_empty_tuple_array(): +def test_rank_zero_scalar_storage_is_writable_scalar_array_storage(): module = parse_pyi_text( """ def update_storage(value: Float64[()]) -> None: ... -def inspect_storage(value: Int32[()]) -> None: ... """, module_name="scalar_storage", ) - update, inspect = module.functions - update_type = update.arguments[0].semantic_type - inspect_type = inspect.arguments[0].semantic_type - + update_type = module.functions[0].arguments[0].semantic_type assert update_type.rank == 0 assert update_type.storage.kind == "array" assert update_type.storage.array.category == "scalar_storage" - assert inspect_type.storage.read_only is False - assert inspect_type.storage.mutable is True - - emitted = emit_module(module) - assert "value: Float64[()]" in emitted - assert "value: Int32[()]" in emitted - assert parse_pyi_text(emitted, module_name="scalar_storage") == module + assert update_type.storage.read_only is False + assert update_type.storage.mutable is True def test_convert_pyi_to_ir_preserves_explicit_array_source_dimensions(): + """Explicit extents keep their source dimensions; an empty step marks a strided axis.""" module = parse_pyi_text( """ def apply( A: Float64[LDA, N], work: Float64[::], + bounded: Float64[0:n:], scratch: Float64[:] ) -> None: ... """, @@ -245,41 +133,18 @@ def apply( assert args["A"].source_shape == ["LDA", "N"] assert args["A"].lower_bounds == [None, None] assert args["A"].upper_bounds == [None, None] - assert args["work"].shape == ["::"] - assert args["work"].axes == ["strided"] - assert args["work"].contiguous is False - assert args["work"].source_shape == [] - assert args["scratch"].shape == [":"] - assert args["scratch"].axes == ["dense"] - assert args["scratch"].contiguous is True - assert args["scratch"].source_shape == [] - - -def test_convert_pyi_to_ir_reads_a_strided_axis_from_its_empty_step(): - """An empty step marks a strided axis; a bounded axis keeps its bounds.""" - module = parse_pyi_text( - """ -unbounded: Float64[::] -bounded: Float64[0:n:] -""", - module_name="strided_axes", - ) - - arrays = [variable.semantic_type.storage.array for variable in module.variables] - assert [array.shape for array in arrays] == [["::"], ["0:n:"]] - assert [array.axes for array in arrays] == [["strided"], ["strided"]] - assert [array.contiguous for array in arrays] == [False, False] - + assert [(args[name].axes, args[name].contiguous, args[name].source_shape) for name in ("work", "bounded")] == [ + (["strided"], False, []), + (["strided"], False, []), + ] + assert args["bounded"].shape == ["0:n:"] + assert (args["scratch"].axes, args["scratch"].contiguous) == (["dense"], True) -@pytest.mark.parametrize("dimension", ["Float64[::Strided]", "Float64[0:n:Strided]", "Float64[::2]"]) -def test_convert_pyi_to_ir_rejects_a_dimension_step(dimension: str): - """A dimension carries bounds only, so the step position spells nothing. - `T[::]` already says strided, so the longer explicit form it replaced is - refused rather than kept as a second way to write the same contract. - """ +def test_convert_pyi_to_ir_rejects_a_dimension_step(): + """A dimension carries bounds only, so the step position spells nothing.""" with pytest.raises(ValueError, match="not part of the contract grammar"): - parse_pyi_text(f"x: {dimension}\n", module_name="rejected_step") + parse_pyi_text("x: Float64[::2]\n", module_name="rejected_step") def test_convert_pyi_to_ir_uses_fortran_native_array_defaults(): @@ -308,22 +173,6 @@ def test_convert_pyi_to_ir_rejects_redundant_fortran_default_array_order(): ) -def test_convert_pyi_to_ir_records_explicit_c_to_fortran_copy_order(): - module = parse_pyi_text( - """ -def consume(values: Annotated[Float64[:, :], ORDER_C, COPY_F]) -> None: ... -""", - module_name="copy_order", - ) - - array = module.functions[0].arguments[0].semantic_type.storage.array - - assert array.order == "ORDER_C" - assert array.copy_order == "ORDER_F" - assert array.rank == 2 - assert array.contiguous is True - - def test_convert_pyi_to_ir_accepts_flat_array_dimension(): module = parse_pyi_text( """ @@ -368,31 +217,16 @@ def test_convert_pyi_to_ir_accepts_flat_array_dimension(): assert [array.order for array in arrays] == [None, "ORDER_F", "ORDER_F", "ORDER_C", "ORDER_C"] -def test_convert_pyi_to_ir_preserves_array_layout_and_nested_selector(): +def test_convert_pyi_to_ir_preserves_rank_selector_and_character_allocatable_marker(): module = parse_pyi_text( """ -value: Float64[:, :] nested: Float64[:, :][rank, kind] name: Annotated[String[16], FortranAllocatable] - -def fill(x: Float64[:]) -> None: ... """, module_name="metadata", ) - value_type = module.variables[0].semantic_type - value = value_type.storage.array - nested = module.variables[1].semantic_type - name = module.variables[2].semantic_type - assert value.order == "ORDER_F" - assert value.allocatable is False - assert value.pointer is False - assert value.contiguous is True - assert value.category is None - assert value.source_shape == [] - assert value.lower_bounds == [] - assert value.upper_bounds == [] - assert value_type.constraints == [] + nested, name = [variable.semantic_type for variable in module.variables] assert nested.metadata["rank_selector"] == "rank, kind" assert nested.storage.array.metadata["rank_selector"] == "rank, kind" assert name.metadata["fortran_character_length"] == "16" @@ -400,6 +234,7 @@ def fill(x: Float64[:]) -> None: ... def test_convert_pyi_to_ir_accepts_array_descriptor_handle_wrappers(): + """`Allocatable[...]`/`Pointer[...]` wrap a plain array as a native descriptor handle.""" module = pyi_text_to_semantic_module( """ from prik.contracts import Allocatable as A, Annotated, Float64 as F64, Pointer as P, SourceName, String as Str @@ -409,87 +244,32 @@ def test_convert_pyi_to_ir_accepts_array_descriptor_handle_wrappers(): labels: P[Str[8][:]] plain_values: F64[:] -def consume(values: A[F64[:]], target: P[F64[:]]) -> None: ... def maybe_consume(values: A[F64[:]] | None = ..., target: P[F64[:]] | None = ...) -> None: ... """, module_name="array_descriptors", ) values, target, labels, plain_values = [variable.semantic_type for variable in module.variables] - assert is_native_array_handle(values) is True - assert native_array_descriptor_kind(values) == "allocatable" - assert values.storage.array.allocatable is True - assert values.storage.array.pointer is False - assert values.metadata[NATIVE_ARRAY_DESCRIPTOR_METADATA] == "allocatable" - assert values.rank == 1 - assert values.shape == [":"] - values_data = native_array_data_type(values) - assert values_data.storage.array.allocatable is False - assert values_data.storage.array.pointer is False - assert values_data.metadata.get(NATIVE_ARRAY_DESCRIPTOR_METADATA) is None - assert values_data == plain_values + assert [native_array_descriptor_kind(item) for item in (values, target, labels)] == [ + "allocatable", + "pointer", + "pointer", + ] assert is_native_array_handle(plain_values) is False - - assert target.storage.array.pointer is True - assert native_array_descriptor_kind(target) == "pointer" - assert target.metadata[NATIVE_ARRAY_DESCRIPTOR_METADATA] == "pointer" - assert target.rank == 2 - target_data = native_array_data_type(target) - assert target_data.storage.array.pointer is False - assert target_data.rank == target.rank - - assert native_array_descriptor_kind(labels) == "pointer" - assert labels.name == "String" - assert labels.rank == 1 - assert labels.shape == [":"] - assert labels.metadata["fortran_character_length"] == "8" - assert labels.storage.array.pointer is True - labels_data = native_array_data_type(labels) - assert labels_data.metadata["fortran_character_length"] == "8" - - values_facts = native_array_handle_facts(values) - assert values_facts.descriptor_kind == "allocatable" - assert values_facts.data_type == plain_values - assert values_facts.element_type.name == "Float64" - assert values_facts.element_type.rank == 0 - assert values_facts.element_type.shape == [] - assert values_facts.dtype == "Float64" - assert values_facts.rank == 1 - assert values_facts.shape == (":",) - assert values_facts.fortran_character_length is None - - target_facts = native_array_handle_facts(target) - assert target_facts.descriptor_kind == "pointer" - assert target_facts.data_type.storage.array.pointer is False - assert target_facts.rank == 2 - assert target_facts.shape == (":", ":") + assert native_array_data_type(values) == plain_values + assert native_array_data_type(target).storage.array.pointer is False + assert native_array_data_type(target).rank == 2 labels_facts = native_array_handle_facts(labels) - assert labels_facts.descriptor_kind == "pointer" - assert labels_facts.element_type.name == "String" - assert labels_facts.element_type.rank == 0 - assert labels_facts.element_type.metadata["fortran_character_length"] == "8" - assert labels_facts.data_type.storage.array.pointer is False - assert labels_facts.dtype == "String" - assert labels_facts.rank == 1 - assert labels_facts.shape == (":",) + assert (labels_facts.dtype, labels_facts.rank, labels_facts.shape) == ("String", 1, (":",)) assert labels_facts.fortran_character_length == "8" - + assert labels_facts.element_type.rank == 0 with pytest.raises(ValueError, match="is not a native array handle"): native_array_handle_facts(plain_values) - assert labels_data.storage.array.pointer is False - - consume_values, consume_target = [arg.semantic_type for arg in module.functions[0].arguments] - assert consume_values.storage.array.allocatable is True - assert consume_target.storage.array.pointer is True - maybe_values, maybe_target = module.functions[1].arguments - assert maybe_values.semantic_type.metadata[NATIVE_ARRAY_DESCRIPTOR_METADATA] == "allocatable" - assert maybe_values.semantic_type.metadata[OPTIONAL_ABSENT_HANDLE_METADATA] is True - assert maybe_values.optional is True - assert maybe_target.semantic_type.metadata[NATIVE_ARRAY_DESCRIPTOR_METADATA] == "pointer" - assert maybe_target.semantic_type.metadata[OPTIONAL_ABSENT_HANDLE_METADATA] is True - assert maybe_target.optional is True + for argument in module.functions[0].arguments: + assert argument.optional is True + assert argument.semantic_type.metadata[OPTIONAL_ABSENT_HANDLE_METADATA] is True def test_convert_pyi_to_ir_preserves_user_private_bound_function_contract(): @@ -505,14 +285,10 @@ def helper(value: Int32) -> None: ... helper = module.functions[0] assert native_contract_issues(module) == [] assert helper.visibility == "private" + assert helper.native_name == "native_helper" assert helper.origin.source_language == "fortran" assert helper.origin.metadata[USER_PRIVATE_METADATA] is True - emitted = emit_module(module) - assert '@private\n@bind("native_helper")\ndef helper(' in emitted - assert " value: Int32" in emitted - assert parse_pyi_text(emitted, module_name="edited") == module - @pytest.mark.parametrize( "source, message", @@ -568,10 +344,6 @@ def test_convert_pyi_to_ir_rejects_additional_invalid_storage_forms(source: str, assert str(error.value) == message -def test_node_text_falls_back_to_node_type_for_empty_unparse(): - assert _node_text(ast.Module(body=[], type_ignores=[])) == "Module" - - def test_native_contract_structurally_accepts_declared_type_and_constraint_edits(): parsed = parse_fortran_file( """ diff --git a/tests/fortran/infrastructure/utilities/test_class_visitor.py b/tests/fortran/infrastructure/utilities/test_class_visitor.py index c7b26cffd..50b68c9bb 100644 --- a/tests/fortran/infrastructure/utilities/test_class_visitor.py +++ b/tests/fortran/infrastructure/utilities/test_class_visitor.py @@ -13,39 +13,26 @@ class ChildNode(BaseNode): """More specific model used to prove MRO dispatch.""" -def test_class_visitor_uses_the_most_specific_available_handler() -> None: - class Visitor(ClassVisitor): - @staticmethod - def _visit_BaseNode(_node): - return "base" - - @staticmethod - def _visit_ChildNode(_node): - return "child" +class GrandchildNode(ChildNode): + """Model whose nearest handler is its parent's.""" - assert Visitor()._visit(ChildNode()) == "child" - -def test_class_visitor_falls_back_to_a_base_model_handler() -> None: - class Visitor(ClassVisitor): - @staticmethod - def _visit_BaseNode(_node): - return "base" - - assert Visitor()._visit(ChildNode()) == "base" - - -def test_class_visitor_supports_a_configured_handler_prefix() -> None: +def test_class_visitor_dispatches_by_mro_and_rejects_unhandled_models() -> None: class ParserVisitor(ClassVisitor): visitor_method_prefix = "_parse" @staticmethod def _parse_BaseNode(node): - return type(node).__name__ + return ("base", type(node).__name__) - assert ParserVisitor()._visit(ChildNode()) == "ChildNode" + @staticmethod + def _parse_ChildNode(node): + return ("child", type(node).__name__) + visitor = ParserVisitor() -def test_class_visitor_reports_an_unsupported_model() -> None: + assert visitor._visit(BaseNode()) == ("base", "BaseNode") + assert visitor._visit(ChildNode()) == ("child", "ChildNode") + assert visitor._visit(GrandchildNode()) == ("child", "GrandchildNode") with pytest.raises(TypeError, match="Unsupported model for class visitor"): - ClassVisitor()._visit(object()) + visitor._visit(object()) diff --git a/tests/fortran/infrastructure/utilities/test_strings.py b/tests/fortran/infrastructure/utilities/test_strings.py deleted file mode 100644 index b1b3c631e..000000000 --- a/tests/fortran/infrastructure/utilities/test_strings.py +++ /dev/null @@ -1,30 +0,0 @@ -"""Tests for prik's self-contained generated-name helpers.""" - -from prik.utilities.strings import create_incremented_string, random_string - - -class _CaseInsensitiveRules: - def has_clash(self, name: object, symbols: set[object]) -> bool: - return str(name).casefold() in {str(symbol).casefold() for symbol in symbols} - - -def test_incremented_name_skips_reserved_spellings_and_reports_next_counter(): - name, counter = create_incremented_string({"bridge_2", "bridge_3"}, prefix="bridge") - - assert name == "bridge_4" - assert counter == 5 - - -def test_incremented_name_uses_the_target_language_collision_rule(): - name, counter = create_incremented_string({"VALUE_2"}, prefix="value", naming_rules=_CaseInsensitiveRules()) - - assert name == "value_3" - assert counter == 4 - - -def test_random_string_has_the_requested_length_and_alphabet(): - value = random_string(24) - - assert len(value) == 24 - assert value.isalnum() - assert value == value.casefold() diff --git a/tests/fortran/memory_management/codegen/test_native_handle_planning.py b/tests/fortran/memory_management/codegen/test_native_handle_planning.py index 9b42d665b..53f8e8d47 100644 --- a/tests/fortran/memory_management/codegen/test_native_handle_planning.py +++ b/tests/fortran/memory_management/codegen/test_native_handle_planning.py @@ -5,21 +5,14 @@ import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.ownership import CodegenAction, ObjectKind, PythonBarrierAction from prik.policy.completion import complete_semantic_policies from prik.policy.models import ( - ArgumentHandoffMode, NativeArrayDescriptorInterop, - NativeArrayDescriptorKind, NativeArrayDescriptorOwnership, - NativeArrayDefaultConstruction, NativeArrayDestroyBehavior, NativeArrayOperation, - NativeArrayOutputProjection, - NativeArrayOwnerStorage, NativeArrayRelease, NativeArrayResultAllocation, - NativeArraySourceKind, NativeDescriptorHandoffABI, ) from prik.pipeline.wrapper import WrapperGenerator @@ -133,13 +126,6 @@ def _functions(plan): return {function.binding.python_name: function for function in plan.namespaces[0].functions} -def _generated_c_function(source: str, name: str) -> str: - signature = f"static PyObject * {name}(PyObject * self, PyObject * args) {{" - start = source.index(signature) - end = source.index("\n}\n", start) + len("\n}\n") - return source[start:end] - - def _module_handle_plan(): module = parse_pyi_text( """ @@ -171,270 +157,111 @@ def _module_handle_plan(): return WrapperPlanner().build(module) -def test_native_handle_plans_keep_datatype_specific_state(): +def _handle_row(handle): + default_operations = handle.default_handle.operations if handle.default_handle else () + return ( + handle.descriptor_kind.value, + handle.handoff.abi.value, + handle.descriptor_ownership.value, + handle.owner_storage.value, + handle.default_handle.construction.value if handle.default_handle else "none", + NativeArrayOperation.DESTROY in handle.operations, + NativeArrayOperation.DESTROY in default_operations, + ) + + +def test_native_handle_plans_select_one_descriptor_abi_and_release_owner_per_origin(): + """Every descriptor slot names how its descriptor crosses and who destroys it. + + Neither an allocatable nor a pointer argument is established from C. An + allocatable cannot be: the standard requires a null base address for that + attribute. A pointer could be, but a descriptor C built is not the + caller's entity, so a callee that re-associates the dummy would change only + that copy. Both therefore take the descriptor the Fortran runtime made, + and a caller-created handle lazily gets wrapper storage its finalizer + destroys. Owned results destroy their own descriptor; borrowed module + descriptors are never destroyed by a handle. + """ plan = _native_handle_plan() functions = _functions(plan) + rows = {name: _handle_row(functions[name].arguments[0].native_array_handle) for name in _ARGUMENT_ROWS} + rows.update({name: _handle_row(functions[name].results[0].native_array_handle) for name in _RESULT_ROWS}) + module_plan = _module_handle_plan() + rows.update({variable.symbol_name: _handle_row(variable.native_array_handle) for variable in module_plan.variables}) - normal = functions["normal"].arguments[0] - assert normal.object_kind is ObjectKind.NUMPY_ARRAY - assert normal.native_array_handle is None - assert normal.native_array_actual is not None - assert normal.native_array_actual.accepted_sources == ( - NativeArraySourceKind.NDARRAY, - NativeArraySourceKind.ALLOCATABLE_HANDLE, - NativeArraySourceKind.POINTER_HANDLE, - ) - assert normal.native_array_actual.require_contiguous is True - assert normal.entrypoint.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER - assert normal.array is normal.projected_call_slot.array - - alloc = functions["alloc"].arguments[0] - pointer = functions["pointer"].arguments[0] - # Neither kind is established from C. An allocatable cannot be: the - # standard requires a null base address for that attribute. A pointer - # could be, but a descriptor C built is not the caller's entity, so a - # callee that re-associates the dummy would change only that copy. Both - # therefore take the descriptor the Fortran runtime made. - for argument, descriptor_kind in ( - (alloc, NativeArrayDescriptorKind.ALLOCATABLE), - (pointer, NativeArrayDescriptorKind.POINTER), - ): - handle = argument.native_array_handle - assert handle is not None - assert handle is argument.projected_call_slot.native_array_handle - assert handle.descriptor_kind is descriptor_kind - assert handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR - assert handle.default_handle.construction is NativeArrayDefaultConstruction.LAZY_OWNED_DESCRIPTOR - assert handle.default_handle.descriptor_ownership is NativeArrayDescriptorOwnership.OWNED - # Lazily attached storage is the wrapper's, so the plan names the slot - # the generated binder allocates and the handle's finalizer releases. - assert handle.default_handle.owner_storage_role == f"{argument.owner_path}:default-owner-storage" - assert NativeArrayOperation.DESTROY in handle.default_handle.operations - assert argument.binding.python_action is PythonBarrierAction.WRAPPER_INSTANCE - assert argument.entrypoint.handoff_mode is ArgumentHandoffMode.NATIVE_DESCRIPTOR + assert rows == {**_ARGUMENT_ROWS, **_RESULT_ROWS, **_MODULE_ROWS} optional = functions["optional"].arguments[0] - assert optional.native_array_handle is not None assert optional.native_array_handle.optional_absent is True assert optional.native_array_handle.handoff.presence_role == optional.entrypoint.presence_role - assert alloc.native_array_handle is not None - assert alloc.native_array_handle.handoff.presence_role is None - - replacement = functions["replace"].arguments[0] - assert replacement.native_array_handle is not None - assert replacement.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR - assert replacement.native_array_handle.output_projection is NativeArrayOutputProjection.PROJECTED_HANDLE - assert ( - replacement.native_array_handle.default_handle.construction - is NativeArrayDefaultConstruction.LAZY_OWNED_DESCRIPTOR + assert functions["alloc"].arguments[0].native_array_handle.handoff.presence_role is None + assert functions["maybe_make"].results[0].native_array_handle.result_allocation is ( + NativeArrayResultAllocation.MAYBE_UNALLOCATED ) - assert replacement.native_array_handle.default_handle.owner_storage_role is not None - assert NativeArrayOperation.DESTROY in replacement.native_array_handle.default_handle.operations - assert replacement.binding.codegen_action is CodegenAction.IN_PLACE_ARGUMENT - - owned = functions["make"].results[0] - assert owned.native_array_handle is not None - assert owned.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE - assert owned.native_array_handle.descriptor_ownership is NativeArrayDescriptorOwnership.OWNED - assert owned.native_array_handle.result_allocation is NativeArrayResultAllocation.ALWAYS_ALLOCATED - assert owned.native_array_handle.handoff.owner_storage_role is not None - assert NativeArrayOperation.DESTROY in owned.native_array_handle.operations - - maybe_owned = functions["maybe_make"].results[0] - assert maybe_owned.native_array_handle is not None - assert maybe_owned.native_array_handle.result_allocation is NativeArrayResultAllocation.MAYBE_UNALLOCATED - - owned_matrix = functions["make_matrix"].results[0] - assert owned_matrix.native_array_handle is not None - assert owned_matrix.native_array_handle.array.rank == 2 - assert owned_matrix.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE - assert owned_matrix.native_array_handle.descriptor_ownership is NativeArrayDescriptorOwnership.OWNED - + assert functions["select_pointer"].results[0].source_kind == "hidden_output" deferred = functions["deferred"].results[0] assert deferred.native_array_handle is None - assert deferred.scalar_descriptor is not None assert deferred.scalar_descriptor.runtime_length is True - assert deferred.scalar_descriptor.presence_role == f"{deferred.owner_path}:present" - - names = functions["make_names"].results[0] - assert names.native_array_handle is not None - assert names.datatype_family.value == "string" - assert names.array.itemsize is None - assert names.native_array_handle.owner_storage is NativeArrayOwnerStorage.FORTRAN_OWNER - assert names.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.FORTRAN_OWNER - assert NativeArrayOperation.ELEMENT_LENGTH in names.native_array_handle.operations - # A deferred length can be resized because the plan carries the width the - # allocation needs; the width is what the entity cannot supply itself. - assert NativeArrayOperation.RESIZE in names.native_array_handle.operations - assert names.native_array_handle.element_length_argument is True - - replacement_names = functions["replace_names"].arguments[0] - assert replacement_names.native_array_handle is not None - assert replacement_names.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR - assert ( - replacement_names.native_array_handle.default_handle.construction - is NativeArrayDefaultConstruction.LAZY_FORTRAN_OWNER - ) - assert replacement_names.native_array_handle.default_handle.owner_storage_role is not None - assert NativeArrayOperation.ELEMENT_LENGTH in replacement_names.native_array_handle.operations - assert plan.required_headers == ("ISO_Fortran_binding.h",) - - pointer_result = functions["make_pointer"].results[0] - assert pointer_result.native_array_handle is not None - assert pointer_result.native_array_handle.descriptor_kind is NativeArrayDescriptorKind.POINTER - assert pointer_result.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE - assert pointer_result.native_array_handle.descriptor_ownership is NativeArrayDescriptorOwnership.OWNED - assert pointer_result.native_array_handle.result_allocation is NativeArrayResultAllocation.NOT_APPLICABLE - assert pointer_result.native_array_handle.target_lifetime == "module" - assert NativeArrayOperation.ASSOCIATE in pointer_result.native_array_handle.operations - assert NativeArrayOperation.ASSOCIATED in pointer_result.native_array_handle.operations - assert NativeArrayOperation.NULLIFY in pointer_result.native_array_handle.operations - assert NativeArrayOperation.CONTIGUOUS in pointer_result.native_array_handle.operations - assert NativeArrayOperation.DESTROY in pointer_result.native_array_handle.operations - - pointer_output = functions["select_pointer"].results[0] - assert pointer_output.source_kind == "hidden_output" - assert pointer_output.native_array_handle is not None - assert pointer_output.native_array_handle.descriptor_kind is NativeArrayDescriptorKind.POINTER - assert pointer_output.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE - assert pointer_output.projected_call_slot is not None - assert pointer_output.projected_call_slot.source_kind == "result" - - managed_pointer = functions["make_managed_pointer"].results[0] - assert managed_pointer.native_array_handle is not None - assert { - NativeArrayOperation.ALLOCATE, - NativeArrayOperation.DEALLOCATE, - NativeArrayOperation.RESIZE, - }.issubset(managed_pointer.native_array_handle.operations) - - -def test_module_variables_use_borrowed_handle_plans_and_operation_sets(): - plan = _module_handle_plan() - variables = {variable.symbol_name: variable for variable in plan.variables} - allocatable = variables["module_allocatable"].native_array_handle - plain = variables["plain_allocatable"].native_array_handle - names = variables["module_names"].native_array_handle - pointer = variables["module_pointer"].native_array_handle - - assert allocatable is not None - assert plain is not None - assert names is not None - assert pointer is not None - assert allocatable.borrowed is plain.borrowed is names.borrowed is pointer.borrowed is True - assert allocatable.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE - assert plain.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE - assert names.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE - assert pointer.descriptor_kind is NativeArrayDescriptorKind.POINTER - assert NativeArrayOperation.DEALLOCATE in allocatable.operations - assert NativeArrayOperation.RESIZE in allocatable.operations - assert NativeArrayOperation.NULLIFY in pointer.operations - assert NativeArrayOperation.ASSOCIATE in pointer.operations - assert NativeArrayOperation.CONTIGUOUS in pointer.operations - assert NativeArrayOperation.DESTROY not in allocatable.operations - assert NativeArrayOperation.ELEMENT_LENGTH in names.operations - assert NativeArrayOperation.RESIZE in names.operations - assert names.element_length_argument is True - # Every other entity allocates from its shape alone. - assert allocatable.element_length_argument is False - assert NativeArrayOperation.DESTROY not in pointer.operations - # A module allocatable reads its own descriptor whether or not it is a - # target, so `Aliased` selects the same interop and headers as a plain one. - assert allocatable.extraction_action.value == "descriptor_view" - assert allocatable.descriptor_interop is NativeArrayDescriptorInterop.MODULE_ALLOCATABLE_C_DESCRIPTOR - assert allocatable.required_headers == ("ISO_Fortran_binding.h",) - assert plain.extraction_action.value == "descriptor_view" - assert plain.descriptor_interop is NativeArrayDescriptorInterop.MODULE_ALLOCATABLE_C_DESCRIPTOR - assert plain.required_headers == ("ISO_Fortran_binding.h",) - assert pointer.required_headers == ("ISO_Fortran_binding.h",) - assert plan.required_headers == ("ISO_Fortran_binding.h",) - - -def test_deferred_character_module_handles_use_runtime_element_length(): - """The live descriptor supplies a deferred character element width.""" - artifacts = WrapperGenerator().generate(_module_handle_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - - assert "out->result = PyLong_FromLongLong((long long)source->elem_len)" in c_source - assert "prik_native_array_read_element_length" in c_source - - -def test_generated_native_handle_artifacts_follow_one_typed_action_vocabulary(): + # Only a deferred length needs a width to allocate from its extents. + assert functions["make_names"].results[0].native_array_handle.element_length_argument is True + assert functions["make"].results[0].native_array_handle.element_length_argument is False + assert plan.required_headers == module_plan.required_headers == ("ISO_Fortran_binding.h",) + + +_LAZY_ARGUMENT = ("direct_standard_descriptor", "borrowed", "borrowed_entity", "lazy_owned_descriptor", False, True) +_OWNED_RESULT = ("owned_result_storage", "owned", "c_descriptor", "none", True, False) +_BORROWED_MODULE = ("direct_standard_descriptor", "borrowed", "borrowed_entity", "none", False, False) +_ARGUMENT_ROWS = { + "alloc": ("allocatable", *_LAZY_ARGUMENT), + "pointer": ("pointer", *_LAZY_ARGUMENT), + "optional": ("allocatable", *_LAZY_ARGUMENT), + "replace": ("allocatable", *_LAZY_ARGUMENT), + "replace_names": ( + "allocatable", + "direct_standard_descriptor", + "borrowed", + "fortran_owner", + "lazy_fortran_owner", + False, + True, + ), +} +_RESULT_ROWS = { + "make": ("allocatable", *_OWNED_RESULT), + "maybe_make": ("allocatable", *_OWNED_RESULT), + "make_matrix": ("allocatable", *_OWNED_RESULT), + "make_names": ("allocatable", "fortran_owner", "owned", "fortran_owner", "none", True, False), + "make_pointer": ("pointer", *_OWNED_RESULT), + "select_pointer": ("pointer", *_OWNED_RESULT), + "make_managed_pointer": ("pointer", *_OWNED_RESULT), +} +_MODULE_ROWS = { + "module_allocatable": ("allocatable", *_BORROWED_MODULE), + "plain_allocatable": ("allocatable", *_BORROWED_MODULE), + "module_names": ("allocatable", *_BORROWED_MODULE), + "module_pointer": ("pointer", *_BORROWED_MODULE), +} + + +def test_absent_optional_allocatable_gets_an_unallocated_placeholder_descriptor(): + """The absent branch pairs a null-address allocatable descriptor with a present flag. + + A null base address is the only form the standard lets C establish for this + attribute (ifx rejects any other), and absence is when there is nothing to + point at. + """ artifacts = WrapperGenerator().generate(_native_handle_plan()) c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - assert artifacts.required_headers == ("ISO_Fortran_binding.h",) - assert "prik_describe_numpy_array(" in c_source - assert '"_native_array_backend_for_binding_positional"' in c_source - assert '"_native_array_handle_from_generated_dispatch"' in c_source - assert '"_bind_contract_native_array_handle"' in c_source - assert "prik_native_array_backend_capsule_new(" in c_source - assert "prik_native_array_backend_for_descriptor(" in c_source - assert "PRIK_NATIVE_ARRAY_KIND_ALLOCATABLE" in c_source - assert "PRIK_NATIVE_ARRAY_KIND_POINTER" in c_source - assert "prik_native_array_backend_release(owner_backend)" in c_source - assert ("bound_values_native_backend = prik_native_array_backend_for_descriptor(bound_values_item") in c_source - assert "prik_bind_default_memory_handles_replace_values" in c_source - assert "prik_owned_memory_handles_replace_values_dispatch" in c_source - assert "bound_values_default_binder" in c_source - assert "CFI_CDESC_T(1)" in c_source - assert "CFI_CDESC_T(2)" in c_source - assert "real(c_double), allocatable, dimension(:) :: values" in bridge_source - assert "real(c_double), pointer, dimension(:) :: values" in bridge_source - assert "real(c_double), allocatable, dimension(:, :) :: result_value" in bridge_source optional_start = bridge_source.index("function bind_c_optional(") - optional_end = bridge_source.index("end function bind_c_optional", optional_start) - optional_bridge = bridge_source[optional_start:optional_end] - assert "real(c_double), allocatable, dimension(:) :: values" in optional_bridge + optional_bridge = bridge_source[ + optional_start : bridge_source.index("end function bind_c_optional", optional_start) + ] assert "type(c_ptr), value :: bound_values_present" in optional_bridge - assert "real(c_double), allocatable, dimension(:), optional :: values" not in optional_bridge optional_c_start = c_source.index("static PyObject * wrap_optional(") - optional_c_end = c_source.index("static PyObject * wrap_replace(", optional_c_start) - optional_binding = c_source[optional_c_start:optional_c_end] - assert "} else {" in optional_binding - # The absent branch hands the bridge an unallocated placeholder to pair - # with its present flag. A null base address is the only form the standard - # lets C establish for this attribute, and absence is when there is nothing - # to point at. + optional_binding = c_source[optional_c_start : c_source.index("static PyObject * wrap_replace(", optional_c_start)] assert "CFI_establish((CFI_cdesc_t *)&bound_values_storage, NULL, CFI_attribute_allocatable" in optional_binding - assert "bound_values = (CFI_cdesc_t *)&bound_values_storage;" in optional_binding - assert "result_value = native_make(n)" in bridge_source - assert "result_value = native_make_matrix(n, m)" in bridge_source - assert "call prik_collect_allocatable_array_result(native_maybe_make(n), result)" in bridge_source - assert "if (allocated(value)) then" in bridge_source - assert "call move_alloc(value, result)" in bridge_source - assert "_deallocate(owner_descriptor);" in c_source - assert "_destroy(owner_descriptor);" in c_source - assert "owner_backend->with_descriptor(owner_backend->context, prik_native_array_read_shape" in c_source - assert "character(kind=c_char, len=:), allocatable :: value_value" in bridge_source - assert "CFI_type_char" in c_source - assert "character(kind=c_char, len=:), allocatable, dimension(:) :: names" in bridge_source - assert "result_owner_status = CFI_establish(result, NULL, CFI_attribute_pointer" in c_source - # An owned result publishes descriptor storage, and says so: v2 discriminates - # the context rather than letting ownership imply what it points at. - assert ( - "PRIK_NATIVE_ARRAY_KIND_POINTER, PRIK_NATIVE_ARRAY_ATTRIBUTE_POINTER, 1, " - "(uint32_t)sizeof(CFI_CDESC_T(1)), CFI_type_double, " - "sizeof(double), PRIK_NATIVE_ARRAY_CONTEXT_DESCRIPTOR, 0, 0, result" in c_source - ) - - -def test_owned_descriptor_handles_publish_one_dispatcher_and_capability_tuple(): - artifacts = WrapperGenerator().generate(_native_handle_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - - dispatch = _generated_c_function( - c_source, - "prik_owned_memory_handles_make_return_dispatch", - ) - assert 'strcmp(operation, "allocated") == 0' in dispatch - assert 'strcmp(operation, "shape") == 0' in dispatch - assert 'strcmp(operation, "to_numpy") == 0' in dispatch - assert 'strcmp(operation, "destroy") == 0' in dispatch - assert "owner_backend" in dispatch - assert "owner_descriptor" in dispatch - assert 'Py_BuildValue("(ssssss)", "allocated", "deallocate", "destroy", "resize", "shape", "to_numpy")' in c_source @pytest.mark.parametrize( diff --git a/tests/fortran/memory_management/runtime/test_handle_lifecycle.py b/tests/fortran/memory_management/runtime/test_handle_lifecycle.py index a444a17ef..ae326e7b5 100644 --- a/tests/fortran/memory_management/runtime/test_handle_lifecycle.py +++ b/tests/fortran/memory_management/runtime/test_handle_lifecycle.py @@ -1,162 +1,80 @@ -"""Runtime ownership, factory, close, and finalizer behavior for native handles.""" +"""Runtime release, finalizer, and fail-closed construction of native handles. + +These paths cannot be reached through a correct generated extension, or only +show up as a leak or a double free there, so they are driven with test doubles. +Observable handle behaviour is proved by the allocatable and pointer +end-to-end suites. +""" import gc + import numpy as np import pytest from prik.runtime.handles import ( AllocatableArray, PointerArray, - _native_array_backend_for_binding, _native_array_handle_from_generated_dispatch, ) from tests.fortran._support.native_array_handles import ( - _ArrayState, - _common_ops, _generated_handle_dispatch, _handle_dispatch, ) -def test_generated_handle_factory_adapts_one_dispatcher_to_runtime_protocol(): - owner = object() - value = np.arange(3, dtype=np.float64) - calls = [] - - def shape(): - calls.append(("shape", ())) - return (3,) - - def allocated(): - calls.append(("allocated", ())) - return True - - def to_numpy(): - calls.append(("to_numpy", ())) - return value - - operations = { - "shape": shape, - "allocated": allocated, - "to_numpy": to_numpy, - } - handle = _native_array_handle_from_generated_dispatch( - "allocatable", - "float64", - 1, - _generated_handle_dispatch(operations), - operations, - owner=owner, - descriptor_ownership="borrowed", - to_numpy_policy="borrowed_view", - generation=9, +def _owned_allocatable(destroy): + return AllocatableArray( + dtype="float64", + rank=1, + **_handle_dispatch( + { + "shape": lambda _handle: (1,), + "allocated": lambda _handle: True, + "destroy": destroy, + } + ), + descriptor_ownership="owned", + to_numpy_policy="unsupported", ) - assert isinstance(handle, AllocatableArray) - assert isinstance(handle.dtype, np.dtype) - assert handle.dtype == np.dtype("float64") - assert handle.owner is owner - assert handle.generation == 9 - assert handle.shape == (3,) - assert handle.allocated is True - assert handle.to_numpy() is value - assert {name for name, _args in calls} == {"allocated", "shape", "to_numpy"} - assert all(args == () for _name, args in calls) - -def test_generated_handle_factory_splats_shape_operations_to_scalar_extents(): +@pytest.mark.parametrize("release", ["close_twice_then_collect", "finalizer_only"]) +def test_owned_handle_destroys_its_descriptor_exactly_once(release: str): calls = [] - operations = { - "shape": lambda: (2, 3), - "allocated": lambda: True, - "resize": lambda *extents: calls.append(("resize", extents)), - } - handle = _native_array_handle_from_generated_dispatch( - "allocatable", - "float64", - 2, - _generated_handle_dispatch(operations), - operations, - to_numpy_policy="unsupported", - ) - - handle.resize((4, 5)) + handle = _owned_allocatable(lambda _handle: calls.append("destroy")) + + if release == "close_twice_then_collect": + assert handle.closed is False + assert handle.close() is None + assert handle.close() is None + assert handle.closed is True + with pytest.raises(ReferenceError, match="allocatable handle is closed"): + _ = handle.shape + with pytest.raises(ReferenceError, match="allocatable handle is closed"): + handle.to_numpy() + del handle + gc.collect() - assert calls == [("resize", (4, 5))] + assert calls == ["destroy"] -def test_generated_owned_handle_factory_passes_persistent_owner_to_every_operation(): +def test_owned_handle_close_marks_closed_when_destroy_raises(): calls = [] - owner = object() - value = np.arange(3, dtype=np.float64) - - def operation(name, result=None): - def call(received_owner, *args): - calls.append((name, received_owner, args)) - return result - - return call - - operations = { - "shape": operation("shape", (3,)), - "allocated": operation("allocated", True), - "to_numpy": operation("to_numpy", value), - "resize": operation("resize"), - "destroy": operation("destroy"), - } - handle = _native_array_handle_from_generated_dispatch( - "allocatable", - "float64", - 1, - _generated_handle_dispatch(operations), - operations, - owner=owner, - descriptor_ownership="owned", - native_backend=owner, - ) - assert handle.shape == (3,) - assert handle.allocated is True - assert handle.to_numpy() is value - assert _native_array_backend_for_binding( - handle, - descriptor_kind="allocatable", - expected_dtype=np.float64, - expected_rank=1, - ) == (owner,) - handle.resize((5,)) - handle.close() - - assert {name for name, _owner, _args in calls} == { - "allocated", - "shape", - "to_numpy", - "resize", - "destroy", - } - assert all(received_owner is owner for _name, received_owner, _args in calls) - assert ("resize", owner, (np.int64(5),)) in calls - assert calls.count(("destroy", owner, ())) == 1 - - -def test_generated_handle_resolves_deferred_character_dtype_from_runtime_element_length(): - state = {"itemsize": 3} - operations = { - "shape": lambda: (2,), - "element_length": lambda: state["itemsize"], - "allocated": lambda: True, - "to_numpy": lambda: np.array([b"red", b"sky"], dtype=f"S{state['itemsize']}"), - } - handle = _native_array_handle_from_generated_dispatch( - "allocatable", - None, - 1, - _generated_handle_dispatch(operations), - operations, - ) + def destroy(_handle): + calls.append("destroy") + raise RuntimeError("boom") + + handle = _owned_allocatable(destroy) - assert handle.dtype == np.dtype("S3") - state["itemsize"] = 5 - assert handle.dtype == np.dtype("S5") + with pytest.raises(RuntimeError, match="boom"): + handle.close() + assert handle.closed is True + assert handle.close() is None + + del handle + gc.collect() + + assert calls == ["destroy"] def test_generated_owned_handle_factory_releases_owner_once_when_construction_fails(): @@ -186,141 +104,219 @@ def destroy(received_owner): assert calls == [("destroy", owner)] -def test_generated_handle_factory_rejects_an_invalid_descriptor_kind(): - ops = { - "shape": lambda: (1,), - "allocated": lambda: True, - "to_numpy": lambda: np.zeros(1, dtype=np.float64), - } - - with pytest.raises(ValueError, match="generated native array handle kind"): - _native_array_handle_from_generated_dispatch( - "target", - "float64", - 1, - _generated_handle_dispatch(ops), - ops, - ) - - -def test_owned_handle_close_calls_destroy_once_and_blocks_later_use(): +@pytest.mark.parametrize("kind", ["allocatable", "pointer"]) +def test_borrowed_handle_close_and_finalizer_do_not_destroy_native_storage(kind: str): calls = [] - state = _ArrayState(shape=(2,), value=np.zeros(2, dtype=np.float64)) - handle = AllocatableArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - **_common_ops(state), - "allocated": lambda _handle: True, - "destroy": lambda _handle: calls.append(("destroy", state.shape, state.value)), - } - ), - descriptor_ownership="owned", - ) - - assert handle.closed is False - assert handle.close() is None - assert handle.closed is True - assert handle.close() is None - assert calls == [("destroy", (2,), state.value)] - with pytest.raises(ReferenceError, match="allocatable handle is closed"): - _ = handle.shape - with pytest.raises(ReferenceError, match="allocatable handle is closed"): - handle.to_numpy() - - -def test_owned_handle_close_marks_closed_when_destroy_raises(): - calls = [] - - def destroy(_handle): - calls.append("destroy") - raise RuntimeError("boom") - - handle = AllocatableArray( + owner = object() + state_operation = {"allocatable": "allocated", "pointer": "associated"}[kind] + handle_type = {"allocatable": AllocatableArray, "pointer": PointerArray}[kind] + handle = handle_type( dtype="float64", rank=1, **_handle_dispatch( { "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - "destroy": destroy, + state_operation: lambda _handle: True, + "nullify": lambda _handle: None, + "destroy": lambda _handle: calls.append("destroy"), } ), - descriptor_ownership="owned", + owner=owner, to_numpy_policy="unsupported", ) - with pytest.raises(RuntimeError, match="boom"): - handle.close() - assert handle.closed is True assert handle.close() is None - + assert handle.closed is False + assert handle.owner is owner del handle gc.collect() - assert calls == ["destroy"] - - -def test_owned_handle_finalizer_calls_destroy_once(): - calls = [] - - handle = AllocatableArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - "destroy": lambda _handle: calls.append("destroy"), - } - ), - descriptor_ownership="owned", - to_numpy_policy="unsupported", - ) + assert calls == [] - del handle - gc.collect() - assert calls == ["destroy"] +def _operations(*names): + return {name: (lambda *_args: None) for name in names} -def test_owned_handle_construction_requires_generated_destroy_operation(): - with pytest.raises(ValueError, match="owned native array handle requires generated operation 'destroy'"): - AllocatableArray( +@pytest.mark.parametrize( + ("construct", "message"), + [ + pytest.param( + lambda: AllocatableArray( + dtype="float64", + rank=1, + **_handle_dispatch(_operations("shape", "allocated")), + descriptor_ownership="owned", + to_numpy_policy="unsupported", + ), + "owned native array handle requires generated operation 'destroy'", + id="owned-without-destroy", + ), + pytest.param( + lambda: AllocatableArray(dtype="float64", rank=1, **_handle_dispatch({})), + "requires generated operation 'shape'", + id="missing-shape", + ), + pytest.param( + lambda: AllocatableArray(dtype="float64", rank=1, **_handle_dispatch(_operations("shape", "to_numpy"))), + "requires generated operation 'allocated'", + id="allocatable-without-allocated", + ), + pytest.param( + lambda: PointerArray(dtype="float64", rank=1, **_handle_dispatch(_operations("shape", "associated"))), + "requires generated operation 'nullify'", + id="pointer-without-nullify", + ), + pytest.param( + lambda: AllocatableArray( + dtype="float64", + rank=1, + **_handle_dispatch(_operations("shape", "allocated")), + to_numpy_policy="borrowed_view", + ), + "requires generated operation 'to_numpy'", + id="view-policy-without-extraction", + ), + pytest.param( + lambda: AllocatableArray( + dtype="float64", + rank=1, + **_handle_dispatch(_operations("shape", "allocated")), + descriptor_ownership="temporary", + ), + "descriptor_ownership must be 'borrowed' or 'owned'", + id="unknown-ownership", + ), + pytest.param( + lambda: AllocatableArray( + dtype="float64", + rank=1, + **_handle_dispatch(_operations("shape", "allocated")), + to_numpy_policy="maybe_copy", + ), + "to_numpy_policy must be one of", + id="unknown-extraction-policy", + ), + pytest.param( + lambda: _native_array_handle_from_generated_dispatch( + "target", + "float64", + 1, + _generated_handle_dispatch({}), + _operations("shape", "allocated", "to_numpy"), + ), + "generated native array handle kind", + id="unknown-descriptor-kind", + ), + ], +) +def test_handle_construction_rejects_an_incomplete_generated_contract(construct, message: str): + with pytest.raises(ValueError, match=message): + construct() + + +@pytest.mark.parametrize( + ("kind", "call", "operation"), + [ + ("allocatable", lambda handle: handle.deallocate(), "deallocate"), + ("allocatable", lambda handle: handle.resize(2), "resize"), + ("pointer", lambda handle: handle.allocate((3,)), "allocate"), + ("pointer", lambda handle: handle.deallocate(), "deallocate"), + ("pointer", lambda handle: handle.resize((4,)), "resize"), + ], +) +def test_operations_outside_the_completed_capabilities_are_refused(kind: str, call, operation: str): + """A handle offers only what completed policy granted, e.g. no target release by default.""" + if kind == "allocatable": + handle = AllocatableArray( + dtype="float64", + rank=1, + **_handle_dispatch({"shape": lambda _handle: (1,), "allocated": lambda _handle: True}), + to_numpy_policy="unsupported", + ) + else: + handle = PointerArray( dtype="float64", rank=1, **_handle_dispatch( { "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, + "associated": lambda _handle: True, + "nullify": lambda _handle: None, } ), - descriptor_ownership="owned", to_numpy_policy="unsupported", ) + with pytest.raises(NotImplementedError, match=f"{kind} handle operation '{operation}' is not available"): + call(handle) -def test_borrowed_handle_close_and_finalizer_do_not_destroy_native_storage(): - calls = [] - handle = PointerArray( - dtype="float64", +@pytest.mark.parametrize( + ("shape", "result", "policy", "error", "message"), + [ + pytest.param( + (2,), [1.0, 2.0], "descriptor_view", TypeError, "must return a NumPy array or None", id="not-numpy" + ), + pytest.param( + (2,), + np.zeros((1, 2), dtype=np.float64), + "descriptor_view", + ValueError, + "to_numpy result rank 2 does not match declared rank 1", + id="wrong-rank", + ), + pytest.param( + (2,), np.zeros(2, dtype=np.int32), "descriptor_view", TypeError, "to_numpy result dtype", id="wrong-dtype" + ), + pytest.param( + (4,), + np.arange(8, dtype=np.float64)[::2], + "contiguous_view", + ValueError, + "must be contiguous", + id="strided-under-contiguous-policy", + ), + ], +) +def test_generated_views_that_disagree_with_the_declared_handle_are_refused(shape, result, policy, error, message): + """A view over native memory is only exposed when it matches what the handle declares.""" + handle = AllocatableArray( + dtype=np.dtype(np.float64), rank=1, **_handle_dispatch( { - "shape": lambda _handle: (1,), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - "destroy": lambda _handle: calls.append("destroy"), + "shape": lambda _handle: shape, + "to_numpy": lambda _handle: result, + "allocated": lambda _handle: True, } ), - to_numpy_policy="unsupported", + to_numpy_policy=policy, ) - assert handle.close() is None - assert handle.closed is False - del handle - gc.collect() + with pytest.raises(error, match=message): + handle.to_numpy() - assert calls == [] + +def test_generated_shapes_are_validated_against_rank_and_sign(): + reported = {"shape": (-1,)} + handle = AllocatableArray( + dtype=np.dtype(np.float64), + rank=1, + **_handle_dispatch( + { + "shape": lambda _handle: reported["shape"], + "allocated": lambda _handle: True, + "resize": lambda _handle, _shape: None, + } + ), + to_numpy_policy="unsupported", + ) + + with pytest.raises(ValueError, match="non-negative"): + _ = handle.shape + with pytest.raises(ValueError, match="non-negative"): + handle.resize(-1) + reported["shape"] = (4, 2) + with pytest.raises(ValueError, match="shape rank 2 does not match declared rank 1"): + _ = handle.shape diff --git a/tests/fortran/modules/codegen/test_module_array_view_lowering.py b/tests/fortran/modules/codegen/test_module_array_view_lowering.py index a4ed679e2..cfadb5321 100644 --- a/tests/fortran/modules/codegen/test_module_array_view_lowering.py +++ b/tests/fortran/modules/codegen/test_module_array_view_lowering.py @@ -69,7 +69,7 @@ def test_addressable_module_array_takes_its_address_directly(): assert "capture_array_address" not in getter -@pytest.mark.parametrize("python_name", ["plain", "counts", "labels"]) +@pytest.mark.parametrize("python_name", ["plain", "labels"]) def test_ordinary_module_array_captures_its_address_in_c(python_name): """Without `target`, the address is taken on the C side, never by `c_loc`. @@ -99,18 +99,24 @@ def test_captured_address_declares_one_typeless_c_interface(): assert captures[0].parameters[0].attributes == ("dimension(*)",) -def test_capture_helper_is_declared_only_where_an_array_needs_it(): - """A module whose arrays are all addressable declares no capture interface.""" +def test_an_all_addressable_module_needs_no_capture_primitive(): + """A module whose arrays are all addressable declares no capture interface. + + Neither side pulls in the capture symbol: the bridge declares no interface + for it and the binding does not opt into the bundled definition. + """ module = parse_pyi_text( "addressable: Annotated[Float64[4], Aliased]\n", module_name="array_state", ) complete_semantic_policies(module) - bridge = FortranBridgeGenerator() - emitted = bridge.visit(WrapperPlanner().build(module)) + plan = WrapperPlanner().build(module) + emitted = FortranBridgeGenerator().visit(plan) names = [procedure.name for interface in emitted.interfaces for procedure in interface.procedures] assert "prik_capture_address" not in names + binding = CBindingGenerator().binding_module(plan) + assert not any(define.name == "PRIK_BINDING_CAPTURE_ADDRESS" for define in binding.defines) def test_binding_opts_into_the_bundled_capture_primitive(): @@ -127,18 +133,6 @@ def test_binding_opts_into_the_bundled_capture_primitive(): assert not any(function.name == "prik_capture_address" for function in binding.functions) -def test_binding_omits_the_capture_primitive_when_no_array_needs_it(): - """An extension whose arrays are all addressable pulls in no capture symbol.""" - module = parse_pyi_text( - "addressable: Annotated[Float64[4], Aliased]\n", - module_name="array_state", - ) - complete_semantic_policies(module) - binding = CBindingGenerator().binding_module(WrapperPlanner().build(module)) - - assert not any(define.name == "PRIK_BINDING_CAPTURE_ADDRESS" for define in binding.defines) - - def _undecided_plan(): """Return a module-array plan whose address mechanism policy never selected.""" module = parse_pyi_text("plain: Float64[3]\n", module_name="array_state") diff --git a/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py b/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py index ef6e6ee5f..186fa9d7f 100644 --- a/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py +++ b/tests/fortran/modules/codegen/test_scalar_module_variable_lowering.py @@ -8,9 +8,6 @@ import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules -from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.policy.ownership import AssignmentMode, SetterAction from prik.policy.completion import complete_semantic_policies from prik.policy.models import ModuleGetterAction @@ -37,42 +34,6 @@ def _plan(): return WrapperPlanner().build(module) -def _computed_constant_plan(): - parsed = parse_fortran_project( - { - "computed_constants.f90": """ -module computed_constants - integer, parameter :: computed = kind(1.0) * 2 - character*1, parameter :: prefix = 'D' -end module computed_constants -""" - } - ) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name="computed_constants_wrapper") - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def _parameter_array_plan(): - parsed = parse_fortran_project( - { - "parameter_array.f90": """ -module parameter_array - use iso_fortran_env, only: real64 - real(real64), parameter :: dpmpar(3) = [epsilon(1.0_real64), tiny(1.0_real64), huge(1.0_real64)] -end module parameter_array -""" - } - ) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name="parameter_array_wrapper") - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - def _source(artifacts, suffix: str) -> str: return next(item.text for item in artifacts.sources if item.path.name.endswith(suffix)) @@ -94,77 +55,6 @@ def _replace_variable(plan, python_name: str, edit): return replace(plan, variables=variables, namespaces=namespaces) -def test_module_variable_plan_contains_only_completed_dispatch_facts(): - plan = _plan() - variables = {variable.bridge.native_name: variable for variable in plan.variables} - - assert variables["limit"].binding.getter_action is ModuleGetterAction.CONSTANT_VALUE - assert variables["limit"].binding.setter_action is SetterAction.OMIT - assert variables["limit"].bridge.native_assignment is AssignmentMode.NONE - assert variables["limit"].binding.constant_value == 12 - assert variables["counter"].binding.getter_action is ModuleGetterAction.DIRECT_VALUE - assert variables["counter"].binding.setter_action is SetterAction.WRITE_THROUGH - assert variables["counter"].bridge.native_assignment is AssignmentMode.VALUE_COPY - assert variables["counter"].binding.initializer == 3 - assert variables["target_scale"].bridge.native_assignment is AssignmentMode.VALUE_COPY - assert variables["optional_scale"].binding.getter_action is ModuleGetterAction.NATIVE_NULLABLE_SCALAR_VIEW - assert variables["optional_scale"].entrypoint.descriptor_kind == "allocatable" - assert variables["optional_scale"].binding.setter_action is SetterAction.WRITE_THROUGH - assert variables["optional_scale"].binding.native_assignment is AssignmentMode.ALLOCATING_COPY - assert variables["optional_scale"].bridge.native_assignment is AssignmentMode.ALLOCATING_COPY - assert variables["selected_scale"].entrypoint.descriptor_kind == "pointer" - assert variables["selected_scale"].binding.native_assignment is AssignmentMode.TARGET_COPY - assert variables["selected_scale"].bridge.native_assignment is AssignmentMode.TARGET_COPY - - -def test_symbolic_source_parameter_reuses_scalar_bridge_getter_for_module_initialization(): - plan = _computed_constant_plan() - variables = {variable.bridge.native_name: variable for variable in plan.variables} - computed = variables["computed"] - assert computed.binding.getter_action is ModuleGetterAction.NATIVE_CONSTANT_VALUE - assert computed.binding.constant_value is None - assert computed.entrypoint.getter_role == "computed_constants.computed:getter" - assert computed.binding.setter_action is SetterAction.OMIT - - artifacts = WrapperGenerator().generate(plan) - c_source = _source(artifacts, ".c") - fortran_source = _source(artifacts, ".f90") - assert "int32_t bind_c_get_computed(void);" in c_source - assert "int32_t constant_computed_constants_computed_value_0 = bind_c_get_computed();" in c_source - assert 'PyUnicode_FromString("D")' in c_source - assert "native_computed => computed" in fortran_source - assert "function bind_c_get_computed()" in fortran_source - assert "result = native_computed" in fortran_source - assert "bind_c_set_computed" not in c_source - assert "bind_c_set_computed" not in fortran_source - - -def test_parameter_array_uses_one_immutable_python_owned_import_snapshot(): - plan = _parameter_array_plan() - variable = next(variable for variable in plan.variables if variable.bridge.native_name == "dpmpar") - assert variable.binding.getter_action is ModuleGetterAction.NATIVE_CONSTANT_ARRAY_VALUE - assert variable.binding.setter_action is SetterAction.OMIT - assert variable.binding.constant_value is None - assert variable.array is not None - assert variable.array.shape == ("3",) - - artifacts = WrapperGenerator().generate(plan) - c_source = _source(artifacts, ".c") - fortran_source = _source(artifacts, ".f90") - assert "void * bind_c_get_dpmpar(int64_t * extent_0);" in c_source - assert "PyArray_EMPTY(1, constant_parameter_array_dpmpar_value_0_dimensions, NPY_FLOAT64, 1)" in c_source - assert "memcpy(PyArray_DATA((PyArrayObject *)constant_parameter_array_dpmpar_object_0)" in c_source - assert ( - "PyArray_CLEARFLAGS((PyArrayObject *)constant_parameter_array_dpmpar_object_0, NPY_ARRAY_WRITEABLE)" in c_source - ) - assert ( - 'PyModule_AddObject(namespace_parameter_array, "dpmpar", constant_parameter_array_dpmpar_object_0)' in c_source - ) - assert "real(c_double), allocatable, target, save, dimension(:) :: parameter_snapshot" in fortran_source - assert "parameter_snapshot = native_dpmpar" in fortran_source - assert "result = c_loc(parameter_snapshot)" in fortran_source - - def test_module_variable_visitors_consume_their_backend_owned_actions(): plan = _plan() counter = next(variable for variable in plan.variables if variable.bridge.native_name == "counter") @@ -205,11 +95,9 @@ def test_fortran_module_setter_rejects_unsupported_bridge_assignment(): @pytest.mark.parametrize( ("python_name", "assignment"), [ - ("counter", AssignmentMode.NONE), - ("counter", AssignmentMode.ALIAS), - ("optional_scale", AssignmentMode.NONE), - ("optional_scale", AssignmentMode.ALIAS), - ("limit", AssignmentMode.VALUE_COPY), + pytest.param("counter", AssignmentMode.ALIAS, id="direct-value-with-alias"), + pytest.param("optional_scale", AssignmentMode.NONE, id="descriptor-without-assignment"), + pytest.param("limit", AssignmentMode.VALUE_COPY, id="constant-with-assignment"), ], ) def test_module_setter_assignment_mismatch_fails_before_backend_preflight_or_lowering( @@ -248,28 +136,6 @@ def test_module_setter_assignment_mismatch_fails_before_backend_preflight_or_low fortran_printer.doprint.assert_not_called() -def test_module_variable_generators_dispatch_get_set_and_rejection_from_plan(): - artifacts = WrapperGenerator().generate(_plan()) - c_source = _source(artifacts, ".c") - fortran_source = _source(artifacts, ".f90") - - assert "scalar_state_root_module_property_setup_getattro" in c_source - assert "scalar_state_root_module_property_setup_setattro" in c_source - assert 'PyModule_AddObject(mod, "limit"' in c_source - assert "bind_c_set_counter(3);" in c_source - assert "return bind_c_get_counter();" not in c_source - assert "bind_c_get_counter()" in c_source - assert "bind_c_set_counter(value)" in c_source - assert "bind_c_set_optional_scale(value)" in c_source - assert "Module variable selected_scale has no pointer target" in c_source - assert "result = native_counter" in fortran_source - assert "native_counter = value" in fortran_source - assert "allocated(native_optional_scale)" in fortran_source - assert "associated(native_selected_scale)" in fortran_source - assert "native_optional_scale = value" in fortran_source - assert "native_selected_scale = value" in fortran_source - - def test_generated_support_procedure_symbol_is_shared_by_both_boundary_lowerers(): plan = _plan() procedure = next( diff --git a/tests/fortran/modules/end_to_end/test_logical_array_views.py b/tests/fortran/modules/end_to_end/test_logical_array_views.py index 14d274a07..01cfe9127 100644 --- a/tests/fortran/modules/end_to_end/test_logical_array_views.py +++ b/tests/fortran/modules/end_to_end/test_logical_array_views.py @@ -37,28 +37,26 @@ def logical_view(tmp_path_factory): return module -def test_a_one_byte_logical_is_a_numpy_boolean(logical_view): - """`logical(c_bool)` holds zero or one in one byte, which is `numpy.bool_`.""" +def test_a_logical_array_reports_the_width_its_elements_occupy(logical_view): + """`logical(c_bool)` holds zero or one in one byte, which is `numpy.bool_`. + + NumPy has no Boolean larger than a byte, so a wider kind states its width + as the integer of matching size instead. + """ assert logical_view.narrow.dtype == np.dtype(np.bool_) assert logical_view.narrow_alloc.to_numpy().dtype == np.dtype(np.bool_) assert logical_view.narrow.tolist() == [True, False, True, False] - -def test_a_wider_logical_reports_the_width_its_elements_occupy(logical_view): - """NumPy has no Boolean larger than a byte, so the width is stated instead.""" wide = logical_view.wide - assert wide.dtype == np.dtype(np.int32) assert wide.astype(bool).tolist() == [True, False, True, False] assert logical_view.wide_alloc.to_numpy().dtype == np.dtype(np.int32) -def test_logical_allocatable_handles_reach_matching_ordinary_dummies(logical_view): +def test_logical_handles_reach_matching_ordinary_dummies(logical_view): + """Allocatable and pointer handles are ordinary array actuals at either width.""" assert logical_view.count_narrow_actual(logical_view.narrow_alloc) == np.int32(2) assert logical_view.count_wide_actual(logical_view.wide_alloc) == np.int32(2) - - -def test_wide_logical_pointer_handles_reach_matching_ordinary_dummies(logical_view): assert logical_view.wide_pointer.dtype == np.dtype(np.int32) assert logical_view.wide_pointer.shape == (4,) assert logical_view.count_wide_actual(logical_view.wide_pointer) == np.int32(2) diff --git a/tests/fortran/modules/end_to_end/test_module_array_storage_forms.py b/tests/fortran/modules/end_to_end/test_module_array_storage_forms.py index 8f5b150d0..ab1618977 100644 --- a/tests/fortran/modules/end_to_end/test_module_array_storage_forms.py +++ b/tests/fortran/modules/end_to_end/test_module_array_storage_forms.py @@ -34,46 +34,39 @@ def array_forms(tmp_path_factory): return module -# A fixed shape is entirely in the declaration, so these are plain views; an -# allocatable or pointer carries runtime state, so those are handles. -@pytest.mark.parametrize( - ("name", "exposed_as", "dtype"), - [ - ("fixed_plain", np.ndarray, "float64"), - ("fixed_target", np.ndarray, "float64"), - ("fixed_matrix", np.ndarray, "float64"), - ("fixed_shifted", np.ndarray, "float64"), - ("fixed_counts", np.ndarray, "int32"), - ("fixed_flags", np.ndarray, "bool"), - ("char_fixed", np.ndarray, "S5"), - ("char_target", np.ndarray, "S5"), - ], -) -def test_fixed_shape_module_arrays_are_plain_views(array_forms, name, exposed_as, dtype): - """A declared shape needs no handle: the value is the storage itself.""" - value = getattr(array_forms, name) - - assert isinstance(value, exposed_as) - assert value.dtype == np.dtype(dtype) - - -@pytest.mark.parametrize( - ("name", "dtype"), - [ - ("alloc_plain", "float64"), - ("alloc_target", "float64"), - ("alloc_matrix", "float64"), - ("alloc_shifted", "float64"), - ("char_alloc", "S5"), - ("char_deferred", "S6"), - ], -) -def test_allocatable_module_arrays_are_handles(array_forms, name, dtype): - """Allocation state is not in the declaration, so these carry it explicitly.""" - handle = getattr(array_forms, name) - - assert handle.allocated is True - assert handle.to_numpy().dtype == np.dtype(dtype) +def test_fixed_shapes_are_plain_views_and_allocatables_are_handles(array_forms): + """A declared shape needs no handle: the value is the storage itself. + + Allocation state is not in the declaration, so an allocatable carries it + explicitly as a handle. + """ + fixed = { + "fixed_plain": "float64", + "fixed_target": "float64", + "fixed_matrix": "float64", + "fixed_shifted": "float64", + "fixed_counts": "int32", + "fixed_flags": "bool", + "char_fixed": "S5", + "char_target": "S5", + } + for name, dtype in fixed.items(): + value = getattr(array_forms, name) + assert isinstance(value, np.ndarray), name + assert value.dtype == np.dtype(dtype), name + + allocatable = { + "alloc_plain": "float64", + "alloc_target": "float64", + "alloc_matrix": "float64", + "alloc_shifted": "float64", + "char_alloc": "S5", + "char_deferred": "S6", + } + for name, dtype in allocatable.items(): + handle = getattr(array_forms, name) + assert handle.allocated is True, name + assert handle.to_numpy().dtype == np.dtype(dtype), name def test_a_bare_pointer_is_a_handle_that_declines_to_hand_out_a_view(array_forms): @@ -98,26 +91,23 @@ def test_derived_array_fields_are_views_through_either_owner(array_forms): assert array_forms.obj_plain.grid.shape == (2, 3) -@pytest.mark.parametrize( - "name", - ["fixed_plain", "fixed_target", "fixed_shifted", "obj_plain", "obj_target"], -) -def test_views_stay_live_across_native_writes(array_forms, name): +def test_views_stay_live_across_native_writes(array_forms): """Every borrowed view names the storage native code writes, not a copy.""" - - def current(): - owner = getattr(array_forms, name) - return owner.grid if name.startswith("obj") else owner - - view = current() - before = float(view.flat[0]) - try: - view.flat[0] = before + 1.0 - assert float(current().flat[0]) == before + 1.0 - finally: - # The storage is shared with every other test in this module, so the - # write is undone rather than left for whatever runs next. - view.flat[0] = before + for name in ("fixed_plain", "fixed_target", "fixed_shifted", "obj_plain", "obj_target"): + + def current(name=name): + owner = getattr(array_forms, name) + return owner.grid if name.startswith("obj") else owner + + view = current() + before = float(view.flat[0]) + try: + view.flat[0] = before + 1.0 + assert float(current().flat[0]) == before + 1.0, name + finally: + # The storage is shared with every other test in this module, so the + # write is undone rather than left for whatever runs next. + view.flat[0] = before def test_every_numeric_form_reaches_one_ordinary_array_dummy(array_forms): diff --git a/tests/fortran/modules/end_to_end/test_module_variable_reexport.py b/tests/fortran/modules/end_to_end/test_module_variable_reexport.py index cf33c5380..ffadd8360 100644 --- a/tests/fortran/modules/end_to_end/test_module_variable_reexport.py +++ b/tests/fortran/modules/end_to_end/test_module_variable_reexport.py @@ -30,29 +30,22 @@ def built(tmp_path_factory): return _import_from_build_dir(result.module_name, result.output_dir) -def test_a_scalar_publication_reads_and_writes_one_native_variable(built): - """Both namespaces name the same storage, so either one observes the other.""" +def test_every_scalar_publication_reads_and_writes_one_native_variable(built): + """Direct, renamed, and multi-hop publications all name the declaring storage. + + A rename changes the Python name a namespace binds, never the variable, and + A -> B -> C publishes what A declares, not a copy B made. + """ built.store_mod.counter = np.int32(11) assert built.facade_mod.counter == np.int32(11) - built.facade_mod.counter = np.int32(23) assert built.store_mod.counter == np.int32(23) - -def test_a_renamed_publication_reaches_the_same_variable(built): - """A rename changes the Python name a namespace binds, never the variable.""" - built.store_mod.counter = np.int32(31) - assert built.renamed_mod.tally == np.int32(31) - + assert built.renamed_mod.tally == np.int32(23) built.renamed_mod.tally = np.int32(37) assert built.store_mod.counter == np.int32(37) - -def test_a_multi_hop_publication_resolves_to_the_declaring_variable(built): - """A -> B -> C publishes what A declares, not a copy B made.""" - built.store_mod.counter = np.int32(41) - assert built.hop_mod.tally == np.int32(41) - + assert built.hop_mod.tally == np.int32(37) built.hop_mod.tally = np.int32(43) assert built.store_mod.counter == np.int32(43) assert built.renamed_mod.tally == np.int32(43) @@ -147,26 +140,6 @@ def test_a_parameter_publishes_a_value_rather_than_shared_storage(built): assert built.store_mod.limit == np.int32(42) -def test_one_native_accessor_serves_every_publication(tmp_path: Path): - """The second namespace adds names, so no second accessor is generated.""" - source = tmp_path / "store.f90" - source.write_text(SOURCE, encoding="utf-8") - - result = build_fortran_extension( - source, - output_dir=tmp_path / "generated", - output_name="accessor_api", - generate_sources=True, - ) - wrapper = (result.output_dir / "accessor_api_wrapper.c").read_text(encoding="utf-8") - - # One getter and one setter definition carry `counter`, however many - # namespaces publish it; the four dispatches all call the same pair. - assert wrapper.count("static PyObject * module_get_counter(void) {") == 1 - assert wrapper.count("static int module_set_counter(PyObject * value_obj) {") == 1 - assert wrapper.count("return module_get_counter();") == 4 - - def test_a_facade_may_publish_a_variable_its_declaring_namespace_hides(tmp_path: Path): """Owning the one variable plan must not put the declaring module in Python.""" source = tmp_path / "store.f90" @@ -239,6 +212,13 @@ def surface(module): assert surface(from_source) == surface(from_contract) + # The second namespace adds names, so no second accessor is generated: one + # getter and one setter carry `counter`, and all four dispatches call them. + wrapper = (source_result.output_dir / "parity_source_wrapper.c").read_text(encoding="utf-8") + assert wrapper.count("static PyObject * module_get_counter(void) {") == 1 + assert wrapper.count("static int module_set_counter(PyObject * value_obj) {") == 1 + assert wrapper.count("return module_get_counter();") == 4 + # The contract route reaches the same native variable, not a copy of it. from_contract.facade_mod.counter = np.int32(61) assert from_contract.store_mod.counter == np.int32(61) diff --git a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py index d36ce9848..2ada2a1ba 100644 --- a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py +++ b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py @@ -227,9 +227,6 @@ def test_fixed_shape_character_module_arrays_expose_one_live_bytes_view(tmp_path assert module.read_grid(np.int32(2), np.int32(1)) == "ZZ " -CHARACTER_MODULE_SCALAR_SOURCE = (NATIVE_FIXTURES / "fchar_module_scalars_f90.f90").read_text(encoding="utf-8") - - def test_scalar_character_module_variables_read_and_write_through(pyi_parity_build_mode: str, tmp_path: Path): """Fixed character storage keeps one native address across reads and writes.""" module = _build_source_or_generated_pyi_and_import( @@ -266,44 +263,22 @@ def test_scalar_character_module_variables_read_and_write_through(pyi_parity_bui assert label[()] == "café!!!".encode() assert module.read_label() == "café!!!" - -@pytest.mark.parametrize("value", ["ab", "abcd"]) -def test_scalar_character_module_variable_rejects_a_wrong_encoded_width(value: str, tmp_path: Path): - """Truncating or padding silently would corrupt native state, so the width is exact.""" - module = _build_text_and_import( - CHARACTER_MODULE_SCALAR_SOURCE, - "fchar_module_scalars_f90.f90", - tmp_path, - { - "bind_c_fchar_module_scalars_f90_wrapper.f90", - "fchar_module_scalars_f90_wrapper.c", - "fchar_module_scalars_f90_wrapper.h", - }, - ) - - with pytest.raises(TypeError, match="exactly 3 bytes"): - module.code = value + # Truncating or padding silently would corrupt native state, so the width is exact. + for value in ("ab", "abcd"): + with pytest.raises(TypeError, match="exactly 3 bytes"): + module.code = value assert module.code[()] == b"abc" -CHARACTER_MODULE_DESCRIPTOR_SOURCE = (NATIVE_FIXTURES / "fchar_module_descriptors_f90.f90").read_text(encoding="utf-8") - - -def _character_descriptor_module(tmp_path: Path): - return _build_text_and_import( - CHARACTER_MODULE_DESCRIPTOR_SOURCE, - "fchar_module_descriptors_f90.f90", - tmp_path, - { - "bind_c_fchar_module_descriptors_f90_wrapper.f90", - "fchar_module_descriptors_f90_wrapper.c", - "fchar_module_descriptors_f90_wrapper.h", - }, - ) - - def test_descriptor_character_module_variables_follow_current_storage(pyi_parity_build_mode: str, tmp_path: Path): - """Each read lends the current storage read-only; assignment writes through the descriptor.""" + """Each read lends the current storage read-only; assignment writes through the descriptor. + + Deallocation and nullification are values Python observes, not stale reads. + A character parameter array has no addressable storage, so it is a + read-only Python-owned copy taken at import with the declared element width + as its dtype; a `len=*` parameter reports the width the Fortran side infers + from its initializer. + """ module = _build_source_or_generated_pyi_and_import( NATIVE_FIXTURES / "fchar_module_descriptors_f90.f90", tmp_path, @@ -315,6 +290,16 @@ def test_descriptor_character_module_variables_follow_current_storage(pyi_parity CONTRACT_FIXTURES / "fchar_module_descriptors_f90", pyi_parity_build_mode, ) + assert module.pair.dtype == np.dtype("S2") + assert module.grid.dtype == np.dtype("S3") + assert module.grid.shape == (2, 2) + assert module.pair.flags["WRITEABLE"] is False + assert module.grid.flags["WRITEABLE"] is False + np.testing.assert_array_equal(module.pair, np.array([b"ab", b"cd"], dtype="S2")) + np.testing.assert_array_equal(module.grid, np.array([[b"aaa", b"ccc"], [b"bbb", b"ddd"]], dtype="S3")) + assert module.inferred.dtype == np.dtype("S5") + np.testing.assert_array_equal(module.inferred, np.array([b"alpha", b"beta ", b"gamma"], dtype="S5")) + assert module.deferred is None assert module.fixed is None assert module.link is None @@ -351,12 +336,6 @@ def test_descriptor_character_module_variables_follow_current_storage(pyi_parity module.fixed = "NARROW" assert module.fixed[()] == b"NARROW" - -def test_descriptor_character_module_variables_report_absence_as_none(tmp_path: Path): - """Deallocation and nullification are values Python observes, not stale reads.""" - module = _character_descriptor_module(tmp_path) - - module.setup() module.clear() assert module.deferred is None assert module.fixed is None @@ -366,43 +345,6 @@ def test_descriptor_character_module_variables_report_absence_as_none(tmp_path: assert f"{name} : ndarray[bytes] or None" in module.__doc__ -def test_character_parameter_arrays_are_read_only_fixed_width_snapshots(tmp_path: Path): - """A character parameter array is copied once, like a numeric one. - - A Fortran parameter has no addressable storage, so the value is a - Python-owned copy taken at import; it must therefore be read-only and keep - the declared element width as its dtype. - """ - module = _character_descriptor_module(tmp_path) - - assert module.pair.dtype == np.dtype("S2") - assert module.grid.dtype == np.dtype("S3") - assert module.grid.shape == (2, 2) - assert module.pair.flags["WRITEABLE"] is False - assert module.grid.flags["WRITEABLE"] is False - np.testing.assert_array_equal(module.pair, np.array([b"ab", b"cd"], dtype="S2")) - np.testing.assert_array_equal( - module.grid, - np.array([[b"aaa", b"ccc"], [b"bbb", b"ddd"]], dtype="S3"), - ) - - -def test_assumed_length_character_parameter_array_reports_its_inferred_width(tmp_path: Path): - """A `len=*` parameter takes its width from its initializer, which prik never reads. - - The width is still a constant the Fortran side knows, so the accessor - reports it beside the extents rather than the binding restating a length - it would have to evaluate the initializer to learn. - """ - module = _character_descriptor_module(tmp_path) - - assert module.inferred.dtype == np.dtype("S5") - np.testing.assert_array_equal( - module.inferred, - np.array([b"alpha", b"beta ", b"gamma"], dtype="S5"), - ) - - DECLARED_LENGTH_CHARACTER_ARRAY_SOURCE = (NATIVE_FIXTURES / "fchar_declared_arrays_f90.f90").read_text(encoding="utf-8") @@ -458,7 +400,7 @@ def test_declared_length_character_module_arrays_compile_and_expose_their_width( assert module.deferred_ptr.shape is None -REEXPORT_SOURCE = (NATIVE_FIXTURES / "reexport.f90").read_text(encoding="utf-8") +REEXPORT_SOURCE = NATIVE_FIXTURES / "reexport.f90" def test_module_variable_reexports_share_one_native_entity_from_source_and_contract( @@ -555,182 +497,48 @@ def test_module_variable_reexports_share_one_native_entity_from_source_and_contr assert bridge.count(signature) == 1 -def test_explicitly_published_import_is_reachable_without_a_second_wrapper(tmp_path: Path): - """Naming an imported procedure in a `public` statement publishes it here. +def test_published_imports_bind_the_declaring_modules_callable(pyi_parity_build_mode: str, tmp_path: Path): + """A published name is an alias, so every route binds the one wrapped callable. - The declaration is not repeated: the published name binds to the one - wrapper its own module exposes, so both namespaces share a single callable. - A default-public module also republishes an accessible imported name. + The declaration is never repeated, whichever way the build was described: + an explicit `public` of an import, a default-public or plain-`use` + republication, a rename, and a further hop all bind the object the + declaring module exposes. A re-export binds a Python attribute, so it + reaches the name the declaring module settled on -- the Python spelling of + a name written in capitals, or `lambda__2` when `lambda` and `lambda_` + collide -- rather than the source spelling. """ - source = tmp_path / "reexport.f90" - source.write_text(REEXPORT_SOURCE, encoding="utf-8") - module = _build_source_and_import( - source, - tmp_path / "build", + module = _build_source_or_generated_pyi_and_import( + REEXPORT_SOURCE, + tmp_path, {"bind_c_reexport_wrapper.f90", "reexport_wrapper.c", "reexport_wrapper.h"}, + None, + pyi_parity_build_mode, ) + scale_value = module.reexport_home_mod.scale_value - assert module.reexport_facade_mod.scale_value is module.reexport_home_mod.scale_value + assert module.reexport_facade_mod.scale_value is scale_value assert module.reexport_facade_mod.scale_value(np.int32(4)) == np.int32(8) - - assert module.reexport_default_mod.scale_value is module.reexport_home_mod.scale_value - - # One wrapper defines the procedure; the facade only names it again. - generated = (tmp_path / "build" / "reexport_wrapper.c").read_text(encoding="utf-8") - assert generated.count("static PyObject * wrap_scale_value") == 1 - - -def test_published_import_resolves_the_python_name_its_declaring_module_bound(tmp_path: Path): - """A re-export binds a Python attribute, which is not a Fortran spelling. - - A Fortran entity written in capitals is exported under its Python name, so - the module publishing it has to reach for that name rather than the source - spelling, which names no attribute at all. - """ - source = tmp_path / "reexport.f90" - source.write_text(REEXPORT_SOURCE, encoding="utf-8") - module = _build_source_and_import( - source, - tmp_path / "build", - {"bind_c_reexport_wrapper.f90", "reexport_wrapper.c", "reexport_wrapper.h"}, - ) + assert module.reexport_default_mod.scale_value is scale_value + assert module.reexport_wildcard_mod.scale_value is scale_value + assert module.reexport_wildcard_mod.scale_value(np.int32(5)) == np.int32(10) + assert module.reexport_renamed_mod.public_scale is scale_value + assert module.reexport_renamed_mod.public_scale(np.int32(6)) == np.int32(12) + assert module.reexport_hop_mod.scale_value is scale_value + assert module.reexport_hop_mod.scale_value(np.int32(7)) == np.int32(14) assert module.reexport_case_mod.scale_loud is module.reexport_shout_mod.scale_loud assert module.reexport_case_mod.scale_loud(np.int32(4)) == np.int32(12) assert not hasattr(module.reexport_case_mod, "SCALE_LOUD") - -def test_renamed_published_import_shares_the_wrapper_it_renames(tmp_path: Path): - """A renamed re-export states a new name for one existing callable.""" - source = tmp_path / "reexport.f90" - source.write_text(REEXPORT_SOURCE, encoding="utf-8") - module = _build_source_and_import( - source, - tmp_path / "build", - {"bind_c_reexport_wrapper.f90", "reexport_wrapper.c", "reexport_wrapper.h"}, - ) - - assert module.reexport_renamed_mod.public_scale is module.reexport_home_mod.scale_value - assert module.reexport_renamed_mod.public_scale(np.int32(6)) == np.int32(12) - - -def test_publishing_a_name_a_plain_use_brought_in_republishes_that_name(tmp_path: Path): - """A plain `use` carries public names that remain accessible by default. - - An explicit `public` statement also publishes the named import; both routes - bind the one wrapper owned by the declaring module. - """ - source = tmp_path / "reexport.f90" - source.write_text(REEXPORT_SOURCE, encoding="utf-8") - module = _build_source_and_import( - source, - tmp_path / "build", - {"bind_c_reexport_wrapper.f90", "reexport_wrapper.c", "reexport_wrapper.h"}, - ) - - assert module.reexport_wildcard_mod.scale_value is module.reexport_home_mod.scale_value - assert module.reexport_wildcard_mod.scale_value(np.int32(5)) == np.int32(10) - assert module.reexport_default_mod.scale_value is module.reexport_home_mod.scale_value - - -def test_publishing_an_already_published_import_follows_it_to_its_declaration(tmp_path: Path): - """A published name may come from a module that published it in turn. - - The module a `use` reads is not always the one declaring the entity, so - each hop is followed until the declaration itself is reached; stopping at - the first module leaves the name looking like nothing at all. - """ - source = tmp_path / "reexport.f90" - source.write_text(REEXPORT_SOURCE, encoding="utf-8") - module = _build_source_and_import( - source, - tmp_path / "build", - {"bind_c_reexport_wrapper.f90", "reexport_wrapper.c", "reexport_wrapper.h"}, - ) - - assert module.reexport_hop_mod.scale_value is module.reexport_home_mod.scale_value - assert module.reexport_hop_mod.scale_value(np.int32(7)) == np.int32(14) - - -def test_published_import_binds_the_declaration_a_collision_moved_aside(tmp_path: Path): - """Two source names may want one Python name, and only one may have it. - - A module holding both `lambda` and `lambda_` publishes them as `lambda_` - and `lambda__2`, so a module publishing the second reaches the name the - declaring module settled on rather than the one its source resembles. - """ - source = tmp_path / "reexport.f90" - source.write_text(REEXPORT_SOURCE, encoding="utf-8") - module = _build_source_and_import( - source, - tmp_path / "build", - {"bind_c_reexport_wrapper.f90", "reexport_wrapper.c", "reexport_wrapper.h"}, - ) - assert module.reexport_collide_mod.lambda_(np.int32(0)) == np.int32(1) assert module.reexport_collide_mod.lambda__2(np.int32(0)) == np.int32(100) assert module.reexport_collide_user_mod.lambda_ is module.reexport_collide_mod.lambda__2 assert module.reexport_collide_user_mod.lambda_(np.int32(0)) == np.int32(100) - -def test_a_reexport_binds_one_callable_from_source_and_from_its_contract(tmp_path: Path): - """A published name is an alias, so both routes bind the same object. - - A re-export names a procedure that is already wrapped, whichever way the - build was described. Wrapping it a second time would give one native - procedure two Python objects, and a renamed re-export is no different: the - name it binds changes, not the callable behind it. - """ - import subprocess - import sys - - from tests.fortran._support.wrapper_build import _compiler, _import_from_build_dir - from prik import build_pyi_extension - - source = tmp_path / "reexport.f90" - source.write_text(REEXPORT_SOURCE, encoding="utf-8") - - from_source = _build_source_and_import( - source, - tmp_path / "source_build", - {"bind_c_reexport_wrapper.f90", "reexport_wrapper.c", "reexport_wrapper.h"}, - ) - assert from_source.reexport_facade_mod.scale_value is from_source.reexport_home_mod.scale_value - assert from_source.reexport_renamed_mod.public_scale is from_source.reexport_home_mod.scale_value - - contracts = tmp_path / "contracts" - subprocess.run( - [ - sys.executable, - "-m", - "prik", - "generate", - "--pyi", - str(source), - "--out", - str(contracts), - "--compiler", - _compiler(), - ], - check=True, - capture_output=True, - ) - result = build_pyi_extension( - contracts / "__init__.pyi", - input_compiler=_compiler(), - native_fortran_sources=[str(source)], - output_dir=tmp_path / "contract_build", - output_name="reexport_contract", - ) - from_contract = _import_from_build_dir(result.module_name, result.output_dir) - - assert from_contract.reexport_facade_mod.scale_value is from_contract.reexport_home_mod.scale_value - assert from_contract.reexport_renamed_mod.public_scale is from_contract.reexport_home_mod.scale_value - assert from_contract.reexport_facade_mod.scale_value(np.int32(4)) == np.int32(8) - - # One wrapper defines the procedure on either route. - generated = (result.output_dir / "reexport_contract_wrapper.c").read_text(encoding="utf-8") - assert generated.count("static PyObject * wrap_scale_value") == 1 + # One wrapper defines the procedure on either route; the facades only name it again. + generated = next(_module_variables_build_dir(tmp_path, pyi_parity_build_mode).glob("*_wrapper.c")) + assert generated.read_text(encoding="utf-8").count("static PyObject * wrap_scale_value") == 1 def test_a_derived_module_variable_argument_is_the_variable_itself(pyi_parity_build_mode: str, tmp_path: Path): diff --git a/tests/fortran/modules/parsing/test_project_scope_models.py b/tests/fortran/modules/parsing/test_project_scope_models.py index 16136c8b8..9f2c72402 100644 --- a/tests/fortran/modules/parsing/test_project_scope_models.py +++ b/tests/fortran/modules/parsing/test_project_scope_models.py @@ -6,7 +6,6 @@ from prik.parsers.fortran import FortranParseError, parse_fortran_file, parse_fortran_project from prik.parsers.fortran.scope import ScopeUses -from prik.parsers.fortran.parser import FortranParser from prik.semantics.fortran2ir import fortran_module_to_semantic_module @@ -308,63 +307,12 @@ def test_public_models_finalize_program_block_data_and_file_level_types_interfac assert "worker" in project.programs -def test_duplicate_program_and_block_data_variables_report_scope_labels(): - with pytest.raises(FortranParseError, match="Duplicate variable 'status' in program 'driver'"): - parse_fortran_file( - """ -program driver - integer :: status - real :: status -end program driver -""", - filename="dup_program_var.f90", - ) - - with pytest.raises(FortranParseError, match="Duplicate variable 'seed' in block data 'init_data'"): - parse_fortran_file( - """ -block data init_data - integer seed - real seed -end block data init_data -""", - filename="dup_block_var.f90", - ) - +def test_directory_project_resolves_imported_kinds_from_other_files(tmp_path): + """Renamed, chained, and plain imports resolve kinds; declared shapes stay symbolic. -def test_directory_project_resolves_module_kinds_and_orders_dependencies(tmp_path): - (tmp_path / "kinds.f90").write_text( - """ -module kinds_mod - integer, parameter :: rk = 8 -end module kinds_mod -""", - encoding="utf-8", - ) - (tmp_path / "solver.f90").write_text( - """ -module solver_mod - use kinds_mod, only: rk -contains - function make_value(x) result(value) - real(kind=rk), intent(in) :: x(1:rk) - real(kind=rk) :: value - end function make_value -end module solver_mod -""", - encoding="utf-8", - ) - - project = parse_fortran_project(tmp_path) - proc = project.procedures["solver_mod.make_value"] - - assert proc.arguments[0].kind == "8" - assert proc.arguments[0].shape == ["1:rk"] - assert proc.result.kind == "8" - assert project.dependencies["solver_mod"] == {"kinds_mod"} - - -def test_directory_project_tracks_renamed_kind_imports_from_other_files(tmp_path): + A derived-type component folds its extent, while a dummy keeps the name the + source spells; dependency order follows the `use`. + """ (tmp_path / "precision.f90").write_text( """ module precision_mod @@ -372,6 +320,8 @@ def test_directory_project_tracks_renamed_kind_imports_from_other_files(tmp_path integer, parameter :: stride = 2 integer, parameter :: wp = word * stride integer, parameter :: wide = wp * stride + integer, parameter :: rk = 8 + integer, parameter :: n = 3 end module precision_mod """, encoding="utf-8", @@ -379,91 +329,44 @@ def test_directory_project_tracks_renamed_kind_imports_from_other_files(tmp_path (tmp_path / "solver.f90").write_text( """ module solver_mod - use precision_mod, only: local_wp => wp, stride, local_wide => wide + use precision_mod, only: local_wp => wp, stride, local_wide => wide, rk, n + type :: sample + real(kind=local_wp) :: values(0:n) + end type sample contains subroutine consume(x, y) real(kind=local_wp), intent(in) :: x(1:stride) complex(kind=local_wide), intent(out) :: y end subroutine consume + function make_value(x) result(value) + real(kind=rk), intent(in) :: x(1:rk) + real(kind=rk) :: value + end function make_value end module solver_mod """, encoding="utf-8", ) project = parse_fortran_project(tmp_path) - proc = project.procedures["solver_mod.consume"] - args = {arg.name: arg for arg in proc.arguments} + consume = project.procedures["solver_mod.consume"] + args = {arg.name: arg for arg in consume.arguments} + make_value = project.procedures["solver_mod.make_value"] + field = project.modules["solver_mod"].derived_types[0].fields[0] - assert args["x"].kind == "8" - assert args["x"].shape == ["1:stride"] + assert (args["x"].kind, args["x"].shape) == ("8", ["1:stride"]) assert args["y"].kind == "16" - assert [(mapping.source, mapping.target) for mapping in ScopeUses(proc.uses).mappings("precision_mod")] == [ + assert (make_value.arguments[0].kind, make_value.arguments[0].shape, make_value.result.kind) == ("8", ["1:rk"], "8") + assert (field.kind, field.shape) == ("8", ["0:3"]) + assert [(mapping.source, mapping.target) for mapping in ScopeUses(consume.uses).mappings("precision_mod")] == [ ("wp", "local_wp"), ("stride", None), ("wide", "local_wide"), + ("rk", None), + ("n", None), ] assert project.dependencies["solver_mod"] == {"precision_mod"} - - -def test_project_compile_time_resolution_uses_models_is_idempotent_and_preserves_symbolic_shapes(): - parser = FortranParser() - kinds_file = parser.parse_file( - """ -module kinds - integer, parameter :: word = 4 - integer, parameter :: rk = word * 2 - integer, parameter :: n = 3 -end module kinds -""", - filename="kinds.f90", - ) - consumer_file = parser.parse_file( - """ -module records - use kinds, only: wp => rk, n - type :: sample - real(kind=wp) :: values(0:n) - end type sample -contains - subroutine consume(values) - real(kind=wp), intent(in) :: values(1:n) - end subroutine consume -end module records -""", - filename="records.f90", - ) - kinds_file.source = None - consumer_file.source = None - - parser._resolve_project_compile_time_facts([kinds_file, consumer_file]) - field = consumer_file.modules[0].derived_types[0].fields[0] - argument = consumer_file.modules[0].procedures[0].arguments[0] - first_result = (field.kind, list(field.shape), argument.kind, list(argument.shape)) - - parser._resolve_project_compile_time_facts([kinds_file, consumer_file]) - - assert first_result == ("8", ["0:3"], "8", ["1:n"]) - assert (field.kind, field.shape, argument.kind, argument.shape) == first_result - - -def test_project_resolves_reexported_intrinsic_kind_renames(): - project = parse_fortran_project( - { - "consumer.f90": """ -subroutine consume(x) - use fftpack_kind, only: dp => rk - real(dp), intent(inout) :: x -end subroutine consume -""", - "kind.f90": """ -module fftpack_kind - use, intrinsic :: iso_fortran_env, only: rk => real64 -end module fftpack_kind -""", - } - ) - - assert project.procedures["consume"].arguments[0].kind == "real64" + ordered = [Path(parsed.filename).name for parsed in project.files] + assert ordered.index("precision.f90") < ordered.index("solver.f90") def test_single_file_project_resolves_intrinsic_kind_rename_for_module_variables(): @@ -489,9 +392,16 @@ def test_single_file_project_resolves_intrinsic_kind_rename_for_module_variables assert module.procedures[0].result.kind == "real64" -def test_project_resolves_submodule_host_associated_kind(): +def test_project_resolves_intrinsic_kind_renames_through_reexports_and_hosts(): + """A kind renamed from an intrinsic module reaches a `use` of the re-exporting module and a submodule host.""" project = parse_fortran_project( { + "consumer.f90": """ +subroutine consume(x) + use precision, only: dp => rk + real(dp), intent(inout) :: x +end subroutine consume +""", "implementation.f90": """ submodule(transform_api) transform_impl contains @@ -520,6 +430,7 @@ def test_project_resolves_submodule_host_associated_kind(): } ) + assert project.procedures["consume"].arguments[0].kind == "real64" procedure = project.submodules["transform_api:transform_impl"].procedures[0] assert procedure.arguments[0].kind == "real64" assert procedure.result.kind == "real64" @@ -553,48 +464,35 @@ def test_directory_project_records_missing_and_parent_only_submodule_dependencie assert project.dependencies["parent_mod:child_mod"] == {"parent_mod", "missing_mod"} -def test_program_and_block_data_scope_errors_use_public_parse_paths(): - with pytest.raises(FortranParseError, match="Unsupported OpenMP declarative directive in program 'driver'"): - parse_fortran_file( - """ -program driver -!$omp threadprivate(counter) -end program driver -""", - filename="program_omp_decl.f90", - ) - - with pytest.raises(FortranParseError, match="Unsupported OpenMP declarative directive in block data 'init_data'"): - parse_fortran_file( - """ -block data init_data -!$omp threadprivate(seed) -end block data init_data -""", - filename="block_omp_decl.f90", - ) - - with pytest.raises(FortranParseError, match="Unknown or unsupported datatype declaration in program 'driver'"): - parse_fortran_file( - """ -program driver - weirdtype state -end program driver -""", - filename="program_unknown_decl.f90", - ) - - with pytest.raises( - FortranParseError, match="Unknown or unsupported datatype declaration in block data 'init_data'" - ): - parse_fortran_file( - """ -block data init_data - weirdtype seed -end block data init_data -""", - filename="block_unknown_decl.f90", - ) +@pytest.mark.parametrize( + ("source", "message"), + [ + pytest.param( + "program driver\n integer :: status\n real :: status\nend program driver\n", + "Duplicate variable 'status' in program 'driver'", + id="duplicate-variable-in-program", + ), + pytest.param( + "block data init_data\n integer seed\n real seed\nend block data init_data\n", + "Duplicate variable 'seed' in block data 'init_data'", + id="duplicate-variable-in-block-data", + ), + pytest.param( + "program driver\n!$omp threadprivate(counter)\nend program driver\n", + "Unsupported OpenMP declarative directive in program 'driver'", + id="openmp-declarative-in-program", + ), + pytest.param( + "block data init_data\n weirdtype seed\nend block data init_data\n", + "Unknown or unsupported datatype declaration in block data 'init_data'", + id="unknown-declaration-in-block-data", + ), + ], +) +def test_program_and_block_data_diagnostics_name_their_scope(source: str, message: str): + """Each scope-level diagnostic names the program unit kind and name it arose in.""" + with pytest.raises(FortranParseError, match=message): + parse_fortran_file(source, filename="units.f90") def test_project_resolution_folds_fortran_real_literal_integer_parameters(): diff --git a/tests/fortran/modules/parsing/test_scope_handling.py b/tests/fortran/modules/parsing/test_scope_handling.py index e5d15c0f7..991cc0eeb 100644 --- a/tests/fortran/modules/parsing/test_scope_handling.py +++ b/tests/fortran/modules/parsing/test_scope_handling.py @@ -5,8 +5,24 @@ from prik.parsers.fortran.scope import ScopeUses -def test_same_argument_name_in_different_procedures_is_allowed(): - code = """ +def _declared_names(parsed) -> list[str]: + """Return every procedure, module variable, and type component a file declares.""" + names = [procedure.name for procedure in parsed.procedures] + for module in parsed.modules: + names.append(module.name) + names += [f"{module.name}.{procedure.name.lower()}" for procedure in module.procedures] + names += [f"{module.name}.{variable.name}" for variable in module.variables] + names += [ + f"{module.name}.{dtype.name}.{field.name}" for dtype in module.derived_types for field in dtype.fields + ] + return sorted(names) + + +@pytest.mark.parametrize( + ("code", "expected"), + [ + pytest.param( + """ subroutine a(x) implicit none integer :: x @@ -16,13 +32,12 @@ def test_same_argument_name_in_different_procedures_is_allowed(): implicit none real(8) :: x end subroutine b -""" - parsed = parse_fortran_file(code, filename="scope_args_ok.f90") - assert [s.name for s in parsed.procedures] == ["a", "b"] - - -def test_interface_argument_names_do_not_conflict_with_host_locals(): - code = """ +""", + ["a", "b"], + id="argument-names-in-different-procedures", + ), + pytest.param( + """ subroutine host(func, x) implicit none interface @@ -33,14 +48,12 @@ def test_interface_argument_names_do_not_conflict_with_host_locals(): end interface real(8) :: x end subroutine host -""" - parsed = parse_fortran_file(code, filename="scope_interface_ok.f90") - assert len(parsed.procedures) == 1 - assert parsed.procedures[0].name == "host" - - -def test_same_contained_procedure_name_in_different_hosts_is_allowed(): - code = """ +""", + ["host"], + id="interface-arguments-and-host-locals", + ), + pytest.param( + """ module m contains subroutine host_a() @@ -57,31 +70,12 @@ def test_same_contained_procedure_name_in_different_hosts_is_allowed(): end subroutine helper end subroutine host_b end module m -""" - parsed = parse_fortran_file(code, filename="scope_contains_ok.f90") - module = parsed.modules[0] - assert [s.name.lower() for s in module.procedures] == ["host_a", "host_b"] - - -def test_duplicate_procedure_name_in_same_scope_still_errors(): - code = """ -module m -contains - subroutine work(n) - integer :: n - end subroutine work - function work(n) result(out) - integer :: n - integer :: out - end function work -end module m -""" - with pytest.raises(FortranParseError, match="Duplicate procedure name"): - parse_fortran_file(code, filename="scope_dup_err.f90") - - -def test_type_components_do_not_conflict_with_host_procedure_locals(): - code = """ +""", + ["m", "m.host_a", "m.host_b"], + id="contained-procedures-in-different-hosts", + ), + pytest.param( + """ module component_vs_local implicit none type :: box_t @@ -93,14 +87,12 @@ def test_type_components_do_not_conflict_with_host_procedure_locals(): integer :: n end subroutine touch end module component_vs_local -""" - parsed = parse_fortran_file(code, filename="scope_component_local_ok.f90") - module = parsed.modules[0] - assert [s.name for s in module.procedures] == ["touch"] - - -def test_module_variable_and_type_component_same_name_is_allowed(): - code = """ +""", + ["component_vs_local", "component_vs_local.box_t.n", "component_vs_local.touch"], + id="type-components-and-procedure-locals", + ), + pytest.param( + """ module module_vs_component implicit none integer :: vals @@ -108,17 +100,12 @@ def test_module_variable_and_type_component_same_name_is_allowed(): integer :: vals end type payload end module module_vs_component -""" - parsed = parse_fortran_file(code, filename="scope_module_component_same_name_ok.f90") - module = parsed.modules[0] - - assert {v.name for v in module.variables} == {"vals"} - assert [t.name for t in module.derived_types] == ["payload"] - assert {f.name for f in module.derived_types[0].fields} == {"vals"} - - -def test_type_component_names_do_not_leak_between_different_modules(): - code = """ +""", + ["module_vs_component", "module_vs_component.payload.vals", "module_vs_component.vals"], + id="module-variable-and-type-component", + ), + pytest.param( + """ module a_mod type :: t integer :: vals @@ -130,15 +117,38 @@ def test_type_component_names_do_not_leak_between_different_modules(): real(8) :: vals end type t end module b_mod -""" - parsed = parse_fortran_file(code, filename="scope_cross_module_components_ok.f90") - assert [m.name for m in parsed.modules] == ["a_mod", "b_mod"] - assert {f.name for f in parsed.modules[0].derived_types[0].fields} == {"vals"} - assert {f.name for f in parsed.modules[1].derived_types[0].fields} == {"vals"} - - -def test_duplicate_type_component_name_still_errors_inside_single_type_scope(): - code = """ +""", + ["a_mod", "a_mod.t.vals", "b_mod", "b_mod.t.vals"], + id="type-components-in-different-modules", + ), + ], +) +def test_the_same_name_in_different_scopes_is_not_a_duplicate(code: str, expected: list[str]): + """Each scope owns its names, so a spelling repeated in another scope parses and keeps both.""" + assert _declared_names(parse_fortran_file(code, filename="scopes.f90")) == expected + + +@pytest.mark.parametrize( + ("code", "message"), + [ + pytest.param( + """ +module m +contains + subroutine work(n) + integer :: n + end subroutine work + function work(n) result(out) + integer :: n + integer :: out + end function work +end module m +""", + "Duplicate procedure name", + id="procedure-in-one-module", + ), + pytest.param( + """ module dup_component_mod implicit none type :: thing @@ -146,9 +156,15 @@ def test_duplicate_type_component_name_still_errors_inside_single_type_scope(): real(8) :: vals end type thing end module dup_component_mod -""" - with pytest.raises(FortranParseError, match="Duplicate field 'vals' in derived type 'thing'"): - parse_fortran_file(code, filename="scope_component_dup_err.f90") +""", + "Duplicate field 'vals' in derived type 'thing'", + id="component-in-one-type", + ), + ], +) +def test_the_same_name_twice_in_one_scope_is_a_duplicate(code: str, message: str): + with pytest.raises(FortranParseError, match=message): + parse_fortran_file(code, filename="duplicates.f90") def test_module_parameter_shape_is_visible_to_contained_function_scope(): @@ -178,19 +194,6 @@ def test_module_parameter_shape_is_visible_to_contained_function_scope(): assert proc.variables == {} -def test_fortran_parser_class_entrypoint(): - source = """ -subroutine touch(x) - integer, intent(inout) :: x -end subroutine -""" - - signatures = parse_fortran_file(source).procedures - - assert len(signatures) == 1 - assert signatures[0].name == "touch" - - def test_repeated_use_of_one_module_accumulates_its_imports(): """A scope may name the same module in several `use` statements. diff --git a/tests/fortran/modules/pipeline/test_generated_module_contracts.py b/tests/fortran/modules/pipeline/test_generated_module_contracts.py index d46fb06f6..e47ccb1bf 100644 --- a/tests/fortran/modules/pipeline/test_generated_module_contracts.py +++ b/tests/fortran/modules/pipeline/test_generated_module_contracts.py @@ -16,17 +16,9 @@ FIXTURES = Path(__file__).parents[1] / "end_to_end" / "fixtures" CONTRACT_ROOT = FIXTURES / "contracts" +# Contracts that an end-to-end test also replays are compared there, in its +# generated-`.pyi` lane; only the ones no build replays are compared here. CASES = ( - GeneratedContractCase( - "fcommon_block_f90", - (FIXTURES / "native" / "fcommon_block_f90.f90",), - CONTRACT_ROOT / "fcommon_block_f90", - ), - GeneratedContractCase( - "fmodule_vars_f90", - (FIXTURES / "native" / "fmodule_vars_f90.f90",), - CONTRACT_ROOT / "fmodule_vars_f90", - ), GeneratedContractCase( "module_exports", (FIXTURES / "native" / "module_exports.f90",), diff --git a/tests/fortran/modules/policy/test_module_variable_policy.py b/tests/fortran/modules/policy/test_module_variable_policy.py index dff1676cb..038c3b5b9 100644 --- a/tests/fortran/modules/policy/test_module_variable_policy.py +++ b/tests/fortran/modules/policy/test_module_variable_policy.py @@ -177,30 +177,3 @@ def test_fixed_module_array_address_mechanism_follows_declared_addressability(): # Neither route hands Python the whole variable back to reassign. assert policies["values"].setter_action is SetterAction.REJECT_REPLACEMENT assert policies["addressable"].setter_action is SetterAction.REJECT_REPLACEMENT - - -def test_logical_module_arrays_are_borrowed_at_every_width(): - """A live view aliases element for element, so the dtype reports the width. - - NumPy has no Boolean wider than one byte, so a logical array is described by - the integer of matching width rather than narrowed to `bool`. The widths - then agree for every Fortran kind and each is borrowed as a live view. - """ - module = parse_pyi_text( - """ -from prik.contracts import Bool, Bool32 - -narrow: Bool[3] -wide: Bool32[3] -""", - module_name="logical_state", - ) - complete_semantic_policies(module) - - policies = { - variable.name: variable.metadata[RESOLVED_MODULE_VARIABLE_POLICY_METADATA] for variable in module.variables - } - for name in ("narrow", "wide"): - assert policies[name].supported is True, name - assert policies[name].getter_action is ModuleGetterAction.BORROWED_ARRAY_VIEW, name - assert policies[name].blockers == (), name diff --git a/tests/fortran/modules/semantics/fixtures/native/complex_module.f90 b/tests/fortran/modules/semantics/fixtures/native/complex_module.f90 deleted file mode 100644 index de6266a3b..000000000 --- a/tests/fortran/modules/semantics/fixtures/native/complex_module.f90 +++ /dev/null @@ -1,30 +0,0 @@ -module fem_mod - -type :: mesh - - integer :: nelements - integer :: nnodes - -end type - -contains - -subroutine assemble(K, coords, connectivity) - - real(8), intent(out) :: K(:, :) - - real(8), intent(in) :: coords(:, :) - - integer, intent(in) :: connectivity(:, :) - -end subroutine - -function compute_norm(x) result(r) - - real(8), intent(in) :: x(:) - - real(8) :: r - -end function - -end module diff --git a/tests/fortran/modules/semantics/test_declaration_publication.py b/tests/fortran/modules/semantics/test_declaration_publication.py index 1aed9e436..c4b5c539e 100644 --- a/tests/fortran/modules/semantics/test_declaration_publication.py +++ b/tests/fortran/modules/semantics/test_declaration_publication.py @@ -32,27 +32,29 @@ def _module(source: str, tmp_path: Path): return module -def test_a_private_prototype_and_generic_state_their_accessibility(tmp_path: Path): - """Semantics records what the module's `private` default says about each.""" +def test_a_private_prototype_and_generic_are_written_but_not_published(tmp_path: Path): + """Semantics records the module's `private` default for both declarations. + + The contract still names both for typing and dispatch, and publishes neither. + """ module = _module(PRIVATE_SOURCE, tmp_path) assert [(item.name, item.visibility) for item in module.prototypes] == [("cb", "private")] assert [(item.name, item.visibility) for item in module.overload_sets] == [("hidden_generic", "private")] - -def test_a_private_prototype_and_generic_are_written_but_not_published(tmp_path: Path): - """The contract names both for typing and dispatch, and publishes neither.""" - module = _module(PRIVATE_SOURCE, tmp_path) contract = PyiPrinter().emit(module) - # `run` annotates its callback with the prototype, so the name must exist. assert "def cb() -> None: ..." in contract assert "def hidden_generic(" in contract assert '__all__ = ["run"]' in contract -def test_two_procedures_may_name_different_interfaces_the_same_way(tmp_path: Path): - """A block inside a procedure is that procedure's, so each keeps its own.""" +def test_a_procedure_local_interface_is_never_a_module_publication(tmp_path: Path): + """A block inside a procedure is that procedure's, so each keeps its own. + + Two procedures may name different interfaces the same way; a `use` of the + module cannot reach either, so the contract writes both and publishes neither. + """ module = _module(LOCAL_INTERFACE_SOURCE, tmp_path) assert [(completed_contract_name(item), item.native_name, item.visibility) for item in module.prototypes] == [ @@ -66,12 +68,7 @@ def test_two_procedures_may_name_different_interfaces_the_same_way(tmp_path: Pat } assert signatures == {"first": ["Int32"], "second": ["Float32"]} - -def test_a_procedure_local_interface_is_never_a_module_publication(tmp_path: Path): - """A `use` of the module cannot reach it, so the contract does not publish it.""" - module = _module(LOCAL_INTERFACE_SOURCE, tmp_path) contract = PyiPrinter(normalize_public_names=True).emit(module) - assert "def first_cb(" in contract assert "def second_cb(" in contract assert "f: first_cb" in contract @@ -118,6 +115,14 @@ def test_a_prototype_is_identified_by_its_scope_rather_than_its_spelling(tmp_pat assert annotations["uses_module_one.g"] == ("first_cb", "Int32") assert annotations["first.f"] == (names[1], "Float32") + # Both prototypes are written, and only the module's own is published. + contract = PyiPrinter(normalize_public_names=True).emit(module) + assert "def first_cb(\n x: Int32[()]\n) -> None: ..." in contract + assert f"def {names[1]}(\n x: Float32[()]\n) -> None: ..." in contract + assert f"f: {names[1]}" in contract + assert "g: first_cb" in contract + assert '__all__ = ["first_cb", "first", "uses_module_one"]' in contract + def test_scopes_whose_joined_spellings_collide_keep_distinct_contract_names(tmp_path: Path): """`a_b` declaring `c` and `a` declaring `b_c` are different prototypes.""" @@ -131,19 +136,6 @@ def test_scopes_whose_joined_spellings_collide_keep_distinct_contract_names(tmp_ assert annotations["a.f"] == (names[1], "Float32") -def test_a_contract_writes_one_prototype_for_each_scope(tmp_path: Path): - """Both prototypes are written, and only the module's own is published.""" - module = _module(MODULE_AND_LOCAL_SOURCE, tmp_path) - contract = PyiPrinter(normalize_public_names=True).emit(module) - local_name = completed_contract_name(module.prototypes[1]) - - assert "def first_cb(\n x: Int32[()]\n) -> None: ..." in contract - assert f"def {local_name}(\n x: Float32[()]\n) -> None: ..." in contract - assert f"f: {local_name}" in contract - assert "g: first_cb" in contract - assert '__all__ = ["first_cb", "first", "uses_module_one"]' in contract - - IMPORT_COLLISION_SOURCE = """\ module helper_mod implicit none diff --git a/tests/fortran/modules/semantics/test_module_contract_semantics.py b/tests/fortran/modules/semantics/test_module_contract_semantics.py index 3a346410c..e8b9b3da4 100644 --- a/tests/fortran/modules/semantics/test_module_contract_semantics.py +++ b/tests/fortran/modules/semantics/test_module_contract_semantics.py @@ -5,6 +5,7 @@ def test_module_common_block_storage_stays_internal(): + """Common-block storage is never a module variable, in the module or in a procedure.""" source = """ module common_mod public :: value, read_value @@ -14,26 +15,15 @@ def test_module_common_block_storage_stays_internal(): real function read_value() read_value = value end function read_value -end module common_mod -""" - - module = fortran_module_to_semantic_module(parse_fortran_source(source)) - - assert module.variables == [] - assert [function.name for function in module.functions] == ["read_value"] - -def test_procedure_common_block_storage_is_allowed(): - source = """ -module procedure_common_mod -contains subroutine work() - real :: value - common /shared/ value + real :: local + common /procedure_shared/ local end subroutine work -end module procedure_common_mod +end module common_mod """ module = fortran_module_to_semantic_module(parse_fortran_source(source)) - assert [function.name for function in module.functions] == ["work"] + assert module.variables == [] + assert [function.name for function in module.functions] == ["read_value", "work"] diff --git a/tests/fortran/modules/semantics/test_modules_and_imports.py b/tests/fortran/modules/semantics/test_modules_and_imports.py index c0d859847..9cb85a756 100644 --- a/tests/fortran/modules/semantics/test_modules_and_imports.py +++ b/tests/fortran/modules/semantics/test_modules_and_imports.py @@ -1,16 +1,6 @@ -"""Tests split by stable ownership concept from `test_compile_time_values.py`.""" +"""Fortran module declarations become the intended semantic module.""" -from pathlib import Path - -from prik.parsers.fortran.models import ( - FortranUseStatement, - FortranArgument, - FortranModule, -) from prik.semantics.fortran2ir import ( - FortranToIRConverter, - _requirement_unit_name, - _resolve_compile_time_text, fortran_file_to_semantic_modules, fortran_module_to_semantic_module, ) @@ -24,57 +14,6 @@ from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.parsers.fortran import parse_fortran_file as parse_fortran_source -NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" - - -def test_converter_normalizes_wrapped_types_and_resolves_wildcard_imports(): - converter = FortranToIRConverter(wrapped_derived_types={("types_mod", "state_t")}) - module = FortranModule( - name="consumer", - uses=[FortranUseStatement("OTHER_MOD"), FortranUseStatement("TYPES_MOD")], - ) - context = converter._module_derived_type_context(module) - - state = converter.visit( - FortranArgument(name="state", base_type="derived", kind="state_t"), - derived_type_context=context, - ).semantic_type - opaque_context = converter._module_derived_type_context( - FortranModule(name="consumer", uses=[FortranUseStatement("OPAQUE_MOD")]) - ) - opaque = converter.visit( - FortranArgument(name="opaque", base_type="derived", kind="opaque_t"), - derived_type_context=opaque_context, - ).semantic_type - merged = FortranToIRConverter()._with_additional_wrapped_types({("TYPES_MOD", "State_T")}) - - assert state.metadata["external_type_ref"] == { - "name": "state_t", - "local_name": "state_t", - "origin_module": "TYPES_MOD", - "wrapped": True, - "representation": "wrapped", - } - assert opaque.metadata["external_type_ref"] == { - "name": "opaque_t", - "local_name": "opaque_t", - "origin_module": "OPAQUE_MOD", - "wrapped": False, - "representation": "opaque", - } - assert merged.wrapped_derived_types == {("types_mod", "state_t")} - custom_type_map = {("integer", None): "CustomInt"} - configured = FortranToIRConverter(type_map=custom_type_map, compile_time_values={"rk": 8}) - configured = configured._with_additional_wrapped_types({("types_mod", "state_t")}) - assert configured.type_map is custom_type_map - assert configured.compile_time_values == {"rk": "8"} - assert FortranToIRConverter(compile_time_values={" ": 4, " RK ": 8}).compile_time_values == {"rk": "8"} - assert _resolve_compile_time_text("n + missing", {"n": "4"}) == "4 + missing" - assert _resolve_compile_time_text("N + missing", {"n": "4"}) == "4 + missing" - assert _requirement_unit_name(module="m") == "m" - assert _requirement_unit_name(unit_name="step") == "step" - assert _requirement_unit_name() == "" - def test_iso_c_module_variable_kinds_map_to_semantic_types(): source = """ @@ -175,81 +114,6 @@ def test_explicit_public_submodule_interface_is_callable_from_parent_contract(): assert module.functions[0].return_type.shape == ["n"] -def test_complex_module(): - source = (NATIVE_FIXTURES / "complex_module.f90").read_text(encoding="utf-8") - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - # -------------------------------------------------------- - # Module structure - # -------------------------------------------------------- - - assert smod.name == "fem_mod" - - assert len(smod.functions) == 2 - - assert len(smod.classes) == 1 - - # -------------------------------------------------------- - # Class checks - # -------------------------------------------------------- - - mesh_cls = get_class(smod, "mesh") - - assert len(mesh_cls.fields) == 2 - - # -------------------------------------------------------- - # Procedure checks - # -------------------------------------------------------- - - assemble = get_function(smod, "assemble") - - assert len(assemble.arguments) == 3 - - K = next(arg for arg in assemble.arguments if arg.name == "K") - - assert K.semantic_type.rank == 2 - - assert array_contract(K.semantic_type).order == "ORDER_F" - - connectivity = next(arg for arg in assemble.arguments if arg.name == "connectivity") - - assert connectivity.semantic_type.name == "Int32" - - # -------------------------------------------------------- - # Function return - # -------------------------------------------------------- - - norm = get_function(smod, "compute_norm") - - assert norm.return_type.name == "Float64" - - -def test_module_conversion_public_api_entrypoint(): - source = """ -module class_mod - -contains - -subroutine touch(x) - - integer, intent(inout) :: x - -end subroutine - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - assert smod.name == "class_mod" - assert get_function(smod, "touch").arguments[0].semantic_type.name == "Int32" - - def test_fortran_to_ir_preserves_module_semantics_from_inline_source(): source = """ module m @@ -285,22 +149,3 @@ def test_fortran_to_ir_preserves_module_semantics_from_inline_source(): assert semantic_dtype.visibility == "private" assert semantic_proc.visibility == "public" assert semantic_file_modules[0].name == "m" - - -def test_declaration_level_private_module_constant_is_not_exported(): - parsed = parse_fortran_source( - """ -module constants - real, parameter, private :: epsilon = 1.0 - real, parameter :: visible = 2.0 -end module constants -""", - filename="constants.f90", - ) - - semantic_module = fortran_module_to_semantic_module(parsed) - - assert [(variable.name, variable.visibility) for variable in semantic_module.variables] == [ - ("epsilon", "private"), - ("visible", "public"), - ] diff --git a/tests/fortran/modules/semantics/test_reexport_accessibility.py b/tests/fortran/modules/semantics/test_reexport_accessibility.py index 86cd48137..455fc6926 100644 --- a/tests/fortran/modules/semantics/test_reexport_accessibility.py +++ b/tests/fortran/modules/semantics/test_reexport_accessibility.py @@ -4,6 +4,12 @@ decides it, otherwise the module's bare `public`/`private` default does, and that default is itself `public`. Those rules cover a use-associated entity, so an ordinary module publishes what it imports without naming it anywhere. + +Every route to a name is read the same way whichever form of `use` it entered +by (bare, `only`, renamed, repeated), and each hop of a transitive chain applies +the rule again. Routes that agree on one declaration name one entity; routes +that disagree, or that pass through a module this project cannot read, leave +the name unresolved rather than guessing. """ from pathlib import Path @@ -17,263 +23,11 @@ DECLARING = (NATIVE_FIXTURES / "declaring_module.f90").read_text(encoding="utf-8") +NAMED_GENERIC = (NATIVE_FIXTURES / "a_plain_use_carries_a_named_generic_interface.f90").read_text(encoding="utf-8") -def _reexports( - tmp_path: Path, - importer: str, - *, - module_name: str = "b_mod", -) -> list[tuple[str, str, str]]: - """Return one module's public use associations as (local, source, origin).""" - source = tmp_path / "project.f90" - source.write_text(f"{DECLARING}\n{importer}", encoding="utf-8") - modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) - importing = next(module for module in modules if module.name == module_name) - return [(item.local_name, item.source_name, item.origin_module) for item in importing.reexports] - - -def test_a_default_public_module_publishes_what_it_imports(tmp_path: Path): - """No access statement is needed: the module default is public.""" - assert _reexports( - tmp_path, - """\ -module b_mod - use a_mod, only : x - implicit none -end module b_mod -""", - ) == [("x", "x", "a_mod")] - - -def test_a_declaration_dependency_remains_a_public_use_association(tmp_path: Path): - """Using an import in a declaration does not change its accessibility.""" - assert _reexports( - tmp_path, - """\ -module b_mod - use a_mod, only : crate => box - implicit none -contains - integer function crate_value(item) result(out) - type(crate), intent(in) :: item - out = item%value - end function crate_value -end module b_mod -""", - ) == [("crate", "box", "a_mod")] - - -def test_a_third_module_resolves_a_declaration_dependency_through_its_importer(tmp_path: Path): - """A public use association remains available to another Fortran module.""" - assert _reexports( - tmp_path, - """\ -module b_mod - use a_mod, only : box - implicit none - type(box) :: stored -end module b_mod - -module c_mod - use b_mod, only : box - implicit none - type(box) :: another -end module c_mod -""", - module_name="c_mod", - ) == [("box", "box", "a_mod")] - - -def test_explicit_public_still_publishes_a_declaration_dependency(tmp_path: Path): - """A named public statement is an explicit publication request.""" - assert _reexports( - tmp_path, - """\ -module b_mod - use a_mod, only : crate => box - implicit none - public :: crate -contains - integer function crate_value(item) result(out) - type(crate), intent(in) :: item - out = item%value - end function crate_value -end module b_mod -""", - ) == [("crate", "box", "a_mod")] - - -@pytest.mark.parametrize( - ("importer", "expected"), - [ - pytest.param( - """\ -module b_mod - use a_mod, only : x - implicit none - private -end module b_mod -""", - [], - id="private-default", - ), - pytest.param( - """\ -module b_mod - use a_mod, only : x - implicit none - private - public :: x -end module b_mod -""", - [("x", "x", "a_mod")], - id="public-name-over-private-default", - ), - pytest.param( - """\ -module b_mod - use a_mod, only : x - implicit none - private :: x -end module b_mod -""", - [], - id="private-name-over-public-default", - ), - pytest.param( - """\ -module b_mod - use a_mod - implicit none - private :: a_mod -end module b_mod -""", - [], - id="private-module-route", - ), - pytest.param( - """\ -module b_mod - use a_mod - implicit none - private - public :: a_mod -end module b_mod -""", - [ - ("box", "box", "a_mod"), - ("scale_value", "scale_value", "a_mod"), - ("x", "x", "a_mod"), - ("y", "y", "a_mod"), - ], - id="public-module-route-over-private-default", - ), - ], -) -def test_accessibility_precedence_for_use_associations( - importer: str, - expected: list[tuple[str, str, str]], - tmp_path: Path, -): - """A named access decision outranks the module's public or private default.""" - assert sorted(_reexports(tmp_path, importer)) == sorted(expected) - - -def test_any_public_route_keeps_a_multiply_accessible_entity_public(tmp_path: Path): - """One public route wins when another route to the same entity is private.""" - assert _reexports( - tmp_path, - """\ -module left_mod - use a_mod, only : x -end module left_mod - -module right_mod - use a_mod, only : x -end module right_mod - -module b_mod - use left_mod - use right_mod - implicit none - private :: left_mod - public :: right_mod -end module b_mod -""", - ) == [("x", "x", "a_mod")] - - -def test_a_renamed_default_public_import_publishes_the_local_name(tmp_path: Path): - """A rename changes the name this module publishes, never the declaration.""" - assert _reexports( - tmp_path, - """\ -module b_mod - use a_mod, only : renamed => y - implicit none -end module b_mod -""", - ) == [("renamed", "y", "a_mod")] - - -def test_a_plain_use_carries_the_public_names_of_what_it_reads(tmp_path: Path): - """A `use` naming no list carries every public name, default rules applying.""" - carried = _reexports( - tmp_path, - """\ -module b_mod - use a_mod - implicit none -end module b_mod -""", - ) - - assert sorted(local for local, _source, _origin in carried) == ["box", "scale_value", "x", "y"] - - -def test_a_plain_use_carries_a_named_generic_interface(tmp_path: Path): - """The offered-name inventory includes named interface declarations.""" - carried = _reexports( - tmp_path, - (NATIVE_FIXTURES / "a_plain_use_carries_a_named_generic_interface.f90").read_text(encoding="utf-8"), - ) - - assert ("convert", "convert", "generic_home") in carried - - -def test_a_plain_use_under_a_private_default_carries_nothing(tmp_path: Path): - """The importing module's default decides what it publishes in turn.""" - assert ( - _reexports( - tmp_path, - """\ -module b_mod - use a_mod - implicit none - private -end module b_mod -""", - ) - == [] - ) - - -@pytest.mark.parametrize("kind", ["variable", "procedure"]) -def test_accessibility_decides_every_re_exportable_kind(kind: str, tmp_path: Path): - """The rule is about accessibility, so it does not single out one kind.""" - name = "x" if kind == "variable" else "scale_value" - published = _reexports( - tmp_path, - f"""\ -module b_mod - use a_mod, only : {name} - implicit none -end module b_mod -""", - ) - - assert published == [(name, name, "a_mod")] +TRANSITIVE_DECLARING = (NATIVE_FIXTURES / "transitive_declaring.f90").read_text(encoding="utf-8") +TRANSITIVE_OTHER = (NATIVE_FIXTURES / "transitive_other.f90").read_text(encoding="utf-8") CALLBACK_HOME = """\ module callback_types @@ -286,180 +40,12 @@ def test_accessibility_decides_every_re_exportable_kind(kind: str, tmp_path: Pat end module callback_types """ - -def _callback_reexports(tmp_path: Path, importer: str, *, module_name: str) -> list[tuple[str, str, str]]: - """Return one module's public use associations over an abstract-interface home.""" - source = tmp_path / "callbacks.f90" - source.write_text(f"{CALLBACK_HOME}\n{importer}", encoding="utf-8") - modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) - importing = next(module for module in modules if module.name == module_name) - return [(item.local_name, item.source_name, item.origin_module) for item in importing.reexports] - - -def test_a_plain_use_carries_an_abstract_interface_procedure(tmp_path: Path): - """An abstract block names no generic; what it declares are its procedures.""" - assert _callback_reexports( - tmp_path, - """\ -module middle_mod - use callback_types - implicit none -end module middle_mod -""", - module_name="middle_mod", - ) == [("unary", "unary", "callback_types")] - - -def test_an_abstract_interface_procedure_survives_a_further_hop(tmp_path: Path): - """Carrying it once makes it importable by name from the carrying module.""" - assert _callback_reexports( - tmp_path, - """\ -module middle_mod - use callback_types - implicit none -end module middle_mod - -module user_mod - use middle_mod, only : unary - implicit none -end module user_mod -""", - module_name="user_mod", - ) == [("unary", "unary", "callback_types")] - - -def test_a_callback_reached_through_a_public_route_stays_public(tmp_path: Path): - """Callback accessibility is the module's accessibility, routes included. - - A bare `private` would hide the name were the used module not named public, - so judging it by the symbol statements alone reaches the wrong answer. - """ - assert _callback_reexports( - tmp_path, - """\ -module facade_mod - use callback_types - implicit none - private - public :: callback_types -end module facade_mod -""", - module_name="facade_mod", - ) == [("unary", "unary", "callback_types")] - - -def test_a_callback_reached_through_a_private_route_is_withheld(tmp_path: Path): - """Naming the used module private withholds what it carried, default aside.""" - assert ( - _callback_reexports( - tmp_path, - """\ -module facade_mod - use callback_types - implicit none - private :: callback_types -end module facade_mod -""", - module_name="facade_mod", - ) - == [] - ) - - -def test_routes_that_agree_on_one_entity_publish_it(tmp_path: Path): - """Two `use` statements naming the same declaration name one entity.""" - assert _reexports( - tmp_path, - """\ -module middle_mod - use a_mod, only : x - implicit none -end module middle_mod - -module b_mod - use a_mod, only : x - use middle_mod, only : x - implicit none -end module b_mod -""", - ) == [("x", "x", "a_mod")] - - -def test_a_readable_route_beside_an_unreadable_one_is_not_guessed(tmp_path: Path): - """An unparsed module may carry the same entity or another one. - - Choosing the readable route would be a guess about the one this project - cannot read, so the name is left out rather than resolved to either. - """ - assert ( - _reexports( - tmp_path, - """\ -module b_mod - use a_mod, only : x - use external_mod, only : x - implicit none -end module b_mod -""", - ) - == [] - ) - - -def test_a_single_unreadable_route_still_names_what_it_reached(tmp_path: Path): - """One route names one entity, whether or not this project can read it.""" - assert _reexports( - tmp_path, - """\ -module b_mod - use external_mod, only : y - implicit none -end module b_mod -""", - ) == [("y", "y", "external_mod")] - - -def test_a_procedure_local_abstract_interface_stays_inside_its_procedure(tmp_path: Path): - """A block written inside a contained procedure declares a name only there. - - Those blocks are stored beside the module's own, so nothing but the - declaring scope distinguishes them. - """ - assert _reexports( - tmp_path, - """\ -module local_home - implicit none -contains - subroutine work() - abstract interface - subroutine local_callback() - end subroutine local_callback - end interface - end subroutine work -end module local_home - -module b_mod - use local_home - implicit none -end module b_mod -""", - ) == [("work", "work", "local_home")] - - -def test_a_procedure_local_generic_stays_inside_its_procedure(tmp_path: Path): - """A named generic declared inside a procedure is that procedure's, too.""" - carried = _reexports( - tmp_path, - """\ +LOCAL_HOME = """\ module local_home implicit none contains subroutine work() - interface local_generic - module procedure work - end interface local_generic +{local} end subroutine work end module local_home @@ -467,209 +53,9 @@ def test_a_procedure_local_generic_stays_inside_its_procedure(tmp_path: Path): use local_home implicit none end module b_mod -""", - ) - - assert [local for local, _source, _origin in carried] == ["work"] - - -def test_a_wildcard_route_beside_an_unreadable_one_is_not_guessed(tmp_path: Path): - """A plain `use` compares routes the way a named import does. - - Discarding the unreadable route would leave the readable one standing - alone and answer for a module this project never read. - """ - assert ( - _reexports( - tmp_path, - """\ -module left_mod - use a_mod, only : x - implicit none -end module left_mod - -module right_mod - use external_mod, only : x - implicit none -end module right_mod - -module b_mod - use left_mod - use right_mod - implicit none -end module b_mod -""", - ) - == [] - ) - - -def test_wildcard_routes_that_agree_on_one_entity_publish_it(tmp_path: Path): - """Repeating a route to the same declaration names one entity.""" - assert _reexports( - tmp_path, - """\ -module left_mod - use a_mod, only : x - implicit none -end module left_mod - -module b_mod - use left_mod - use a_mod, only : x - implicit none -end module b_mod -""", - ) == [("x", "x", "a_mod")] - - -def test_a_name_spelled_inside_a_character_literal_is_not_a_dependency(tmp_path: Path): - """A literal's contents are its value, not a reference to what they spell.""" - source = tmp_path / "project.f90" - source.write_text( - f"""{DECLARING} -module b_mod - use a_mod, only : box - implicit none - character(len=3), parameter :: label = "box" -end module b_mod -""", - encoding="utf-8", - ) - modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) - importing = next(module for module in modules if module.name == "b_mod") - - assert [(item.local_name, item.declaration_dependency) for item in importing.reexports] == [("box", False)] - - -def test_a_type_a_declaration_names_is_a_dependency(tmp_path: Path): - """Declaring with an imported type is what makes it a dependency.""" - source = tmp_path / "project.f90" - source.write_text( - f"""{DECLARING} -module b_mod - use a_mod, only : box - implicit none - type(box) :: item -end module b_mod -""", - encoding="utf-8", - ) - modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) - importing = next(module for module in modules if module.name == "b_mod") - - assert [(item.local_name, item.declaration_dependency) for item in importing.reexports] == [("box", True)] - - -def test_a_compile_time_symbol_is_not_substituted_inside_a_character_literal(): - """A literal's contents are data, so a symbol spelled there is not a reference.""" - from prik.semantics.fortran2ir import _resolve_compile_time_text - - values = {"runtime": "4"} - - assert _resolve_compile_time_text('len("runtime")', values) == 'len("runtime")' - # A reference outside the literal is still resolved. - assert _resolve_compile_time_text("runtime + 1", values) == "4 + 1" - assert _resolve_compile_time_text('len("runtime") + runtime', values) == 'len("runtime") + 4' - - -TRANSITIVE_DECLARING = (NATIVE_FIXTURES / "transitive_declaring.f90").read_text(encoding="utf-8") - -TRANSITIVE_OTHER = (NATIVE_FIXTURES / "transitive_other.f90").read_text(encoding="utf-8") - - -def _project_modules(tmp_path: Path, *sources: str): - """Parse one throwaway project and return its semantic modules by name.""" - (tmp_path / "project.f90").write_text("\n".join(sources), encoding="utf-8") - modules = fortran_project_to_semantic_modules(parse_fortran_project(str(tmp_path))) - return {module.name: module for module in modules} - - -def test_a_private_name_in_an_intermediate_module_ends_the_chain(tmp_path: Path): - """Each hop applies the accessibility rule, so a `private` stops the walk. - - `middle` imports `x` and makes it private, so `outer` cannot reach the - declaration behind it however `middle` got there. - """ - modules = _project_modules( - tmp_path, - TRANSITIVE_DECLARING, - """\ -module middle_mod - use a_mod, only : x - implicit none - private :: x -end module middle_mod - -module outer_mod - use middle_mod, only : x - implicit none -end module outer_mod -""", - ) - - reexports = {item.local_name: item for item in modules["outer_mod"].reexports} - assert reexports["x"].entity_kind == "unknown" - assert reexports["x"].origin_module == "middle_mod" - - -def test_routes_disagreeing_inside_an_intermediate_module_stay_unresolved(tmp_path: Path): - """`middle` reaches two different `x`, so no hop through it names one.""" - modules = _project_modules( - tmp_path, - TRANSITIVE_DECLARING, - TRANSITIVE_OTHER, - """\ -module middle_mod - use a_mod, only : x - use c_mod, only : x - implicit none -end module middle_mod - -module outer_mod - use middle_mod, only : x - implicit none -end module outer_mod -""", - ) - - reexports = {item.local_name: item for item in modules["outer_mod"].reexports} - assert reexports["x"].entity_kind == "unknown" - assert reexports["x"].origin_module == "middle_mod" - - -def test_an_ordinary_chain_still_reaches_the_declaring_module(tmp_path: Path): - """One accessible, unambiguous route per hop resolves to the declaration.""" - modules = _project_modules( - tmp_path, - TRANSITIVE_DECLARING, - """\ -module middle_mod - use a_mod, only : x - implicit none - public :: x -end module middle_mod - -module outer_mod - use middle_mod, only : x - implicit none -end module outer_mod -""", - ) - - reexports = {item.local_name: item for item in modules["outer_mod"].reexports} - assert (reexports["x"].entity_kind, reexports["x"].origin_module, reexports["x"].source_name) == ( - "variable", - "a_mod", - "x", - ) - +""" -def test_an_enumerator_is_carried_and_classified_as_the_constant_it_is(tmp_path: Path): - """An enum names constants, which is how every later stage models them.""" - modules = _project_modules( - tmp_path, - """\ +ENUM_HOME = """\ module colors_mod implicit none enum, bind(c) @@ -677,242 +63,439 @@ def test_an_enumerator_is_carried_and_classified_as_the_constant_it_is(tmp_path: enumerator :: green = 2 end enum end module colors_mod +""" -module facade_mod - use colors_mod - implicit none -end module facade_mod - -module named_facade_mod - use colors_mod, only : red - implicit none -end module named_facade_mod -""", - ) - - # A plain `use` carries every public name, enumerators included. - carried = {item.local_name: item.entity_kind for item in modules["facade_mod"].reexports} - assert carried == {"red": "variable", "green": "variable"} - - named = {item.local_name: item for item in modules["named_facade_mod"].reexports} - assert named["red"].entity_kind == "variable" - assert (named["red"].origin_module, named["red"].source_name) == ("colors_mod", "red") - - -def test_an_enumerator_initializer_is_a_declaration_dependency(tmp_path: Path): - """A name an enum's value reads expresses a declaration, so it is a dependency.""" - modules = _project_modules( - tmp_path, - """\ -module constants_mod - implicit none - integer, parameter :: base = 10 -end module constants_mod - -module colors_mod - use constants_mod, only : base - implicit none - enum, bind(c) - enumerator :: red = base - end enum -end module colors_mod -""", - ) - - reexports = {item.local_name: item for item in modules["colors_mod"].reexports} - assert reexports["base"].declaration_dependency is True - - -def test_a_named_and_a_wildcard_route_to_different_entities_stay_unresolved(tmp_path: Path): - """How a route entered says nothing about what it carries. - - `b_mod` reaches two different `x`, one through an `only` list and one - through a plain `use`. Examining the named route first would publish - `a_mod::x` as the canonical one, and a re-exported module variable - generates native access to that owner directly, so the Fortran compiler - never gets to diagnose the ambiguity. - """ - modules = _project_modules( - tmp_path, - TRANSITIVE_DECLARING, - TRANSITIVE_OTHER, - """\ -module b_mod - use a_mod, only : x - use c_mod +Q_HOME = """\ +module a_mod implicit none -end module b_mod -""", - ) - - assert [item.local_name for item in modules["b_mod"].reexports] == [] - + integer :: q = 1 + integer :: other = 2 +end module a_mod +""" -def test_a_named_and_a_wildcard_route_to_one_entity_resolve_together(tmp_path: Path): - """Two routes naming one declaration are not a disagreement.""" - modules = _project_modules( - tmp_path, - TRANSITIVE_DECLARING, - """\ -module pass_mod - use a_mod +USER_ISO_FORTRAN_ENV = """\ +module iso_fortran_env implicit none -end module pass_mod + integer :: my_value = 7 +end module iso_fortran_env +""" +CRATE_IMPORTER = """\ module b_mod - use a_mod, only : x - use pass_mod + use a_mod, only : crate => box implicit none +{access} +contains + integer function crate_value(item) result(out) + type(crate), intent(in) :: item + out = item%value + end function crate_value end module b_mod -""", - ) - - reexports = {item.local_name: item for item in modules["b_mod"].reexports} - assert (reexports["x"].entity_kind, reexports["x"].origin_module) == ("variable", "a_mod") +""" -def test_an_unparsed_plain_use_carries_no_assumed_name(tmp_path: Path): - """PRIK cannot enumerate an unread module, so it is not a route for a name.""" - modules = _project_modules( - tmp_path, - TRANSITIVE_DECLARING, - """\ -module b_mod - use a_mod, only : x - use external_mod - implicit none -end module b_mod -""", - ) - - reexports = {item.local_name: item for item in modules["b_mod"].reexports} - assert (reexports["x"].entity_kind, reexports["x"].origin_module) == ("variable", "a_mod") +def _modules(tmp_path: Path, *sources: str): + """Parse one throwaway project and return its semantic modules by name.""" + (tmp_path / "project.f90").write_text("\n".join(sources), encoding="utf-8") + modules = fortran_project_to_semantic_modules(parse_fortran_project(str(tmp_path))) + return {module.name: module for module in modules} -def test_mixed_routes_through_an_intermediate_module_stay_unresolved(tmp_path: Path): - """The rule is the same at every hop, whichever way each route entered.""" - modules = _project_modules( - tmp_path, - TRANSITIVE_DECLARING, - TRANSITIVE_OTHER, - """\ -module middle_mod - use a_mod, only : x - use c_mod - implicit none -end module middle_mod - -module outer_mod - use middle_mod, only : x - implicit none -end module outer_mod -""", - ) +def _row(id_: str, sources: tuple[str, ...], expected: list[tuple[str, str, str, str]], module: str = "b_mod"): + return pytest.param(sources, module, expected, id=id_) + + +A_MOD_PUBLIC = [ + ("box", "box", "a_mod", "derived_type"), + ("scale_value", "scale_value", "a_mod", "procedure"), + ("x", "x", "a_mod", "variable"), + ("y", "y", "a_mod", "variable"), +] +X_FROM_A = [("x", "x", "a_mod", "variable")] +X_UNRESOLVED_AT_MIDDLE = [("x", "x", "middle_mod", "unknown")] +UNARY = [("unary", "unary", "callback_types", "prototype")] + +PUBLICATION_CASES = [ + # Accessibility precedence over the module default. + _row("default-public-only-list", (DECLARING, "module b_mod\n use a_mod, only : x\nend module b_mod\n"), X_FROM_A), + _row( + "default-public-procedure", + (DECLARING, "module b_mod\n use a_mod, only : scale_value\nend module b_mod\n"), + [("scale_value", "scale_value", "a_mod", "procedure")], + ), + _row( + "declaration-dependency-stays-public", + (DECLARING, CRATE_IMPORTER.format(access="")), + [("crate", "box", "a_mod", "derived_type")], + ), + _row( + "explicit-public-declaration-dependency", + (DECLARING, CRATE_IMPORTER.format(access=" public :: crate")), + [("crate", "box", "a_mod", "derived_type")], + ), + _row( + "third-module-resolves-through-importer", + ( + DECLARING, + "module b_mod\n use a_mod, only : box\n type(box) :: stored\nend module b_mod\n", + "module c_mod\n use b_mod, only : box\n type(box) :: another\nend module c_mod\n", + ), + [("box", "box", "a_mod", "derived_type")], + module="c_mod", + ), + _row("private-default", (DECLARING, "module b_mod\n use a_mod, only : x\n private\nend module b_mod\n"), []), + _row( + "public-name-over-private-default", + (DECLARING, "module b_mod\n use a_mod, only : x\n private\n public :: x\nend module b_mod\n"), + X_FROM_A, + ), + _row( + "private-name-over-public-default", + (DECLARING, "module b_mod\n use a_mod, only : x\n private :: x\nend module b_mod\n"), + [], + ), + _row("private-module-route", (DECLARING, "module b_mod\n use a_mod\n private :: a_mod\nend module b_mod\n"), []), + _row( + "public-module-route-over-private-default", + (DECLARING, "module b_mod\n use a_mod\n private\n public :: a_mod\nend module b_mod\n"), + A_MOD_PUBLIC, + ), + _row( + "any-public-route-keeps-an-entity-public", + ( + DECLARING, + "module left_mod\n use a_mod, only : x\nend module left_mod\n", + "module right_mod\n use a_mod, only : x\nend module right_mod\n", + "module b_mod\n use left_mod\n use right_mod\n private :: left_mod\n public :: right_mod\nend module b_mod\n", + ), + X_FROM_A, + ), + _row( + "rename-publishes-the-local-name", + (DECLARING, "module b_mod\n use a_mod, only : renamed => y\nend module b_mod\n"), + [("renamed", "y", "a_mod", "variable")], + ), + _row( + "plain-use-carries-every-public-name", + (DECLARING, "module b_mod\n use a_mod\nend module b_mod\n"), + A_MOD_PUBLIC, + ), + _row( + "plain-use-carries-a-named-generic-interface", + (NAMED_GENERIC,), + [ + ("convert", "convert", "generic_home", "generic"), + ("convert_i", "convert_i", "generic_home", "procedure"), + ("convert_r", "convert_r", "generic_home", "procedure"), + ], + ), + _row( + "plain-use-under-private-default-carries-nothing", + (DECLARING, "module b_mod\n use a_mod\n private\nend module b_mod\n"), + [], + ), + # An abstract block names no generic; what it declares are its procedures. + _row( + "plain-use-carries-an-abstract-interface-procedure", + (CALLBACK_HOME, "module middle_mod\n use callback_types\nend module middle_mod\n"), + UNARY, + module="middle_mod", + ), + _row( + "abstract-interface-procedure-survives-a-further-hop", + ( + CALLBACK_HOME, + "module middle_mod\n use callback_types\nend module middle_mod\n", + "module user_mod\n use middle_mod, only : unary\nend module user_mod\n", + ), + UNARY, + module="user_mod", + ), + # Callback accessibility is the module's accessibility, routes included: a + # bare `private` would hide the name were the used module not named public. + _row( + "callback-through-a-public-route-stays-public", + ( + CALLBACK_HOME, + "module facade_mod\n use callback_types\n private\n public :: callback_types\nend module facade_mod\n", + ), + UNARY, + module="facade_mod", + ), + _row( + "callback-through-a-private-route-is-withheld", + ( + CALLBACK_HOME, + "module facade_mod\n use callback_types\n private :: callback_types\nend module facade_mod\n", + ), + [], + module="facade_mod", + ), + # Agreeing and disagreeing routes. + _row( + "routes-agreeing-on-one-entity-publish-it", + ( + DECLARING, + "module middle_mod\n use a_mod, only : x\nend module middle_mod\n", + "module b_mod\n use a_mod, only : x\n use middle_mod, only : x\nend module b_mod\n", + ), + X_FROM_A, + ), + # An unparsed module may carry the same entity or another one, so choosing + # the readable route would be a guess about the one this project cannot read. + _row( + "readable-route-beside-an-unreadable-one-is-not-guessed", + (DECLARING, "module b_mod\n use a_mod, only : x\n use external_mod, only : x\nend module b_mod\n"), + [], + ), + _row( + "single-unreadable-route-names-what-it-reached", + (DECLARING, "module b_mod\n use external_mod, only : y\nend module b_mod\n"), + [("y", "y", "external_mod", "unknown")], + ), + _row( + "wildcard-route-beside-an-unreadable-one-is-not-guessed", + ( + DECLARING, + "module left_mod\n use a_mod, only : x\nend module left_mod\n", + "module right_mod\n use external_mod, only : x\nend module right_mod\n", + "module b_mod\n use left_mod\n use right_mod\nend module b_mod\n", + ), + [], + ), + _row( + "wildcard-routes-agreeing-on-one-entity-publish-it", + ( + DECLARING, + "module left_mod\n use a_mod, only : x\nend module left_mod\n", + "module b_mod\n use left_mod\n use a_mod, only : x\nend module b_mod\n", + ), + X_FROM_A, + ), + # How a route entered says nothing about what it carries: examining the + # named route first would publish `a_mod::x`, and a re-exported module + # variable generates native access to that owner directly. + _row( + "named-and-wildcard-routes-to-different-entities-stay-unresolved", + ( + TRANSITIVE_DECLARING, + TRANSITIVE_OTHER, + "module b_mod\n use a_mod, only : x\n use c_mod\nend module b_mod\n", + ), + [], + ), + _row( + "named-and-wildcard-routes-to-one-entity-resolve-together", + ( + TRANSITIVE_DECLARING, + "module pass_mod\n use a_mod\nend module pass_mod\n", + "module b_mod\n use a_mod, only : x\n use pass_mod\nend module b_mod\n", + ), + X_FROM_A, + ), + # PRIK cannot enumerate an unread module, so it is not a route for a name. + _row( + "unparsed-plain-use-carries-no-assumed-name", + (TRANSITIVE_DECLARING, "module b_mod\n use a_mod, only : x\n use external_mod\nend module b_mod\n"), + X_FROM_A, + ), + # A block written inside a contained procedure declares a name only there. + _row( + "procedure-local-abstract-interface-stays-in-its-procedure", + ( + LOCAL_HOME.format( + local=" abstract interface\n subroutine local_callback()\n end subroutine local_callback\n" + " end interface" + ), + ), + [("work", "work", "local_home", "procedure")], + ), + _row( + "procedure-local-generic-stays-in-its-procedure", + ( + LOCAL_HOME.format( + local=" interface local_generic\n module procedure work\n end interface local_generic" + ), + ), + [("work", "work", "local_home", "procedure")], + ), + # Each hop of a transitive chain applies the same rule. + _row( + "private-name-in-an-intermediate-module-ends-the-chain", + ( + TRANSITIVE_DECLARING, + "module middle_mod\n use a_mod, only : x\n private :: x\nend module middle_mod\n", + "module outer_mod\n use middle_mod, only : x\nend module outer_mod\n", + ), + X_UNRESOLVED_AT_MIDDLE, + module="outer_mod", + ), + _row( + "routes-disagreeing-inside-an-intermediate-module-stay-unresolved", + ( + TRANSITIVE_DECLARING, + TRANSITIVE_OTHER, + "module middle_mod\n use a_mod, only : x\n use c_mod, only : x\nend module middle_mod\n", + "module outer_mod\n use middle_mod, only : x\nend module outer_mod\n", + ), + X_UNRESOLVED_AT_MIDDLE, + module="outer_mod", + ), + _row( + "mixed-routes-through-an-intermediate-module-stay-unresolved", + ( + TRANSITIVE_DECLARING, + TRANSITIVE_OTHER, + "module middle_mod\n use a_mod, only : x\n use c_mod\nend module middle_mod\n", + "module outer_mod\n use middle_mod, only : x\nend module outer_mod\n", + ), + X_UNRESOLVED_AT_MIDDLE, + module="outer_mod", + ), + _row( + "ordinary-chain-reaches-the-declaring-module", + ( + TRANSITIVE_DECLARING, + "module middle_mod\n use a_mod, only : x\n public :: x\nend module middle_mod\n", + "module outer_mod\n use middle_mod, only : x\nend module outer_mod\n", + ), + X_FROM_A, + module="outer_mod", + ), + # An enum names constants, which is how every later stage models them. + _row( + "plain-use-carries-enumerators-as-constants", + (ENUM_HOME, "module facade_mod\n use colors_mod\nend module facade_mod\n"), + [("green", "green", "colors_mod", "variable"), ("red", "red", "colors_mod", "variable")], + module="facade_mod", + ), + _row( + "only-list-names-an-enumerator", + (ENUM_HOME, "module facade_mod\n use colors_mod, only : red\nend module facade_mod\n"), + [("red", "red", "colors_mod", "variable")], + module="facade_mod", + ), + # ONLY, renaming, and repeated USE statements share one route interpretation. + # A rename hides the source spelling; `only` narrows to what it lists (see + # the default-public rows above). + _row( + "rename-without-only-hides-the-source-name", + (Q_HOME, "module b_mod\n use a_mod, p => q\nend module b_mod\n"), + [("other", "other", "a_mod", "variable"), ("p", "q", "a_mod", "variable")], + ), + _row("empty-only-list", (TRANSITIVE_DECLARING, "module b_mod\n use a_mod, only :\nend module b_mod\n"), []), + _row( + "repeated-statements", + (Q_HOME, "module b_mod\n use a_mod, only : p => q\n use a_mod\nend module b_mod\n"), + [("other", "other", "a_mod", "variable"), ("p", "q", "a_mod", "variable")], + ), + # A user module may share an intrinsic module's name; the `use` nature decides which is meant. + _row( + "non-intrinsic-names-the-user-module", + ( + USER_ISO_FORTRAN_ENV, + "module facade\n use, non_intrinsic :: iso_fortran_env, only: my_value\nend module facade\n", + ), + [("my_value", "my_value", "iso_fortran_env", "variable")], + module="facade", + ), + _row( + "unstated-nature-prefers-the-parsed-module", + (USER_ISO_FORTRAN_ENV, "module facade\n use iso_fortran_env, only: my_value\nend module facade\n"), + [("my_value", "my_value", "iso_fortran_env", "variable")], + module="facade", + ), + _row( + "intrinsic-names-the-processor-module", + (USER_ISO_FORTRAN_ENV, "module facade\n use, intrinsic :: iso_fortran_env, only: int32\nend module facade\n"), + [("int32", "int32", "iso_fortran_env", "intrinsic")], + module="facade", + ), + # Semantic resolution and source discovery share one inventory of processor modules. + _row( + "plain-use-of-an-ieee-module-names-the-processor-module", + ("module facade\n use ieee_arithmetic, only: ieee_is_nan\nend module facade\n",), + [("ieee_is_nan", "ieee_is_nan", "ieee_arithmetic", "intrinsic")], + module="facade", + ), +] + + +@pytest.mark.parametrize(("sources", "module", "expected"), PUBLICATION_CASES) +def test_a_module_publishes_exactly_its_accessible_use_associations( + sources: tuple[str, ...], + module: str, + expected: list[tuple[str, str, str, str]], + tmp_path: Path, +): + """Each published name records its local spelling, declaration, owner, and kind.""" + reexports = _modules(tmp_path, *sources)[module].reexports - reexports = {item.local_name: item for item in modules["outer_mod"].reexports} - assert reexports["x"].entity_kind == "unknown" - assert reexports["x"].origin_module == "middle_mod" + published = [(item.local_name, item.source_name, item.origin_module, item.entity_kind) for item in reexports] + assert sorted(published) == sorted(expected) @pytest.mark.parametrize( - ("sources", "expected"), + ("sources", "module", "name", "dependency"), [ pytest.param( - ( - """\ -module a_mod - implicit none - integer :: q = 1 - integer :: other = 2 -end module a_mod - -module b_mod - use a_mod, p => q - implicit none -end module b_mod -""", - ), - {"p": ("a_mod", "q"), "other": ("a_mod", "other")}, - id="rename-without-only", - ), - pytest.param( - ( - """\ -module a_mod - implicit none - integer :: q = 1 - integer :: other = 2 -end module a_mod - -module b_mod - use a_mod, only : q - implicit none -end module b_mod -""", - ), - {"q": ("a_mod", "q")}, - id="only-list", + (DECLARING, CRATE_IMPORTER.format(access="")), + "b_mod", + "crate", + True, + id="a-dummy-type-is-a-dependency", ), pytest.param( - ( - """\ -module a_mod - implicit none - integer :: q = 1 -end module a_mod - -module b_mod - use a_mod, p => q - implicit none -end module b_mod -""", - ), - {"p": ("a_mod", "q")}, - id="renamed-source-name-hidden", + (DECLARING, "module b_mod\n use a_mod, only : box\n type(box) :: item\nend module b_mod\n"), + "b_mod", + "box", + True, + id="a-declared-type-is-a-dependency", ), pytest.param( ( - TRANSITIVE_DECLARING, - """\ -module b_mod - use a_mod, only : - implicit none -end module b_mod -""", + DECLARING, + 'module b_mod\n use a_mod, only : box\n character(len=3), parameter :: label = "box"\nend module b_mod\n', ), - {}, - id="empty-only-list", + "b_mod", + "box", + False, + id="a-name-spelled-in-a-character-literal-is-not", ), pytest.param( ( - """\ -module a_mod - implicit none - integer :: q = 1 - integer :: other = 2 -end module a_mod - -module b_mod - use a_mod, only : p => q - use a_mod - implicit none -end module b_mod -""", + "module constants_mod\n integer, parameter :: base = 10\nend module constants_mod\n", + "module colors_mod\n use constants_mod, only : base\n enum, bind(c)\n" + " enumerator :: red = base\n end enum\nend module colors_mod\n", ), - {"p": ("a_mod", "q"), "other": ("a_mod", "other")}, - id="repeated-statements", + "colors_mod", + "base", + True, + id="an-enumerator-initializer-is-a-dependency", ), ], ) -def test_use_statement_forms_define_the_accessible_local_names( - sources: tuple[str, ...], - expected: dict[str, tuple[str, str]], - tmp_path: Path, +def test_a_name_a_declaration_reads_is_a_declaration_dependency( + sources: tuple[str, ...], module: str, name: str, dependency: bool, tmp_path: Path ): - """ONLY, renaming, and repeated USE statements share one route interpretation.""" - modules = _project_modules(tmp_path, *sources) - reexports = {item.local_name: (item.origin_module, item.source_name) for item in modules["b_mod"].reexports} - assert reexports == expected + """A literal's contents are its value, not a reference to what they spell.""" + reexports = {item.local_name: item for item in _modules(tmp_path, *sources)[module].reexports} + + assert reexports[name].declaration_dependency is dependency + + +def test_a_compile_time_symbol_is_not_substituted_inside_a_character_literal(): + """A literal's contents are data, so a symbol spelled there is not a reference.""" + from prik.semantics.fortran2ir import _resolve_compile_time_text + + values = {"runtime": "4"} + + assert _resolve_compile_time_text('len("runtime")', values) == 'len("runtime")' + # A reference outside the literal is still resolved, whatever its case, and + # an unknown name is left as written. + assert _resolve_compile_time_text("runtime + 1", values) == "4 + 1" + assert _resolve_compile_time_text("RUNTIME + missing", values) == "4 + missing" + assert _resolve_compile_time_text('len("runtime") + runtime', values) == 'len("runtime") + 4' def test_a_non_only_rename_still_carries_imported_compile_time_symbols(tmp_path: Path): @@ -949,45 +532,6 @@ def test_a_non_only_rename_still_carries_imported_compile_time_symbols(tmp_path: assert (declared.kind, declared.shape) == ("8", ["4"]) -USER_ISO_FORTRAN_ENV = """\ -module iso_fortran_env - implicit none - integer :: my_value = 7 -end module iso_fortran_env -""" - - -@pytest.mark.parametrize( - ("statement", "expected"), - [ - pytest.param( - "use, non_intrinsic :: iso_fortran_env, only: my_value", - ("my_value", "variable", "iso_fortran_env"), - id="non-intrinsic-names-the-user-module", - ), - pytest.param( - "use iso_fortran_env, only: my_value", - ("my_value", "variable", "iso_fortran_env"), - id="unstated-prefers-the-parsed-module", - ), - pytest.param( - "use, intrinsic :: iso_fortran_env, only: int32", - ("int32", "intrinsic", "iso_fortran_env"), - id="intrinsic-names-the-processor-module", - ), - ], -) -def test_use_nature_decides_whether_an_intrinsic_name_is_the_users_module(tmp_path: Path, statement, expected): - """A user module may share an intrinsic module's name; the ``use`` nature decides which is meant.""" - source = tmp_path / "project.f90" - source.write_text(f"{USER_ISO_FORTRAN_ENV}\nmodule facade\n {statement}\nend module facade\n", encoding="utf-8") - - modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) - facade = next(module for module in modules if module.name == "facade") - - assert [(item.local_name, item.entity_kind, item.origin_module) for item in facade.reexports] == [expected] - - USER_IEEE_ARITHMETIC = """\ module ieee_arithmetic implicit none @@ -1040,11 +584,7 @@ def test_an_intrinsic_use_reads_nothing_from_a_same_named_user_module( end subroutine apply end module consumer """ - source = tmp_path / "project.f90" - source.write_text(f"{USER_IEEE_ARITHMETIC}\n{consumer}", encoding="utf-8") - - modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) - module = next(item for item in modules if item.name == "consumer") + module = _modules(tmp_path, USER_IEEE_ARITHMETIC, consumer)["consumer"] callback, _count, values = next(function for function in module.functions if function.name == "apply").arguments (bound_call,) = values.semantic_type.storage.array.expression_callables[0] @@ -1067,22 +607,8 @@ def test_a_wildcard_use_resolves_a_derived_type_by_its_nature(tmp_path: Path, na f"module consumer\n use, {nature} :: ieee_arithmetic\ncontains\n subroutine inspect(value)\n" " type(ieee_class_type), intent(in) :: value\n end subroutine inspect\nend module consumer\n" ) - source = tmp_path / "project.f90" - source.write_text(f"{user}\n{consumer}", encoding="utf-8") - - modules = fortran_project_to_semantic_modules(parse_fortran_project([source])) - argument = next(module for module in modules if module.name == "consumer").functions[0].arguments[0] + argument = _modules(tmp_path, user, consumer)["consumer"].functions[0].arguments[0] reference = argument.semantic_type.metadata["external_type_ref"] assert reference["origin_module"] == "ieee_arithmetic" assert (bool(reference.get("processor")), reference["wrapped"]) == (processor, wrapped) - - -def test_a_plain_use_of_an_ieee_module_names_the_processor_module(tmp_path: Path): - """Semantic resolution and source discovery share one inventory of processor modules.""" - source = tmp_path / "facade.f90" - source.write_text("module facade\n use ieee_arithmetic, only: ieee_is_nan\nend module facade\n", encoding="utf-8") - - (facade,) = fortran_project_to_semantic_modules(parse_fortran_project([source])) - - assert [(item.local_name, item.entity_kind) for item in facade.reexports] == [("ieee_is_nan", "intrinsic")] diff --git a/tests/fortran/optional_arguments/codegen/test_optional_lowering.py b/tests/fortran/optional_arguments/codegen/test_optional_lowering.py index 37841cfa4..9d42fc27b 100644 --- a/tests/fortran/optional_arguments/codegen/test_optional_lowering.py +++ b/tests/fortran/optional_arguments/codegen/test_optional_lowering.py @@ -10,14 +10,11 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from prik.pipeline.pyi import pyi_file_to_semantic_module from prik.policy.completion import complete_semantic_policies -from prik.policy.models import BridgeDataAction, OptionalMode +from prik.policy.models import OptionalMode from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -OPTIONAL_FIXED_CONTRACT = ( - Path(__file__).parents[1] / "end_to_end" / "fixtures" / "contracts" / "foptional_fixed" / "__init__.pyi" -) OPTIONAL_MIXED_CONTRACT = ( Path(__file__).parents[1] / "end_to_end" / "fixtures" / "contracts" / "foptional_f90" / "foptional_f90.pyi" ) @@ -37,59 +34,6 @@ def _replace_root_function(plan, function): return replace(plan, namespaces=(replace(root, functions=(function,)), *plan.namespaces[1:])) -def test_optional_scalar_lowering_distinguishes_absent_or_none_from_value(): - module = pyi_file_to_semantic_module(OPTIONAL_FIXED_CONTRACT, module_name="foptional_fixed") - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - factor = plan.namespaces[0].functions[0].arguments[1] - - assert factor.binding.optional_mode is OptionalMode.NULLABLE_VALUE - assert factor.entrypoint.optional_mode is OptionalMode.NULLABLE_VALUE - artifacts = WrapperGenerator().generate(plan) - c_source = _source(artifacts, ".c") - fortran_source = _source(artifacts, ".f90") - - assert 'PyArg_ParseTupleAndKeywords(args, kwargs, "O|O"' in c_source - assert "PyObject * bound_factor_obj = Py_None;" in c_source - assert "if (bound_factor_obj != Py_None)" in c_source - assert "bound_factor_nullable = bound_factor_storage;" in c_source - assert "bind_c_optional_scale(base, bound_factor)" in fortran_source - assert "if (c_associated(bound_factor)) then" in fortran_source - assert "result = optional_scale(base=base, factor=prik_optional_factor)" in fortran_source - assert fortran_source.count("result = optional_scale(") == 1 - - -def test_optional_descriptor_lowering_records_presence_and_nullable_value_handoffs(): - module = parse_pyi_text( - """ -@native_call([Allocatable(Arg(0))]) -def alloc_state(value: Annotated[Float64, Immutable] | None = ...) -> Int32: ... -""", - module_name="scalar_optional_descriptors", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - value = plan.namespaces[0].functions[0].arguments[0] - - assert value.binding.optional_mode is OptionalMode.DESCRIPTOR - assert value.entrypoint.presence_role == "scalar_optional_descriptors.alloc_state.value:present" - assert value.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - assert value.bridge.copy_reason == "materialize owned Fortran allocatable scalar storage from the binding value" - artifacts = WrapperGenerator().generate(plan) - c_source = _source(artifacts, ".c") - fortran_source = _source(artifacts, ".f90") - - assert "PyObject * bound_value_obj = NULL;" in c_source - assert "if (bound_value_obj != NULL)" in c_source - assert "bound_value_present = &bound_value;" in c_source - assert "(bound_value_obj != NULL) && (bound_value_obj != Py_None)" in c_source - assert "bind_c_alloc_state(bound_value_nullable, bound_value_present)" in c_source - assert "type(c_ptr), value :: bound_value_present" in fortran_source - assert "if (c_associated(bound_value_present)) then" in fortran_source - assert "result = native_alloc_state(value=prik_optional_value)" in fortran_source - assert fortran_source.count("result = native_alloc_state(") == 1 - - def test_optional_arguments_with_hidden_literals_materialize_the_literal_in_the_binding(): module = parse_pyi_text( """ diff --git a/tests/fortran/optional_arguments/end_to_end/test_optional_direct_entrypoint_routing.py b/tests/fortran/optional_arguments/end_to_end/test_optional_direct_entrypoint_routing.py index 0bfe3d406..60aebc53b 100644 --- a/tests/fortran/optional_arguments/end_to_end/test_optional_direct_entrypoint_routing.py +++ b/tests/fortran/optional_arguments/end_to_end/test_optional_direct_entrypoint_routing.py @@ -6,7 +6,6 @@ import pytest from tests.fortran._support.wrapper_build import ( - _build_inline_pyi_contract_module, _build_source_or_generated_pyi_and_import, ) @@ -75,24 +74,3 @@ def test_optional_mixed_route_adapts_only_optional_value_dummy( ) assert "bind_c_adapted_optional_value_state" in bridge assert "direct_optional_state" not in bridge - - -def test_optional_mixed_route_matches_edited_source_free_contract(tmp_path: Path): - stem = "optional_arguments_mixed_bind_c_f90" - source = (FIXTURES / "native" / f"{stem}.f90").read_text(encoding="utf-8") - contract = (FIXTURES / "contracts" / stem / f"{stem}.pyi").read_text(encoding="utf-8") - contract = contract.replace("from prik.contracts import ", "from prik.contracts import nogil, ") - contract = contract.replace("def direct_optional_state(", "@nogil\ndef direct_optional_state(").replace( - "def adapted_optional_value_state(", "@nogil\ndef adapted_optional_value_state(" - ) - module, result = _build_inline_pyi_contract_module( - tmp_path, module_name=stem, source_text=source, contract_text=contract - ) - - assert module.direct_optional_state() == np.int32(0) - assert module.direct_optional_state(np.float64(2.0)) == np.int32(1) - assert module.adapted_optional_value_state() == np.int32(0) - assert module.adapted_optional_value_state(np.float64(2.0)) == np.int32(2) - bridge = (result.output_dir / f"bind_c_{stem}_wrapper.f90").read_text(encoding="utf-8").casefold() - assert "bind_c_adapted_optional_value_state" in bridge - assert "function bind_c_direct_optional_state" not in bridge diff --git a/tests/fortran/optional_arguments/pipeline/test_generated_optional_contracts.py b/tests/fortran/optional_arguments/pipeline/test_generated_optional_contracts.py index d874dd0a6..60b9639d9 100644 --- a/tests/fortran/optional_arguments/pipeline/test_generated_optional_contracts.py +++ b/tests/fortran/optional_arguments/pipeline/test_generated_optional_contracts.py @@ -1,4 +1,8 @@ -"""Reviewed generated contracts for optional Fortran arguments.""" +"""Reviewed generated contract for fixed-form optional arguments. + +The free-form ``foptional_f90`` contract is compared against its fixture by the +generated-pyi lane of ``end_to_end/test_optional_runtime.py``. +""" from __future__ import annotations @@ -20,11 +24,6 @@ (FIXTURES / "native" / "foptional_fixed.f",), CONTRACT_ROOT / "foptional_fixed", ), - GeneratedContractCase( - "foptional_f90", - (FIXTURES / "native" / "foptional_f90.f90",), - CONTRACT_ROOT / "foptional_f90", - ), ) diff --git a/tests/fortran/optional_arguments/policy/test_optional_policy.py b/tests/fortran/optional_arguments/policy/test_optional_policy.py index 294d9ac50..290e9f201 100644 --- a/tests/fortran/optional_arguments/policy/test_optional_policy.py +++ b/tests/fortran/optional_arguments/policy/test_optional_policy.py @@ -1,118 +1,79 @@ -from pathlib import Path +"""Optional presence completes one three-state boundary decision before planning.""" +import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from tests.fortran._support.wrapper_build import wrapper_source -from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig, read_fortran_source -from prik.pipeline.pyi import pyi_file_to_semantic_module -from prik.semantics.fortran2ir import fortran_project_to_semantic_modules -from prik.semantics.models import ( - RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, -) +from prik.semantics.models import RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - ArgumentHandoffMode, - BridgeDataAction, - FunctionWrapperPolicy, - OptionalMode, -) +from prik.policy.models import ArgumentHandoffMode, FunctionWrapperPolicy, OptionalMode from prik.policy.construction import completed_function_wrapper_policy -FMATH_CONTRACT = Path("tests/fortran/data_types/end_to_end/fixtures/contracts/fmath/__init__.pyi") - - -def _source_semantic_module(filename: str, *, module_name: str): - source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name=module_name) - complete_semantic_policies(module) - return module - - -def test_optional_scalar_policy_completes_nullable_value_presence_before_planning(): - module = pyi_file_to_semantic_module( - Path(__file__).parents[1] / "end_to_end" / "fixtures" / "contracts" / "foptional_fixed" / "__init__.pyi", - module_name="foptional_fixed", - ) - complete_semantic_policies(module) - - policy = completed_function_wrapper_policy(module.functions[0]) - - assert policy.supported is True - assert [argument.optional_mode for argument in policy.arguments] == [ - OptionalMode.REQUIRED, - OptionalMode.NULLABLE_VALUE, - ] - assert policy.native_module is None - assert policy.native_is_subroutine is False - - -def test_optional_descriptor_policy_completes_three_state_boundary_before_planning(): - module = parse_pyi_text( - """ -@native_call([Allocatable(Arg(0))]) -def alloc_state(value: Annotated[Float64, Immutable] | None = ...) -> Int32: ... -""", - module_name="scalar_optional_descriptors", - ) - complete_semantic_policies(module) - - policy = completed_function_wrapper_policy(module.functions[0]) - value = policy.arguments[0] - - assert policy.supported is True - assert value.optional_mode is OptionalMode.DESCRIPTOR - assert value.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - assert value.bridge_copy_reason == "materialize owned Fortran allocatable scalar storage from the binding value" - assert value.nullable is True - assert value.descriptor_boundary is True - assert policy.native_module == "scalar_optional_descriptors" - -def test_optional_projected_array_keeps_nullable_value_separate_from_descriptor_storage(): - module = parse_pyi_text( - """ -@native_call([Addr(Arg(0)), Arg(1)]) -def fill_optional( - n: Int32, - values: Float64[::] = ..., -) -> Returns["values", Float64[::]] | None: ... -""", - module_name="optional_array_storage", - ) +@pytest.mark.parametrize( + ("contract", "position", "optional_mode", "nullable", "descriptor_boundary", "handoff_mode"), + [ + ( + "def scale(base: Float64, factor: Float64 = ...) -> Float64: ...", + 1, + OptionalMode.NULLABLE_VALUE, + False, + False, + None, + ), + ( + "@native_call([Allocatable(Arg(0))])\n" + "def alloc_state(value: Annotated[Float64, Immutable] | None = ...) -> Int32: ...", + 0, + OptionalMode.DESCRIPTOR, + True, + True, + None, + ), + ( + "@native_call([Allocatable(Arg(0))])\ndef alloc_state(value: Float64 | None) -> Int32: ...", + 0, + OptionalMode.REQUIRED_DESCRIPTOR, + True, + True, + None, + ), + ( + "@native_call([Addr(Arg(0)), Arg(1)])\n" + 'def fill_optional(n: Int32, values: Float64[::] = ...) -> Returns["values", Float64[::]] | None: ...', + 1, + OptionalMode.NULLABLE_VALUE, + True, + False, + ArgumentHandoffMode.ARRAY_BUFFER, + ), + ], + ids=[ + "optional-scalar-nullable-value", + "optional-descriptor-three-state", + "required-descriptor-nullable-at-native-boundary", + "optional-array-buffer-separate-from-descriptor-storage", + ], +) +def test_optional_policy_completes_presence_mode_before_planning( + contract, position, optional_mode, nullable, descriptor_boundary, handoff_mode +): + """Omitted, ``None``, and a value are three states; policy says which ones the ABI carries. + + A plain optional passes a null pointer for omitted or ``None``. A + descriptor dummy also distinguishes a present-but-unallocated descriptor, + and a required descriptor argument still accepts ``None`` for that state. + """ + module = parse_pyi_text(contract, module_name="optional_policy") complete_semantic_policies(module) - policy = completed_function_wrapper_policy(module.functions[0]) - values = policy.arguments[1] + argument = policy.arguments[position] assert policy.supported is True - assert values.optional_mode is OptionalMode.NULLABLE_VALUE - assert values.nullable is True - assert values.descriptor_boundary is False - assert values.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER - - -def test_required_descriptor_policy_keeps_required_python_argument_nullable_at_native_boundary(): - module = parse_pyi_text( - """ -@native_call([Allocatable(Arg(0))]) -def alloc_state(value: Float64 | None) -> Int32: ... -""", - module_name="scalar_required_descriptors", - ) - complete_semantic_policies(module) - - value = completed_function_wrapper_policy(module.functions[0]).arguments[0] - - assert value.optional is False - assert value.optional_mode is OptionalMode.REQUIRED_DESCRIPTOR - assert value.nullable is True - assert value.descriptor_boundary is True - assert value.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION + assert argument.optional_mode is optional_mode + assert argument.nullable is nullable + assert argument.descriptor_boundary is descriptor_boundary + if handoff_mode is not None: + assert argument.handoff_mode is handoff_mode def test_optional_value_callback_dummy_is_blocked_before_codegen(): diff --git a/tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py b/tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py index c7799bb7e..70db87536 100644 --- a/tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py +++ b/tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py @@ -1,200 +1,64 @@ -"""Tests split by stable ownership concept from `test_compile_time_values.py`.""" +"""Optional presence keeps an output dummy visible so the caller can omit it.""" +import pytest + +from prik.parsers.fortran import parse_fortran_file as parse_fortran_source from prik.semantics.fortran2ir import fortran_module_to_semantic_module from prik.semantics.metadata import SCALAR_STORAGE_CATEGORY from prik.semantics.models import ProjectionMapping -from tests.fortran._support.semantic_conversion import ( - array_contract, - get_function, +from tests.fortran._support.semantic_conversion import array_contract, get_function + + +@pytest.mark.parametrize( + ("declaration", "python_position", "result_position"), + [ + ("integer(4), optional :: value", 0, 0), + ("integer(4), intent(out), optional :: value", 0, 0), + ("real(8), allocatable, intent(out), optional :: value(:)", 0, 0), + ("real(8), pointer, intent(out) :: value(:)", None, 0), + ("real(8), pointer, optional, intent(out) :: value(:)", 0, 0), + ("real(8), pointer, intent(inout) :: value(:)", 0, None), + ("type(point), intent(out), optional :: value", 0, None), + ], + ids=[ + "optional-without-intent-conservative-replacement", + "optional-scalar-output-visible-storage", + "optional-allocatable-output-visible", + "required-pointer-output-hidden", + "optional-pointer-output-visible", + "pointer-inout-visible-without-result", + "optional-derived-output-visible-without-result", + ], ) -from prik.semantics.metadata import PROJECTED_OUTPUT_METADATA -from prik.parsers.fortran import parse_fortran_file as parse_fortran_source - - -def test_optional_argument(): - source = """ -module opt_mod - -contains - -subroutine solve(A, tol) - - real(8), intent(in) :: A(:, :) - real(8), intent(in), optional :: tol - -end subroutine - -end module -""" - - fmod = parse_fortran_source(source) - - smod = fortran_module_to_semantic_module(fmod) - - func = get_function(smod, "solve") - - tol = func.arguments[1] - - assert tol.optional is True - - -def test_optional_without_intent_uses_visible_conservative_replacement_projection(): - source = """ -module no_intent_optional_mod -contains -subroutine adjust(value) - integer(4), optional :: value -end subroutine adjust -end module no_intent_optional_mod -""" - - module = fortran_module_to_semantic_module(parse_fortran_source(source)) - function = get_function(module, "adjust") - value = function.arguments[0] - - assert value.optional is True - assert value.metadata[PROJECTED_OUTPUT_METADATA] is True - assert function.projection == [ - ProjectionMapping( - python_name="value", - native_name="value", - native_position=0, - python_position=0, - result_position=0, - ) - ] - - -def test_optional_scalar_output_remains_visible_scalar_storage(): - source = """ -module opt_out_mod -contains -subroutine maybe_status(status) - integer(4), intent(out), optional :: status -end subroutine maybe_status -end module opt_out_mod -""" - - smod = fortran_module_to_semantic_module(parse_fortran_source(source)) - - func = get_function(smod, "maybe_status") - status = func.arguments[0] - - assert status.optional is True - assert array_contract(status.semantic_type).category == SCALAR_STORAGE_CATEGORY - assert func.projection == [ - ProjectionMapping( - python_name="status", - native_name="status", - native_position=0, - python_position=0, - result_position=0, - ) - ] - - -def test_optional_allocatable_output_remains_visible(): - source = """ -module opt_alloc_out_mod -contains -subroutine maybe_values(values) - real(8), allocatable, intent(out), optional :: values(:) -end subroutine maybe_values -end module opt_alloc_out_mod -""" - - smod = fortran_module_to_semantic_module(parse_fortran_source(source)) - - func = get_function(smod, "maybe_values") - values = func.arguments[0] - - assert values.optional is True - assert array_contract(values.semantic_type).allocatable is True - assert func.projection == [ - ProjectionMapping( - python_name="values", - native_name="values", - native_position=0, - python_position=0, - result_position=0, - ) - ] - - -def test_pointer_array_output_visibility_follows_intent_and_optional_presence(): - source = """ -module pointer_output_mod -contains -subroutine create_values(values) - real(8), pointer, intent(out) :: values(:) -end subroutine create_values - -subroutine maybe_create_values(values) - real(8), pointer, optional, intent(out) :: values(:) -end subroutine maybe_create_values - -subroutine replace_values(values) - real(8), pointer, intent(inout) :: values(:) -end subroutine replace_values -end module pointer_output_mod -""" - - smod = fortran_module_to_semantic_module(parse_fortran_source(source)) - create = get_function(smod, "create_values") - maybe_create = get_function(smod, "maybe_create_values") - replace = get_function(smod, "replace_values") - - assert create.arguments[0].metadata[PROJECTED_OUTPUT_METADATA] is True - assert create.projection == [ - ProjectionMapping( - python_name="values", - native_name="values", - native_position=0, - python_position=None, - result_position=0, - ) - ] - assert maybe_create.projection == [ - ProjectionMapping( - python_name="values", - native_name="values", - native_position=0, - python_position=0, - result_position=0, - ) - ] - assert replace.projection == [ - ProjectionMapping( - python_name="values", - native_name="values", - native_position=0, - python_position=0, - ) - ] - - -def test_optional_scalar_derived_output_stays_visible_without_result_projection(): - source = """ -module outputs +def test_optional_output_visibility_follows_intent_and_presence(declaration, python_position, result_position): + """A required output can be produced for the caller, so it leaves the signature. + + An optional one must stay a visible argument, because only the caller can + say whether it is present; its result slot follows the object kind. + """ + source = f""" +module optional_outputs type :: point real(8) :: x end type point contains -subroutine fill(value) - type(point), intent(out), optional :: value -end subroutine fill -end module outputs +subroutine update(value) + {declaration} +end subroutine update +end module optional_outputs """ - smod = fortran_module_to_semantic_module(parse_fortran_source(source)) - fill = get_function(smod, "fill") + update = get_function(fortran_module_to_semantic_module(parse_fortran_source(source)), "update") - assert PROJECTED_OUTPUT_METADATA not in fill.arguments[0].metadata - assert fill.projection == [ + assert update.projection == [ ProjectionMapping( python_name="value", native_name="value", native_position=0, - python_position=0, + python_position=python_position, + result_position=result_position, ) ] + if declaration == "integer(4), intent(out), optional :: value": + # The caller lends rank-zero storage the procedure may fill. + assert array_contract(update.arguments[0].semantic_type).category == SCALAR_STORAGE_CATEGORY diff --git a/tests/fortran/pointers/codegen/test_pointer_lowering.py b/tests/fortran/pointers/codegen/test_pointer_lowering.py index be5368ae1..adcc31ef9 100644 --- a/tests/fortran/pointers/codegen/test_pointer_lowering.py +++ b/tests/fortran/pointers/codegen/test_pointer_lowering.py @@ -1,142 +1,13 @@ -"""Pointer descriptor lowering from completed wrapper policy.""" +"""Pointer lowering details that the gfortran end-to-end suite cannot distinguish.""" from tests.fortran._support.ownership_policy import parse_pyi_text from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - NativeArrayDescriptorKind, - NativeArrayDescriptorOwnership, - NativeArrayOperation, - NativeArrayResultAllocation, - NativeDescriptorHandoffABI, -) from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -def _pointer_plan(): - module = parse_pyi_text( - """ -from prik.contracts import Annotated, Arg, Float64, Int32, Pointer, PointerAssociation, PointerPolicy, Return, native_call - -module_pointer: Annotated[ - Pointer[Float64[:]], - PointerAssociation("runtime"), - PointerPolicy( - nullable=True, - transfer="call_local", - target_owner="module", - lifetime="module", - deallocation="never", - shape_source="pointer_bounds", - contiguity="strided", - reassociation="never", - aliasing="borrowed", - mutability="view", - ), -] - -def make_pointer(n: Int32) -> Annotated[ - Pointer[Float64[:]], - PointerPolicy( - nullable=True, - transfer="call_local", - target_owner="module", - lifetime="module", - deallocation="never", - shape_source="pointer_bounds", - contiguity="strided", - reassociation="never", - aliasing="borrowed", - mutability="view", - ), -]: ... - -@native_call([Arg(0), Return("selected", 0)]) -def select_pointer(n: Int32) -> Annotated[ - Pointer[Float64[:]], - PointerPolicy( - nullable=True, - transfer="call_local", - target_owner="module", - lifetime="module", - deallocation="never", - shape_source="pointer_bounds", - contiguity="strided", - reassociation="never", - aliasing="borrowed", - mutability="view", - ), -]: ... -""", - module_name="pointer_lowering", - ) - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def test_pointer_plans_complete_descriptor_ownership_and_operations_before_lowering(): - plan = _pointer_plan() - namespace = plan.namespaces[0] - module_pointer = plan.variables[0].native_array_handle - functions = {function.binding.python_name: function for function in namespace.functions} - pointer_result = functions["make_pointer"].results[0].native_array_handle - pointer_output = functions["select_pointer"].results[0] - - assert module_pointer is not None - assert module_pointer.descriptor_kind is NativeArrayDescriptorKind.POINTER - assert module_pointer.borrowed is True - assert module_pointer.required_headers == ("ISO_Fortran_binding.h",) - assert NativeArrayOperation.ASSOCIATE in module_pointer.operations - assert NativeArrayOperation.NULLIFY in module_pointer.operations - assert NativeArrayOperation.DESTROY not in module_pointer.operations - - assert pointer_result is not None - assert pointer_result.handoff.abi is NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE - assert pointer_result.descriptor_ownership is NativeArrayDescriptorOwnership.OWNED - assert pointer_result.result_allocation is NativeArrayResultAllocation.NOT_APPLICABLE - assert pointer_result.target_lifetime == "module" - assert NativeArrayOperation.DESTROY in pointer_result.operations - - assert pointer_output.source_kind == "hidden_output" - assert pointer_output.native_array_handle is not None - assert pointer_output.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE - - -def test_pointer_lowering_assigns_descriptors_and_emits_manual_target_release(): - artifacts = WrapperGenerator().generate(_pointer_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "void bind_c_module_pointer_associate(CFI_cdesc_t * source);" in c_source - assert "bind_c_module_pointer_associate(source_descriptor);" in c_source - assert "native_module_pointer => source" in bridge_source - - result_start = bridge_source.index("subroutine bind_c_make_pointer(") - result_end = bridge_source.index("end subroutine", result_start) - result_procedure = bridge_source[result_start:result_end] - assert "real(c_double), pointer, dimension(:), intent(out) :: result" in result_procedure - assert "result_value => native_make_pointer(n)" in result_procedure - assert "result => result_value" in result_procedure - assert "move_alloc" not in result_procedure - - output_start = bridge_source.index("subroutine bind_c_select_pointer(") - output_end = bridge_source.index("end subroutine", output_start) - output_procedure = bridge_source[output_start:output_end] - assert "real(c_double), pointer, dimension(:), intent(out) :: selected" in output_procedure - assert "call native_select_pointer(n, selected_value)" in output_procedure - assert "selected => selected_value" in output_procedure - - operations_start = bridge_source.index("end subroutine bind_c_make_pointer") - operations_end = bridge_source.index("subroutine bind_c_select_pointer(", operations_start) - pointer_operations = bridge_source[operations_start:operations_end] - assert "result => source" in pointer_operations - assert "nullify(result)" in pointer_operations - # Release is manual and caller-driven, matching the ``deallocate`` a Fortran - # caller would write for the same pointer; prik never runs it on its own. - assert "deallocate(result)" in pointer_operations - - def test_nullable_scalar_pointer_result_uses_attribute_independent_storage_sizing(): + """Sizing uses ``storage_size``, which does not depend on the pointer attribute.""" module = parse_pyi_text( """ from prik.contracts import Addr, Aliased, Annotated, Arg, Destruction, Float64, Ownership, Pointer, Return, Transfer, native_call diff --git a/tests/fortran/pointers/end_to_end/test_pointer_handles.py b/tests/fortran/pointers/end_to_end/test_pointer_handles.py index e1267a773..e9a9d8c21 100644 --- a/tests/fortran/pointers/end_to_end/test_pointer_handles.py +++ b/tests/fortran/pointers/end_to_end/test_pointer_handles.py @@ -213,6 +213,10 @@ def test_module_and_derived_pointer_handles_track_native_association( assert field_handle.shape == (3,) assert module.sum_values(field_handle) == np.float64(24.0) + # Association hands over the source's descriptor facts, strides included. + module.associate_module_slice() + field_handle.associate(module_handle) + assert module.sum_pointer_descriptor(field_handle) == np.float64(6.0) module.associate_module_contiguous() field_handle.associate(module_handle) assert module.sum_pointer_descriptor(field_handle) == np.float64(9.0) @@ -278,7 +282,7 @@ def test_caller_created_pointer_crosses_separately_built_extensions(tmp_path: Pa assert values.closed is True -def test_a_reversed_pointer_target_keeps_its_data_pointer_strides_and_span(tmp_path: Path): +def test_pointer_descriptor_views_preserve_strides_span_and_parent_lifetime(tmp_path: Path): """A negative stride reaches the view exactly as the descriptor records it. The descriptor's base address is the first element in Fortran order and its @@ -304,10 +308,7 @@ def test_a_reversed_pointer_target_keeps_its_data_pointer_strides_and_span(tmp_p module.associate_module_reversed() np.testing.assert_allclose(handle.to_numpy(), np.array([50.0, 4.0, 3.0, 2.0])) - -def test_pointer_descriptor_views_preserve_slice_shape_strides_and_parent_lifetime(tmp_path: Path): - module = _pointer_descriptor_view_module(tmp_path) - + # A strided slice keeps its strides, and a field view outlives its parent. module_handle = module.module_values module.associate_module_slice() module_view = module_handle.to_numpy() @@ -352,8 +353,8 @@ def test_pointer_descriptor_views_preserve_slice_shape_strides_and_parent_lifeti np.testing.assert_allclose(field_view, np.array([6.0, 12.0], dtype=np.float64)) -def test_module_native_array_handles_use_canonical_plan(tmp_path: Path): - """Replay module pointer/allocatable handles without derived-field owners.""" +def test_module_native_array_handles_and_caller_created_pointers_use_canonical_plan(tmp_path: Path): + """Replay module pointer/allocatable handles and a native pointer output from an edited contract.""" source = tmp_path / "native" / "fpointer_handles_f90.f90" source.parent.mkdir() source.write_text(POINTER_HANDLE_SOURCE, encoding="utf-8") @@ -361,7 +362,7 @@ def test_module_native_array_handles_use_canonical_plan(tmp_path: Path): contract = tmp_path / "pointer_handles" / "fpointer_handles_f90.pyi" contract.parent.mkdir() contract.write_text( - """from prik.contracts import Aliased, Allocatable, Annotated, Float64, Pointer, PointerAssociation, PointerPolicy, bind + """from prik.contracts import Aliased, Allocatable, Annotated, Float64, Pointer, PointerAssociation, PointerPolicy, Returns, bind module_values: Annotated[ Pointer[Float64[:]], @@ -389,6 +390,25 @@ def sum_values(values: Float64[:]) -> Float64: ... def sum_four(values: Float64[4]) -> Float64: ... def sum_pointer_descriptor(values: Pointer[Float64[:]]) -> Float64: ... def sum_allocatable_descriptor(values: Allocatable[Float64[:]]) -> Float64: ... + +def select_module_values( + values: Annotated[ + Pointer[Float64[:]], + PointerAssociation("runtime"), + PointerPolicy( + nullable=True, + transfer="call_local", + target_owner="module", + lifetime="module", + deallocation="never", + shape_source="pointer_bounds", + contiguity="contiguous", + reassociation="native", + aliasing="borrowed", + mutability="view", + ), + ], +) -> Returns["values", Pointer[Float64[:]]]: ... """, encoding="utf-8", ) @@ -431,52 +451,11 @@ def sum_allocatable_descriptor(values: Allocatable[Float64[:]]) -> Float64: ... allocatable_handle.deallocate() assert allocatable_handle.allocated is False - -def test_caller_created_pointer_handle_tracks_native_output_association(tmp_path: Path): - source = tmp_path / "native" / "fpointer_handles_f90.f90" - source.parent.mkdir() - source.write_text(POINTER_HANDLE_SOURCE, encoding="utf-8") - native_object = _compile_native_object(source, tmp_path / "native_build") - contract = tmp_path / "contracts" / "fpointer_handles_f90.pyi" - contract.parent.mkdir() - pointer_type = """Annotated[ - Pointer[Float64[:]], - PointerAssociation("runtime"), - PointerPolicy( - nullable=True, - transfer="call_local", - target_owner="module", - lifetime="module", - deallocation="never", - shape_source="pointer_bounds", - contiguity="contiguous", - reassociation="native", - aliasing="borrowed", - mutability="view", - ), -]""" - contract.write_text( - f"""from prik.contracts import Annotated, Float64, Pointer, PointerAssociation, PointerPolicy, Returns - -def select_module_values( - values: {pointer_type}, -) -> Returns["values", {pointer_type}]: ... - -def sum_pointer_descriptor(values: Pointer[Float64[:]]) -> Float64: ... -""", - encoding="utf-8", - ) - result = build_pyi_extension( - contract, - input_compiler=_compiler(), - native_objects=[native_object], - native_include_dirs=[native_object.parent], - output_dir=tmp_path / "build", - ) - module = _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) - + # A caller-created handle tracks the association a native output makes, + # and a pending association reaches native storage once it is attached. handle = Pointer[Float64[:]]() assert handle.associated is False + assert handle.owned is True assert module.sum_pointer_descriptor(handle) == np.float64(-1.0) assert module.select_module_values(handle) is handle assert handle.associated is True diff --git a/tests/fortran/pointers/policy/fixtures/native/native_array_handle_policies_complete_before_ir_lowering.f90 b/tests/fortran/pointers/policy/fixtures/native/native_array_handle_policies_complete_before_ir_lowering.f90 index e0fecb057..022bef419 100644 --- a/tests/fortran/pointers/policy/fixtures/native/native_array_handle_policies_complete_before_ir_lowering.f90 +++ b/tests/fortran/pointers/policy/fixtures/native/native_array_handle_policies_complete_before_ir_lowering.f90 @@ -1,5 +1,6 @@ values: Allocatable[Float64[:]] target_values: Annotated[Allocatable[Float64[:]], Aliased] +module_target: Pointer[Float64[:]] class box: values: Allocatable[Float64[:]] diff --git a/tests/fortran/pointers/policy/test_pointer_ownership_policy.py b/tests/fortran/pointers/policy/test_pointer_ownership_policy.py index aa39d2181..5023b9e8c 100644 --- a/tests/fortran/pointers/policy/test_pointer_ownership_policy.py +++ b/tests/fortran/pointers/policy/test_pointer_ownership_policy.py @@ -6,7 +6,6 @@ from prik.printers import PyiPrinter from prik.semantics.models import ( RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA, - RESOLVED_OWNERSHIP_POLICY_METADATA, ) from prik.policy.native_array_handles import ( NativeArrayBuildRequirement, @@ -16,7 +15,6 @@ from prik.policy.ownership import ( CodegenAction, DestructionPolicy, - NativeBarrierAction, OwnershipContext, OwnershipOwner, TransferMode, @@ -39,26 +37,6 @@ NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" -def test_native_array_handle_dispatcher_routes_completed_policy_to_named_method(): - class Subject: - name = "values" - - class Target: - def handle(self, subject, policy, marker): - return marker, subject.name, policy.descriptor_kind, policy.handle_kind - - dispatcher = NativeArrayHandlePolicyDispatcher( - {("allocatable", "borrowed_module_descriptor"): "handle"}, - ) - - assert dispatcher.dispatch(Target(), Subject(), _native_array_policy(), "seen") == ( - "seen", - "values", - "allocatable", - "borrowed_module_descriptor", - ) - - def test_native_array_handle_dispatcher_rejects_missing_completed_policy_pair(): dispatcher = NativeArrayHandlePolicyDispatcher({}) @@ -69,75 +47,6 @@ def test_native_array_handle_dispatcher_rejects_missing_completed_policy_pair(): ) -def test_native_array_handle_build_requirements_include_default_pointer_descriptor_accessors(): - module = parse_pyi_text( - """ -target: Pointer[Float64[:]] - -class box: - target: Pointer[Float64[:]] -""", - module_name="native_handle_no_interop", - ) - - complete_semantic_policies(module) - - requirements = native_array_handle_build_requirements(module) - - assert requirements.pointer_c_descriptor_interop is True - assert requirements.requires_iso_fortran_binding is True - assert requirements.headers == ("ISO_Fortran_binding.h",) - assert requirements.items == ( - NativeArrayBuildRequirement( - owner="native_handle_no_interop.target", - item="target", - descriptor_kind="pointer", - handle_kind="borrowed_module_descriptor", - descriptor_interop="pointer_c_descriptor", - headers=("ISO_Fortran_binding.h",), - ), - NativeArrayBuildRequirement( - owner="native_handle_no_interop.box.target", - item="target", - descriptor_kind="pointer", - handle_kind="borrowed_field_descriptor", - descriptor_interop="pointer_c_descriptor", - headers=("ISO_Fortran_binding.h",), - ), - ) - - -def test_hidden_pointer_handle_output_owns_descriptor_but_not_target_policy(): - module = parse_pyi_text( - """ -@native_call([Return("values", 0)]) -def select_values() -> Pointer[Float64[:]]: ... -""", - module_name="hidden_pointer_handle_result", - ) - complete_semantic_policies(module) - - argument = module.functions[0].arguments[0] - decision = argument.metadata[RESOLVED_OWNERSHIP_POLICY_METADATA] - policy = argument.metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - - assert decision.owner is OwnershipOwner.WRAPPER - assert decision.transfer is TransferMode.WRAPPER_INSTANCE - assert decision.destruction is DestructionPolicy.WRAPPER_DEALLOC - assert decision.codegen_action is CodegenAction.WRAPPER_INSTANCE - assert decision.native_barrier_action is NativeBarrierAction.PASS_NATIVE_DESCRIPTOR - assert policy.descriptor_kind == "pointer" - assert policy.handle_kind == "owned_result_descriptor" - assert policy.origin == "projected_result" - assert policy.owner_retention == "wrapper_owner_storage" - assert policy.descriptor_ownership == "owned" - assert policy.output_projection == "projected_handle" - assert policy.target_lifetime == "unknown" - assert policy.destroy_behavior == "handle_finalizer" - assert policy.to_numpy == "unsupported" - assert set(policy.operations) == {"associate", "associated", "deallocate", "nullify", "to_numpy"} - - @pytest.mark.parametrize( ("owner", "transfer", "destruction", "context"), [ @@ -176,19 +85,21 @@ def test_explicit_supported_ownership_triples_remain_codegen_ready( @pytest.mark.parametrize( - ("owner", "transfer", "destruction"), + ("owner", "transfer", "destruction", "blocker"), [ - ("native", "copy_return", "native_owner"), - ("native", "borrowed_view", "python_refcount"), - ("python", "copy_return", "native_owner"), - ("python", "borrowed_view", "python_refcount"), - ("wrapper", "wrapper_instance", "python_refcount"), + ("native", "copy_return", "native_owner", "native/copy_return/native_owner"), + ("native", "borrowed_view", "python_refcount", "native/borrowed_view/python_refcount"), + ("python", "copy_return", "native_owner", "python/copy_return/native_owner"), + ("python", "borrowed_view", "python_refcount", "python/borrowed_view/python_refcount"), + ("wrapper", "wrapper_instance", "python_refcount", "wrapper/wrapper_instance/python_refcount"), + ("native", "blocked", "native_owner", "blocked by ownership policy"), ], ) -def test_contradictory_ownership_triples_fail_closed( +def test_contradictory_or_explicitly_blocked_ownership_triples_fail_closed( owner: str, transfer: str, destruction: str, + blocker: str, ): metadata: dict[str, object] = {} set_ownership_metadata( @@ -204,30 +115,11 @@ def test_contradictory_ownership_triples_fail_closed( ) assert decision.is_blocked - assert decision.owner is OwnershipOwner.UNKNOWN - assert decision.transfer is TransferMode.BLOCKED - assert decision.destruction is DestructionPolicy.BLOCKED - assert f"{owner}/{transfer}/{destruction}" in decision.blocker - - -def test_explicit_blocked_policy_normalizes_all_lifetime_axes(): - metadata: dict[str, object] = {} - set_ownership_metadata( - metadata, - owner="native", - transfer="blocked", - destruction="native_owner", - ) - - decision = default_ownership_policy.decide_semantic_type( - _array_type(metadata=metadata), - OwnershipContext.result(), - ) - assert decision.owner is OwnershipOwner.UNKNOWN assert decision.transfer is TransferMode.BLOCKED assert decision.destruction is DestructionPolicy.BLOCKED assert decision.codegen_action is CodegenAction.BLOCKED + assert blocker in decision.blocker def test_documented_transfer_and_destruction_modes_resolve_or_fail_closed(): @@ -280,21 +172,7 @@ def test_documented_transfer_and_destruction_modes_resolve_or_fail_closed(): assert decision.codegen_action is not CodegenAction.BLOCKED, label -def test_pyi_policy_metadata_round_trips_pointer_array_handle_policy(): - default_type = _array_type(pointer=True) - default_field = default_ownership_policy.decide_semantic_type(default_type, OwnershipContext.field()) - assert not default_field.is_blocked - assert default_field.owner is OwnershipOwner.WRAPPER - assert default_field.transfer is TransferMode.BORROWED_VIEW - assert default_field.destruction is DestructionPolicy.WRAPPER_DEALLOC - assert default_field.borrowed is True - - default_module = default_ownership_policy.decide_semantic_type(default_type, OwnershipContext.module_variable()) - assert not default_module.is_blocked - assert default_module.owner is OwnershipOwner.NATIVE - assert default_module.transfer is TransferMode.BORROWED_VIEW - assert default_module.destruction is DestructionPolicy.NATIVE_OWNER - +def test_pointer_container_ownership_is_fixed_by_its_native_parent(): metadata: dict[str, object] = {} set_ownership_metadata( metadata, @@ -351,48 +229,6 @@ class box: assert 'Destruction("python_refcount")' in emitted -def test_plain_pointer_array_container_policy_completes_default_handle_profile(): - module = parse_pyi_text( - """ -value: Pointer[Float64[:]] - -class box: - target: Pointer[Float64[:]] -""", - module_name="pointer_default_profile", - ) - - complete_semantic_policies(module) - - module_policy = module.variables[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - field_policy = module.classes[0].fields[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - - assert not module_policy.is_blocked - assert module_policy.handle_kind == "borrowed_module_descriptor" - assert module_policy.getter_behavior == "handle" - assert module_policy.to_numpy == "unsupported" - assert module_policy.descriptor_interop == "pointer_c_descriptor" - assert module_policy.requires_pointer_c_descriptor_interop is True - assert module_policy.target_lifetime == "module" - assert module_policy.destroy_behavior == "none" - assert set(module_policy.operations) == {"associate", "associated", "deallocate", "nullify", "to_numpy"} - # Release is available manually, as it is for an allocatable module array. - # Allocation and resize still need PointerPolicy, because they establish a - # new target rather than releasing the one the module already names. - assert "allocate" not in module_policy.operations - assert "resize" not in module_policy.operations - - assert not field_policy.is_blocked - assert field_policy.handle_kind == "borrowed_field_descriptor" - assert field_policy.getter_behavior == "handle" - assert field_policy.to_numpy == "unsupported" - assert field_policy.descriptor_interop == "pointer_c_descriptor" - assert field_policy.requires_pointer_c_descriptor_interop is True - assert field_policy.target_lifetime == "parent_wrapper" - assert field_policy.destroy_behavior == "parent_wrapper_finalizer" - assert set(field_policy.operations) == {"associate", "associated", "deallocate", "nullify", "to_numpy"} - - def test_deferred_character_pointer_arguments_select_an_opaque_fortran_owner(): module = parse_pyi_text( """ @@ -422,40 +258,6 @@ def inspect(values: Pointer[String[:][:]]) -> None: ... assert set(argument_policy.operations) == {"associate", "associated", "nullify"} -def test_contiguous_deferred_character_pointer_selects_zero_copy_view_policy(): - module = parse_pyi_text( - """ -from prik.contracts import Annotated, Pointer, PointerAssociation, PointerPolicy, String - -def inspect(values: Annotated[ - Pointer[String[:][:]], - PointerAssociation("runtime"), - PointerPolicy( - nullable=True, - transfer="call_local", - target_owner="wrapper", - lifetime="wrapper", - deallocation="deallocate_resize", - shape_source="pointer_bounds", - contiguity="contiguous", - reassociation="allocate_resize", - aliasing="descriptor", - mutability="mutable", - ), -]) -> None: ... -""", - module_name="deferred_character_pointer_view_policy", - ) - - complete_semantic_policies(module) - policy = module.functions[0].arguments[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - - assert policy.descriptor_inquiries is False - assert policy.owner_storage == "fortran_owner" - assert policy.to_numpy == "contiguous_view" - assert "to_numpy" in policy.operations - - def test_complete_pointer_policy_metadata_round_trips_without_overriding_container_ownership(): module = parse_pyi_text( """ @@ -506,39 +308,6 @@ def test_complete_pointer_policy_metadata_round_trips_without_overriding_contain assert decision.destruction is DestructionPolicy.NATIVE_OWNER -def test_copy_oriented_pointer_policy_still_exposes_live_contiguous_view(): - module = parse_pyi_text( - """ -value: Annotated[ - Pointer[Float64[:]], - PointerPolicy( - nullable=True, - transfer="snapshot_copy", - target_owner="module", - lifetime="module", - deallocation="never", - shape_source="pointer_bounds", - contiguity="contiguous", - reassociation="snapshot_final", - aliasing="independent_copy", - mutability="copy", - ), -] -""", - module_name="pointer_policy", - ) - - complete_semantic_policies(module) - - handle_policy = module.variables[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - assert not handle_policy.is_blocked - assert handle_policy.getter_behavior == "handle" - assert handle_policy.to_numpy == "contiguous_view" - assert handle_policy.descriptor_interop == "pointer_c_descriptor" - assert handle_policy.requires_pointer_c_descriptor_interop is True - assert handle_policy.blocker is None - - def test_pointer_policy_metadata_requires_every_fact(): with pytest.raises(ValueError, match="missing: lifetime"): parse_pyi_text( @@ -601,7 +370,18 @@ def consume( assert 'deallocation="unsafe_deallocate"' in emitted +_BORROWED_ALLOCATABLE_OPERATIONS = ("allocated", "deallocate", "resize", "to_numpy") +_DEFAULT_POINTER_OPERATIONS = ("associate", "associated", "deallocate", "nullify", "to_numpy") + + def test_native_array_handle_policies_complete_before_ir_lowering(): + """Each handle origin completes one ownership, lifetime, and operation profile. + + Borrowed descriptors are never destroyed by their handle; owned result + descriptors are, and a pointer result owns its descriptor but not its + target. Release of a default pointer target is a manual operation, while + allocation needs an explicit `PointerPolicy`. + """ module = parse_pyi_text( (NATIVE_FIXTURES / "native_array_handle_policies_complete_before_ir_lowering.f90").read_text(encoding="utf-8"), module_name="native_handles", @@ -609,156 +389,137 @@ def test_native_array_handle_policies_complete_before_ir_lowering(): complete_semantic_policies(module) - values = module.variables[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - target_values = module.variables[1].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - field_values = module.classes[0].fields[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - field_target = module.classes[0].fields[1].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - argument_values = module.functions[0].arguments[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - managed_target = module.functions[0].arguments[1].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - optional_target = module.functions[0].arguments[2].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - allocatable_result = module.functions[1].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - pointer_result = module.functions[2].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - - assert values.descriptor_kind == "allocatable" - assert values.handle_kind == "borrowed_module_descriptor" - assert values.origin == "module_variable" - assert values.owner == "native" - assert values.owner_retention == "native_module" - assert values.descriptor_ownership == "borrowed" - assert values.target_lifetime == "module" - assert values.destroy_behavior == "none" - assert values.to_numpy == "descriptor_view" - assert values.descriptor_interop == "module_allocatable_c_descriptor" - assert values.requires_pointer_c_descriptor_interop is False - assert values.requires_c_descriptor_interop is True - assert values.storage_mode == "heap" - assert set(values.operations) == {"allocated", "deallocate", "resize", "to_numpy"} - - assert target_values.handle_kind == "borrowed_module_descriptor" - assert target_values.owner_retention == "native_module" - assert target_values.target_lifetime == "module" - # A module allocatable reports its own descriptor whether or not it is a - # target, so `Aliased` selects neither a different NumPy exposure nor a - # different interop mechanism. - assert target_values.to_numpy == "descriptor_view" - assert target_values.descriptor_interop == "module_allocatable_c_descriptor" - assert target_values.requires_pointer_c_descriptor_interop is False - - assert field_values.handle_kind == "borrowed_field_descriptor" - assert field_values.owner == "wrapper" - assert field_values.owner_retention == "parent_wrapper" - assert field_values.release == "wrapper_dealloc" - assert field_values.target_lifetime == "parent_wrapper" - assert field_values.destroy_behavior == "parent_wrapper_finalizer" - - assert field_target.descriptor_kind == "pointer" - assert field_target.handle_kind == "borrowed_field_descriptor" - assert field_target.target_lifetime == "parent_wrapper" - assert field_target.destroy_behavior == "parent_wrapper_finalizer" - assert field_target.getter_behavior == "handle" - assert field_target.to_numpy == "unsupported" - assert field_target.descriptor_interop == "pointer_c_descriptor" - assert field_target.requires_pointer_c_descriptor_interop is True - assert field_target.is_blocked is False - assert set(field_target.operations) == {"associate", "associated", "deallocate", "nullify", "to_numpy"} - - assert argument_values.handle_kind == "argument_descriptor" - assert argument_values.origin == "argument" - assert argument_values.owner_retention == "caller_handle" - assert argument_values.target_lifetime == "call" - assert argument_values.destroy_behavior == "none" - assert argument_values.is_blocked is False - assert argument_values.blocker is None - assert argument_values.descriptor_interop == "none" - assert argument_values.requires_pointer_c_descriptor_interop is False - assert set(argument_values.operations) == {"allocated", "to_numpy"} - # A non-optional descriptor argument is handed a descriptor the Fortran - # runtime built, so a caller-created handle needs storage of its own. - assert argument_values.default_construction == "lazy_owned_descriptor" - assert argument_values.default_descriptor_ownership == "owned" - assert argument_values.default_release == "wrapper_dealloc" - assert argument_values.default_destroy_behavior == "handle_finalizer" - assert "destroy" in argument_values.default_operations - - assert optional_target.handle_kind == "optional_absent_handle" - assert optional_target.optional_absent is True - assert optional_target.nullable is True - assert optional_target.owner_retention == "optional_argument" - assert optional_target.target_lifetime == "absent_or_call" - assert optional_target.destroy_behavior == "none" - assert optional_target.is_blocked is False - assert optional_target.blocker is None - assert optional_target.descriptor_interop == "pointer_c_descriptor" - assert optional_target.requires_pointer_c_descriptor_interop is True - assert set(optional_target.operations) == {"associate", "associated", "nullify", "to_numpy"} - # An optional argument is a descriptor argument like any other when it is - # present, so a caller-created handle needs storage of its own to hand over. - assert optional_target.default_construction == "lazy_owned_descriptor" - assert "destroy" in optional_target.default_operations - assert "allocate" not in optional_target.operations - assert "deallocate" not in optional_target.operations - assert "resize" not in optional_target.operations - - assert managed_target.handle_kind == "argument_descriptor" - assert managed_target.descriptor_kind == "pointer" - assert managed_target.target_lifetime == "call" - assert managed_target.destroy_behavior == "none" - assert managed_target.is_blocked is False - assert managed_target.blocker is None - assert managed_target.to_numpy == "contiguous_view" - assert managed_target.descriptor_interop == "pointer_c_descriptor" - assert managed_target.requires_pointer_c_descriptor_interop is True - assert set(managed_target.operations) == { - "allocate", - "associate", - "associated", - "deallocate", - "nullify", - "resize", - "to_numpy", + entities = {variable.name: variable for variable in module.variables} + entities.update({f"box.{field.name}": field for field in module.classes[0].fields}) + entities.update({f"consume.{argument.name}": argument for argument in module.functions[0].arguments}) + entities.update({function.name: function for function in module.functions[1:]}) + completed = { + name: ( + policy.descriptor_kind, + policy.handle_kind, + policy.descriptor_ownership, + policy.target_lifetime, + policy.destroy_behavior, + policy.to_numpy, + tuple(sorted(policy.operations)), + policy.default_construction, + ) + for name, entity in entities.items() + for policy in (entity.metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA],) + if not policy.is_blocked } - assert allocatable_result.handle_kind == "owned_result_descriptor" - assert allocatable_result.origin == "result" - assert allocatable_result.owner == "wrapper" - assert allocatable_result.owner_retention == "wrapper_owner_storage" - assert allocatable_result.descriptor_ownership == "owned" - assert allocatable_result.output_projection == "handle_result" - assert allocatable_result.release == "wrapper_dealloc" - assert allocatable_result.target_lifetime == "wrapper_owner_storage" - assert allocatable_result.destroy_behavior == "handle_finalizer" - assert allocatable_result.is_blocked is False - assert allocatable_result.descriptor_interop == "owned_allocatable_c_descriptor" - assert allocatable_result.requires_pointer_c_descriptor_interop is False - assert allocatable_result.requires_c_descriptor_interop is True - assert set(allocatable_result.operations) == {"allocated", "deallocate", "resize", "to_numpy"} - assert allocatable_result.default_construction == "none" - assert allocatable_result.default_operations == () - - assert pointer_result.handle_kind == "owned_result_descriptor" - assert pointer_result.origin == "result" - assert pointer_result.owner == "wrapper" - assert pointer_result.owner_retention == "wrapper_owner_storage" - assert pointer_result.descriptor_ownership == "owned" - assert pointer_result.output_projection == "handle_result" - assert pointer_result.release == "wrapper_dealloc" - assert pointer_result.target_lifetime == "unknown" - assert pointer_result.destroy_behavior == "handle_finalizer" - assert pointer_result.is_blocked is False - assert pointer_result.blocker is None - assert pointer_result.descriptor_interop == "pointer_c_descriptor" - assert pointer_result.requires_pointer_c_descriptor_interop is True - assert pointer_result.requires_c_descriptor_interop is True - assert set(pointer_result.operations) == {"associate", "associated", "deallocate", "nullify", "to_numpy"} - assert pointer_result.default_construction == "none" + module_allocatable = ( + "allocatable", + "borrowed_module_descriptor", + "borrowed", + "module", + "none", + "descriptor_view", + _BORROWED_ALLOCATABLE_OPERATIONS, + "none", + ) + assert completed == { + "values": module_allocatable, + # `Aliased` selects neither another NumPy exposure nor another interop. + "target_values": module_allocatable, + "module_target": ( + "pointer", + "borrowed_module_descriptor", + "borrowed", + "module", + "none", + "unsupported", + _DEFAULT_POINTER_OPERATIONS, + "none", + ), + "box.values": ( + "allocatable", + "borrowed_field_descriptor", + "borrowed", + "parent_wrapper", + "parent_wrapper_finalizer", + "borrowed_view", + _BORROWED_ALLOCATABLE_OPERATIONS, + "none", + ), + "box.target": ( + "pointer", + "borrowed_field_descriptor", + "borrowed", + "parent_wrapper", + "parent_wrapper_finalizer", + "unsupported", + _DEFAULT_POINTER_OPERATIONS, + "none", + ), + # A descriptor argument is handed a descriptor the Fortran runtime + # built, so a caller-created handle lazily gets storage of its own. + "consume.values": ( + "allocatable", + "argument_descriptor", + "borrowed", + "call", + "none", + "borrowed_view", + ("allocated", "to_numpy"), + "lazy_owned_descriptor", + ), + "consume.managed_target": ( + "pointer", + "argument_descriptor", + "borrowed", + "call", + "none", + "contiguous_view", + ("allocate", "associate", "associated", "deallocate", "nullify", "resize", "to_numpy"), + "lazy_owned_descriptor", + ), + "consume.maybe_target": ( + "pointer", + "optional_absent_handle", + "borrowed", + "absent_or_call", + "none", + "unsupported", + ("associate", "associated", "nullify", "to_numpy"), + "lazy_owned_descriptor", + ), + "make_values": ( + "allocatable", + "owned_result_descriptor", + "owned", + "wrapper_owner_storage", + "handle_finalizer", + "borrowed_view", + _BORROWED_ALLOCATABLE_OPERATIONS, + "none", + ), + "make_target": ( + "pointer", + "owned_result_descriptor", + "owned", + "unknown", + "handle_finalizer", + "unsupported", + _DEFAULT_POINTER_OPERATIONS, + "none", + ), + } def test_native_array_handle_build_requirements_are_selected_from_completed_policy(): + """Copy and contiguous pointers expose a contiguous view; a strided one exposes the descriptor.""" module = parse_pyi_text( """ values: Allocatable[Float64[:]] default_target: Pointer[Float64[:]] +class box: + target: Pointer[Float64[:]] + +def make_values() -> Allocatable[Float64[:]]: ... + def inspect( copy_target: Annotated[ Pointer[Float64[:]], @@ -812,9 +573,10 @@ def inspect( complete_semantic_policies(module) - copy_target = module.functions[0].arguments[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - contiguous_target = module.functions[0].arguments[1].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - descriptor_target = module.functions[0].arguments[2].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] + arguments = module.functions[1].arguments + copy_target, contiguous_target, descriptor_target = ( + argument.metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] for argument in arguments + ) assert copy_target.to_numpy == "contiguous_view" assert copy_target.requires_pointer_c_descriptor_interop is True @@ -845,6 +607,22 @@ def inspect( descriptor_interop="pointer_c_descriptor", headers=("ISO_Fortran_binding.h",), ), + NativeArrayBuildRequirement( + owner="native_handle_build.box.target", + item="target", + descriptor_kind="pointer", + handle_kind="borrowed_field_descriptor", + descriptor_interop="pointer_c_descriptor", + headers=("ISO_Fortran_binding.h",), + ), + NativeArrayBuildRequirement( + owner="native_handle_build.make_values.return", + item="return", + descriptor_kind="allocatable", + handle_kind="owned_result_descriptor", + descriptor_interop="owned_allocatable_c_descriptor", + headers=("ISO_Fortran_binding.h",), + ), NativeArrayBuildRequirement( owner="native_handle_build.inspect.copy_target", item="copy_target", diff --git a/tests/fortran/pointers/runtime/test_pointer_contract_handles.py b/tests/fortran/pointers/runtime/test_pointer_contract_handles.py index 3e97b047f..dc2daa8ac 100644 --- a/tests/fortran/pointers/runtime/test_pointer_contract_handles.py +++ b/tests/fortran/pointers/runtime/test_pointer_contract_handles.py @@ -1,168 +1,67 @@ -"""Pointer contract constructors and native-storage attachment.""" +"""Contract handle constructors and generated-storage attachment checks.""" import numpy as np import pytest import prik.contracts as contracts -from prik.runtime.handles import ( - AllocatableArray, - PointerArray, - _bind_contract_native_array_handle, - _numpy_view_from_descriptor_facts, -) -from tests.fortran._support.native_array_handles import ( - _absent_descriptor_facts, - _descriptor_facts_for_array, - _generated_handle_dispatch, - _handle_dispatch, -) - - -def test_contract_default_handle_constructors_preserve_dtype_rank_and_empty_state(): - allocatable = contracts.Allocatable[contracts.Float64[:]]() - pointer = contracts.Pointer[contracts.Int32[:, :]]() - - assert isinstance(allocatable, AllocatableArray) - assert allocatable.dtype == np.dtype(np.float64) - assert allocatable.rank == 1 - assert allocatable.owned is True - assert allocatable.allocated is False - assert allocatable.shape is None - assert allocatable.to_numpy() is None - - assert isinstance(pointer, PointerArray) - assert pointer.dtype == np.dtype(np.int32) - assert pointer.rank == 2 - assert pointer.owned is True - assert pointer.associated is False - assert pointer.shape is None - assert pointer.to_numpy() is None - - -def test_fresh_pointer_associate_copies_association_without_following_source_descriptor(): - value = np.arange(6, dtype=np.float64)[::2] - source_state = {"facts": _descriptor_facts_for_array(value)} - - def source_nullify(_handle): - source_state["facts"] = _absent_descriptor_facts("float64", 1) - - source = PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: value.shape if source_state["facts"][0] else None, - "descriptor": lambda _handle: source_state["facts"], - "to_numpy": lambda _handle: _numpy_view_from_descriptor_facts(source_state["facts"], "float64"), - "associated": lambda _handle: source_state["facts"][0] != 0, - "associate": lambda _handle, facts: source_state.update(facts=facts), - "nullify": source_nullify, - } - ), - to_numpy_policy="descriptor_view", - ) - target = contracts.Pointer[contracts.Float64[:]]() - - target.associate(source) - assert target.associated is True - assert target.shape == (3,) - np.testing.assert_array_equal(target.to_numpy(), value) - - source.nullify() - assert source.associated is False - assert target.associated is True - np.testing.assert_array_equal(target.to_numpy(), value) - - target.associate(source) - assert target.associated is False - assert target.to_numpy() is None - +from prik.runtime.handles import AllocatableArray, _bind_contract_native_array_handle +from tests.fortran._support.native_array_handles import _generated_handle_dispatch, _handle_dispatch -def test_fresh_pointer_pending_association_is_applied_when_native_storage_attaches(): - value = np.arange(4, dtype=np.float64) - facts = _descriptor_facts_for_array(value) - source = PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: value.shape, - "descriptor": lambda _handle: facts, - "to_numpy": lambda _handle: _numpy_view_from_descriptor_facts(facts, "float64"), - "associated": lambda _handle: True, - "associate": lambda _handle, _facts: None, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="descriptor_view", - ) - target = contracts.Pointer[contracts.Float64[:]]() - target.associate(source) - owner = object() - received = [] - state = {"associated": False} - - def associate(received_owner, facts): - received.append((received_owner, facts)) - state["associated"] = True - - operations = { - "shape": lambda _owner: value.shape if state["associated"] else None, - "descriptor": lambda _owner: facts, - "associated": lambda _owner: state["associated"], - "associate": associate, - "nullify": lambda _owner: state.update(associated=False), - "destroy": lambda _owner: None, - } - _bind_contract_native_array_handle( - target, - "pointer", - "float64", - 1, - _generated_handle_dispatch(operations), - operations, - owner, - "owned", - "unsupported", - ) - assert target.associated is True - assert received == [ - ( - owner, - ( - int(value.ctypes.data), - 8, - 1, - 1, - 4, - 8, - ), - ) - ] +def _closed_allocatable(): + handle = contracts.Allocatable[contracts.Float64[:]]() + handle.close() + return handle @pytest.mark.parametrize( ("prepare", "descriptor_kind", "dtype", "rank", "error", "message"), [ - ( + pytest.param( lambda: contracts.Allocatable[contracts.Float64[:]](), "pointer", "float64", 1, TypeError, "cannot attach pointer descriptor storage", + id="allocatable-into-pointer-storage", ), + pytest.param( + lambda: AllocatableArray( + dtype="float64", + rank=1, + **_handle_dispatch({"shape": lambda _handle: None, "allocated": lambda _handle: False}), + to_numpy_policy="unsupported", + ), + "allocatable", + "float64", + 1, + TypeError, + "fresh contract handle", + id="not-a-contract-handle", + ), + pytest.param( + lambda: contracts.Allocatable[contracts.Float64[:]](), + "allocatable", + "float64", + 2, + ValueError, + "does not match generated rank 2", + id="rank", + ), + pytest.param( + lambda: contracts.Allocatable[contracts.Float64[:]](), + "allocatable", + "int32", + 1, + TypeError, + "does not match generated dtype", + id="dtype", + ), + pytest.param(_closed_allocatable, "allocatable", "float64", 1, ReferenceError, "handle is closed", id="closed"), ], ) -def test_generated_storage_rejects_incompatible_contract_handles( - prepare, - descriptor_kind, - dtype, - rank, - error, - message, -): +def test_generated_storage_rejects_incompatible_contract_handles(prepare, descriptor_kind, dtype, rank, error, message): handle = prepare() with pytest.raises(error, match=message): @@ -180,20 +79,6 @@ def test_generated_storage_rejects_incompatible_contract_handles( ) -def test_pointer_association_rejects_closed_handles(): - target = contracts.Pointer[contracts.Float64[:]]() - source = contracts.Pointer[contracts.Float64[:]]() - target.close() - - with pytest.raises(ReferenceError, match="pointer handle is closed"): - target.associate(source) - - target = contracts.Pointer[contracts.Float64[:]]() - source.close() - with pytest.raises(ReferenceError, match="source pointer handle is closed"): - target.associate(source) - - def test_non_array_descriptor_and_ordinary_array_annotations_are_not_factories(): with pytest.raises(TypeError, match="scalar allocatable contracts"): contracts.Allocatable[contracts.Float64]() @@ -221,11 +106,3 @@ def test_character_array_contracts_create_fixed_and_deferred_handle_types(): for width in (True, 0, -1): with pytest.raises(TypeError, match="positive integer width or ':'"): contracts.Pointer[contracts.String[width][:]]() - - -def test_unattached_character_pointer_association_is_refused_instead_of_deferred(): - source = contracts.Pointer[contracts.String[4][:]]() - target = contracts.Pointer[contracts.String[4][:]]() - - with pytest.raises(TypeError, match="target handle to be attached"): - target.associate(source) diff --git a/tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py b/tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py index 4050e3a1a..99755e5b7 100644 --- a/tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py +++ b/tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py @@ -1,4 +1,8 @@ -"""Descriptor handoff and view construction through the runtime ABI.""" +"""Fail-closed checks at the runtime descriptor boundary. + +A handle of the wrong kind, dtype, rank, or state must be refused before its +backend reaches native code; the successful handoffs are proved end to end. +""" import numpy as np import pytest @@ -7,19 +11,13 @@ from prik.runtime.handles import ( AllocatableArray, PointerArray, - _bind_contract_native_array_handle, _native_array_backend_for_binding, - _native_array_backend_for_binding_positional, _numpy_view_from_descriptor_facts, ) -from tests.fortran._support.native_array_handles import ( - _descriptor_facts_for_array, - _generated_handle_dispatch, - _handle_dispatch, -) +from tests.fortran._support.native_array_handles import _handle_dispatch -def _bound_pointer(backend, *, dtype=np.float64, rank=1): +def _bound_pointer(*, dtype=np.float64, rank=1): """Return a pointer handle standing for storage a wrapper attached.""" handle = PointerArray( dtype=np.dtype(dtype), @@ -30,151 +28,104 @@ def _bound_pointer(backend, *, dtype=np.float64, rank=1): "associated": lambda _handle: False, "nullify": lambda _handle: None, "descriptor": lambda _handle: None, + "associate": lambda _handle, _facts: None, } ), to_numpy_policy="unsupported", ) - handle._native_backend = backend + handle._native_backend = object() return handle -def test_descriptor_argument_hands_over_the_backend_the_handle_publishes(): - """The backend is what crosses; nothing rebuilds a descriptor in Python.""" - backend = object() - handle = _bound_pointer(backend) - - assert _native_array_backend_for_binding( - handle, - descriptor_kind="pointer", - expected_dtype=np.float64, - expected_rank=1, - ) == (backend,) - assert _native_array_backend_for_binding( - None, - descriptor_kind="pointer", - optional_absent=True, - ) == (None, None) - - -def test_an_optional_descriptor_argument_reports_presence_alongside_its_backend(): - backend = object() - supplied, presence = _native_array_backend_for_binding( - _bound_pointer(backend), - descriptor_kind="pointer", - expected_dtype=np.float64, - expected_rank=1, - optional_absent=True, +def _unattached_allocatable(): + return AllocatableArray( + dtype=np.dtype(np.float64), + rank=1, + **_handle_dispatch({"shape": lambda _handle: None, "allocated": lambda _handle: False}), + to_numpy_policy="unsupported", ) - assert supplied is backend - assert isinstance(presence, int) - assert presence > 0 - @pytest.mark.parametrize( - ("value", "error", "message"), + ("value", "kind", "expected", "error", "message"), [ - (np.zeros(2, dtype=np.float64), TypeError, "expected pointer native array handle"), - (None, TypeError, "handle argument is required"), - ( - AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch({"shape": lambda _handle: None, "allocated": lambda _handle: False}), - to_numpy_policy="unsupported", - ), + pytest.param(np.zeros(2), "pointer", {}, TypeError, "expected pointer native array handle", id="plain-array"), + pytest.param(None, "pointer", {}, TypeError, "handle argument is required", id="none"), + pytest.param( + _unattached_allocatable(), + "pointer", + {}, TypeError, "expected pointer native array handle", + id="allocatable-for-pointer", + ), + pytest.param( + _bound_pointer(), "pointer", {"expected_rank": 2}, ValueError, "does not match expected rank 2", id="rank" + ), + pytest.param( + _bound_pointer(), + "pointer", + {"expected_dtype": np.int32}, + TypeError, + "does not match expected dtype", + id="dtype", + ), + pytest.param( + contracts.Pointer[contracts.Float64[:]](), + "pointer", + {}, + TypeError, + "requires generated persistent descriptor storage", + id="fresh-pointer-without-storage", + ), + pytest.param( + contracts.Allocatable[contracts.Float64[:]](), + "allocatable", + {"expected_dtype": np.float64, "expected_rank": 1}, + TypeError, + "requires generated persistent descriptor storage", + id="fresh-allocatable-without-storage", ), ], ) -def test_descriptor_argument_rejects_values_that_are_not_the_declared_handle(value, error, message: str): +def test_descriptor_argument_refuses_a_handle_that_is_not_the_declared_one(value, kind, expected, error, message): with pytest.raises(error, match=message): - _native_array_backend_for_binding(value, descriptor_kind="pointer") - - -def test_descriptor_argument_rejects_a_mismatched_dtype_or_rank(): - handle = _bound_pointer(object()) - - with pytest.raises(ValueError, match="does not match expected rank 2"): - _native_array_backend_for_binding(handle, descriptor_kind="pointer", expected_rank=2) - with pytest.raises(TypeError, match="does not match expected dtype"): - _native_array_backend_for_binding(handle, descriptor_kind="pointer", expected_dtype=np.int32) - - -def test_descriptor_argument_refuses_a_handle_that_has_no_storage_yet(): - """A fresh contract handle publishes no backend until a binder attaches one.""" - handle = contracts.Pointer[contracts.Float64[:]]() - - with pytest.raises(TypeError, match="requires generated persistent descriptor storage"): - _native_array_backend_for_binding(handle, descriptor_kind="pointer") + _native_array_backend_for_binding(value, descriptor_kind=kind, **expected) -def test_descriptor_argument_binds_a_fresh_contract_handle_then_reads_its_backend(): - handle = contracts.Pointer[contracts.Float64[:]]() - backend = object() - - def bind_default(value): - operations = { - "shape": lambda _owner: None, - "associated": lambda _owner: False, - "nullify": lambda _owner: None, - "descriptor": lambda _owner: None, - "associate": lambda _owner, _facts: None, - "destroy": lambda _owner: None, - } - _bind_contract_native_array_handle( - value, - "pointer", - "float64", - 1, - _generated_handle_dispatch(operations), - operations, - backend, - "owned", - "unsupported", - native_backend=backend, - ) - - assert _native_array_backend_for_binding_positional( - handle, - "pointer", - np.float64, - 1, - False, - bind_default, - ) == (backend,) - - -def test_view_from_facts_preserves_a_strided_target(): - source = np.arange(8, dtype=np.float64) - strided = source[::2] - - view = _numpy_view_from_descriptor_facts(_descriptor_facts_for_array(strided), np.float64) - - assert view.shape == (4,) - assert view.strides == (16,) - np.testing.assert_allclose(view, strided) - view[1] = np.float64(99.0) - assert source[2] == np.float64(99.0) +def test_view_from_facts_reports_absent_storage_and_a_disagreeing_element_width(): + assert _numpy_view_from_descriptor_facts((0, 8, 1, 0, 0, 8), np.float64) is None + with pytest.raises(ValueError, match="does not match NumPy dtype itemsize"): + _numpy_view_from_descriptor_facts((1024, 4, 1, 1, 2, 4), np.float64) -def test_view_from_facts_preserves_a_negative_stride_target(): - """A reversed target keeps its data pointer, strides and accessible span.""" - source = np.arange(6, dtype=np.float64) - reversed_view = source[::-1] - view = _numpy_view_from_descriptor_facts(_descriptor_facts_for_array(reversed_view), np.float64) +@pytest.mark.parametrize( + ("other", "error", "message"), + [ + pytest.param(object(), TypeError, "requires another PointerArray", id="not-a-pointer"), + pytest.param(_bound_pointer(dtype=np.int32), TypeError, "dtype", id="dtype"), + pytest.param(_bound_pointer(rank=2), ValueError, "rank", id="rank"), + ], +) +def test_pointer_associate_rejects_incompatible_sources(other, error, message): + with pytest.raises(error, match=message): + _bound_pointer().associate(other) - assert view.shape == (6,) - assert view.strides == (-8,) - np.testing.assert_allclose(view, source[::-1]) - view[0] = np.float64(42.0) - assert source[5] == np.float64(42.0) +def test_pointer_association_rejects_closed_and_unattached_character_handles(): + target = contracts.Pointer[contracts.Float64[:]]() + source = contracts.Pointer[contracts.Float64[:]]() + target.close() + with pytest.raises(ReferenceError, match="pointer handle is closed"): + target.associate(source) -def test_view_from_facts_reports_absent_storage_and_a_disagreeing_element_width(): - assert _numpy_view_from_descriptor_facts((0, 8, 1, 0, 0, 8), np.float64) is None + target = contracts.Pointer[contracts.Float64[:]]() + source.close() + with pytest.raises(ReferenceError, match="source pointer handle is closed"): + target.associate(source) - with pytest.raises(ValueError, match="does not match NumPy dtype itemsize"): - _numpy_view_from_descriptor_facts((1024, 4, 1, 1, 2, 4), np.float64) + # A character pointer cannot record a pending association, so it refuses + # instead of deferring one until storage is attached. + with pytest.raises(TypeError, match="target handle to be attached"): + contracts.Pointer[contracts.String[4][:]]().associate(contracts.Pointer[contracts.String[4][:]]()) diff --git a/tests/fortran/pointers/runtime/test_pointer_handle_protocol.py b/tests/fortran/pointers/runtime/test_pointer_handle_protocol.py deleted file mode 100644 index ef2560845..000000000 --- a/tests/fortran/pointers/runtime/test_pointer_handle_protocol.py +++ /dev/null @@ -1,553 +0,0 @@ -"""Pointer handle state, association, extraction, and operation protocols.""" - -import numpy as np -import pytest -from prik.runtime.handles import ( - AllocatableArray, - NativeArrayHandleBase, - PointerArray, - _native_array_handle_from_generated_dispatch, - _numpy_view_from_descriptor_facts, -) -from tests.fortran._support.native_array_handles import ( - _ArrayState, - _common_ops, - _descriptor_facts_for_array, - _generated_handle_dispatch, - _handle_dispatch, -) - - -def test_pointer_to_numpy_reports_unassociated_state_before_an_unsupported_policy(): - """There is nothing to expose either way when the target is not there. - - A policy that blocks extraction still has to answer an unassociated - pointer with None rather than refusing, because no view is being withheld. - """ - handle = PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="unsupported", - ) - - assert handle.to_numpy() is None - - -def test_shape_reports_absent_descriptor_state_without_being_asked_first(): - def fail_state(_handle): - pytest.fail("the shape inquiry reads the descriptor, which records absence itself") - - allocatable = AllocatableArray( - dtype="float64", - rank=1, - **_handle_dispatch({"shape": lambda _handle: None, "allocated": fail_state}), - to_numpy_policy="unsupported", - ) - pointer = PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: None, - "associated": fail_state, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="unsupported", - ) - - assert allocatable.shape is None - assert pointer.shape is None - - -def test_to_numpy_contiguous_view_policy_rejects_non_contiguous_storage(): - source = np.arange(8, dtype=np.float64) - strided = source[::2] - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: strided.shape, - "to_numpy": lambda _handle: strided, - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="contiguous_view", - ) - - with pytest.raises(ValueError, match="must be contiguous"): - handle.to_numpy() - - -def test_to_numpy_descriptor_view_policy_never_copies_storage(): - source = np.arange(4, dtype=np.float64) - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: source.shape, - "to_numpy": lambda _handle: source, - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="descriptor_view", - ) - - view = handle.to_numpy() - - assert np.shares_memory(view, source) is True - assert view.flags.writeable is True - view[0] = 99.0 - assert source[0] == 99.0 - - -@pytest.mark.parametrize( - "policy", - ["borrowed_view", "contiguous_view", "descriptor_view"], -) -def test_to_numpy_rejects_generated_non_numpy_results(policy: str): - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (2,), - "to_numpy": lambda _handle: [1.0, 2.0], - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - } - ), - to_numpy_policy=policy, - ) - - with pytest.raises(TypeError, match="must return a NumPy array or None"): - handle.to_numpy() - - -def test_to_numpy_rejects_generated_array_with_wrong_rank_or_dtype(): - wrong_rank = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (2,), - "to_numpy": lambda _handle: np.zeros((1, 2), dtype=np.float64), - "allocated": lambda _handle: True, - } - ), - ) - with pytest.raises(ValueError, match="to_numpy result rank 2 does not match declared rank 1"): - wrong_rank.to_numpy() - - wrong_dtype = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (2,), - "to_numpy": lambda _handle: np.zeros(2, dtype=np.int32), - "allocated": lambda _handle: True, - } - ), - ) - with pytest.raises(TypeError, match="to_numpy result dtype"): - wrong_dtype.to_numpy() - - -def test_runtime_handle_shapes_reject_negative_extents(): - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (-1,), - "allocated": lambda _handle: True, - "resize": lambda _handle, _shape: None, - } - ), - to_numpy_policy="unsupported", - ) - - with pytest.raises(ValueError, match="non-negative"): - _ = handle.shape - with pytest.raises(ValueError, match="non-negative"): - handle.resize(-1) - - -def test_pointer_handle_uses_common_base_and_nullify_operation(): - state = _ArrayState(shape=(5,), value=np.zeros(5, dtype=np.int32)) - - def nullify(_handle): - state.shape = None - state.value = None - - ops = { - **_common_ops(state), - "associated": lambda _handle: state.shape is not None, - "nullify": nullify, - "destroy": lambda _handle: None, - } - handle = PointerArray(dtype="int32", rank=1, **_handle_dispatch(ops), descriptor_ownership="owned") - - assert isinstance(handle, NativeArrayHandleBase) - assert handle.descriptor_kind == "pointer" - assert handle.owned is True - assert handle.associated is True - assert handle.shape == (5,) - assert handle.to_numpy() is state.value - - handle.nullify() - assert handle.associated is False - assert handle.shape is None - assert handle.to_numpy() is None - - -def test_pointer_associate_copies_the_target_as_it_stands_and_does_not_follow_it(): - """A pointer assignment snapshots the source's target, it does not track it.""" - first_value = np.arange(3, dtype=np.float64) - second_value = np.arange(4, dtype=np.float64) - absent = (0, 8, 1, 0, 0, 8) - - def pointer(state): - return PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (state["facts"][4],) if state["facts"][0] else None, - "descriptor": lambda _handle: state["facts"], - "to_numpy": lambda _handle: _numpy_view_from_descriptor_facts(state["facts"], "float64"), - "associated": lambda _handle: state["facts"][0] != 0, - "associate": lambda _handle, facts: state.update(facts=facts), - "nullify": lambda _handle: state.update(facts=absent), - } - ), - to_numpy_policy="descriptor_view", - ) - - destination = pointer({"facts": _descriptor_facts_for_array(first_value)}) - source = pointer({"facts": _descriptor_facts_for_array(second_value)}) - - destination.associate(source) - assert destination.associated is True - assert destination.shape == (4,) - np.testing.assert_array_equal(destination.to_numpy(), second_value) - - source.nullify() - assert destination.associated is True - destination.associate(source) - assert destination.associated is False - - -def test_generated_pointer_associate_hands_over_flat_descriptor_facts(): - value = np.arange(6, dtype=np.float64)[::2] - source = PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: value.shape, - "descriptor": lambda _handle: _descriptor_facts_for_array(value), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - "associate": lambda _handle, _facts: None, - } - ), - to_numpy_policy="unsupported", - ) - received = [] - operations = { - "shape": lambda: None, - "descriptor": lambda: None, - "associated": lambda: False, - "associate": lambda facts: received.append(facts), - "nullify": lambda: None, - } - destination = _native_array_handle_from_generated_dispatch( - "pointer", - "float64", - 1, - _generated_handle_dispatch(operations), - operations, - to_numpy_policy="unsupported", - ) - - destination.associate(source) - - assert received == [(int(value.ctypes.data), 8, 1, 1, 3, 16)] - - -@pytest.mark.parametrize( - ("other", "error", "message"), - [ - (object(), TypeError, "requires another PointerArray"), - ( - PointerArray( - dtype="int32", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="unsupported", - ), - TypeError, - "dtype", - ), - ( - PointerArray( - dtype="float64", - rank=2, - **_handle_dispatch( - { - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="unsupported", - ), - ValueError, - "rank", - ), - ], -) -def test_pointer_associate_rejects_incompatible_sources(other, error, message): - destination = PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "associate": lambda _handle, _descriptor: None, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="unsupported", - ) - - with pytest.raises(error, match=message): - destination.associate(other) - - -def test_pointer_allocation_operations_are_policy_gated_by_capabilities(): - state = _ArrayState(shape=(1,), value=object()) - handle = PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - **_common_ops(state), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - } - ), - ) - - with pytest.raises(NotImplementedError, match="pointer handle operation 'allocate' is not available"): - handle.allocate((3,)) - with pytest.raises(NotImplementedError, match="pointer handle operation 'deallocate' is not available"): - handle.deallocate() - with pytest.raises(NotImplementedError, match="pointer handle operation 'resize' is not available"): - handle.resize((4,)) - - -def test_pointer_allocation_operations_route_when_capabilities_exist(): - state = _ArrayState(shape=None, value=None) - - def allocate(_handle, shape): - state.shape = shape - state.value = object() - - def deallocate(_handle): - state.shape = None - state.value = None - - def resize(_handle, shape): - state.shape = shape - state.value = object() - - handle = PointerArray( - dtype="float64", - rank=2, - **_handle_dispatch( - { - **_common_ops(state), - "associated": lambda _handle: state.shape is not None, - "nullify": lambda _handle: deallocate(_handle), - "allocate": allocate, - "deallocate": deallocate, - "resize": resize, - } - ), - ) - - assert handle.associated is False - handle.allocate((2, 3)) - assert handle.associated is True - assert handle.shape == (2, 3) - - handle.resize([4, 5]) - assert handle.shape == (4, 5) - - handle.deallocate() - assert handle.associated is False - assert handle.shape is None - - -def test_pointer_to_numpy_reports_missing_descriptor_extraction(): - handle = PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (2,), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="unsupported", - ) - - with pytest.raises(NotImplementedError, match="to_numpy extraction is unsupported by completed policy"): - handle.to_numpy() - - -def test_to_numpy_policy_unsupported_reports_completed_policy_block(): - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (2,), - "to_numpy": lambda _handle: pytest.fail("unsupported policy must not call generated extraction"), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - } - ), - to_numpy_policy="unsupported", - ) - - with pytest.raises(NotImplementedError, match="to_numpy extraction is unsupported by completed policy"): - handle.to_numpy() - - -def test_common_shape_dispatch_validates_rank(): - handle = AllocatableArray( - dtype="float64", - rank=2, - **_handle_dispatch( - { - "shape": lambda _handle: (4,), - "to_numpy": lambda _handle: None, - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - } - ), - ) - - with pytest.raises(ValueError, match="shape rank 1 does not match declared rank 2"): - _ = handle.shape - - -def test_common_handle_rejects_invalid_descriptor_kind(): - with pytest.raises(ValueError, match="descriptor_kind must be 'allocatable' or 'pointer'"): - NativeArrayHandleBase( - dtype="float64", - rank=1, - invoke=lambda _operation: None, - capabilities=(), - descriptor_kind="target", - descriptor_ownership="borrowed", - ) - - -def test_common_handle_rejects_invalid_dispatch_contract(): - with pytest.raises(TypeError, match="dispatcher must be callable"): - AllocatableArray(dtype="float64", rank=1, invoke=None, capabilities={"shape", "allocated"}) - with pytest.raises(TypeError, match="capability names must be strings"): - AllocatableArray(dtype="float64", rank=1, invoke=lambda _operation: None, capabilities={1}) - - -def test_common_handle_requires_generated_shape_operation(): - with pytest.raises(ValueError, match="requires generated operation 'shape'"): - AllocatableArray(dtype="float64", rank=1, invoke=lambda _operation: None, capabilities=()) - - -def test_extraction_enabled_handle_requires_generated_to_numpy_operation(): - with pytest.raises(ValueError, match="requires generated operation 'to_numpy'"): - AllocatableArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - } - ), - to_numpy_policy="borrowed_view", - ) - - -def test_pointer_handle_requires_generated_associated_and_nullify_operations(): - with pytest.raises(ValueError, match="requires generated operation 'associated'"): - PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (1,), - "nullify": lambda _handle: None, - } - ), - ) - with pytest.raises(ValueError, match="requires generated operation 'nullify'"): - PointerArray( - dtype="float64", - rank=1, - **_handle_dispatch( - { - "shape": lambda _handle: (1,), - "associated": lambda _handle: True, - } - ), - ) - - -def test_common_handle_rejects_invalid_descriptor_ownership(): - with pytest.raises(ValueError, match="descriptor_ownership must be 'borrowed' or 'owned'"): - AllocatableArray( - dtype="float64", - rank=1, - invoke=lambda _operation: None, - capabilities=(), - descriptor_ownership="temporary", - ) - - -def test_common_handle_rejects_invalid_to_numpy_policy(): - with pytest.raises(ValueError, match="to_numpy_policy must be one of"): - AllocatableArray( - dtype="float64", - rank=1, - invoke=lambda _operation: None, - capabilities=(), - to_numpy_policy="maybe_copy", - ) diff --git a/tests/fortran/pointers/semantics/test_pointer_semantics.py b/tests/fortran/pointers/semantics/test_pointer_semantics.py index cfaacccc5..543d893f4 100644 --- a/tests/fortran/pointers/semantics/test_pointer_semantics.py +++ b/tests/fortran/pointers/semantics/test_pointer_semantics.py @@ -1,95 +1,10 @@ -"""Fortran and edited-`.pyi` pointer semantic contracts.""" +"""Edited-`.pyi` pointer contract diagnostics.""" import re import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.semantics.fortran2ir import fortran_module_to_semantic_module -from tests.fortran._support.semantic_conversion import ( - array_contract, - get_function, -) -from prik.semantics.metadata import NATIVE_ARRAY_DESCRIPTOR_METADATA, OPTIONAL_ABSENT_HANDLE_METADATA -from prik.semantics.native_array_handles import native_array_descriptor_kind -from prik.parsers.fortran import parse_fortran_file as parse_fortran_source - - -def test_fortran_pointer_arrays_and_scalars_preserve_descriptor_semantics(): - source = """ -module pointer_semantics -contains - subroutine inspect(values, scalar) - real(8), pointer, intent(inout) :: values(:) - real(8), pointer, intent(in) :: scalar - end subroutine inspect -end module pointer_semantics -""" - - module = fortran_module_to_semantic_module(parse_fortran_source(source)) - values, scalar = get_function(module, "inspect").arguments - - assert array_contract(values.semantic_type).pointer is True - assert native_array_descriptor_kind(values.semantic_type) == "pointer" - assert scalar.semantic_type.metadata["fortran_pointer"] is True - assert scalar.semantic_type.metadata["fortran_pointer_association"] == "runtime" - assert scalar.semantic_type.storage.pointer_depth == 1 - - -def test_fortran_optional_array_descriptor_preserves_absent_handle_state(): - source = """ -module optional_descriptor_semantics -contains - subroutine inspect(values) - real(8), allocatable, optional, intent(in) :: values(:) - end subroutine inspect -end module optional_descriptor_semantics -""" - - module = fortran_module_to_semantic_module(parse_fortran_source(source)) - values = get_function(module, "inspect").arguments[0] - - assert values.optional is True - assert values.semantic_type.metadata[OPTIONAL_ABSENT_HANDLE_METADATA] is True - - -def test_pyi_pointer_handles_preserve_rank_optionality_and_scalar_state(): - module = parse_pyi_text( - """ -module_values: Pointer[Float64[:, :]] -current: Pointer[Int32] - -def consume(values: Pointer[Float64[:]], maybe_values: Pointer[Float64[:]] | None = ...) -> None: ... -""", - module_name="pointer_contracts", - ) - - module_values, current = [variable.semantic_type for variable in module.variables] - values, maybe_values = module.functions[0].arguments - - assert module_values.metadata[NATIVE_ARRAY_DESCRIPTOR_METADATA] == "pointer" - assert module_values.storage.array.pointer is True - assert module_values.rank == 2 - assert current.metadata["fortran_pointer"] is True - assert current.storage.pointer_depth == 1 - assert values.semantic_type.metadata[NATIVE_ARRAY_DESCRIPTOR_METADATA] == "pointer" - assert maybe_values.semantic_type.metadata[NATIVE_ARRAY_DESCRIPTOR_METADATA] == "pointer" - assert maybe_values.semantic_type.metadata[OPTIONAL_ABSENT_HANDLE_METADATA] is True - assert maybe_values.optional is True - - -@pytest.mark.parametrize( - ("annotation", "message"), - [("Annotated[Float64[:], Pointer]", "use Pointer")], -) -def test_convert_pyi_to_ir_rejects_legacy_array_descriptor_metadata(annotation: str, message: str): - with pytest.raises(ValueError, match=message): - parse_pyi_text( - f""" -values: {annotation} -""", - module_name="legacy_array_descriptors", - ) @pytest.mark.parametrize( @@ -105,6 +20,6 @@ def test_convert_pyi_to_ir_rejects_legacy_array_descriptor_metadata(annotation: ), ], ) -def test_scalar_pointer_results_reject_legacy_descriptor_spellings(source: str, message: str): +def test_scalar_pointer_results_require_a_nullable_value_annotation(source: str, message: str): with pytest.raises(ValueError, match=re.escape(message)): parse_pyi_text(source, module_name="invalid_pointer_projection") diff --git a/tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py b/tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py index b74be1f8e..898f5cef5 100644 --- a/tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py +++ b/tests/fortran/raw_addresses/codegen/test_raw_array_lowering.py @@ -6,20 +6,14 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from prik.policy.ownership import ( - CodegenAction, - DestructionPolicy, NativeBarrierAction, ObjectKind, - OwnershipOwner, PythonBarrierAction, - StorageMode, - TransferMode, ) from prik.policy.completion import complete_semantic_policies from prik.policy.models import ArgumentHandoffMode, BridgeDataAction from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -from prik.planning.models import DatatypeFamily def _raw_array_module(): @@ -54,41 +48,6 @@ def _functions(plan): return {function.binding.python_name: function for function in plan.namespaces[0].functions} -def test_raw_array_addresses_use_one_shared_transfer_and_shape_plan(): - module = _raw_array_module() - function = _functions(WrapperPlanner().build(module))["raw_vector"] - argument = function.arguments[1] - assert argument.projected_call_slot is function.entrypoint.projected_slots[argument.native_position] - assert argument.array is argument.projected_call_slot.array - assert argument.object_kind is ObjectKind.NUMPY_ARRAY - assert argument.ownership_owner is OwnershipOwner.CALLER - assert argument.transfer_mode is TransferMode.IN_PLACE - assert argument.destruction_policy is DestructionPolicy.CALLER - assert argument.storage_mode is StorageMode.STACK - assert argument.boundary_storage_mode is StorageMode.STACK - assert argument.datatype_family is DatatypeFamily.REAL - assert argument.binding.python_action is PythonBarrierAction.RAW_ADDRESS - assert argument.binding.codegen_action is CodegenAction.IN_PLACE_ARGUMENT - assert argument.bridge.native_action is NativeBarrierAction.PASS_RAW_ADDRESS - assert argument.entrypoint.handoff_mode is ArgumentHandoffMode.OPAQUE_ADDRESS - assert argument.bridge.data_action is BridgeDataAction.ASSOCIATE_VIEW - assert argument.bridge.copy_reason is None - - assert argument.array is not None - assert argument.array.rank == 1 - assert argument.array.shape == ("n",) - assert argument.array.axes == ("dense",) - assert argument.array.contiguous is True - assert argument.array.category == "raw_address" - assert argument.array.data_role == argument.entrypoint.handoff_role - assert argument.array.extent_reference_roles == (("raw_array_addresses.raw_vector.n:value",),) - assert argument.array.extent_roles == () - assert argument.array.upper_bound_roles == () - assert argument.array.stride_roles == () - assert argument.array.runtime_rank_role is None - assert argument.array.itemsize_role is None - - def test_raw_array_addresses_reuse_integer_extraction_and_named_array_bridge_association(): artifacts = WrapperGenerator().generate(_raw_array_plan()) c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") diff --git a/tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py b/tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py index 479f2d06e..f5f1b9513 100644 --- a/tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py +++ b/tests/fortran/raw_addresses/codegen/test_scalar_address_lowering.py @@ -1,113 +1,45 @@ -"""Direct-plan scalar storage and raw-address boundary lowering.""" +"""Scalar storage and raw-address boundaries stay distinct plans.""" from __future__ import annotations - from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.ownership import CodegenAction, NativeBarrierAction, ObjectKind, PythonBarrierAction +from prik.policy.ownership import NativeBarrierAction, PythonBarrierAction from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ArgumentHandoffMode, BridgeDataAction, DirectResultABI +from prik.policy.models import BridgeDataAction from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -def _scalar_boundary_plan(): +def test_checked_scalar_storage_and_raw_address_complete_distinct_plans_and_guards(): + """Checked storage validates the NumPy object; a raw address trusts the integer. + + The end-to-end suites prove dtype, rank and writeability checks; byte order + and alignment guards are only visible in the generated binding. + """ module = parse_pyi_text( """ def storage(x: Float64[()]) -> None: ... def raw(x: Addr(Float64)) -> None: ... -def direct_storage_result() -> Float64[()]: ... -@native_call([Return("out", 0)]) -def hidden_storage_result() -> Float64[()]: ... """, module_name="scalar_boundaries", ) complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def test_scalar_storage_and_raw_address_plans_keep_explicit_boundary_facts(): - plan = _scalar_boundary_plan() + plan = WrapperPlanner().build(module) functions = {function.binding.python_name: function for function in plan.namespaces[0].functions} - storage_function = functions["storage"] - raw_function = functions["raw"] - direct_function = functions["direct_storage_result"] - hidden_function = functions["hidden_storage_result"] - storage = storage_function.arguments[0] - raw = raw_function.arguments[0] - direct_result = direct_function.results[0] - hidden_result = hidden_function.results[0] - - assert storage.projected_call_slot is storage_function.entrypoint.projected_slots[storage.native_position] - assert storage.object_kind is ObjectKind.NUMPY_ARRAY - assert storage.array.rank == 0 - assert storage.array.category == "scalar_storage" - assert storage.binding.python_action is PythonBarrierAction.SCALAR_STORAGE - assert storage.binding.writable is True - assert storage.bridge.native_action is NativeBarrierAction.PASS_STORAGE_ADDRESS - assert storage.entrypoint.handoff_mode is ArgumentHandoffMode.OPAQUE_ADDRESS - assert storage.bridge.data_action is BridgeDataAction.ASSOCIATE_VIEW - assert storage.bridge.copy_reason is None - assert raw.projected_call_slot is raw_function.entrypoint.projected_slots[raw.native_position] - assert raw.binding.python_action is PythonBarrierAction.RAW_ADDRESS - assert raw.bridge.native_action is NativeBarrierAction.PASS_RAW_ADDRESS - assert raw.entrypoint.handoff_mode is ArgumentHandoffMode.OPAQUE_ADDRESS - assert raw.bridge.data_action is BridgeDataAction.ASSOCIATE_VIEW - assert raw.bridge.copy_reason is None - assert direct_result.object_kind is ObjectKind.NUMPY_ARRAY - assert direct_result.array.rank == 0 - assert direct_result.array.category == "scalar_storage" - assert direct_result.binding.codegen_action is CodegenAction.COPY_OUT - assert direct_result.bridge.native_action is NativeBarrierAction.NONE - assert direct_result.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - assert direct_result.entrypoint.direct_result_abi is DirectResultABI.NOT_APPLICABLE - assert hidden_result.object_kind is ObjectKind.NUMPY_ARRAY - assert hidden_result.array.rank == 0 - assert hidden_result.array.category == "scalar_storage" - assert hidden_result.binding.codegen_action is CodegenAction.COPY_OUT - assert hidden_result.bridge.native_action is NativeBarrierAction.PASS_STORAGE_ADDRESS - assert hidden_result.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION + storage = functions["storage"].arguments[0] + raw = functions["raw"].arguments[0] + assert (storage.binding.python_action, storage.bridge.native_action) == ( + PythonBarrierAction.SCALAR_STORAGE, + NativeBarrierAction.PASS_STORAGE_ADDRESS, + ) + assert (raw.binding.python_action, raw.bridge.native_action) == ( + PythonBarrierAction.RAW_ADDRESS, + NativeBarrierAction.PASS_RAW_ADDRESS, + ) + assert storage.bridge.data_action is raw.bridge.data_action is BridgeDataAction.ASSOCIATE_VIEW -def test_scalar_storage_and_raw_address_lower_to_direct_named_paths(): - artifacts = WrapperGenerator().generate(_scalar_boundary_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "void bind_c_storage(void * x);" in c_source - assert "PyArray_TYPE((PyArrayObject *)bound_x_obj) != NPY_FLOAT64" in c_source - assert "PyArray_NDIM((PyArrayObject *)bound_x_obj) != 0" in c_source + c_source = next(source.text for source in WrapperGenerator().generate(plan).sources if source.path.suffix == ".c") assert "PyArray_ISNOTSWAPPED((PyArrayObject *)bound_x_obj)" in c_source assert "PyArray_ISALIGNED((PyArrayObject *)bound_x_obj)" in c_source - assert "PyArray_ISWRITEABLE((PyArrayObject *)bound_x_obj)" in c_source - assert "bound_x = PyArray_DATA((PyArrayObject *)bound_x_obj);" in c_source - assert "bind_c_storage(bound_x);" in c_source - assert "void bind_c_raw(void * x);" in c_source - assert "if (!PyLong_Check(bound_x_obj))" in c_source assert "bound_x = PyLong_AsVoidPtr(bound_x_obj);" in c_source - assert "bind_c_raw(bound_x);" in c_source - assert "void * bind_c_direct_storage_result(void);" in c_source - assert "void bind_c_hidden_storage_result(void ** out);" in c_source - assert c_source.count("PyArray_New(&PyArray_Type, 0, NULL, NPY_FLOAT64") == 2 - assert "bind_c_hidden_storage_result(&out);" in c_source - - assert 'subroutine bind_c_storage(bound_x) bind(c, name="bind_c_storage")' in bridge_source - assert 'subroutine bind_c_raw(bound_x) bind(c, name="bind_c_raw")' in bridge_source - assert bridge_source.count("type(c_ptr), value :: bound_x") == 2 - assert bridge_source.count("call c_f_pointer(bound_x, x)") == 2 - assert "call native_storage(x)" in bridge_source - assert "call native_raw(x)" in bridge_source - assert ( - 'function bind_c_direct_storage_result() result(result) bind(c, name="bind_c_direct_storage_result")' - in bridge_source - ) - assert 'subroutine bind_c_hidden_storage_result(out) bind(c, name="bind_c_hidden_storage_result")' in bridge_source - assert "real(c_double) :: result_value" in bridge_source - assert "real(c_double), pointer :: result_copy" in bridge_source - assert "call c_f_pointer(result, result_copy)" in bridge_source - assert "result_copy = result_value" in bridge_source - assert "real(c_double) :: out_value" in bridge_source - assert "real(c_double), pointer :: out_copy" in bridge_source - assert "call c_f_pointer(out, out_copy)" in bridge_source - assert "out_copy = out_value" in bridge_source - assert "dimension()" not in bridge_source diff --git a/tests/fortran/raw_addresses/codegen/test_string_address_lowering.py b/tests/fortran/raw_addresses/codegen/test_string_address_lowering.py index 543b95aa9..4f8256c01 100644 --- a/tests/fortran/raw_addresses/codegen/test_string_address_lowering.py +++ b/tests/fortran/raw_addresses/codegen/test_string_address_lowering.py @@ -1,4 +1,4 @@ -"""Direct-plan mutable fixed string storage and raw-address lowering.""" +"""Fixed-string storage and raw-address plans fail closed when edited.""" from __future__ import annotations @@ -6,22 +6,10 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from prik.policy.ownership import ( - CodegenAction, - DestructionPolicy, - NativeBarrierAction, - ObjectKind, OwnershipOwner, - PythonBarrierAction, StorageMode, - TransferMode, ) from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - ArgumentHandoffMode, - BridgeDataAction, - RAW_STRING_ADDRESS_COPY_REASON, - STRING_STORAGE_COPY_REASON, -) from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner @@ -46,76 +34,6 @@ def _functions(plan): return {function.binding.python_name: function for function in plan.namespaces[0].functions} -def test_string_address_plans_keep_completed_ownership_length_and_copy_facts(): - module = _string_address_module() - functions = _functions(WrapperPlanner().build(module)) - storage = functions["storage"].arguments[0] - raw = functions["raw"].arguments[0] - for argument in (storage, raw): - assert argument.character_length == 8 - assert argument.object_kind is ObjectKind.STRING - assert argument.ownership_owner is OwnershipOwner.CALLER - assert argument.transfer_mode is TransferMode.IN_PLACE - assert argument.destruction_policy is DestructionPolicy.CALLER - assert argument.binding.codegen_action is CodegenAction.IN_PLACE_ARGUMENT - assert argument.bridge.codegen_action is CodegenAction.IN_PLACE_ARGUMENT - assert argument.entrypoint.handoff_mode is ArgumentHandoffMode.OPAQUE_ADDRESS - assert argument.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - assert argument.entrypoint.length_handoff_role is None - assert argument.mutates_native is True - assert argument.projects_result is False - - assert storage.binding.python_action is PythonBarrierAction.STRING_STORAGE - assert storage.bridge.native_action is NativeBarrierAction.PASS_STORAGE_ADDRESS - assert storage.storage_mode is StorageMode.ALIAS - assert storage.boundary_storage_mode is StorageMode.ALIAS - assert storage.bridge.copy_reason == STRING_STORAGE_COPY_REASON - assert raw.binding.python_action is PythonBarrierAction.RAW_ADDRESS - assert raw.bridge.native_action is NativeBarrierAction.PASS_RAW_ADDRESS - assert raw.storage_mode is StorageMode.STACK - assert raw.boundary_storage_mode is StorageMode.STACK - assert raw.bridge.copy_reason == RAW_STRING_ADDRESS_COPY_REASON - - -def test_string_addresses_dispatch_to_named_binding_and_bridge_lowering(): - artifacts = WrapperGenerator().generate(_string_address_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - # NumPy-backed storage reports the caller's itemsize beside the address; a - # raw address has no Python object to measure, so it carries only the width - # the contract declared. - assert "void bind_c_storage(void * label, int64_t label_length);" in c_source - assert "PyArray_TYPE((PyArrayObject *)bound_label_obj) != NPY_STRING" in c_source - assert "PyArray_NDIM((PyArrayObject *)bound_label_obj) != 0" in c_source - assert "PyArray_ITEMSIZE((PyArrayObject *)bound_label_obj) != 8" in c_source - assert "PyArray_ISNOTSWAPPED((PyArrayObject *)bound_label_obj)" in c_source - assert "PyArray_ISALIGNED((PyArrayObject *)bound_label_obj)" in c_source - assert "PyArray_ISWRITEABLE((PyArrayObject *)bound_label_obj)" in c_source - assert "bound_label = PyArray_DATA((PyArrayObject *)bound_label_obj);" in c_source - # Every scalar string reports a width beside its address, so the adapter - # has one shape; a raw address states the contract's width. - assert "void bind_c_raw(void * label, int64_t label_length);" in c_source - assert "if (!PyLong_Check(bound_label_obj))" in c_source - assert "bound_label = PyLong_AsVoidPtr(bound_label_obj);" in c_source - assert "prik_malloc" not in c_source - - assert 'subroutine bind_c_storage(bound_label, label_length) bind(c, name="bind_c_storage")' in bridge_source - assert 'subroutine bind_c_raw(bound_label, label_length) bind(c, name="bind_c_raw")' in bridge_source - assert bridge_source.count("type(c_ptr), value :: bound_label") == 2 - # Every scalar string reports a width beside its address, so both shapes - # receive the same two parameters and name the caller's storage directly. - # The binding decides where the width comes from; the callee writes the - # caller's bytes, so nothing is copied back. - assert bridge_source.count("integer(c_int64_t), value :: label_length") == 2 - assert bridge_source.count("character(kind=c_char, len=label_length), pointer :: label") == 2 - assert bridge_source.count("call c_f_pointer(bound_label, label)") == 2 - assert "call native_storage(label)" in bridge_source - assert "call native_raw(label)" in bridge_source - assert "label_bytes" not in bridge_source - assert "transfer(" not in bridge_source - - @pytest.mark.parametrize( ("edit", "diagnostic"), [ diff --git a/tests/fortran/raw_addresses/pipeline/test_invalid_raw_address_contracts.py b/tests/fortran/raw_addresses/pipeline/test_invalid_raw_address_contracts.py index 5bb96c69e..b001d082a 100644 --- a/tests/fortran/raw_addresses/pipeline/test_invalid_raw_address_contracts.py +++ b/tests/fortran/raw_addresses/pipeline/test_invalid_raw_address_contracts.py @@ -7,38 +7,19 @@ from prik import build_pyi_extension -@pytest.mark.parametrize( - ("contract_text", "message"), - [ - ( - "from prik.contracts import Addr, Float64\n\n" - "class particle:\n value: Float64\n\n" - "def invalid(value: Addr(particle)) -> None: ...\n", - r"Addr\(WrappedType\) is not allowed", - ), - ( - "from prik.contracts import Addr, Arg, Float64, native_call\n\n" - "@native_call([Addr(Arg(0))])\n" - "def invalid(values: Float64[:]) -> None: ...\n", - "only valid for primitive scalar values", - ), - ( - "from prik.contracts import Addr, Float64\n\ndef invalid(values: Addr(Float64[:])) -> None: ...\n", - "raw arrays require a fully resolved rank and shape", - ), - ], -) -def test_pyi_python_api_rejects_invalid_address_contracts_before_codegen( - tmp_path: Path, - contract_text: str, - message: str, -): +def test_pyi_python_api_rejects_invalid_address_contracts_before_codegen(tmp_path: Path): + """The public build stops at policy completion; every diagnostic is covered in semantics.""" contract = tmp_path / "invalid_address.pyi" - contract.write_text(contract_text, encoding="utf-8") + contract.write_text( + "from prik.contracts import Addr, Float64\n\n" + "class particle:\n value: Float64\n\n" + "def invalid(value: Addr(particle)) -> None: ...\n", + encoding="utf-8", + ) native_object = tmp_path / "native.o" native_object.touch() - with pytest.raises(ValueError, match=message): + with pytest.raises(ValueError, match=r"Addr\(WrappedType\) is not allowed"): build_pyi_extension(contract, native_objects=[native_object], output_dir=tmp_path / "build") assert not list((tmp_path / "build").glob("*_wrapper.*")) diff --git a/tests/fortran/raw_addresses/policy/test_raw_address_policy.py b/tests/fortran/raw_addresses/policy/test_raw_address_policy.py index 4dc79be48..f208c85f9 100644 --- a/tests/fortran/raw_addresses/policy/test_raw_address_policy.py +++ b/tests/fortran/raw_addresses/policy/test_raw_address_policy.py @@ -1,169 +1,31 @@ -from pathlib import Path +"""Raw address forms that policy must block before codegen.""" +import pytest -from tests.fortran._support.ownership_policy import parse_pyi_text -from tests.fortran._support.wrapper_build import wrapper_source -from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig, read_fortran_source -from prik.semantics.fortran2ir import fortran_project_to_semantic_modules -from prik.semantics.models import ( - RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, -) -from prik.policy.ownership import ( - CodegenAction, - DestructionPolicy, - NativeBarrierAction, - ObjectKind, - OwnershipOwner, - PythonBarrierAction, - StorageMode, - TransferMode, -) from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - RAW_STRING_ADDRESS_COPY_REASON, - STRING_STORAGE_COPY_REASON, - ArgumentHandoffMode, - BridgeDataAction, -) - -FMATH_CONTRACT = Path("tests/fortran/data_types/end_to_end/fixtures/contracts/fmath/__init__.pyi") - - -def _source_semantic_module(filename: str, *, module_name: str): - source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name=module_name) - complete_semantic_policies(module) - return module - - -def test_wrapper_policy_completes_primitive_raw_address_handoff(): - module = parse_pyi_text( - "def update(value: Addr(Float64)) -> None: ...", - module_name="primitive_raw_address", - ) - complete_semantic_policies(module) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - argument = policy.arguments[0] - - assert argument.python_barrier_action is PythonBarrierAction.RAW_ADDRESS - assert argument.native_barrier_action is NativeBarrierAction.PASS_RAW_ADDRESS - assert argument.ownership.owner is OwnershipOwner.CALLER - assert argument.ownership.transfer is TransferMode.IN_PLACE - assert argument.ownership.destruction is DestructionPolicy.CALLER - - -def test_wrapper_policy_completes_required_raw_array_address_handoff(): - module = parse_pyi_text( - """ -def raw_values(n: Int32[()], values: Addr(Float64[n])) -> None: ... -def raw_matrix(n: Int32, m: Int32, values: Addr(Float64[n, m])) -> None: ... -def raw_labels(n: Int32, labels: Addr(String[8][n])) -> None: ... -""", - module_name="raw_array_arguments", - ) - complete_semantic_policies(module) - policies = { - function.name: function.metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] for function in module.functions - } - - assert all(policy.supported for policy in policies.values()) - values = policies["raw_values"].arguments[1] - assert values.ownership.kind is ObjectKind.NUMPY_ARRAY - assert values.python_barrier_action is PythonBarrierAction.RAW_ADDRESS - assert values.native_barrier_action is NativeBarrierAction.PASS_RAW_ADDRESS - assert values.handoff_mode is ArgumentHandoffMode.OPAQUE_ADDRESS - assert values.bridge_data_action is BridgeDataAction.ASSOCIATE_VIEW - assert values.bridge_copy_reason is None - assert values.array is not None - assert values.array.rank == 1 - assert values.array.shape == ("n",) - assert values.array.axes == ("dense",) - assert values.array.category == "raw_address" - assert values.array.contiguous is True - assert values.array.extent_references == (("n",),) - assert policies["raw_values"].native_call_slots[1].array == values.array - - matrix = policies["raw_matrix"].arguments[2] - assert matrix.array is not None - assert matrix.array.order == "ORDER_F" - assert matrix.array.shape == ("n", "m") - - labels = policies["raw_labels"].arguments[1] - assert labels.ownership.kind is ObjectKind.NUMPY_ARRAY - assert labels.character_length == 8 - assert labels.array is not None - assert labels.array.itemsize == 8 - - -def test_wrapper_policy_keeps_optional_raw_array_addresses_blocked(): - module = parse_pyi_text( - "def optional_raw(n: Int32, values: Addr(Float64[n]) = ...) -> None: ...", - module_name="optional_raw_array", - ) - complete_semantic_policies(module) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert policy.supported is False - assert "argument 'values' optional raw array addresses are not supported" in policy.blockers +from prik.semantics.models import RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA +from tests.fortran._support.ownership_policy import parse_pyi_text -def test_wrapper_policy_keeps_projected_raw_array_addresses_blocked(): - module = parse_pyi_text( - 'def projected_raw(n: Int32, values: Addr(Float64[n])) -> Returns["values", Float64[n]]: ...', - module_name="projected_raw_array", - ) +@pytest.mark.parametrize( + ("contract", "blocker"), + [ + pytest.param( + "def optional_raw(n: Int32, values: Addr(Float64[n]) = ...) -> None: ...", + "argument 'values' optional raw array addresses are not supported", + id="optional", + ), + pytest.param( + 'def projected_raw(n: Int32, values: Addr(Float64[n])) -> Returns["values", Float64[n]]: ...', + "argument 'values' raw array address cannot project a Python result", + id="projected-result", + ), + ], +) +def test_wrapper_policy_blocks_raw_array_addresses_it_cannot_prove(contract: str, blocker: str): + module = parse_pyi_text(contract, module_name="blocked_raw_array") complete_semantic_policies(module) policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] assert policy.supported is False - assert "argument 'values' raw array address cannot project a Python result" in policy.blockers - - -def test_wrapper_policy_completes_fixed_string_storage_and_raw_address_ownership(): - module = parse_pyi_text( - """ -def storage(label: String[8][()]) -> None: ... -def raw(label: Addr(String[8])) -> None: ... -""", - module_name="fixed_string_addresses", - ) - complete_semantic_policies(module) - storage = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - raw = module.functions[1].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert storage.supported is True - storage_argument = storage.arguments[0] - assert storage_argument.ownership.kind is ObjectKind.STRING - assert storage_argument.ownership.owner is OwnershipOwner.CALLER - assert storage_argument.ownership.transfer is TransferMode.IN_PLACE - assert storage_argument.ownership.destruction is DestructionPolicy.CALLER - assert storage_argument.storage_mode is StorageMode.ALIAS - assert storage_argument.boundary_storage_mode is StorageMode.ALIAS - assert storage_argument.codegen_action is CodegenAction.IN_PLACE_ARGUMENT - assert storage_argument.python_barrier_action is PythonBarrierAction.STRING_STORAGE - assert storage_argument.native_barrier_action is NativeBarrierAction.PASS_STORAGE_ADDRESS - assert storage_argument.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - assert storage_argument.bridge_copy_reason == STRING_STORAGE_COPY_REASON - assert storage_argument.character_length == 8 - assert storage.writeback_actions == () - - assert raw.supported is True - raw_argument = raw.arguments[0] - assert raw_argument.ownership.kind is ObjectKind.STRING - assert raw_argument.ownership.owner is OwnershipOwner.CALLER - assert raw_argument.ownership.transfer is TransferMode.IN_PLACE - assert raw_argument.ownership.destruction is DestructionPolicy.CALLER - assert raw_argument.storage_mode is StorageMode.STACK - assert raw_argument.boundary_storage_mode is StorageMode.STACK - assert raw_argument.codegen_action is CodegenAction.IN_PLACE_ARGUMENT - assert raw_argument.python_barrier_action is PythonBarrierAction.RAW_ADDRESS - assert raw_argument.native_barrier_action is NativeBarrierAction.PASS_RAW_ADDRESS - assert raw_argument.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - assert raw_argument.bridge_copy_reason == RAW_STRING_ADDRESS_COPY_REASON - assert raw_argument.character_length == 8 - assert raw.writeback_actions == () + assert blocker in policy.blockers diff --git a/tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py b/tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py index cacf30f25..4d483752d 100644 --- a/tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py +++ b/tests/fortran/raw_addresses/semantics/test_raw_address_semantics.py @@ -48,52 +48,6 @@ def raw_label(label: Addr(String[8])) -> None: ... assert parse_pyi_text(emitted, module_name="raw_address") == module -def test_wrapped_type_raw_address_is_rejected_during_policy_completion(): - module = parse_pyi_text( - """ -class particle: - value: Float64 - -def move(value: Addr(particle)) -> None: ... -""", - module_name="wrapped_address", - ) - - storage = module.functions[0].arguments[0].semantic_type.storage - assert storage.kind == "address" - assert storage.metadata[ADDRESS_ROLE_METADATA] == ADDRESS_ROLE_RAW - assert ( - emit_module(module) - .strip() - .endswith("class particle:\n value: Float64\n\ndef move(\n value: Addr(particle)\n) -> None: ...") - ) - with pytest.raises(ValueError, match=r"Addr\(WrappedType\) is not allowed"): - complete_semantic_policies(module) - - -def test_raw_address_policy_accepts_only_complete_primitive_layouts(): - module = parse_pyi_text( - """ -def raw_access( - n: Int32, - scalar: Addr(Float64), - label: Addr(String[8]), - values: Addr(Float64[n]) -) -> Addr(Int32): ... - -def raw_access_with_storage_extent( - n: Int32[()], - values: Addr(Float64[n]) -) -> None: ... -""", - module_name="raw_addresses", - ) - - complete_semantic_policies(module) - - assert module.metadata["policy_completion_prepared"] is True - - @pytest.mark.parametrize( ("annotation", "message"), [ diff --git a/tests/fortran/strings/codegen/fixtures/native/descriptor_local.f90 b/tests/fortran/strings/codegen/fixtures/native/descriptor_local.f90 deleted file mode 100644 index f4533e3d1..000000000 --- a/tests/fortran/strings/codegen/fixtures/native/descriptor_local.f90 +++ /dev/null @@ -1,23 +0,0 @@ -module descriptor_locals - implicit none -contains - subroutine fixed_allocatable(value, length) - character(len=4), allocatable, intent(in) :: value - integer(4), intent(out) :: length - length = len(value) - end subroutine fixed_allocatable - subroutine deferred_pointer(value, length) - character(len=:), pointer, intent(in) :: value - integer(4), intent(out) :: length - length = len(value) - end subroutine deferred_pointer - subroutine fixed_pointer(value, length) - character(len=4), pointer, intent(in) :: value - integer(4), intent(out) :: length - length = len(value) - end subroutine fixed_pointer - subroutine pointer_update(value) - character(len=:), pointer, intent(inout) :: value - if (associated(value)) value = 'z' - end subroutine pointer_update -end module descriptor_locals diff --git a/tests/fortran/strings/codegen/test_character_array_lowering.py b/tests/fortran/strings/codegen/test_character_array_lowering.py index 39e2770d8..a28e83013 100644 --- a/tests/fortran/strings/codegen/test_character_array_lowering.py +++ b/tests/fortran/strings/codegen/test_character_array_lowering.py @@ -5,12 +5,9 @@ import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.ownership import CodegenAction, NativeBarrierAction, ObjectKind from prik.policy.completion import complete_semantic_policies -from prik.policy.models import BridgeDataAction from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -from prik.planning.models import DatatypeFamily def _later_array_plan(): @@ -52,24 +49,6 @@ def test_character_itemsize_edit_fails_before_backend_lowering(): WrapperGenerator().generate(plan) -def test_fixed_width_character_array_results_reuse_the_ordinary_array_copy_plan(): - direct_function, hidden_function = _character_array_result_plan().namespaces[0].functions - direct = direct_function.results[0] - hidden = hidden_function.results[0] - - for result, itemsize in ((direct, 5), (hidden, 4)): - assert result.object_kind is ObjectKind.NUMPY_ARRAY - assert result.datatype_family is DatatypeFamily.STRING - assert result.array is not None - assert result.array.itemsize == itemsize - assert result.character_length == itemsize - assert result.binding.codegen_action is CodegenAction.COPY_OUT - assert result.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - assert direct.bridge.native_action is NativeBarrierAction.NONE - assert hidden.bridge.native_action is NativeBarrierAction.PASS_ARRAY_BUFFER - assert hidden.projected_call_slot.object_kind is ObjectKind.NUMPY_ARRAY - - def test_fixed_width_character_array_results_lower_itemsize_into_both_backends(): artifacts = WrapperGenerator().generate(_character_array_result_plan()) c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") diff --git a/tests/fortran/strings/codegen/test_fixed_string_result_lowering.py b/tests/fortran/strings/codegen/test_fixed_string_result_lowering.py index d2fb2944b..772778775 100644 --- a/tests/fortran/strings/codegen/test_fixed_string_result_lowering.py +++ b/tests/fortran/strings/codegen/test_fixed_string_result_lowering.py @@ -7,23 +7,12 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from prik.semantics.models import RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA from prik.policy.ownership import ( - CodegenAction, - DestructionPolicy, - NativeBarrierAction, ObjectKind, - OwnershipOwner, - PythonBarrierAction, - StorageMode, - TransferMode, ) from prik.policy.completion import complete_semantic_policies from prik.policy.models import BridgeDataAction from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -from prik.planning.models import DatatypeFamily - - -_COPY_REASON = "copy fixed-length Fortran character output into C-owned null-terminated storage" def _fixed_string_module(): @@ -44,82 +33,11 @@ def _fixed_string_plan(): return WrapperPlanner().build(_fixed_string_module()) -def test_fixed_strings_reuse_ordered_result_plans_with_completed_length_and_copy_facts(): - module = _fixed_string_module() - plan = WrapperPlanner().build(module) - functions = {function.binding.python_name: function for function in plan.namespaces[0].functions} - direct = functions["direct_label"].results[0] - hidden = functions["hidden_label"].results[0] - - for result in (direct, hidden): - assert result.datatype_family is DatatypeFamily.STRING - assert result.character_length == 8 - assert result.object_kind is ObjectKind.STRING - assert result.ownership_owner is OwnershipOwner.PYTHON - assert result.transfer_mode is TransferMode.COPY_RETURN - assert result.destruction_policy is DestructionPolicy.PYTHON_REFCOUNT - assert result.storage_mode is StorageMode.STACK - assert result.boundary_storage_mode is StorageMode.STACK - assert result.nullable is False - assert result.binding.python_action is PythonBarrierAction.NONE - assert result.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - assert result.bridge.copy_reason == _COPY_REASON - - assert direct.source_kind == "direct_return" - assert direct.binding.codegen_action is CodegenAction.COPY_OUT - assert direct.bridge.native_action is NativeBarrierAction.NONE - assert direct.projected_call_slot is None - - assert hidden.source_kind == "hidden_output" - assert hidden.binding.codegen_action is CodegenAction.COPY_OUT - assert hidden.bridge.native_action is NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS - assert hidden.projected_call_slot is functions["hidden_label"].entrypoint.projected_slots[0] - assert hidden.projected_call_slot.object_kind is ObjectKind.STRING - assert hidden.projected_call_slot.character_length == hidden.character_length - - -def test_fixed_string_results_dispatch_to_named_binding_and_bridge_copy_lowering(): - artifacts = WrapperGenerator().generate(_fixed_string_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "void * bind_c_direct_label(void);" in c_source - assert "void * result = NULL;" in c_source - assert "result = bind_c_direct_label();" in c_source - assert "if (result == NULL)" in c_source - assert 'Py_BuildValue("s", (const char *)result)' in c_source - assert "free(result);" in c_source - assert "void bind_c_hidden_label(void ** label);" in c_source - assert "bind_c_hidden_label(&label);" in c_source - assert 'Py_BuildValue("s", (const char *)label)' in c_source - assert "free(label);" in c_source - - assert 'function bind_c_direct_label() result(result) bind(c, name="bind_c_direct_label")' in bridge_source - assert "type(c_ptr) :: result" in bridge_source - assert "character(kind=c_char, len=8) :: result_value" in bridge_source - assert "result_value = native_direct_label()" in bridge_source - assert "result = c_malloc(9_c_size_t)" in bridge_source - assert "result_copy(1:8) = transfer(result_value, result_copy(1:8))" in bridge_source - assert "result_copy(9) = c_null_char" in bridge_source - assert 'subroutine bind_c_hidden_label(label) bind(c, name="bind_c_hidden_label")' in bridge_source - assert "character(kind=c_char, len=8) :: label_value" in bridge_source - assert "call native_hidden_label(label_value)" in bridge_source - assert "label = c_malloc(9_c_size_t)" in bridge_source - - @pytest.mark.parametrize( ("edit", "diagnostic"), [ ("missing-length", "invalid-result-character-length"), ("wrong-copy", "invalid-string-result-data-action"), - ("wrong-copy-reason", "invalid-string-result-copy-reason"), - ("wrong-object-kind", "invalid-scalar-result-datatype-family"), - ("wrong-owner", "invalid-string-result-owner"), - ("wrong-transfer", "invalid-string-result-transfer"), - ("wrong-destruction", "invalid-string-result-destruction"), - ("wrong-storage", "invalid-string-result-storage"), - ("wrong-boundary-storage", "invalid-string-result-boundary-storage"), - ("nullable", "nullable-fixed-string-result"), ("slot-length-drift", "inconsistent-result-character-length"), ], ) @@ -134,22 +52,6 @@ def test_fixed_string_result_plan_edits_fail_before_backend_lowering(edit: str, elif edit == "wrong-copy": direct.bridge.data_action = BridgeDataAction.DIRECT_TRANSFER direct.bridge.copy_reason = None - elif edit == "wrong-copy-reason": - direct.bridge.copy_reason = "an edited reason" - elif edit == "wrong-object-kind": - direct.object_kind = ObjectKind.SCALAR - elif edit == "wrong-owner": - direct.ownership_owner = OwnershipOwner.NATIVE - elif edit == "wrong-transfer": - direct.transfer_mode = TransferMode.BORROWED_VIEW - elif edit == "wrong-destruction": - direct.destruction_policy = DestructionPolicy.NATIVE_OWNER - elif edit == "wrong-storage": - direct.storage_mode = StorageMode.HEAP - elif edit == "wrong-boundary-storage": - direct.boundary_storage_mode = StorageMode.HEAP - elif edit == "nullable": - direct.nullable = True else: hidden.projected_call_slot.character_length = 7 diff --git a/tests/fortran/strings/codegen/test_fixed_string_writeback.py b/tests/fortran/strings/codegen/test_fixed_string_writeback.py index 27db3d440..caa41af1f 100644 --- a/tests/fortran/strings/codegen/test_fixed_string_writeback.py +++ b/tests/fortran/strings/codegen/test_fixed_string_writeback.py @@ -7,25 +7,15 @@ import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.ownership import ( - CodegenAction, - DestructionPolicy, - ObjectKind, - OwnershipOwner, - StorageMode, - TransferMode, -) from prik.policy.completion import complete_semantic_policies from prik.policy.models import ( - BridgeDataAction, OptionalMode, PythonExceptionKind, - STRING_REPLACEMENT_COPY_REASON, WritebackPhase, ) from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -from prik.planning.models import BindingStatusErrorPlan, DatatypeFamily +from prik.planning.models import BindingStatusErrorPlan def _fixed_writeback_module(): @@ -48,69 +38,6 @@ def _functions(plan): return {function.binding.python_name: function for function in plan.namespaces[0].functions} -def test_fixed_replacement_projects_completed_argument_and_lifecycle_facts(): - module = _fixed_writeback_module() - functions = _functions(WrapperPlanner().build(module)) - replacement = functions["replace_name"] - argument = replacement.arguments[0] - assert argument.character_length == 8 - assert argument.object_kind is ObjectKind.STRING - assert argument.ownership_owner is OwnershipOwner.PYTHON - assert argument.transfer_mode is TransferMode.COPY_RETURN - assert argument.destruction_policy is DestructionPolicy.PYTHON_REFCOUNT - assert argument.storage_mode is StorageMode.STACK - assert argument.boundary_storage_mode is StorageMode.STACK - assert argument.nullable is False - assert argument.mutates_native is False - assert argument.projects_result is True - assert argument.result_position == 0 - assert argument.binding.codegen_action is CodegenAction.COPY_IN_OUT - assert argument.bridge.codegen_action is CodegenAction.COPY_IN_OUT - assert argument.projected_call_slot.adapter.codegen_action is CodegenAction.COPY_IN_OUT - assert argument.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - assert argument.bridge.copy_reason == STRING_REPLACEMENT_COPY_REASON - assert tuple(action.phase for action in replacement.writeback_actions) == tuple(WritebackPhase) - assert all(action.semantic_type_name == "String" for action in replacement.writeback_actions) - assert all(action.datatype_family is DatatypeFamily.STRING for action in replacement.writeback_actions) - - identity = functions["discard_name"] - assert identity.arguments[0].binding.codegen_action is CodegenAction.CALL_LOCAL_INPUT - assert identity.arguments[0].bridge.codegen_action is CodegenAction.CALL_LOCAL_INPUT - assert identity.arguments[0].projects_result is False - assert identity.writeback_actions == () - - -def test_fixed_string_writeback_dispatches_to_named_binding_and_bridge_lowering(): - artifacts = WrapperGenerator().generate(_fixed_writeback_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "void bind_c_replace_name(char * name, int64_t name_length);" in c_source - assert "const char * bound_name_source = NULL;" in c_source - assert "char * bound_name = NULL;" in c_source - assert "bound_name = (char *)prik_malloc((size_t)bound_name_length + 1);" in c_source - assert 'PyExc_MemoryError, "Unable to allocate mutable string buffer for argument name."' in c_source - assert "memcpy(bound_name, bound_name_source, (size_t)bound_name_length);" in c_source - assert "bound_name[bound_name_length] = '\\0';" in c_source - assert "bind_c_replace_name(bound_name, (int64_t)bound_name_length);" in c_source - # Converting the updated value and releasing the copied buffer is one - # runtime call, made before the result is checked. - conversion = c_source.index("prik_character_result(bound_name_obj, &bound_name, ") - assert conversion < c_source.index("if (result_obj == NULL)", conversion) - assert "void bind_c_discard_name(const char * name, int64_t name_length);" in c_source - assert "bind_c_discard_name(bound_name, (int64_t)bound_name_length);" in c_source - - # The local names the binding's buffer instead of copying it in and back - # out, so a mutating callee has already written the storage the binding - # will read, and the terminator it wrote past the width is out of reach. - assert "character(kind=c_char, len=name_length), pointer :: name" in bridge_source - assert "call c_f_pointer(bound_name, name)" in bridge_source - assert "call native_replace_name(name)" in bridge_source - assert "call native_discard_name(name)" in bridge_source - assert "name_bytes" not in bridge_source - assert "transfer(" not in bridge_source - - def test_fixed_string_replacements_validate_first_and_cleanup_every_live_buffer(): module = parse_pyi_text( """ @@ -159,110 +86,39 @@ def replace_and_return( assert "if (__return_0 != NULL) { free(__return_0); __return_0 = NULL; }" in writeback_failure -def test_assumed_and_optional_string_replacements_reuse_runtime_length_and_presence_facts(): - module = parse_pyi_text( - """ -def assumed(name: String) -> Returns["name", String]: ... -def optional(label: String = ...) -> Returns["label", String] | None: ... -def optional_identity(label: String = ...) -> None: ... -""", - module_name="assumed_optional_string_writeback", - ) - complete_semantic_policies(module) - functions = _functions(WrapperPlanner().build(module)) - for name in ("assumed", "optional", "optional_identity"): - argument = functions[name].arguments[0] - assert argument.character_length is None - assert argument.projected_call_slot.character_length is None - assert functions["assumed"].arguments[0].binding.optional_mode is OptionalMode.REQUIRED - assert functions["optional"].arguments[0].binding.optional_mode is OptionalMode.NULLABLE_VALUE - assert functions["optional"].arguments[0].nullable is True - assert functions["optional_identity"].arguments[0].binding.codegen_action is CodegenAction.CALL_LOCAL_INPUT - - -def test_assumed_and_optional_string_lowering_guards_presence_copyback_and_cleanup(): - module = parse_pyi_text( - """ -def assumed(name: String) -> Returns["name", String]: ... -def optional(label: String = ...) -> Returns["label", String] | None: ... -def optional_identity(label: String = ...) -> None: ... -""", - module_name="assumed_optional_string_writeback", - ) - complete_semantic_policies(module) - artifacts = WrapperGenerator().generate(WrapperPlanner().build(module)) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "void bind_c_assumed(char * name, int64_t name_length);" in c_source - assert "void bind_c_optional(char * label, int64_t label_length);" in c_source - assert "PyObject * bound_label_obj = Py_None;" in c_source - assert "if (bound_label_obj != Py_None)" in c_source - assert "bind_c_optional(bound_label, (int64_t)bound_label_length);" in c_source - assert "if (bound_label == NULL)" in c_source - assert "Py_INCREF(Py_None);" in c_source - assert 'result_obj = Py_BuildValue("s", (const char *)bound_label);' in c_source - assert "void bind_c_optional_identity(const char * label, int64_t label_length);" in c_source - - # Required and optional locals both name the binding's buffer; presence - # only decides whether the argument is passed, not how it is reached. - assert "character(kind=c_char, len=name_length), pointer :: name" in bridge_source - assert "character(kind=c_char, len=label_length), pointer :: label" in bridge_source - assert "if (c_associated(bound_label)) then" in bridge_source - assert "call c_f_pointer(bound_label, label)" in bridge_source - assert "call native_optional(label=prik_optional_label)" in bridge_source - assert bridge_source.count("call native_optional(") == 1 - # A mutating callee wrote the binding's bytes, so nothing is copied back. - assert "label_bytes" not in bridge_source - assert "transfer(" not in bridge_source - - @pytest.mark.parametrize( ("edit", "diagnostic"), [ - ("wrong-owner", "invalid-string-replacement-owner"), - ("wrong-copy-reason", "invalid-string-copy-reason"), ("missing-cleanup", "missing-writeback-phase"), ("lifecycle-type-drift", "inconsistent-lifecycle-type"), ("descriptor-presence", "invalid-string-optional-mode"), + ("status-error", "string-writeback-with-status-error"), ], ) def test_fixed_string_writeback_plan_edits_fail_before_backend_lowering(edit: str, diagnostic: str): plan = _fixed_writeback_plan() function = _functions(plan)["replace_name"] argument = function.arguments[0] - if edit == "wrong-owner": - argument.ownership_owner = OwnershipOwner.NATIVE - elif edit == "wrong-copy-reason": - argument.bridge.copy_reason = "an edited copy reason" - argument.projected_call_slot.adapter.bridge_copy_reason = "an edited copy reason" - elif edit == "missing-cleanup": + if edit == "missing-cleanup": function.writeback_actions = tuple( action for action in function.writeback_actions if action.phase is not WritebackPhase.CLEANUP ) elif edit == "lifecycle-type-drift": copy_out = next(action for action in function.writeback_actions if action.phase is WritebackPhase.COPY_OUT) copy_out.semantic_type_name = "Int32" - else: + elif edit == "descriptor-presence": argument.binding.optional_mode = OptionalMode.DESCRIPTOR argument.entrypoint.optional_mode = OptionalMode.DESCRIPTOR + else: + function.binding = replace( + function.binding, + status_error=BindingStatusErrorPlan( + status_role="missing:status", + message_role=None, + success=0, + exception_kind=PythonExceptionKind.RUNTIME_ERROR, + ), + ) with pytest.raises(ValueError, match=diagnostic): WrapperGenerator().generate(plan) - - -def test_fixed_string_writeback_status_edit_fails_at_generator_validation(): - plan = _fixed_writeback_plan() - function = _functions(plan)["replace_name"] - function.binding = replace( - function.binding, - status_error=BindingStatusErrorPlan( - status_role="missing:status", - message_role=None, - success=0, - exception_kind=PythonExceptionKind.RUNTIME_ERROR, - ), - ) - - with pytest.raises(ValueError, match="string-writeback-with-status-error"): - WrapperGenerator().generate(plan) diff --git a/tests/fortran/strings/codegen/test_string_input_lowering.py b/tests/fortran/strings/codegen/test_string_input_lowering.py index 99832e9ad..27c72272a 100644 --- a/tests/fortran/strings/codegen/test_string_input_lowering.py +++ b/tests/fortran/strings/codegen/test_string_input_lowering.py @@ -1,93 +1,19 @@ -"""Direct-plan required scalar string-value input lowering.""" +"""Plan validation for scalar string-value handoffs and deferred-length updates. -from __future__ import annotations +Runtime behavior of every lane is proved end to end in +``tests/fortran/strings/end_to_end/``; these checks keep the validator that +stops an incomplete plan before lowering. +""" -from pathlib import Path +from __future__ import annotations import pytest from tests.fortran._support.ownership_policy import parse_pyi_text -from prik.policy.ownership import CodegenAction, NativeBarrierAction, PythonBarrierAction from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - ArgumentHandoffMode, - BridgeDataAction, - NativeArrayDescriptorKind, - OptionalMode, -) +from prik.policy.models import ArgumentHandoffMode, BridgeDataAction from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner -from prik.planning.models import DatatypeFamily - -NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" - - -def _string_input_module(): - module = parse_pyi_text( - """ -def fixed(text: String[8]) -> Int32: ... -def assumed(text: String) -> Int32: ... -""", - module_name="string_inputs", - ) - complete_semantic_policies(module) - return module - - -def _string_input_plan(): - return WrapperPlanner().build(_string_input_module()) - - -def test_required_string_values_reuse_argument_plan_with_character_handoff_facts(): - module = _string_input_module() - plan = WrapperPlanner().build(module) - functions = {function.binding.python_name: function for function in plan.namespaces[0].functions} - fixed = functions["fixed"].arguments[0] - assumed = functions["assumed"].arguments[0] - - for function_name, argument in (("fixed", fixed), ("assumed", assumed)): - assert argument.datatype_family is DatatypeFamily.STRING - assert argument.binding.python_action is PythonBarrierAction.STRING_VALUE - assert argument.bridge.native_action is NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS - assert argument.entrypoint.handoff_mode is ArgumentHandoffMode.CHARACTER_BUFFER - assert argument.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - assert argument.bridge.copy_reason == ( - "materialize Fortran character storage from the binding UTF-8 byte buffer" - ) - assert argument.projected_call_slot.adapter.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - assert argument.entrypoint.length_handoff_role == f"{argument.owner_path}:length" - assert argument.projected_call_slot is functions[function_name].entrypoint.projected_slots[0] - assert argument.projected_call_slot.adapter.codegen_action is CodegenAction.CALL_LOCAL_INPUT - - assert fixed.projected_call_slot.character_length == 8 - assert assumed.projected_call_slot.character_length is None - - -def test_required_string_values_dispatch_to_named_binding_and_bridge_lowering(): - artifacts = WrapperGenerator().generate(_string_input_plan()) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "#include " in c_source - assert "const char * bound_text = NULL;" in c_source - # A fixed width also accepts rank-zero bytes; both routes and their checks - # are one runtime call with the width and the NUL rule. - assert 'prik_character_input(bound_text_obj, 8, 0, 0, "text", &bound_text, &bound_text_length)' in c_source - # An assumed length has no storage route, so it validates encoded text inline. - assert "bound_text = PyUnicode_AsUTF8AndSize(bound_text_obj, &bound_text_length);" in c_source - assert "strlen(bound_text) != bound_text_length" in c_source - assert "bind_c_fixed(bound_text, (int64_t)bound_text_length)" in c_source - assert "bind_c_assumed(bound_text, (int64_t)bound_text_length)" in c_source - - assert "type(c_ptr), value :: bound_text" in bridge_source - assert "integer(c_int64_t), value :: text_length" in bridge_source - # The local names the binding's buffer rather than copying it through a - # byte array, as a rank-zero numeric argument already does. - assert "character(kind=c_char, len=text_length), pointer :: text" in bridge_source - assert "call c_f_pointer(bound_text, text)" in bridge_source - assert "text_bytes" not in bridge_source - assert "native_fixed(text)" in bridge_source - assert "native_assumed(text)" in bridge_source @pytest.mark.parametrize( @@ -99,7 +25,9 @@ def test_required_string_values_dispatch_to_named_binding_and_bridge_lowering(): ], ) def test_string_handoff_plan_edits_fail_before_backend_lowering(edit: str, diagnostic: str): - plan = _string_input_plan() + module = parse_pyi_text("def fixed(text: String[8]) -> Int32: ...", module_name="string_inputs") + complete_semantic_policies(module) + plan = WrapperPlanner().build(module) argument = plan.namespaces[0].functions[0].arguments[0] if edit == "missing-length": argument.entrypoint.length_handoff_role = None @@ -127,19 +55,6 @@ def test_string_handoff_plan_edits_fail_before_backend_lowering(edit: str, diagn """ -DEFERRED_INPUT_SOURCE = """ -module deferred_input - implicit none -contains - subroutine measure(value, length) - character(len=:), allocatable, intent(in) :: value - integer(4), intent(out) :: length - length = len(value) - end subroutine measure -end module deferred_input -""" - - def _source_route_plan(tmp_path, text: str, module_name: str): from prik.parsers.fortran.parser import parse_fortran_project from prik.pipeline.build import ( @@ -159,102 +74,10 @@ def _source_route_plan(tmp_path, text: str, module_name: str): return WrapperPlanner().build(module) -def _deferred_input_plan(tmp_path): - return _source_route_plan(tmp_path, DEFERRED_INPUT_SOURCE, "deferred_input") - - def _deferred_update_plan(tmp_path): return _source_route_plan(tmp_path, DEFERRED_UPDATE_SOURCE, "deferred_update") -def test_deferred_length_string_input_plans_an_allocatable_adapter_local(tmp_path): - """The bridge facet carries the deferred fact; the shared entrypoint does not. - - A deferred-length dummy cannot appear in a ``bind(C)`` interface, so the - adapter local is adapter-local conversion rather than part of the C ABI. - """ - plan = _deferred_input_plan(tmp_path) - function = next( - function - for namespace in plan.namespaces - for function in namespace.functions - if function.binding.python_name == "measure" - ) - argument = function.arguments[0] - - assert argument.bridge.character_local is not None - assert argument.bridge.character_local.deferred_length is True - assert argument.bridge.character_local.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE - assert argument.entrypoint.handoff_mode is ArgumentHandoffMode.CHARACTER_BUFFER - assert argument.bridge.data_action is BridgeDataAction.COPY_REPRESENTATION - - -def test_deferred_length_string_input_lowers_to_allocatable_local_without_changing_the_binding(tmp_path): - """The adapter allocates on assignment; the C binding keeps the byte buffer.""" - artifacts = WrapperGenerator().generate(_deferred_input_plan(tmp_path)) - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - - assert "character(kind=c_char, len=:), allocatable :: value" in bridge_source - assert "transfer(value_bytes, repeat(' ', value_length))" in bridge_source - assert "character(kind=c_char, len=value_length)" not in bridge_source - # The shared C ABI is unchanged: the binding still hands over bytes plus a length. - assert "bind_c_measure" in c_source - - -def test_deferred_length_string_update_plans_one_input_and_one_output_group(tmp_path): - """The update adds an output group beside its input, not a descriptor argument. - - The Python-visible argument keeps the plain character-buffer handoff, so the - C ABI gains only the descriptor output group the reallocated value needs. - """ - plan = _deferred_update_plan(tmp_path) - function = next( - function - for namespace in plan.namespaces - for function in namespace.functions - if function.binding.python_name == "grow" - ) - argument = function.arguments[0] - result = function.results[0] - - assert argument.entrypoint.handoff_mode is ArgumentHandoffMode.CHARACTER_BUFFER - assert argument.binding.optional_mode is OptionalMode.REQUIRED - assert argument.binding.descriptor_boundary is False - assert argument.entrypoint.descriptor_output_role is None - assert argument.projects_character_descriptor_update is True - - assert result.updates_argument is True - assert result.owner_path == argument.owner_path - assert result.projected_call_slot is argument.projected_call_slot - assert result.scalar_descriptor is not None - assert result.entrypoint.parameter_name == "value_output" - assert tuple( - (parameter.owner_path, parameter.source_kind) - for parameter in sorted(function.entrypoint.parameters, key=lambda item: item.position) - ) == ((argument.owner_path, "argument"), (result.owner_path, "hidden_result")) - - -def test_deferred_length_string_update_copies_the_reallocated_local_into_c_storage(tmp_path): - """The adapter reads back the same local the native procedure may reallocate. - - Reading a separate output local would return the value the caller passed in, - which compiles and imports but silently discards the update. - """ - artifacts = WrapperGenerator().generate(_deferred_update_plan(tmp_path)) - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - - assert "character(kind=c_char, len=:), allocatable :: value" in bridge_source - assert "call native_grow(value)" in bridge_source - assert "if (allocated(value)) then" in bridge_source - assert "value_output_length = len(value, kind=c_int64_t)" in bridge_source - assert "transfer(value, value_output_copy(1:value_output_length))" in bridge_source - # No separate output local exists to read the pre-call value from. - assert "value_output_value" not in bridge_source - assert "bind_c_grow(bound_value, (int64_t)bound_value_length, &value_output, " in c_source - - @pytest.mark.parametrize( ("edit", "diagnostic"), [ @@ -288,52 +111,3 @@ def test_deferred_length_string_update_plan_edits_fail_before_backend_lowering( with pytest.raises(ValueError, match=diagnostic): WrapperGenerator().generate(plan) - - -DESCRIPTOR_LOCAL_SOURCE = (NATIVE_FIXTURES / "descriptor_local.f90").read_text(encoding="utf-8") - - -def _descriptor_local_source(tmp_path) -> str: - artifacts = WrapperGenerator().generate(_source_route_plan(tmp_path, DESCRIPTOR_LOCAL_SOURCE, "descriptor_locals")) - return next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - -def test_descriptor_character_locals_carry_the_attribute_the_native_dummy_declares(tmp_path): - """An allocatable or pointer dummy rejects a plain local as its actual argument. - - The local is the only thing that changes: each of these arguments still - crosses the C ABI as a byte buffer and a length. - """ - bridge_source = _descriptor_local_source(tmp_path) - - assert "character(kind=c_char, len=4), allocatable :: value" in bridge_source - assert "character(kind=c_char, len=:), pointer :: value" in bridge_source - assert "character(kind=c_char, len=4), pointer :: value" in bridge_source - - -def test_descriptor_character_locals_are_allocated_before_the_copy_that_needs_them(tmp_path): - """Only a deferred-length allocatable is established by assignment alone. - - A pointer has no storage until it is allocated, and a fixed-length - allocatable would otherwise be moulded from storage that does not exist. - """ - bridge_source = _descriptor_local_source(tmp_path) - - assert "allocate(character(kind=c_char, len=value_length) :: value)" in bridge_source - assert "allocate(value)" in bridge_source - - -def test_pointer_character_locals_release_the_storage_the_adapter_allocated(tmp_path): - """A read-only pointer dummy cannot reassociate, so its allocation is always still ours. - - An update dummy may be reassociated or deallocated by the native procedure, - so the adapter compares against the seed it recorded and leaves native-owned - storage alone. - """ - bridge_source = _descriptor_local_source(tmp_path) - - assert "value => null()" in bridge_source - assert "if (associated(value)) then" in bridge_source - assert "deallocate(value)" in bridge_source - assert "value_seed => value" in bridge_source - assert "if (associated(value, value_seed)) then" in bridge_source diff --git a/tests/fortran/strings/end_to_end/test_assumed_width_character_storage.py b/tests/fortran/strings/end_to_end/test_assumed_width_character_storage.py index 4f664dd3c..e19e9aa32 100644 --- a/tests/fortran/strings/end_to_end/test_assumed_width_character_storage.py +++ b/tests/fortran/strings/end_to_end/test_assumed_width_character_storage.py @@ -14,17 +14,13 @@ pytestmark = pytest.mark.fortran_end_to_end -SCALAR_SOURCE = """module {name} +SOURCE = """module assumed_width contains subroutine stamp(text) character(len=*), intent(inout) :: text text = "abc" end subroutine -end module -""" -ARRAY_SOURCE = """module {name} -contains integer function stamp_all(text) character(len=*), intent(inout) :: text(:) stamp_all = size(text) * 100 + len(text) @@ -33,87 +29,50 @@ end module """ +CONTRACT = """from prik.contracts import Int32, String, bind -def _build(tmp_path: Path, name: str, source: str, contract: str): - (tmp_path / f"{name}.f90").write_text(source.format(name=name), encoding="utf-8") - (tmp_path / f"{name}.pyi").write_text(contract, encoding="utf-8") - result = build_pyi_extension( - tmp_path / f"{name}.pyi", - native_fortran_sources=[tmp_path / f"{name}.f90"], - output_dir=tmp_path / f"build_{name}", - output_name=name, - ) - adapter = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".f90") - return result, adapter +def stamp(text: String[...][()]) -> None: ... + +@bind("stamp") +def stamp_declared(text: String[8][()]) -> None: ... +def stamp_all(text: String[...][:]) -> Int32: ... -def test_assumed_width_scalar_storage_accepts_any_caller_itemsize(tmp_path: Path): - """``String[...][()]`` declares its adapter local from the runtime width.""" - result, adapter = _build( - tmp_path, - "assumed_scalar_any", - SCALAR_SOURCE, - "from prik.contracts import String\n\ndef stamp(text: String[...][()]) -> None: ...\n", +@bind("stamp_all") +def stamp_all_declared(text: String[8][:]) -> Int32: ... +""" + + +def test_assumed_width_storage_accepts_any_itemsize_and_declared_width_still_checks(tmp_path: Path): + """``String[...]`` takes its width from the caller's buffer; a stated width keeps its check. + + The width always crosses beside the address, declared or not, so a scalar + buffer of any itemsize is stamped in place and a character array reports + the itemsize the ABI already carries. + """ + (tmp_path / "assumed_width.f90").write_text(SOURCE, encoding="utf-8") + (tmp_path / "assumed_width.pyi").write_text(CONTRACT, encoding="utf-8") + result = build_pyi_extension( + tmp_path / "assumed_width.pyi", + native_fortran_sources=[tmp_path / "assumed_width.f90"], + output_dir=tmp_path / "build", + output_name="assumed_width", ) module = result.import_module() - # The local names the caller's storage rather than copying it, so it is a - # pointer -- the same shape a rank-zero numeric argument already uses. - assert "character(kind=c_char, len=text_length), pointer :: text" in adapter for width, expected in (("S8", b"abc "), ("S32", b"abc" + b" " * 29)): buffer = np.array(b"Z", dtype=width) assert module.stamp(buffer) is None assert buffer.tobytes() == expected + declared = np.array(b"Z", dtype="S8") + assert module.stamp_declared(declared) is None + assert declared.tobytes() == b"abc " - -def test_declared_and_assumed_scalar_storage_share_one_adapter_shape(tmp_path: Path): - """The width always crosses beside the address, declared or not.""" - _, assumed = _build( - tmp_path, - "assumed_scalar_shape", - SCALAR_SOURCE, - "from prik.contracts import String\n\ndef stamp(text: String[...][()]) -> None: ...\n", - ) - (tmp_path / "declared").mkdir() - _, declared = _build( - tmp_path / "declared", - "declared_scalar_shape", - SCALAR_SOURCE, - "from prik.contracts import String\n\ndef stamp(text: String[8][()]) -> None: ...\n", - ) - - signature = 'subroutine bind_c_stamp(bound_text, text_length) bind(c, name="bind_c_stamp")' - assert signature in assumed - assert signature in declared - - -def test_assumed_width_character_array_accepts_any_caller_itemsize(tmp_path: Path): - """``String[...][:]`` names the itemsize the ABI already reports.""" - result, adapter = _build( - tmp_path, - "assumed_array_any", - ARRAY_SOURCE, - "from prik.contracts import Int32, String\n\ndef stamp_all(text: String[...][:]) -> Int32: ...\n", - ) - module = result.import_module() - - assert "character(kind=c_char, len=text_itemsize)" in adapter for width in ("S8", "S16", "S32"): values = np.array([b"alpha", b"beta"], dtype=width) assert module.stamp_all(values) == np.int32(200 + int(width[1:])) assert values[0] == b"Zlpha" - -def test_declared_array_width_still_checks_the_caller_itemsize(tmp_path: Path): - """A stated width keeps its validation; only an assumed one accepts any.""" - result, _ = _build( - tmp_path, - "declared_array_width", - ARRAY_SOURCE, - "from prik.contracts import Int32, String\n\ndef stamp_all(text: String[8][:]) -> Int32: ...\n", - ) - module = result.import_module() - - assert module.stamp_all(np.array([b"alpha"], dtype="S8")) == np.int32(108) + assert module.stamp_all_declared(np.array([b"alpha"], dtype="S8")) == np.int32(108) with pytest.raises(TypeError, match="itemsize 8"): - module.stamp_all(np.array([b"alpha"], dtype="S16")) + module.stamp_all_declared(np.array([b"alpha"], dtype="S16")) diff --git a/tests/fortran/strings/end_to_end/test_character_array_handle_arguments.py b/tests/fortran/strings/end_to_end/test_character_array_handle_arguments.py index 7f350ce2d..ff604dc0f 100644 --- a/tests/fortran/strings/end_to_end/test_character_array_handle_arguments.py +++ b/tests/fortran/strings/end_to_end/test_character_array_handle_arguments.py @@ -142,7 +142,10 @@ def test_owner_close_is_idempotent_and_finalization_is_safe(allocatable_owner_mo ]""" -def _build_pointer_owner_module(tmp_path: Path): +@pytest.fixture(scope="module") +def pointer_owner_module(tmp_path_factory: pytest.TempPathFactory): + """Build the pointer-owner contract once; each test uses its own handles.""" + tmp_path = tmp_path_factory.mktemp("character-owner-pointer") source = tmp_path / "fcharacter_owner_pointer.f90" source.write_text(POINTER_SOURCE, encoding="utf-8") native_object = _compile_native_object(source, tmp_path / "native") @@ -192,8 +195,8 @@ def _gnu_fortran_version() -> tuple[int, int, int] | None: return tuple(int(parts[index]) if index < len(parts) else 0 for index in range(3)) -def test_fixed_character_pointer_owner_supports_association_and_target_mutation(tmp_path: Path): - module = _build_pointer_owner_module(tmp_path) +def test_fixed_character_pointer_owner_supports_association_and_target_mutation(pointer_owner_module): + module = pointer_owner_module source = Pointer[String[4][:]]() alias = Pointer[String[4][:]]() @@ -224,10 +227,10 @@ def test_fixed_character_pointer_owner_supports_association_and_target_mutation( managed.close() -def test_deferred_character_pointer_owner_supports_zero_copy_view(tmp_path: Path): +def test_deferred_character_pointer_owner_supports_zero_copy_view(pointer_owner_module): if (gnu_version := _gnu_fortran_version()) is not None and gnu_version < (13, 3, 0): pytest.skip("deferred-length character pointer reassociation requires GNU Fortran 13.3 or newer") - module = _build_pointer_owner_module(tmp_path) + module = pointer_owner_module values = Pointer[String[:][:]]() assert module.deferred_state(values) == np.int32(0) assert module.repoint_deferred(values) is values diff --git a/tests/fortran/strings/end_to_end/test_character_constant_quoting.py b/tests/fortran/strings/end_to_end/test_character_constant_quoting.py index 38c319b9b..b04eb9783 100644 --- a/tests/fortran/strings/end_to_end/test_character_constant_quoting.py +++ b/tests/fortran/strings/end_to_end/test_character_constant_quoting.py @@ -21,36 +21,29 @@ SOURCE = (NATIVE_FIXTURES / "fcharacter_constant_quoting.f90").read_text(encoding="utf-8") -@pytest.fixture(scope="module") -def built(tmp_path_factory): - """Build the quoting source once for the read-only checks.""" - tmp_path = tmp_path_factory.mktemp("character_quoting") +def test_character_constants_hold_their_declared_characters_in_extension_and_contract(tmp_path: Path): + """Each constant holds exactly the characters its declared length counts. + + A doubled quote is one quote, a literal's kind is a type fact rather than + part of the value, and the generated contract publishes the same value the + built extension returns. + """ source = tmp_path / "quoting.f90" source.write_text(SOURCE, encoding="utf-8") result = build_fortran_extension(source, output_dir=tmp_path / "build", output_name="quoting_api") - return _import_from_build_dir(result.module_name, result.output_dir) - - -def test_a_doubled_quote_reaches_python_as_one_quote(built): - """Each constant holds exactly the characters its declared length counts.""" - assert built.quoting_mod.word == "don't" - assert built.quoting_mod.pair == 'a"b' - assert built.quoting_mod.plain == "abcd" - - -def test_a_literal_states_its_kind_without_the_kind_joining_the_value(built): - """A literal's kind is a type fact, so only its characters are the value.""" - assert built.quoting_mod.tagged == "abc" - assert built.quoting_mod.numbered == "xyz" - assert built.quoting_mod.tagged_quote == "don't" + built = _import_from_build_dir(result.module_name, result.output_dir) + + expected = { + "word": "don't", + "pair": 'a"b', + "plain": "abcd", + "tagged": "abc", + "numbered": "xyz", + "tagged_quote": "don't", + } + assert {name: getattr(built.quoting_mod, name) for name in expected} == expected - -def test_a_generated_contract_states_the_declared_characters(tmp_path: Path): - """The contract publishes the same value the extension returns.""" - source = tmp_path / "quoting.f90" - source.write_text(SOURCE, encoding="utf-8") contracts = tmp_path / "contracts" - _generate_checked_pyi_contract(source, contracts, None) contract = (contracts / "quoting_mod.pyi").read_text(encoding="utf-8") diff --git a/tests/fortran/strings/end_to_end/test_scalar_string_descriptors.py b/tests/fortran/strings/end_to_end/test_scalar_string_descriptors.py index 127003669..2f8e944f7 100644 --- a/tests/fortran/strings/end_to_end/test_scalar_string_descriptors.py +++ b/tests/fortran/strings/end_to_end/test_scalar_string_descriptors.py @@ -14,24 +14,26 @@ pytestmark = pytest.mark.fortran_end_to_end -@pytest.fixture -def compiled_descriptor_module(pyi_parity_build_mode: str, tmp_path: Path): +@pytest.fixture(scope="module", params=("source", "generated-pyi"), ids=("source", "generated-pyi")) +def compiled_descriptor_module(request: pytest.FixtureRequest, tmp_path_factory: pytest.TempPathFactory): """Build the same module from Fortran source and from its generated contract. Every descriptor form here has a contract spelling, so both routes must reach the same runtime behavior; building only from source would hide a contract that no longer describes the procedure it was generated from. + Each route is built once for the module: the calls below leave no native + state behind, so the tests share one extension per route. """ return _build_source_or_generated_pyi_and_import( DESCRIPTOR_SOURCE, - tmp_path, + tmp_path_factory.mktemp(f"string_descriptors_{request.param}"), { "bind_c_fstring_descriptors_f90_wrapper.f90", "fstring_descriptors_f90_wrapper.c", "fstring_descriptors_f90_wrapper.h", }, CONTRACT_FIXTURES / "fstring_descriptors_f90", - pyi_parity_build_mode, + request.param, wrapper_c_flags=FAULT_INJECTION_C_FLAGS, ) diff --git a/tests/fortran/strings/end_to_end/test_string_direct_entrypoint_routing.py b/tests/fortran/strings/end_to_end/test_string_direct_entrypoint_routing.py index 408e50e4f..4c962f00b 100644 --- a/tests/fortran/strings/end_to_end/test_string_direct_entrypoint_routing.py +++ b/tests/fortran/strings/end_to_end/test_string_direct_entrypoint_routing.py @@ -6,7 +6,6 @@ import pytest from tests.fortran._support.wrapper_build import ( - _build_inline_pyi_contract_module, _build_source_or_generated_pyi_and_import, ) @@ -77,22 +76,3 @@ def test_strings_mixed_route_keeps_only_fixed_length_adapter( ) assert "bind_c_adapted_fixed_code" in bridge assert "direct_char_code" not in bridge - - -def test_strings_mixed_route_matches_edited_source_free_contract(tmp_path: Path): - stem = "strings_mixed_bind_c_f90" - source = (FIXTURES / "native" / f"{stem}.f90").read_text(encoding="utf-8") - contract = (FIXTURES / "contracts" / stem / f"{stem}.pyi").read_text(encoding="utf-8") - contract = contract.replace("from prik.contracts import ", "from prik.contracts import nogil, ") - contract = contract.replace("def direct_char_code(", "@nogil\ndef direct_char_code(").replace( - "def adapted_fixed_code(", "@nogil\ndef adapted_fixed_code(" - ) - module, result = _build_inline_pyi_contract_module( - tmp_path, module_name=stem, source_text=source, contract_text=contract - ) - - assert module.direct_char_code("C") == np.int32(67) - assert module.adapted_fixed_code("D ") == np.int32(68) - bridge = (result.output_dir / f"bind_c_{stem}_wrapper.f90").read_text(encoding="utf-8").casefold() - assert "bind_c_adapted_fixed_code" in bridge - assert "function bind_c_direct_char_code" not in bridge diff --git a/tests/fortran/strings/pipeline/test_generated_string_contracts.py b/tests/fortran/strings/pipeline/test_generated_string_contracts.py deleted file mode 100644 index 0cd0d06dd..000000000 --- a/tests/fortran/strings/pipeline/test_generated_string_contracts.py +++ /dev/null @@ -1,30 +0,0 @@ -"""Generated `.pyi` package fixtures for character wrapper inputs.""" - -from __future__ import annotations - -from pathlib import Path - -import pytest - -from tests.fortran._support.generated_contracts import ( - GeneratedContractCase, - assert_generated_contract_matches_fixture, - contract_case_id, -) - -FIXTURE_ROOT = Path(__file__).resolve().parents[1] / "end_to_end" / "fixtures" -CONTRACT_ROOT = FIXTURE_ROOT / "contracts" -CASES = tuple( - GeneratedContractCase(source.stem, (source,), CONTRACT_ROOT / source.stem) - for source in ( - FIXTURE_ROOT / "native" / "fcharacter_edges_f90.f90", - FIXTURE_ROOT / "native" / "fstring_descriptors_f90.f90", - FIXTURE_ROOT / "native" / "fstrings.f", - FIXTURE_ROOT / "native" / "fstrings_f90.f90", - ) -) - - -@pytest.mark.parametrize("case", CASES, ids=contract_case_id) -def test_string_generated_pyi_contract_matches_fixture(case: GeneratedContractCase, tmp_path: Path): - assert_generated_contract_matches_fixture(case, tmp_path) diff --git a/tests/fortran/strings/policy/test_string_wrapper_policy.py b/tests/fortran/strings/policy/test_string_wrapper_policy.py index 09be15cf6..403331371 100644 --- a/tests/fortran/strings/policy/test_string_wrapper_policy.py +++ b/tests/fortran/strings/policy/test_string_wrapper_policy.py @@ -1,385 +1,120 @@ +"""Character dummy attributes select one completed policy lane before planning. + +Policy owns which adapter local a character dummy gets, who releases it, and +whether a mutable dummy writes back through the caller's buffer or returns its +reallocated value as a descriptor result. No ``bind(C)`` interface can declare +a deferred-length or descriptor character dummy, so the generated Fortran +adapter must build these locals; the bridge only implements the decision. +Runtime behavior of every lane is proved in ``tests/fortran/strings/end_to_end/``. +""" + from pathlib import Path import pytest -from tests.fortran._support.ownership_policy import parse_pyi_text -from tests.fortran._support.wrapper_build import wrapper_source from prik.parsers.fortran.parser import parse_fortran_project from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.policy.completion import complete_semantic_policies +from prik.policy.models import CharacterLocalRelease, NativeArrayDescriptorKind, WritebackPhase +from prik.policy.ownership import CodegenAction, OwnershipOwner from prik.preprocessing import PreprocessingConfig, read_fortran_source from prik.semantics.fortran2ir import fortran_project_to_semantic_modules -from prik.semantics.models import ( - RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA, -) -from prik.policy.ownership import ( - CodegenAction, - DestructionPolicy, - NativeBarrierAction, - ObjectKind, - OwnershipOwner, - PythonBarrierAction, - TransferMode, -) -from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ( - ArgumentConversionPhase, - CharacterLocalRelease, - NativeArrayDescriptorKind, - NativeArrayDefaultConstruction, - NativeArrayOwnerStorage, - NativeDescriptorHandoffABI, - ArgumentHandoffMode, - BridgeDataAction, - OptionalMode, - WritebackPhase, -) - -FMATH_CONTRACT = Path("tests/fortran/data_types/end_to_end/fixtures/contracts/fmath/__init__.pyi") - - -def _source_semantic_module(filename: str, *, module_name: str): - source = wrapper_source(filename) - parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name=module_name) - complete_semantic_policies(module) - return module - - -def test_wrapper_policy_completes_required_read_only_string_value_handoff(): - module = parse_pyi_text( - "def consume(value: String) -> None: ...", - module_name="string_argument", - ) - complete_semantic_policies(module) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert policy.supported is True - assert policy.blockers == () - argument = policy.arguments[0] - assert argument.python_barrier_action is PythonBarrierAction.STRING_VALUE - assert argument.native_barrier_action is NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS - assert argument.codegen_action is CodegenAction.CALL_LOCAL_INPUT - assert argument.handoff_mode is ArgumentHandoffMode.CHARACTER_BUFFER - assert argument.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - assert argument.bridge_copy_reason == ("materialize Fortran character storage from the binding UTF-8 byte buffer") - - -def test_wrapper_policy_completes_fixed_string_direct_and_hidden_copy_results(): - module = parse_pyi_text( - """ -def direct_label() -> String[8]: ... - -@native_call([Return("label", 0)]) -def hidden_label() -> String[8]: ... -""", - module_name="fixed_string_results", - ) - complete_semantic_policies(module) - direct_policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - hidden_policy = module.functions[1].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert direct_policy.supported is True - direct = direct_policy.results[0] - assert direct.ownership.kind is ObjectKind.STRING - assert direct.codegen_action is CodegenAction.COPY_OUT - assert direct.native_barrier_action is NativeBarrierAction.NONE - assert direct.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - assert direct.character_length == 8 +from prik.semantics.models import RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA - assert hidden_policy.supported is True - hidden = hidden_policy.results[0] - assert hidden.ownership.kind is ObjectKind.STRING - assert hidden.codegen_action is CodegenAction.COPY_OUT - assert hidden.native_barrier_action is NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS - assert hidden.bridge_data_action is BridgeDataAction.COPY_REPRESENTATION - assert hidden.character_length == 8 - assert hidden_policy.native_call_slots[0].character_length == hidden.character_length +ALLOCATABLE = NativeArrayDescriptorKind.ALLOCATABLE +POINTER = NativeArrayDescriptorKind.POINTER -def test_wrapper_policy_completes_fixed_string_replacement_and_discarded_identity(): - module = parse_pyi_text( - """ -def replace_name(name: String[8]) -> Returns["name", String[8]]: ... -def discard_name(name: String[8]) -> None: ... +def _completed_policy(declaration: str, tmp_path: Path): + source = tmp_path / "character_lane.f90" + source.write_text( + f""" +module character_lane + implicit none +contains + subroutine work(value) + {declaration} :: value + end subroutine work +end module character_lane """, - module_name="fixed_string_writeback", + encoding="utf-8", ) - complete_semantic_policies(module) - replacement = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - identity = module.functions[1].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert replacement.supported is True - argument = replacement.arguments[0] - assert argument.ownership.kind is ObjectKind.STRING - assert argument.ownership.owner is OwnershipOwner.PYTHON - assert argument.ownership.transfer is TransferMode.COPY_RETURN - assert argument.ownership.destruction is DestructionPolicy.PYTHON_REFCOUNT - assert argument.codegen_action is CodegenAction.COPY_IN_OUT - assert argument.conversion_phase is ArgumentConversionPhase.DEFERRED_REPLACEMENT - assert argument.character_length == 8 - assert argument.projects_result is True - # The native call mutates a binding-owned replacement, not the immutable - # Python string supplied at the public boundary. - assert argument.writable is False - assert tuple(action.phase for action in replacement.writeback_actions) == tuple(WritebackPhase) - - assert identity.supported is True - assert identity.arguments[0].codegen_action is CodegenAction.CALL_LOCAL_INPUT - assert identity.arguments[0].projects_result is False - assert identity.writeback_actions == () - - -def _semantic_module_from_text(source_text: str, tmp_path: Path, *, module_name: str): - """Complete policy for one inline Fortran source without a shared fixture.""" - source = tmp_path / f"{module_name}.f90" - source.write_text(source_text, encoding="utf-8") parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) modules = fortran_project_to_semantic_modules(parsed) _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name=module_name) + module = _merge_wrapper_modules(modules, name="character_lane") complete_semantic_policies(module) - return module - - -def test_read_only_deferred_length_string_argument_completes_deferred_policy(tmp_path: Path): - """A ``character(len=:)`` input records the fact the adapter needs. - - No ``bind(C)`` interface can declare a deferred-length dummy, so the - generated Fortran adapter must build the allocatable local itself. Policy - owns that fact; the bridge only implements it. - """ - module = _semantic_module_from_text( - """ -module deferred_input - implicit none -contains - subroutine measure(value, length) - character(len=:), allocatable, intent(in) :: value - integer(4), intent(out) :: length - length = len(value) - end subroutine measure -end module deferred_input -""", - tmp_path, - module_name="deferred_input", - ) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert policy.supported is True - argument = policy.arguments[0] - assert argument.character_local is not None - assert argument.character_local.deferred_length is True - assert argument.character_local.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE - assert argument.character_local.release is CharacterLocalRelease.NONE - assert argument.character_length is None - assert argument.handoff_mode is ArgumentHandoffMode.CHARACTER_BUFFER - - -def test_fixed_and_assumed_length_string_arguments_stay_plain_locals(): - """Only a descriptor attribute selects a descriptor adapter local. - - ``character(len=8)`` and ``character(len=*)`` are neither allocatable nor - pointer, so both keep the plain fixed-length local and owe no release. - """ - module = parse_pyi_text( - """ -def fixed(text: String[8]) -> Int32: ... -def assumed(text: String) -> Int32: ... -""", - module_name="non_deferred_strings", - ) - complete_semantic_policies(module) - - for index in (0, 1): - policy = module.functions[index].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - local = policy.arguments[0].character_local - assert local is not None - assert local.descriptor_kind is None - assert local.deferred_length is False - assert local.release is CharacterLocalRelease.NONE + return module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] @pytest.mark.parametrize( - ("intent", "release"), + ("declaration", "descriptor_kind", "deferred", "release", "lane"), [ - ("in", CharacterLocalRelease.DEALLOCATE), - ("inout", CharacterLocalRelease.DEALLOCATE_IF_RETAINED), + ("character(len=8), intent(in)", None, False, CharacterLocalRelease.NONE, "input"), + ("character(len=*), intent(in)", None, False, CharacterLocalRelease.NONE, "input"), + ("character(len=8), intent(inout)", None, False, CharacterLocalRelease.NONE, "writeback"), + ("character(len=:), allocatable, intent(in)", ALLOCATABLE, True, CharacterLocalRelease.NONE, "input"), + ("character(len=:), allocatable, intent(inout)", ALLOCATABLE, True, CharacterLocalRelease.NONE, "update"), + ("character(len=8), allocatable, intent(inout)", ALLOCATABLE, False, CharacterLocalRelease.NONE, "update"), + ("character(len=:), pointer, intent(in)", POINTER, True, CharacterLocalRelease.DEALLOCATE, "input"), + ( + "character(len=:), pointer, intent(inout)", + POINTER, + True, + CharacterLocalRelease.DEALLOCATE_IF_RETAINED, + "update", + ), + ], + ids=[ + "fixed-input", + "assumed-input", + "plain-fixed-inout-writes-back", + "deferred-allocatable-input", + "deferred-allocatable-update", + "fixed-allocatable-update", + "pointer-input-always-released", + "pointer-update-released-only-if-retained", ], ) -def test_character_pointer_arguments_complete_their_release_responsibility( - intent: str, +def test_character_dummy_attributes_select_local_release_and_update_lane( + declaration: str, + descriptor_kind: NativeArrayDescriptorKind | None, + deferred: bool, release: CharacterLocalRelease, + lane: str, tmp_path: Path, ): - """A pointer local is storage the adapter allocated, so policy must say who frees it. - - An ``intent(in)`` dummy cannot change its association, so the allocation is - always still the adapter's to release. A mutable dummy may be reassociated - or deallocated by the native procedure, so the adapter may only release the - allocation while the dummy still identifies it. + """The descriptor attribute, not the length, selects the adapter local and update lane. + + A plain dummy accepts the caller's buffer, so an ``inout`` one writes back + through it. An allocatable or pointer dummy will not accept that buffer, so + a mutable one takes a call-local input plus a nullable descriptor result + that returns whatever length the native procedure chose. An ``intent(in)`` + pointer cannot reassociate, so its allocation is always the adapter's to + free; a mutable one may only be freed while the dummy still identifies it. """ - module = _semantic_module_from_text( - f""" -module pointer_input - implicit none -contains - subroutine consume(value, length) - character(len=:), pointer, intent({intent}) :: value - integer(4), intent(out) :: length - length = 0 - if (associated(value)) length = len(value) - end subroutine consume -end module pointer_input -""", - tmp_path, - module_name="pointer_input", - ) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert policy.supported is True - local = policy.arguments[0].character_local - assert local is not None - assert local.descriptor_kind is NativeArrayDescriptorKind.POINTER - assert local.deferred_length is True - assert local.release is release - - -def test_deferred_length_string_update_completes_input_plus_descriptor_result(tmp_path: Path): - """A mutable ``character(len=:)`` dummy keeps its input and gains a result facet. - - The caller's ``str`` cannot carry back a length chosen during the call, so - policy completes two decisions for the one dummy: a call-local character - buffer for the input, and a nullable descriptor result that owns the - reallocated storage. Argument writeback stays absent because the value - travels as that result. - """ - module = _semantic_module_from_text( - """ -module deferred_update - implicit none -contains - subroutine grow(value) - character(len=:), allocatable, intent(inout) :: value - if (allocated(value)) value = value // '!' - end subroutine grow -end module deferred_update -""", - tmp_path, - module_name="deferred_update", - ) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] + policy = _completed_policy(declaration, tmp_path) assert policy.supported is True argument = policy.arguments[0] - assert argument.codegen_action is CodegenAction.CALL_LOCAL_INPUT - assert argument.handoff_mode is ArgumentHandoffMode.CHARACTER_BUFFER - assert argument.optional_mode is OptionalMode.REQUIRED - assert argument.descriptor_boundary is False - assert argument.nullable is False - assert argument.projects_character_descriptor_update is True - assert policy.writeback_actions == () - - result = policy.results[0] - assert result.updates_argument is True - assert result.owner_path == argument.owner_path - assert result.codegen_action is CodegenAction.COPY_OUT - assert result.ownership.owner is OwnershipOwner.PYTHON - assert result.ownership.python_visible is False - assert result.scalar_descriptor is not None - assert result.scalar_descriptor.runtime_length is True - assert result.scalar_descriptor.nullable is True - assert result.scalar_descriptor.release_owner is OwnershipOwner.PYTHON - - -def test_fixed_length_allocatable_string_update_takes_the_descriptor_result_lane(tmp_path: Path): - """The descriptor attribute, not the length, selects the update lane. - - A copy-in/copy-out replacement writes back through the caller's buffer, - which means passing that buffer as the actual argument. An allocatable - dummy will not accept one, so a fixed-length allocatable takes the same - call-local input and projected descriptor result a deferred length does. - """ - module = _semantic_module_from_text( - """ -module fixed_update - implicit none -contains - subroutine relabel(value) - character(len=8), allocatable, intent(inout) :: value - value = 'fixed' - end subroutine relabel -end module fixed_update -""", - tmp_path, - module_name="fixed_update", - ) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] + local = argument.character_local + assert local is not None + assert (local.descriptor_kind, local.deferred_length, local.release) == (descriptor_kind, deferred, release) - assert policy.supported is True - argument = policy.arguments[0] + if lane == "writeback": + assert argument.codegen_action is CodegenAction.COPY_IN_OUT + assert tuple(action.phase for action in policy.writeback_actions) == tuple(WritebackPhase) + assert policy.results == () + return assert argument.codegen_action is CodegenAction.CALL_LOCAL_INPUT - assert argument.projects_character_descriptor_update is True - assert argument.character_local is not None - assert argument.character_local.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE - assert argument.character_local.deferred_length is False assert policy.writeback_actions == () - assert policy.results[0].updates_argument is True - - -@pytest.mark.parametrize("attribute", ["allocatable", "pointer"]) -def test_fixed_length_character_array_arguments_select_a_fortran_owner(tmp_path: Path, attribute: str): - module = _semantic_module_from_text( - f""" -module fixed_array_descriptor - implicit none -contains - subroutine inspect(values) - character(len=5), {attribute}, intent(in) :: values(:) - end subroutine inspect -end module fixed_array_descriptor -""", - tmp_path, - module_name="fixed_array_descriptor", - ) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - assert policy.supported is True - handle = policy.arguments[0].native_array_handle - assert handle is not None - assert handle.owner_storage is NativeArrayOwnerStorage.FORTRAN_OWNER - assert handle.handoff.abi is NativeDescriptorHandoffABI.FORTRAN_OWNER - assert handle.default_handle.construction is NativeArrayDefaultConstruction.LAZY_FORTRAN_OWNER - assert handle.owner_type_name - assert handle.owner_signature - assert handle.call_lease is True - - -def test_plain_fixed_length_string_update_keeps_copy_in_out_replacement(tmp_path: Path): - """A dummy with no descriptor attribute keeps the caller-buffer replacement. - - Nothing about that dummy rejects the caller's buffer as the actual - argument, so it stays on the writeback lane rather than gaining a result. - """ - module = _semantic_module_from_text( - """ -module plain_update - implicit none -contains - subroutine relabel(value) - character(len=8), intent(inout) :: value - value = 'fixed' - end subroutine relabel -end module plain_update -""", - tmp_path, - module_name="plain_update", - ) - policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] - - argument = policy.arguments[0] - assert argument.codegen_action is CodegenAction.COPY_IN_OUT - assert argument.projects_character_descriptor_update is False - assert policy.results == () - assert tuple(action.phase for action in policy.writeback_actions) == tuple(WritebackPhase) + assert argument.projects_character_descriptor_update is (lane == "update") + if lane == "update": + (result,) = policy.results + assert result.updates_argument is True + assert result.owner_path == argument.owner_path + assert result.scalar_descriptor.runtime_length is True + assert result.scalar_descriptor.nullable is True + assert result.scalar_descriptor.release_owner is OwnershipOwner.PYTHON + else: + assert policy.results == () diff --git a/tests/fortran/strings/semantics/test_string_pyi_semantics.py b/tests/fortran/strings/semantics/test_string_pyi_semantics.py index 7f9ddb587..96a4934ca 100644 --- a/tests/fortran/strings/semantics/test_string_pyi_semantics.py +++ b/tests/fortran/strings/semantics/test_string_pyi_semantics.py @@ -5,28 +5,6 @@ from tests.fortran._support.pyi_conversion import parse_pyi_text -def test_rank_zero_string_storage_round_trips_as_empty_tuple_array(): - module = parse_pyi_text( - """ -def rewrite_label(label: String[8][()]) -> None: ... -""", - module_name="string_storage", - ) - - label_type = module.functions[0].arguments[0].semantic_type - - assert label_type.name == "String" - assert label_type.rank == 0 - assert label_type.shape == [] - assert label_type.metadata["fortran_character_length"] == "8" - assert label_type.storage.kind == "array" - assert label_type.storage.array.category == "scalar_storage" - - emitted = emit_module(module) - assert "label: String[8][()]" in emitted - assert parse_pyi_text(emitted, module_name="string_storage") == module - - def test_string_length_and_shape_axes_round_trip(): module = parse_pyi_text( """ diff --git a/tests/fortran/subroutines/codegen/test_hidden_scalar_outputs.py b/tests/fortran/subroutines/codegen/test_hidden_scalar_outputs.py index 877594f08..fca3cdb13 100644 --- a/tests/fortran/subroutines/codegen/test_hidden_scalar_outputs.py +++ b/tests/fortran/subroutines/codegen/test_hidden_scalar_outputs.py @@ -8,36 +8,6 @@ from prik.planning import WrapperPlanner -def test_hidden_scalar_result_is_one_bridge_output_and_one_python_result(): - module = parse_pyi_text( - """ -@bind("SCALE_OUT") -@standalone -@native_call([Addr(Arg(0)), Return("result", 0)]) -def scale(x: Float64) -> Float64: ... -""", - module_name="hidden_result", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - function = plan.namespaces[0].functions[0] - result = function.results[0] - - assert result.projected_call_slot is function.entrypoint.projected_slots[result.projected_call_slot.native_position] - - artifacts = WrapperGenerator().generate(plan) - c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - fortran_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "void bind_c_scale(double * x, double * result);" in c_source - assert "bind_c_scale(bound_x_storage, &result);" in c_source - assert "PyObject * result_obj = prik_float64_to_numpy(&result);" in c_source - assert 'subroutine bind_c_scale(x, result) bind(c, name="bind_c_scale")' in fortran_source - assert "external :: SCALE_OUT" in fortran_source - assert "subroutine SCALE_OUT(" not in fortran_source - assert "call SCALE_OUT(x, result)" in fortran_source - - def test_required_explicit_interface_declares_hidden_result_in_native_order(): module = parse_pyi_text( """ diff --git a/tests/fortran/subroutines/codegen/test_scalar_subroutine_writeback_validation.py b/tests/fortran/subroutines/codegen/test_scalar_subroutine_writeback_validation.py index 49921de15..90898e529 100644 --- a/tests/fortran/subroutines/codegen/test_scalar_subroutine_writeback_validation.py +++ b/tests/fortran/subroutines/codegen/test_scalar_subroutine_writeback_validation.py @@ -1,8 +1,9 @@ -"""Scalar subroutine optional, descriptor, and writeback lowering tests.""" +"""Scalar writeback plans are validated before lowering.""" from __future__ import annotations from dataclasses import replace + import pytest from tests.fortran._support.ownership_policy import parse_pyi_text @@ -12,75 +13,51 @@ from prik.planning import WrapperPlanner -def _artifacts(module): - complete_semantic_policies(module) - return WrapperGenerator().generate(WrapperPlanner().build(module)) - - -def _source(artifacts, suffix: str) -> str: - return next(item.text for item in artifacts.sources if item.path.name.endswith(suffix)) - - -def _replace_root_function(plan, function): - root = plan.namespaces[0] - return replace(plan, namespaces=(replace(root, functions=(function,)), *plan.namespaces[1:])) - - -def test_generator_rejects_incomplete_writeback_phase_group(): - module = parse_pyi_text( - 'def bump(value: Annotated[Int32, Immutable]) -> Returns["value", Int32]: ...', - module_name="invalid_writeback", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - function = plan.namespaces[0].functions[0] - invalid = _replace_root_function( - plan, - replace(function, writeback_actions=function.writeback_actions[:-1]), - ) - - with pytest.raises(ValueError, match="missing-writeback-phase"): - WrapperGenerator().generate(invalid) +def _drop_cleanup(actions): + return actions[:-1] -def test_generator_rejects_writeback_without_python_result_target(): - module = parse_pyi_text( - 'def bump(value: Annotated[Int32, Immutable]) -> Returns["value", Int32]: ...', - module_name="invalid_writeback_target", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - function = plan.namespaces[0].functions[0] - actions = tuple( +def _drop_python_target(actions): + return tuple( replace(action, binding=replace(action.binding, python_result_role=None)) if action.phase is WritebackPhase.COPY_OUT else action - for action in function.writeback_actions + for action in actions ) - invalid = _replace_root_function(plan, replace(function, writeback_actions=actions)) - - with pytest.raises(ValueError, match="missing-python-writeback-target"): - WrapperGenerator().generate(invalid) -def test_generator_rejects_writeback_from_an_unavailable_handoff(): - module = parse_pyi_text( - 'def bump(value: Annotated[Int32, Immutable]) -> Returns["value", Int32]: ...', - module_name="invalid_writeback_source", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - function = plan.namespaces[0].functions[0] - actions = tuple( +def _unavailable_source(actions): + return tuple( replace( action, source_role="missing:value", binding=(replace(action.binding, source_role="missing:value") if action.binding is not None else None), bridge=(replace(action.bridge, source_role="missing:value") if action.bridge is not None else None), ) - for action in function.writeback_actions + for action in actions ) - invalid = _replace_root_function(plan, replace(function, writeback_actions=actions)) - with pytest.raises(ValueError, match=r"unavailable-.*-role"): + +@pytest.mark.parametrize( + ("edit", "diagnostic"), + [ + (_drop_cleanup, "missing-writeback-phase"), + (_drop_python_target, "missing-python-writeback-target"), + (_unavailable_source, r"unavailable-.*-role"), + ], + ids=["incomplete-phase-group", "no-python-result-target", "unavailable-handoff"], +) +def test_generator_rejects_incomplete_scalar_writeback_plans(edit, diagnostic): + module = parse_pyi_text( + 'def bump(value: Annotated[Int32, Immutable]) -> Returns["value", Int32]: ...', + module_name="invalid_writeback", + ) + complete_semantic_policies(module) + plan = WrapperPlanner().build(module) + function = plan.namespaces[0].functions[0] + root = plan.namespaces[0] + invalid_function = replace(function, writeback_actions=edit(function.writeback_actions)) + invalid = replace(plan, namespaces=(replace(root, functions=(invalid_function,)), *plan.namespaces[1:])) + + with pytest.raises(ValueError, match=diagnostic): WrapperGenerator().generate(invalid) diff --git a/tests/fortran/subroutines/end_to_end/test_assumed_scalar_intent.py b/tests/fortran/subroutines/end_to_end/test_assumed_scalar_intent.py index 8515b18c0..a6ee8edab 100644 --- a/tests/fortran/subroutines/end_to_end/test_assumed_scalar_intent.py +++ b/tests/fortran/subroutines/end_to_end/test_assumed_scalar_intent.py @@ -29,6 +29,7 @@ def _module(workdir: Path, *, assume_intent_in_scalars: bool): def test_conservative_default_returns_every_undeclared_scalar(tmp_path: Path): + """Primitive and character scalars without intent share one conservative default.""" module = _module(tmp_path, assume_intent_in_scalars=False) values = np.array([1.0, 2.0, 3.0], dtype=np.float64) @@ -37,39 +38,24 @@ def test_conservative_default_returns_every_undeclared_scalar(tmp_path: Path): np.int32(3), np.float64(2.0), ) + assert module.label_width("abcd") == (np.int32(4), "abcd") + +def test_assumed_scalar_intent_drops_only_undeclared_scalar_results(tmp_path: Path): + """The option reaches undeclared scalars, primitive and character alike. -def test_assumed_scalar_intent_returns_only_the_function_result(tmp_path: Path): + A declared intent keeps its replacement result, and arrays and derived + objects keep writing back in place. + """ module = _module(tmp_path, assume_intent_in_scalars=True) values = np.array([1.0, 2.0, 3.0], dtype=np.float64) assert module.weighted(np.int32(3), values, np.float64(2.0)) == np.float64(12.0) + assert module.label_width("abcd") == np.int32(4) + assert module.declared(np.float64(4.0)) == np.float64(5.0) - -def test_assumed_scalar_intent_keeps_array_and_derived_writeback(tmp_path: Path): - module = _module(tmp_path, assume_intent_in_scalars=True) item = module.Sample(x=np.float64(1.0)) values = np.array([1.0, 2.0, 3.0], dtype=np.float64) - assert module.touch(np.int32(5), item, values) is None assert item.x == np.float64(2.0) np.testing.assert_array_equal(values, np.array([2.0, 4.0, 6.0])) - - -def test_undeclared_character_scalar_follows_the_same_conservative_default(tmp_path: Path): - """A character dummy with no intent is returned exactly like a primitive one.""" - module = _module(tmp_path, assume_intent_in_scalars=False) - - assert module.label_width("abcd") == (np.int32(4), "abcd") - - -def test_assumed_scalar_intent_also_drops_the_character_result(tmp_path: Path): - module = _module(tmp_path, assume_intent_in_scalars=True) - - assert module.label_width("abcd") == np.int32(4) - - -def test_assumed_scalar_intent_does_not_change_a_declared_intent(tmp_path: Path): - module = _module(tmp_path, assume_intent_in_scalars=True) - - assert module.declared(np.float64(4.0)) == np.float64(5.0) diff --git a/tests/fortran/subroutines/end_to_end/test_subroutine_direct_entrypoint_routing.py b/tests/fortran/subroutines/end_to_end/test_subroutine_direct_entrypoint_routing.py index 38e63c91c..2a1ee8a14 100644 --- a/tests/fortran/subroutines/end_to_end/test_subroutine_direct_entrypoint_routing.py +++ b/tests/fortran/subroutines/end_to_end/test_subroutine_direct_entrypoint_routing.py @@ -6,7 +6,6 @@ import pytest from tests.fortran._support.wrapper_build import ( - _build_inline_pyi_contract_module, _build_source_or_generated_pyi_and_import, ) @@ -65,23 +64,3 @@ def test_subroutine_mixed_route_matches_results_and_adapts_only_ordinary_operati ) assert "bind_c_adapted_outputs" in bridge assert "direct_outputs" not in bridge - - -def test_subroutine_mixed_route_matches_edited_source_free_contract(tmp_path: Path): - stem = "subroutines_mixed_bind_c_f90" - source = (FIXTURES / "native" / f"{stem}.f90").read_text(encoding="utf-8") - contract = (FIXTURES / "contracts" / stem / f"{stem}.pyi").read_text(encoding="utf-8") - contract = contract.replace("from prik.contracts import ", "from prik.contracts import nogil, ") - contract = contract.replace("def direct_outputs(", "@nogil\ndef direct_outputs(").replace( - "def adapted_outputs(", "@nogil\ndef adapted_outputs(" - ) - module, result = _build_inline_pyi_contract_module( - tmp_path, module_name=stem, source_text=source, contract_text=contract - ) - - expected = (np.int32(4), np.int32(6)) - assert module.direct_outputs(np.int32(3)) == expected - assert module.adapted_outputs(np.int32(3)) == expected - bridge = (result.output_dir / f"bind_c_{stem}_wrapper.f90").read_text(encoding="utf-8").casefold() - assert "bind_c_adapted_outputs" in bridge - assert "subroutine bind_c_direct_outputs" not in bridge diff --git a/tests/fortran/subroutines/policy/test_subroutine_output_policy.py b/tests/fortran/subroutines/policy/test_subroutine_output_policy.py index 54aee1ac7..bc835f966 100644 --- a/tests/fortran/subroutines/policy/test_subroutine_output_policy.py +++ b/tests/fortran/subroutines/policy/test_subroutine_output_policy.py @@ -1,43 +1,6 @@ -from pathlib import Path - - from tests.fortran._support.ownership_policy import parse_pyi_text -from tests.fortran._support.paths import FORTRAN_ROOT -from prik.parsers.fortran.parser import parse_fortran_project -from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules -from prik.preprocessing import PreprocessingConfig, read_fortran_source -from prik.semantics.fortran2ir import fortran_project_to_semantic_modules -from prik.policy.ownership import ( - NativeBarrierAction, -) from prik.policy.completion import complete_semantic_policies -from prik.policy.construction import ( - completed_function_wrapper_policy, -) - -FMATH_CONTRACT = Path("tests/fortran/data_types/end_to_end/fixtures/contracts/fmath/__init__.pyi") - - -CALLS_NATIVE = ( - FORTRAN_ROOT - / "infrastructure" - / "semantic_pyi" - / "contracts" - / "calls_and_results" - / "end_to_end" - / "fixtures" - / "native" -) - - -def _source_semantic_module(filename: str, *, module_name: str): - source = CALLS_NATIVE / filename - parsed = parse_fortran_project({str(source): read_fortran_source(source, PreprocessingConfig()).source}) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name=module_name) - complete_semantic_policies(module) - return module +from prik.policy.construction import completed_function_wrapper_policy def test_native_call_policy_maps_visible_positions_when_hidden_output_precedes_input(): @@ -59,14 +22,3 @@ def mapped_status(base: Int32) -> Int32: ... ("scalar_native_order.mapped_status.status", "result", 0), ("scalar_native_order.mapped_status.base", "projection", 1), ] - - -def test_source_hidden_scalar_output_completes_call_local_address_before_planning(): - module = _source_semantic_module("foutputs_f90.f90", module_name="foutputs_f90") - function = next(function for function in module.functions if function.name == "scalar_status") - policy = completed_function_wrapper_policy(function) - - hidden = policy.results[0] - assert hidden.source_kind == "hidden_output" - assert hidden.native_barrier_action is NativeBarrierAction.PASS_CALL_LOCAL_ADDRESS - assert policy.native_call_slots[1].native_barrier_action is hidden.native_barrier_action diff --git a/tests/fortran/subroutines/semantics/fixtures/native/assumed_intent.f90 b/tests/fortran/subroutines/semantics/fixtures/native/assumed_intent.f90 deleted file mode 100644 index 61ef8a430..000000000 --- a/tests/fortran/subroutines/semantics/fixtures/native/assumed_intent.f90 +++ /dev/null @@ -1,18 +0,0 @@ -module legacy - type :: pt - real(8) :: x = 0.0d0 - end type pt -contains -subroutine touch(count, item, values, label, declared) - integer(4) :: count - type(pt) :: item - real(8) :: values(:) - character(len=4) :: label - integer(4), intent(inout) :: declared - count = count + 1 - item%x = item%x + 1.0d0 - values = values * 2.0d0 - label = "zzzz" - declared = declared + 1 -end subroutine touch -end module legacy diff --git a/tests/fortran/subroutines/semantics/test_subroutine_argument_projection.py b/tests/fortran/subroutines/semantics/test_subroutine_argument_projection.py index d23ed4181..86fcdac8f 100644 --- a/tests/fortran/subroutines/semantics/test_subroutine_argument_projection.py +++ b/tests/fortran/subroutines/semantics/test_subroutine_argument_projection.py @@ -1,134 +1,58 @@ -"""Tests split by stable ownership concept from `test_compile_time_values.py`.""" +"""Fortran intent and object kind decide whether a dummy projects a replacement result.""" -from pathlib import Path +import pytest +from prik.parsers.fortran import parse_fortran_file as parse_fortran_source from prik.semantics.fortran2ir import fortran_module_to_semantic_module +from prik.semantics.metadata import PROJECTED_OUTPUT_METADATA from prik.semantics.models import ProjectionMapping from tests.fortran._support.semantic_conversion import get_function -from prik.semantics.metadata import PROJECTED_OUTPUT_METADATA -from prik.parsers.fortran import parse_fortran_file as parse_fortran_source - -NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" - - -def test_primitive_scalar_inout_stays_visible_and_projects_replacement_return(): - source = """ -module outputs -contains -subroutine scale_in_place(value, factor) - real(8), intent(inout) :: value - real(8), intent(in) :: factor - value = factor * value -end subroutine scale_in_place -end module outputs -""" - - smod = fortran_module_to_semantic_module(parse_fortran_source(source)) - scale = get_function(smod, "scale_in_place") - - assert scale.arguments[0].metadata[PROJECTED_OUTPUT_METADATA] is True - assert scale.projection == [ - ProjectionMapping( - python_name="value", - native_name="value", - native_position=0, - python_position=0, - result_position=0, - ), - ProjectionMapping( - python_name="factor", - native_name="factor", - native_position=1, - python_position=1, - ), - ] -def test_ordinary_array_output_stays_visible_without_result_projection(): - source = """ -module outputs -contains -subroutine fill(values) - real(8), intent(out) :: values(:) -end subroutine fill -end module outputs -""" - - smod = fortran_module_to_semantic_module(parse_fortran_source(source)) - fill = get_function(smod, "fill") - - assert PROJECTED_OUTPUT_METADATA not in fill.arguments[0].metadata - assert fill.projection == [ - ProjectionMapping( - python_name="values", - native_name="values", - native_position=0, - python_position=0, - ) - ] - - -def test_scalar_derived_output_stays_visible_without_result_projection(): - source = """ +@pytest.mark.parametrize( + ("declaration", "projects_result"), + [ + ("real(8), intent(inout) :: value", True), + ("real(8), intent(out) :: value(:)", False), + ("type(point), intent(out) :: value", False), + ], + ids=[ + "primitive-scalar-inout-returns-replacement", + "array-output-stays-caller-storage", + "derived-output-stays-caller-object", + ], +) +def test_output_dummies_stay_visible_and_only_primitive_scalars_project_a_result( + declaration: str, projects_result: bool +): + """A primitive scalar is immutable in Python, so its update returns as a result. + + Arrays and derived objects are caller-owned storage the native procedure + writes in place, so they stay visible arguments without a result slot. + """ + source = f""" module outputs type :: point real(8) :: x end type point contains -subroutine fill(value) - type(point), intent(out) :: value -end subroutine fill +subroutine update(value, factor) + {declaration} + real(8), intent(in) :: factor +end subroutine update end module outputs """ - smod = fortran_module_to_semantic_module(parse_fortran_source(source)) - fill = get_function(smod, "fill") + update = get_function(fortran_module_to_semantic_module(parse_fortran_source(source)), "update") - assert PROJECTED_OUTPUT_METADATA not in fill.arguments[0].metadata - assert fill.projection == [ + assert update.arguments[0].metadata.get(PROJECTED_OUTPUT_METADATA, False) is projects_result + assert update.projection == [ ProjectionMapping( python_name="value", native_name="value", native_position=0, python_position=0, - ) + result_position=0 if projects_result else None, + ), + ProjectionMapping(python_name="factor", native_name="factor", native_position=1, python_position=1), ] - - -ASSUMED_INTENT_SOURCE = (NATIVE_FIXTURES / "assumed_intent.f90").read_text(encoding="utf-8") - - -def _touch_result_names(*, assume_intent_in_scalars): - smod = fortran_module_to_semantic_module( - parse_fortran_source(ASSUMED_INTENT_SOURCE), - assume_intent_in_scalars=assume_intent_in_scalars, - ) - touch = get_function(smod, "touch") - return [mapping.native_name for mapping in touch.projection if mapping.result_position is not None] - - -def test_undeclared_intent_scalar_projects_a_replacement_result_by_default(): - """Primitive and character scalars share one conservative default.""" - assert _touch_result_names(assume_intent_in_scalars=False) == ["count", "label", "declared"] - - -def test_assumed_scalar_intent_drops_only_the_undeclared_scalar_results(): - """The assumption reaches undeclared scalars, primitive and character alike. - - A declared ``intent(inout)`` scalar keeps its replacement result, and - arrays and derived-type objects were never projected as results, so their - in-place contract is unchanged either way. - """ - assert _touch_result_names(assume_intent_in_scalars=True) == ["declared"] - - -def test_assumed_scalar_intent_leaves_undeclared_non_scalars_writable(): - smod = fortran_module_to_semantic_module( - parse_fortran_source(ASSUMED_INTENT_SOURCE), - assume_intent_in_scalars=True, - ) - arguments = {argument.name: argument for argument in get_function(smod, "touch").arguments} - - assert arguments["count"].semantic_type.ownership.mutable is False - assert arguments["item"].semantic_type.ownership.mutable is True - assert arguments["values"].semantic_type.ownership.mutable is True diff --git a/tests/tools/test_build_time_benchmark.py b/tests/tools/test_build_time_benchmark.py index 946fccc5c..1903efba2 100644 --- a/tests/tools/test_build_time_benchmark.py +++ b/tests/tools/test_build_time_benchmark.py @@ -65,20 +65,6 @@ def test_build_commands_use_the_same_sources_and_complete_optimization_flags(tmp assert "--opt=-O0" in f2py -def test_tool_order_alternates_between_rounds() -> None: - assert build_time.tool_order("prik", 0) == ("prik", "f2py") - assert build_time.tool_order("prik", 1) == ("f2py", "prik") - assert build_time.tool_order("f2py", 0) == ("f2py", "prik") - assert build_time.tool_order("f2py", 1) == ("prik", "f2py") - - -def test_build_benchmark_defaults_to_four_measured_rounds() -> None: - args = build_time.parse_args([]) - - assert args.runs == 4 - assert args.warmups == 1 - - def test_timed_build_excludes_post_build_import_verification(tmp_path: Path, monkeypatch) -> None: workload = build_time.BuildWorkload("test", (), (), ()) case = build_time.BuildCase(build_time.BUILD_PROFILES[0], workload) diff --git a/tests/tools/test_check_codegen_complexity_cli.py b/tests/tools/test_check_codegen_complexity_cli.py index f680cbfbd..34ca2724f 100644 --- a/tests/tools/test_check_codegen_complexity_cli.py +++ b/tests/tools/test_check_codegen_complexity_cli.py @@ -15,7 +15,7 @@ def _recommendation() -> WrapperCodegenViolation: ) -def test_codegen_review_is_advisory_by_default(monkeypatch, capsys) -> None: +def test_codegen_review_is_advisory_unless_strict(monkeypatch, capsys) -> None: monkeypatch.setattr(check_codegen_complexity, "check_codegen_package", lambda: (_recommendation(),)) assert check_codegen_complexity.main([]) == 0 @@ -23,8 +23,4 @@ def test_codegen_review_is_advisory_by_default(monkeypatch, capsys) -> None: assert "example recommendation" in output assert "advisory codegen recommendation" in output - -def test_codegen_review_can_be_requested_as_strict(monkeypatch) -> None: - monkeypatch.setattr(check_codegen_complexity, "check_codegen_package", lambda: (_recommendation(),)) - assert check_codegen_complexity.main(["--strict"]) == 1 diff --git a/tests/tools/test_check_static_analysis_versions.py b/tests/tools/test_check_static_analysis_versions.py index 2e151b419..bb6875a80 100644 --- a/tests/tools/test_check_static_analysis_versions.py +++ b/tests/tools/test_check_static_analysis_versions.py @@ -6,28 +6,19 @@ ) -def test_static_analysis_version_errors_reports_missing_and_mismatched_tools(): - installed = { - "bandit": None, - "radon": "6.0.1", - "ruff": "0.11.7", - "vulture": "2.16", - } +def test_static_analysis_version_errors_accept_exact_pins_and_report_drift(): + assert static_analysis_version_errors(EXPECTED_STATIC_ANALYSIS_VERSIONS) == [] + installed = {**EXPECTED_STATIC_ANALYSIS_VERSIONS, "bandit": None, "ruff": "0.0.1"} assert static_analysis_version_errors(installed) == [ - "bandit: not installed, expected 1.9.4", - "ruff: installed 0.11.7, expected 0.15.17", + f"bandit: not installed, expected {EXPECTED_STATIC_ANALYSIS_VERSIONS['bandit']}", + f"ruff: installed 0.0.1, expected {EXPECTED_STATIC_ANALYSIS_VERSIONS['ruff']}", ] -def test_static_analysis_version_errors_accepts_exact_pins(): - assert static_analysis_version_errors(EXPECTED_STATIC_ANALYSIS_VERSIONS) == [] - - def test_static_analysis_version_pins_match_qa_extra(): pyproject = Path("pyproject.toml").read_text(encoding="utf-8") + extras = {"bandit": "bandit[toml]", "radon": "radon[toml]"} - assert '"bandit[toml]==1.9.4"' in pyproject - assert '"radon[toml]==6.0.1"' in pyproject - assert '"ruff==0.15.17"' in pyproject - assert '"vulture==2.16"' in pyproject + for package, expected in EXPECTED_STATIC_ANALYSIS_VERSIONS.items(): + assert f'"{extras.get(package, package)}=={expected}"' in pyproject, f"{package} is not pinned to {expected}" diff --git a/tests/tools/test_direct_entrypoint_benchmark.py b/tests/tools/test_direct_entrypoint_benchmark.py index 5503d9896..fc981f27f 100644 --- a/tests/tools/test_direct_entrypoint_benchmark.py +++ b/tests/tools/test_direct_entrypoint_benchmark.py @@ -246,33 +246,3 @@ def timeit(self, name: str, **_kwargs) -> None: "direct.call.scalar_function", "direct.call.scalar_subroutine", ] - - -def test_run_and_workflows_keep_direct_results_out_of_default_population() -> None: - run_script = Path("benchmarks/run.sh").read_text(encoding="utf-8") - generator = Path("tools/generate_performance_docs.py").read_text(encoding="utf-8") - workflows = "\n".join( - Path(path).read_text(encoding="utf-8") - for path in (".github/workflows/docs.yml", ".github/workflows/merge-validation.yml") - ) - - assert "python3 direct_preflight.py" in run_script - assert "python3 direct_build_time.py" in run_script - assert "direct_runtime.py" in run_script - assert '--output "results/$binding_tool.json"' in run_script - assert '--output "results/$direct_route.json"' in run_script - assert "prik-adapted.json" in run_script - assert "direct_runtime_passes=(forward reverse)" in run_script - assert "direct_routes=(prik-adapted f2py-direct prik-direct)" in run_script - assert "direct" not in generator.partition("DEFAULT_F2PY_RESULTS")[0] - assert workflows.count("name: direct-entrypoint-preflight") == 2 - assert workflows.count("path: benchmarks/build/direct-runtime") == 2 - for name in ( - "f2py-direct.json", - "prik-direct.json", - "prik-adapted.json", - "f2py-direct-build.json", - "prik-direct-build.json", - "prik-adapted-build.json", - ): - assert workflows.count(name) == 4 diff --git a/tests/tools/test_generate_performance_docs.py b/tests/tools/test_generate_performance_docs.py index 2b6491b92..e30394989 100644 --- a/tests/tools/test_generate_performance_docs.py +++ b/tests/tools/test_generate_performance_docs.py @@ -16,8 +16,6 @@ _load_direct_snapshots, generate, load_snapshot, - render_build_chart, - render_chart, render_page, ) @@ -323,48 +321,6 @@ def test_render_page_rejects_missing_or_duplicate_markers(tmp_path: Path) -> Non ) -def test_render_chart_is_valid_accessible_svg(tmp_path: Path) -> None: - f2py, prik = _paired_suites(tmp_path) - snapshot = load_snapshot( - f2py, - prik, - operating_system=TEST_OS, - compiler_version="GNU Fortran 13.3.0", - commit="1234567890abcdef", - ) - - chart = render_chart(snapshot) - root = ElementTree.fromstring(chart) - - assert root.attrib["role"] == "img" - assert root.attrib["aria-labelledby"] == "title description" - assert "PRIK performance relative to f2py" in chart - assert "no significant difference" in chart - assert "Geometric mean:" in chart - - -def test_render_build_chart_is_valid_accessible_svg(tmp_path: Path) -> None: - f2py, prik = _paired_build_suites(tmp_path) - snapshot = load_snapshot( - f2py, - prik, - operating_system=TEST_OS, - compiler_version="GNU Fortran 13.3.0", - commit="1234567890abcdef", - metadata_keys=BUILD_SHARED_METADATA, - ) - - chart = render_build_chart(snapshot) - root = ElementTree.fromstring(chart) - - assert root.attrib["role"] == "img" - assert root.attrib["aria-labelledby"] == "build-title build-description" - assert "Clean build time for PRIK and f2py" in chart - assert "Development · small module" in chart - assert "Optimized · full reference BLAS" in chart - assert "lower is better" in chart - - def test_load_snapshot_rejects_incompatible_platforms(tmp_path: Path) -> None: f2py, _prik = _paired_suites(tmp_path) prik = _write_suite( @@ -459,10 +415,23 @@ def test_generate_writes_page_and_chart(tmp_path: Path) -> None: ) assert "August 2, 2026" in page.read_text(encoding="utf-8") - assert chart.is_file() - assert build_chart.is_file() - ElementTree.parse(chart) - ElementTree.parse(build_chart) + + runtime_svg = chart.read_text(encoding="utf-8") + runtime_root = ElementTree.fromstring(runtime_svg) + assert runtime_root.attrib["role"] == "img" + assert runtime_root.attrib["aria-labelledby"] == "title description" + assert "PRIK performance relative to f2py" in runtime_svg + assert "no significant difference" in runtime_svg + assert "Geometric mean:" in runtime_svg + + build_svg = build_chart.read_text(encoding="utf-8") + build_root = ElementTree.fromstring(build_svg) + assert build_root.attrib["role"] == "img" + assert build_root.attrib["aria-labelledby"] == "build-title build-description" + assert "Clean build time for PRIK and f2py" in build_svg + assert "Development · small module" in build_svg + assert "Optimized · full reference BLAS" in build_svg + assert "lower is better" in build_svg def test_current_performance_page_has_one_complete_marker_pair_per_generated_block() -> None: @@ -471,9 +440,3 @@ def test_current_performance_page_has_one_complete_marker_pair_per_generated_blo for name in ("summary", "table", "direct", "build", "direct-build", "environment"): assert page.count(f"") == 1 assert page.count(f"") == 1 - - -def test_pyperf_is_pinned_for_documentation_and_generator_tests() -> None: - pyproject = Path("pyproject.toml").read_text(encoding="utf-8") - - assert pyproject.count('"pyperf==2.10.0"') == 2 diff --git a/tests/tools/test_runtime_benchmark.py b/tests/tools/test_runtime_benchmark.py index 835c4a4c7..524288780 100644 --- a/tests/tools/test_runtime_benchmark.py +++ b/tests/tools/test_runtime_benchmark.py @@ -14,30 +14,18 @@ RUNTIME_SCRIPT = Path("benchmarks/runtime.py") -@pytest.mark.parametrize( - ("group", "processes", "values", "expected_names"), - [ - ("calls", 16, 4, ("call.noop", "call.add_scalars")), - ("vector-latency", 16, 4, ("array.increment_vector.n=1", "array.increment_vector.n=16")), - ("vector-bulk", 4, 3, ("array.increment_vector.n=1024", "array.increment_vector.n=1000000")), - ("matrix-sum-latency", 16, 4, ("matrix.sum.4x4.order=F",)), - ( - "matrix-sum-bulk", - 2, - 3, - ("matrix.sum.32x32.order=F", "matrix.sum.256x256.order=F", "matrix.sum.1024x1024.order=F"), - ), - ("matrix-update-latency", 16, 4, ("matrix.update.4x4.order=F", "matrix.update.256x256.order=F")), - ("matrix-update-bulk", 8, 3, ("matrix.update.1024x1024.order=F",)), - ], +RUNTIME_GROUPS = ( + "calls", + "vector-latency", + "vector-bulk", + "matrix-sum-latency", + "matrix-sum-bulk", + "matrix-update-latency", + "matrix-update-bulk", ) -def test_runtime_groups_assign_more_samples_only_to_noisy_cases( - monkeypatch: pytest.MonkeyPatch, - group: str, - processes: int, - values: int, - expected_names: tuple[str, ...], -) -> None: + + +def _run_runtime_group(monkeypatch: pytest.MonkeyPatch, group: str) -> dict[str, object]: observed: dict[str, object] = {"names": []} class FakeRunner: @@ -62,16 +50,28 @@ def timeit(self, name: str, **_kwargs) -> None: monkeypatch.setattr(pyperf, "Runner", FakeRunner) runpy.run_path(RUNTIME_SCRIPT, run_name="__main__") + return observed + - assert observed["processes"] == processes - assert observed["values"] == values - assert observed["metadata"]["cpu_model_name"] == "Published Benchmark CPU" - assert observed["metadata"]["runtime_order_pass"] == "prik-first" - assert observed["metadata"]["runtime_order_protocol"] == "balanced_ab_ba" - assert observed["names"] == list(expected_names) +def test_runtime_groups_partition_all_cases_and_sample_latency_cases_more(monkeypatch: pytest.MonkeyPatch) -> None: + every_case = _run_runtime_group(monkeypatch, "all")["names"] + groups = {group: _run_runtime_group(monkeypatch, group) for group in RUNTIME_GROUPS} + # The reduced per-group budget still measures every public case exactly once, in table order. + assert [name for group in RUNTIME_GROUPS for name in groups[group]["names"]] == every_case + for observed in groups.values(): + assert observed["metadata"]["cpu_model_name"] == "Published Benchmark CPU" + assert observed["metadata"]["runtime_order_pass"] == "prik-first" + assert observed["metadata"]["runtime_order_protocol"] == "balanced_ab_ba" -def test_run_script_balances_reduced_runtime_budget_in_public_table_order() -> None: + # Nanosecond-scale latency cases are the noisy ones and get more processes and values than bulk cases. + samples = {group: observed["processes"] * observed["values"] for group, observed in groups.items()} + latency = [samples[group] for group in RUNTIME_GROUPS if group == "calls" or group.endswith("-latency")] + bulk = [samples[group] for group in RUNTIME_GROUPS if group.endswith("-bulk")] + assert min(latency) > max(bulk) + + +def test_run_script_balances_order_passes_and_keeps_direct_results_separate() -> None: source = Path("benchmarks/run.sh").read_text(encoding="utf-8") positions = [ @@ -100,6 +100,15 @@ def test_run_script_balances_reduced_runtime_budget_in_public_table_order() -> N assert "PRIK_BUILD_BENCHMARK_RUNS:-4" in source assert "PRIK_BENCHMARK_CPU_MODEL" in source + # The direct-entrypoint cohort is preflighted, runs its own balanced passes, + # and merges into per-route results that never join the default population. + assert "python3 direct_preflight.py" in source + assert "python3 direct_build_time.py" in source + assert "direct_runtime.py" in source + assert "direct_runtime_passes=(forward reverse)" in source + assert "direct_routes=(prik-adapted f2py-direct prik-direct)" in source + assert '--output "results/$direct_route.json"' in source + def test_pyperf_merge_preserves_both_runtime_order_passes(tmp_path: Path) -> None: pass_paths = [] diff --git a/tests/tools/test_warm_real_library_native_cache.py b/tests/tools/test_warm_real_library_native_cache.py index 62ccf0041..f0ff0cc1f 100644 --- a/tests/tools/test_warm_real_library_native_cache.py +++ b/tests/tools/test_warm_real_library_native_cache.py @@ -1,10 +1,12 @@ from pathlib import Path from types import SimpleNamespace +import pytest + from tools import warm_real_library_native_cache -def test_warm_real_library_native_cache_defaults_to_all_libraries(monkeypatch, capsys): +def test_warm_real_library_native_cache_builds_selected_or_all_libraries(monkeypatch, capsys): calls = [] def build_reference_library(library: str): @@ -21,7 +23,6 @@ def build_reference_library(library: str): ) assert warm_real_library_native_cache.main([]) == 0 - assert calls == ["blas", "lapack"] assert capsys.readouterr().out.splitlines() == [ "native cache root: /cache", @@ -29,45 +30,15 @@ def build_reference_library(library: str): "lapack: /cache/libprik_full_lapack.so", ] - -def test_warm_real_library_native_cache_accepts_selected_libraries(monkeypatch, capsys): - calls = [] - - def build_reference_library(library: str): - calls.append(library) - return SimpleNamespace(shared_library=Path("/cache") / f"libprik_full_{library}.so") - - monkeypatch.setattr( - warm_real_library_native_cache, - "_native_library_module", - lambda: SimpleNamespace( - native_cache_root=lambda: Path("/cache"), - build_reference_library=build_reference_library, - ), - ) - + calls.clear() assert warm_real_library_native_cache.main(["lapack"]) == 0 - assert calls == ["lapack"] assert capsys.readouterr().out.splitlines() == [ "native cache root: /cache", "lapack: /cache/libprik_full_lapack.so", ] - -def test_warm_real_library_native_cache_rejects_unknown_library(monkeypatch): - monkeypatch.setattr( - warm_real_library_native_cache, - "_native_library_module", - lambda: SimpleNamespace( - native_cache_root=lambda: Path("/cache"), - build_reference_library=lambda library: SimpleNamespace(shared_library=Path("/cache") / library), - ), - ) - - try: + with pytest.raises(SystemExit) as stopped: warm_real_library_native_cache.main(["unknown"]) - except SystemExit as exc: - assert exc.code == 2 - else: - raise AssertionError("Expected invalid library to stop argument parsing") + assert stopped.value.code == 2 + assert calls == ["lapack"] diff --git a/tests/workflows/test_openmpi_integration.py b/tests/workflows/test_openmpi_integration.py index 29b2a2d21..a2944437d 100644 --- a/tests/workflows/test_openmpi_integration.py +++ b/tests/workflows/test_openmpi_integration.py @@ -69,22 +69,27 @@ def _info(text: str) -> str: return f"{shutil.which('cat')} <<'EOF'\n{text}EOF" +# Each unusable-helper reason is one row; the rows alternate between a local run +# (skip) and a provisioned run (fail) so both outcomes stay covered. @pytest.mark.parametrize( - ("ompi_info", "reason"), + ("ompi_info", "reason", "required"), [ - pytest.param(None, "ompi_info is unavailable", id="missing"), - pytest.param("exit 3", "ompi_info is unavailable", id="failing"), + pytest.param(None, "ompi_info is unavailable", False, id="local-missing"), + pytest.param("exit 3", "ompi_info is unavailable", True, id="required-failing"), pytest.param( - _info(INFO.replace("config:host:buildhost\n", "")), "configure host is not recorded", id="incomplete" + _info(INFO.replace("config:host:buildhost\n", "")), + "configure host is not recorded", + False, + id="local-incomplete", ), pytest.param( _info(INFO.replace("bindings:use_mpi_f08:yes\n", "")), "does not provide the mpi_f08 module", - id="without-mpi-f08", + True, + id="required-without-mpi-f08", ), ], ) -@pytest.mark.parametrize("required", [False, True], ids=["local", "required"]) def test_an_unusable_open_mpi_helper_skips_locally_and_fails_when_required( tmp_path: Path, monkeypatch, ompi_info: str | None, reason: str, required: bool ): From df0f9c729df70acb034856008ec7446ee8deec61 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 19:36:26 +0100 Subject: [PATCH 38/47] codex: retain compiler probe artifact isolation test --- .../compiling/test_support_probe_artifacts.py | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 tests/fortran/infrastructure/building/compiling/test_support_probe_artifacts.py diff --git a/tests/fortran/infrastructure/building/compiling/test_support_probe_artifacts.py b/tests/fortran/infrastructure/building/compiling/test_support_probe_artifacts.py new file mode 100644 index 000000000..5ae0e67d0 --- /dev/null +++ b/tests/fortran/infrastructure/building/compiling/test_support_probe_artifacts.py @@ -0,0 +1,19 @@ +"""Keep compiler capability probes isolated from the invoking test session.""" + +from pathlib import Path + +from tests.fortran._support import wrapper_build + + +def test_allocatable_result_probe_keeps_fortran_modules_out_of_invocation_directory( + tmp_path: Path, + monkeypatch, +) -> None: + invocation_dir = tmp_path / "invocation" + invocation_dir.mkdir() + monkeypatch.chdir(invocation_dir) + wrapper_build._supports_maybe_unallocated_function_result.cache_clear() + + wrapper_build._supports_maybe_unallocated_function_result() + + assert not tuple(invocation_dir.glob("*.mod")) From 2519d541bd94da30b3c4a9cda23990c9f6abf9c9 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 20:06:06 +0100 Subject: [PATCH 39/47] codex: remove redundant module array lowering checks --- .../test_primitive_scalar_type_catalogue.py | 14 +-- .../end_to_end/test_source_build_modes.py | 18 +-- .../test_module_array_view_lowering.py | 107 +----------------- 3 files changed, 7 insertions(+), 132 deletions(-) diff --git a/tests/fortran/data_types/codegen/test_primitive_scalar_type_catalogue.py b/tests/fortran/data_types/codegen/test_primitive_scalar_type_catalogue.py index 91899872a..22bb77e4c 100644 --- a/tests/fortran/data_types/codegen/test_primitive_scalar_type_catalogue.py +++ b/tests/fortran/data_types/codegen/test_primitive_scalar_type_catalogue.py @@ -1,11 +1,8 @@ -"""Readable NumPy projection and primitive backend catalogue invariants.""" +"""Readable NumPy projection for resolved primitive semantic types.""" import pytest -from prik.codegen.primitive_scalar_types import ( - NumpyDtypeRegistry, - PrimitiveScalarTypeRegistry, -) +from prik.codegen.primitive_scalar_types import NumpyDtypeRegistry def test_numpy_projection_catalogue_uses_resolved_semantic_names(): @@ -23,10 +20,3 @@ def test_numpy_projection_rejects_unresolved_and_unknown_semantic_dtypes(): with pytest.raises(KeyError, match="No NumPy dtype mapping for semantic dtype 'Int'"): NumpyDtypeRegistry.expression_for("Int") - - -def test_backend_catalogue_returns_detached_records(): - scalar = PrimitiveScalarTypeRegistry.type_for("Int32") - scalar.c_spelling = "changed" - - assert PrimitiveScalarTypeRegistry.type_for("Int32").c_spelling == "int32_t" diff --git a/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py b/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py index 07548d97e..6d78b2093 100644 --- a/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py +++ b/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py @@ -69,20 +69,10 @@ def test_verbose_mode_prints_full_direct_build_commands(tmp_path: Path): native_object = tmp_path / "verbose_api.o" bridge_object = tmp_path / "bind_c_verbose_api_wrapper.o" binding_object = tmp_path / "verbose_api_wrapper.o" - assert step_lines[:4] == [ - "Complete wrapper policies", - "Generate binding source", - "Generate bridge source", - "Generate binding header", - ] - binding_generation = command_lines.index(">> Generate binding source") - bridge_generation = command_lines.index(">> Generate bridge source") - header_generation = command_lines.index(">> Generate binding header") - assert bridge_generation == binding_generation + 2 - assert header_generation == bridge_generation + 2 - assert command_lines[binding_generation + 1].startswith(">> Timing: ") - assert command_lines[bridge_generation + 1].startswith(">> Timing: ") - assert command_lines[header_generation + 1].startswith(">> Timing: ") + assert "Complete wrapper policies" in step_lines + assert "Generate binding source" in step_lines + assert "Generate bridge source" in step_lines + assert "Generate binding header" in step_lines assert f"Compile native source: {source} -> {native_object}" in step_lines assert f"Write bridge source: {bridge_source}" in step_lines assert f"Write binding source: {binding_source}" in step_lines diff --git a/tests/fortran/modules/codegen/test_module_array_view_lowering.py b/tests/fortran/modules/codegen/test_module_array_view_lowering.py index cfadb5321..15d8b5f1e 100644 --- a/tests/fortran/modules/codegen/test_module_array_view_lowering.py +++ b/tests/fortran/modules/codegen/test_module_array_view_lowering.py @@ -1,4 +1,4 @@ -"""Bridge lowering for the two fixed module-array address mechanisms.""" +"""Module-array plan validation and character descriptor lowering.""" from __future__ import annotations @@ -6,7 +6,6 @@ import pytest -from prik.codegen.c.binding import CBindingGenerator from prik.codegen.fortran.bridge import FortranBridgeGenerator from prik.parsers.fortran.parser import parse_fortran_project from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules @@ -18,17 +17,6 @@ from tests.fortran._support.ownership_policy import parse_pyi_text -MODULE_ARRAY_SOURCE = """ -module array_state - use iso_fortran_env, only: int32, real64 - implicit none - real(real64) :: plain(2, 3) - integer(int32) :: counts(3) - character(len=5) :: labels(2) - real(real64), target :: addressable(4) -end module array_state -""" - DESCRIPTOR_CHARACTER_SOURCE = """ module char_state implicit none @@ -40,99 +28,6 @@ """ -def _plan(): - parsed = parse_fortran_project({"array_state.f90": MODULE_ARRAY_SOURCE}) - modules = fortran_project_to_semantic_modules(parsed) - _apply_source_python_exports(modules) - module = _merge_wrapper_modules(modules, name="array_state") - complete_semantic_policies(module) - return WrapperPlanner().build(module) - - -def _bridge_module(): - return FortranBridgeGenerator().visit(_plan()) - - -def _lowered_getters(): - plan = _plan() - bridge = FortranBridgeGenerator() - bridge.visit(plan) - printer = FortranSourcePrinter() - return {variable.bridge.native_name: printer.visit(bridge.visit(variable)[0]) for variable in plan.variables} - - -def test_addressable_module_array_takes_its_address_directly(): - """A `target` declaration lets `c_loc` name the array, so nothing else is emitted.""" - getter = _lowered_getters()["addressable"] - - assert "c_loc(native_addressable)" in getter - assert "capture_array_address" not in getter - - -@pytest.mark.parametrize("python_name", ["plain", "labels"]) -def test_ordinary_module_array_captures_its_address_in_c(python_name): - """Without `target`, the address is taken on the C side, never by `c_loc`. - - `c_loc` requires the variable it names to be a target, so an ordinary - declaration has no Fortran route to its own address. The getter hands the - whole array to a `bind(C)` procedure instead: an assumed-type assumed-size - dummy is passed as the bare base address, so C receives where the module - variable lives. The bridge forms no pointer and claims no target, and one - interface covers every element type including character. - """ - getter = _lowered_getters()[python_name] - - assert f"prik_capture_address(native_{python_name})" in getter - assert "c_loc" not in getter - assert "target" not in getter - - -def test_captured_address_declares_one_typeless_c_interface(): - """One assumed-type interface serves every captured element type.""" - module = _bridge_module() - - interfaces = [procedure for interface in module.interfaces for procedure in interface.procedures] - captures = [procedure for procedure in interfaces if procedure.name == "prik_capture_address"] - assert len(captures) == 1 - assert captures[0].bind_name == "prik_capture_address" - assert captures[0].parameters[0].type_name == "type(*)" - assert captures[0].parameters[0].attributes == ("dimension(*)",) - - -def test_an_all_addressable_module_needs_no_capture_primitive(): - """A module whose arrays are all addressable declares no capture interface. - - Neither side pulls in the capture symbol: the bridge declares no interface - for it and the binding does not opt into the bundled definition. - """ - module = parse_pyi_text( - "addressable: Annotated[Float64[4], Aliased]\n", - module_name="array_state", - ) - complete_semantic_policies(module) - plan = WrapperPlanner().build(module) - emitted = FortranBridgeGenerator().visit(plan) - - names = [procedure.name for interface in emitted.interfaces for procedure in interface.procedures] - assert "prik_capture_address" not in names - binding = CBindingGenerator().binding_module(plan) - assert not any(define.name == "PRIK_BINDING_CAPTURE_ADDRESS" for define in binding.defines) - - -def test_binding_opts_into_the_bundled_capture_primitive(): - """The C side selects the runtime definition of the symbol the bridge calls. - - The helper lives in the bundled support header rather than in emitted code, - because it is a fixed ABI primitive rather than something a plan describes. - It needs external linkage for the bridge to call it, so the binding opts in - once per extension and the header defines it in that translation unit alone. - """ - binding = CBindingGenerator().binding_module(_plan()) - - assert any(define.name == "PRIK_BINDING_CAPTURE_ADDRESS" for define in binding.defines) - assert not any(function.name == "prik_capture_address" for function in binding.functions) - - def _undecided_plan(): """Return a module-array plan whose address mechanism policy never selected.""" module = parse_pyi_text("plain: Float64[3]\n", module_name="array_state") From b5025861db69b623ba7400af2348380e40e18e77 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 20:18:58 +0100 Subject: [PATCH 40/47] codex: consolidate Fortran scalar test evidence --- .../parsing/test_declarations_and_shapes.py | 59 ++----------------- .../pipeline/test_type_mapping_report.py | 42 ++++--------- .../probes/test_fortran_type_probes.py | 34 ++++------- 3 files changed, 30 insertions(+), 105 deletions(-) diff --git a/tests/fortran/data_types/parsing/test_declarations_and_shapes.py b/tests/fortran/data_types/parsing/test_declarations_and_shapes.py index 782575fab..1079c2637 100644 --- a/tests/fortran/data_types/parsing/test_declarations_and_shapes.py +++ b/tests/fortran/data_types/parsing/test_declarations_and_shapes.py @@ -1,4 +1,4 @@ -"""Tests split by stable ownership concept from `test_procedures_and_interfaces.py`.""" +"""Fortran scalar declarations, kinds, and parameter expressions.""" from pathlib import Path @@ -10,7 +10,6 @@ collect_project_procedure_signatures, parse_fortran_modules, ) -from prik.parsers.fortran.parser import FortranParser NATIVE_FIXTURES = Path(__file__).parent / "fixtures" / "native" @@ -96,17 +95,6 @@ def test_builtin_star_kind_declarations_preserve_all_intrinsic_kinds(): assert args["ch2"].character_length_syntax is True -def test_legacy_character_star_kind_sets_length_metadata_when_character_prefix_is_preparsed(): - parsed = FortranParser()._parse_declaration_left("character*8", parse_character_star=False) - - assert parsed is not None - declaration, attributes = parsed - assert declaration.base_type == "character" - assert declaration.kind == "8" - assert declaration.character_length_syntax is True - assert attributes == [] - - def test_kind_resolution_from_imported_module_across_files(): files = { "kinds.f90": """ @@ -162,29 +150,6 @@ def test_fixed_form_parameter_statement_after_typed_constants(): assert sigs[0].variables == {} -def test_duplicate_declaration_raises_error(): - code = """ -subroutine dup(x) - real :: x - integer :: x -end subroutine dup -""" - with pytest.raises(ValueError, match="Duplicate declaration"): - _ = parse_fortran_file(code, filename="dup.f90").procedures - - -def test_fixed_form_parameter_without_typed_declaration_raises_error(): - code = """ - subroutine cst(a) - implicit none - real a - parameter ( zero = 0.0e+0 ) - end -""" - with pytest.raises(ValueError, match="Unknown datatype for PARAMETER symbol"): - _ = parse_fortran_file(code, filename="legacy.f").procedures - - def test_fixed_form_parameter_without_typed_declaration_allowed_with_implicit_typing(): code = """ subroutine cst() @@ -318,8 +283,6 @@ def test_compiler_dependent_parameter_expressions_remain_symbolic_with_value_at_ } sig = collect_project_procedure_signatures(files)[0] assert sig.arguments[0].shape == ["1:ip"] - assert sig.arguments[0].shape == ["1:ip"] - assert sig.arguments[0].shape == ["1:ip"] def test_local_compiler_dependent_parameter_expressions_remain_symbolic_with_value(): @@ -331,8 +294,6 @@ def test_local_compiler_dependent_parameter_expressions_remain_symbolic_with_val """ sig = parse_fortran_file(code).procedures[0] assert sig.arguments[0].shape == ["1:ip"] - assert sig.arguments[0].shape == ["1:ip"] - assert sig.arguments[0].shape == ["1:ip"] def test_compile_time_parameter_expression_resolves_deep_dependency_chains(): @@ -352,8 +313,11 @@ def test_compile_time_parameter_expression_resolves_deep_dependency_chains(): end module dims_mod """ } - sig = collect_project_procedure_signatures(files)[0] - assert sig.arguments[0].shape[0].startswith("1:") + project = parse_fortran_project(files) + parameters = {variable.name: variable.value for variable in project.modules["dims_mod"].variables} + + assert parameters["n5"] == "6" + assert project.procedures["use_expr"].arguments[0].shape == ["1:n5"] def test_big_compile_time_expression_suite(): @@ -432,14 +396,3 @@ def test_unevaluated_module_parameter_keeps_symbolic_value_without_literal_value assert variables["rk"].value is None assert variables["rk"].symbolic_value == "selected_real_kind(12)" assert module.procedures[0].arguments[0].kind == "selected_real_kind(12)" - - -def test_star_kind_is_parsed_in_modern_fortran_file(): - code = """ -subroutine bad(x) - real*8 :: x -end subroutine bad -""" - proc = parse_fortran_file(code, filename="bad.f90").procedures[0] - assert proc.arguments[0].base_type == "real" - assert proc.arguments[0].kind == "8" diff --git a/tests/fortran/data_types/pipeline/test_type_mapping_report.py b/tests/fortran/data_types/pipeline/test_type_mapping_report.py index 18384cfda..f622cee64 100644 --- a/tests/fortran/data_types/pipeline/test_type_mapping_report.py +++ b/tests/fortran/data_types/pipeline/test_type_mapping_report.py @@ -34,11 +34,21 @@ def test_type_mapping_markdown_covers_target_native_semantic_and_numpy_types( compiler, native_header, representative, + tmp_path, + monkeypatch, + capsys, ): if shutil.which(compiler) is None: pytest.skip(f"{compiler} is required for the target-specific mapping report") - report = _mapping_markdown(language, compiler=compiler) + monkeypatch.chdir(tmp_path) + assert ( + type_mapping_report.main( + ["--language", language, "--compiler", compiler, "--cache-dir", str(tmp_path / "cache"), "--refresh"] + ) + == 0 + ) + report = capsys.readouterr().out assert report.startswith(f"Target profile: `{type_mapping_report.target_profile()}`") assert native_header in report @@ -84,26 +94,6 @@ def test_type_mapping_report_records_structured_measurements(language, compiler) assert str(entry["target_fact"]["bits"]) in entry["native_fact"] -def test_type_mapping_report_main_selects_language(monkeypatch, capsys): - monkeypatch.setattr( - type_mapping_report, - "c_type_mapping_report", - lambda *, compiler, compiler_args, **options: f"C:{compiler}:{','.join(compiler_args)}:{options['refresh']}", - ) - monkeypatch.setattr( - type_mapping_report, - "fortran_type_mapping_report", - lambda *, compiler, compiler_args, **options: f"F:{compiler}:{','.join(compiler_args)}:{options['refresh']}", - ) - monkeypatch.setattr(type_mapping_report, "type_mapping_markdown", lambda report: report) - - assert type_mapping_report.main(["--language", "c", "--compiler", "clang", "--compiler-arg=-m32", "--refresh"]) == 0 - assert capsys.readouterr().out == "C:clang:-m32:True\n" - - assert type_mapping_report.main(["--language", "fortran"]) == 0 - assert capsys.readouterr().out == "F:gfortran::False\n" - - def test_fortran_type_mapping_uses_compiler_dependent_defaults(): if shutil.which("gfortran") is None: pytest.skip("gfortran is required for the target-specific mapping report") @@ -128,6 +118,7 @@ def test_fortran_type_mapping_includes_legacy_and_modern_spellings(): assert "| `complex*8` | 64-bit storage | `Complex64` | `numpy.complex64` |" in report assert "| `double precision` | 64-bit storage | `Float64` | `numpy.float64` |" in report assert "| `double complex` | 128-bit storage | `Complex128` | `numpy.complex128` |" in report + assert "| `character(kind=c_char)` | 8-bit storage | `String` | `numpy.str_ / ABI bytes` |" in report assert "| `character*8` | 8-bit storage | `String` | `numpy.str_ / ABI bytes` |" in report @@ -138,15 +129,6 @@ def test_target_profile_normalizes_common_machine_names(monkeypatch): assert type_mapping_report.target_profile() == "linux-x86_64" -def test_character_mapping_fact_is_modeled_without_compiler_probe_metadata(): - semantic_type = type("SemanticType", (), {"metadata": {}})() - - fact = type_mapping_report._fortran_target_fact(semantic_type, ("character", "c_char")) - - assert fact == {"bits": 8} - assert type_mapping_report._fortran_fact_text(fact) == "8-bit storage" - - def test_expression_probe_markdown_renders_measured_values(): if shutil.which("gfortran") is None: pytest.skip("gfortran is required for the Fortran expression probe") diff --git a/tests/fortran/data_types/probes/test_fortran_type_probes.py b/tests/fortran/data_types/probes/test_fortran_type_probes.py index 97e765170..0d129da6f 100644 --- a/tests/fortran/data_types/probes/test_fortran_type_probes.py +++ b/tests/fortran/data_types/probes/test_fortran_type_probes.py @@ -19,7 +19,6 @@ FortranTypeProbeRecipe, FortranTypeProbeReport, FortranTypeProbeError, - _value_for_expression, build_fortran_type_probe_source, evaluate_fortran_type_facts, evaluate_fortran_type_requirements, @@ -216,20 +215,6 @@ def probe(config, expressions, *, runner=None): assert len(calls) == 3 -def test_fortran_type_probe_expressions_extracts_semantic_requirement_inputs(): - requirements = [ - {"code": "parameter_value", "symbol": "blank", "expression": " "}, - {"code": "parameter_value", "symbol": "rk", "expression": "selected_real_kind(12)"}, - {"code": "unsupported_kind", "symbol": "x", "expression": "selected_real_kind(12)"}, - {"code": "parameter_value", "symbol": "ik", "expression": "selected_int_kind(9)"}, - ] - - assert fortran_type_probe_expressions(requirements) == [ - "selected_real_kind(12)", - "selected_int_kind(9)", - ] - - def test_fortran_type_probe_report_resolves_only_matching_parameter_requirements(): report = FortranTypeProbeReport( values={"Selected_Real_Kind(12)": 8}, @@ -251,8 +236,6 @@ def test_fortran_type_probe_report_resolves_only_matching_parameter_requirements assert report.to_compile_time_values() == {"Selected_Real_Kind(12)": 8} assert report.to_compile_time_values(requirements)["rk"] == 8 assert "not_added" not in report.to_compile_time_values(requirements) - assert _value_for_expression({}, "not_present") is None - assert evaluate_fortran_type_requirements(PreprocessingConfig(mode="compiler"), []) == {} @pytest.mark.parametrize( @@ -502,7 +485,7 @@ def test_prik_semantics_cli_uses_compiler_dependent_kinds_and_collected_requirem assert semantic_types["legacy_value"]["metadata"]["fortran_type_fact_source"] == "legacy_star_storage" -def test_probe_skips_expressions_naming_project_symbols(): +def test_probe_collects_unique_resolvable_expressions(): """The probe program cannot `use` a module that has not been compiled yet. An expression naming a kind parameter declared elsewhere in the project is @@ -515,11 +498,18 @@ def test_probe_skips_expressions_naming_project_symbols(): assert not fortran_type_probe.probe_can_resolve_expression("wp") requirements = [ - {"expression": "real64"}, - {"expression": "storage_size(1_ip, kind=ip)"}, - {"expression": "selected_int_kind(9)"}, + {"code": "parameter_value", "symbol": "blank", "expression": " "}, + {"code": "parameter_value", "symbol": "rk", "expression": "real64"}, + {"code": "unsupported_kind", "symbol": "x", "expression": "selected_real_kind(12)"}, + {"code": "parameter_value", "symbol": "size", "expression": "storage_size(1_ip, kind=ip)"}, + {"code": "parameter_value", "symbol": "ik", "expression": "selected_int_kind(9)"}, + {"code": "parameter_value", "symbol": "ik_again", "expression": "selected_int_kind(9)"}, + ] + assert fortran_type_probe_expressions(requirements) == [ + "real64", + "selected_real_kind(12)", + "selected_int_kind(9)", ] - assert fortran_type_probe_expressions(requirements) == ["real64", "selected_int_kind(9)"] def test_probe_source_compiles_for_a_module_using_imported_kind_parameters(tmp_path): From aa4aba69d182512b6cdac4ce1be55fc690416099 Mon Sep 17 00:00:00 2001 From: said Date: Sat, 26 Sep 2026 20:26:03 +0100 Subject: [PATCH 41/47] codex: remove redundant C infrastructure tests --- CHANGELOG.md | 2 + prik/parsers/c/parser.py | 2 - tests/c/README.md | 1 - .../test_c_parser_developer_tutorial.py | 64 ------------------- .../parsing/test_c_model_serialization.py | 12 +--- .../parsing/test_c_parser_benchmark.py | 20 ------ .../pipeline/test_c_pyi_contract_fixtures.py | 10 --- 7 files changed, 3 insertions(+), 108 deletions(-) delete mode 100644 tests/c/infrastructure/execution_examples/test_c_parser_developer_tutorial.py delete mode 100644 tests/c/infrastructure/parsing/test_c_parser_benchmark.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 5f997b7f7..f6dd3dd1b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,8 @@ release tags add a leading `v` to the package version. - The test suite consolidates overlapping checks around compiled workflows and retains focused parser, semantic, diagnostic, and ABI boundary coverage; contributor guidance now favors observable behavior over implementation shape. + The parked C parser benchmark and redundant parser walkthroughs no longer + appear in test collection. - Generated bindings pass a wrapped derived-type object about twice as fast: they read its native storage through attribute names interned once instead diff --git a/prik/parsers/c/parser.py b/prik/parsers/c/parser.py index 1a5c0f833..4a7ed521b 100644 --- a/prik/parsers/c/parser.py +++ b/prik/parsers/c/parser.py @@ -60,8 +60,6 @@ It records includes as graph facts; it never follows includes to discover more parser inputs. -Executable walkthroughs live in -``tests/c/infrastructure/execution_examples/test_c_parser_developer_tutorial.py``. """ from __future__ import annotations diff --git a/tests/c/README.md b/tests/c/README.md index 47e87478d..bb089e205 100644 --- a/tests/c/README.md +++ b/tests/c/README.md @@ -34,7 +34,6 @@ The active owners are: | `infrastructure/preprocessing/` | C recipes, dependencies, mappings, execution, and diagnostics | | `infrastructure/semantic_ir/` | C parser-model conversion to semantic IR | | `infrastructure/semantic_pyi/` | C semantic `.pyi` conversion and source/generated-contract parity | -| `infrastructure/execution_examples/` | Executable C parser walkthroughs kept runnable as documentation | | `fixtures/native/` | C source and include inputs | | `fixtures/parser/` | C parser snapshots and update commands | | `fixtures/pyi/` | checked C generated-contract packages | diff --git a/tests/c/infrastructure/execution_examples/test_c_parser_developer_tutorial.py b/tests/c/infrastructure/execution_examples/test_c_parser_developer_tutorial.py deleted file mode 100644 index 28dbc6445..000000000 --- a/tests/c/infrastructure/execution_examples/test_c_parser_developer_tutorial.py +++ /dev/null @@ -1,64 +0,0 @@ -"""Executable walkthroughs for maintainers reading `c_parser.parser`. - -These tests intentionally expose the internal declaration gateway named in the -module-level parsing sketch. Behavioral coverage remains in the feature test -modules; this file keeps the parser's control flow easy to inspect. -""" - - -def test_tutorial_shared_declarator_backend_builds_layered_variable_type(): - from prik.parsers.c import CArray, CConst, CInt, CParser, CPointer - - parser = CParser() - specifiers, declarator = parser._split_declaration_specifiers("const int *values[4]") - name, type_, storage, function_specifiers, direct_function = parser._build_declared_type( - specifiers, - declarator, - ) - - assert (specifiers, declarator) == ("const int", "*values[4]") - assert name == "values" - assert storage == [] - assert function_specifiers == [] - assert direct_function is None - assert [type(component) for component in type_.components] == [CArray, CPointer, CInt] - assert type_.components[-1].qualifiers == [CConst()] - - -def test_tutorial_parse_file_dispatches_declaration_roles_through_one_model(): - from prik.parsers.c import CParser, CStruct - - parsed = CParser().parse_file( - """ -typedef int api_status; -struct request { int id; }; -api_status submit(struct request *request); -extern int request_count; -""", - filename="tutorial.h", - ) - - assert [typedef.name for typedef in parsed.typedefs] == ["api_status"] - assert [struct.name for struct in parsed.structs] == ["request"] - assert isinstance(parsed.functions[0].parameters[0].type.components[-1], CStruct) - assert [function.name for function in parsed.functions] == ["submit"] - assert [variable.name for variable in parsed.variables] == ["request_count"] - assert parsed.diagnostics == [] - - -def test_tutorial_preprocessed_input_reuses_parsing_and_remaps_locations(): - from prik.parsers.c import CParser - - parsed = CParser().parse_file( - '# 24 "include/api.h"\nint expanded_api(void);\n', - filename="generated.i", - ) - - function = parsed.functions[0] - assert parsed.preprocessing == "preprocessed" - assert parsed.preprocessed_source_path == "generated.i" - assert parsed.original_source_paths == ["include/api.h"] - assert function.name == "expanded_api" - assert function.origin == "preprocessed" - assert function.source_location.filename == "include/api.h" - assert function.source_location.line == 24 diff --git a/tests/c/infrastructure/parsing/test_c_model_serialization.py b/tests/c/infrastructure/parsing/test_c_model_serialization.py index 16f99ea66..d1e663d1c 100644 --- a/tests/c/infrastructure/parsing/test_c_model_serialization.py +++ b/tests/c/infrastructure/parsing/test_c_model_serialization.py @@ -1,14 +1,8 @@ -"""Minimal JSON-shape coverage for C parser model dataclasses.""" +"""C parser diagnostic rendering, including color and source locations.""" import prik.parsers.c.models as models -def _type_payload(model: str, **extra): - payload = {"model": model, "qualifiers": [], "source_text": ""} - payload.update(extra) - return payload - - def _make_parse_error(**kwargs): return models.CParseError("unexpected token", **kwargs) @@ -27,10 +21,6 @@ def test_c_parse_error_diagnostic_rendering_contract(monkeypatch): assert str(error) == plain assert error.format_diagnostic(color=False, debug=False) == plain assert error.format_diagnostic(color=True, debug=False) == colored - assert error.parser_file is not None - assert error.parser_line_number > 0 - assert error.parser_function is not None - monkeypatch.setenv("C_PARSER_DEBUG", "yes") assert "note: parser raised at" in error.format_diagnostic(color=False) colored_debug = error.format_diagnostic(color=True, debug=True) diff --git a/tests/c/infrastructure/parsing/test_c_parser_benchmark.py b/tests/c/infrastructure/parsing/test_c_parser_benchmark.py deleted file mode 100644 index 98dd6b347..000000000 --- a/tests/c/infrastructure/parsing/test_c_parser_benchmark.py +++ /dev/null @@ -1,20 +0,0 @@ -"""Representative C-parser performance benchmark.""" - -from __future__ import annotations - -import pytest - -from prik.parsers.c import parse_c_file - - -pytestmark = pytest.mark.skip(reason="Benchmarks are parked until benchmark adoption resumes.") - -_C_HEADER = "".join(f"int fn_{index}(int x_{index}, double y_{index});\n" for index in range(200)) - - -@pytest.mark.benchmark -def test_parse_representative_c_header(benchmark): - parsed = benchmark(parse_c_file, _C_HEADER, filename="benchmark.h") - - assert len(parsed.functions) == 200 - assert parsed.diagnostics == [] diff --git a/tests/c/infrastructure/semantic_pyi/pipeline/test_c_pyi_contract_fixtures.py b/tests/c/infrastructure/semantic_pyi/pipeline/test_c_pyi_contract_fixtures.py index 2ce84ac1d..1ea162b1c 100644 --- a/tests/c/infrastructure/semantic_pyi/pipeline/test_c_pyi_contract_fixtures.py +++ b/tests/c/infrastructure/semantic_pyi/pipeline/test_c_pyi_contract_fixtures.py @@ -17,16 +17,6 @@ C_FIXTURE_PROJECTS = iter_general_c_fixture_projects() -def test_c_pyi_fixtures_do_not_contain_unknown_types(): - unknown_fixtures = [ - str(path.relative_to(C_PYI_FIXTURE_DIR)) - for path in C_PYI_FIXTURE_DIR.rglob("*.pyi") - if "Unknown" in path.read_text(encoding="utf-8") - ] - - assert not unknown_fixtures, f"Unknown semantic types in C .pyi fixtures: {unknown_fixtures[:20]}" - - @pytest.mark.parametrize( ("project_key", "fixtures"), C_FIXTURE_PROJECTS, From d8246c5435b529fd031adcdb985c06b1643c0796 Mon Sep 17 00:00:00 2001 From: said Date: Sun, 27 Sep 2026 00:14:23 +0100 Subject: [PATCH 42/47] codex: point the C parser fixture README at surviving tests The cJSON corpus test and the C parser developer walkthrough were removed in the test consolidation; cJSON stays covered by the json project golden. Co-Authored-By: Claude Opus 5.5 --- tests/c/fixtures/parser/README.md | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/tests/c/fixtures/parser/README.md b/tests/c/fixtures/parser/README.md index 0f6779992..8d847f091 100644 --- a/tests/c/fixtures/parser/README.md +++ b/tests/c/fixtures/parser/README.md @@ -16,7 +16,7 @@ Guidelines: ## Active cJSON Regression `tests/c/fixtures/native/json/cJSON.h` and `cJSON.c` exercise the header, source and -project paths in `test_c_corpus.py`; a separately pinned copy with license and +project paths through the `json` project golden in `test_c_fixture_suite.py`; a separately pinned copy with license and source provenance remains documentation work rather than a disabled test. The exact compiler-preprocessed project snapshot is a Linux reference golden and is skipped on other platforms. Portable parser and preprocessing behavior @@ -29,17 +29,8 @@ Checked-in compatibility snapshots cover grouped projects from `tests/c/fixtures `tests/c/fixtures/native/nanosvg/`, plus top-level C inputs from `tests/c/fixtures/native/stb/`. They preserve the compiler-preprocessed Linux reference payload and historical JSON shape. Do not refresh them from macOS or another compiler/libc reference -environment. - -## Developer Walkthrough - -`test_c_parser_developer_tutorial.py` is an executable reading guide for -`prik/parsers/c/parser.py`. It shows the shared declaration/declarator gateway, the -`parse_file` routing of declaration roles, and the preprocessed linemarker -path without replacing the feature-focused test modules. - -`test_c_fixture_suite.py` keeps fixture grouping coverage and verifies that -representative macro-heavy fixtures fail clearly in raw mode. +environment. `test_c_fixture_suite.py` compares every grouped project with its +golden. ## Error Goldens From b048123800974e71afbb3b6f3a6fcbd70e8bee37 Mon Sep 17 00:00:00 2001 From: said Date: Sun, 27 Sep 2026 00:20:27 +0100 Subject: [PATCH 43/47] codex: read a character entity's *length as its length, not a dimension In `character(len=4) :: label*(8)` the parser took `(8)` as an array spec and dropped the length, and an entity copy for `label*8` kept the statement's length expression, which semantics reads first. The entity splitter now owns `name[(array-spec)][*char-length]` and the entity record carries its own length expression. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 3 + prik/parsers/fortran/parser.py | 72 +++++++++++-------- .../test_declaration_and_interface_edges.py | 30 +++++++- 3 files changed, 73 insertions(+), 32 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f6dd3dd1b..5e19947a9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,9 @@ release tags add a leading `v` to the package version. contributor guidance now favors observable behavior over implementation shape. The parked C parser benchmark and redundant parser walkthroughs no longer appear in test collection. +- A character entity's own length now wins over its statement's: in + `character(len=4) :: label*(8), code*8`, `label` and `code` are scalars of + length 8 rather than `label` becoming a four-character array of extent 8. - Generated bindings pass a wrapped derived-type object about twice as fast: they read its native storage through attribute names interned once instead diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index a1e85a7e1..cf00dfb59 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -337,6 +337,12 @@ class EnumUnit(SourceUnit): _DeclarationRole = Literal["procedure_symbol", "type_field", "module_variable"] +_PARENTHESIZED = r"\((?:[^()]|\([^()]*\))*\)" +_CHARACTER_ENTITY = re.compile( + rf"(?P[A-Za-z_]\w*)\s*(?P{_PARENTHESIZED})?" + rf"\s*\*\s*(?P{_PARENTHESIZED}|\d+|[A-Za-z_]\w*)" + rf"\s*(?P{_PARENTHESIZED})?" +) _DECLARATION_FLAG_FIELDS = MappingProxyType( { "optional": "optional", @@ -4494,11 +4500,14 @@ def _declaration_entities( entities: list[tuple[str, list[str], str | None, _Declaration]] = [] for entity in split_csv(right): declared_entity, initializer = split_declaration_assignment(entity) - raw_name, shape = self._var(declared_entity) + raw_name, shape, entity_length = self._var( + declared_entity, + character=declaration.base_type == "character", + ) if not raw_name: continue - entity_declaration = self._entity_declaration(raw_name, declaration) - normalized_name = self._normalize_declared_name(raw_name, entity_declaration) + entity_declaration = self._entity_declaration(entity_length, declaration) + normalized_name = self._normalize_declared_name(raw_name) if normalized_name: entities.append((normalized_name, shape, initializer, entity_declaration)) return entities @@ -4519,27 +4528,21 @@ def _record_declaration_visibility( symbols.append(var.name) @staticmethod - def _entity_declaration(raw_name: str, declaration: _Declaration) -> _Declaration: - """Return the effective declaration for one entity spelling. + def _entity_declaration(length: str | None, declaration: _Declaration) -> _Declaration: + """Return the effective declaration for one entity's character length. - Non-character entities and declarations without an entity-level star - return the shared record unchanged. For ``character`` entities such as - ``label*8``, a copied declaration records kind ``8`` without mutating - sibling entities from the same statement. + Entities without an entity-level length share the statement record. + For ``character(len=4) :: label*8``, a copied declaration records + length ``8`` without mutating sibling entities from the same statement. """ - if declaration.base_type != "character": - return declaration - match = re.search(r"\*\s*(\([^)]*\)|\*|[A-Za-z_]\w*|\d+)\s*$", raw_name) - if match is None: + if length is None: return declaration - length = match.group(1).strip() - if length.startswith("(") and length.endswith(")"): - length = length[1:-1].strip() return replace( declaration, kind=length, shape=list(declaration.shape), character_length_syntax=True, + character_length_expression=length, ) @staticmethod @@ -4634,15 +4637,9 @@ def _apply_declaration_attributes( declaration.rank = len(shape) @staticmethod - def _normalize_declared_name(name: str, declaration: _Declaration) -> str: - """Strip legacy entity-local spelling from a declared symbol name.""" - normalized_name = re.sub(r"^\*\s*[0-9]+\s*", "", name).strip() - if declaration.base_type == "character" and "*" in normalized_name: - # Legacy CHARACTER declarations may carry entity-local length - # specifiers (e.g. NAME*(*) or SUBNAM*6). Strip the `*len` - # suffix so symbol lookup matches procedure arguments. - normalized_name = normalized_name.split("*", 1)[0].strip() - return normalized_name + def _normalize_declared_name(name: str) -> str: + """Strip a legacy leading ``*len`` from a declared symbol name.""" + return re.sub(r"^\*\s*[0-9]+\s*", "", name).strip() @staticmethod def _strip_legacy_star_kind_prefix(left: str) -> str: @@ -4655,15 +4652,30 @@ def _strip_legacy_star_kind_prefix(left: str) -> str: ).strip() @staticmethod - def _var(entry: str): - """Split one declaration entity into its name and inline dimensions.""" + def _var(entry: str, *, character: bool = False) -> tuple[str, list[str], str | None]: + """Split one declaration entity into its name, inline dimensions, and length. + + A character entity is ``name[(array-spec)][*char-length]``, so in + ``label*(8)`` the parenthesized ``8`` is a length, not a dimension; the + legacy ``name*len(array-spec)`` order is accepted as well. The length is + returned without its parentheses, e.g. ``*`` for ``name*(*)``. + """ e, _initializer = split_declaration_assignment(entry) if not e: # pragma: no cover - split_csv omits empty declaration entities for valid declarations. - return "", [] + return "", [], None + if character: + match = _CHARACTER_ENTITY.fullmatch(e.strip()) + if match is not None and match.group("length") is not None: + length = match.group("length").strip() + if length.startswith("(") and length.endswith(")"): + length = length[1:-1].strip() + dims = match.group("dims") or match.group("late_dims") + shape = split_csv(dims.strip()[1:-1]) if dims else [] + return match.group("name").strip(), shape, length if "(" in e and e.endswith(")"): name = e[: e.find("(")].strip() - return name, split_csv(e[e.find("(") + 1 : -1]) - return e, [] + return name, split_csv(e[e.find("(") + 1 : -1]), None + return e, [], None @staticmethod def _apply_declaration(arg: FortranArgument, declaration: _Declaration, shape: list[str]) -> None: diff --git a/tests/fortran/infrastructure/parsing/test_declaration_and_interface_edges.py b/tests/fortran/infrastructure/parsing/test_declaration_and_interface_edges.py index fa4a5fdcc..282496767 100644 --- a/tests/fortran/infrastructure/parsing/test_declaration_and_interface_edges.py +++ b/tests/fortran/infrastructure/parsing/test_declaration_and_interface_edges.py @@ -1,5 +1,7 @@ """Declaration forms, compile-time kind and shape resolution, and type definitions.""" +import pytest + from prik.parsers.fortran import parse_fortran_file, parse_fortran_project from prik.parsers.fortran.scope import ScopeUses @@ -120,6 +122,7 @@ def test_module_specification_keeps_variables_and_skips_other_statements(): assert (variables["kept"].base_type, variables["values"].shape) == ("integer", ["2"]) assert (variables["wide"].base_type, variables["wide"].kind) == ("real", "8") # An entity's own character length does not leak into the next entity. + assert (variables["label"].kind, variables["label"].shape) == ("name_len", []) assert (variables["other"].kind, variables["other"].shape) == ("len=default_len", []) dtype = module.derived_types[0] assert [field.name for field in dtype.fields] == ["x", "y"] @@ -234,8 +237,31 @@ def test_local_parameter_chains_resolve_kinds_and_shapes(): assert [arg.kind for arg in modern.arguments] == ["8", "16"] assert [arg.shape for arg in legacy.arguments] == [["2"], ["3"]] - # A parameter defined through another resolves; an entity length stays on its entity. - assert [(arg.kind, arg.shape) for arg in dependent.arguments] == [("len=4", ["8"]), ("len=4", []), ("", ["4"])] + # A parameter defined through another resolves; an entity length is a length on its entity only. + assert [(arg.kind, arg.shape) for arg in dependent.arguments] == [("8", []), ("len=4", []), ("", ["4"])] + assert [arg.character_length_expression for arg in dependent.arguments[:2]] == ["8", "4"] + + +@pytest.mark.parametrize( + ("declaration", "expected"), + [ + pytest.param("character(len=4) :: x*8", ("8", []), id="literal-length-overrides-statement"), + pytest.param("character(len=4) :: x*(*)", ("*", []), id="assumed-length"), + pytest.param("character(len=4) :: x(3)*2", ("2", ["3"]), id="array-spec-then-length"), + pytest.param("character(len=4) :: x*2(3)", ("2", ["3"]), id="legacy-length-then-array-spec"), + pytest.param("character*3 :: x*5", ("5", []), id="legacy-statement-length"), + ], +) +def test_character_entity_length_is_a_length_not_a_dimension(declaration, expected): + """``name[(array-spec)][*char-length]`` gives the entity its own length.""" + procedure = parse_fortran_file( + f"subroutine sized(x)\n {declaration}\nend subroutine sized\n", + filename="entity_length.f90", + ).procedures[0] + (argument,) = procedure.arguments + + assert (argument.kind, argument.shape) == expected + assert argument.character_length_expression == expected[0] def test_local_compile_time_arithmetic_is_folded_for_shapes_and_parameters(): From 2a291ade2ad29f62b27e786290884b1392400337 Mon Sep 17 00:00:00 2001 From: said Date: Sun, 27 Sep 2026 00:37:00 +0100 Subject: [PATCH 44/47] codex: print an edited .pyi contract back as it was written Reprinting a loaded contract dropped a variable or field SourceName (the printer compared the Python name, not origin.native_name), dropped a class's @bind (the rename check was gated on Fortran-sourced classes rather than excluding C structs), and spelled a plain tuple return as an explicit @native_call. A loaded contract now states @native_call exactly when it wrote one, read from the loader's native_projection record. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 3 ++ prik/printers/pyi.py | 43 ++++++++++++++----- .../semantics/test_round_trip_properties.py | 12 ++++++ 3 files changed, 47 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5e19947a9..b9add7866 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,9 @@ release tags add a leading `v` to the package version. - A character entity's own length now wins over its statement's: in `character(len=4) :: label*(8), code*8`, `label` and `code` are scalars of length 8 rather than `label` becoming a four-character array of extent 8. +- Printing an edited `.pyi` contract again keeps what it stated: a + `SourceName(...)` on a module variable or class field, a class's `@bind(...)`, + and a plain `tuple[...]` return no longer gain an explicit `@native_call`. - Generated bindings pass a wrapped derived-type object about twice as fast: they read its native storage through attribute names interned once instead diff --git a/prik/printers/pyi.py b/prik/printers/pyi.py index a9b93f580..b816aa3cd 100644 --- a/prik/printers/pyi.py +++ b/prik/printers/pyi.py @@ -97,6 +97,8 @@ class _PyiEmissionContext: contract_aliases: dict[str, str] = field(default_factory=dict) contract_imports: set[str] = field(default_factory=set) public_namespace: tuple[str, ...] = () + # A contract read back from .pyi: its native layouts were stated there. + contract_loaded: bool = False def contract(self, name: str) -> str: """Return one local contract spelling and record its required import.""" @@ -177,6 +179,7 @@ def _emission_context(self, node) -> _PyiEmissionContext: if cls.origin.source_language == "fortran" and cls.origin.source_kind == "derived_type" ), contract_aliases=self._contract_aliases_for_module(node), + contract_loaded=bool(node.metadata.get(PYI_LOADED_METADATA)), ) @staticmethod @@ -445,11 +448,11 @@ def _visit_SemanticClass( decorators.append(f"@{context.contract('abstract')}") if self._class_uses_c_abi(cls): decorators.append(f'@{context.contract("native_abi")}("c")') - # Only a Fortran type states a separate native name here. A C struct - # keeps its native spelling -- `struct node` for `node` -- through its - # own representation rules, which state it without a decorator. + # A C struct keeps its native spelling -- `struct node` for `node` -- + # through its own representation rules, which state it without a + # decorator. Every other class states a separate native name here. if ( - cls.origin.source_language == "fortran" + cls.origin.source_language != "c" and cls.native_name and self._renames_native_entity(cls, cls.native_name, emitted_name) ): @@ -1045,7 +1048,7 @@ def _emit_data_member( self._annotation_target(name), variable, context, - original_name=variable.name if self._renames_native_entity(variable, variable.name, name) else None, + original_name=self._renamed_native_spelling(variable, name), ) def _emit_module_variable( @@ -1059,7 +1062,7 @@ def _emit_module_variable( self._annotation_target(name), arg, context, - original_name=arg.name if self._renames_native_entity(arg, arg.name, name) else None, + original_name=self._renamed_native_spelling(arg, name), ) @staticmethod @@ -1465,10 +1468,11 @@ def _constructor_argument( or self._python_literal_text(field.default_value) or "..." ) - if self._renames_native_entity(field, field.name, name): + native_spelling = self._renamed_native_spelling(field, name) + if native_spelling is not None: type_text = self._annotated_type_text( type_text, - [f"{context.contract('SourceName')}({json.dumps(field.name)})"], + [f"{context.contract('SourceName')}({json.dumps(native_spelling)})"], context, ) return f"{name}: {type_text} = {default_value}" @@ -1986,7 +1990,7 @@ def _decorators( decorators.append(f"{indent}@{context.contract('pure')}") overload_target = func.metadata.get(OVERLOAD_TARGET_METADATA) is_specific_declaration = not overload_target or str(func.name).casefold() == str(overload_target).casefold() - if is_specific_declaration and self._requires_native_call(func): + if is_specific_declaration and self._requires_native_call(func, contract_loaded=context.contract_loaded): decorators.append( f"{indent}{self._native_call(self._pyi_projection(func), context, self._native_result_projection(func), func)}" ) @@ -2400,8 +2404,15 @@ def _native_value_ref( raise ValueError(f"Unsupported native_call value reference: {kind!r}") @staticmethod - def _requires_native_call(func: SemanticFunction) -> bool: - """Return whether requires native call.""" + def _requires_native_call(func: SemanticFunction, *, contract_loaded: bool = False) -> bool: + """Return whether requires native call. + + A loaded contract states a native layout with ``@native_call`` exactly + when it wrote one; otherwise its annotations produced the layout and + printing them again restates it. + """ + if contract_loaded and not func.metadata.get(NATIVE_PROJECTION_METADATA): + return False if isinstance(func, SemanticMethod) and func.name == "__init__" and func.metadata.get(BIND_TARGET_METADATA): return True if PyiPrinter._scalar_descriptor_kind(func.return_type) is not None: @@ -2476,6 +2487,16 @@ def _is_private(node) -> bool: """Return whether is private.""" return getattr(node, "visibility", "public") == "private" + @classmethod + def _renamed_native_spelling(cls, declaration: SemanticVariable, emitted_name: str) -> str | None: + """Return the native spelling a data declaration must record, if any. + + A contract's ``SourceName`` lives on the origin, so a loaded contract + prints it back; a source-built declaration is named natively already. + """ + native = declaration.origin.native_name or declaration.name + return native if cls._renames_native_entity(declaration, native, emitted_name) else None + @staticmethod def _renames_native_entity(declaration: object, native_name: object, emitted_name: str) -> bool: """Return whether an emitted name has to record the spelling it came from. diff --git a/tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py b/tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py index be817a5d1..69d777bc9 100644 --- a/tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py +++ b/tests/fortran/infrastructure/semantic_pyi/semantics/test_round_trip_properties.py @@ -222,6 +222,18 @@ def score( def transform( values: Float64[:] ) -> Float64: ... +""", + "native-spellings-of-renamed-entities": """ +@bind("native_state") +class state: + tally: Annotated[Int32, SourceName("TALLY")] + +counter: Annotated[Int32, SourceName("COUNTER")] +""", + "plain-tuple-return": """ +def pair( + x: Int32 +) -> tuple[Int32, Float64]: ... """, "visibility-and-module-state": """ import iso_c_binding From 2133359a8cca1e31b31689b2c2f9fd70caeefe64 Mon Sep 17 00:00:00 2001 From: said Date: Sun, 27 Sep 2026 00:38:30 +0100 Subject: [PATCH 45/47] codex: stop documenting optional callbacks as unsupported Optional dummy procedures work on both routes (test_optional_callbacks.py); only by-value optional prototype dummies and optional procedure pointers are blocked. The feature matrix, Fortran support page, optional-arguments limitations, and coverage index now say so and cite the end-to-end test. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 3 +++ docs/user/guide/optional-arguments.md | 3 ++- docs/user/language-support/feature-matrix.md | 2 +- docs/user/language-support/fortran-support.md | 5 +++-- tests/fortran/CONTRACT_COVERAGE.md | 6 +++--- 5 files changed, 12 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b9add7866..7e58d915c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,9 @@ release tags add a leading `v` to the package version. - Printing an edited `.pyi` contract again keeps what it stated: a `SourceName(...)` on a module variable or class field, a class's `@bind(...)`, and a plain `tuple[...]` return no longer gain an explicit `@native_call`. +- The feature matrix and Fortran support pages no longer list optional + callbacks as unsupported: an optional dummy procedure may be omitted or + passed `None`, as the Callbacks guide describes. - Generated bindings pass a wrapped derived-type object about twice as fast: they read its native storage through attribute names interned once instead diff --git a/docs/user/guide/optional-arguments.md b/docs/user/guide/optional-arguments.md index 563dd46a7..ce52cca3a 100644 --- a/docs/user/guide/optional-arguments.md +++ b/docs/user/guide/optional-arguments.md @@ -206,7 +206,8 @@ association. If its updated value is returned, Python receives a scalar or ## Limitations -- Optional procedure pointers and passed procedures are not yet supported. +- Optional procedure pointers are not yet supported. An optional dummy procedure + is supported; see [Callbacks](callbacks.md). - PRIK does not invent default values. The Fortran procedure handles missing arguments. diff --git a/docs/user/language-support/feature-matrix.md b/docs/user/language-support/feature-matrix.md index 4dc18877c..70bb9dc11 100644 --- a/docs/user/language-support/feature-matrix.md +++ b/docs/user/language-support/feature-matrix.md @@ -74,7 +74,7 @@ where they apply. | Scalar kind coverage | Supported | [Data types](../guide/data-types.md) | [Scalar kind tests](../../../tests/fortran/data_types/end_to_end/test_primitive_scalar_runtime.py) | Real and complex storage wider than the target's `long double` is blocked. Logical scalars use Python `bool`; arrays use their documented NumPy dtype. | | Multi-source builds, Makefiles, verbose mode, and output placement | Supported | [Building the shared library](../guide/building-shared-library.md) | [Multi-source tests](../../../tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py), [compiler verbose tests](../../../tests/fortran/infrastructure/building/compiling/test_compiler_verbose.py) | Wrapped project sources compile in dependency order derived from their module/`use` graph, falling back to the given order when a compiled source was not parsed. PRIK does not discover sources you did not name, prebuilt module paths, or external libraries. | | Visibility, naming, keyword escaping, and collision policy | Supported | [Generic interfaces](../guide/generic-interfaces.md#key-rules) | [Visibility/naming tests](../../../tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_visibility_naming.py) | Strict mode rejects names that default mode can normalize. | -| Immediate call-scoped Python callbacks | Supported | [Callbacks](../guide/callbacks.md) | [Callback plan tests](../../../tests/fortran/callbacks/codegen/test_callback_planning.py), [scalar callback tests](../../../tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py), [array callback tests](../../../tests/fortran/callbacks/end_to_end/test_array_callbacks.py), [combined shape tests](../../../tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py) | Direct wrapper-plan generation supports entering-thread callbacks only. Stored, optional, asynchronous, or cross-thread callbacks are unsupported. | +| Immediate call-scoped Python callbacks | Supported | [Callbacks](../guide/callbacks.md) | [Callback plan tests](../../../tests/fortran/callbacks/codegen/test_callback_planning.py), [scalar callback tests](../../../tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py), [array callback tests](../../../tests/fortran/callbacks/end_to_end/test_array_callbacks.py), [combined shape tests](../../../tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py), [optional callback tests](../../../tests/fortran/callbacks/end_to_end/test_optional_callbacks.py) | Direct wrapper-plan generation supports entering-thread callbacks only; an optional callback may be omitted or `None`. Stored, asynchronous, or cross-thread callbacks are unsupported. | | Runtime error projection, GIL policy, recursion, OpenMP path, and GNU ABI checks | Supported | [Error handling](../guide/error-handling.md) | [Status projection runtime](../../../tests/fortran/error_handling/end_to_end/test_status_projection.py), [status and GIL lowering](../../../tests/fortran/error_handling/codegen/test_status_error_lowering.py), [recursion tests](../../../tests/fortran/error_handling/end_to_end/test_runtime_recursion.py), [OpenMP tests](../../../tests/fortran/error_handling/end_to_end/test_openmp_runtime.py), [ABI tests](../../../tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py) | OpenMP and ABI evidence is compiler/platform-specific; callers still own native synchronization. | | Fortran source wrapper builds | Supported | [Building the shared library](../guide/building-shared-library.md) | [Build modes](../../../tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py), [runtime ABI](../../../tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py) | Implemented for ordered Fortran source inputs. | | `value` arguments and existing `bind(C)` procedures | Supported | [Data types](../guide/data-types.md) | [`value` and `bind(C)` tests](../../../tests/fortran/data_types/end_to_end/test_value_and_bind_c.py) | Existing `bind(C)` support is deliberately ABI-guarded. | diff --git a/docs/user/language-support/fortran-support.md b/docs/user/language-support/fortran-support.md index b812124e0..c6e80e0ab 100644 --- a/docs/user/language-support/fortran-support.md +++ b/docs/user/language-support/fortran-support.md @@ -44,8 +44,9 @@ Fortran declaration: - Pointer arrays and projections have useful supported forms, but target lifetime, deallocation, and writable reassociation remain policy-gated. See [Pointers](../guide/pointers.md) and [Memory Management](../guide/memory-management.md). -- Callbacks are immediate and call-scoped. Stored, asynchronous, optional, and - cross-thread callbacks are unsupported. See [Callbacks](../guide/callbacks.md). +- Callbacks are immediate and call-scoped, and an optional callback may be + omitted. Stored, asynchronous, and cross-thread callbacks are unsupported. See + [Callbacks](../guide/callbacks.md). - Scalar derived types are supported, but arrays of derived types and several mutable or result polymorphic forms are not. See [Wrapping Derived Types](../guide/wrapping-derived-types.md). diff --git a/tests/fortran/CONTRACT_COVERAGE.md b/tests/fortran/CONTRACT_COVERAGE.md index 404cda425..bc583be7d 100644 --- a/tests/fortran/CONTRACT_COVERAGE.md +++ b/tests/fortran/CONTRACT_COVERAGE.md @@ -81,7 +81,7 @@ Authoritative sources: | [Optional Arguments: Key Rules](../../docs/user/guide/optional-arguments.md#key-rules) | Supported | scalar; array; string; derived input; output visibility; conservative no-`intent`; presence without result inflation | `tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py::test_optional_output_visibility_follows_intent_and_presence`
`tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_policy_completes_presence_mode_before_planning` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_array_buffers_preserve_omission_and_identity` | — | canonical | | [Optional Arguments: Scalar Allocatables and Pointers](../../docs/user/guide/optional-arguments.md#scalar-allocatables-and-pointers) | Supported | allocatable and pointer; omitted; present-unallocated/unassociated `None`; present concrete value; exact type rejection | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_policy_completes_presence_mode_before_planning`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_scalar_descriptors_distinguish_omitted_none_and_value` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_scalar_descriptors_distinguish_omitted_none_and_value` | — | canonical | | [Optional Arguments: Optional Outputs](../../docs/user/guide/optional-arguments.md#optional-outputs) | Supported | ordinary scalar and array outputs; derived output remains visible; allocatable and pointer output visibility; absent, explicit `None`, caller storage; in-place identity; stable result projection | `tests/fortran/optional_arguments/semantics/test_optional_fortran_semantics.py::test_optional_output_visibility_follows_intent_and_presence` | `tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_array_buffers_preserve_omission_and_identity` | — | canonical | -| [Optional Arguments: Limitations](../../docs/user/guide/optional-arguments.md#limitations) | Blocked | optional passed procedure; no invented native default; optional native literals cannot replace native procedure behavior | — | — | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_value_callback_dummy_is_blocked_before_codegen` (`policy`)
`tests/fortran/optional_arguments/codegen/test_optional_lowering.py::test_optional_arguments_with_hidden_literals_materialize_the_literal_in_the_binding` (`codegen`) | canonical | +| [Optional Arguments: Limitations](../../docs/user/guide/optional-arguments.md#limitations) | Blocked | optional procedure pointer; by-value optional callback prototype dummy; no invented native default; optional native literals cannot replace native procedure behavior | — | — | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_value_callback_dummy_is_blocked_before_codegen` (`policy`)
`tests/fortran/optional_arguments/codegen/test_optional_lowering.py::test_optional_arguments_with_hidden_literals_materialize_the_literal_in_the_binding` (`codegen`) | canonical | | [Generic Interfaces: Complete Example](../../docs/user/guide/generic-interfaces.md#complete-example) | Supported | free-form source; fixed-form parsing and contract generation; reviewed generated `.pyi`; source build; generated-`.pyi` replay | `tests/fortran/generic_interfaces/parsing/test_generic_interface_syntax.py::test_fixed_form_generic_interface_preserves_specific_procedures`
`tests/fortran/generic_interfaces/pipeline/test_generated_generic_contracts.py::test_generated_generic_contract_matches_fixture[foverloads_fixed]` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]`
`tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[generated-pyi]` | — | canonical | | [Generic Interfaces: Generated Contract](../../docs/user/guide/generic-interfaces.md#complete-example) | Supported | private link targets; one exact overload candidate per declaration; public-generic `@bind`; native target precedence | `tests/fortran/generic_interfaces/semantics/test_pyi_overload_semantics.py::test_convert_pyi_to_ir_resolves_prik_overload_by_explicit_specific_name`
`tests/fortran/generic_interfaces/policy/test_generic_policy.py::test_module_overload_bind_takes_precedence_per_candidate` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]` | — | canonical | | [Generic Interfaces: Usage in Python](../../docs/user/guide/generic-interfaces.md#complete-example) | Supported | exact `Int32`, `Float64`, and `Complex128`; scalar and rank-one dispatch; generated-class dispatch; no implicit coercion | `tests/fortran/generic_interfaces/codegen/test_overload_dispatch_plan.py::test_plan_records_one_exact_numpy_scalar_predicate_per_candidate` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]` | `tests/fortran/generic_interfaces/end_to_end/test_generic_interfaces.py::test_fortran_generic_interfaces_dispatch_in_generated_c_extension[source]` (`runtime`) | canonical | @@ -147,9 +147,9 @@ Authoritative sources: | [Callbacks: Small Example](../../docs/user/guide/callbacks.md#small-example) | Supported | immediate scalar reference callback; exact scalar result; source and generated-`.pyi` replay | — | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]`
`tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[generated-pyi]` | — | canonical | | [Callbacks: Choosing The Prototype Spelling](../../docs/user/guide/callbacks.md#choosing-the-prototype-spelling) | Supported | native value/reference ABI; named arguments; shape dependencies; imported prototypes | `tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_uses_value_default_and_explicit_reference_callbacks`
`tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_imported_prototype_resolves_as_module_interface_definition` | `tests/fortran/callbacks/end_to_end/test_array_callbacks.py::test_immediate_dummy_procedure_converts_array_arguments_and_results[source]` | `tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_rejects_invalid_prototype_address_wrappers[Addr(String[8])]` (`semantics`)
`tests/fortran/callbacks/semantics/test_pyi_callback_semantics.py::test_convert_pyi_to_ir_rejects_redundant_or_invalid_prototype_value_wrappers[Value(Float64)]` (`semantics`) | canonical | | [Callbacks: Key Rules](../../docs/user/guide/callbacks.md#key-rules) | Supported | call scope; same-thread nested entry; exact scalar result; reference cleanup; live array and derived storage | — | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]`
`tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]` | — | canonical | -| [Callbacks: Important Limitations](../../docs/user/guide/callbacks.md#important-limitations) | Blocked | persistent, optional, asynchronous, and cross-thread callbacks; optional or descriptor prototype forms; fatal exceptions and invalid returns | — | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` | `tests/fortran/callbacks/policy/test_callback_policy.py::test_unsupported_callback_forms_are_blocked_before_codegen[def callback_shape(value: Allocatable[Float64]) -> None: ...-callback argument 'value' uses unsupported allocatable, pointer, polymorphic, or assumed-type storage]` (`policy`)
`tests/fortran/callbacks/policy/test_callback_policy.py::test_unsupported_callback_forms_are_blocked_before_codegen[def callback_shape(value: Float64 = ...) -> None: ...-callback argument 'value' cannot be both optional and passed by value; use a reference dummy so absence has a null-pointer ABI]` (`policy`) | canonical | +| [Callbacks: Important Limitations](../../docs/user/guide/callbacks.md#important-limitations) | Blocked | persistent, asynchronous, and cross-thread callbacks; by-value optional or descriptor prototype forms; fatal exceptions and invalid returns | — | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` | `tests/fortran/callbacks/policy/test_callback_policy.py::test_unsupported_callback_forms_are_blocked_before_codegen[def callback_shape(value: Allocatable[Float64]) -> None: ...-callback argument 'value' uses unsupported allocatable, pointer, polymorphic, or assumed-type storage]` (`policy`)
`tests/fortran/callbacks/policy/test_callback_policy.py::test_unsupported_callback_forms_are_blocked_before_codegen[def callback_shape(value: Float64 = ...) -> None: ...-callback argument 'value' cannot be both optional and passed by value; use a reference dummy so absence has a null-pointer ABI]` (`policy`) | canonical | | [Callbacks: Key Rules](../../docs/user/guide/callbacks.md#key-rules) | Supported | completed callback ABI, conversion, lifecycle, adapter symbols, bridge declaration, source/generated parity | — | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]`
`tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[generated-pyi]` | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_callback_plan_edits_fail_central_validation_before_backend_emission[symbols-invalid-callback-symbols]` (`codegen`) | canonical | -| [Feature Matrix: Immediate Call-Scoped Python Callbacks](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | scalar, array, string, and derived conversions; entering thread; source/generated parity | — | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]` | — | canonical | +| [Feature Matrix: Immediate Call-Scoped Python Callbacks](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | scalar, array, string, and derived conversions; entering thread; source/generated parity; omitted or `None` optional callback | — | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]`
`tests/fortran/callbacks/end_to_end/test_optional_callbacks.py::test_optional_callback_and_optional_dummies_preserve_each_presence_state` | — | canonical | | [Feature Matrix: Persistent Callbacks And Procedure Pointers](../../docs/user/language-support/feature-matrix.md#unsupported-or-blocked-forms) | Blocked | stored or post-call invocation; call-scoped context | — | — | `tests/fortran/callbacks/codegen/test_callback_planning.py::test_callback_plan_edits_fail_central_validation_before_backend_emission[lifecycle-unbalanced-callback-lifecycle]` (`codegen`) | canonical | | [Enumerations: Complete Example](../../docs/user/guide/enumerations.md#complete-example) | Supported | `enum, bind(C)`; explicit, implicit, negative, and symbolic values; source generation; reviewed contract; source/generated replay | `tests/fortran/enumerations/parsing/test_enum_syntax.py::test_valid_enum_subunit_accepts_optional_separator_and_multiple_enumerators` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]`
`tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[generated-pyi]` | — | canonical | | [Enumerations: Usage In Python](../../docs/user/guide/enumerations.md#usage-in-python) | Supported | module constants; integer procedure input and result; exact `np.int32` values | `tests/fortran/enumerations/semantics/test_enum_semantics.py::test_fortran_enums_preserve_values_in_generated_pyi_contract` | `tests/fortran/enumerations/end_to_end/test_enum_runtime.py::test_fortran_enums_preserve_integer_runtime_surface[source]` | — | canonical | From a5f2c5603c28c54a22007baa5f99d5450b99e8ff Mon Sep 17 00:00:00 2001 From: said Date: Sun, 27 Sep 2026 00:49:13 +0100 Subject: [PATCH 46/47] codex: fold review-flagged duplicate tests into shared cases - Drop the semantic dependent-type selection test; the export-symbol E2E proves parent, component, cross-file nested component, exclusion, and the exact published set in both lanes. - Source build modes: one verbose build checks the step log and custom flag routing (default release flags stay pinned by compiling tests); the default and --out-dir alias placements and the two unusable-source rejections are parametrized. - C project resolution: one project proves quoted include-dir resolution, system-include recording, and that neither header is parsed. - .pyi imports: six import-spelling tests become one table on the invariant that a contract imports the exact name its dependency binds; the two Fortran source-name recording tests share one module. Co-Authored-By: Claude Opus 5.5 --- .../parsing/test_c_project_resolution.py | 52 +-- tests/fortran/CONTRACT_COVERAGE.md | 8 +- .../test_fortran_export_symbol_selection.py | 34 -- .../end_to_end/test_source_build_modes.py | 190 ++++----- .../test_pyi_printer_imports_and_packages.py | 371 ++++++++---------- 5 files changed, 254 insertions(+), 401 deletions(-) diff --git a/tests/c/infrastructure/parsing/test_c_project_resolution.py b/tests/c/infrastructure/parsing/test_c_project_resolution.py index f8cd61dca..30e2b069c 100644 --- a/tests/c/infrastructure/parsing/test_c_project_resolution.py +++ b/tests/c/infrastructure/parsing/test_c_project_resolution.py @@ -20,7 +20,13 @@ def test_project_include_graph_tracks_local_system_missing_and_cycles(tmp_path: assert any(diag.code == "C_UNRESOLVED_INCLUDE" for diag in project.files["a.h"].diagnostics) -def test_project_resolves_quoted_includes_through_include_dirs(tmp_path: Path): +def test_project_resolves_quoted_includes_through_include_dirs_without_parsing_them(tmp_path: Path): + """A quoted include resolves through ``include_dirs``; a system include is only recorded. + + Neither header is parsed into the project: the resolved local header is an + edge in the include graph, and a local file shadowing a system header is + neither searched for nor read. + """ from prik.parsers.c import parse_c_project include_dir = tmp_path / "include" @@ -30,47 +36,19 @@ def test_project_resolves_quoted_includes_through_include_dirs(tmp_path: Path): types = include_dir / "types.h" api = src_dir / "api.h" types.write_text("typedef int api_int;\n", encoding="utf-8") - api.write_text('#include "types.h"\napi_int answer(void);\n', encoding="utf-8") - - project = parse_c_project([api], include_dirs=[include_dir]) - - include = project.files[str(api)].includes[0] - assert include.target == "types.h" - assert include.resolved_path == str(types) - assert project.unresolved_includes[str(api)] == set() - - -def test_project_records_local_include_without_recursively_parsing_resolved_header(tmp_path: Path): - from prik.parsers.c import parse_c_project - - include_dir = tmp_path / "generated" - include_dir.mkdir() - generated = include_dir / "generated_types.h" - api = tmp_path / "api.h" - generated.write_text("typedef int generated_int;\n", encoding="utf-8") - api.write_text('#include "generated_types.h"\nint run(void);\n', encoding="utf-8") + (include_dir / "stddef.h").write_text("typedef unsigned long size_t;\n", encoding="utf-8") + api.write_text('#include "types.h"\n#include \napi_int answer(void);\n', encoding="utf-8") project = parse_c_project([api], include_dirs=[include_dir]) + quoted, system = project.files[str(api)].includes + assert (quoted.target, quoted.resolved_path) == ("types.h", str(types)) + assert (system.target, system.resolved_path) == ("stddef.h", None) assert set(project.files) == {str(api)} - assert project.files[str(api)].includes[0].resolved_path == str(generated) - assert project.include_graph[str(api)] == {str(generated)} - assert "generated_int" not in project.typedefs - - -def test_project_records_system_include_without_searching_or_parsing_local_copy(tmp_path: Path): - from prik.parsers.c import parse_c_project - - local_system_header = tmp_path / "stddef.h" - api = tmp_path / "api.h" - local_system_header.write_text("typedef unsigned long size_t;\n", encoding="utf-8") - api.write_text("#include \nint run(void);\n", encoding="utf-8") - - project = parse_c_project([api], include_dirs=[tmp_path]) - - assert set(project.files) == {str(api)} - assert project.files[str(api)].includes[0].resolved_path is None + assert project.include_graph[str(api)] == {str(types)} assert project.system_includes[str(api)] == {"stddef.h"} + assert project.unresolved_includes[str(api)] == set() + assert "api_int" not in project.typedefs assert "size_t" not in project.typedefs diff --git a/tests/fortran/CONTRACT_COVERAGE.md b/tests/fortran/CONTRACT_COVERAGE.md index bc583be7d..3fc73dec1 100644 --- a/tests/fortran/CONTRACT_COVERAGE.md +++ b/tests/fortran/CONTRACT_COVERAGE.md @@ -175,8 +175,8 @@ Authoritative sources: | [Error Handling: Best Practices](../../docs/user/guide/error-handling.md#best-practices) | Supported | full diagnostic first; verbose command replay; debug traceback only on demand; edited-contract inspection; risky callback isolation | `tests/fortran/error_handling/parsing/test_fortran_diagnostics.py::test_parse_error_message_includes_filename_and_lineno`
`tests/fortran/error_handling/compiling/test_verbose_commands.py::test_run_command_verbose_prints_replayable_command` | `tests/fortran/error_handling/pipeline/test_concise_cli_diagnostics.py::test_cli_formats_parse_errors_concisely_and_reraises_with_debug`
`tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` | — | canonical | | [Error Handling: Failure Stages](../../docs/user/guide/error-handling.md#failure-stages) | Supported | ordinary wrapper exceptions; no inferred application convention; opt-in status/message projection; cleanup after failure; native termination remains unrecoverable | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers`
`tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_status_error_plan_edits_fail_before_backend_lowering` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/callbacks/end_to_end/test_scalar_callbacks.py::test_immediate_scalar_dummy_procedure_calls_python_callback[source]` (`runtime`) | canonical | | [Feature Matrix: Runtime Error Projection, GIL Policy, Recursion, OpenMP Path, And GNU ABI Checks](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | status error and message; completed GIL envelope; recursion/OpenMP/ABI remain separately owned; no caller synchronization inference | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | — | canonical | -| [Building The Shared Library: Build](../../docs/user/guide/building-shared-library.md#build) | Supported | source input; default and explicit module names; build directory; generated sources; importable shared library | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_result_records_structured_native_plan`
`tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_generated_pyi_replay_matches_source_build_without_source_reparse` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_documented_readme_points_example_builds_and_imports` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_wrapper_build_rejects_empty_source_list` (`pipeline`)
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_wrapper_build_rejects_missing_source` (`pipeline`) | canonical | -| [Building The Shared Library: Import](../../docs/user/guide/building-shared-library.md#import) | Supported | ABI-suffixed artifact; stable module import name; explicit output name; root-function name collision avoidance | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_names_importable_shared_library`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_default_module_name_does_not_collide_with_root_function` | — | canonical | +| [Building The Shared Library: Build](../../docs/user/guide/building-shared-library.md#build) | Supported | source input; default and explicit module names; build directory; generated sources; importable shared library | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_result_records_structured_native_plan`
`tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_generated_pyi_replay_matches_source_build_without_source_reparse` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_documented_readme_points_example_builds_and_imports` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_wrapper_build_rejects_unusable_sources` (`pipeline`) | canonical | +| [Building The Shared Library: Import](../../docs/user/guide/building-shared-library.md#import) | Supported | ABI-suffixed artifact; stable module import name; explicit output name; root-function name collision avoidance | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_keeps_its_import_alias_beside_the_invocation` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_names_importable_shared_library`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_default_module_name_does_not_collide_with_root_function` | — | canonical | | [Building The Shared Library: Multiple Source Files](../../docs/user/guide/building-shared-library.md#multiple-source-files) | Supported | dependency-ordered compilation; contained-module namespaces; standalone externals; one merged extension; generated and edited contract parity | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_multi_source_generated_contract_build_matches_source_runtime_and_link_order` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_multi_file_modules_build_one_merged_extension`
`tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_multi_file_standalone_procedures_build_one_merged_extension` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_missing_module_directory_reports_compile_error` (`compiling`) | canonical | | [Building The Shared Library: Use A Makefile](../../docs/user/guide/building-shared-library.md#use-a-makefile) | Supported | generation without compilation; editable compiler and flags; ordered source dependencies; GNU Make build; manifest regeneration and replay | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_makefile_manifest_and_replay_workflows` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_makefile_mode_reproduces_multi_source_build` | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_wrapper_build_rejects_generation_verbose_combination[makefile]` (`pipeline`) | canonical | | [Building The Shared Library: Compatibility](../../docs/user/guide/building-shared-library.md#compatibility) | Supported | target ABI; debug and optimized wrappers; top-level kind flags; platform-specific extension; rebuildable native artifacts | `tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py::test_top_level_native_kind_flags_drive_internal_type_measurement` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_makefile_mode_reproduces_multi_source_build` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_incompatible_native_artifact_reports_linker_error` (`compiling`) | canonical | @@ -191,7 +191,7 @@ Authoritative sources: | [`.pyi`: Standalone Procedures](../../docs/user/reference/pyi-format.md#standalone-procedures) | Supported | `@standalone`; entry placement; multiple root procedures; no invented module scope | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_generated_contract_package_matches_reviewed_layout[contract_standalone_only]`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_imports_and_packages.py::test_generated_native_scope_comes_from_contract_filename` | — | — | canonical | | [`.pyi`: Source-To-Contract Layout](../../docs/user/reference/pyi-format.md#source-to-contract-layout) | Supported | module-only, standalone-only, mixed, multi-module, same-name, and transitive-import source layouts | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_generated_contract_package_matches_reviewed_layout[contract_import_graph]`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_multi_module_generation_keeps_each_native_namespace` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback` | — | canonical | | [`.pyi`: Root Export Contract](../../docs/user/reference/pyi-format.md#root-export-contract) | Supported | module import, selective symbol export, alias, support-import exclusion, and collision rejection | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_checked_entry_discovers_its_complete_contract_package[contract_import_graph]`
`tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/policy/test_export_and_initializer_policy.py::test_policy_completion_prunes_unexported_entry_declarations_before_lowering` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_selects_child_flattened_aliased_and_bound_exports` | `tests/fortran/infrastructure/semantic_pyi/contracts/exports_and_modules/end_to_end/test_package_exports.py::test_entry_contract_rejects_colliding_wildcard_exports` (`pipeline`) | canonical | -| [`.pyi`: Entry Contract And Extension Identity](../../docs/user/reference/pyi-format.md#entry-contract-and-extension-identity) | Supported | `__init__.pyi` parent identity; explicit output identity; leaf identity; ABI-suffixed shared object | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_generated_contract_package_matches_reviewed_layout[contract_same_name]` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias` | — | canonical | +| [`.pyi`: Entry Contract And Extension Identity](../../docs/user/reference/pyi-format.md#entry-contract-and-extension-identity) | Supported | `__init__.pyi` parent identity; explicit output identity; leaf identity; ABI-suffixed shared object | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_package_generation.py::test_generated_contract_package_matches_reviewed_layout[contract_same_name]` | `tests/fortran/infrastructure/semantic_pyi/end_to_end/test_authoritative_contract_runtime.py::test_generated_contract_rebuilds_without_native_source_fallback`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_keeps_its_import_alias_beside_the_invocation` | — | canonical | | [`.pyi`: Contract Import Graph](../../docs/user/reference/pyi-format.md#contract-import-graph) | Supported | recursive relative imports; deterministic discovery order; parse cache; missing file and cycle diagnostics | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_pyi_contract_bundle_reuses_import_discovery_conversion_cache`
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_checked_entry_discovers_its_complete_contract_package[contract_import_graph]` | — | `tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_recursive_graph_reports_missing_relative_contract_before_native_validation` (`pipeline`)
`tests/fortran/infrastructure/semantic_pyi/pipeline/test_contract_loading.py::test_recursive_graph_reports_cycles_before_codegen` (`pipeline`) | canonical | | [`.pyi`: Semantic Type Names](../../docs/user/reference/pyi-format.md#semantic-type-names) | Supported | canonical primitive, wrapper, nested, qualified, aliased, callback, and storage type spellings | `tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_dispatches_nested_and_qualified_semantic_types`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_follows_arbitrary_contract_aliases` | — | — | canonical | | [`.pyi`: Metadata With `Annotated`](../../docs/user/reference/pyi-format.md#metadata-with-annotated) | Supported | constraints; source names; layout/copy; immutability; native descriptor and provenance metadata; stable round trip | `tests/fortran/infrastructure/semantic_pyi/parsing/test_python_ast_contracts.py::test_pyi_parser_preserves_generic_constraints_as_annotation_metadata`
`tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_preserves_rank_selector_and_character_allocatable_marker` | — | `tests/fortran/infrastructure/semantic_pyi/semantics/test_types_and_values.py::test_convert_pyi_to_ir_rejects_additional_invalid_storage_forms[value: Annotated[Int32, 'bad']\n-Unsupported Annotated metadata: "'bad'"]` (`semantics`) | canonical | @@ -219,7 +219,7 @@ Authoritative sources: | [`.pyi` Calls And Results: Edit Types Shapes Layout And Optionality](../../docs/user/reference/pyi-contracts/calls-and-results.md#edit-types-shapes-layout-and-optionality) | Supported | fixed/open shapes; exact dtype, rank, layout, writeability, byte order, alignment, and zero-size checks; Fortran-order default; supported nullable/defaulted native optionals | `tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py::test_unavailable_dense_extent_role_fails_before_backend_lowering`
`tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_policy_completes_presence_mode_before_planning` | `tests/fortran/arrays/end_to_end/test_array_contract_validation.py::test_remaining_array_contracts_are_validated_before_fortran_calls[source]`
`tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py::test_optional_arguments_drive_fortran_present_behavior[source]` | `tests/fortran/optional_arguments/policy/test_optional_policy.py::test_optional_value_callback_dummy_is_blocked_before_codegen` (`policy`) | canonical | | [`.pyi` Calls And Results: Translate Status Results Into Exceptions](../../docs/user/reference/pyi-contracts/calls-and-results.md#translate-status-results-into-exceptions) | Supported | named hidden scalar integer status; optional hidden string message; configurable success value; consumed projected outputs | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_status_projection_accepts_an_optional_missing_message_target`
`tests/fortran/error_handling/codegen/test_status_error_lowering.py::test_status_error_plan_edits_fail_before_backend_lowering` | `tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_runtime_status_policy_rejects_invalid_output_contracts[@raises(status="status", message="message")\ndef solve() -> tuple[Returns["status", Int32], Returns["message", Int32]]: ...-must be a scalar string hidden output]` (`policy`) | canonical | | [`.pyi` Calls And Results: Release The GIL For A Native Call](../../docs/user/reference/pyi-contracts/calls-and-results.md#release-the-gil-for-a-native-call) | Supported | ordinary held call; explicit released call; status conversion after reacquisition; callback trampoline reacquisition | `tests/fortran/error_handling/semantics/test_status_contract_semantics.py::test_runtime_policy_decorators_round_trip_through_pyi`
`tests/fortran/error_handling/end_to_end/test_status_projection.py::test_status_projection_consumes_outputs_raises_message_and_recovers` | `tests/fortran/callbacks/end_to_end/test_supported_callback_shapes.py::test_immediate_callbacks_cover_all_supported_argument_shapes[source]` | — | canonical | -| [Feature Matrix: Multi-Source Builds, Makefiles, Verbose Mode, And Output Placement](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | dependency-ordered compilation; direct and Makefile builds; replayable verbose commands; ABI artifact and stable alias placement | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_verbose_mode_prints_full_direct_build_commands` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_makefile_mode_reproduces_multi_source_build`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias` | — | canonical | +| [Feature Matrix: Multi-Source Builds, Makefiles, Verbose Mode, And Output Placement](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | dependency-ordered compilation; direct and Makefile builds; replayable verbose commands; ABI artifact and stable alias placement | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_verbose_mode_prints_each_build_step_and_routes_custom_flags` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_makefile_mode_reproduces_multi_source_build`
`tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_fortran_wrapper_keeps_its_import_alias_beside_the_invocation` | — | canonical | | [Feature Matrix: Fortran Source Wrapper Builds](../../docs/user/language-support/feature-matrix.md#supported-runtime-features) | Supported | ordered Fortran source inputs; generated contracts; structured native plan; ABI-compatible import | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_result_records_structured_native_plan`
`tests/fortran/infrastructure/building/pipeline/test_source_generated_contracts.py::test_source_build_generated_pyi_contract_matches_fixture[fdefault_output]` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_makefile_mode_reproduces_multi_source_build` | — | canonical | | [Feature Matrix: Semantic `.pyi` Wrapper Builds From Explicit Native Artifacts](../../docs/user/language-support/feature-matrix.md#supported-inspection-features) | Partially supported | exactly one entry contract; explicit native input; source-free object build; ordered link items; current runtime subset | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_python_api_accepts_exactly_one_entry_contract` | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_generated_pyi_replay_matches_source_build_without_source_reparse` | `tests/fortran/infrastructure/building/pipeline/test_pyi_build_modes.py::test_pyi_python_api_rejects_a_missing_native_artifact` (`pipeline`) | canonical | | [Feature Matrix: Advanced Multi-Source Dependency Discovery And External-Library Integration](../../docs/user/language-support/feature-matrix.md#unsupported-or-blocked-forms) | Blocked | undeclared sources, prebuilt module paths, and external-library discovery are caller/build-system responsibilities; dependency ordering among parsed sources is PRIK's; explicit paths remain supported | `tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py::test_source_build_reuses_native_plan_for_additional_compile_and_link_inputs` | `tests/fortran/infrastructure/building/end_to_end/test_multi_source_builds.py::test_imported_contracts_resolve_from_one_archive_or_shared_library[archive]` | `tests/fortran/infrastructure/building/end_to_end/test_native_bundles.py::test_missing_module_directory_reports_compile_error` (`compiling`) | canonical | diff --git a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py index f55b2fc79..40fcc8d07 100644 --- a/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py +++ b/tests/fortran/functions/semantics/test_fortran_export_symbol_selection.py @@ -184,40 +184,6 @@ def test_external_root_cannot_satisfy_a_module_qualified_identity(): select_fortran_export_symbols([external], ["foo::external"]) -def test_selection_retains_component_and_parent_types_of_a_selected_signature(): - """A selected type brings the types its components and parent declare, and no others.""" - module = fortran_module_to_semantic_module( - parse_fortran_file( - """ -module shapes - implicit none - type :: base_t - integer :: b = 1 - end type base_t - type :: inner_t - integer :: a = 2 - end type inner_t - type, extends(base_t) :: outer_t - type(inner_t) :: inner - end type outer_t - type :: unrelated_t - integer :: u = 0 - end type unrelated_t -contains - subroutine use_outer(x) - type(outer_t), intent(in) :: x - end subroutine use_outer -end module shapes -""" - ).modules[0] - ) - - selected = select_fortran_export_symbols([module], ["shapes::use_outer"]).primary_modules[0] - - assert sorted(cls.name for cls in selected.classes) == ["base_t", "inner_t", "outer_t"] - assert set(selected.exported_names) == {"use_outer", "base_t", "inner_t", "outer_t"} - - def test_selection_through_a_two_level_facade_reaches_each_declaring_module(): """A generic, a procedure, and a variable re-exported twice resolve to where they are declared.""" project = parse_fortran_project( diff --git a/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py b/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py index 6d78b2093..2fc2ef3db 100644 --- a/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py +++ b/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py @@ -26,9 +26,20 @@ pytestmark = pytest.mark.fortran_end_to_end -def test_verbose_mode_prints_full_direct_build_commands(tmp_path: Path): - source = tmp_path / "verbose_api.f90" - shutil.copyfile(VERBOSE_SOURCE, source) +def test_verbose_mode_prints_each_build_step_and_routes_custom_flags(tmp_path: Path): + """Verbose output names every step and the exact commands, with each flag on its own stage. + + Default release flags are pinned by the compile-command tests in + ``compiling/test_compiler_verbose.py``; this build checks what verbose mode + reports and where selected flags, includes, and the compiler reach. + """ + source = tmp_path / SCALE_SOURCE.name + shutil.copyfile(SCALE_SOURCE, source) + include_dir = tmp_path / "include" + include_dir.mkdir() + selected_compiler = tmp_path / "selected-gfortran" + selected_compiler.symlink_to(shutil.which("gfortran")) + build_dir = tmp_path / "build" / "SCALE_debug" result = subprocess.run( [ @@ -36,9 +47,18 @@ def test_verbose_mode_prints_full_direct_build_commands(tmp_path: Path): "-m", "prik", str(source), - "--verbose", + "--out", + "SCALE_debug", "--out-dir", - str(tmp_path), + str(build_dir), + "--verbose", + "--compiler", + str(selected_compiler), + "-I", + str(include_dir), + "--native-compile-flags=-O1", + "--wrapper-fortran-flags=-O2", + "--wrapper-c-flags=-O2", ], capture_output=True, text=True, @@ -47,15 +67,25 @@ def test_verbose_mode_prints_full_direct_build_commands(tmp_path: Path): ) command_lines = result.stdout.splitlines() - assert any(str(source) in line and "-c" in line for line in command_lines) - assert any("bind_c_verbose_api_wrapper.f90" in line and "-c" in line for line in command_lines) - assert any("verbose_api_wrapper.c" in line and "-c" in line for line in command_lines) - c_wrapper_command = next(line for line in command_lines if "verbose_api_wrapper.c" in line and "-c" in line) - c_wrapper_parts = shlex.split(c_wrapper_command) - assert "-O3" in c_wrapper_parts - assert "-DNDEBUG" in c_wrapper_parts - assert "-g" not in c_wrapper_parts - link_command = next(line for line in command_lines if "-shared" in line and "verbose_api" in line) + native_command = next(line for line in command_lines if str(source) in line and "-c" in line) + fortran_wrapper_command = next( + line for line in command_lines if "bind_c_SCALE_debug_wrapper.f90" in line and "-c" in line + ) + c_wrapper_command = next(line for line in command_lines if "SCALE_debug_wrapper.c" in line and "-c" in line) + link_command = next(line for line in command_lines if "-shared" in line and "SCALE_debug" in line) + assert "-O1" in shlex.split(native_command) + assert "-O2" not in shlex.split(native_command) + assert "-O2" in shlex.split(fortran_wrapper_command) + assert "-O2" in shlex.split(c_wrapper_command) + assert "-O2" in shlex.split(link_command) + assert shlex.split(native_command)[0] == str(selected_compiler) + assert shlex.split(fortran_wrapper_command)[0] == str(selected_compiler) + assert shlex.split(link_command)[0] == str(selected_compiler) + for command in (native_command, fortran_wrapper_command, c_wrapper_command): + tokens = shlex.split(command) + include_values = tuple(tokens[index + 1] for index, token in enumerate(tokens) if token == "-I") + assert str(include_dir) in include_values + link_parts = shlex.split(link_command) link_output = link_parts[link_parts.index("-o") + 1] step_lines = [ @@ -63,22 +93,19 @@ def test_verbose_mode_prints_full_direct_build_commands(tmp_path: Path): for line in command_lines if line.startswith(">> ") and not line.startswith((">> Timing", ">> Total build time")) ] - bridge_source = tmp_path / "bind_c_verbose_api_wrapper.f90" - binding_source = tmp_path / "verbose_api_wrapper.c" - header = tmp_path / "verbose_api_wrapper.h" - native_object = tmp_path / "verbose_api.o" - bridge_object = tmp_path / "bind_c_verbose_api_wrapper.o" - binding_object = tmp_path / "verbose_api_wrapper.o" + bridge_source = build_dir / "bind_c_SCALE_debug_wrapper.f90" + binding_source = build_dir / "SCALE_debug_wrapper.c" + header = build_dir / "SCALE_debug_wrapper.h" assert "Complete wrapper policies" in step_lines assert "Generate binding source" in step_lines assert "Generate bridge source" in step_lines assert "Generate binding header" in step_lines - assert f"Compile native source: {source} -> {native_object}" in step_lines + assert f"Compile native source: {source} -> {build_dir / 'scale.o'}" in step_lines assert f"Write bridge source: {bridge_source}" in step_lines assert f"Write binding source: {binding_source}" in step_lines assert f"Write binding header: {header}" in step_lines - assert f"Compile bridge source: {bridge_source} -> {bridge_object}" in step_lines - assert f"Compile binding source: {binding_source} -> {binding_object}" in step_lines + assert f"Compile bridge source: {bridge_source} -> {build_dir / 'bind_c_SCALE_debug_wrapper.o'}" in step_lines + assert f"Compile binding source: {binding_source} -> {build_dir / 'SCALE_debug_wrapper.o'}" in step_lines assert f"Create shared library: {link_output}" in step_lines assert any(line.startswith(">> Timing: ") for line in command_lines) assert command_lines[-1].startswith(">> Total build time: ") @@ -112,84 +139,17 @@ def test_verbose_mode_prints_failing_compiler_command_before_execution(tmp_path: assert "Native compiler command failed:" in result.stderr -def test_verbose_mode_prints_custom_wrapper_flags(tmp_path: Path): - source = tmp_path / SCALE_SOURCE.name - shutil.copyfile(SCALE_SOURCE, source) - include_dir = tmp_path / "include" - include_dir.mkdir() - selected_compiler = tmp_path / "selected-gfortran" - selected_compiler.symlink_to(shutil.which("gfortran")) - - result = subprocess.run( - [ - sys.executable, - "-m", - "prik", - str(source), - "--out", - "SCALE_debug", - "--out-dir", - str(tmp_path / "build" / "SCALE_debug"), - "--verbose", - "--compiler", - str(selected_compiler), - "-I", - str(include_dir), - "--native-compile-flags=-O1", - "--wrapper-fortran-flags=-O2", - "--wrapper-c-flags=-O2", - ], - capture_output=True, - text=True, - check=True, - cwd=tmp_path, - ) - command_lines = result.stdout.splitlines() - - native_command = next(line for line in command_lines if str(source) in line and "-c" in line) - fortran_wrapper_command = next( - line for line in command_lines if "bind_c_SCALE_debug_wrapper.f90" in line and "-c" in line - ) - c_wrapper_command = next(line for line in command_lines if "SCALE_debug_wrapper.c" in line and "-c" in line) - link_command = next(line for line in command_lines if "-shared" in line and "SCALE_debug" in line) - assert "-O1" in shlex.split(native_command) - assert "-O2" not in shlex.split(native_command) - assert "-O2" in shlex.split(fortran_wrapper_command) - assert "-O2" in shlex.split(c_wrapper_command) - assert "-O2" in shlex.split(link_command) - assert shlex.split(native_command)[0] == str(selected_compiler) - assert shlex.split(fortran_wrapper_command)[0] == str(selected_compiler) - assert shlex.split(link_command)[0] == str(selected_compiler) - for command in (native_command, fortran_wrapper_command, c_wrapper_command): - tokens = shlex.split(command) - include_values = tuple(tokens[index + 1] for index, token in enumerate(tokens) if token == "-I") - assert str(include_dir) in include_values - - -def test_fortran_wrapper_default_places_artifacts_in_invocation_directory(tmp_path: Path): - source_dir = tmp_path / "source" - source_dir.mkdir() - run_dir = tmp_path / "run" - run_dir.mkdir() - source = source_dir / DEFAULT_OUTPUT_SOURCE.name - shutil.copyfile(DEFAULT_OUTPUT_SOURCE, source) - - cmd = [sys.executable, "-m", "prik", str(source), "--json"] - result = subprocess.run(cmd, capture_output=True, text=True, check=True, cwd=run_dir) - payload = json.loads(result.stdout) - - build_dir = run_dir / "__prik__" - shared_library = Path(payload["shared_library"]) - assert shared_library.parent == run_dir - assert shared_library.name == "fdefault_output.so" - assert shared_library.exists() - assert Path(payload["output_dir"]) == build_dir - assert (build_dir / "bind_c_fdefault_output_wrapper.f90").exists() - assert len(tuple(build_dir.glob("fdefault_output.*.so"))) == 1 - assert not list(source_dir.glob("*_wrapper.c")) - - -def test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias(tmp_path: Path): +@pytest.mark.parametrize( + ("location_args", "build_dir_name"), + [ + pytest.param((), "__prik__", id="default-build-directory"), + pytest.param(("--out-dir", "build"), "build", id="explicit-out-dir"), + ], +) +def test_fortran_wrapper_keeps_its_import_alias_beside_the_invocation( + tmp_path: Path, location_args: tuple[str, ...], build_dir_name: str +): + """The importable alias lands where the command ran; ABI artifacts stay in the build directory.""" source_dir = tmp_path / "source" source_dir.mkdir() run_dir = tmp_path / "run" @@ -198,7 +158,7 @@ def test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias(tmp_path: shutil.copyfile(DEFAULT_OUTPUT_SOURCE, source) result = subprocess.run( - [sys.executable, "-m", "prik", str(source), "--out-dir", "build", "--json"], + [sys.executable, "-m", "prik", str(source), *location_args, "--json"], capture_output=True, text=True, check=True, @@ -206,11 +166,14 @@ def test_fortran_wrapper_out_dir_separates_abi_artifact_from_cli_alias(tmp_path: ) payload = json.loads(result.stdout) - build_dir = run_dir / "build" + build_dir = run_dir / build_dir_name assert Path(payload["shared_library"]) == run_dir / "fdefault_output.so" assert (run_dir / "fdefault_output.so").is_file() + assert run_dir / payload["output_dir"] == build_dir + assert (build_dir / "bind_c_fdefault_output_wrapper.f90").exists() assert len(tuple(build_dir.glob("fdefault_output.*.so"))) == 1 assert not (build_dir / "fdefault_output.so").exists() + assert not list(source_dir.glob("*_wrapper.c")) def test_fortran_wrapper_default_module_name_does_not_collide_with_root_function(tmp_path: Path): @@ -530,16 +493,19 @@ def test_native_link_plan_serializes_interleaved_item_kinds(): ] -def test_wrapper_build_rejects_empty_source_list(tmp_path: Path): - with pytest.raises(ValueError, match="at least one Fortran source"): - build_fortran_extension([], output_dir=tmp_path) - - -def test_wrapper_build_rejects_missing_source(tmp_path: Path): - missing = tmp_path / "missing.f90" +@pytest.mark.parametrize( + ("sources", "error", "message"), + [ + pytest.param([], ValueError, "at least one Fortran source", id="empty-source-list"), + pytest.param("missing.f90", FileNotFoundError, "Fortran source not found", id="missing-source"), + ], +) +def test_wrapper_build_rejects_unusable_sources(tmp_path: Path, sources, error, message): + if isinstance(sources, str): + sources = tmp_path / sources - with pytest.raises(FileNotFoundError, match="Fortran source not found"): - build_fortran_extension(missing, output_dir=tmp_path) + with pytest.raises(error, match=message): + build_fortran_extension(sources, output_dir=tmp_path) @pytest.mark.parametrize("mode", ["makefile", "sources"]) diff --git a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py index 7c8f761ca..952422a62 100644 --- a/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py +++ b/tests/fortran/infrastructure/semantic_pyi/pipeline/test_pyi_printer_imports_and_packages.py @@ -444,83 +444,64 @@ def test_emit_module_aliases_standalone_only_for_actual_name_collisions(): assert "@prik_standalone_2\ndef standalone() -> Int32: ..." in twice_colliding -def test_generated_contract_imports_a_name_under_the_spelling_its_definition_uses(): - """An import binds the name the module it reads from actually defines. - - A source-derived contract writes its declarations under Python names, so a - Fortran entity spelled in capitals is declared lower case. An import asking - for the source spelling names nothing the dependency contract defines, and - loading the package back fails on it. - """ - consts = parse_fortran_source(""" +_CONSTS_MOD = """ module consts_mod implicit none integer, parameter :: IK = 4 end module consts_mod -""") - infos = parse_fortran_source(""" -module infos_mod -use consts_mod, only : IK -implicit none -end module infos_mod -""") - - stubs = emit_module_stubs( - [fortran_module_to_semantic_module(consts), fortran_module_to_semantic_module(infos)], - normalize_public_names=True, - ) - - assert "ik: Final[Int32]" in stubs["consts_mod"] - assert "from .consts_mod import ik" in stubs["infos_mod"] - assert "import IK" not in stubs["infos_mod"] - - -def test_generated_contract_renames_an_imported_name_under_both_spellings(): - """A renamed import binds the defined name to this contract's own name.""" - consts = parse_fortran_source(""" -module consts_mod -implicit none -integer, parameter :: IK = 4 -end module consts_mod -""") - renaming = parse_fortran_source(""" -module renaming_mod -use consts_mod, only : MY_IK => IK -implicit none -end module renaming_mod -""") - - stubs = emit_module_stubs( - [fortran_module_to_semantic_module(consts), fortran_module_to_semantic_module(renaming)], - normalize_public_names=True, - ) - - assert "from .consts_mod import ik as my_ik" in stubs["renaming_mod"] - - -def test_generated_contract_imports_a_prototype_under_its_declared_spelling(): - """A prototype keeps its spelling, so the import that binds it keeps it too. - - A contract writes a prototype under the name its own declaration states, and - an annotation naming that prototype is written the same way, so normalizing - the import would bind a name no declaration defines. - """ - declares = parse_fortran_source(""" -module pintrf_mod +""" +_CALLBACK_MOD = """ +module callback_mod implicit none private public :: OBJ abstract interface subroutine OBJ(x) implicit none -real(8), intent(in) :: x(:) +real(8), intent(in) :: x end subroutine OBJ end interface -end module pintrf_mod -""") - solver = parse_fortran_source(""" +end module callback_mod +""" + + +@pytest.mark.parametrize( + ("sources", "present", "absent"), + [ + pytest.param( + ( + _CONSTS_MOD, + """ +module infos_mod +use consts_mod, only : IK +implicit none +end module infos_mod +""", + ), + {"consts_mod": ["ik: Final[Int32]"], "infos_mod": ["from .consts_mod import ik"]}, + {"infos_mod": ["import IK"]}, + id="python-spelled-definition", + ), + pytest.param( + ( + _CONSTS_MOD, + """ +module renaming_mod +use consts_mod, only : MY_IK => IK +implicit none +end module renaming_mod +""", + ), + {"renaming_mod": ["from .consts_mod import ik as my_ik"]}, + {}, + id="renamed-import-under-both-spellings", + ), + pytest.param( + ( + _CALLBACK_MOD, + """ module solver_mod -use pintrf_mod, only : OBJ +use callback_mod, only : OBJ implicit none contains subroutine solve(calfun, x) @@ -528,54 +509,130 @@ def test_generated_contract_imports_a_prototype_under_its_declared_spelling(): real(8), intent(inout) :: x(:) end subroutine solve end module solver_mod -""") +""", + ), + { + "callback_mod": ["def OBJ("], + "solver_mod": ["from .callback_mod import OBJ", "calfun: OBJ"], + }, + {}, + id="prototype-keeps-its-declared-spelling", + ), + pytest.param( + ( + _CALLBACK_MOD, + """ +module values_mod +implicit none +integer, parameter :: OBJ = 1 +end module values_mod +""", + """ +module consumer_mod +use values_mod, only : OBJ +implicit none +end module consumer_mod +""", + ), + { + "callback_mod": ["def OBJ("], + "values_mod": ["obj: Final[Int32]"], + "consumer_mod": ["from .values_mod import obj"], + }, + {"consumer_mod": ["import OBJ"]}, + id="prototype-spelling-belongs-to-its-declaring-module-only", + ), + pytest.param( + ( + _CALLBACK_MOD, + """ +module user_mod +use callback_mod, only : obj +implicit none +contains +subroutine run(f, v) +procedure(obj) :: f +real(8), intent(in) :: v +end subroutine run +end module user_mod +""", + ), + { + "callback_mod": ["def OBJ("], + "user_mod": ["from .callback_mod import OBJ as obj", "f: obj"], + }, + {}, + id="prototype-used-under-another-case", + ), + pytest.param( + ( + """ +module collide_home +implicit none +contains +subroutine lambda(x) +integer, intent(inout) :: x +end subroutine lambda +subroutine lambda_(x) +integer, intent(inout) :: x +end subroutine lambda_ +end module collide_home +""", + """ +module collide_user +use collide_home, only : lambda_ +implicit none +private +public :: lambda_ +end module collide_user +""", + ), + { + "collide_home": ["def lambda__2("], + "collide_user": ["from .collide_home import lambda__2", '__all__ = ["lambda_"]'], + }, + {}, + id="name-a-collision-moved-aside", + ), + ], +) +def test_a_contract_imports_the_exact_name_its_dependency_binds(sources, present, absent): + """An import names what the dependency contract actually defines, under this contract's name. + Generated declarations follow Python naming, prototypes keep their declared + spelling, and a collision can move a name aside; an import that asked for + the Fortran spelling instead would bind nothing when the package loads. + """ stubs = emit_module_stubs( - [fortran_module_to_semantic_module(declares), fortran_module_to_semantic_module(solver)], + [fortran_module_to_semantic_module(parse_fortran_source(source)) for source in sources], normalize_public_names=True, ) - assert "def OBJ(" in stubs["pintrf_mod"] - assert "from .pintrf_mod import OBJ" in stubs["solver_mod"] - assert "calfun: OBJ" in stubs["solver_mod"] + for module_name, lines in present.items(): + for line in lines: + assert line in stubs[module_name], (module_name, line) + for module_name, lines in absent.items(): + for line in lines: + assert line not in stubs[module_name], (module_name, line) -def test_fortran_contract_records_no_source_name_for_a_case_only_python_name(): - """Writing a Fortran entity in lower case renames nothing worth recording. +def test_fortran_contract_records_a_source_name_only_where_python_renames_the_entity(): + """Only a real rename keeps the Fortran spelling beside the Python name. - Fortran names entities without regard to case, so a capitalized source - spelling and the lower-case Python name are the same entity and the - generated Fortran reaches it either way. + Fortran names entities without regard to case, so ``IK`` written as ``ik`` + renames nothing. A keyword, or a name a collision moved aside, is a real + rename and records the spelling the generated Fortran must reach. """ source = """ -module consts_mod +module naming_mod implicit none integer, parameter :: IK = 4 +integer :: lambda +integer :: LAMBDA_ contains subroutine SCALE_VALUE(x) integer, intent(in) :: x end subroutine SCALE_VALUE -end module consts_mod -""" - - module = fortran_module_to_semantic_module(parse_fortran_source(source)) - complete_python_export_policy(module) - code = emit_module(module, normalize_public_names=True) - - assert "ik: Final[Int32]" in code - assert "def scale_value(" in code - assert "SourceName" not in code - assert "@bind(" not in code - - -def test_fortran_contract_records_a_source_name_python_cannot_spell(): - """A name Python cannot hold as written keeps the spelling it came from.""" - source = """ -module naming_mod -implicit none -integer :: lambda -integer :: LAMBDA_ -contains subroutine ASSERT(x) integer, intent(in) :: x end subroutine ASSERT @@ -586,9 +643,13 @@ def test_fortran_contract_records_a_source_name_python_cannot_spell(): complete_python_export_policy(module) code = emit_module(module, normalize_public_names=True) + assert "ik: Final[Int32]" in code + assert "def scale_value(" in code assert 'lambda_: Annotated[Int32[()], SourceName("lambda")]' in code assert 'lambda__2: Annotated[Int32[()], SourceName("LAMBDA_")]' in code assert '@bind("ASSERT")\n@native_call([Addr(Arg(0))])\ndef assert_(' in code + assert code.count("SourceName(") == 2 + assert code.count("@bind(") == 1 def test_non_fortran_declaration_compares_its_native_spelling_exactly(): @@ -641,124 +702,6 @@ def test_generated_contract_binds_a_class_only_when_its_python_name_renames_its_ assert ("@bind(" in code) is binds -def test_prototype_spelling_is_kept_only_for_the_module_that_declares_one(): - """A prototype identity names its module, not a spelling used anywhere. - - One module may declare a prototype while another spells an ordinary - declaration the same way. The second follows Python naming, so an import - reading from it asks for the name that module actually defines. - """ - callbacks = parse_fortran_source(""" -module callback_mod -implicit none -private -public :: OBJ -abstract interface -subroutine OBJ(x) -implicit none -real(8), intent(in) :: x -end subroutine OBJ -end interface -end module callback_mod -""") - values = parse_fortran_source(""" -module values_mod -implicit none -integer, parameter :: OBJ = 1 -end module values_mod -""") - consumer = parse_fortran_source(""" -module consumer_mod -use values_mod, only : OBJ -implicit none -end module consumer_mod -""") - - stubs = emit_module_stubs( - [fortran_module_to_semantic_module(item) for item in (callbacks, values, consumer)], - normalize_public_names=True, - ) - - assert "def OBJ(" in stubs["callback_mod"] - assert "obj: Final[Int32]" in stubs["values_mod"] - assert "from .values_mod import obj" in stubs["consumer_mod"] - assert "import OBJ" not in stubs["consumer_mod"] - - -def test_prototype_import_uses_the_declared_spelling_whatever_case_names_it(): - """Fortran reaches a prototype without regard to case; a contract does not. - - A module may write `use callback_mod, only : obj` for a prototype declared - as `OBJ`, and the annotation then names it that way. The import binds the - declared spelling under the name this contract uses. - """ - callbacks = parse_fortran_source(""" -module callback_mod -implicit none -public :: OBJ -abstract interface -subroutine OBJ(x) -implicit none -real(8), intent(in) :: x -end subroutine OBJ -end interface -end module callback_mod -""") - user = parse_fortran_source(""" -module user_mod -use callback_mod, only : obj -implicit none -contains -subroutine run(f, v) -procedure(obj) :: f -real(8), intent(in) :: v -end subroutine run -end module user_mod -""") - - stubs = emit_module_stubs( - [fortran_module_to_semantic_module(item) for item in (callbacks, user)], - normalize_public_names=True, - ) - - assert "def OBJ(" in stubs["callback_mod"] - assert "from .callback_mod import OBJ as obj" in stubs["user_mod"] - assert "f: obj" in stubs["user_mod"] - - -def test_import_binds_the_name_a_collision_made_the_declaring_contract_use(): - """A collision moves a name aside, and the import follows it there.""" - home = parse_fortran_source(""" -module collide_home -implicit none -contains -subroutine lambda(x) -integer, intent(inout) :: x -end subroutine lambda -subroutine lambda_(x) -integer, intent(inout) :: x -end subroutine lambda_ -end module collide_home -""") - user = parse_fortran_source(""" -module collide_user -use collide_home, only : lambda_ -implicit none -private -public :: lambda_ -end module collide_user -""") - - stubs = emit_module_stubs( - [fortran_module_to_semantic_module(item) for item in (home, user)], - normalize_public_names=True, - ) - - assert "def lambda__2(" in stubs["collide_home"] - assert "from .collide_home import lambda__2" in stubs["collide_user"] - assert '__all__ = ["lambda_"]' in stubs["collide_user"] - - def test_generated_contract_states_the_names_its_source_publishes(): """A contract names its whole public surface, not only its re-exports. From 13f3277f6522f409e941b5a5ad6f35486c68d095 Mon Sep 17 00:00:00 2001 From: said Date: Sun, 27 Sep 2026 00:57:40 +0100 Subject: [PATCH 47/47] use AGENTS.d --- CLAUDE.md | 189 +----------------------------------------------------- 1 file changed, 1 insertion(+), 188 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index f45100ba1..43c994c2d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,188 +1 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## What this is - -PRIK (Python Runtime Interop Kit) generates native Python bindings from Fortran -projects: importable CPython extensions plus editable `.pyi` contracts for -reshaping the generated Python API. The active codebase is entirely Python -(`prik/`); Fortran/C fixture and source files (`*.f90`, `*.f95`, `*.for`, `*.c`, -`*.h`) are inputs/outputs of the tool, not implementation — don't spend -analysis effort on them unless explicitly asked to. - -This repo also has an `AGENTS.md` with detailed, load-bearing contributor -policy (test philosophy, changelog rules, policy-completion boundary, QA -gating). Read it — the summary below pulls out what matters most day to day, -but AGENTS.md is authoritative for edge cases. - -## Commands - -Install with dev/QA extras: - -```bash -python3 -m pip install -e ".[qa]" -``` - -Run the full suite: - -```bash -PYTHONPATH=. python3 -m pytest -q -``` - -Run a single test / focused owner (always prefer the narrowest owning path over the full suite): - -```bash -python3 -m pytest -q path/to/tests -python3 -m pytest -q path/to/test_file.py::test_name -``` - -The five independently-collecting suite roots: - -```bash -python3 -m pytest -q tests/fortran -m "not real_library" -python3 -m pytest -q tests/c -python3 -m pytest -q tests/docs -python3 -m pytest -q tests/tools -python3 -m pytest -q tests/workflows -``` - -For documentation-only changes, run only: - -```bash -python3 -m pytest -q tests/docs -git diff --check -``` - -Static analysis suite (blocking for any code/test/build/tooling change; run before pushing): - -```bash -python3 -m ruff check . -python3 -m ruff format --check . -python3 tools/check_static_analysis_versions.py -python3 tools/check_codegen_complexity.py # advisory -python3 -m bandit -c pyproject.toml -r prik --severity-level medium --confidence-level medium -python3 -m vulture -python3 tools/check_radon_policy.py --base-ref auto # blocking (changed code) -python3 -m radon cc prik -n C -s --total-average # advisory, still run -python3 -m radon mi prik -s # advisory, still run -``` - -Enable the tracked pre-push hook once per clone (`git config core.hooksPath .githooks`) — it runs the static-analysis gate plus focused doc/wrapper/tools/workflows smoke tests. - -Full CI-style coverage (only for explicit pre-merge/PR verification or investigating a coverage failure — mirror this exactly, a plain local `coverage run` does not match CI): - -```bash -COVERAGE_PROCESS_START=pyproject.toml PYTHONPATH=. python3 -m coverage run -m pytest -q --randomly-seed=1 -python3 -m coverage combine -python3 -m coverage report # fail_under = 90 -``` - -Run pytest with at most `-n 2` — never `-n 4`, `-n 8`, or `-n auto`. This -machine has 12 cores but only ~7 GB of RAM; each xdist worker loads NumPy -while the Fortran end-to-end tests fork gfortran and cc per test, so higher -parallelism exhausts memory, thrashes swap, and has hard-frozen the machine. -Prefer the narrowest owning test path, and commit verified work promptly -rather than batching it behind a long run. - -Don't run LAPACK wrapper tests locally unless explicitly asked (leave to GitHub Actions); ordinary local runs also exclude `real_library` generally. `examples/blas`, `examples/lapack`, `examples/fftpack`, `examples/minpack` are full-library correctness projects with their own workflows. - -Alternate-compiler toolchain lane (when changing compiler portability/native generation): - -```bash -python3 tools/run_fortran_toolchain_lane.py --compiler=/path/to/compiler -``` - -## Architecture - -PRIK is a strict pipeline: source facts flow forward through owned stages, and -**meaning moves forward only** — a downstream stage implements an upstream -decision, it never reinterprets or overrides it. - -``` -preprocessing/ -> parsers/ -> semantics/ -> policy/ -> planning/ -> codegen/ -> printers/ -> compiler/ -> runtime/ -``` - -| Package | Owns | -| --- | --- | -| `preprocessing/` | Source prep, provenance, includes, compiler-derived target/type probes | -| `parsers/` | Syntax facts only (`fortran/`, `pyi/` — a deferred `c/` frontend exists but isn't the published contributor architecture) | -| `semantics/` | Language-neutral semantic IR (`SemanticModule`); shared meaning, not a Python API or emitted code | -| `policy/` | **Every** interoperability decision: object kind, ownership, transfer, destruction, mutability/writeback, nullability, output projection, release responsibility, storage mode (stack/heap/alias), getter/setter behavior, support | -| `planning/` | Projects policy-complete IR into a deterministic, backend-neutral `ModulePlan` (`WrapperPlanner.build()`) — orders/names/validates, invents nothing | -| `codegen/` | Dispatches the plan into named C-binding and Fortran-bridge lowering mechanisms; backend scalar projection | -| `printers/` | Serializes formed nodes to C/Fortran/`.pyi` text — no behavior decisions | -| `compiler/` | Compiler commands, compile objects, native-support install, linking | -| `runtime/` | Python runtime objects + bundled native support used by generated extensions | -| `pipeline/` | End-to-end build orchestration (`build.py`, `wrapper.py`, `pyi.py`) tying the stages together | -| `contracts/` | Public names usable in semantic `.pyi` contracts (deliberately public — its import path is part of `.pyi` syntax) | -| `naming/` | Shared public-name and generated-symbol policy | -| `utilities/` | Stage-neutral helpers only (parsing/normalization/rendering/evaluation/visitor) | - -Two input routes converge at `SemanticModule` and share everything after it: -Fortran source (`preprocessing` → `parsers/fortran` → `semantics/fortran2ir.py`) -and semantic `.pyi` contracts (`parsers/pyi` → `semantics/pyi2ir.py`). - -**The hard boundary is before `WrapperPlanner.build()`.** By that point policy -must be fully decided. Binding/bridge generators (`codegen/`) must never infer -or override policy from datatype, Fortran `intent`, alias shape, storage -layout, or a local memory check, and must not add a silent fallback — if a -decision is missing, that's a bug in `policy/`, not something to patch around -in codegen. When changing behavior, prefer expressing it in completed policy -or the shared wrapper plan; touch binding/bridge lowering only when the plan -already requires a genuinely new emitted-code mechanism. - -**Deciding binding vs. bridge, or any ABI question:** ask how it would work for -a `bind(C)` procedure, where there is no bridge. A direct entrypoint has only -the binding and the user's C ABI symbol, so whatever the direct route must do -is binding-owned; the bridge owns exactly the remainder that makes an ordinary -non-`bind(C)` procedure reachable through the same completed plan. When a form -*cannot* be `bind(C)` at all — e.g. a deferred-length `character(len=:)` dummy, -which the standard rejects there because `bind(C)` character dummies must have -length 1 — that proves a generated Fortran adapter is mandatory and names what -it must construct. See AGENTS.md for the full rule. - -Array declaration expressions specifically cross packages in a fixed order: -`utilities/declaration_expressions.py` (parse/normalize text) → `semantics/` -(record native callable provenance) → `policy/` (complete support) → -`codegen/` (consume the completed plan only). - -Root entry points: `prik.__init__` exposes `build_fortran_extension`, -`build_pyi_extension`, and `__version__` only; `prik/cli.py` is the `python3 -m prik` -dispatcher into the same stage owners. Deeper docs: `docs/developer/architecture.md`, -`docs/developer/codebase-map.md`, `docs/developer/packages/*.md`. - -## Test tree - -Tests mirror the pipeline and are organized `tests////` -(stage names: `parsing/`, `probes/`, `preprocessing/`, `semantics/`, `policy/`, -`codegen/`, `printers/`, `compiling/`, `pipeline/`, `runtime/`, `end_to_end/`). -Give each test one primary invariant, placed at the earliest stage that can -prove it; add `end_to_end/` only when generation/compilation/import/runtime -behavior contributes a distinct claim. Genuinely internal (non-public-behavior) -mechanisms live under `tests/fortran/infrastructure//`, -mirroring the production module. See `tests/README.md` for the full stage -table and markers (`fortran_end_to_end`, `real_library`, `toolchain_smoke`, `property`, `regression`, `slow`, `benchmark`). - -## Working conventions (see AGENTS.md for full detail) - -- Update `CHANGELOG.md` under **Unreleased** for any user/maintainer-visible - change (public APIs, features, examples, CI/build workflow, benchmark - methodology, documented limitations). Skip it for invisible internal cleanup. -- One question, one path: when two entry points (single file vs. project, - library vs. CLI, discovery vs. compile ordering) answer the same question, - they call the same owner and differ only in inputs. Never add a second - list, regex, or route that re-derives an existing owner's answer; merge - duplicates into one owner and test both entry points against each other. -- When asked to move/change an API, import path, command, or behavior, remove - the old path — do not add compatibility shims, aliases, or fallbacks unless - explicitly asked to keep them. -- Tests are evidence for a named invariant (observable behavior, a public API, - a documented diagnostic/serialized format, ABI/ownership/lifetime/build - behavior, a stage boundary, or tooling-consumed structure) — not a freeze on - prose, private names, file inventories, or incidental layout. Remove tests - that only pin removed behavior; don't add tests that only prevent refactors. -- After finishing an implementation task, summarize which pipeline stages - actually changed (parsing / semantic IR / policy / planning / codegen / - bridge / compilation / docs) and what changed there, plus the tests - touched and how they were verified. +@AGENTS.md