codex: update parser grammar documentation

Author: saidctb

README.md

@@ -453,15 +453,20 @@ The parser exposes stable file/project entrypoints:
 - `parse_fortran_project(...)` for many sources returning `FortranProject`.
 - `assess_wrap_readiness(...)` for wrappability diagnostics.
 
-Internally, `FortranParser.visit_file` uses a recursive source-unit parser:
-the file is sliced into direct modules/submodules/programs/procedures/block
-data/interfaces/types, then each unit visitor parses only its own substring and
-recurses into direct children. Shared declaration helpers parse variables,
-procedure arguments/results, and type fields, then push them into the active
-scope. Procedure execution bodies and internal subprograms are ignored for
-wrapper metadata; procedure-local interfaces are retained for callback typing.
+Internally, `FortranParser.visit_file` uses a recursive grammar-style
+source-unit parser. The file is first sliced into direct
+modules/submodules/programs/procedures/block-data/interfaces/types. Each unit
+visitor then parses only its own substring, splits it into header,
+specification, optional execution, and optional `contains` regions, and recurses
+into direct child units where that grammar allows children. Shared declaration
+helpers parse variables, procedure arguments/results, and type fields, then
+push them into the active scope. Procedure execution bodies and internal
+subprograms are ignored for wrapper metadata; procedure-local interfaces are
+retained for callback typing.
 Parameter variables keep both `value` (best resolved value) and runtime
 `symbolic_value` (the original expression) when the parser has that information.
+Module-level parameters used in procedure argument shapes remain symbolic in
+the signature while still being valid scoped references for readiness checks.
 
 The semantics layer consumes `FortranFile`/`FortranModule` objects and projects them into language-independent semantic IR (`SemanticModule`, `SemanticFunction`, `SemanticClass`, `SemanticType`). This keeps the semantic API model independent from parser internals, matching the project goal that parser output is a helper and the semantic interface/IR is the source of truth.
 

fortran_parser.md

@@ -85,8 +85,9 @@ sections so maintainers can navigate the file by concern instead of by history:
 - Module-level helper blocks (source-form rules, preprocessor logic,
   diagnostics, compile-time expression resolution, dependency ordering)
 - `FortranParser` internals grouped by domain:
-  - visitor-style API entrypoints (`visit_file`, `visit_project`,
-    `visit_wrap_readiness`)
+  - internal visitor entrypoints (`visit_file`, `visit_project`,
+    `visit_wrap_readiness`). The public API remains the module-level wrappers
+    listed above.
   - source-unit visitors for files, modules, submodules, programs,
     procedures, interfaces, derived types, and block data
   - recursive source-unit slicing (`header`, specification part, execution
@@ -101,10 +102,17 @@ sections so maintainers can navigate the file by concern instead of by history:
 
 `visit_file` is the central orchestration path. It first slices the source into
 direct file-level units, then each unit visitor parses only its own substring
-and recursively slices direct children. Procedure execution parts are ignored
-for wrapper metadata, and procedure-internal subprograms are not exported as
-file/module procedures. Procedure-local interface blocks are still visited
-enough to type callback dummy arguments and to preserve interface metadata.
+and recursively slices direct children. This is the key parser design: each
+Fortran grammar unit has a header, a specification region, optional execution
+region, and optional `contains` region. The differences between modules,
+programs, procedures, derived types, interfaces, and block data are expressed
+by small visitor decisions and grammar flags rather than separate whole-file
+parsing loops.
+
+Procedure execution parts are ignored for wrapper metadata, and
+procedure-internal subprograms are not exported as file/module procedures.
+Procedure-local interface blocks are still visited enough to type callback
+dummy arguments and to preserve interface metadata.
 
 ### 2.1 Recursive parser sketch
 
@@ -171,6 +179,13 @@ folding; `symbolic_value` preserves the original parameter initializer for
 validation, debugging, and downstream diagnostics without changing the legacy
 JSON fixture shape.
 
+Procedure-local parameters may be folded into argument shapes during procedure
+finalization. Module-level and `use`-associated parameters used in procedure
+argument shapes are kept symbolic in the signature (`x(n)` remains `["n"]`)
+and are treated as valid scope references for readiness checks. Module/program
+variable shapes and parameter values can be resolved through the compile-time
+resolver when enough information is available.
+
 ## 3) Terminal usage and expected outputs
 
 ### 3.1 Basic CLI invocation

fortran_parser/parser.py

@@ -26,13 +26,17 @@
      file/project models.
 
 Recommended reading order for maintainers:
-- Start from `FortranParser.visit_file` / `visit_project`
+- Start from the module-level public wrappers (`parse_fortran_file`,
+  `parse_fortran_project`, `assess_wrap_readiness`)
+- Then read `FortranParser.visit_file` / `visit_project`
 - Then read the high-level unit visitor methods at the top of the class
 - Then drill into `_helper_*` implementations and low-level helpers
 
 `FortranParser` class layout (top -> bottom):
-- Visitor API: `visit_file`, `visit_project`, `visit_wrap_readiness`, plus
-  small `visit_*_unit` methods for each already-sliced source unit.
+- Internal visitor entrypoints: `visit_file`, `visit_project`,
+  `visit_wrap_readiness`, plus small `visit_*_unit` methods for each
+  already-sliced source unit. The stable public API is the module-level wrapper
+  layer at the bottom of the file.
 - Core `_helper_*` methods for source slicing, grammar-region splitting,
   scoped specification-part visits, declaration parsing, symbol pushing,
   preprocessor branch selection, and same-level duplicate checks.
@@ -1629,33 +1633,48 @@ class FortranParser:
 
     State carried on the instance:
     - `macro_defines`: optional macro-selection configuration used while
-      collecting procedures when conditional branches are present.
+      selecting preprocessor branches before source-unit slicing.
 
     Parsing pipeline used by `visit_file`:
     1. Preprocess source into normalized lines (`_preprocessed_lines`).
-    2. Parse signatures/types/interfaces/program units.
-    3. Attach parsed members to owning module/submodule scopes.
-    4. Build `FortranFile` symbol table and standalone entity lists.
+    2. Slice direct file-level source units (`module`, `submodule`,
+       `program`, standalone `procedure`, `block data`, file-level
+       `interface`, and file-level derived type).
+    3. Dispatch each `_SourceUnit` to a small `visit_*_unit` method.
+    4. Each unit visitor parses only that unit's own substring, builds its own
+       `_ParserScope`, splits the unit into grammar regions, visits the
+       specification part, and recursively slices direct children where the
+       grammar allows them.
+    5. Shared declaration helpers push variables, procedure symbols, and type
+       fields into the active scope model.
+    6. Build `FortranFile` symbol table and standalone entity lists.
 
     Class section map:
-    - Public API methods first (developer discovery).
-    - High-level unit parsing methods next (top-down by Fortran block size).
-    - Internal `_helper_*` methods after that (full scoped parsing logic).
+    - Internal visitor entrypoints first (developer discovery).
+    - Unit visitors next (one visitor per grammar-level source unit).
+    - Internal `_helper_*` methods after that (reusable scoped parsing logic).
     - Lower-level declaration/header helpers and assembly utilities last.
 
     Scope behavior summary:
-    - `current_module` tracks ownership for procedures/types/interfaces.
-    - `interface_depth`/stacks track when declarations belong to interface blocks.
-    - Per-procedure state tracks declaration-part vs executable-part boundaries.
-    - Type parsing tracks `contains` sub-region for bindings/generics vs fields.
-    - Program/module/submodule parsers collect specification-part declarations
-      and stop collecting variable declarations after `contains`.
+    - `_ParserScope` is passed explicitly into shared helpers; there is no
+      ambient `current_module` or interface stack.
+    - Module/submodule scopes own contained procedures, interfaces, and derived
+      types; program and block-data scopes collect their specification
+      variables only.
+    - Procedure scopes parse only wrapper-relevant specification declarations;
+      execution statements and internal procedures after `contains` are
+      ignored, except procedure-local interfaces are revisited to type callback
+      dummy arguments.
+    - Derived-type scopes parse fields in the specification region and
+      type-bound procedure/generic bindings in the `contains` region.
+    - Same-level unit names are validated by the slicer with preprocessor
+      branch-awareness, while identical names in different scopes remain valid.
 
     `visit_project` composes multiple `FortranFile` objects into one
     `FortranProject` registry and validates duplicate symbols by scope.
     """
     # ------------------------------------------------------------------
-    # Public API (kept first for discoverability)
+    # Internal visitor entrypoints (kept first for developer discovery)
     # ------------------------------------------------------------------
 
     def __init__(self, macro_defines: set[str] | dict[str, int | bool | str] | None = None):

parser_implementation_reference.md

@@ -105,7 +105,10 @@ another source language.
   including renamed imports from parsed modules. Safely evaluable arithmetic
   parameter chains are folded to their final integer kind; compiler-dependent
   intrinsics such as `selected_real_kind(...)` remain as resolved expressions.  
-- Shape expressions resolved using available symbol dictionary.  
+- Procedure-local parameter expressions can be folded into argument shapes
+  during procedure finalization. Module-level and `use`-associated parameters
+  used in procedure argument shapes remain symbolic in the signature while
+  still being considered valid scope references for readiness diagnostics.
 - Module/program variable parameter values, character lengths, and shapes are
   resolved through the same cached compile-time resolver where safe.
 - The resolver caches symbol and expression results per scope and evaluates a
@@ -230,6 +233,8 @@ Covers, among others:
 - storage directives (`save`, `common`) in procedures are recognized and skipped as non-declaration statements
 - module variable and `use` parsing
 - module children attachment (procedures/types/interfaces)
+- executable parser-internals tutorial in
+  `tests/parser/test_parser_developer_tutorial.py`
 - ignoring local vars in external signatures
 - external callback declarations (including typed `real, external :: f`) under `implicit none`
 - ignoring internal procedures in `contains`
@@ -345,13 +350,15 @@ ask it to implement each of these layers explicitly:
 11. Project namespace parser with dependency ordering.  
 12. Readiness validator with unsupported-pattern rules + unknown type checks.  
 13. CLI with tree output + JSON output + file emission.  
-11. Unit tests per feature + fixture/golden regression suite + golden
+14. Unit tests per feature + fixture/golden regression suite + golden
     regeneration script.  
 
 
 ### 6.1 Parser control-flow example
 
-Use this as the mental model when changing `fortran_parser/parser.py`:
+Use this as the mental model when changing `fortran_parser/parser.py`: parsing
+is recursive over source units, and each unit is handled by the same grammar
+shape before grammar-specific exceptions are applied.
 
 ```fortran
 module m
@@ -384,6 +391,20 @@ The control flow is:
    build a scope, split the unit, visit the relevant specification part, and
    push declarations into that scope.
 
+The grammar shape is the design rule:
+
+- every sliced source unit has a header and a specification part
+- procedures and programs can also have an execution part
+- modules, submodules, programs, procedures, and derived types can have a
+  `contains` part, but visitors decide whether children in that region matter
+  for wrapping
+- block data is specification-only
+- interfaces use the same child-slicing mechanism for procedure declarations
+
+That means new parser behavior should usually be implemented by extending the
+grammar profile, unit splitter, or shared specification/declaration helpers,
+not by adding a new whole-file scan.
+
 Declaration parsing is deliberately shared. Module variables, program/block
 data variables, procedure arguments/results, and derived-type fields all call
 `_helper_parse_declaration_line`, then `_helper_push_declaration_to_scope`.
@@ -443,6 +464,11 @@ implemented today:
 - **Module specification scope tracking**: module `parameter` collection is
   restricted to the module specification part and stops at `contains`, avoiding
   leakage from executable regions.
+- **Module parameter references in contained procedure shapes**: a contained
+  procedure argument such as `real :: x(n)` may refer to a module-level
+  parameter `n`. The signature keeps the shape token symbolic (`"n"`) while
+  readiness validation treats it as a valid scoped reference. This protects
+  module-level parameters from being mistaken for undeclared procedure locals.
 - **Interface scope tracking**: procedures parsed inside `interface ... end
   interface` are represented separately and flagged as interface procedures.
   Interface-local argument declarations do not conflict with host declarations;