feat: make AWS::LanguageExtensions local processing opt-in via --language-extensions flag#9033
Conversation
Add language_extensions_enabled kwarg to SamLocalStackProvider.get_stacks with default=False to match pre-1.160.0 behavior. Forward to expand_language_extensions and recursive get_stacks calls for nested stacks.
…alidator Wire the --language-extensions flag from validate command through to SamTemplateValidator and expand_language_extensions. Validator takes a resolved bool (not Optional) since resolution happens at command boundary. Add language_extensions parameter to __init__, forward enabled kwarg to expand_language_extensions call, and update all test call sites.
…pection-path get_stacks calls
aws-sam-cli-bot
left a comment
There was a problem hiding this comment.
Code Review Results
Reviewed: 9ffa5ed..ed0d4f9
Files: 66
Comments: 1
Comments on lines outside the diff:
[samcli/lib/sync/infra_sync_executor.py:609] [BUG] With the new opt-in semantics, _detect_foreach_code_changes can crash on templates that declare AWS::LanguageExtensions but where the user did not pass --language-extensions. The early-return guard only short-circuits when the template has no LE transform at all:
if not check_using_language_extension(current_template):
return
...
current_expanded = expand_language_extensions(
current_template,
parameter_values=current_params,
enabled=self._sync_context.language_extensions_enabled,
).expanded_templateWhen the transform is declared but enabled=False, expand_language_extensions now returns a passthrough (per the new if not enabled branch in sam_integration.py), so current_expanded still contains the unexpanded Fn::ForEach::* keys whose values are lists, not dicts. The subsequent loop then does:
for resource_id, resource_dict in current_resources.items():
if resource_id in original_regular_keys: # Fn::ForEach key is excluded from this set
continue
resource_type = resource_dict.get("Type") # AttributeError: 'list' object has no attribute 'get'This is the exact regression scenario the PR aims to support (template declares the transform, user opts out), and the call site in _auto_skip_infra_sync has no surrounding try/except, so the AttributeError propagates and breaks sam sync's auto-skip-infra path.
Add an early return for the disabled case, e.g.:
if not self._sync_context.language_extensions_enabled:
return
if not check_using_language_extension(current_template):
returnOr guard the loop body to skip is_foreach_key(resource_id) entries when expansion was a passthrough.
|
Re: PackageContext not threaded in deploy command (review 2026-05-20T17:51) Fixed in ed0d4f9 — |
|
Re: _detect_foreach_code_changes crash when LE disabled (review 2026-05-20T18:04) Fixed in 9f5934b — added early return when |
valerena
left a comment
There was a problem hiding this comment.
Just a couple of questions, but it looks good.
After #9033, language extensions are opt-in. The two ForEach zipfile tests added by this PR omitted the flag, causing SAM transform plugins to choke on unexpanded Fn::ForEach blocks.
PR #9033 made AWS::LanguageExtensions local processing opt-in. Two API changes broke three tests on this branch after the merge from develop: - expand_language_extensions() now requires a keyword-only `enabled` arg - PackageContext reads self._language_extensions_enabled, set in __init__ Pass enabled=True to expand_language_extensions in the two artifact-exporter tests, and set _language_extensions_enabled on the PackageContext stub that bypasses __init__ via __new__. All three tests exercise templates with Transform: AWS::LanguageExtensions, so enabling LE is the intended behavior.
… build merge (aws#9029) (aws#9031) * fix(cfn-lang-ext): preserve inline-source intrinsics through LE-aware build merge (aws#9029) Under `Transform: AWS::LanguageExtensions`, `sam build` was overwriting `Code: {ZipFile: !Sub ...}` with the LE-resolved value, baking default pseudo-parameter substitutions (`us-east-1`, `123456789012`) into the built template. The same merge path covered three sites; gate all three on a single artifacts-lookup primitive matching the non-LE `ApplicationBuilder.update_template` early-skip: - Root-level merge (`_update_original_template_paths`): skip resources whose `get_full_path(stack_path, key)` is absent from `artifacts`. - ForEach static branch (`_merge_static_artifact_path`): same check per expanded key. - ForEach dynamic branch (`_collect_dynamic_mapping_entries` + nested): same check; reject with `UserException` naming the non-buildable property when no iteration produced an artifact, since Mappings hold only static strings and the loop variable cannot survive deploy. Adds case A/B/C integration tests + testdata templates and unit tests covering the new skip and refusal paths. * refactor(cfn-lang-ext): address PR aws#9031 review feedback - Hoist inline `get_full_path` imports to the top of build_context.py. - Allow inline `Code: {ZipFile: !Sub ...}` inside Fn::ForEach bodies that reference the loop variable: skip the merge instead of raising, since CFN's LanguageExtensions transform expands the body at deploy time and substitutes the loop variable per iteration. Drop the unused `_raise_dynamic_no_artifacts_unsupported` helper and flip the related integration test to assert pass-through preservation. - Stop coercing `Optional[Dict[str, str]]` artifacts to `{}` in functions that only forward the value. Where it's used directly, fold the None guard into the existing membership check (`not artifacts or ...`). `_merge_static_artifact_path` becomes Optional to match its callers. * test(cfn-lang-ext): flip dynamic-no-artifact unit test to pass-through Companion to the prior commit: the dynamic-branch path no longer raises when an inline-source Fn::ForEach iteration has no build artifact. Update the unit test to assert that no Mapping is generated and the original Code (with Fn::Sub intact) is left untouched, so CFN's LanguageExtensions transform can substitute the loop variable per-iteration at deploy time. * refactor(build_context): hoist all inline imports, drop redundant `or {}` Hoist every inline import in build_context.py to the module-level import block (`itertools`, the cfn_language_extensions models/sam_integration symbols, and all language_extensions_packaging helpers). None of these introduce a circular dependency — language_extensions_packaging and cfn_language_extensions only import from samcli.lib.* and samcli.commands. validate, never from samcli.commands.build.build_context. Drop the redundant `stack_output_template_path_by_stack_path or {}` at the `_update_original_template_paths` call site: the receiver declares the parameter Optional and already coerces to {} internally. * test(integ): opt-in to --language-extensions for foreach zipfile tests After aws#9033, language extensions are opt-in. The two ForEach zipfile tests added by this PR omitted the flag, causing SAM transform plugins to choke on unexpanded Fn::ForEach blocks.
…tension functions (aws#9030) * refactor(package): replace METADATA_EXPORT_LIST and GLOBAL_EXPORT_DICT with typed registries Migrate two flat package exporter registries to typed dataclass form: - METADATA_EXPORT_LIST (list of Resource subclasses) -> METADATA_EXPORTS (List[MetadataExportSpec]). Each spec captures both the metadata_type ('AWS::ServerlessRepo::Application') and the per-property exporter classes that handle LicenseUrl/ReadmeUrl. Template._export_metadata now dispatches via a metadata_type lookup instead of a per-class RESOURCE_TYPE filter. - GLOBAL_EXPORT_DICT (dict keyed by Fn::Transform) -> GLOBAL_TRANSFORM_EXPORTS (List[GlobalTransformExportSpec]). Each spec carries a discriminator callable (e.g. _is_aws_include for matching AWS::Include) plus the handler. _export_global_artifacts now dispatches through the discriminator so future Fn::Transform variants can register without touching the walker. The typed shape is the contract a follow-up commit will read from to process AWS::Include before language-extension expansion. * feat(package): merge SAR Metadata exports back into LE child templates Extend merge_language_extensions_s3_uris with a registry-driven Metadata pass that copies rewritten property values (LicenseUrl, ReadmeUrl) from the exported template back into the original (Fn::ForEach-preserving) template after LE expansion. Without this pass, when a child stack uses Transform: AWS::LanguageExtensions and declares AWS::ServerlessRepo::Application metadata, sam package silently dropped the License/Readme S3 URLs from the merged output — they were uploaded but never wired into the template the user deploys. Implementation iterates METADATA_EXPORTS (the registry added in the prior commit) so future metadata types pick up merge support without touching the merge walker. * fix(package): process AWS::Include before language-extension expansion (aws#9027) Closes aws#9027. Symptom: when a template uses both Transform: AWS::LanguageExtensions and contains Fn::Transform: AWS::Include buried inside an LE function (e.g. Fn::ToJsonString), sam package fails to rewrite the include's Location to an S3 URL. CloudFormation then rejects the deploy with 'The location parameter is not a valid S3 uri.' Root cause: PackageContext._export ran expand_language_extensions BEFORE the artifact exporter walked Fn::Transform nodes. LE functions like Fn::ToJsonString json.dumps() their argument, collapsing the structural Fn::Transform subtree into a JSON-string literal. By the time the exporter ran, the include was no longer a structural dict node and was invisible to the global-transform walker. Fix: process AWS::Include (and any other GLOBAL_TRANSFORM_EXPORTS handler) on the original template BEFORE LE expansion runs, mirroring CloudFormation's own server-side transform ordering — CFN resolves inline Fn::Transform macros before evaluating AWS::LanguageExtensions. Implementation: - Extract Template._export_global_artifacts to a module-level _export_global_artifacts_pass(template_dict, uploader, template_dir). The instance method becomes a one-line delegate so existing callers keep working. - Call _export_global_artifacts_pass on the original template before expand_language_extensions in PackageContext._export (root flow) and in CloudFormationStackResource.do_export (nested-stack child flow). Dynamic-Location AWS::Include inside Fn::ForEach (e.g. Location: ./swagger-${Name}.yaml) is not supported by sam package: a local file path with literal ${...} placeholders does not exist on disk, so is_local_file fails and the existing InvalidLocalPathError fires — which is the right user-facing failure. CloudFormation does not substitute loop variables into Fn::Transform paths server-side either, so the limitation matches CFN's actual capability. * test(package): integration coverage for AWS::Include + SAR Metadata in LE templates Three end-to-end tests that exercise the package_context._export-equivalent flow on language-extension templates: - test_le_template_with_top_level_aws_include_merges_location verifies AWS::Include in Outputs gets its Location rewritten to s3:// after the pre-LE pass. - test_le_template_with_serverless_repo_metadata_merges_license_url verifies SAR LicenseUrl/ReadmeUrl in Metadata get merged back. - TestPackageContextIssue9027 reproduces the user template from aws#9027 (Fn::ToJsonString over Fn::Transform: AWS::Include buried inside AWS::SSM::Parameter) and asserts the buried Location is rewritten to s3://. Locks down the regression. * docs(cfn-lang-ext): document AWS::Include processing order vs language extensions Add a section explaining that sam package processes Fn::Transform: AWS::Include before language-extension expansion, mirroring CloudFormation's server-side transform ordering. This means AWS::Include Location rewrites work correctly even when the include lives buried inside language-extension functions like Fn::ToJsonString or Fn::ForEach bodies. * test(package): align LE tests with opt-in API from aws#9033 PR aws#9033 made AWS::LanguageExtensions local processing opt-in. Two API changes broke three tests on this branch after the merge from develop: - expand_language_extensions() now requires a keyword-only `enabled` arg - PackageContext reads self._language_extensions_enabled, set in __init__ Pass enabled=True to expand_language_extensions in the two artifact-exporter tests, and set _language_extensions_enabled on the PackageContext stub that bypasses __init__ via __new__. All three tests exercise templates with Transform: AWS::LanguageExtensions, so enabling LE is the intended behavior. * test(package): hoist inline imports in TestPackageContextBuriedAWSInclude Move os, Destination/Uploaders, and yaml_parse to module-level imports; drop the redundant inline tempfile/PackageContext (already at module level). Addresses inline-imports review comment on aws#9030. * refactor(package): hoist InvalidSamDocumentException and expand_language_extensions imports Move the two inline imports inside _export() to module scope. No behavior change; this is preparation for the upcoming _export() split which references these from a new branch method. * refactor(package): add _export_without_language_extensions (off-path branch) New private method that mirrors pre-1.160.0 sam-cli behavior: a tight Template.export() pipeline with no LE machinery. Unused by _export() until the dispatcher is cut over in a follow-up commit. Also adds two structural-gate smoke tests that stay red until the dispatcher is wired up — they assert that the off path never invokes expand_language_extensions or the pre-LE _export_global_artifacts_pass. * refactor(package): add _export_with_language_extensions (on-path branch) New private method containing the existing aws#9027 ordering: pre-LE include pass, LE expansion, Template.export(), merge, Mappings. Unused by _export() until the dispatcher is cut over in the next commit. * refactor(package): split _export() into LE / non-LE branches gated on flag _export() becomes a thin dispatcher: read the template, branch on self._language_extensions_enabled, dump. The two branch methods added in the prior two commits now own the actual work. Treats --language-extensions as a hard correctness boundary: when off, no LE machinery is invoked at all (no expand_language_extensions, no pre-LE _export_global_artifacts_pass, no merge, no Mappings). Template.export() still handles AWS::Include for non-LE templates via its own internal _export_global_artifacts pass — that path has worked since long before the aws#9027 fix and is unchanged by this refactor. Public surface (PackageContext._export(template_path, use_json) -> str) is preserved; all existing _export tests pass unchanged. * test(package): repoint expand_language_extensions patches to package_context use site The hoist in 756c502 moved `expand_language_extensions` to a module-level import in `package_context.py`. The use-site binding is now captured at module load, so existing tests that patched the source module `samcli.lib.cfn_language_extensions.sam_integration.expand_language_extensions` no longer intercept calls from `_export()`. Repoint two patches at the use-site `samcli.commands.package.package_context.expand_language_extensions`: - `test_phase_separation.py::test_package_context_export_calls_expand_language_extensions` was failing intermittently in CI depending on test-collection order. - `test_package_context.py::test_off_path_does_not_invoke_expand_language_extensions` is retargeted for consistency now that the inline-import shadow is gone (post-Task-4 cutover); the dead `had_language_extensions=False` scaffolding can also go since the off path no longer calls expand. Other source-module patches in `test_artifact_exporter.py` are correct as-is — those tests cover `Template.export()` paths whose use site is the artifact_exporter module. * refactor(package): hoist do_export inline imports and repoint test patches Hoist expand_language_extensions, IntrinsicsSymbolTable, generate_and_apply_artifact_mappings, and merge_language_extensions_s3_uris to module-level imports in artifact_exporter.py — preparation for the LE / non-LE structural-gate split of CloudFormationStackResource.do_export. Module-level binding requires existing tests that patched samcli.lib.cfn_language_extensions.sam_integration.expand_language_extensions (the source module) to repoint at the use-site samcli.lib.package.artifact_exporter.expand_language_extensions, since the use-site name is captured at module load and source-module patches no longer intercept once the inline import is gone. Same lesson as PR aws#9030 commit 45dea4b for package_context. * refactor(package): add _do_export_without_language_extensions branch method LE-off branch for CloudFormationStackResource: path-based Template construction with no pre-LE pass, no parameter_values, no deepcopy. Method is unused until the dispatcher rewrite — wiring lands in the follow-up commit. Adds TestDoExportLanguageExtensionsStructuralGate with two red structural-gate assertions that the dispatcher rewrite will turn green. * refactor(package): add _do_export_with_language_extensions branch method Lifts the existing LE-expansion body from CloudFormationStackResource.do_export into a dedicated method (resource_id, template_path, parent_dir, abs_template_path, resource_dict) -> Dict. No behavior change yet — the dispatcher in the follow-up commit will route LE-on calls into it. Threads resource_dict so the branch can read nested-stack Parameters without the dispatcher passing them separately. Returns the exported template dict so the dispatcher owns the final yaml_dump + upload tail. * refactor(package): split do_export into LE / non-LE branches gated on flag CloudFormationStackResource.do_export is now a thin dispatcher that: 1) does the early-exit guards (None / S3 URL / non-local file) 2) routes to _do_export_with_language_extensions or _do_export_without_language_extensions based on self.language_extensions_enabled 3) owns the final yaml_dump + upload + property mutation tail. The off path is now structurally LE-free: no expand_language_extensions call, no _export_global_artifacts_pass, no IntrinsicsSymbolTable pseudo-param plumbing, no parent_parameter_values, no copy.deepcopy. Template.export() runs its own internal _export_global_artifacts so AWS::Include still resolves on the off path. The structural-gate tests added in the previous commit are now green. Existing LE tests that rely on language_extensions_enabled = True now set the flag explicitly (where they previously depended on the LE machinery running unconditionally as a passthrough). * chore(package): apply make format to do_export refactor Black collapsed multi-line calls and signatures whose single-line form fits under the 120-char limit. No behavior change. * refactor(package): add _build_child_parameter_values helper Adds a CFN-parity helper that builds the parameter_values dict for a nested child stack: pseudo-params (with parent pseudo-name overrides honored) plus parent-rebound values via the nested-stack Parameters property. Non-pseudo parent names are not copied; child Defaults are read by the LE expander itself via parsed_template.parameters. Helper is unused production code in this commit. Wired into CloudFormationStackResource._do_export_with_language_extensions in the next commit. * fix(package): tighten parent_parameter_values scoping in LE path CloudFormationStackResource._do_export_with_language_extensions used to bulk-copy the parent stack's full parameter_values into the child, so a parent's `Foo=parent_foo` could silently shadow an unrelated child Parameter named Foo. This diverged from CloudFormation's nested-stack contract (only parent-rebound names reach the child) and from the non-LE path (which threads no parameters at all). Replace the merge block with _build_child_parameter_values, which returns pseudo-params (with parent pseudo-name overrides honored) plus parent-rebound values from the nested-stack Parameters property. Child template Defaults still resolve via the LE expander's parsed_template fallback (resolvers/fn_ref.py:_resolve_parameter). Adds end-to-end coverage: - non-pseudo parent param does not leak into child's parameter_values - child Default still resolves via resolver fallback after the change * refactor(package): trim Task 2 redundant comments and verbose test docstrings Code-review polish for the LE parent-param scoping fix: - drop call-site comment block at _do_export_with_language_extensions that duplicated _build_child_parameter_values's docstring - shorten the two new test docstrings in TestCloudFormationStackResourceChildExpansion to match neighbor style (no internal module paths) * refactor(tests): hoist inline imports added in LE parent-param fix Move imports introduced by the previous two commits from inside test bodies up to the module-level import block: - _build_child_parameter_values, IntrinsicsSymbolTable - yaml_dump (samcli.yamlhelper), shutil, inspect (stdlib) None of these symbols are @patch targets, so hoisting does not affect any mock binding (use-site patches in artifact_exporter remain correct). * refactor(tests): hoist remaining inline imports and extract LE fixtures Address review feedback on PR aws#9030: inline imports should live at module scope, and on-the-fly file writes for LE child templates should use checked-in fixtures instead of tempfile.mkdtemp() + yaml_dump(). Hoist inline imports (LanguageExtensionResult, _resolve_nested_stack_parameters, yaml_dump, shutil, the packageable_resources block) to the module-level import block. None are @patch targets, so use-site patches remain correct. Add tests/unit/lib/package/test_data/ following the precedent in tests/unit/lib/intrinsic_resolver/test_data, with TEST_DATA_PATH constant defined as Path(__file__).resolve().parent / "test_data". Convert the LE parent-param test methods, the buried-AWS::Include test, the expansion-error-handling pair, and the structural-gate pair to read their child templates (and the export-events.json include target) from the fixture tree. The on-the-fly tempdir + yaml_dump scaffolding is removed entirely along with the corresponding _write_child / _write_minimal_child helpers. No production code change; all 110 unit tests in test_artifact_exporter still pass. * refactor(tests): extract TestPackageContextBuriedAWSInclude inline fixture Replace the on-the-fly tempfile + open()+write() scaffolding in TestPackageContextBuriedAWSInclude.setUp with a checked-in fixture under tests/unit/commands/package/test_data/buried_aws_include/. Mirrors the TEST_DATA_PATH pattern already in tests/unit/lib/package/test_data and tests/unit/lib/intrinsic_resolver/test_data. setUp now resolves template_path from TEST_DATA_PATH; the export-events include target lives next to template.yaml so the relative Location in the template still resolves at sam-package time. No production code change; all 146 tests in test_package_context still pass.
…tension functions (aws#9030) * refactor(package): replace METADATA_EXPORT_LIST and GLOBAL_EXPORT_DICT with typed registries Migrate two flat package exporter registries to typed dataclass form: - METADATA_EXPORT_LIST (list of Resource subclasses) -> METADATA_EXPORTS (List[MetadataExportSpec]). Each spec captures both the metadata_type ('AWS::ServerlessRepo::Application') and the per-property exporter classes that handle LicenseUrl/ReadmeUrl. Template._export_metadata now dispatches via a metadata_type lookup instead of a per-class RESOURCE_TYPE filter. - GLOBAL_EXPORT_DICT (dict keyed by Fn::Transform) -> GLOBAL_TRANSFORM_EXPORTS (List[GlobalTransformExportSpec]). Each spec carries a discriminator callable (e.g. _is_aws_include for matching AWS::Include) plus the handler. _export_global_artifacts now dispatches through the discriminator so future Fn::Transform variants can register without touching the walker. The typed shape is the contract a follow-up commit will read from to process AWS::Include before language-extension expansion. * feat(package): merge SAR Metadata exports back into LE child templates Extend merge_language_extensions_s3_uris with a registry-driven Metadata pass that copies rewritten property values (LicenseUrl, ReadmeUrl) from the exported template back into the original (Fn::ForEach-preserving) template after LE expansion. Without this pass, when a child stack uses Transform: AWS::LanguageExtensions and declares AWS::ServerlessRepo::Application metadata, sam package silently dropped the License/Readme S3 URLs from the merged output — they were uploaded but never wired into the template the user deploys. Implementation iterates METADATA_EXPORTS (the registry added in the prior commit) so future metadata types pick up merge support without touching the merge walker. * fix(package): process AWS::Include before language-extension expansion (aws#9027) Closes aws#9027. Symptom: when a template uses both Transform: AWS::LanguageExtensions and contains Fn::Transform: AWS::Include buried inside an LE function (e.g. Fn::ToJsonString), sam package fails to rewrite the include's Location to an S3 URL. CloudFormation then rejects the deploy with 'The location parameter is not a valid S3 uri.' Root cause: PackageContext._export ran expand_language_extensions BEFORE the artifact exporter walked Fn::Transform nodes. LE functions like Fn::ToJsonString json.dumps() their argument, collapsing the structural Fn::Transform subtree into a JSON-string literal. By the time the exporter ran, the include was no longer a structural dict node and was invisible to the global-transform walker. Fix: process AWS::Include (and any other GLOBAL_TRANSFORM_EXPORTS handler) on the original template BEFORE LE expansion runs, mirroring CloudFormation's own server-side transform ordering — CFN resolves inline Fn::Transform macros before evaluating AWS::LanguageExtensions. Implementation: - Extract Template._export_global_artifacts to a module-level _export_global_artifacts_pass(template_dict, uploader, template_dir). The instance method becomes a one-line delegate so existing callers keep working. - Call _export_global_artifacts_pass on the original template before expand_language_extensions in PackageContext._export (root flow) and in CloudFormationStackResource.do_export (nested-stack child flow). Dynamic-Location AWS::Include inside Fn::ForEach (e.g. Location: ./swagger-${Name}.yaml) is not supported by sam package: a local file path with literal ${...} placeholders does not exist on disk, so is_local_file fails and the existing InvalidLocalPathError fires — which is the right user-facing failure. CloudFormation does not substitute loop variables into Fn::Transform paths server-side either, so the limitation matches CFN's actual capability. * test(package): integration coverage for AWS::Include + SAR Metadata in LE templates Three end-to-end tests that exercise the package_context._export-equivalent flow on language-extension templates: - test_le_template_with_top_level_aws_include_merges_location verifies AWS::Include in Outputs gets its Location rewritten to s3:// after the pre-LE pass. - test_le_template_with_serverless_repo_metadata_merges_license_url verifies SAR LicenseUrl/ReadmeUrl in Metadata get merged back. - TestPackageContextIssue9027 reproduces the user template from aws#9027 (Fn::ToJsonString over Fn::Transform: AWS::Include buried inside AWS::SSM::Parameter) and asserts the buried Location is rewritten to s3://. Locks down the regression. * docs(cfn-lang-ext): document AWS::Include processing order vs language extensions Add a section explaining that sam package processes Fn::Transform: AWS::Include before language-extension expansion, mirroring CloudFormation's server-side transform ordering. This means AWS::Include Location rewrites work correctly even when the include lives buried inside language-extension functions like Fn::ToJsonString or Fn::ForEach bodies. * test(package): align LE tests with opt-in API from aws#9033 PR aws#9033 made AWS::LanguageExtensions local processing opt-in. Two API changes broke three tests on this branch after the merge from develop: - expand_language_extensions() now requires a keyword-only `enabled` arg - PackageContext reads self._language_extensions_enabled, set in __init__ Pass enabled=True to expand_language_extensions in the two artifact-exporter tests, and set _language_extensions_enabled on the PackageContext stub that bypasses __init__ via __new__. All three tests exercise templates with Transform: AWS::LanguageExtensions, so enabling LE is the intended behavior. * test(package): hoist inline imports in TestPackageContextBuriedAWSInclude Move os, Destination/Uploaders, and yaml_parse to module-level imports; drop the redundant inline tempfile/PackageContext (already at module level). Addresses inline-imports review comment on aws#9030. * refactor(package): hoist InvalidSamDocumentException and expand_language_extensions imports Move the two inline imports inside _export() to module scope. No behavior change; this is preparation for the upcoming _export() split which references these from a new branch method. * refactor(package): add _export_without_language_extensions (off-path branch) New private method that mirrors pre-1.160.0 sam-cli behavior: a tight Template.export() pipeline with no LE machinery. Unused by _export() until the dispatcher is cut over in a follow-up commit. Also adds two structural-gate smoke tests that stay red until the dispatcher is wired up — they assert that the off path never invokes expand_language_extensions or the pre-LE _export_global_artifacts_pass. * refactor(package): add _export_with_language_extensions (on-path branch) New private method containing the existing aws#9027 ordering: pre-LE include pass, LE expansion, Template.export(), merge, Mappings. Unused by _export() until the dispatcher is cut over in the next commit. * refactor(package): split _export() into LE / non-LE branches gated on flag _export() becomes a thin dispatcher: read the template, branch on self._language_extensions_enabled, dump. The two branch methods added in the prior two commits now own the actual work. Treats --language-extensions as a hard correctness boundary: when off, no LE machinery is invoked at all (no expand_language_extensions, no pre-LE _export_global_artifacts_pass, no merge, no Mappings). Template.export() still handles AWS::Include for non-LE templates via its own internal _export_global_artifacts pass — that path has worked since long before the aws#9027 fix and is unchanged by this refactor. Public surface (PackageContext._export(template_path, use_json) -> str) is preserved; all existing _export tests pass unchanged. * test(package): repoint expand_language_extensions patches to package_context use site The hoist in 756c502 moved `expand_language_extensions` to a module-level import in `package_context.py`. The use-site binding is now captured at module load, so existing tests that patched the source module `samcli.lib.cfn_language_extensions.sam_integration.expand_language_extensions` no longer intercept calls from `_export()`. Repoint two patches at the use-site `samcli.commands.package.package_context.expand_language_extensions`: - `test_phase_separation.py::test_package_context_export_calls_expand_language_extensions` was failing intermittently in CI depending on test-collection order. - `test_package_context.py::test_off_path_does_not_invoke_expand_language_extensions` is retargeted for consistency now that the inline-import shadow is gone (post-Task-4 cutover); the dead `had_language_extensions=False` scaffolding can also go since the off path no longer calls expand. Other source-module patches in `test_artifact_exporter.py` are correct as-is — those tests cover `Template.export()` paths whose use site is the artifact_exporter module. * refactor(package): hoist do_export inline imports and repoint test patches Hoist expand_language_extensions, IntrinsicsSymbolTable, generate_and_apply_artifact_mappings, and merge_language_extensions_s3_uris to module-level imports in artifact_exporter.py — preparation for the LE / non-LE structural-gate split of CloudFormationStackResource.do_export. Module-level binding requires existing tests that patched samcli.lib.cfn_language_extensions.sam_integration.expand_language_extensions (the source module) to repoint at the use-site samcli.lib.package.artifact_exporter.expand_language_extensions, since the use-site name is captured at module load and source-module patches no longer intercept once the inline import is gone. Same lesson as PR aws#9030 commit 45dea4b for package_context. * refactor(package): add _do_export_without_language_extensions branch method LE-off branch for CloudFormationStackResource: path-based Template construction with no pre-LE pass, no parameter_values, no deepcopy. Method is unused until the dispatcher rewrite — wiring lands in the follow-up commit. Adds TestDoExportLanguageExtensionsStructuralGate with two red structural-gate assertions that the dispatcher rewrite will turn green. * refactor(package): add _do_export_with_language_extensions branch method Lifts the existing LE-expansion body from CloudFormationStackResource.do_export into a dedicated method (resource_id, template_path, parent_dir, abs_template_path, resource_dict) -> Dict. No behavior change yet — the dispatcher in the follow-up commit will route LE-on calls into it. Threads resource_dict so the branch can read nested-stack Parameters without the dispatcher passing them separately. Returns the exported template dict so the dispatcher owns the final yaml_dump + upload tail. * refactor(package): split do_export into LE / non-LE branches gated on flag CloudFormationStackResource.do_export is now a thin dispatcher that: 1) does the early-exit guards (None / S3 URL / non-local file) 2) routes to _do_export_with_language_extensions or _do_export_without_language_extensions based on self.language_extensions_enabled 3) owns the final yaml_dump + upload + property mutation tail. The off path is now structurally LE-free: no expand_language_extensions call, no _export_global_artifacts_pass, no IntrinsicsSymbolTable pseudo-param plumbing, no parent_parameter_values, no copy.deepcopy. Template.export() runs its own internal _export_global_artifacts so AWS::Include still resolves on the off path. The structural-gate tests added in the previous commit are now green. Existing LE tests that rely on language_extensions_enabled = True now set the flag explicitly (where they previously depended on the LE machinery running unconditionally as a passthrough). * chore(package): apply make format to do_export refactor Black collapsed multi-line calls and signatures whose single-line form fits under the 120-char limit. No behavior change. * refactor(package): add _build_child_parameter_values helper Adds a CFN-parity helper that builds the parameter_values dict for a nested child stack: pseudo-params (with parent pseudo-name overrides honored) plus parent-rebound values via the nested-stack Parameters property. Non-pseudo parent names are not copied; child Defaults are read by the LE expander itself via parsed_template.parameters. Helper is unused production code in this commit. Wired into CloudFormationStackResource._do_export_with_language_extensions in the next commit. * fix(package): tighten parent_parameter_values scoping in LE path CloudFormationStackResource._do_export_with_language_extensions used to bulk-copy the parent stack's full parameter_values into the child, so a parent's `Foo=parent_foo` could silently shadow an unrelated child Parameter named Foo. This diverged from CloudFormation's nested-stack contract (only parent-rebound names reach the child) and from the non-LE path (which threads no parameters at all). Replace the merge block with _build_child_parameter_values, which returns pseudo-params (with parent pseudo-name overrides honored) plus parent-rebound values from the nested-stack Parameters property. Child template Defaults still resolve via the LE expander's parsed_template fallback (resolvers/fn_ref.py:_resolve_parameter). Adds end-to-end coverage: - non-pseudo parent param does not leak into child's parameter_values - child Default still resolves via resolver fallback after the change * refactor(package): trim Task 2 redundant comments and verbose test docstrings Code-review polish for the LE parent-param scoping fix: - drop call-site comment block at _do_export_with_language_extensions that duplicated _build_child_parameter_values's docstring - shorten the two new test docstrings in TestCloudFormationStackResourceChildExpansion to match neighbor style (no internal module paths) * refactor(tests): hoist inline imports added in LE parent-param fix Move imports introduced by the previous two commits from inside test bodies up to the module-level import block: - _build_child_parameter_values, IntrinsicsSymbolTable - yaml_dump (samcli.yamlhelper), shutil, inspect (stdlib) None of these symbols are @patch targets, so hoisting does not affect any mock binding (use-site patches in artifact_exporter remain correct). * refactor(tests): hoist remaining inline imports and extract LE fixtures Address review feedback on PR aws#9030: inline imports should live at module scope, and on-the-fly file writes for LE child templates should use checked-in fixtures instead of tempfile.mkdtemp() + yaml_dump(). Hoist inline imports (LanguageExtensionResult, _resolve_nested_stack_parameters, yaml_dump, shutil, the packageable_resources block) to the module-level import block. None are @patch targets, so use-site patches remain correct. Add tests/unit/lib/package/test_data/ following the precedent in tests/unit/lib/intrinsic_resolver/test_data, with TEST_DATA_PATH constant defined as Path(__file__).resolve().parent / "test_data". Convert the LE parent-param test methods, the buried-AWS::Include test, the expansion-error-handling pair, and the structural-gate pair to read their child templates (and the export-events.json include target) from the fixture tree. The on-the-fly tempdir + yaml_dump scaffolding is removed entirely along with the corresponding _write_child / _write_minimal_child helpers. No production code change; all 110 unit tests in test_artifact_exporter still pass. * refactor(tests): extract TestPackageContextBuriedAWSInclude inline fixture Replace the on-the-fly tempfile + open()+write() scaffolding in TestPackageContextBuriedAWSInclude.setUp with a checked-in fixture under tests/unit/commands/package/test_data/buried_aws_include/. Mirrors the TEST_DATA_PATH pattern already in tests/unit/lib/package/test_data and tests/unit/lib/intrinsic_resolver/test_data. setUp now resolves template_path from TEST_DATA_PATH; the export-events include target lives next to template.yaml so the relative Location in the template still resolves at sam-package time. No production code change; all 146 tests in test_package_context still pass.
Summary
AWS::LanguageExtensionsprocessing opt-in (off by default), restoring pre-1.160.0 behavior for templates that declare the transform but don't need local expansion--language-extensions / --no-language-extensionsCLI flag to all 8 affected commands: build, package, deploy, sync, validate, local invoke, local start-api, local start-lambdaSAM_CLI_ENABLE_LANGUAGE_EXTENSIONS=1env var as fallback (flag wins)samconfig.tomlpersistence vialanguage_extensions = trueMotivation
SAM CLI 1.160.0 enabled local processing of
AWS::LanguageExtensionsimplicitly when the transform was declared. This caused regressions (#9004, #9005, #9027, #9029) for templates that declared the transform for non-Fn::ForEachreasons or as a forward-compat placeholder. Making it opt-in lets customers adopt at their own pace.Changes
expand_language_extensions()now requiresenabled: boolas a keyword-only arg — unwired callers fail with TypeErrorresolve_language_extensions_enabled(flag_value: Optional[bool]) -> boolresolver (flag → env var → False)@language_extensions_optionClick decorator shared across all commandsSamLocalStackProvider.get_stacks(..., language_extensions_enabled=False)kwarg threaded through recursionTemplateclass in artifact_exporter wired for nested-stack recursionFalse--language-extensionsfor ForEach templatesTest plan
pytest tests/unit/ -q -W ignore::pytest.PytestUnknownMarkWarning)--language-extensionsparameter registered on all 8 commands (verified programmatically)make formatcleanmake lintshows zero errorstest_build_without_flag_does_not_expand_foreachverifies opt-out behavior--language-extensionsflag