diff --git a/docs/skills/dx-sap-spartacus-import.md b/docs/skills/dx-sap-spartacus-import.md index 36e952a..cac56bb 100644 --- a/docs/skills/dx-sap-spartacus-import.md +++ b/docs/skills/dx-sap-spartacus-import.md @@ -21,7 +21,7 @@ When a developer has added the connector to a Spartacus app (`ng add @contentsta - **Idempotent seed and publish**: entries upsert by their SAP uid, so re-runs update in place. - **Verification**: a round-trip comparison against SAP and a Delivery API read of the home page. - **Localization**: localized content and images per language from SAP's own localized responses. -- **SAP product picker**: optionally installs and configures the SAP Commerce Cloud marketplace app, so editors pick carousel products from SAP instead of typing SKUs. The agent checks the picker, repairs it at most twice, then lets the user fall back to the plain SKU list. +- **SAP product picker**: optionally installs and configures the SAP Commerce Cloud marketplace app, so editors pick carousel products from SAP instead of typing SKUs. It works for B2C and B2B sites (for example the electronics and powertools demos), and is set up only when the installed connector can render it: the npm connector 0.1.0 keeps the plain SKU list, while a connector that reads `sap_products` gets the picker. Before installing, a preflight checks that the app's SAP server has a trusted certificate, the base site, a working product search and the carousel codes, and stops the run if not. After the run the agent checks the picker, repairs it at most twice, then lets the user fall back to the plain SKU list. - **Your own login**: writes with a management token, or with the user's `csdx auth:login --oauth` session (required for the marketplace app). ## Example prompts @@ -31,6 +31,7 @@ When a developer has added the connector to a Spartacus app (`ng add @contentsta - "Re-run the SAP import; we changed the homepage banners." - "The import says a component type is unknown. What do I do?" - "Let our editors pick the carousel products from SAP instead of typing SKUs." +- "Set up the SAP product picker on our powertools B2B demo stack too." ## References diff --git a/evals/routing-dx-sap-spartacus-import-3/graders/loads-dx-sap-spartacus-import.md b/evals/routing-dx-sap-spartacus-import-3/graders/loads-dx-sap-spartacus-import.md new file mode 100644 index 0000000..cb33de8 --- /dev/null +++ b/evals/routing-dx-sap-spartacus-import-3/graders/loads-dx-sap-spartacus-import.md @@ -0,0 +1,7 @@ +--- +type: tool_used +tool: Skill +input_match: '"skill"\s*:\s*"(?:[\w-]+:)?dx-sap-spartacus-import"' +min: 1 +--- +The agent loads the `dx-sap-spartacus-import` skill. diff --git a/evals/routing-dx-sap-spartacus-import-3/prompt.md b/evals/routing-dx-sap-spartacus-import-3/prompt.md new file mode 100644 index 0000000..e4e7ab6 --- /dev/null +++ b/evals/routing-dx-sap-spartacus-import-3/prompt.md @@ -0,0 +1,7 @@ +--- +name: routing-dx-sap-spartacus-import-3 +tags: [routing, dx-sap-spartacus-import] +max_turns: 3 +timeout_seconds: 180 +--- +We imported our SAP powertools-spa B2B storefront into Contentstack with the Spartacus connector. Now set up the SAP Commerce Cloud product picker on the powertools carousels, like we did for electronics. diff --git a/skills/dx-sap-spartacus-import/SKILL.md b/skills/dx-sap-spartacus-import/SKILL.md index c05be3d..0759d1b 100644 --- a/skills/dx-sap-spartacus-import/SKILL.md +++ b/skills/dx-sap-spartacus-import/SKILL.md @@ -28,35 +28,38 @@ Use another skill for: Confirm, without asking for secret values: 1. The current directory is the Spartacus project root and `node_modules/@contentstack/contentstack-spartacus-connector/import-export/starter-pack/content_types` exists. If not, the connector isn't installed: run `ng add @contentstack/contentstack-spartacus-connector` first, or point `CONNECTOR_STARTER_PACK` at a connector checkout. -2. The SAP baseSite uid (for example `electronics-spa`) and the OCC v2 base URL (`https://HOST/occ/v2`). List baseSites with `curl -s "$OCC_BASE_URL/basesites"`. +2. The SAP baseSite uid (for example `electronics-spa`) and the OCC v2 base URL (`https://HOST/occ/v2`). List baseSites with `curl -s "$OCC_BASE_URL/basesites"` (add `-k` only when the user set `OCC_INSECURE_TLS=1` for a self-signed dev server). 3. The target stack is **new or dedicated to this import**. The publish step publishes every entry and asset in the stack. 4. `python3 --version` is 3.9 or later. The Python scripts use only the standard library. 5. How the run writes: with `CS_MANAGEMENT_TOKEN`, or, without one, as the user's own `csdx auth:login --oauth` session. The SAP Commerce Cloud app step always needs that csdx session (a management token can't install apps), plus Node.js and `@contentstack/cli` on `PATH`. +6. For the product picker: the SAP server the app will search (`SAP_APP_API_BASE_URL`, with a publicly trusted certificate), its base site (`SAP_APP_BASE_SITE`: `electronics` for the electronics demo, `powertools-spa` for the B2B powertools demo) and its Backoffice URL. It must carry the same catalog as the storefront's OCC. See [references/sap-commerce-app.md](references/sap-commerce-app.md). ## Instructions 1. Resolve ``: the folder this `SKILL.md` was loaded from. Run every script by absolute path, `python3 "/scripts/orchestrate.py"`, from the Spartacus project root. -2. Ask the user to export the variables under Environment Variables in their own shell. Never ask them to paste values into chat. -3. **Dry run** (writes nothing): +2. Ask the user to export the variables under Environment Variables in their own shell. Never ask them to paste values into chat. Required: `CS_API_KEY` and `OCC_BASE_URL`. Optional: `OCC_INSECURE_TLS`, `CS_MANAGEMENT_TOKEN` (else the csdx login), the `SAP_APP_*` picker settings, and `CS_DELIVERY_TOKEN` (only for the final delivery check; a fresh stack gets one after the run). +3. **Dry run** (writes nothing, about 2 minutes). Pass the same `--locales` and `--run-dir` you'll use for the write, so the gate reports show the full plan: ```bash - python3 "/scripts/orchestrate.py" --site electronics-spa + python3 "/scripts/orchestrate.py" --site electronics-spa \ + --locales de=de-de,ja=ja-jp,zh=zh-cn --run-dir ``` It extracts the site, builds the plan and prints both gate reports. Summarize them for the user: pages, component entries, assets, which slots stay with SAP, any unknown component types, where carousel SKUs go (`products` or `sap_products`), and whether writes run as the management token or the csdx login. 4. **Gate 1, model approval.** If the report lists unknown typeCodes, stop. Each one is either authored (add it to `component_map` in `scripts/registry/connector.registry.json` with its connector content type and field mapping) or functional (add it to `occ_only_types.types`). Re-run the dry run. Pass `--approve-model` only after the user agrees. -5. **Gate 2, pre-publish.** Show the counts and the target stack and environment. The run stops here if the plan writes a content type or field the installed starter pack lacks. The fix is a newer connector (`npm i @contentstack/contentstack-spartacus-connector@latest`), not editing the stack by hand. -6. With the user's explicit yes, run the write: +5. **Gate 2, pre-publish.** Show the counts and the target stack and environment: gate 2 prints the stack's name and whether it's empty, the content types (the connector's full starter pack, which replaces existing types with the same uids), and components, navigation entries and pages separately. The run stops here if the plan writes a content type or field the installed starter pack lacks. The fix is a newer connector (`npm i @contentstack/contentstack-spartacus-connector@latest`), not editing the stack by hand. +6. With the user's explicit yes, run the write. It takes a while (a demo site with three languages: about 45 minutes, roughly 10 per language), so run it in the background with its output in a log, check the log until it ends, and never start a second write while one is running: ```bash python3 "/scripts/orchestrate.py" --site electronics-spa \ - --approve-model --approve-publish --env development + --approve-model --approve-publish --env development \ + --locales de=de-de,ja=ja-jp,zh=zh-cn --run-dir > /write.log 2>&1 ``` This sets up the SAP Commerce Cloud app when carousels use the picker, imports the starter-pack content types, uploads images, upserts entries, creates the environment if needed, publishes, and runs the verify steps. -7. **Carousel products.** With `SAP_APP_API_BASE_URL`, `SAP_APP_BASE_SITE` and `SAP_APP_BACKOFFICE_URL` exported, carousels go into `sap_products`, the SAP Commerce Cloud app's product picker; otherwise into the plain `products` SKU list. Force either with `--carousel-field products|sap_products`. After a picker run, follow "Verify, retry, then ask" in [references/sap-commerce-app.md](references/sap-commerce-app.md): check the picker, repair at most twice, then let the user choose between fixing it and falling back to `products`. +7. **Carousel products.** Carousels go into `sap_products`, the SAP Commerce Cloud app's product picker, when `SAP_APP_API_BASE_URL`, `SAP_APP_BASE_SITE` and `SAP_APP_BACKOFFICE_URL` are exported **and** the installed connector reads `sap_products` (the connector release, or a build or checkout of the connector repo's main branch). Otherwise they go into the plain `products` SKU list: the npm connector 0.1.0 reads only `products`, so with it the picker isn't set up and gate 2 says why. The version number can't tell them apart (a build from main also says 0.1.0), so the orchestrator checks the connector's code. Force either with `--carousel-field products|sap_products`. It works the same for B2C and B2B sites. Gate 2 runs a preflight on the picker's SAP server (trusted certificate, base site present, product search returns products without a login, carousel codes found); if it fails, the write stops with exit code 4 before the app is installed. Quote the `FAIL` line to the user with its fix. If the picker is forced onto a connector that reads only `products`, gate 2 warns that the carousels would render empty; tell the user before the write. You may have no browser for the picker check: then ask the user to do it (open a carousel entry, check the SAP Products field lists products, click Add Product(s)). After a picker run, follow "Verify, retry, then ask" in [references/sap-commerce-app.md](references/sap-commerce-app.md): check the picker, repair at most twice, then let the user choose between fixing it and falling back to `products`. 8. For more languages, add `--locales de=de-de,ja=ja-jp,zh=zh-cn` (SAP isocode = Contentstack locale). Locales are created when missing, with the master locale as fallback. -9. Report the result from `manifest.json` in the run directory, then point the user at the app config: `cmsPageContentType: 'landing_page'`, a `localeMapping` that matches the locales imported, and `occFallback: true`. +9. Report the result from `manifest.json` in the run directory. `warn` on seed, publish or a locale means some items failed (their ERR lines are in the output); re-running is safe. A `check` on the delivery read with "no CS_DELIVERY_TOKEN yet" is expected on a fresh stack: tell the user to create a delivery token for the environment and re-run `verify/cda_read.py`, as the manifest note says. Then point the user at the app config: `cmsPageContentType: 'landing_page'`, a `localeMapping` that matches the locales imported, and `occFallback: true`. ### How the pipeline works -`extract → nav-resolve → model → gate 1 → gate 2 → sap-app → seed → publish → verify → localize`. Stage details, run-directory files and flags are in [references/pipeline.md](references/pipeline.md). The mapping rules (which SAP templates, slots and typeCodes become which content types) are in [references/mapping-profile.md](references/mapping-profile.md). The SAP Commerce Cloud app stage, its auth and its recovery steps are in [references/sap-commerce-app.md](references/sap-commerce-app.md). +`extract → nav-resolve → model → gate 1 → gate 2 (+ SAP picker preflight) → sap-app → seed → publish → verify → localize`. Stage details, run-directory files and flags are in [references/pipeline.md](references/pipeline.md). The mapping rules (which SAP templates, slots and typeCodes become which content types) are in [references/mapping-profile.md](references/mapping-profile.md). The SAP Commerce Cloud app stage, its auth and its recovery steps are in [references/sap-commerce-app.md](references/sap-commerce-app.md). ### Rules @@ -109,7 +112,7 @@ Never print, echo, log or store the management token, delivery token, API key or - `CS_MASTER_LOCALE`: defaults to `en-us`. - `CS_PREVIEW_URL`: storefront URL set on the environment for every locale, so Live Preview loads (default `http://localhost:4200`). - `CONNECTOR_STARTER_PACK`: override the starter-pack path. -- `SAP_APP_API_BASE_URL` (host, no scheme), `SAP_APP_BASE_SITE`, `SAP_APP_BACKOFFICE_URL`: the SAP server the app's picker searches. Setting all three turns on the picker. +- `SAP_APP_API_BASE_URL` (host, no scheme), `SAP_APP_BASE_SITE` (for example `electronics` or `powertools-spa`), `SAP_APP_BACKOFFICE_URL`: the SAP server the app's picker searches. Setting all three turns on the picker. - `SAP_APP_API_ROUTE` (default `/occ/v2/`), `SAP_APP_CONFIG_NAME` (default `hti`), and optionally `SAP_APP_UID`, `CS_ORG_UID`. ## References @@ -126,5 +129,8 @@ Never print, echo, log or store the management token, delivery token, API key or User: Import our SAP electronics storefront into my new stack. Assistant: Runs the dry run, then reports: "6 pages (/, /faq, /contact, /sale, shared /product and /category layouts), 34 components, 79 navigation entries, 37 images. SearchBox, MiniCart, SiteContext and SiteLinks stay with SAP. No unknown component types. Gate 2 will write to stack 'SAP demo' and publish everything in it to 'development'. Shall I run it?" and waits. +User: Set up the SAP product picker on our powertools demo stack too. +Assistant: Asks for the SAP server and Backoffice URL the app should search (base site `powertools-spa`), runs the dry run with the `SAP_APP_*` variables exported, and reports the preflight: "powertools-spa exists, search returns 252 products, 14 of 14 carousel codes found." Then it asks before the write, which installs and configures the app once, fills both carousels' `sap_products`, and checks the picker in the browser. + User: Just publish it, skip the dry run. Assistant: Explains that the write step only runs after the gate reports have been reviewed, runs the dry run (it writes nothing and takes about a minute), and asks for the go-ahead with the counts in front of the user. diff --git a/skills/dx-sap-spartacus-import/references/pipeline.md b/skills/dx-sap-spartacus-import/references/pipeline.md index 7168af8..1daf874 100644 --- a/skills/dx-sap-spartacus-import/references/pipeline.md +++ b/skills/dx-sap-spartacus-import/references/pipeline.md @@ -14,13 +14,13 @@ extract ─▶ nav-resolve ─▶ model ─▶ [gate 1] ─▶ [gate 2 + pack ch | 2 | nav-resolve | `storefront-extract/nav_resolve.py` | Resolves navigation link entries into full link detail and finds the navigation roots. Adds `navigationLinks` and `navigationRoots` to the inventory. Skipped when `--inventory` is reused without an OCC base. | | 3 | model | `content-model/model_connector.py` | Maps the inventory onto the connector model with `registry/connector.registry.json`. Writes `connector-plan.json` (entries, assets, content types needed) and `connector-report.json`. Drops page slots the target content type lacks in the starter pack. With `--carousel-field sap_products`, carousel SKUs become a `sap_products` placeholder the seed resolves. | | – | gate 1 | orchestrator | Blocks when the site uses a component typeCode the profile doesn't know. Pass with `--approve-model` once it's mapped. | -| – | gate 2 | orchestrator | Prints what will be written. Checks that every content type and field the plan writes exists in the starter pack, and exits with code 3 if not. Without `--approve-publish` the run ends here as a dry run. | +| – | gate 2 | orchestrator | Names the target stack and says whether it's empty (`cs-seed/stack_info.py`, read-only), then prints what will be written: the connector's full starter pack of content types, assets, and components, navigation entries and pages separately. Checks that every content type and field the plan writes exists in the starter pack, and exits with code 3 if not. With carousels in `sap_products`, also runs the SAP picker preflight (`marketplace-app/sap_preflight.py`) and, on `--approve-publish`, exits with code 4 before any write if it fails. Without `--approve-publish` the run ends here as a dry run. | | 4 | sap-app | `marketplace-app/sap_app.py` (+ `sap_app.cjs`) | Only when carousels use `sap_products`: installs and configures the SAP Commerce Cloud app on the user's csdx login, finds its Product field extension, writes `sap-app.json`. See [sap-commerce-app.md](sap-commerce-app.md). | -| 5 | seed | `cs-seed/cma_seed_connector.py` | Imports the starter-pack content types (adds a `sap_uid` field where the pack lacks one, and the `sap_products` picker field on the carousel when the app is set up), uploads images, then upserts component entries, the `global_slots` shell entry and page entries in reference order. | -| 6 | publish | `cs-seed/publish.py` | Creates the environment when missing (with a Live Preview URL per locale) and publishes every entry and asset in the stack to it in the master locale. | +| 5 | seed | `cs-seed/cma_seed_connector.py` | Imports the starter-pack content types (adds a `sap_uid` field where the pack lacks one, and the `sap_products` picker field on the carousel when the app is set up), uploads images, then upserts component entries, the `global_slots` shell entry and page entries in reference order. Exits 3 when items failed (recorded as `warn`; the rest is written). | +| 6 | publish | `cs-seed/publish.py` | Creates the environment when missing (with a Live Preview URL per locale) and publishes every entry and asset in the stack to it in the master locale (`CS_MASTER_LOCALE`, default `en-us`). Ends with a totals line; exits 3 when some publishes failed (recorded as `warn`). | | 7 | verify, round trip | `verify/verify_connector.py` | Reads the seeded entries back and compares each with its original OCC component, including the rebuilt navigation tree. | -| 8 | verify, delivery | `verify/cda_read.py` | Reads the home page (`landing_page`, url `/`) through the Delivery API with `CS_DELIVERY_TOKEN`. | -| 9 | localize | `cs-seed/localize.py` | Per `--locales` pair: a localized extract and model, the locale's images, a localized version of every entry, and a publish in that locale. | +| 8 | verify, delivery | `verify/cda_read.py` | Reads the home page (`landing_page`, url `/`) through the Delivery API with `CS_DELIVERY_TOKEN`. Without a token (always the case on a fresh stack) it exits 5 and the stage is `check` with a note: create a token for the environment, then re-run this script. | +| 9 | localize | `cs-seed/localize.py` | Per `--locales` pair: a localized extract and model, the locale's images, a localized version of every entry, and a publish in that locale. A failed entry update is retried once; if anything still fails the locale is recorded as `warn` and named in the final banner. Re-running it is safe. | `cs-seed/cma_seed.py` holds the shared Content Management API helpers the seed, publish and localize stages import. `cs-seed/cs_auth.py` picks the auth header: `CS_MANAGEMENT_TOKEN`, or the user's csdx session through `cs-seed/csdx_auth.cjs` (refreshed once after a 401). `marketplace-app/use_products.py` is the user-chosen fallback from the picker back to `products`. @@ -35,7 +35,7 @@ extract ─▶ nav-resolve ─▶ model ─▶ [gate 1] ─▶ [gate 2 + pack ch | `--env` | `development` | Target environment for publishing. | | `--approve-model` | off | Pass gate 1. | | `--approve-publish` | off | Pass gate 2 and write. | -| `--carousel-field` | `auto` | `sap_products` when the `SAP_APP_*` configuration is exported, else `products`. | +| `--carousel-field` | `auto` | `sap_products` when the `SAP_APP_*` configuration is exported and the installed connector reads `sap_products` (checked in its code), else `products`. | | `--skip-app-install` | off | The SAP app is already set up: only discover its field extension. | | `--reinstall-app` | off | Uninstall, reinstall and configure the SAP app once (repairs a broken install). | | `--locales` | none | `sapIso=csLocale` pairs, for example `de=de-de,ja=ja-jp,zh=zh-cn`. | @@ -53,6 +53,8 @@ extract ─▶ nav-resolve ─▶ model ─▶ [gate 1] ─▶ [gate 2 + pack ch - `0`: done, or dry run complete. - `2`: gate 1 blocked on unknown typeCodes. - `3`: gate 2 blocked on a starter-pack mismatch. +- `4`: gate 2 blocked because the SAP picker preflight failed (nothing installed or written). +- Per-stage `warn` (seed, publish, a locale) doesn't stop the run: some items failed and are listed as ERR lines; re-running is safe. - Anything else: a stage failed. Its output is printed and recorded in `manifest.json`. ## Common problems @@ -61,6 +63,9 @@ extract ─▶ nav-resolve ─▶ model ─▶ [gate 1] ─▶ [gate 2 + pack ch - **Starter pack not found**: run from the Spartacus project root after `ng add @contentstack/contentstack-spartacus-connector`, or set `CONNECTOR_STARTER_PACK`. - **Gate 2 pack mismatch**: the installed connector is older than the mapping profile expects. Update the connector package. - **`csdx not found` or `not logged in`**: the run has no management token, or reached the SAP app stage. Install `@contentstack/cli` and run `csdx auth:login --oauth`, or export `CS_MANAGEMENT_TOKEN` and pass `--carousel-field products`. +- **SAP picker preflight fails** (exit `4`): the server in `SAP_APP_API_BASE_URL` can't serve the picker. The line marked `FAIL` names the cause: an untrusted certificate, a missing base site, an empty search index, product search behind a login, or a catalog without the carousel codes. See "Before install: preflight" in [sap-commerce-app.md](sap-commerce-app.md). +- **A locale ends as `warn`**: an entry update in that locale failed twice (the Management API can answer a transient 422). Re-run with the same `--locales`; it updates in place. - **SAP Products field errors or HTTP 414**: see "Verify, retry, then ask" in [sap-commerce-app.md](sap-commerce-app.md). -- **Empty carousels on the storefront**: the connector is older than the `sap_products` release. Update it, or re-run with `--carousel-field products`. +- **"SAP product picker not set up" at gate 2**: the `SAP_APP_*` settings are exported, but the installed connector reads only `products` (npm 0.1.0). Update to a connector that reads `sap_products` (the release, or a build of the connector repo's main branch) and re-run. +- **Empty carousels on the storefront**: the picker was forced (`--carousel-field sap_products`) onto a connector that reads only `products` (gate 2 warns about this). Update the connector, or re-run with `--carousel-field products`. - **Blank storefront after import**: check the app's `cmsPageContentType` is `landing_page`, the `localeMapping` covers the imported locales, and the environment name matches `CS_ENVIRONMENT`. diff --git a/skills/dx-sap-spartacus-import/references/sap-commerce-app.md b/skills/dx-sap-spartacus-import/references/sap-commerce-app.md index d4019eb..58b46a3 100644 --- a/skills/dx-sap-spartacus-import/references/sap-commerce-app.md +++ b/skills/dx-sap-spartacus-import/references/sap-commerce-app.md @@ -2,9 +2,36 @@ The optional `sap-app` stage runs between gate 2 and the seed. It sets up the **SAP Commerce Cloud** marketplace app on the target stack, so carousel products come from the app's product picker (`sap_products`) instead of a hand-typed SKU list (`products`). Editors then add, remove and reorder products visually, and Live Preview's Edit on a carousel opens the picker. -The stage runs when carousels are seeded into `sap_products`: by default when `SAP_APP_API_BASE_URL`, `SAP_APP_BASE_SITE` and `SAP_APP_BACKOFFICE_URL` are all exported, or always with `--carousel-field sap_products`. With `--carousel-field products`, or without that configuration, it's skipped and carousels keep the `products` list. +The stage runs when carousels are seeded into `sap_products`: by default when `SAP_APP_API_BASE_URL`, `SAP_APP_BASE_SITE` and `SAP_APP_BACKOFFICE_URL` are all exported **and** the installed connector reads `sap_products`, or always with `--carousel-field sap_products`. Otherwise it's skipped and carousels keep the `products` list. -The storefront must run a connector release that reads `sap_products` (it falls back to `products`). Connector 0.1.0 reads only `products` and would render these carousels empty. +The orchestrator decides from the connector's code, not its version: it looks for `sap_products` in the installed package's bundle (node_modules, a `file:` tarball or an npm link), else in a connector checkout's `dist/` build or `src/` (with `CONNECTOR_STARTER_PACK` pointing into the checkout). The npm package 0.1.0 reads only `products`; a build of the connector repo's main branch reads `sap_products`, though it also says 0.1.0. + +The storefront must run a connector that reads `sap_products` (it falls back to `products` when `sap_products` is absent). Connector 0.1.0 reads only `products`, so by default the picker isn't set up with it; if you force `--carousel-field sap_products`, gate 2 warns that its carousels would render empty. + +## Which SAP server and base site + +The picker doesn't call SAP from the browser: the app's own backend calls the server in `SAP_APP_API_BASE_URL`. Point it at a server that carries the **same catalog as the storefront's OCC**: the picker saves product codes, and the storefront looks those codes up on its own server. + +| Storefront site | `SAP_APP_BASE_SITE` | Notes | +| --- | --- | --- | +| `electronics-spa` (B2C) | `electronics` or `electronics-spa` | The electronics sample catalog. | +| `powertools-spa` (B2B) | `powertools-spa` | Created by `spartacussampledata` once the Powertools sample data (`powertoolsstore`) is imported. Product search must work without a login. | +| Your own site | that site's uid | Any site that passes the preflight below. | + +Each stack gets its own app configuration, so an electronics stack and a powertools stack can point at the same SAP server with different base sites. + +## Before install: preflight + +At gate 2, with carousels going into `sap_products`, the orchestrator runs `scripts/marketplace-app/sap_preflight.py` against that server, with TLS verified (`OCC_INSECURE_TLS` doesn't apply: the app's backend verifies). The dry run prints the result; a write stops with exit code `4` before the app is installed or anything is written if a check fails. + +| Check | Fails when | Fix | +| --- | --- | --- | +| tls | The certificate isn't publicly trusted. SAP's default development certificate (`CN=hybris`) is the usual cause. | Give the server a trusted certificate, for example Let's Encrypt behind a reverse proxy. | +| site | `SAP_APP_BASE_SITE` isn't one of the server's base sites. | Import the store's sample data and `spartacussampledata` on that server, or fix the uid. | +| search | Product search returns 0 products, or answers 401/403. | Run a full Solr index for the site (for example `full-powertoolsIndex-cronJob`), or allow anonymous product search on the B2B site. | +| codes | None of the carousels' product codes exist there. Some missing is only a warning; those codes are left out of the picker value. | Point the app at a server with the storefront's catalog. | + +Run it on its own with the `SAP_APP_*` variables exported: `python3 "/scripts/marketplace-app/sap_preflight.py" --plan /connector-plan.json`. ## What it does @@ -39,7 +66,7 @@ python3 "/scripts/marketplace-app/sap_app.py" --status python3 "/scripts/marketplace-app/sap_app.py" --out /sap-app.json # install/configure once + discover python3 "/scripts/marketplace-app/sap_app.py" --out /sap-app.json --skip-install # already set up: discover only python3 "/scripts/marketplace-app/sap_app.py" --out /sap-app.json --reinstall # repair a broken install -python3 "/scripts/marketplace-app/use_products.py" --plan /connector-plan.json --env # fallback to products +python3 "/scripts/marketplace-app/use_products.py" --plan /connector-plan.json --env --starter-pack # fallback to products ``` ## Verify, retry, then ask @@ -48,7 +75,7 @@ The pipeline can't tell on its own whether the picker works: the app only talks **Check (one attempt):** -1. Open a seeded carousel entry: `https://app.contentstack.com/#!/stack//content-type/product_carousel_component/en-us/entries` (use the region's app host outside North America). +1. Open a seeded carousel entry: `https://app.contentstack.com/#!/stack//content-type/product_carousel_component/en-us/entries` (use the region's app host outside North America). The demo sites have two: "Our Bestselling Products" and "What's New" (electronics), "Our Bestselling Products" and "WHAT'S NEW" (powertools). 2. Wait about 20 seconds. **Pass** if the SAP Products field shows "N Products" with names. **Fail** if it shows "Something went wrong while fetching data". 3. Click **Add Product(s)**. **Pass** if the picker lists products. To read the app server's status code, run this in the picker tab with a product code from the carousel: ```js @@ -69,7 +96,7 @@ The pipeline can't tell on its own whether the picker works: the app only talks Explain the fallback in a sentence or two: the storefront shows the same products either way, because the connector reads `sap_products` first and falls back to `products`. With `products`, editors type or paste SAP product codes into a list instead of searching and picking them, and the entry shows codes rather than names. -**If the user picks `products`**, run `use_products.py --plan /connector-plan.json --env `. It re-imports the carousel content type without `sap_products`, writes each carousel's SKUs into `products` (same order, every locale the entry exists in) and publishes them. Later runs for that stack: `--carousel-field products`. The app stays installed; the user can uninstall it from Marketplace. +**If the user picks `products`**, run `use_products.py --plan /connector-plan.json --env --starter-pack `. It re-imports the carousel content type without `sap_products`, writes each carousel's SKUs into `products` (same order, every locale the entry exists in) and publishes them. Later runs for that stack: `--carousel-field products`. The app stays installed; the user can uninstall it from Marketplace. ## Limits diff --git a/skills/dx-sap-spartacus-import/scripts/cs-seed/cma_seed_connector.py b/skills/dx-sap-spartacus-import/scripts/cs-seed/cma_seed_connector.py index 5881222..499f19e 100644 --- a/skills/dx-sap-spartacus-import/scripts/cs-seed/cma_seed_connector.py +++ b/skills/dx-sap-spartacus-import/scripts/cs-seed/cma_seed_connector.py @@ -72,10 +72,12 @@ def visit(uid, stack): def import_content_types(needed, starter, sap_ext=None, verbose=True): + """Returns the number of content types that failed to import.""" # Import the FULL shipped model (topo-ordered) so every reference_to resolves, # even for types whose entries we defer (nav). Inject sap_uid on component types, # and the SAP app's product picker on the carousel when the app is set up. order = topo_types(starter) + errors = 0 for uid in order: ct = json.loads(json.dumps(starter[uid])) is_component = uid not in PAGE_TYPES and uid != GLOBAL_TYPE @@ -94,8 +96,10 @@ def import_content_types(needed, starter, sap_ext=None, verbose=True): else: st2, r = low.cma("POST", "/content_types", body) res = "created" if st2 in (200, 201) else f"ERR {st2}:{json.dumps(r)[:200]}" - if verbose: + if verbose or res.startswith("ERR"): print(f" {uid}: {res}") + errors += res.startswith("ERR") + return errors def resolve(v, uid_map, asset_cache, sap=None): @@ -160,6 +164,7 @@ def main(): stages = set(s.strip() for s in a.stages.split(",")) plan = json.load(open(a.plan)) uid_map, asset_cache = {}, {} + failures = 0 # per-item errors; exit 3 at the end so the orchestrator reports the stage as warn sap_ext, sap = None, None if plan.get("carousel_field") == "sap_products": if not a.sap_app: @@ -170,7 +175,7 @@ def main(): if "types" in stages: print("[content types] (starter pack, +sap_uid on components" f"{', +sap_products on carousel' if sap_ext else ''})") - import_content_types(plan["content_types_needed"], load_starter_types(a.starter_pack), sap_ext) + failures += import_content_types(plan["content_types_needed"], load_starter_types(a.starter_pack), sap_ext) if "assets" in stages: print(f"[assets] {len(plan['assets'])}") @@ -182,6 +187,7 @@ def main(): if r.startswith("ERR"): print(f" {asset['code']}: {r}") print(f" assets: {ok} ok, {err} err") + failures += err if "entries" in stages: if not asset_cache: # resolve existing assets for banner refs @@ -200,6 +206,7 @@ def main(): if err <= 6: print(f" {e['_key']['value']}: {res}") print(f" components: {ok} ok, {err} err") + failures += err if sap: print(f" sap_products: {len(sap.cache) - len(sap.missing)} products resolved on the app's SAP server" + (f", NOT FOUND (left out): {[c for c, _ in sap.missing]}" if sap.missing else "")) @@ -207,6 +214,7 @@ def main(): if gs: res, _ = upsert_entry(gs, uid_map, asset_cache, sap) print(f"[global_slots] {res}") + failures += res.startswith("ERR") pages = plan["entries"]["pages"] print(f"[pages] {len(pages)}") ok = err = 0 @@ -216,6 +224,10 @@ def main(): if res.startswith("ERR"): err += 1; print(f" {e['_key']['value']}: {res}") print(f" pages: {ok} ok, {err} err") + failures += err + if failures: + print(f"[seed] {failures} item(s) failed — see the ERR lines above; re-running is safe") + sys.exit(3) if __name__ == "__main__": diff --git a/skills/dx-sap-spartacus-import/scripts/cs-seed/localize.py b/skills/dx-sap-spartacus-import/scripts/cs-seed/localize.py index efa74c9..f619a20 100644 --- a/skills/dx-sap-spartacus-import/scripts/cs-seed/localize.py +++ b/skills/dx-sap-spartacus-import/scripts/cs-seed/localize.py @@ -15,10 +15,14 @@ The base (master en-us) entries must already be seeded. Matches entries by their key (sap_uid / url / title). Idempotent. Creds from env (CS_API_KEY, CS_MANAGEMENT_TOKEN). +A failed entry update is retried once (the Management API can answer a transient 422, for +example on global_slots). Exits 3 when an entry or asset still failed, so the orchestrator +reports the locale instead of marking it ok; re-running the locale is safe. + Usage: python3 localize.py --locale de-de --plan /connector-plan.json --env development """ -import argparse, json, os, sys, urllib.parse +import argparse, json, os, sys, time, urllib.parse sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "marketplace-app")) import cma_seed as low # cma(), upsert_asset(), asset_lookup(), download_occ @@ -109,6 +113,7 @@ def main(): {"asset": {"environments": [a.env], "locales": [a.master, loc]}}) pub += 1 if st in (200, 201) else 0 print(f"[{loc}] assets: {ok} ok, {err} err, {len(asset_cache)} resolved, {pub} published") + failures = err # resolve a reference target's entry uid by sap_uid (shared across locales), cached ref_cache = {} @@ -119,6 +124,7 @@ def ref_uid(ct, sap_uid): return ref_cache[key] def do(entries, label): + nonlocal failures put_ok = pub_ok = miss = e_err = 0 for e in entries: ct = e["_content_type"]; kf = e["_key"]["field"]; kv = e["_key"]["value"] @@ -126,7 +132,11 @@ def do(entries, label): if not uid: miss += 1; continue fields = loc_fields(e, asset_cache, ref_uid, sap) - st, r = low.cma("PUT", f"/content_types/{ct}/entries/{uid}?locale={loc}", {"entry": fields}) + path = f"/content_types/{ct}/entries/{uid}?locale={loc}" + st, r = low.cma("PUT", path, {"entry": fields}) + if st not in (200, 201, 404): # transient 422/5xx: retry once + time.sleep(2) + st, r = low.cma("PUT", path, {"entry": fields}) if st not in (200, 201): e_err += 1 if e_err <= 4: @@ -137,11 +147,15 @@ def do(entries, label): {"entry": {"environments": [a.env], "locales": [loc]}}) pub_ok += 1 if st2 in (200, 201) else 0 print(f" [{loc}] {label}: localized {put_ok}, published {pub_ok}, missing {miss}, err {e_err}") + failures += e_err do(plan["entries"]["components"], "components+nav") if plan["entries"].get("global_slots"): do([plan["entries"]["global_slots"]], "global_slots") do(plan["entries"]["pages"], "pages") + if failures: + print(f"[{loc}] {failures} failed after a retry — re-run this locale (safe: it updates in place)") + sys.exit(3) if __name__ == "__main__": diff --git a/skills/dx-sap-spartacus-import/scripts/cs-seed/publish.py b/skills/dx-sap-spartacus-import/scripts/cs-seed/publish.py index 6a1152d..3ed24ef 100644 --- a/skills/dx-sap-spartacus-import/scripts/cs-seed/publish.py +++ b/skills/dx-sap-spartacus-import/scripts/cs-seed/publish.py @@ -47,7 +47,7 @@ def list_all(path, key): def main(): ap = argparse.ArgumentParser() ap.add_argument("--env", required=True) - ap.add_argument("--locale", default="en-us") + ap.add_argument("--locale", default=os.environ.get("CS_MASTER_LOCALE", "en-us")) ap.add_argument("--preview-locales", default="", help="comma list of extra locales that need a Live Preview URL, e.g. de-de,ja-jp") ap.add_argument("--preview-url", default=os.environ.get("CS_PREVIEW_URL", "http://localhost:4200"), @@ -68,6 +68,7 @@ def main(): {"asset": {"environments": envs, "locales": loc}}) ok += st in (200, 201); err += st not in (200, 201) print(f" assets: {ok} ok, {err} err") + totals = {"assets": ok, "entries": 0, "errors": err} cts = [c["uid"] for c in list_all("/content_types", "content_types")] # order: components, then global_slots, then pages @@ -87,6 +88,12 @@ def main(): if st not in (200, 201) and err <= 3: print(f" {ct}/{e['uid']}: {st} {json.dumps(r)[:120]}") print(f"[publish {ct}] {ok} ok, {err} err") + totals["entries"] += ok; totals["errors"] += err + # one summary line last, so a caller that shows only the tail still sees the totals + print(f"[publish] {totals['assets']} assets and {totals['entries']} entries published to " + f"'{a.env}' ({a.locale}), {totals['errors']} errors") + if totals["errors"]: + sys.exit(3) if __name__ == "__main__": diff --git a/skills/dx-sap-spartacus-import/scripts/cs-seed/stack_info.py b/skills/dx-sap-spartacus-import/scripts/cs-seed/stack_info.py new file mode 100644 index 0000000..91dc990 --- /dev/null +++ b/skills/dx-sap-spartacus-import/scripts/cs-seed/stack_info.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python3 +""" +stack_info — read-only: name the target stack and say what's in it, so gate 2 can tell the +user which stack will be written and published, and whether it's empty (the skill +recommends a new, dedicated stack: the publish step publishes everything in it). + +Prints one line, never the API key: + target stack: 'SAP demo' — empty (0 content types, 0 assets; locales en-us; no environments) + +Usage: python3 stack_info.py (CS_API_KEY + CS_MANAGEMENT_TOKEN or a csdx login) +""" +import os, sys +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +import cma_seed as low # cma() + + +def count(path, key): + st, r = low.cma("GET", f"{path}?include_count=true&limit=1") + return r.get("count", len(r.get(key) or [])) if st == 200 else None + + +def main(): + if not (low.KEY and low.TOKEN): + print("target stack: unknown (set CS_API_KEY, and CS_MANAGEMENT_TOKEN or a csdx login)") + return + st, r = low.cma("GET", "/stacks") + if st != 200: + print(f"target stack: couldn't read it (HTTP {st}) — check CS_API_KEY and the login") + sys.exit(1) + name = (r.get("stack") or {}).get("name", "?") + n_ct, n_assets = count("/content_types", "content_types"), count("/assets", "assets") + st_l, rl = low.cma("GET", "/locales") + locales = [l["code"] for l in (rl.get("locales") or [])] if st_l == 200 else [] + st_e, re_ = low.cma("GET", "/environments") + envs = [e["name"] for e in (re_.get("environments") or [])] if st_e == 200 else [] + empty = not n_ct and not n_assets + print(f"target stack: '{name}' — {'empty' if empty else 'NOT empty'} " + f"({n_ct} content types, {n_assets} assets; locales {', '.join(locales) or 'none'}; " + f"environments {', '.join(envs) or 'none'})") + + +if __name__ == "__main__": + main() diff --git a/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_app.cjs b/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_app.cjs index 3a62fc5..6b76ba5 100644 --- a/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_app.cjs +++ b/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_app.cjs @@ -6,10 +6,10 @@ * user's own Contentstack CLI session (`csdx auth:login --oauth`), through csdx's bundled * @contentstack/cli-utilities (the same client csdx plugins use). Nothing is stored here. * - * node sap_app.cjs status --api-key # read-only: installation + config summary - * node sap_app.cjs ensure --api-key # install + configure once if missing; + * CS_API_KEY= node sap_app.cjs status # read-only: installation + config summary + * CS_API_KEY= node sap_app.cjs ensure # install + configure once if missing; * # an existing install is only checked - * node sap_app.cjs ensure --api-key --reinstall # uninstall, install, configure once + * CS_API_KEY= node sap_app.cjs ensure --reinstall # uninstall, install, configure once * * The configuration applied is one named multi-config (SAP_APP_CONFIG_NAME, default "hti"), * set as default, using a Custom URL, plus "Keys to Save" = Custom Fields: code + name @@ -21,7 +21,7 @@ * SAP_APP_CONFIG_NAME default "hti" * SAP_APP_API_BASE_URL SAP OCC host, no scheme (required for ensure) * SAP_APP_API_ROUTE default "/occ/v2/" - * SAP_APP_BASE_SITE e.g. "electronics" (required for ensure) + * SAP_APP_BASE_SITE e.g. "electronics", "powertools-spa" (required for ensure) * SAP_APP_BACKOFFICE_URL backoffice host or URL (required for ensure) * CS_ORG_UID optional; otherwise read from the stack * @@ -133,7 +133,7 @@ async function main() { const cmd = process.argv[2]; const apiKey = arg('api-key', process.env.CS_API_KEY); if (!['status', 'ensure'].includes(cmd) || !apiKey) { - fail('usage: node sap_app.cjs --api-key '); + fail('usage: CS_API_KEY= node sap_app.cjs (or --api-key, which shows in ps)'); } const U = loadCliUtilities(); const authType = U.configHandler.get('authorisationType'); diff --git a/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_app.py b/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_app.py index 03eed3e..7296199 100644 --- a/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_app.py +++ b/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_app.py @@ -25,8 +25,11 @@ def node(cmd, *extra): - p = subprocess.run(["node", os.path.join(HERE, "sap_app.cjs"), cmd, "--api-key", low.KEY, *extra], - capture_output=True, text=True) + # The stack API key goes to node through the environment (CS_API_KEY), not argv, + # so it never shows in the process list. + env = dict(os.environ, CS_API_KEY=low.KEY) + p = subprocess.run(["node", os.path.join(HERE, "sap_app.cjs"), cmd, *extra], + capture_output=True, text=True, env=env) if p.returncode: low.die((p.stderr or p.stdout).strip()) return json.loads(p.stdout) diff --git a/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_preflight.py b/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_preflight.py new file mode 100644 index 0000000..efaa573 --- /dev/null +++ b/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_preflight.py @@ -0,0 +1,142 @@ +#!/usr/bin/env python3 +""" +sap_preflight — check, before anything is installed or written, that the SAP server the SAP +Commerce Cloud app's product picker will search can actually serve the carousels. + +The picker never talks to SAP from the browser: the app's own backend calls the server in +SAP_APP_API_BASE_URL. So that server has to work for an anonymous caller over HTTPS with a +publicly trusted certificate, for the base site in SAP_APP_BASE_SITE. This holds for a B2C +site (electronics) and a B2B site (powertools-spa) alike; B2B sites are the ones that most +often fail it (site not imported, search index not built, or search behind a login). + +Checks, all with TLS verified (OCC_INSECURE_TLS does not apply: the app's backend verifies): + 1. tls — the server's certificate is trusted + 2. site — SAP_APP_BASE_SITE is one of the server's base sites + 3. search — product search on that site returns products (needs a built Solr index and + anonymous access; a 401/403 means the site requires a login) + 4. codes — the carousel product codes in the plan exist on that server (the picker saves + codes the storefront looks up on its own OCC; both must carry the same catalog) + +Exit 0 = pass (missing codes are a warning: they are left out of the picker value and +reported), 4 = fail: nothing should be installed or written. The orchestrator runs this at +gate 2 and stops before any write when it fails. + +Usage: + python3 sap_preflight.py [--plan /connector-plan.json] +""" +import argparse, json, os, ssl, sys, urllib.error, urllib.parse, urllib.request + +FAIL = 4 + + +def server(): + host = (os.environ.get("SAP_APP_API_BASE_URL") or "").strip().rstrip("/") + if not host: + return None, None, None + if not host.startswith("http"): + host = "https://" + host + route = "/" + (os.environ.get("SAP_APP_API_ROUTE") or "/occ/v2/").strip("/") + "/" + return host, route, os.environ.get("SAP_APP_BASE_SITE") + + +def get(url): + """GET with TLS verified. Returns (status, json-or-None, error-kind, detail).""" + try: + with urllib.request.urlopen(urllib.request.Request(url, headers={"Accept": "application/json"}), + context=ssl.create_default_context(), timeout=30) as r: + return r.status, json.loads(r.read().decode() or "{}"), None, "" + except urllib.error.HTTPError as e: + return e.code, None, "http", e.read().decode(errors="replace")[:200] + except urllib.error.URLError as e: + if isinstance(e.reason, ssl.SSLCertVerificationError) or "CERTIFICATE_VERIFY_FAILED" in str(e.reason): + return 0, None, "tls", str(e.reason)[:200] + return 0, None, "net", str(e.reason)[:200] + except Exception as e: # timeouts, bad JSON + return 0, None, "net", str(e)[:200] + + +def carousel_codes(plan): + codes = [] + for e in plan.get("entries", {}).get("components", []): + if e.get("_content_type") != "product_carousel_component": + continue + sp = e.get("sap_products") + if isinstance(sp, dict) and sp.get("_sap_products"): + codes += sp.get("codes") or [] + elif isinstance(e.get("products"), list): + codes += [c for c in e["products"] if isinstance(c, str)] + return list(dict.fromkeys(codes)) + + +def run(plan_path=None): + """Returns (ok, lines).""" + lines, ok = [], True + host, route, site = server() + if not host or not site: + return False, ["SAP_APP_API_BASE_URL and SAP_APP_BASE_SITE must be set for the picker"] + base = f"{host}{route}" + lines.append(f"picker server: {base} · base site {site}") + + st, body, kind, detail = get(f"{base}basesites?fields=baseSites(uid)") + if kind == "tls": + return False, lines + [ + "FAIL tls: the server's certificate isn't publicly trusted. The app's backend rejects it " + "(SAP's default development certificate, CN=hybris, is the usual cause). Give the server " + "a trusted certificate (for example Let's Encrypt behind a reverse proxy) and re-run.", + f" {detail}"] + if kind: + return False, lines + [f"FAIL server: {base}basesites answered {st or kind} — {detail}"] + lines.append("ok tls: certificate trusted") + sites = [s.get("uid") for s in (body or {}).get("baseSites", [])] + if site not in sites: + return False, lines + [ + f"FAIL site: '{site}' isn't a base site on this server (it has: {', '.join(sites) or 'none'}).", + " On SAP Commerce Cloud the -spa sites come from the spartacussampledata extension; " + "a store's sample data (for example powertoolsstore) has to be imported for its site to exist."] + lines.append(f"ok site: {site} exists") + + st, body, kind, detail = get(f"{base}{site}/products/search?pageSize=1&fields=pagination(totalResults)") + total = ((body or {}).get("pagination") or {}).get("totalResults", 0) if not kind else 0 + if kind == "http" and st in (401, 403): + return False, lines + [ + f"FAIL search: product search on {site} needs a login (HTTP {st}). The picker searches " + "anonymously; allow anonymous product search on this B2B site, or point the app at one that does."] + if kind: + return False, lines + [f"FAIL search: {st or kind} — {detail}"] + if not total: + return False, lines + [ + f"FAIL search: product search on {site} returns 0 products, so the picker would list nothing.", + " Run a full Solr index for the site's catalog (for example full-powertoolsIndex-cronJob) and re-run."] + lines.append(f"ok search: {total} products") + + if plan_path: + codes = carousel_codes(json.load(open(plan_path))) + missing = [] + for c in codes: + st, _, kind, _ = get(f"{base}{site}/products/{urllib.parse.quote(c)}?fields=code") + if st != 200: + missing.append(c) + if codes and len(missing) == len(codes): + return False, lines + [ + f"FAIL codes: none of the {len(codes)} carousel product codes exist on this server. " + "It doesn't carry the storefront's catalog; point the app at a server that does."] + if missing: + lines.append(f"warn codes: {len(codes) - len(missing)}/{len(codes)} found; missing {missing} " + "will be left out of the picker value") + else: + lines.append(f"ok codes: {len(codes)}/{len(codes)} carousel product codes found") + return ok, lines + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--plan", help="connector-plan.json, to check the carousel product codes") + a = ap.parse_args() + ok, lines = run(a.plan) + for l in lines: + print(f" {l}") + sys.exit(0 if ok else FAIL) + + +if __name__ == "__main__": + main() diff --git a/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_products.py b/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_products.py index fc9c2bc..9182971 100644 --- a/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_products.py +++ b/skills/dx-sap-spartacus-import/scripts/marketplace-app/sap_products.py @@ -22,7 +22,7 @@ Env: SAP_APP_API_BASE_URL host of the app's SAP OCC (no scheme), e.g. api.<...>.commerce.ondemand.com SAP_APP_API_ROUTE default /occ/v2/ - SAP_APP_BASE_SITE e.g. electronics + SAP_APP_BASE_SITE e.g. electronics (B2C) or powertools-spa (B2B) SAP_APP_CONFIG_NAME the app configuration name stored in cs_metadata, default hti Falls back to OCC_BASE_URL + the run's site when SAP_APP_API_BASE_URL is unset. """ diff --git a/skills/dx-sap-spartacus-import/scripts/orchestrate.py b/skills/dx-sap-spartacus-import/scripts/orchestrate.py index 8966dd6..3db7a73 100644 --- a/skills/dx-sap-spartacus-import/scripts/orchestrate.py +++ b/skills/dx-sap-spartacus-import/scripts/orchestrate.py @@ -19,7 +19,9 @@ GATES (fail-closed): the pipeline STOPS at each gate unless the matching approval flag is passed, so nothing is created or published without an explicit human go. Gate 1 model approval -> pass --approve-model (only blocks if unknown typeCodes found) - Gate 2 pre-publish -> pass --approve-publish (always blocks before any write) + Gate 2 pre-publish -> pass --approve-publish (always blocks before any write; with the + SAP picker on, also stops with exit 4 if the picker's SAP server + fails the preflight in marketplace-app/sap_preflight.py) Default (no approval flags) = a full DRY RUN: extract→model + both gate reports, no writes. @@ -94,6 +96,41 @@ def load_pack(dir_): return pack +def connector_reads_sap_products(starter_pack): + """(version, reads) for the connector the starter pack came from. reads is True when its + code reads the picker field `sap_products`, False when it doesn't (npm 0.1.0 reads only + `products`), None when no code was found next to the starter pack. + + The version can't decide it: a build from the connector repo's main branch also says + 0.1.0. So look at the code: the installed package's bundle (node_modules, a `file:` tgz or + an npm link), else a repo checkout's build (dist/) or its source (src/, specs excluded).""" + root = os.path.abspath(os.path.join(starter_pack, "..", "..", "..")) + version = None + try: + version = json.load(open(os.path.join(root, "package.json"))).get("version") + except Exception: + pass + found_code = False + for d in ("fesm2022", "fesm2020", "fesm2015", "bundles", os.path.join("dist", "fesm2022"), "dist"): + folder = os.path.join(root, d) + if not os.path.isdir(folder): + continue + for f in os.listdir(folder): + if f.endswith((".mjs", ".js")): + found_code = True + if "sap_products" in open(os.path.join(folder, f), errors="ignore").read(): + return version, True + src = os.path.join(root, "src") + if not found_code and os.path.isdir(src): + for dirpath, _, files in os.walk(src): + for f in files: + if f.endswith(".ts") and not f.endswith(".spec.ts"): + found_code = True + if "sap_products" in open(os.path.join(dirpath, f), errors="ignore").read(): + return version, True + return version, (False if found_code else None) + + def pack_mismatches(plan, pack): """Every content type and field the plan writes must exist in the starter pack, or the seed fails half-way. sap_uid is allowed: the seeder adds it when the pack lacks it.""" @@ -131,7 +168,9 @@ def main(): ap.add_argument("--approve-publish", action="store_true", help="pass Gate 2 (enables writes)") ap.add_argument("--locales", default="", help="optional i18n: sapIso=csLocale,... e.g. de=de-de,ja=ja-jp,zh=zh-cn") ap.add_argument("--carousel-field", choices=["auto", "products", "sap_products"], default="auto", - help="where carousel SKUs go; auto = sap_products for electronics-spa, products otherwise") + help="where carousel SKUs go; auto = sap_products when SAP_APP_API_BASE_URL, " + "SAP_APP_BASE_SITE and SAP_APP_BACKOFFICE_URL are set (any site, B2C or B2B) AND " + "the installed connector reads sap_products; products otherwise") ap.add_argument("--skip-app-install", action="store_true", help="SAP app already installed + configured: only discover its field extension") ap.add_argument("--reinstall-app", action="store_true", @@ -147,8 +186,21 @@ def main(): "`ng add @contentstack/contentstack-spartacus-connector`, or set CONNECTOR_STARTER_PACK.") locale_map = dict(p.split("=", 1) for p in a.locales.split(",") if "=" in p) if a.locales else {} carousel_field = a.carousel_field + picker_configured = all(os.environ.get(k) for k in SAP_APP_ENV) + conn_ver, conn_reads = connector_reads_sap_products(a.starter_pack) + carousel_reason = "" if carousel_field == "auto": - carousel_field = "sap_products" if all(os.environ.get(k) for k in SAP_APP_ENV) else "products" + if picker_configured and conn_reads: + carousel_field = "sap_products" + carousel_reason = "SAP_APP_* set and the connector reads sap_products" + else: + carousel_field = "products" + if not picker_configured: + carousel_reason = "SAP_APP_* not set" + elif conn_reads is False: + carousel_reason = f"the installed connector ({conn_ver or 'unknown version'}) reads only `products`" + else: + carousel_reason = "couldn't find the connector's code to check whether it reads sap_products" run_dir = a.run_dir or os.path.join(tempfile.gettempdir(), f"sap-spartacus-import-{a.site}") os.makedirs(run_dir, exist_ok=True) @@ -163,7 +215,7 @@ def record(stage, status, note=""): banner(f"CONTENT-MODELING AGENT · {a.site} · profile connector-reference-fields") print(f" run dir: {run_dir} env: {a.env} starter pack: {a.starter_pack}") - print(f" carousel SKUs -> {carousel_field}") + print(f" carousel SKUs -> {carousel_field}{' (' + carousel_reason + ')' if carousel_reason else ''}") print(f" occ: {a.base} region: {os.environ.get('CS_REGION_BASE', 'https://api.contentstack.io')}") print(f" approvals: model={'YES' if a.approve_model else 'no'} " f"publish={'YES' if a.approve_publish else 'no'}" @@ -227,16 +279,28 @@ def record(stage, status, note=""): n_pages = len(plan["entries"]["pages"]) print() banner("GATE 2 · PRE-PUBLISH", "gate") - print(f" will UPSERT to stack + publish to '{a.env}':") - print(f" content types : {len(plan['content_types_needed'])}") + if os.environ.get("CS_API_KEY"): + _, out_si = run("cs-seed/stack_info.py", []) + print(" " + (out_si.strip().splitlines() or ["target stack: unknown"])[-1]) + else: + print(" target stack: not set yet (export CS_API_KEY before the write)") + pack = load_pack(a.starter_pack) + try: + report = json.load(open(os.path.join(run_dir, "connector-report.json"))) + n_nav = sum((report.get("nav_entries") or {}).values()) + except Exception: + n_nav = 0 + print(f" will UPSERT to that stack + publish to '{a.env}':") + print(f" content types : {len(pack)} (the connector's full starter pack; this site's entries " + f"use {len(plan['content_types_needed'])}). Existing types with these uids are replaced.") print(f" assets : {len(plan['assets'])}") - print(f" component entries: {n_comp} pages: {n_pages} global_slots: " - f"{'1' if plan['entries'].get('global_slots') else '0'}") + print(f" entries : {n_comp - n_nav} components, {n_nav} navigation, {n_pages} pages, " + f"{'1' if plan['entries'].get('global_slots') else '0'} global_slots") print(f" pages: {[p['_key']['value'] for p in plan['entries']['pages']]}") if locale_map: print(f" + localize into: {list(locale_map.values())} " f"(fully-localized OCC extract per language, incl. per-locale images)") - mismatches = pack_mismatches(plan, load_pack(a.starter_pack)) + mismatches = pack_mismatches(plan, pack) if mismatches: print(f" {C['err']}starter pack mismatch ({len(mismatches)}){C['z']} — the plan writes types/fields the pack lacks:") for m in mismatches[:25]: @@ -247,6 +311,18 @@ def record(stage, status, note=""): sys.exit(3) print(f" {C['ok']}starter pack check: every planned type and field exists{C['z']}") print(f" carousel SKUs -> {carousel_field}") + if carousel_field == "sap_products" and conn_reads is False: + # only when forced with --carousel-field sap_products + print(f" {C['warn']}the installed connector ({conn_ver or 'unknown version'}) reads only `products`:{C['z']} " + f"carousels seeded into sap_products render empty on the storefront until you update to " + f"the connector release that reads sap_products (or use --carousel-field products).") + record("connector", "warn", f"{conn_ver} reads only products; picker forced") + elif carousel_field == "products" and picker_configured and a.carousel_field == "auto": + print(f" {C['warn']}SAP product picker not set up:{C['z']} {carousel_reason}. The carousels get the plain " + f"`products` list, which this connector renders. Update to a connector that reads sap_products " + f"(the release, or a build of the connector repo's main branch) and re-run to use the picker, " + f"or force it with --carousel-field sap_products.") + record("connector", "warn", carousel_reason) auth = os.environ.get("CS_AUTH", "auto").lower() uses_token = auth == "token" or (auth == "auto" and os.environ.get("CS_MANAGEMENT_TOKEN")) print(f" writes as: {'management token' if uses_token else 'your csdx login (csdx auth:login --oauth)'}") @@ -255,19 +331,40 @@ def record(stage, status, note=""): print(" SAP Commerce Cloud app: already set up (--skip-app-install) — only find its Product field") else: print(f" SAP Commerce Cloud app: install if missing + config " - f"'{os.environ.get('SAP_APP_CONFIG_NAME', 'hti')}' -> " - f"{os.environ.get('SAP_APP_API_BASE_URL', '')}" - f"{os.environ.get('SAP_APP_API_ROUTE', '/occ/v2/')}" + f"'{os.environ.get('SAP_APP_CONFIG_NAME', 'hti')}': server SAP_APP_API_BASE_URL" + f"{os.environ.get('SAP_APP_API_ROUTE', '/occ/v2/')}, base site " f"{os.environ.get('SAP_APP_BASE_SITE', '')}" f", keys code+name (uses your csdx login)") missing = [k for k in SAP_APP_ENV if not os.environ.get(k)] if missing: print(f" {C['warn']}missing env for the SAP app: {missing}{C['z']}") + # The picker's SAP server must serve this site to an anonymous caller over trusted TLS, + # with a built search index and the carousels' product codes (B2C and B2B sites alike). + # Checked here so the dry run shows it, and so a failing server stops the run before + # the app is installed or anything is written. + preflight_failed = False + if carousel_field == "sap_products" and all(os.environ.get(k) for k in SAP_APP_ENV[:2]): + rc_pf, out_pf = run("marketplace-app/sap_preflight.py", + ["--plan", os.path.join(run_dir, "connector-plan.json")]) + print(" SAP picker preflight:") + print("\n".join(" " + line for line in out_pf.rstrip().splitlines())) + record("picker-preflight", "pass" if not rc_pf else "fail", + " | ".join(l.strip() for l in out_pf.splitlines() + if l.strip().startswith(("ok", "warn", "FAIL")))[:300]) + preflight_failed = bool(rc_pf) if not a.approve_publish: record("gate2", "dry-run") + if preflight_failed: + print(f"\n {C['err']}The SAP picker preflight failed{C['z']} — a write would stop before " + f"the app is installed. Fix the SAP server, or use --carousel-field products.") print(f"\n {C['warn']}DRY RUN complete{C['z']} — nothing written. " f"Re-run with --approve-publish (and CS_* env) to seed + publish.") sys.exit(0) + if preflight_failed: + record("gate2", "blocked", "SAP picker preflight failed") + print(f" {C['err']}stopped: the SAP picker preflight failed{C['z']} — nothing installed or written. " + f"Fix the SAP server, or re-run with --carousel-field products.") + sys.exit(4) if not os.environ.get("CS_API_KEY"): record("gate2", "error", "no creds") sys.exit(f"{C['err']}--approve-publish set but CS_API_KEY missing.{C['z']}") @@ -305,15 +402,19 @@ def record(stage, status, note=""): rc, out = run("cs-seed/cma_seed_connector.py", seed_args) tail = "\n ".join(out.strip().splitlines()[-6:]) print(" " + tail) - record("seed", "ok" if not rc else "error", out[-300:]) - if rc: + # exit 3 = some items failed (their ERR lines are in the output); the rest was written + record("seed", "ok" if not rc else "warn" if rc == 3 else "error", out[-300:]) + if rc and rc != 3: sys.exit("seed failed") + if rc == 3: + print(f" {C['warn']}some items failed (ERR lines above); re-running is safe{C['z']}") # 7. PUBLISH step("6/8 publish") - rc, out = run("cs-seed/publish.py", ["--env", a.env, "--locale", "en-us"]) + master = os.environ.get("CS_MASTER_LOCALE", "en-us") + rc, out = run("cs-seed/publish.py", ["--env", a.env, "--locale", master]) print(" " + "\n ".join(out.strip().splitlines()[-4:])) - record("publish", "ok" if not rc else "error") + record("publish", "ok" if not rc else "warn" if rc == 3 else "error", out.strip().splitlines()[-1] if out.strip() else "") # 8. VERIFY step("7/8 verify — round-trip") @@ -326,7 +427,10 @@ def record(stage, status, note=""): venv = os.environ.copy(); venv["CS_ENVIRONMENT"] = a.env rc, out = run("verify/cda_read.py", ["--content-type", "landing_page", "--url", "/"], env=venv) print(" " + "\n ".join(out.strip().splitlines()[-2:])) - record("verify-cda", "pass" if rc == 0 else "check") + # exit 5 = no delivery token yet (expected on a fresh stack): re-run cda_read.py after creating one + record("verify-cda", "pass" if rc == 0 else "check", + "no CS_DELIVERY_TOKEN yet: create one for the environment, then re-run verify/cda_read.py" + if rc == 5 else out.strip().splitlines()[-1] if out.strip() else "") # 9. LOCALIZE (optional) — full localized OCC extract per language, then localize+publish. for sap_iso, cs_loc in locale_map.items(): @@ -348,19 +452,27 @@ def record(stage, status, note=""): rc, out = run("cs-seed/localize.py", ["--locale", cs_loc, "--plan", os.path.join(lmodel, "connector-plan.json"), "--env", a.env]) print(" " + "\n ".join(out.strip().splitlines()[-4:])) - record(f"localize:{cs_loc}", "ok" if not rc else "error", out[-200:]) + # localize.py exits 3 when some entries or assets failed after a retry (re-running is safe) + record(f"localize:{cs_loc}", "ok" if not rc else "warn" if rc == 3 else "error", out[-200:]) print() loc_ok = [l for l in locale_map.values() if manifest["stages"].get(f"localize:{l}", {}).get("status") == "ok"] loc_failed = [l for l in locale_map.values() if l not in loc_ok] banner(f"DONE · {a.site} · round-trip {'PASS' if rt_ok else 'CHECK'}" f"{' · +' + str(len(loc_ok)) + ' locales' if loc_ok else ''}" - f"{' · FAILED locales: ' + ', '.join(loc_failed) if loc_failed else ''}", + f"{' · locales with errors (re-run is safe): ' + ', '.join(loc_failed) if loc_failed else ''}", "ok" if rt_ok and not loc_failed else "warn") print(f" manifest: {run_dir}/manifest.json") + stages = manifest["stages"] + for name in ("seed", "publish"): + if stages.get(name, {}).get("status") == "warn": + print(f" {C['warn']}{name}: some items failed{C['z']} — see the ERR lines above; re-running is safe") + if stages.get("verify-cda", {}).get("status") == "check": + print(f" delivery check: {stages['verify-cda'].get('note') or 'see the output above'}") if carousel_field == "sap_products": print(f"\n {C['gate']}NEXT · verify the SAP product picker in the browser{C['z']} — " - f"references/sap-commerce-app.md › 'Verify, retry, then ask'.\n" + f"references/sap-commerce-app.md › 'Verify, retry, then ask'. Without a browser, ask the " + f"user to open a carousel entry and check that the SAP Products field lists products.\n" f" Repair (max 2): python3 {os.path.join(SKILLS, 'marketplace-app', 'sap_app.py')} " f"--reinstall --out {sap_app_path}\n" f" Fallback (user's choice): python3 {os.path.join(SKILLS, 'marketplace-app', 'use_products.py')} " diff --git a/skills/dx-sap-spartacus-import/scripts/verify/cda_read.py b/skills/dx-sap-spartacus-import/scripts/verify/cda_read.py index 29520d8..ad72ec4 100644 --- a/skills/dx-sap-spartacus-import/scripts/verify/cda_read.py +++ b/skills/dx-sap-spartacus-import/scripts/verify/cda_read.py @@ -40,8 +40,14 @@ def main(): ap.add_argument("--content-type", default="landing_page") ap.add_argument("--url", default="/") a = ap.parse_args() - if not (KEY and DELIVERY): - sys.exit("set CS_API_KEY and CS_DELIVERY_TOKEN") + if not KEY: + sys.exit("set CS_API_KEY") + if not DELIVERY: + # A fresh stack has no delivery token until the run has created the environment. + print(f"[cda_read] skipped: CS_DELIVERY_TOKEN isn't set. Create a delivery token for '{ENV}' " + f"(Settings > Tokens), export it as CS_DELIVERY_TOKEN, then re-run this check: " + f"python3 {os.path.abspath(__file__)} --content-type {a.content_type} --url {a.url}") + sys.exit(5) params = [("environment", ENV), ("query", json.dumps({"url": a.url}))] for f in SLOT_FIELDS: