diff --git a/.github/workflows/inspect-longport-token-expiry.yml b/.github/workflows/inspect-longport-token-expiry.yml new file mode 100644 index 0000000..21efb9c --- /dev/null +++ b/.github/workflows/inspect-longport-token-expiry.yml @@ -0,0 +1,67 @@ +name: Inspect LongPort Token Expiry + +# Read-only: decode JWT exp from Secret Manager token secrets. +# Never calls /v1/token/refresh, never adds SM versions, never prints secret values. +on: + workflow_dispatch: + inputs: + target: + description: "Account target whose Access Token JWT expiry to inspect." + required: true + type: choice + options: + - paper + - hk + - sg + warn_days: + description: "Fail when days remaining is below this threshold." + required: false + type: string + default: "30" + +permissions: + contents: read + +concurrency: + group: inspect-longport-token-expiry-${{ inputs.target }} + cancel-in-progress: false + +jobs: + inspect: + if: github.repository == 'QuantStrategyLab/LongBridgePlatform' && github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: read + id-token: write + env: + GCP_PROJECT_ID: longbridgequant + GCP_WORKLOAD_IDENTITY_PROVIDER: projects/252919773759/locations/global/workloadIdentityPools/github-actions/providers/github-main + GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT: longbridge-platform-deploy@longbridgequant.iam.gserviceaccount.com + TARGET: ${{ inputs.target }} + WARN_DAYS: ${{ inputs.warn_days }} + steps: + - name: Checkout + uses: actions/checkout@v5 + + - name: Authenticate to Google Cloud (deploy SA) + uses: google-github-actions/auth@v3 + with: + workload_identity_provider: ${{ env.GCP_WORKLOAD_IDENTITY_PROVIDER }} + service_account: ${{ env.GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT }} + + - name: Set up Python + uses: actions/setup-python@v6 + with: + python-version: "3.12" + + - name: Install Secret Manager client + run: python -m pip install --quiet 'google-cloud-secret-manager>=2.20' + + - name: Inspect token expiry (names + days only) + run: | + set -euo pipefail + python scripts/inspect_longport_token_expiry.py \ + --project "$GCP_PROJECT_ID" \ + --target "$TARGET" \ + --warn-days "$WARN_DAYS" | tee "$GITHUB_STEP_SUMMARY" diff --git a/docs/longport_token_refresh_runbook.md b/docs/longport_token_refresh_runbook.md new file mode 100644 index 0000000..e11bd48 --- /dev/null +++ b/docs/longport_token_refresh_runbook.md @@ -0,0 +1,106 @@ +# LongPort Access Token durability (paper / HK / SG) + +## Recommendation (one line) + +**Prefer path A now:** keep Legacy App Key/Secret/Access Token, fix and schedule **pre-expiry** refresh into Secret Manager (paper → SG → HK last); defer OAuth2 (path B) until A is durable and SDK 4.x is planned. + +## Why this exists + +LongPort Legacy Access Tokens expire (docs: ~90 days / three months). After expiry, `GET /v1/token/refresh` returns **401003** and **cannot** revive the token; recovery is Developer Portal reset + Secret Manager rotate (HK was healed this way on 2026-10-09). Probe/heartbeat and paused strategy must not be the only auth paths. + +Authoritative vendor docs: + +- [Getting Started (Legacy refresh + OAuth)](https://open.longportapp.com/docs/getting-started.md) +- [Refresh Access Token](https://open.longportapp.com/docs/refresh-token-api.md) +- [Error codes (401003)](https://open.longportapp.com/docs/error-codes.md) + +## 1. How each account stores credentials today + +Public inventory: [`config/runtime_targets.manifest.json`](../config/runtime_targets.manifest.json). + +| Target | GitHub Environment | Cloud Run service | SM token | SM app key | SM app secret | +| --- | --- | --- | --- | --- | --- | +| paper | `longbridge-paper` | `longbridge-quant-paper-service` | `longport_token_paper` | `longport-app-key-paper` | `longport-app-secret-paper` | +| hk | `longbridge-hk` | `longbridge-quant-hk-service` | `longport_token_hk` | `longport-app-key-hk` | `longport-app-secret-hk` | +| sg | `longbridge-sg` | `longbridge-quant-sg-service` | `longport_token_sg` | `longport-app-key-sg` | `longport-app-secret-sg` | + +Runtime wiring (Actions deploy + Cloud Run): + +- Env vars `LONGPORT_SECRET_NAME` / `LONGPORT_APP_KEY_SECRET_NAME` / `LONGPORT_APP_SECRET_SECRET_NAME` point at the SM **names** above (per Environment / deploy inputs). +- **App Key / App Secret** are mounted into the revision as Cloud Run secret env refs (`LONGPORT_APP_KEY` / `LONGPORT_APP_SECRET` → `:latest`). +- **Access Token** is **not** relied on as a static revision env value for broker auth: bootstrap reads SM `projects/.../secrets//versions/latest` via `quant_platform_kit.longbridge.auth.fetch_token_from_secret` (pinned QPK in `pyproject.toml`). +- Manual portal-reset write path: [`.github/workflows/rotate-longport-secrets.yml`](../.github/workflows/rotate-longport-secrets.yml) (temporary repo secrets → SM versions). Deploy SA needs `roles/secretmanager.secretVersionAdder` on those secrets (HK was granted separately). + +Never log or commit secret **values**; this runbook only names resources. + +## 2. Pre-expiry refresh: implemented, but historically ineffective + +### What exists + +- QPK: `quant_platform_kit.longbridge.auth.refresh_token_if_needed` calls Legacy `GET https://openapi.longportapp.com/v1/token/refresh` with HMAC headers, then writes the new token to SM when `code == 0`. +- Platform: `TOKEN_REFRESH_THRESHOLD_DAYS = 30` in `main.py`; `LongBridgeRuntimeBootstrap.build_contexts()` invokes refresh before building quote/trade contexts. +- Read-only paths **intentionally skip** refresh: `build_read_only_contexts`, account-snapshot metadata read, paper command consumer, and HK/SG history probe branch that uses `build_account_snapshot_broker_contexts()`. + +### Why tokens still expired + +1. **Refresh only rides strategy bootstrap.** When `RUNTIME_TARGET_ENABLED=false` (or strategy cycles are otherwise not running), the refresh path never runs. Daily probe / heartbeat still need a valid token but use read-only context builders. +2. **Past expiry is terminal for refresh.** Vendor 401003 + docs: refresh must happen **before** expiry; portal reset is the only recovery afterward (matches HK incident). +3. **Soft-fail hides pre-expiry API errors.** If refresh returns non-zero **and** JWT `exp` is still in the future, QPK returns the old token instead of failing closed—so a broken refresh can sit silent until hard expiry. +4. **Missing `expired_at` query param.** Vendor API marks `expired_at` required; official SDK `Config.refresh_access_token(expired_at=...)` sends it (default ~90 days). Current QPK call signs/sends **empty** params. That is a likely contributor to silent refresh failure (covered by soft-fail above). Fix belongs in QPK + tests, then platform pin bump. +5. **IAM for writers.** Runtime/Action writers need SM add-version permission; without it, even a successful LongPort refresh cannot persist (rotate workflow surfaced this for HK). + +## 3. OAuth2 availability (path B) + +LongPort OpenAPI now documents **OAuth 2.0 as recommended** for new integrations: register client (`/oauth2/register`), browser `authorization_code` + `refresh_token`, SDK `Config.from_oauth` / `OAuthBuilder`, token file under `~/.longport/openapi/tokens/`, SDK auto-refresh. Legacy App Key mode remains compatible; Legacy `refresh_access_token` is **not** supported in OAuth mode. + +For Cloud Run paper/HK/SG this is **not** a small drop-in: + +- Needs interactive (or carefully automated) consent **per account**. +- Must replace HMAC App Key/Secret signing with Bearer OAuth tokens end-to-end in QPK `build_contexts` and any custom signed HTTP. +- Must store OAuth refresh material in SM (not home-directory files on ephemeral containers). +- Platform currently pins `longport==3.0.23`; OAuth-first SDK surface is on newer 4.x lines—bump is a separate risk. + +Treat OAuth as a **follow-on migration** after path A is green, not the emergency durability fix. + +## 4. Recommended path and blast radius + +| Phase | Action | Targets | Blast radius | +| --- | --- | --- | --- | +| A0 (this PR) | Runbook + read-only JWT expiry inspector (no refresh, no SM write) | paper / hk / sg | Docs + optional manual inspect only | +| A1 | QPK: send `expired_at`; fail closed on pre-expiry refresh failure; keep SM write via store_rw | kit first | Shared kit; pin bump in platform after | +| A2 | Dedicated refresh job (Actions `workflow_dispatch` → then schedule) calling refresh **before** expiry, independent of `RUNTIME_TARGET_ENABLED` | **paper first**, then **sg**, **hk last** | Mutates token SM version; invalidates previous Access Token on success | +| A3 | Alert when days-to-exp < threshold (reuse inspector) | all | Notify only | +| B (later) | OAuth spike on paper only, SDK 4.x, SM-backed refresh token | paper → sg → hk | Large: auth model + SDK + deploy | + +**HK last** because it was just healed via portal reset + SM rotate + QRS binding; do not experiment with token invalidation there until paper/SG prove A1+A2. + +Constraints this plan respects: no `independent_get` / ingress changes, no production strategy enablement flips, no secret values in docs/logs, no strategy logic changes. + +## 5. Exact next PR scope (do not fold into this PR) + +1. **QuantPlatformKit** – `longbridge/auth.py` + `tests/test_longbridge_auth.py`: + - Pass ISO-8601 `expired_at` on `/v1/token/refresh` (and include it in the signed `params` string). + - On non-zero refresh while still pre-expiry: raise (or structured error) instead of returning the old token. + - Optionally prefer official SDK `Config.refresh_access_token` once pin allows; keep SM persistence in kit. +2. **LongBridgePlatform** – after QPK pin: + - `workflow_dispatch` refresh job per target (reuse rotate’s target matrix / WIF deploy SA); schedule only after paper dry-run succeeds. + - Confirm `secretVersionAdder` (or store_rw equivalent) on paper + sg (+ hk when ready). + - Keep probe/snapshot read-only (no refresh on those paths). +3. **Ops** – calendar reminder / alert from expiry inspector until A2 is scheduled. +4. **Not in next PR:** OAuth client registration, `longport` 4.x bump, enabling `RUNTIME_TARGET_ENABLED`, ingress, or strategy changes. + +## Emergency recovery (already proven on HK) + +1. Developer Portal: reset Access Token (and confirm App Key/Secret if needed) for that account. +2. Set temporary repo secrets `ROTATE_LONGPORT_TOKEN` / `ROTATE_LONGPORT_APP_KEY` / `ROTATE_LONGPORT_APP_SECRET`. +3. Run **Rotate LongPort Secrets** with `target=paper|hk|sg`. +4. Delete temporary repo secrets. +5. Re-run heartbeat / probe; if account-history source binding changes, update QRS expected binding (HK required this after rotate). + +## Related code pointers + +- Platform bootstrap: `application/runtime_bootstrap_adapters.py` +- Composer read-only vs refresh: `application/runtime_composer.py` +- Probe history branch (no refresh): `main.py` `run_probe` +- Rotate workflow: `.github/workflows/rotate-longport-secrets.yml` +- Read-only expiry inspect: `.github/workflows/inspect-longport-token-expiry.yml` + `scripts/inspect_longport_token_expiry.py` diff --git a/scripts/inspect_longport_token_expiry.py b/scripts/inspect_longport_token_expiry.py new file mode 100755 index 0000000..0f264fb --- /dev/null +++ b/scripts/inspect_longport_token_expiry.py @@ -0,0 +1,144 @@ +#!/usr/bin/env python3 +"""Read-only LongPort Access Token expiry inspector. + +Decodes JWT ``exp`` from a Secret Manager token secret and prints days remaining. +Never prints secret values, never calls /v1/token/refresh, never adds SM versions. + +Usage (ADC or GHA WIF already authenticated): + + python scripts/inspect_longport_token_expiry.py \\ + --project longbridgequant \\ + --secret longport_token_paper \\ + --warn-days 30 +""" + +from __future__ import annotations + +import argparse +import base64 +import json +import sys +import time +from typing import Any + + +TARGET_SECRETS = { + "paper": "longport_token_paper", + "hk": "longport_token_hk", + "sg": "longport_token_sg", +} + + +def decode_token_expiry_unix(token: str) -> float | None: + """Return JWT exp as unix seconds, or None if the token is not a decodable JWT.""" + try: + parts = token.split(".") + if len(parts) <= 1: + return None + payload_b64 = parts[1] + padded = payload_b64 + "=" * (-len(payload_b64) % 4) + payload = json.loads(base64.urlsafe_b64decode(padded).decode("utf-8")) + expiry = payload.get("exp") + if expiry is None: + return None + return float(expiry) + except Exception: + return None + + +def days_until_expiry(expiry_unix: float, *, now: float | None = None) -> float: + current = time.time() if now is None else float(now) + return (float(expiry_unix) - current) / 86400.0 + + +def access_secret_latest(project_id: str, secret_name: str) -> str: + try: + import google.cloud.secretmanager_v1 as secret_manager + except ImportError: # pragma: no cover - depends on install layout + from google.cloud import secret_manager + + client = secret_manager.SecretManagerServiceClient() + name = f"projects/{project_id}/secrets/{secret_name}/versions/latest" + response = client.access_secret_version(request={"name": name}) + return response.payload.data.decode("UTF-8").strip() + + +def inspect_one( + *, + project_id: str, + secret_name: str, + warn_days: float, + now: float | None = None, + token_reader: Any | None = None, +) -> dict[str, Any]: + reader = token_reader or access_secret_latest + token = reader(project_id, secret_name) + expiry = decode_token_expiry_unix(token) + result: dict[str, Any] = { + "project_id": project_id, + "secret_name": secret_name, + "jwt_exp_present": expiry is not None, + "warn_days": float(warn_days), + } + if expiry is None: + result["status"] = "undecodable" + result["ok"] = False + return result + remaining = days_until_expiry(expiry, now=now) + result["days_until_expiry"] = round(remaining, 3) + result["expired"] = remaining <= 0 + if remaining <= 0: + result["status"] = "expired" + result["ok"] = False + elif remaining < float(warn_days): + result["status"] = "warn" + result["ok"] = False + else: + result["status"] = "ok" + result["ok"] = True + return result + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--project", required=True, help="GCP project id") + parser.add_argument( + "--target", + choices=sorted(TARGET_SECRETS), + help="Named runtime target; sets --secret from the public manifest mapping", + ) + parser.add_argument( + "--secret", + help="Explicit Secret Manager token secret name (overrides --target)", + ) + parser.add_argument( + "--warn-days", + type=float, + default=30.0, + help="Fail (exit 2) when days remaining is below this threshold (default 30)", + ) + return parser + + +def main(argv: list[str] | None = None) -> int: + args = build_parser().parse_args(argv) + secret_name = args.secret + if not secret_name: + if not args.target: + print("error: provide --target or --secret", file=sys.stderr) + return 2 + secret_name = TARGET_SECRETS[args.target] + result = inspect_one( + project_id=args.project, + secret_name=secret_name, + warn_days=args.warn_days, + ) + # Never include token material; result is names + numeric expiry distance only. + print(json.dumps(result, sort_keys=True)) + if result.get("ok"): + return 0 + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/test_inspect_longport_token_expiry.py b/tests/test_inspect_longport_token_expiry.py new file mode 100644 index 0000000..3b52afb --- /dev/null +++ b/tests/test_inspect_longport_token_expiry.py @@ -0,0 +1,89 @@ +from __future__ import annotations + +import base64 +import json +import sys +import time +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +if str(ROOT) not in sys.path: + sys.path.insert(0, str(ROOT)) + +from scripts.inspect_longport_token_expiry import ( # noqa: E402 + TARGET_SECRETS, + days_until_expiry, + decode_token_expiry_unix, + inspect_one, + main, +) + + +def _jwt_with_exp(exp: int) -> str: + payload = base64.urlsafe_b64encode(json.dumps({"exp": exp}).encode("utf-8")).decode("utf-8").rstrip("=") + return f"hdr.{payload}.sig" + + +def test_target_secret_names_match_public_manifest(): + assert TARGET_SECRETS == { + "paper": "longport_token_paper", + "hk": "longport_token_hk", + "sg": "longport_token_sg", + } + + +def test_decode_token_expiry_unix_reads_exp(): + token = _jwt_with_exp(1_700_000_000) + assert decode_token_expiry_unix(token) == 1_700_000_000.0 + + +def test_decode_token_expiry_unix_returns_none_for_opaque(): + assert decode_token_expiry_unix("not-a-jwt") is None + + +def test_inspect_one_ok_warn_expired_without_leaking_token(): + now = time.time() + secret_token = _jwt_with_exp(int(now + 60 * 86400)) + ok = inspect_one( + project_id="demo", + secret_name="longport_token_paper", + warn_days=30, + now=now, + token_reader=lambda *_a, **_k: secret_token, + ) + assert ok["status"] == "ok" + assert ok["ok"] is True + dumped = json.dumps(ok) + assert secret_token not in dumped + assert "hdr." not in dumped + + warn = inspect_one( + project_id="demo", + secret_name="longport_token_paper", + warn_days=30, + now=now, + token_reader=lambda *_a, **_k: _jwt_with_exp(int(now + 10 * 86400)), + ) + assert warn["status"] == "warn" + assert warn["ok"] is False + + expired = inspect_one( + project_id="demo", + secret_name="longport_token_paper", + warn_days=30, + now=now, + token_reader=lambda *_a, **_k: _jwt_with_exp(int(now - 86400)), + ) + assert expired["status"] == "expired" + assert expired["ok"] is False + + +def test_days_until_expiry_math(): + assert abs(days_until_expiry(100_000.0, now=100_000.0 - 2 * 86400) - 2.0) < 1e-9 + + +def test_main_requires_target_or_secret(capsys): + code = main(["--project", "demo"]) + assert code == 2 + err = capsys.readouterr().err + assert "provide --target or --secret" in err