Skip to content

feat(api): add ENABLE_INGESTION_API_KEY to honor INGESTION_API_KEY on the app image - #3633

Merged
kodiakhq[bot] merged 3 commits into
ClickHouse:mainfrom
andrewdunndev:ai/app-image-ingestion-key
Nov 21, 2026
Merged

kodiakhq[bot] merged 3 commits into
ClickHouse:mainfrom
andrewdunndev:ai/app-image-ingestion-key

Conversation

@andrewdunndev

Copy link
Copy Markdown
Contributor

Summary

On the app image, the OpAMP config ignores INGESTION_API_KEY, so an external collector has to authenticate with the team's generated key. That key changes whenever the team is recreated, for example after the database is rebuilt, and the collector is locked out until someone copies the new key from Team Settings into its config.

This adds an opt-in ENABLE_INGESTION_API_KEY=true, in the same style as ENABLE_DATADOG_RECEIVER. With it set, INGESTION_API_KEY is accepted alongside the team key, so a deployment can pin a key that survives a rebuild. Nothing changes when it is unset. I kept the existing all-in-one and dev gate as it is, since its security reasoning isn't written down anywhere. Happy to rename the variable, or to drop the gate instead if you'd prefer that.

How to reproduce

Starting point: the app image (HYPERDX_IMAGE=hyperdx, NODE_ENV=production) with the standalone collector in OpAMP supervisor mode, and collector authentication enforced for the team (the default for a new team). yarn dev does not show the problem, because IS_DEV already honors the key. The new unit tests in opampController.test.ts cover the same three cases.

On main:

  1. Set INGESTION_API_KEY=preset-key on the API and restart it.
  2. Send OTLP to the collector with Authorization: preset-key: it is rejected, and preset-key is missing from the bearertokenauth/hyperdx tokens in the OpAMP config.

On this branch:

  1. Set INGESTION_API_KEY=preset-key and ENABLE_INGESTION_API_KEY=true and restart the API.
  2. The same request is accepted, and the team key still works. With the flag unset, the behaviour matches main.

How to test on Vercel preview

N/A, non-UI change.

References

@andrewdunndev
andrewdunndev requested a review from a team as a code owner November 12, 2026 17:46
@changeset-bot

changeset-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1399601

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@hyperdx/api Patch
@hyperdx/app Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Nov 12, 2026

Copy link
Copy Markdown

@andrewdunndev is attempting to deploy a commit to the HyperDX Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 P0/P1 in code this PR adds or changes: fix before asking for review.
  • 🟡 P2 in code this PR changes: your call. Fix if small, otherwise reply.
  • 🔵 P3 is collapsed at the bottom: nobody needs to act on it before merge.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    priority. Reply with one sentence; if it is P0, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

1 finding(s): 🔴 0 P0/P1 · 🟡 1 P2 · 🔵 0 P3

1 posted as inline comment(s) on the changed lines.


Priority comes from a separate grading pass over the finished review. P3 is folded here instead of posted inline. No finding blocks merge automatically; a maintainer decides.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found.

This is a small, well-contained change that adds an opt-in ENABLE_INGESTION_API_KEY flag, mirroring the existing ENABLE_DATADOG_RECEIVER idiom. With the flag unset the gate condition reduces byte-for-byte to the pre-existing IS_ALL_IN_ONE_IMAGE || IS_LOCAL_APP_MODE || IS_DEV, so default behavior is unchanged; the if (config.INGESTION_API_KEY) guard already covers the empty-key case. The three new tests cover the off, on, and on-with-empty-key branches with exact-match assertions, and the in-code comment was updated to reflect the new opt-in branch.

No P2 findings. The residual considerations below are informational only — each is pre-existing to the all-in-one/dev path and merely becomes reachable on the app image through an explicit, default-off opt-in, so none is a regression introduced by this diff:

  • When ENABLE_INGESTION_API_KEY=true on the app image, the preset INGESTION_API_KEY is a static value shared across teams; data pushed with it is attributed by the same shared-key mechanism that already exists for all-in-one and dev builds.
  • If a team's apiKey equals INGESTION_API_KEY, bearerTokenVariants emits duplicate tokens — harmless, and already true for the existing honored builds.
  • The otlp/hyperdx receiver is attached without auth whenever teams[0].collectorAuthenticationEnforced is false, independent of the preset key — unchanged by this diff.

Reviewers (4): correctness, security, testing, maintainability.

Testing gaps: None substantive — the new opt-in branch (on/off/empty-key) is covered; the one "gap" raised (asserting the preset key is only emitted when collectorAuthenticationEnforced is true) concerns pre-existing, unchanged behavior.

config.IS_ALL_IN_ONE_IMAGE ||
config.IS_LOCAL_APP_MODE ||
config.IS_DEV ||
config.ENABLE_INGESTION_API_KEY

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 P2 — With ENABLE_INGESTION_API_KEY on and no team yet, the app image opens an OTLP receiver with no auth

buildOtelCollectorConfig gets teams from getAllTeams (line 604). When that list is empty (fresh install, or a rebuilt DB before anyone registers, which is the case this PR targets), apiKeys ends up as [INGESTION_API_KEY]. That passes the apiKeys.length > 0 check at line 348, so otlp/hyperdx gets wired into every pipeline. But collectorAuthenticationEnforced comes from teams[0]?.collectorAuthenticationEnforced, which is undefined here, so the bearertokenauth block at line 379 is skipped and the receiver accepts anything. On main the same state leaves apiKeys empty on the app image, so no receiver opens. .github/workflows/release.yml:1117-1142 relies on the collector never opening a keyless receiver. Fix: when the preset key is what makes apiKeys non-empty, enforce auth with it, for example collectorAuthenticationEnforced = teams[0]?.collectorAuthenticationEnforced ?? (teams.length === 0 && pushedPresetKey). Add a test that calls buildOtelCollectorConfig([]) with the flag set and asserts either that otlp/hyperdx has auth or that it is not attached.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#3633 merged before the fix reached its branch, so the fix is in #3760. With no team, an opted-in INGESTION_API_KEY now turns authentication on. A new test covers the no-team case.

@pulpdrew pulpdrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the contribution

@kodiakhq
kodiakhq Bot merged commit ef81fe6 into ClickHouse:main Nov 21, 2026
25 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow forcing ingestion key

2 participants