Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .changeset/app-image-ingestion-key.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
'@hyperdx/api': patch
---

feat: add `ENABLE_INGESTION_API_KEY` to honor `INGESTION_API_KEY` on the app image

The app image ignores `INGESTION_API_KEY`, so an external collector has to use
the team's generated key, which changes whenever the team is recreated. Set
`ENABLE_INGESTION_API_KEY=true` on the API to accept a preset key alongside the
team key. Behavior is unchanged when unset.
2 changes: 1 addition & 1 deletion agent_docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ pings `/v1/opamp` with status, and the server returns updated config when needed
- **Model**: `models/agent.ts` - agent state persistence
- **Proto**: `proto/` - Protocol Buffer definitions for OpAMP and anyvalue

Config is derived from the team document with the ingestion API key.
Config is derived from the team document with the ingestion API key. `INGESTION_API_KEY` is also accepted on all-in-one and dev builds, or anywhere with `ENABLE_INGESTION_API_KEY=true`.

### Background Tasks (`src/tasks/`)

Expand Down
3 changes: 3 additions & 0 deletions packages/api/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ export const IS_LOCAL_IMAGE = HYPERDX_IMAGE === 'all-in-one-noauth';
export const IS_INLINE_API = env.HDX_PREVIEW_INLINE_API === 'true';
export const FRONTEND_REDIRECT_BASE = IS_INLINE_API ? '' : FRONTEND_URL;
export const INGESTION_API_KEY = env.INGESTION_API_KEY ?? '';
// Opt-in: honor INGESTION_API_KEY on the app image too, where it is ignored
// by default. The key is accepted alongside the team's key.
export const ENABLE_INGESTION_API_KEY = env.ENABLE_INGESTION_API_KEY === 'true';
// Opt-in: emit the contrib `datadogreceiver` on the collector so a
// Datadog Agent can ship APM traces (DD trace API -> OTLP -> ClickHouse).
// Off by default because the receiver has no per-team bearer-token auth like
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ const configState = {
IS_LOCAL_APP_MODE: false,
IS_DEV: false,
INGESTION_API_KEY: '' as string,
ENABLE_INGESTION_API_KEY: false,
IS_PROMQL_ENABLED: false,
ENABLE_DATADOG_RECEIVER: false,
};
Expand All @@ -22,6 +23,9 @@ jest.mock('@/config', () => ({
get INGESTION_API_KEY() {
return configState.INGESTION_API_KEY;
},
get ENABLE_INGESTION_API_KEY() {
return configState.ENABLE_INGESTION_API_KEY;
},
get IS_PROMQL_ENABLED() {
return configState.IS_PROMQL_ENABLED;
},
Expand All @@ -40,6 +44,7 @@ const resetConfig = () => {
configState.IS_LOCAL_APP_MODE = false;
configState.IS_DEV = false;
configState.INGESTION_API_KEY = '';
configState.ENABLE_INGESTION_API_KEY = false;
configState.IS_PROMQL_ENABLED = false;
configState.ENABLE_DATADOG_RECEIVER = false;
};
Expand Down Expand Up @@ -171,6 +176,40 @@ describe('opampController', () => {
);
});

it('ignores INGESTION_API_KEY outside all-in-one and dev unless opted in', () => {
configState.INGESTION_API_KEY = 'ingest-key';

const cfg = buildOtelCollectorConfig([
{ apiKey: 'k1', collectorAuthenticationEnforced: true },
]);

const tokens = cfg.extensions['bearertokenauth/hyperdx'].tokens;
expect(tokens).toEqual(bearerTokenVariants(['k1']));
});

it('includes INGESTION_API_KEY alongside the team key when ENABLE_INGESTION_API_KEY is set', () => {
configState.ENABLE_INGESTION_API_KEY = true;
configState.INGESTION_API_KEY = 'ingest-key';

const cfg = buildOtelCollectorConfig([
{ apiKey: 'k1', collectorAuthenticationEnforced: true },
]);

const tokens = cfg.extensions['bearertokenauth/hyperdx'].tokens;
expect(tokens).toEqual(bearerTokenVariants(['k1', 'ingest-key']));
});

it('adds nothing when ENABLE_INGESTION_API_KEY is set but INGESTION_API_KEY is empty', () => {
configState.ENABLE_INGESTION_API_KEY = true;

const cfg = buildOtelCollectorConfig([
{ apiKey: 'k1', collectorAuthenticationEnforced: true },
]);

const tokens = cfg.extensions['bearertokenauth/hyperdx'].tokens;
expect(tokens).toEqual(bearerTokenVariants(['k1']));
});

it('bearerTokenVariants emits the bare token plus the bearer scheme case variants', () => {
expect(bearerTokenVariants(['tok'])).toEqual([
'tok',
Expand Down
10 changes: 8 additions & 2 deletions packages/api/src/opamp/controllers/opampController.ts
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,14 @@ export const buildOtelCollectorConfig = (
): CollectorConfig => {
const apiKeys = teams.filter(team => team.apiKey).map(team => team.apiKey);

if (config.IS_ALL_IN_ONE_IMAGE || config.IS_LOCAL_APP_MODE || config.IS_DEV) {
// Only allow INGESTION_API_KEY for dev or all-in-one images for security reasons
if (
config.IS_ALL_IN_ONE_IMAGE ||
config.IS_LOCAL_APP_MODE ||
config.IS_DEV ||
config.ENABLE_INGESTION_API_KEY

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 P2 — With ENABLE_INGESTION_API_KEY on and no team yet, the app image opens an OTLP receiver with no auth

buildOtelCollectorConfig gets teams from getAllTeams (line 604). When that list is empty (fresh install, or a rebuilt DB before anyone registers, which is the case this PR targets), apiKeys ends up as [INGESTION_API_KEY]. That passes the apiKeys.length > 0 check at line 348, so otlp/hyperdx gets wired into every pipeline. But collectorAuthenticationEnforced comes from teams[0]?.collectorAuthenticationEnforced, which is undefined here, so the bearertokenauth block at line 379 is skipped and the receiver accepts anything. On main the same state leaves apiKeys empty on the app image, so no receiver opens. .github/workflows/release.yml:1117-1142 relies on the collector never opening a keyless receiver. Fix: when the preset key is what makes apiKeys non-empty, enforce auth with it, for example collectorAuthenticationEnforced = teams[0]?.collectorAuthenticationEnforced ?? (teams.length === 0 && pushedPresetKey). Add a test that calls buildOtelCollectorConfig([]) with the flag set and asserts either that otlp/hyperdx has auth or that it is not attached.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#3633 merged before the fix reached its branch, so the fix is in #3760. With no team, an opted-in INGESTION_API_KEY now turns authentication on. A new test covers the no-team case.

) {
// Only allow INGESTION_API_KEY for dev or all-in-one images for security
// reasons, unless the deployment opts in with ENABLE_INGESTION_API_KEY
if (config.INGESTION_API_KEY) {
apiKeys.push(config.INGESTION_API_KEY);
}
Expand Down
3 changes: 2 additions & 1 deletion packages/otel-collector/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,8 @@ config for logs).
In OpAMP supervisor mode, when collector authentication is enforced the
receiver authenticates against the team API keys (same as `otlp/hyperdx`) via
the `DD-API-KEY` header. Set the Datadog Agent's `DD_API_KEY` to a HyperDX
team ingestion API key.
team ingestion API key, or to `INGESTION_API_KEY` where the API honors it
(all-in-one and dev builds, or with `ENABLE_INGESTION_API_KEY=true`).

## Computing span metrics from ingested traces

Expand Down
Loading