From ecd2905795e226570e8aca6137c8b7b4b3d11b83 Mon Sep 17 00:00:00 2001 From: Rodrigo Roca Date: Thu, 24 Sep 2026 16:01:53 +0200 Subject: [PATCH] Document Datadog OTLP job log export --- data/llm_descriptions.yml | 2 +- .../monitoring_and_observability/tracing.md | 14 ++++++++++++++ .../integrations/observability/datadog.md | 6 ++++++ 3 files changed, 21 insertions(+), 1 deletion(-) diff --git a/data/llm_descriptions.yml b/data/llm_descriptions.yml index a1f1cd703ce..695f5f44da7 100644 --- a/data/llm_descriptions.yml +++ b/data/llm_descriptions.yml @@ -139,7 +139,7 @@ pipelines/integrations/notifications/cc-menu: "Using the cctray.xml feed to display build status in CCMenu, CCTray, and build dashboards." pipelines/integrations/notifications/plugins: "Directory of Buildkite plugins for delivering notifications." pipelines/integrations/observability/overview: "Overview of observability integrations: Datadog, Honeycomb, OpenTelemetry, Amazon EventBridge, and community plugins." - pipelines/integrations/observability/datadog: "Configuring Datadog APM tracing and CI Pipeline Visibility for Buildkite Pipelines." + pipelines/integrations/observability/datadog: "Configuring Datadog APM tracing, OTLP job logs, and CI Pipeline Visibility for Buildkite Pipelines." pipelines/integrations/observability/honeycomb: "Integrating Honeycomb with Buildkite using buildevents binary, OpenTelemetry tracing, or the Markers plugin." pipelines/integrations/observability/opentelemetry: "Sending OpenTelemetry traces from Buildkite agents and the dashboard to your OTLP endpoint." pipelines/integrations/observability/amazon-eventbridge: "Streaming real-time build, job, and agent events to Amazon EventBridge with full event schema reference." diff --git a/pages/agent/self_hosted/monitoring_and_observability/tracing.md b/pages/agent/self_hosted/monitoring_and_observability/tracing.md index 544438fc39b..b0e8346efdb 100644 --- a/pages/agent/self_hosted/monitoring_and_observability/tracing.md +++ b/pages/agent/self_hosted/monitoring_and_observability/tracing.md @@ -113,6 +113,20 @@ export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf" buildkite-agent start ``` +### Sending job logs to Datadog + +Configure the Buildkite agent to export job logs directly to the Datadog OTLP logs intake endpoint: + +```bash +export BUILDKITE_JOB_LOGS_OTLP=true +export OTEL_EXPORTER_OTLP_LOGS_PROTOCOL="http/protobuf" +export OTEL_EXPORTER_OTLP_LOGS_ENDPOINT="" +export OTEL_EXPORTER_OTLP_LOGS_HEADERS="dd-api-key=" +buildkite-agent start +``` + +Use the [Datadog OTLP Logs Intake Endpoint documentation](https://docs.datadoghq.com/opentelemetry/setup/otlp_ingest/logs/) to find the endpoint for your Datadog site and review the authentication details. Logs sent this way are available in [Datadog Log Management](https://docs.datadoghq.com/logs/). To correlate the logs with pipeline and job executions in Datadog CI Pipeline Visibility, also configure the [Datadog Pipeline Visibility notification service](/docs/pipelines/integrations/observability/datadog#configuring-the-datadog-integration-in-buildkite). + ### What each record contains Each record carries a line of job output as its body, with the same `[REDACTED]` markers as the Buildkite job log. Records are built from the already-redacted job log stream, so they inherit the [job log redaction](/docs/pipelines/configure/managing-log-output#redacted-environment-variables) automatically, including secrets split across separate writes and secrets added mid-job through the [Job API](/docs/agent/self-hosted/configure/experiments#promoted-experiments-job-api). Lines longer than the maximum record size of 64 KiB are split across multiple records. Both child-process output and the bootstrap control output, such as section headers, prompts, comments, and warnings, are exported, so the records match the downloadable Buildkite job log. diff --git a/pages/pipelines/integrations/observability/datadog.md b/pages/pipelines/integrations/observability/datadog.md index efa3784c07b..b86ebe3bc6b 100644 --- a/pages/pipelines/integrations/observability/datadog.md +++ b/pages/pipelines/integrations/observability/datadog.md @@ -21,6 +21,12 @@ Once the Buildkite agent is running with `--opentelemetry-tracing`, run at least Once Datadog APM is integrated with Buildkite Pipelines, you gain full visibility into your CI pipeline through detailed tracing of Buildkite agent activity. Each job execution is captured as a trace with individual spans representing key phases such as hook execution, command runtime, and lifecycle events like pre-exit or post-command. These spans provide real-time insights into duration, performance bottlenecks, and potential failures across your builds. With built-in filtering, service tagging, Datadog enables deep observability into your CI workflows, making it easier to troubleshoot, optimize, and maintain high pipeline reliability. +## Sending job logs to Datadog + +The Buildkite agent can export job logs as OpenTelemetry log records to the Datadog OTLP logs intake endpoint. Follow the [Datadog job log export instructions](/docs/agent/self-hosted/monitoring-and-observability/tracing#exporting-job-logs-as-opentelemetry-logs-sending-job-logs-to-datadog) to configure the export, endpoint, and authentication. + +The exported logs are available in [Datadog Log Management](https://docs.datadoghq.com/logs/). If you also send pipeline data to Datadog CI Pipeline Visibility through the [Datadog Pipeline Visibility notification service](#configuring-the-datadog-integration-in-buildkite), Datadog correlates the logs with the corresponding pipeline and job executions. For more information, see [Buildkite Setup for CI Visibility](https://docs.datadoghq.com/continuous_integration/pipelines/buildkite/#collect-job-logs). + ## Configuring the Datadog integration in Buildkite To set up the Datadog's CI Pipeline Visibility integration for Buildkite: