feat(otel): add opt-in gzip compression for the default OTLP exporter - #1921
DavidTraina wants to merge 3 commits into
Conversation
| if raw_value is None: | ||
| setting = LANGFUSE_OTEL_COMPRESSION | ||
| raw_value = os.environ.get(LANGFUSE_OTEL_COMPRESSION, "") | ||
|
|
||
| value = raw_value.strip().lower() | ||
| if not value: | ||
| return None |
There was a problem hiding this comment.
Blank argument skips environment setting
If otel_compression is empty or contains only spaces while LANGFUSE_OTEL_COMPRESSION=gzip, this return skips the valid environment setting and silently falls back to the standard OTEL settings. Treat a blank argument as unset, or reject it consistently with other invalid values.
| if raw_value is None: | |
| setting = LANGFUSE_OTEL_COMPRESSION | |
| raw_value = os.environ.get(LANGFUSE_OTEL_COMPRESSION, "") | |
| value = raw_value.strip().lower() | |
| if not value: | |
| return None | |
| if raw_value is None or not raw_value.strip(): | |
| setting = LANGFUSE_OTEL_COMPRESSION | |
| raw_value = os.environ.get(LANGFUSE_OTEL_COMPRESSION, "") | |
| value = raw_value.strip().lower() | |
| if not value: | |
| return None |
Prompt To Fix With AI
This is a comment left during a code review.
Path: langfuse/_client/span_processor.py
Line: 78-84
Comment:
**Blank argument skips environment setting**
If `otel_compression` is empty or contains only spaces while `LANGFUSE_OTEL_COMPRESSION=gzip`, this return skips the valid environment setting and silently falls back to the standard OTEL settings. Treat a blank argument as unset, or reject it consistently with other invalid values.
```suggestion
if raw_value is None or not raw_value.strip():
setting = LANGFUSE_OTEL_COMPRESSION
raw_value = os.environ.get(LANGFUSE_OTEL_COMPRESSION, "")
value = raw_value.strip().lower()
if not value:
return None
```
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Leaving this as is. otel_compression is typed Literal["gzip", "none"], so a blank string is not a supported value for the argument. The blank check exists for LANGFUSE_OTEL_COMPRESSION set to an empty string in the environment. The JS SDK treats its compression option the same way.
There was a problem hiding this comment.
That distinction is reasonable. Since otel_compression is typed as Literal["gzip", "none"], blank values are only expected from LANGFUSE_OTEL_COMPRESSION, where the existing empty-string handling is sufficient. I’ll withdraw this concern; no change is needed.
Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.
What does this PR do?
Adds an opt-in way to gzip the span batches sent by the default Langfuse OTLP exporter. It works through a new
Langfuse(otel_compression=...)argument or a newLANGFUSE_OTEL_COMPRESSIONenv var.Today the only way to compress is the standard
OTEL_EXPORTER_OTLP_(TRACES_)COMPRESSIONenv vars. They work only because Langfuse never passescompressiontoOTLPSpanExporter. They also apply to every OTLP exporter in the process, and the SDK does not mention them.Behavior:
gzipandnone, trimmed and case-insensitive. Other values such asdeflatelog a warning and are ignored.otel_compression>LANGFUSE_OTEL_COMPRESSION>OTEL_EXPORTER_OTLP_TRACES_COMPRESSION>OTEL_EXPORTER_OTLP_COMPRESSION> none.compression=None. Existing setups that use the OTEL env vars behave as before.span_exporterdocstring now says a custom exporter must configure compression itself. Passingotel_compressiontogether withspan_exporterlogs a warning.LANGFUSE_OTEL_MAX_BATCH_SIZE_BYTEScap is unchanged. It measures uncompressed bytes, which matches the server's limit after decompression.Server support:
web/src/pages/api/public/otel/v1/traces/index.ts.deflateis not offered, even though the OTel Python exporter supports it.Measured against a local Langfuse v4.50.0, with one batch of 11 spans carrying about 5 KB of chat-style input and output each:
otel_compression="gzip"LANGFUSE_OTEL_COMPRESSION=gzipAll three batches were ingested with input and output intact.
Companion JS PR: langfuse/langfuse-js#974
Type of change
Verification
List the main commands you ran:
ruff format --check .also flagstests/unit/test_media.py. That file is unchanged here and already fails on main.The new tests in
tests/unit/test_span_processor.pyexport through the real exporter to the localotlp_http_serverfixture. They checkContent-Encodingand decode the body. They cover the argument through the full client, the env var, and precedence against the OTEL env vars. Withcompression=Nonehardcoded, 4 of the 5 fail. The remaining one checks that the OTEL env vars still apply when nothing is set.Not run:
tests/e2eandtests/live_provider, which need a server and provider keys.Checklist
code_review.md..env.templateif needed.