Repository navigation
perf(aggregate): buffer trace payload fragments until sampling - #217
Merged
Merged
Conversation
改动链路:tracing Ingest 保存有界压缩分片并合并摘要,到期整组决策;新增分片结果接口供发送端逐片接管,避免重复解压和重编码。旧单包接口和复杂规则保持兼容,并对长期驻留的过量容量真实缩容。 风险点:每组超过 256 片或 8 MiB 压缩数据、混合协议和 payload spiller 回退旧路径;复杂规则到期仍需物化,调用方必须对全部保留分片施加发送预算。 验证:go test ./aggregate -count=1、go test -race ./aggregate -count=1 通过;固定 128 points、2/8/32/128 分包微基准通过;golangci-lint run --new-from-rev=HEAD 为 0 issues。 已执行 LINT_FIX=false make lint、make copyright_check 和 make test;全库历史 lint/版权问题、未修改的 diskcache/WebSocket 测试及临时目录配额问题使完整检查未通过。
leecha
approved these changes
Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
同一条 trace 的 point 总量固定而分包数增加时,当前时间轮会在每个后续分片到达时解压已有 payload、拼接并重新压缩,在 shard 锁内反复处理历史数据。此改动允许先保留压缩分片和合并摘要,到期对整组做一次采样决定,支持摘要的规则可直接返回原始保留分片,省去重复编解码。
EnableTracePayloadFragments()和TailSamplingFragmentOutcomes();调用方需发送Packets中的全部保留分片,并按组记录决策指标。验证结果:
go test ./aggregate -count=1:通过。go test -race ./aggregate -count=1:通过。go test ./point -run 'TestV2Encode(TrimProgress|ProtobufBoundary|BoundaryErrors)$' -count=1:通过。make lint、make test、go mod verify通过;另覆盖了预算等待、磁盘开关、HTTP query/header/body 透传、分片完整输出及关停后仅重放未接管分片。固定 128 个 point,比较同一实现中即时合并和分片缓冲两种模式(两者均含缩容),入轮及决策微基准三次中位数如下,不包含下游发送:
命令:
GOMAXPROCS=2 go test ./aggregate -run '^$' -bench BenchmarkTracePayloadFragments -benchmem -benchtime=50x -count=3,环境为 Go 1.26.10、Linux amd64。累计分配不代表常驻堆或 RSS,微基准也不代表端到端吞吐收益。完整检查尚未全绿:
LINT_FIX=false make lint和make copyright_check报告未修改代码中的既有问题;全库make test遇到未修改的 diskcache 用例磁盘配额不足、WebSocket 连接失败,且运行到 180 秒时被外层超时终止。调用方仍需对分片施加发送字节预算和停止处理;本改动不提供无限过载接收能力,回退路径仍有原有合并开销。