Skip to content

Fix spring-cloud-gateway-4.x propagating another request's context - #830

Draft
wu-sheng wants to merge 1 commit into
mainfrom
fix/gateway-4.x-per-request-context-snapshot
Draft

wu-sheng wants to merge 1 commit into
mainfrom
fix/gateway-4.x-per-request-context-snapshot

Conversation

@wu-sheng

Copy link
Copy Markdown
Member

Follows up on the report in apache/skywalking#14095.

Problem

Since Spring Cloud Gateway 4.1.2, NettyRoutingFilter#filter only assembles the outbound chain; the chain is subscribed later:

Flux<HttpClientResponse> responseFlux = getHttpClientMono(route, exchange)   // eager, inside filter()
    .flatMapMany(httpClient -> httpClient.headers(...)                       // runs at SUBSCRIPTION
        .request(method).uri(url).send(...)...);

That is why v412x carries a ContextSnapshot forward instead of reading the thread context at send() the way v4x does. The snapshot was parked on the HttpClient returned by NettyRoutingFilter#getHttpClient — but that method returns the shared this.httpClient bean itself unless a connect timeout is configured, so the write lands on one object for the whole JVM lifetime, while the read (HttpClientConnect#duplicate via HttpClient#headers) happens at subscription time.

Any request that reaches getHttpClient in between overwrites the snapshot, and the earlier request's outbound sw8 header then carries a context that belongs to another request. The downstream service joins the wrong trace.

Fix

Hold the snapshot on a client derived per request. HttpClient#headers duplicates the client and copies the header map only, which Spring Cloud Gateway copies again one line later, so a chain that already duplicates on headers(), request() and uri() gains a single duplication and no connection observer. doOnRequest would also force the copy, but it flips reactor-netty off its defaultConnectionObserver() fast path and installs an HttpClientDoOn observer on every request, so it is not used here.

After this change the shared bean is never written, so HttpClientConnectDuplicateV412Interceptor reads null from it and no stale value can be carried forward. Every node of the chain is now per request. Untraced requests derive nothing, because the ContextManager.isActive() guard runs first.

Tests

NettyRoutingGetHttpClientV412InterceptorTest now asserts that the shared client is never stamped and that two requests get distinct snapshots. Both assertions fail against the previous interceptor.

A note for the reporter

This fixes a real cross-request bleed, but it may not be the whole of what apache/skywalking#14095 observes: a concurrent overwrite yields a sibling request's trace from milliseconds ago, not one from ~832 seconds ago on the same thread. If the stale sw8 survives this patch, the remaining suspect is a leaked TracingContext on the event loop — TraceSegment#relatedGlobalTrace only replaces the trace id while it is still a NewDistributedTraceId, so a reused leaked context keeps emitting its original trace id. Local verification against a real gateway is very welcome.

Since Spring Cloud Gateway 4.1.2 the outbound chain is assembled in
NettyRoutingFilter#filter but subscribed later, so the v412x plugin parked the
request's ContextSnapshot on the HttpClient returned by
NettyRoutingFilter#getHttpClient. That method returns the shared this.httpClient
bean unless a connect timeout is configured, so the write lands on one object for
the whole JVM while the read, HttpClientConnect#duplicate through
HttpClient#headers, happens at subscription time. Any request reaching
getHttpClient in between overwrites the snapshot, and the outbound sw8 header
carries a context that belongs to another request.

Hold the snapshot on a client derived per request instead. HttpClient#headers
duplicates the client and copies the header map only, which Spring Cloud Gateway
copies again one line later, so a chain that already duplicates on headers(),
request() and uri() gains a single duplication and no observer. The shared bean is
never written again, so HttpClientConnectDuplicateV412Interceptor cannot carry a
stale value forward, and untraced requests derive nothing because the isActive()
guard runs first.
@wu-sheng wu-sheng added bug Something isn't working plugin labels Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant