Needed advice for a fix in collector service

[AnmolxSingh]

Trying to fix a TODO

jaeger/cmd/collector/app/span_processor.go

Line 290 in 952c77a

// TODO calling sanitizer here contradicts the comment in enqueueSpan about immutable Process.

this TODO requires the sanitizer function to be performed on spans either before they enter the queue or after they exit the queue but not when they are in the queue because they share resources.

jaeger/cmd/collector/app/span_processor.go

Line 316 in 952c77a

// Note: spans may share the Process object, so no changes should be made to Process

so i just wanted to know if we wanted to fix this what would be a better approach.

1. change the calling of sanitizer func from current position i.e. before enqueue or after dequeue.
   OR
2. make a deep copy and then call sanitizer func.

Thanks!

[dosubot[bot]]

The best approach is to ensure the sanitizer function is only called when span ownership is clear—either before the span enters the queue or after it exits, but not while it is in the queue. This is because the queue stores pointers to spans, so spans are shared by reference and mutating them in place while in the queue can cause race conditions or side effects if accessed concurrently by multiple goroutines or components. The sanitizer functions in Jaeger mutate spans in place and do not create deep copies, and there is no existing utility for deep copying spans in the codebase (span_processor.go, sanitizer.go).

Making a deep copy before sanitization would prevent shared resource issues but would introduce a performance cost and add complexity, since deep copying is not natively supported. The current code and related issues reinforce that sanitizer functions should only be called when the span is not shared (issue #7221).

So, the recommended fix is to move the sanitizer call to a point where the span is not shared—either before enqueueing (if you can guarantee no other references exist) or after dequeuing but before any further sharing. Avoid deep copying unless you have a strong reason and have measured the performance impact.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[AnmolxSingh]

I have thought to sanitize the spans them right after they are converted from otel traces but cannot find where it is done.

[dosubot[bot]]

For OpenTelemetry (OTel) traces, sanitization happens right after conversion in the collector: the pushTraces method in cmd/collector/app/span_processor.go calls sanitizerv2.Sanitize(td) on the batch of OTel traces before writing them to storage or further processing. This means OTel traces are sanitized immediately after conversion, before any queuing or sharing occurs. You can see this in the implementation of pushTraces, where sanitization is the first step after receiving the traces from the OTel exporter helper (source).

For non-OTel spans, sanitization is still performed after dequeuing, which is a different code path. If you want to ensure all spans are sanitized immediately after conversion (before entering the queue), you would need to adjust the enqueue logic for those formats as well.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}