feat(llmobs): allow span processor to return None to omit spans#14260
feat(llmobs): allow span processor to return None to omit spans#14260Kyle-Verhoog merged 4 commits intomainfrom
Conversation
|
|
Kyle-Verhoog
force-pushed
the
llmobs/span-proc-drop
branch
from
c76893e to
e5b2d4c
Compare
Allow users to omit any traces they want by returning none in the user_span_processor method. Motivation: I am trying to prevent some auto traces from cluttering my Datadog observability dashboards. These traces create noise and make it harder to focus on the more critical traces. My LLM observability overview is filled with what is clustered as empty input which is incorrect. e.g. all embedding traces are are just a spammer. To do that, I am using the new span_processor, and the new way to omit specific spans, will be to return null by the relevant span processor. I have added a test for that. The only risk is that there is no indication that span was omitted but I was afraid that a debug log would be too spammy. But please let me know if that will help. Also there are some telemetric collected about sent span which might also need to be changed, will be great to hear your thoughts
Kyle-Verhoog
force-pushed
the
llmobs/span-proc-drop
branch
from
e5b2d4c to
e5f59b7
Compare
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 278 ± 6 ms. The average import time from base is: 277 ± 4 ms. The import time difference between this PR and base is: 0.3 ± 0.2 ms. The difference is not statistically significant (z = 1.25). Import time breakdownThe following import paths have shrunk:
|
Kyle-Verhoog
force-pushed
the
llmobs/span-proc-drop
branch
from
534d14a to
2bd5c8a
Compare
Performance SLOsCandidate: llmobs/span-proc-drop (06e9fef) 🔵 No Baseline Data (24 suites)🔵 coreapiscenario - 12/12 (2 unstable)🔵 No baseline data available for this suite
|
Allow users to omit any traces they want by returning none in the user_span_processor method. Motivation: I am trying to prevent some auto traces from cluttering my Datadog observability dashboards. These traces create noise and make it harder to focus on the more critical traces. My LLM observability overview is filled with what is clustered as empty input which is incorrect. e.g. all embedding traces are are just a spammer. To do that, I am using the new span_processor, and the new way to omit specific spans, will be to return null by the relevant span processor. I have added a test for that. The only risk is that there is no indication that span was omitted but I was afraid that a debug log would be too spammy. But please let me know if that will help. Also there are some telemetric collected about sent span which might also need to be changed, will be great to hear your thoughts Co-authored-by: amirbenami <amir@linxsecurity.io>
3 participants
Allow users to omit any traces they want by returning none in the user_span_processor method.
Motivation:
I am trying to prevent some auto traces from cluttering my Datadog observability dashboards. These traces create noise and make it harder to focus on the more critical traces. My LLM observability overview is filled with what is clustered as empty input which is incorrect. e.g. all embedding traces are are just a spammer.
To do that, I am using the new span_processor, and the new way to omit specific spans, will be to return null by the relevant span processor.
I have added a test for that.
The only risk is that there is no indication that span was omitted but I was afraid that a debug log would be too spammy. But please let me know if that will help. Also there are some telemetric collected about sent span which might also need to be changed, will be great to hear your thoughts
Checklist
Reviewer Checklist