fix(ci_visibility): itr skipped counts suites or tests#14681
Conversation
This reverts commit 410e991.
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 266 ± 2 ms. The average import time from base is: 268 ± 2 ms. The import time difference between this PR and base is: -1.72 ± 0.07 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate revert-14257-gnufede/itr-skipped-count-fix (e617e45) with baseline main (6d6cc54) 📈 Performance Regressions (2 suites)📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 4.376µs (SLO: <10.000µs 📉 -56.2%) vs baseline: +1.2% Memory: ✅ 37.631MB (SLO: <39.000MB -3.5%) vs baseline: +4.8% ✅ ospathbasename_noaspectTime: ✅ 1.081µs (SLO: <10.000µs 📉 -89.2%) vs baseline: -0.8% Memory: ✅ 37.631MB (SLO: <39.000MB -3.5%) vs baseline: +4.6% ✅ ospathjoin_aspectTime: ✅ 6.060µs (SLO: <10.000µs 📉 -39.4%) vs baseline: ~same Memory: ✅ 37.729MB (SLO: <39.000MB -3.3%) vs baseline: +4.9% ✅ ospathjoin_noaspectTime: ✅ 2.299µs (SLO: <10.000µs 📉 -77.0%) vs baseline: +1.1% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.9% ✅ ospathnormcase_aspectTime: ✅ 3.492µs (SLO: <10.000µs 📉 -65.1%) vs baseline: +0.6% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.8% ✅ ospathnormcase_noaspectTime: ✅ 0.566µs (SLO: <10.000µs 📉 -94.3%) vs baseline: -0.7% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +4.8% ✅ ospathsplit_aspectTime: ✅ 5.835µs (SLO: <10.000µs 📉 -41.7%) vs baseline: 📈 +18.1% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +5.0% ✅ ospathsplit_noaspectTime: ✅ 1.601µs (SLO: <10.000µs 📉 -84.0%) vs baseline: +0.6% Memory: ✅ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +4.7% ✅ ospathsplitdrive_aspectTime: ✅ 3.725µs (SLO: <10.000µs 📉 -62.8%) vs baseline: +0.7% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.9% ✅ ospathsplitdrive_noaspectTime: ✅ 0.697µs (SLO: <10.000µs 📉 -93.0%) vs baseline: -0.4% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +4.7% ✅ ospathsplitext_aspectTime: ✅ 4.611µs (SLO: <10.000µs 📉 -53.9%) vs baseline: +0.7% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +5.0% ✅ ospathsplitext_noaspectTime: ✅ 1.380µs (SLO: <10.000µs 📉 -86.2%) vs baseline: -1.8% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +4.9% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.143µs (SLO: <20.000µs 📉 -84.3%) vs baseline: +0.8% Memory: ✅ 32.145MB (SLO: <34.000MB -5.5%) vs baseline: +4.7% ✅ 1-count-metrics-100-timesTime: ✅ 214.101µs (SLO: <250.000µs 📉 -14.4%) vs baseline: +0.6% Memory: ✅ 32.165MB (SLO: <34.000MB -5.4%) vs baseline: +5.1% ✅ 1-distribution-metric-1-timesTime: ✅ 2.995µs (SLO: <20.000µs 📉 -85.0%) vs baseline: +2.4% Memory: ✅ 32.067MB (SLO: <34.000MB -5.7%) vs baseline: +5.0% ✅ 1-distribution-metrics-100-timesTime: ✅ 192.027µs (SLO: <220.000µs 📉 -12.7%) vs baseline: +0.3% Memory: ✅ 32.165MB (SLO: <34.000MB -5.4%) vs baseline: +5.2% ✅ 1-gauge-metric-1-timesTime: ✅ 2.388µs (SLO: <20.000µs 📉 -88.1%) vs baseline: 📈 +14.6% Memory: ✅ 32.165MB (SLO: <34.000MB -5.4%) vs baseline: +4.9% ✅ 1-gauge-metrics-100-timesTime: ✅ 126.514µs (SLO: <150.000µs 📉 -15.7%) vs baseline: +0.8% Memory: ✅ 32.086MB (SLO: <34.000MB -5.6%) vs baseline: +4.9% ✅ 1-rate-metric-1-timesTime: ✅ 3.129µs (SLO: <20.000µs 📉 -84.4%) vs baseline: +0.4% Memory: ✅ 32.145MB (SLO: <34.000MB -5.5%) vs baseline: +4.9% ✅ 1-rate-metrics-100-timesTime: ✅ 216.714µs (SLO: <250.000µs 📉 -13.3%) vs baseline: +1.6% Memory: ✅ 32.067MB (SLO: <34.000MB -5.7%) vs baseline: +4.9% ✅ 100-count-metrics-100-timesTime: ✅ 21.814ms (SLO: <23.500ms -7.2%) vs baseline: +0.3% Memory: ✅ 32.165MB (SLO: <34.000MB -5.4%) vs baseline: +5.0% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.038ms (SLO: <2.250ms -9.4%) vs baseline: +2.3% Memory: ✅ 32.086MB (SLO: <34.000MB -5.6%) vs baseline: +4.5% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.294ms (SLO: <1.550ms 📉 -16.5%) vs baseline: -0.7% Memory: ✅ 32.165MB (SLO: <34.000MB -5.4%) vs baseline: +4.9% ✅ 100-rate-metrics-100-timesTime: ✅ 2.224ms (SLO: <2.550ms 📉 -12.8%) vs baseline: +0.6% Memory: ✅ 32.106MB (SLO: <34.000MB -5.6%) vs baseline: +4.8% ✅ flush-1-metricTime: ✅ 4.103µs (SLO: <20.000µs 📉 -79.5%) vs baseline: -0.9% Memory: ✅ 32.106MB (SLO: <34.000MB -5.6%) vs baseline: +4.8% ✅ flush-100-metricsTime: ✅ 182.472µs (SLO: <250.000µs 📉 -27.0%) vs baseline: +0.7% Memory: ✅ 32.165MB (SLO: <34.000MB -5.4%) vs baseline: +5.2% ✅ flush-1000-metricsTime: ✅ 2.219ms (SLO: <2.500ms 📉 -11.3%) vs baseline: +1.4% Memory: ✅ 32.834MB (SLO: <34.500MB -4.8%) vs baseline: +4.5% 🟡 Near SLO Breach (4 suites)🟡 djangosimple - 30/30✅ appsecTime: ✅ 20.497ms (SLO: <22.300ms -8.1%) vs baseline: -0.4% Memory: ✅ 65.327MB (SLO: <67.000MB -2.5%) vs baseline: +4.9% ✅ exception-replay-enabledTime: ✅ 1.348ms (SLO: <1.450ms -7.0%) vs baseline: -0.3% Memory: ✅ 64.077MB (SLO: <67.000MB -4.4%) vs baseline: +4.6% ✅ iastTime: ✅ 20.424ms (SLO: <22.250ms -8.2%) vs baseline: -0.5% Memory: ✅ 65.290MB (SLO: <67.000MB -2.6%) vs baseline: +4.7% ✅ profilerTime: ✅ 15.348ms (SLO: <16.550ms -7.3%) vs baseline: +0.4% Memory: ✅ 53.533MB (SLO: <54.500MB 🟡 -1.8%) vs baseline: +4.8% ✅ resource-renamingTime: ✅ 20.572ms (SLO: <21.750ms -5.4%) vs baseline: +0.1% Memory: ✅ 65.287MB (SLO: <67.000MB -2.6%) vs baseline: +4.8% ✅ span-code-originTime: ✅ 26.161ms (SLO: <28.200ms -7.2%) vs baseline: ~same Memory: ✅ 67.236MB (SLO: <69.500MB -3.3%) vs baseline: +4.7% ✅ tracerTime: ✅ 20.474ms (SLO: <21.750ms -5.9%) vs baseline: ~same Memory: ✅ 65.329MB (SLO: <67.000MB -2.5%) vs baseline: +4.9% ✅ tracer-and-profilerTime: ✅ 22.120ms (SLO: <23.500ms -5.9%) vs baseline: -0.1% Memory: ✅ 66.230MB (SLO: <67.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ tracer-dont-create-db-spansTime: ✅ 19.310ms (SLO: <21.500ms 📉 -10.2%) vs baseline: ~same Memory: ✅ 65.330MB (SLO: <66.000MB 🟡 -1.0%) vs baseline: +4.8% ✅ tracer-minimalTime: ✅ 16.652ms (SLO: <17.500ms -4.8%) vs baseline: ~same Memory: ✅ 65.017MB (SLO: <66.000MB 🟡 -1.5%) vs baseline: +4.8% ✅ tracer-nativeTime: ✅ 20.505ms (SLO: <21.750ms -5.7%) vs baseline: +0.2% Memory: ✅ 71.074MB (SLO: <72.500MB 🟡 -2.0%) vs baseline: +5.0% ✅ tracer-no-cachesTime: ✅ 18.456ms (SLO: <19.650ms -6.1%) vs baseline: ~same Memory: ✅ 65.391MB (SLO: <67.000MB -2.4%) vs baseline: +5.0% ✅ tracer-no-databasesTime: ✅ 18.816ms (SLO: <20.100ms -6.4%) vs baseline: ~same Memory: ✅ 64.937MB (SLO: <67.000MB -3.1%) vs baseline: +4.9% ✅ tracer-no-middlewareTime: ✅ 20.148ms (SLO: <21.500ms -6.3%) vs baseline: ~same Memory: ✅ 65.312MB (SLO: <67.000MB -2.5%) vs baseline: +4.8% ✅ tracer-no-templatesTime: ✅ 20.302ms (SLO: <22.000ms -7.7%) vs baseline: ~same Memory: ✅ 65.371MB (SLO: <67.000MB -2.4%) vs baseline: +4.9% 🟡 errortrackingdjangosimple - 6/6✅ errortracking-enabled-allTime: ✅ 18.078ms (SLO: <19.850ms -8.9%) vs baseline: -0.2% Memory: ✅ 65.185MB (SLO: <66.500MB 🟡 -2.0%) vs baseline: +4.7% ✅ errortracking-enabled-userTime: ✅ 18.088ms (SLO: <19.400ms -6.8%) vs baseline: -0.1% Memory: ✅ 65.246MB (SLO: <66.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ tracer-enabledTime: ✅ 18.167ms (SLO: <19.450ms -6.6%) vs baseline: +0.3% Memory: ✅ 65.198MB (SLO: <66.500MB 🟡 -2.0%) vs baseline: +5.0% 🟡 flasksqli - 6/6✅ appsec-enabledTime: ✅ 3.954ms (SLO: <4.200ms -5.9%) vs baseline: ~same Memory: ✅ 63.332MB (SLO: <66.000MB -4.0%) vs baseline: +4.9% ✅ iast-enabledTime: ✅ 2.452ms (SLO: <2.800ms 📉 -12.4%) vs baseline: ~same Memory: ✅ 58.843MB (SLO: <60.000MB 🟡 -1.9%) vs baseline: +4.7% ✅ tracer-enabledTime: ✅ 2.082ms (SLO: <2.250ms -7.5%) vs baseline: +0.2% Memory: ✅ 52.005MB (SLO: <54.500MB -4.6%) vs baseline: +4.6% 🟡 otelspan - 22/22✅ add-eventTime: ✅ 45.255ms (SLO: <47.150ms -4.0%) vs baseline: +0.4% Memory: ✅ 45.153MB (SLO: <47.000MB -3.9%) vs baseline: +5.0% ✅ add-metricsTime: ✅ 322.729ms (SLO: <344.800ms -6.4%) vs baseline: +0.2% Memory: ✅ 552.739MB (SLO: <562.000MB 🟡 -1.6%) vs baseline: +4.8% ✅ add-tagsTime: ✅ 290.992ms (SLO: <314.000ms -7.3%) vs baseline: ~same Memory: ✅ 553.894MB (SLO: <563.500MB 🟡 -1.7%) vs baseline: +4.7% ✅ get-contextTime: ✅ 82.752ms (SLO: <92.350ms 📉 -10.4%) vs baseline: +0.4% Memory: ✅ 40.117MB (SLO: <46.500MB 📉 -13.7%) vs baseline: +4.7% ✅ is-recordingTime: ✅ 43.020ms (SLO: <44.500ms -3.3%) vs baseline: +0.7% Memory: ✅ 44.524MB (SLO: <47.500MB -6.3%) vs baseline: +4.8% ✅ record-exceptionTime: ✅ 61.795ms (SLO: <67.650ms -8.7%) vs baseline: +0.3% Memory: ✅ 40.451MB (SLO: <47.000MB 📉 -13.9%) vs baseline: +4.8% ✅ set-statusTime: ✅ 48.834ms (SLO: <50.400ms -3.1%) vs baseline: +0.2% Memory: ✅ 44.552MB (SLO: <47.000MB -5.2%) vs baseline: +4.9% ✅ startTime: ✅ 42.194ms (SLO: <43.450ms -2.9%) vs baseline: +0.1% Memory: ✅ 44.584MB (SLO: <47.000MB -5.1%) vs baseline: +5.0% ✅ start-finishTime: ✅ 83.148ms (SLO: <88.000ms -5.5%) vs baseline: ~same Memory: ✅ 34.603MB (SLO: <46.500MB 📉 -25.6%) vs baseline: +4.9% ✅ start-finish-telemetryTime: ✅ 84.661ms (SLO: <89.000ms -4.9%) vs baseline: ~same Memory: ✅ 34.583MB (SLO: <46.500MB 📉 -25.6%) vs baseline: +4.9% ✅ update-nameTime: ✅ 44.118ms (SLO: <45.150ms -2.3%) vs baseline: ~same Memory: ✅ 44.811MB (SLO: <47.000MB -4.7%) vs baseline: +4.8%
|
also fix comment in regression test
CI Visibility: Fixes some bad behavior in test skipping count, introduced in #14257