[uss_qualifier/reports] Add timing report#1336
[uss_qualifier/reports] Add timing report#1336BenjaminPelletier merged 5 commits intointeruss:mainfrom
Conversation
| <td>{{ format_time(row.total_time) }}</td> | ||
| <td>{{ format_time(row.average_time) }}</td> | ||
| <td>{{ round(row.query_fraction * 100, 1) }}%</td> | ||
| <td>{{ round(row.delay_fraction * 100, 1) }}%</td> |
There was a problem hiding this comment.
Since populating delay is a TODO, I would suggest to not display this value in the report.
| scenario=scenario_type, | ||
| total_time=summary.total_time, | ||
| average_time=summary.total_time / summary.instances, | ||
| query_fraction=summary.query_time.total_seconds() |
There was a problem hiding this comment.
In the u-space report, HeavyConcurrentTraffic represents 178% of queries. See below.
If my understanding is correct, I would expect the sum of all fractions to be 100%.
There was a problem hiding this comment.
Ah, yes, the queries in that case constitute 175.8% of the time the scenario takes because there are multiple queries happening at once (interestingly, I would have expected an average concurrency of more than 1.758). The current algorithm naively just adds up the time each query takes and divides that by total scenario time which is how this number gets bigger than 1. I've added some additional context information that should hopefully help a reader understand this interpretation.
Co-authored-by: Michael Barroco <michael@orbitalize.com> 65e179b
|
|
||
| class _GenerationError(RuntimeError): | ||
| def __init__(self, msg: str): | ||
| super().__init__(msg) |
There was a problem hiding this comment.
why having the __init__ constructor ? It doesn't seems to add anything
3 participants
This PR adds a new timing report artifact to uss_qualifier which reports where and how time is spent during the test.
It also cleans up how end_time is populated in scenario reports as previously it was often being accidentally omitted. Now, all different places it was populated have been replaced by population when the report is retrieved. This changes the timestamp from what it would have been by tiny fractions of a second, but I believe that is worthwhile for the reduced complexity and increased likelihood of consistency.