Benchmarks workflow failing

[kylebarron]

Run CodSpeedHQ/action@v4
Run RUNNER_VERSION=""
Run actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830
/usr/bin/docker exec  178f0317695dce9f82139275f9081a8aec73ef04f7dd2701b66832263a13c34f sh -c "cat /etc/*release | grep ^ID"
Cache hit for restore-key: codspeed-instruments-simulation-Linux-X64-6.14.0-1017-azure-4.10.6
Received 28216592 of 28216592 (100.0%), 96.1 MBs/sec
Cache Size: ~27 MB (28216592 B)
/usr/bin/tar -xf /__w/_temp/5417054b-fcd5-4185-bbe0-b17ffa02120d/cache.tgz -P -C /__w/async-tiff/async-tiff -z
Cache restored successfully
Cache restored from key: codspeed-instruments-simulation-Linux-X64-6.14.0-1017-azure-4.10.6
Run # Validate required inputs
/__w/_temp/16bbb15d-db27-4465-9102-f1d880b33748.sh: line 67: jq: command not found

https://github.com/developmentseed/async-tiff/actions/runs/22463565344/job/65063912947

cc @weiji14

I disabled this workflow for now

[weiji14]

TIL you could disable workflows at https://github.com/developmentseed/async-tiff/actions! I've re-enabled it to test #256, hoping that an apt install jq fixes it.

[kylebarron]

The workflow failed again https://github.com/developmentseed/async-tiff/actions/runs/22679967761/job/65747358424

Do you get any value from that CI job? I've never even looked at it.

[kylebarron]

I think we should at least reduce the frequency that the benchmarks workflow gets run. I heard at team week that async-tiff is our most expensive repo for Github Actions,

[weiji14]

Yeah let's trigger the benchmarks on an as needed basis (since it's easy to do in the web UI). We racked up just over 1000min YTD on the bench.yml workflow (not as much as the python workflows that were at least 3x higher, but still...)

That said, I reckon we should still run the benchmarks on releases to have some rolling baselines every version? Edit: actually doesn't work because release events are not supported by codspeed action 😅

[kylebarron]

Are the benchmarks over time collected somewhere? Or we need to go back into each actions run to look?

[weiji14]

https://codspeed.io/developmentseed/async-tiff

[kylebarron]

Oh wow I didn't know that existed.

Still, I feel like we aren't consistently making edits that are likely to change the benchmarking. Most of the time is just handling geotiff edge cases. For PRs that we expect to change performance, I think it's ok to run cargo bench locally on both branches 🤷‍♂️

[weiji14]

Benchmarking locally is a little fragile... I've opened #273 to have the benchmarks be triggered on workflow_dispatch or by labelling a PR with run/benchmark. See if that looks reasonable.