Build error: cannot use t (variable of type `*codspeed.T`) as `*testing.T` value

[marten-seemann]

I'm working on enabling Codspeed for quic-go, but the job is failing due to a compilation error:

  [INFO  codspeed_go_runner] Building binary for package: quic
  [WARN  codspeed_go_runner::builder] Build command output: 
  [WARN  codspeed_go_runner::builder] Build command error output: # github.com/quic-go/quic-go/internal/synctest
  Error:     internal/synctest/synctest_go125.go:11:16: cannot use t (variable of type *codspeed.T) as *"testing".T value in argument to synctest.Test
  Error:     internal/synctest/synctest_go125.go:11:19: cannot use f (variable of type func(t *codspeed.T)) as func(*"testing".T) value in argument to synctest.Test

Here's my theory (without having done a lot of research): It looks like Codspeed is replacing testing.T with codspeed.T, and this doesn't work in this case. As I'm passing around testing.T a lot, I assume that the problem only occurs when testing.T is passed to a function across package boundaries.

Is my suspicion correct? What can I do to properly build my code?

[not-matthias]

Hey, thanks for creating this issue!

Here's my theory (without having done a lot of research): It looks like Codspeed is replacing testing.T with codspeed.T, and this doesn't work in this case. As I'm passing around testing.T a lot, I assume that the problem only occurs when testing.T is passed to a function across package boundaries.

Yep, that's somewhat correct. We automatically patch the imports to point to our fork of testing which allows us to internally add instrumentation to gather the results and communicate with our runner. In your case, it looks like there's 2 issues:

1. Our testing fork is based on Go 1.24 where the testing/synctest package was still experimental
2. We're not patching the import of sub-packages in testing

I'll bump our fork and modify the logic to properly handle syntest correctly.

[not-matthias]

Hey @marten-seemann, we've fixed the issue and released a new version.

The CodSpeed action will automatically pull the latest version so you don't have to do anything except rerunning your CI workflow! Since you have a lot of benchmarks, I would recommend creating multiple shards to run them in parallel to speedup your workflow: https://codspeed.io/docs/benchmarks/go#sharding-benchmarks-in-parallel-ci-jobs