feat: support external test packages in go#26
Merged
not-matthias merged 5 commits intomainfrom Oct 20, 2025
Merged
Conversation
94573f4 to
5670a2c
Compare
There was a problem hiding this comment.
Pull Request Overview
Add support for external Go test packages (packages with the _test suffix) in discovery, patching, templating, and build flow to fix missing benchmark discovery and execution (Fixes #23).
- Discovery: Detect and handle external test packages; surface
is_externaland select correct files/imports/paths. - Build pipeline: Patch package declarations only where needed, move/rename external test files into a codspeed sub-package, and update the runner template to call unqualified functions for external tests.
- Tests/fixtures: Add a new example project using external test packages and update integration and discovery snapshots accordingly.
Reviewed Changes
Copilot reviewed 37 out of 37 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| go-runner/testdata/projects/example-with-test-package/math_test.go | Add external test package benchmarks for math package |
| go-runner/testdata/projects/example-with-test-package/math.go | Add simple math package used by external tests |
| go-runner/testdata/projects/example-with-test-package/go.mod | New module for the example project |
| go-runner/testdata/projects/example-with-test-package/fib_external/fib_test.go | Add internal tests in separate directory |
| go-runner/testdata/projects/example-with-test-package/fib_external/fib_integration_test.go | Add external test package tests + benchmark |
| go-runner/testdata/projects/example-with-test-package/fib_external/fib.go | Fibonacci implementation for tests |
| go-runner/testdata/projects/example-with-test-package/fib/fib_test.go | Internal tests and benchmarks |
| go-runner/testdata/projects/example-with-test-package/fib/fib_integration_test.go | External test package tests |
| go-runner/testdata/projects/example-with-test-package/fib/fib.go | Fibonacci implementation for tests |
| go-runner/testdata/projects/example-with-test-package/README.md | Document the external test package scenario |
| go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@zerolog.snap | Update integration results snapshot for zerolog |
| go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@zap.snap | Update integration results snapshot for zap |
| go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@opentelemetry-go.snap | Update integration results snapshot for opentelemetry-go |
| go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@hugo.snap | Update integration results snapshot for hugo |
| go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@example-with-test-package.snap | New snapshot asserting external test package results |
| go-runner/src/integration_tests.rs | Add new integration test case for example-with-test-package |
| go-runner/src/builder/templater.rs | Handle external test packages: patch package decls, move/rename test files, install dep, generate runner in codspeed subdir |
| go-runner/src/builder/template.go | Template changes to import only internal tests and call unqualified names for external tests |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@zerolog.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@zap.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@opentelemetry-go.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@hugo__identity.snap | New discovery snapshot segment |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@hugo__common__htime.snap | New discovery snapshot segment |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@hugo.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@golang-benchmarks.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@fzf.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@fuego.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example-with-test-package.snap | New discovery snapshot for test package example |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example-with-main.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example-with-helper.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example-with-dot-go-folder.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@cli-runtime.snap | Update discovery snapshot |
| go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@caddy.snap | Update discovery snapshot |
| go-runner/src/builder/patcher.rs | Split import patching vs package patching; add install_codspeed_dependency; add helpers to patch specific files or dir |
| go-runner/src/builder/mod.rs | Build the whole codspeed package directory instead of a single file |
| go-runner/src/builder/discovery.rs | Detect external test packages, select proper file/import lists, include is_external in benchmarks |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
go-runner/testdata/projects/example-with-test-package/README.md
Outdated
Show resolved
Hide resolved
go-runner/testdata/projects/example-with-test-package/fib_external/fib_integration_test.go
Outdated
Show resolved
Hide resolved
go-runner/testdata/projects/example-with-test-package/fib_external/fib_integration_test.go
Show resolved
Hide resolved
CodSpeed Performance ReportMerging #26 will not alter performanceComparing Summary
|
5670a2c to
ba2150a
Compare
32dc34e to
9532997
Compare
art049
approved these changes
Oct 20, 2025
Member
art049
left a comment
There was a problem hiding this comment.
lot of changes but looks good, and makes sense!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #23
Lots of changes required, because external test packages are quite different:
codspeed/repository