Skip to content

feat: support external test packages in go#26

Merged
not-matthias merged 5 commits intomainfrom
cod-1505-support-external-test-packages-in-go
Oct 20, 2025
Merged

feat: support external test packages in go#26
not-matthias merged 5 commits intomainfrom
cod-1505-support-external-test-packages-in-go

Conversation

@not-matthias
Copy link
Copy Markdown
Member

@not-matthias not-matthias commented Oct 15, 2025

Fixes #23

Lots of changes required, because external test packages are quite different:

  • Added an example that reproduces the error
  • Added more tests for the benchmark discovery
  • Support external tests by moving them to the codspeed/ repository

@not-matthias not-matthias force-pushed the cod-1505-support-external-test-packages-in-go branch from 94573f4 to 5670a2c Compare October 17, 2025 17:01
@not-matthias not-matthias marked this pull request as ready for review October 17, 2025 17:01
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Add support for external Go test packages (packages with the _test suffix) in discovery, patching, templating, and build flow to fix missing benchmark discovery and execution (Fixes #23).

  • Discovery: Detect and handle external test packages; surface is_external and select correct files/imports/paths.
  • Build pipeline: Patch package declarations only where needed, move/rename external test files into a codspeed sub-package, and update the runner template to call unqualified functions for external tests.
  • Tests/fixtures: Add a new example project using external test packages and update integration and discovery snapshots accordingly.

Reviewed Changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
go-runner/testdata/projects/example-with-test-package/math_test.go Add external test package benchmarks for math package
go-runner/testdata/projects/example-with-test-package/math.go Add simple math package used by external tests
go-runner/testdata/projects/example-with-test-package/go.mod New module for the example project
go-runner/testdata/projects/example-with-test-package/fib_external/fib_test.go Add internal tests in separate directory
go-runner/testdata/projects/example-with-test-package/fib_external/fib_integration_test.go Add external test package tests + benchmark
go-runner/testdata/projects/example-with-test-package/fib_external/fib.go Fibonacci implementation for tests
go-runner/testdata/projects/example-with-test-package/fib/fib_test.go Internal tests and benchmarks
go-runner/testdata/projects/example-with-test-package/fib/fib_integration_test.go External test package tests
go-runner/testdata/projects/example-with-test-package/fib/fib.go Fibonacci implementation for tests
go-runner/testdata/projects/example-with-test-package/README.md Document the external test package scenario
go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@zerolog.snap Update integration results snapshot for zerolog
go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@zap.snap Update integration results snapshot for zap
go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@opentelemetry-go.snap Update integration results snapshot for opentelemetry-go
go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@hugo.snap Update integration results snapshot for hugo
go-runner/src/snapshots/codspeed_go_runner__integration_tests__assert_results_snapshots@example-with-test-package.snap New snapshot asserting external test package results
go-runner/src/integration_tests.rs Add new integration test case for example-with-test-package
go-runner/src/builder/templater.rs Handle external test packages: patch package decls, move/rename test files, install dep, generate runner in codspeed subdir
go-runner/src/builder/template.go Template changes to import only internal tests and call unqualified names for external tests
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@zerolog.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@zap.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@opentelemetry-go.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@hugo__identity.snap New discovery snapshot segment
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@hugo__common__htime.snap New discovery snapshot segment
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@hugo.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@golang-benchmarks.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@fzf.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@fuego.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example-with-test-package.snap New discovery snapshot for test package example
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example-with-main.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example-with-helper.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@example-with-dot-go-folder.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@cli-runtime.snap Update discovery snapshot
go-runner/src/builder/snapshots/codspeed_go_runner__builder__discovery__tests__discover_benchmarks@caddy.snap Update discovery snapshot
go-runner/src/builder/patcher.rs Split import patching vs package patching; add install_codspeed_dependency; add helpers to patch specific files or dir
go-runner/src/builder/mod.rs Build the whole codspeed package directory instead of a single file
go-runner/src/builder/discovery.rs Detect external test packages, select proper file/import lists, include is_external in benchmarks

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Oct 17, 2025

CodSpeed Performance Report

Merging #26 will not alter performance

Comparing cod-1505-support-external-test-packages-in-go (9532997) with main (90629b1)

Summary

✅ 24 untouched

@not-matthias not-matthias force-pushed the cod-1505-support-external-test-packages-in-go branch from 5670a2c to ba2150a Compare October 17, 2025 17:18
@not-matthias not-matthias force-pushed the cod-1505-support-external-test-packages-in-go branch from 32dc34e to 9532997 Compare October 20, 2025 09:26
Copy link
Copy Markdown
Member

@art049 art049 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lot of changes but looks good, and makes sense!

@not-matthias not-matthias merged commit 9532997 into main Oct 20, 2025
13 checks passed
@not-matthias not-matthias deleted the cod-1505-support-external-test-packages-in-go branch October 20, 2025 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Testing package name isn't recognised correctlly

3 participants