feat(toolkit-docs-generator): majority-version coherence filter#907
feat(toolkit-docs-generator): majority-version coherence filter#907
Conversation
Engine API returns tools at mixed versions for the same toolkit (e.g. Github tools at @3.1.3 alongside stale notification tools at @2.0.1). These stale tools survive through the pipeline because the data source has no version filtering, and --skip-unchanged preserves them indefinitely since the API response stays consistent. Add filterToolsByMajorityVersion() that computes the version shared by the most tools in a toolkit and drops tools at other versions. Applied in both fetchAllToolkitsData() and fetchToolkitData() (when no explicit version is passed), so stale tools are removed before reaching the diff or merger layers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
- Move extractVersion to utils/fp.ts as the single source of truth; data-merger.ts re-exports it for backward compatibility - Replace lexicographic version tie-break with numeric semver comparison so multi-digit components (e.g. 9.0.0 vs 10.0.0) sort correctly - Remove redundant array spread in fetchAllToolkitsData filter - Add test for multi-digit version tie-break edge case Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Should it be "most" or "greatest"? What if we drop 80 bad tools and keep 10 good ones? |
arcade-mcp's normalize_version() allows semver with pre-release tags
(1.2.3-alpha.1) and build metadata (1.2.3+build.456). The previous
compareVersions used .split(".").map(Number) which produced NaN for
these suffixes. Now strips pre-release and build metadata before
parsing numeric MAJOR.MINOR.PATCH for comparison.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
The majority-version approach was wrong: if a new release has fewer tools (e.g. consolidation), the filter would keep the OLD version with more tools and drop the new one. The correct logic is to always keep tools at the highest (newest) version — stale tools are always at older versions. Renamed: getMajorityVersion → getHighestVersion, filterToolsByMajorityVersion → filterToolsByHighestVersion. Added test: "keeps newer version even when it has fewer tools". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| } | ||
|
|
||
| return best || null; | ||
| }; |
There was a problem hiding this comment.
Version comparison and filtering use inconsistent matching
Low Severity
The getHighestVersion function, using compareVersions, ignores pre-release tags, causing it to pick the first encountered version in a tie. If a pre-release version is selected as 'highest' due to input order, the subsequent exact string filter in filterToolsByHighestVersion will incorrectly drop all stable tools with the same numeric core, violating semver precedence.
Reviewed by Cursor Bugbot for commit ea5ab6b. Configure here.
When the version filter removes stale tools, the tool list changes and the summary signature no longer matches. Without this fix, if no LLM generator is configured (--skip-summary), the summary is silently dropped. Now falls back to the previous summary instead of losing it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 417a51f. Configure here.
| }; | ||
| // Re-export extractVersion so existing consumers (e.g. toolkit-diff.ts) | ||
| // that import it from this module continue to work. | ||
| export { extractVersion } from "../utils/fp.js"; |
There was a problem hiding this comment.
Ambiguous star-export silently drops extractVersion from package API
Medium Severity
The new extractVersion definition in utils/fp.ts is picked up by utils/index.ts via export * from "./fp.js". The backward-compat re-export added in data-merger.ts (export { extractVersion } from "../utils/fp.js") is picked up by merger/index.ts via export * from "./data-merger.js". Since src/index.ts does export * from both merger/index.js and utils/index.js, TypeScript treats extractVersion as an ambiguous binding and silently drops it from the package's public exports — a regression from the previous state where it was only exported via merger.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 417a51f. Configure here.
|
wouldn't we want to continue documenting the deprecated tools (but with a deprecated banner) since the engine is still hosting them? |
4 participants
Summary
filterToolsByMajorityVersion()utility that computes the version shared by the most tools in a toolkit and drops tools at other versionsfetchAllToolkitsData()andfetchToolkitData()(when no explicit version is passed) inCombinedToolkitDataSource@2.0.1alongside 35+ tools at@3.1.3) surviving through the docs pipeline indefinitelyProblem
The Engine
/v1/tool_metadataendpoint returns tools at mixed versions for the same toolkit. The docs generator trusts this as-is, grouping tools by toolkit ID with no version filtering. The--skip-unchangedflow compounds this — it correctly sees no change run after run (because the API keeps returning the same stale tools), preserving old output indefinitely.Solution
After grouping tools by toolkit ID, compute the majority version (the version shared by the most tools), then keep only tools at that version. This drops stale tools before they reach the diff or merger. Explicit version parameters bypass the filter.
Test plan
getMajorityVersionandfilterToolsByMajorityVersion(11 tests)CombinedToolkitDataSourceversion filtering (3 new tests)🤖 Generated with Claude Code
Note
Medium Risk
Changes which tools are included in generated docs/diffs by automatically dropping older-version tools when Engine returns mixed versions, which can cause large output churn if current data has inconsistencies.
Overview
Adds a highest-version coherence filter so each toolkit keeps only tools at the numerically highest
@versionwhen no explicit version is requested, dropping stale tools from older releases in bothfetchToolkitData()andfetchAllToolkitsData().Introduces
utils/version-coherence.ts(plus exports) and movesextractVersionintoutils/fp.tswhile re-exporting it fromdata-merger.tsfor compatibility. Updates the merger to preserve a previous toolkit summary when the tool signature changes but summary generation is disabled (no LLM), and adds unit/scenario tests covering mixed-version filtering and diff/summary behavior.Reviewed by Cursor Bugbot for commit 417a51f. Bugbot is set up for automated code reviews on this repo. Configure here.