Add ADR-0020: Foundry Evals integration#4731
Merged
alliscode merged 2 commits intomicrosoft:mainfrom Mar 20, 2026
Merged
Conversation
markwallace-microsoft
added
the
documentation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new Architectural Decision Record (ADR) documenting the proposed architecture for integrating Azure AI Foundry Evaluations with the agent-framework, including shared concepts and intended Python/.NET API shapes.
Changes:
- Introduces ADR text describing evaluator protocol + orchestration approach, including
EvalItem/split strategies and Foundry/local evaluators. - Documents intended cross-language (Python/.NET) API parity and .NET MEAI (
Microsoft.Extensions.AI.Evaluation) alignment. - Adds usage examples for agent/workflow evaluation and mixed evaluator providers.
You can also share your feedback on Copilot code review. Take the survey.
alliscode
force-pushed
the
af-foundry-evals-adr
branch
from
1cd0d6b to
2fecc22
Compare
moonbox3
reviewed
Mar 17, 2026
alliscode
force-pushed
the
af-foundry-evals-adr
branch
2 times, most recently
from
39201a4 to
deed5e4
Compare
alliscode
force-pushed
the
af-foundry-evals-adr
branch
4 times, most recently
from
2d1b3f1 to
93ef327
Compare
TaoChenOSU
reviewed
Mar 17, 2026
alliscode
changed the title
alliscode
force-pushed
the
af-foundry-evals-adr
branch
2 times, most recently
from
7e6c4f6 to
2e6fa21
Compare
crickman
reviewed
Mar 18, 2026
alliscode
force-pushed
the
af-foundry-evals-adr
branch
from
2e6fa21 to
1502d83
Compare
crickman
reviewed
Mar 18, 2026
crickman
approved these changes
Mar 18, 2026
alliscode
force-pushed
the
af-foundry-evals-adr
branch
3 times, most recently
from
aeae780 to
a33aad4
Compare
Captures the design for integrating Azure AI Foundry Evaluations with agent-framework. Key decisions: - EvalItem with conversation (list[Message]) as single source of truth - query/response derived from configurable conversation split strategies - Tools as list[FunctionTool] (including auto-extracted MCP tools) - FoundryEvals provider with auto-detection of evaluator capabilities - LocalEvaluator with @function_evaluator decorator for local checks - Consistent Python/C# APIs: evaluate_agent, evaluate_workflow Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
alliscode
force-pushed
the
af-foundry-evals-adr
branch
from
a33aad4 to
f986539
Compare
alliscode
changed the title
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TaoChenOSU
approved these changes
Mar 20, 2026
This was referenced Apr 3, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Captures the design for integrating Azure AI Foundry Evaluations with agent-framework. Key decisions:
Motivation and Context
Description
Contribution Checklist