Skip to content

Event app: .NET Africa Conference 2026

Event page

Continuous Quality Monitoring for .NET AI Agents with Amazon Bedrock

Time to be announced

Speakers

About this session

Most teams test an AI agent by chatting with it a few times, deciding it looks fine, and shipping it. That works until someone changes the prompt, swaps the model, or edits a tool definition, and the agent starts picking the wrong tool or giving confidently wrong answers. Nothing crashes. No error gets logged. The only signal is a support ticket a few days later.

This talk covers how to catch that kind of regression before it reaches users, using Amazon Bedrock AgentCore Evaluations. We'll build and instrument a .NET agent with OpenTelemetry, then run three demos live, scoring one session against built-in evaluators for correctness, helpfulness, and tool-use accuracy; adding an evaluation step to a GitHub Actions pipeline so a pull request fails automatically when agent quality drops; and checking evaluation scores against production traffic to catch drift after release. Attendees will walk away with working code and a pattern they can apply to their own .NET agents, regardless of which agent framework they're using, since AgentCore Evaluations works off OpenTelemetry traces rather than a specific SDK.