Copilot Agent Kit

Copilot Agent Kit, published by the Microsoft Copilot Accelerator Team, is a comprehensive set of capabilities designed to compliment and enhance the customer experience with the Microsoft Copilot Studio and Microsoft 365 Copilot products. The kit helps makers test custom agents, use a large language model to validate AI-generated content, and track aggregated key performance indicators.

Testing capabilities

Makers can configure and run test sets against the Copilot Studio APIs to verify that the agent responses are as expected. Test results can be enriched with additional data points from Azure Application Insights and from conversation transcripts in Dataverse. For AI-generated answers, AI Builder prompts are used to compare the generated answer with a sample answer or validation instructions. Rubrics Refinement is a powerful capability in Copilot Agent Kit that enables you to create, test, and iteratively improve reusable evaluation standards (rubrics) for AI-generated responses.

Agent Inventory

Agent Inventory can be used to get a tenant-wide view to all Copilot Studio custom agents in the organization, including detailed information on the features they are using, authentication mode, knowledge sources, orchestration type and more. Agent Inventory ships with a dashboard and the data can be exported for use in other applications.

Agent Review Tool

Agent Review Tool helps makers and administrators assess Copilot Studio agents, Microsoft 365 Declarative Agents, skills, and automations for quality, safety, configuration, orchestration, test coverage, and efficiency. It provides prioritized findings with supporting evidence and actionable recommendations.

Agent & Component Library

Agent & Component Library is a centralized hub for discovering, installing, and managing pre-built agent templates and reusable Copilot Studio components. Makers can browse a curated catalog, import only the assets they need, and quickly customize agents using ready-to-use templates and components without custom development.

Compliance Hub

Compliance Hub helps organizations define and enforce governance policies for Copilot Studio agents at scale. It continuously evaluates agent configurations collected via Agent Inventory against configurable risk thresholds, automatically creating compliance cases when violations are detected and tracking them through an SLA-driven review lifecycle.

Agent Insights Hub

Agent Insights Hub is a unified analytics dashboard for Microsoft Copilot Studio agents, combining telemetry, conversation data, performance metrics, error tracking, topic analytics, and credit consumption insights into a single view.

Agent Debugger

Agent Debugger is a diagnostic tool inside the Copilot Agent Kit Admin app that lets administrators load any recorded conversation and inspect every decision the agent made — step by step, with timing, token usage, knowledge sources, arguments, and observations — without leaving the Power Platform.

Agent Change Tracker

Agent Change Tracker helps makers monitor changes to Microsoft Copilot Studio agents. After an agent is added for tracking, supported saved changes from any maker are automatically captured and displayed in a chronological history showing what changed, who made the change, and when it occurred.

PowerShield

PowerShield enables controlled connector access in Power Platform through an approval-based DLP workflow. Makers can request access via a guided self-service wizard, while admins review and manage requests. All DLP policy changes are linked to PowerShield requests, ensuring governance, compliance, and auditability.

Conversation KPIs

Conversation KPIs help makers monitor custom agent performance by providing aggregated conversation insights in Dataverse, eliminating the need to analyze complex Copilot Studio conversation transcripts.

Conversation Analyzer

Conversation Analyzer allows makers to analyze the conversations of their custom agents using custom prompts to get additional insights.

Agent Value Summary

The Agent Value Component is a modular tool for classifying agents in Microsoft Copilot Studio by type, behavior, and value, helping organizations understand their role, align them with strategy, and measure their impact. Results are presented visually on Agent Value dashboard.