Tutorials/Copilot Studio/Automated Testing for Copilot Studio Agents with Microsoft Auth
Copilot Studiointermediate

Automated Testing for Copilot Studio Agents with Microsoft Auth

Streamline your deployment pipeline by automating agent validation using the Copilot Studio Kit and Azure AD authentication.

NA
Narmer Abader
@narmer · Published June 3, 2026

Every time you push a change to a Copilot Studio agent, you risk introducing regressions in its responses. Manually testing every conversation path is impractical, especially when you're iterating quickly. The Copilot Studio Kit solves this by letting you define automated test suites: you send a set of predefined messages to the agent and verify the replies.

When your agent uses Microsoft Authentication (backed by Entra ID), the test harness needs its own identity to obtain tokens. This guide shows you how to set up an Azure app registration, grant the right permissions, and wire everything together so your automated tests can run seamlessly.

Example Scenario

Let's say your team manages a customer support agent named Contoso Support. You want to create a test configuration that runs every time a build is promoted from development to production. The agent uses Microsoft Authentication, so you'll need to register a test application in Azure and configure the Copilot Studio Kit accordingly.

Agent Configuration PropertyExample Value
Configuration NameContoso Support – Auto Test
Configuration TypeTest Automation
User AuthenticationMicrosoft Authentication
Agent Identifiercontoso-support-bot (schema name)
Environment Id00000000-0000-0000-0000-000000000000
Client ID (from app registration)11111111-1111-1111-1111-111111111111
Tenant ID (from app registration)22222222-2222-2222-2222-222222222222

The rest of the article walks through how to collect and populate these values.

Step 1 – Create an Agent Configuration in Copilot Studio Kit

Open the Copilot Studio Kit solution and navigate to the Agent Configurations table. Create a new record with the following settings:

  • Name: Use a descriptive label (e.g., Contoso Support – Auto Test).
  • Configuration: Select Test Automation.
  • User Authentication: Choose Microsoft Authentication.

Do not save yet; you still need several values from the next steps.

Step 2 – Set the Agent to Use Microsoft Authentication

Inside Copilot Studio, open your agent and go to Settings. Under Authentication, choose Authenticate with Microsoft. After you publish the agent, it will only accept tokens issued by your Entra ID tenant, which secures the conversation in Teams, SharePoint, Power Apps, and Microsoft 365 Copilot.

Step 3 – Capture Agent Metadata

Still in the agent settings, go to Advanced > Metadata. Copy these two pieces of information:

  • Schema Name – This is the unique identifier of your agent. Map it to the Agent Identifier field in your configuration.
  • Environment ID – The Power Platform environment that hosts the agent. Map it to the Environment Id field.

Step 4 – Register an Application in Azure

Now you need a service principal that the Copilot Studio Kit can use to authenticate.

  1. Go to the Azure portal and open App registrations.
  2. Select New registration.
  3. Provide a name, e.g., CopilotStudio-Test-Automation.
  4. Choose Accounts in this organizational directory only (single tenant).
  5. Click Register.

When the registration is complete, note the Application (client) ID and Directory (tenant) ID from the overview page. These will be mapped to Client ID and Tenant ID in your agent configuration.

Step 5 – Find the Environment URL

The test automation flow needs a redirect URI that points to your agent's environment. In Copilot Studio, click the gear icon and open Session details. Under Power Platform Environment details, copy the Instance URL. It typically looks like https://contoso-environment.crm.dynamics.com/.

Step 6 – Add a Redirect URI for a Single‑Page Application

Return to your app registration in Azure. Navigate to Authentication and click Add a platform.

  • Choose Single-page application.
  • Paste the environment URL into the Redirect URIs field.
  • Enable both Access tokens and ID tokens.
  • Confirm Supported account types is set to My organization only (single tenant).

Save the changes.

Step 7 – Grant Power Platform API Permissions

Test automation calls the Power Platform API to invoke your agent. You must add the appropriate permission to the app registration.

  1. Go to API permissions and select Add a permission.
  2. Switch to the APIs my organization uses tab.
  3. Search for Power Platform API. If it does not appear, run the optional PowerShell script below first.
  4. Choose Delegated permissions and tick copilotstudio.copilots.invoke.
  5. Click Add permissions, then Grant admin consent for your organization.
Admin consent required

Without admin consent, the test harness will fail when it tries to acquire a token with the delegated permission. Make sure a Global Administrator or an appropriate Privileged Role Administrator performs this step.

Optional – Enable the Power Platform API

If the Power Platform API is not listed, it may not have a service principal in your tenant. Run the following PowerShell script as an administrator to create it:

powershellInstall Graph module and create Power Platform API service principal
Install-Module Microsoft.Graph -Scope CurrentUser -Repository PSGallery -Force
Connect-MgGraph
New-MgServicePrincipal -AppId 8578e004-a5c6-46e7-913e-12f58912df43 -DisplayName "Power Platform API"

After the script completes, refresh the API permissions blade; the Power Platform API will now appear.

Step 8 – Save the Agent Configuration

Fill any remaining fields in your agent configuration using the values you collected:

  • Client ID → Application (client) ID from the app registration.
  • Tenant ID → Directory (tenant) ID from the app registration.
  • Agent Identifier → Schema name from agent metadata.
  • Environment Id → Environment ID from agent metadata.

Save the record. Your configuration is now ready to be used in test runs.

Security Considerations

  • Use the principle of least privilege. The delegated permission copilotstudio.copilots.invoke only allows the test tool to invoke agents; it cannot create or modify them.
  • Keep the app registration in a single tenant to avoid unintended access.
  • Rotate secrets if you ever switch from delegated to application permissions (not covered here).
  • Monitor token issuance in Entra ID sign‑in logs to detect unusual authentication patterns.

Common Mistakes and Troubleshooting

  • Power Platform API not visible – Run the PowerShell script from Step 7 to create the service principal.
  • Redirect URI mismatch – The environment URL must exactly match the one from the session details. A trailing slash can cause a mismatch.
  • Admin consent not granted – Without consent, the test harness will receive an invalid_grant error when requesting a token.
  • Wrong authentication method – Ensure the agent is set to Authenticate with Microsoft and not another provider.
  • Missing Environment ID – Double‑check you copied the value from Advanced > Metadata, not a different section.

Recommendation

Incorporate the Copilot Studio Kit test runner into your CI/CD pipeline. After you configure Microsoft Authentication as described above, you can run tests automatically on every pull request or release. This catches conversation failures early and gives your team confidence when promoting changes to production.

References