user-invocable: false description: > Authenticate for Copilot Studio evaluation API and SDK chat. Caches a token that is shared across run-eval and chat-sdk skills. Run this before any eval or SDK chat workflow. Requires an App Registration with MakerOperations and Copilots.Invoke permissions. allowed-tools: Bash(node *eval-api.bundle.js auth *), Read, Glob context: fork agent: copilot-studio-test
Test Agent Authentication
Authenticate with the Power Platform API for evaluation and SDK chat workflows.
This caches a token in the "test-agent" slot that is shared by run-eval and chat-sdk.
Step 1: Find the workspace
Locate the agent workspace (directory containing .mcs/conn.json):
Glob: **/agent.mcs.yml
The conn.json provides the tenant ID automatically — no need to ask the user for it.
Step 2: Get the client ID
CRITICAL: You MUST present the FULL configuration checklist below when asking for the client ID. Do NOT omit any items — users frequently miss the redirect URI and public client settings, which causes auth failures.
Ask the user for their App Registration Client ID and show them all of the following requirements in your message:
What is your App Registration Client ID for the Evaluation API?
Before sharing it, please verify your App Registration has all of these settings:
- Redirect URI: Under Authentication > Mobile and desktop applications > redirect URI set to
http://localhost- Public client: Under Authentication > Advanced settings > Allow public client flows set to Yes
- API permissions: Under API permissions, these delegated permissions on the Power Platform API:
CopilotStudio.MakerOperations.Read(list test sets, get results)CopilotStudio.MakerOperations.ReadWrite(start evaluation runs)CopilotStudio.Copilots.Invoke(SDK chat — send utterances to the agent)- Admin consent: Granted for the tenant (green checkmarks in the API permissions list)
Missing any of these will cause authentication to fail.
If the user doesn't have one, guide them:
How to create an App Registration:
- Go to Azure Portal > App Registrations
- Click New registration, name it (e.g., "Copilot Studio Eval")
- Under API permissions > Add a permission > APIs my organization uses > search Power Platform API
- Select Delegated permissions and add:
CopilotStudio.MakerOperations.ReadCopilotStudio.MakerOperations.ReadWriteCopilotStudio.Copilots.Invoke- Click Grant admin consent for the tenant
- Under Authentication > Add a platform > Mobile and desktop applications > set redirect URI to
http://localhost- Still in Authentication, scroll to Advanced settings and set Allow public client flows to Yes (required — without this the interactive browser login will fail with an AADSTS error)
- Copy the Application (client) ID from the Overview page
Step 3: Authenticate
Run the auth command (opens a browser — no device code needed):
node ${CLAUDE_SKILL_DIR}/../../scripts/eval-api.bundle.js auth --workspace <path> --client-id <id>
If successful, you'll see:
{"status": "ok", "message": "Authentication successful. Token cached for subsequent commands."}
Return the client ID and workspace path to the caller — they need these for subsequent run-eval and chat-sdk calls.
Troubleshooting
- HTTP 403 / InsufficientDelegatedPermissions: The app registration is missing required permissions. Guide the user through adding them (Step 2).
- AADSTS700016 / app not found in directory: The client ID doesn't exist in the agent's tenant. The app must be registered in the same tenant as the Copilot Studio environment (check
conn.json>AccountInfo.TenantId). - AADSTS7000218 / request body must contain client_secret or client_assertion: The app registration is not configured as a public client. Go to Authentication > Advanced settings > set Allow public client flows to Yes.
- SDK chat hangs after auth: The cached token may have stale permissions. Clear the cache and re-authenticate:
- Delete
~/.copilot-studio-cli/test-agent.cache.json - Run
security delete-generic-password -s "copilot-studio-cli" -a "test-agent"to clear the macOS Keychain entry - Re-run the auth command
- Delete