{"title":"Write agent tests","url":"https://jonas-paprotka.de/docs/dev-itpro/developer/ai-test-copilot-agent-tests/","by":"JOP-AL","license":"Copyright (c) Microsoft Corporation. CC BY 4.0, https://creativecommons.org/licenses/by/4.0/; code samples MIT. From https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/ai-test-copilot-agent-tests, MicrosoftDocs/dynamics365smb-devitpro-pb at 828b6ee (2026-11-19). Changed: includes filled in, links to this site, and for agents shortened without dropping a fact.","credit":"Microsoft Corporation, CC BY 4.0","origin":"https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/ai-test-copilot-agent-tests","source":{"name":"Business Central developer and IT pro docs","repo":"https://github.com/MicrosoftDocs/dynamics365smb-devitpro-pb","commit":"828b6ee72fcd75755627cd6011f3d20899217ab2","date":"2026-11-19"},"checked":"2026-11-19","description":"Learn how to write AL tests for Business Central agents using the Library - Agent helpers to verify end-to-end agent behavior.","date":"2026-10-02","meta":{"topic":"concept-article"},"outline":[{"level":1,"title":"Write agent tests","slug":"write-agent-tests","at":0,"chars":18970},{"level":2,"title":"Step-by-step: write an agent test","slug":"step-by-step-write-an-agent-test","at":1268,"chars":11718},{"level":3,"title":"Step 1 - define the test codeunit and run the turn loop","slug":"step-1---define-the-test-codeunit-and-run-the-turn-loop","at":1517,"chars":5827},{"level":4,"title":"Initialize the agent","slug":"initialize-the-agent","at":2601,"chars":2347},{"level":4,"title":"Run the turn loop","slug":"run-the-turn-loop","at":4948,"chars":2396},{"level":3,"title":"Step 2 - author the dataset","slug":"step-2---author-the-dataset","at":7344,"chars":1472},{"level":3,"title":"Step 3 - handle interventions declaratively","slug":"step-3---handle-interventions-declaratively","at":8816,"chars":1679},{"level":3,"title":"Step 4 - prepare per-turn data","slug":"step-4---prepare-per-turn-data","at":10495,"chars":1266},{"level":3,"title":"Step 5 - configure suite-level setup (optional)","slug":"step-5---configure-suite-level-setup-optional","at":11761,"chars":1225},{"level":2,"title":"Configure the evaluation suite","slug":"configure-the-evaluation-suite","at":12986,"chars":565},{"level":2,"title":"Advanced - manual task management","slug":"advanced---manual-task-management","at":13551,"chars":3906},{"level":3,"title":"Task lifecycle","slug":"task-lifecycle","at":13977,"chars":1663},{"level":3,"title":"Manual intervention handling","slug":"manual-intervention-handling","at":15640,"chars":1307},{"level":3,"title":"Manual output","slug":"manual-output","at":16947,"chars":510},{"level":2,"title":"Backwards compatibility: `question:` and GetQuestion","slug":"backwards-compatibility-question-and-getquestion","at":17457,"chars":933},{"level":2,"title":"Related information","slug":"related-information","at":18390,"chars":580}],"text":"# Write agent tests\nAn agent test evaluates the end-to-end behavior of a Business Central agent by sending it natural-language input and then asserting that it took the correct actions in the system. Unlike AI tests that evaluate prompt outputs, agent tests verify the full task execution loop: the agent receives input, decides what to do, interacts with Business Central, and completes - or pauses to ask the user for input - before continuing.\nThe AI Test Suite is the main driver of an agent test. It iterates the dataset, dispatches each turn to your test codeunit, and collects evaluation results. The `Library - Agent` codeunit is a reusable helper for interacting with the agent during a test. It reads the defined YAML structure and streamlines the agent interactions, so individual tests don't have to handle that wiring directly.\n> NOTE\n> Agent tests run against a live agent service and consume Copilot credits. Run them in sandbox environments with prepaid credits. See ai-test-copilot-testtool#copilot-credit-limits-for-agent-evaluation-suites for credit limits.\n> TIP\n> A complete working sample is available at SalesValidationAgent3P<https://github.com/microsoft/BCTech/tree/master/samples/BCAgents/SalesValidationAgent/test> on the BCTech repository.\n## Step-by-step: write an agent test\nThe recommended pattern is data-driven: the dataset describes the input to the agent and the expected outcome, and your test code runs a small turn loop that delegates the agent interaction to `Library - Agent`.\n### Step 1 - define the test codeunit and run the turn loop\nAn agent test is a standard AL test codeunit with `TestType = AITest`. It uses `Library - Agent` to run each turn and `AIT Test Context` to read dataset values. The codeunit follows a small pattern: an Initialize procedure that resolves and activates the agent, and a `[Test]` procedure that drives a `repeat ... until` loop over the dataset turns.\n```al\ncodeunit 50200 \"My Agent Accuracy Test\"\n{\n    Subtype = Test;\n    TestType = AITest;\n    TestPermissions = Disabled;\n    RequiredTestIsolation = Disabled;\n\n    var\n        AITTestContext: Codeunit \"AIT Test Context\";\n        LibraryAgent: Codeunit \"Library - Agent\";\n        Assert: Codeunit \"Library Assert\";\n        AgentUserSecurityId: Guid;\n        Initialized: Boolean;\n}\n```\n`RequiredTestIsolation = Disabled` is needed because agent tasks span transactions in a different session.\n> NOTE\n> Agent tests depend on datasets that describe the input sent to the agent and the expected outcome. Learn how to create one in ai-test-copilot-datasets#agent-test-datasets.\n#### Initialize the agent\nInitialize runs before the first turn. It resolves the agent under test, cleans up any tasks left from prior runs, and activates the agent. The agent identity can come from the evaluation suite (when set) or from your own setup - fall back to creating the agent programmatically if neither is available.\n```al\nlocal procedure Initialize()\nbegin\n    if Initialized then\n        exit;\n\n    // Optionally read the agent under test from the evaluation suite.\n    // This enables A/B testing — you can point a suite at a different agent\n    // without changing test code. Skip the call if your test always targets\n    // a specific agent.\n    LibraryAgent.GetAgentUnderTest(AgentUserSecurityId);\n\n    // GetAgentUnderTest returns a null GUID when no agent is configured\n    // on the suite. Fall back to your own lookup or to creating the agent\n    // programmatically.\n    if IsNullGuid(AgentUserSecurityId) then\n        AgentUserSecurityId := GetOrCreateAgent();\n\n    // Clean up any tasks left from prior runs before activating the agent.\n    // Running cleanup at the start of Initialize replaces a separate\n    // [TearDown] step and makes the test resilient to previous failures\n    // that didn't shut down cleanly.\n    LibraryAgent.StopTasks(AgentUserSecurityId);\n\n    LibraryAgent.EnsureAgentIsActive(AgentUserSecurityId);\n\n    Initialized := true;\nend;\n```\nGetOrCreateAgent is application-specific. A common pattern is to look up the agent identity in a setup record and create it on first run by using the Agent SDK:\n```al\nlocal procedure GetOrCreateAgent(): Guid\nvar\n    MyAgentSetup: Record \"My Agent Setup\";\nbegin\n    if MyAgentSetup.FindFirst() then\n        exit(MyAgentSetup.\"Agent User Security ID\");\n\n    // First run — create the agent and persist its identity.\n    // See \"Define and register an agent programmatically\" for the APIs.\n    exit(CreateMyAgent());\nend;\n```\n> TIP\n> Calling GetAgentUnderTest is optional. Use it when you want the evaluation suite to control which agent the test targets - for example, to A/B test two agent versions against the same dataset. Tests that always run against a specific agent can skip the call and assign AgentUserSecurityId directly. For details on creating an agent in code, see Define and register an agent programmatically<../ai/ai-agent-sdk-define-register>.\n#### Run the turn loop\nWith Initialize in place, write the `[Test]` procedure. It runs a `repeat ... until` loop that delegates each turn to `Library - Agent.RunTurnAndWait` and then to `Library - Agent.FinalizeTurn`. RunTurnAndWait reads the current turn's `query:`, dispatches it to the agent, and waits for completion. FinalizeTurn writes the turn output, validates intervention expectations declared in the dataset, and advances to the next turn.\n```al\n[Test]\nprocedure TestAgentReleasesOrders()\nvar\n    AgentTask: Record \"Agent Task\";\n    TurnSuccessful, ContinueWithNextTurn : Boolean;\n    ErrorReason: Text;\n    AgentStatusErr: Label 'The agent task did not complete successfully. Status: %1.', Comment = '%1 = task status';\nbegin\n    Initialize();\n\n    repeat\n        Clear(ErrorReason);\n\n        // [GIVEN] Prepare Business Central state for this turn\n        ApplyTurnSetup();\n\n        // [WHEN] Send the turn input to the agent and wait\n        TurnSuccessful := LibraryAgent.RunTurnAndWait(AgentUserSecurityId, AgentTask);\n\n        if TurnSuccessful then\n            TurnSuccessful := ValidateTurnCompletedSuccessfully(ErrorReason)\n        else\n            ErrorReason := StrSubstNo(AgentStatusErr, AgentTask.Status);\n\n        // [THEN] Record turn outcome and decide whether to continue\n        ContinueWithNextTurn := LibraryAgent.FinalizeTurn(AgentTask, TurnSuccessful, ErrorReason);\n    until not ContinueWithNextTurn;\n\n    Assert.IsTrue(TurnSuccessful, ErrorReason);\nend;\n```\nValidation reads the turn's expected_data from `AIT Test Context` and checks Business Central state. Return false with a populated ErrorReason on mismatch rather than calling `Error()` - that lets FinalizeTurn log the failure on the turn and (optionally) continue with the next turn.\n```al\nlocal procedure ValidateTurnCompletedSuccessfully(var ErrorReason: Text): Boolean\nvar\n    ExpectedData: Codeunit \"Test Input Json\";\n    ExpectedReleased: Integer;\n    ActualReleased: Integer;\nbegin\n    ExpectedData := AITTestContext.GetExpectedData();\n    ExpectedReleased := ExpectedData.Element('expected_released_count').ValueAsInteger();\n\n    ActualReleased := CountReleasedSalesOrders();\n    if ActualReleased <> ExpectedReleased then begin\n        ErrorReason := StrSubstNo('Expected %1 released orders, found %2.', ExpectedReleased, ActualReleased);\n        exit(false);\n    end;\n\n    exit(true);\nend;\n```\n### Step 2 - author the dataset\nAgent test datasets use the `query:` element to describe what the agent receives. A task-input query has at minimum a title (used as the task subject) and typically a from (sender display name) and a message (the natural-language instruction).\n```yaml\nsuite_setup: MY-AGENT-SETUP\ntests:\n  - name: AGENT_TEST_01\n    description: Process sales orders with a future shipment date\n    turns:\n      - query:\n          from: Jane Doe\n          title: \"Release all sales orders for $DateFormula-<CW+1M>$\"\n          message: \"Release all sales orders for $DateFormula-<CW+1M>$\"\n        expected_data:\n          expected_released_count: 2\n```\nA few things to call out:\n Always use the `turns:` array, even for single-turn tests. Multi-turn syntax is the supported format for both single-turn and multi-turn tests.\n Dates are expressed through placeholders - `$DateFormula-<formula>$` is calculated relative to WorkDate so the test doesn't drift over time. See ai-test-copilot-datasets#placeholders-for-dates for the full reference.\n expected_data is your validation contract. Most keys you put there are read by your own validator. One sub-key, intervention_request, is recognized by the framework - see Step 3.\n> NOTE\n> Older datasets use the legacy `question:` element instead of `query:`. Both are accepted by the framework; new datasets should use `query:`. See ai-test-copilot-datasets#backwards-compatibility-question about migration details.\n### Step 3 - handle interventions declaratively\nSome agents pause to ask the user for input before continuing. The framework validates these interventions for you based on what the dataset declares.\nDeclare an expected intervention in expected_data.intervention_request. The type value must be one of Assistance, ReviewMessage, ReviewRecord, ResumeTask, Retry, Approval, or Permission (the English names of the `Agent User Int Request Type` enum). suggestions is optional - when present, every code listed must appear in the agent's actual request.\n```yaml\nturns:\n  - query:\n      from: Jane Doe\n      title: \"Release all open sales orders\"\n      message: \"Release all open sales orders\"\n    expected_data:\n      intervention_request:\n        type: Assistance\n        suggestions:\n          - PROVIDE_DATE\n      expected_released_count: 0\n\n  - query:\n      intervention:\n        instruction: \"Use shipment date $DateFormula-<CW+1M>$\"\n    expected_data:\n      expected_released_count: 1\n```\nThe second turn uses a continuation query instead of a task input. You can respond either with an `instruction:` (free-text reply) or a `suggestion:` (one of the codes offered by the agent - mutually exclusive with instruction).\nFinalizeTurn enforces the contract automatically:\n If a turn declares intervention_request, the agent must have paused for an intervention with the matching type. If suggestions is listed, all expected codes must be present.\n If a turn does not declare intervention_request, the agent must not have paused for one. An unexpected intervention fails the turn.\nSo declare intervention_request on every turn where you expect the agent to pause; otherwise omit it.\n### Step 4 - prepare per-turn data\nA turn's `turn_setup:` block contains data to materialize in Business Central before the agent runs. The shape is application-defined - the framework hands the YAML sub-tree to your test code, which dispatches to record-creation handlers.\nA common convention is to use setup_actions with action_type and action_data:\n```yaml\nturn_setup:\n  setup_actions:\n    - action_type: create_sales_order\n      action_data:\n        - \"Sell-to Customer No.\": CUST001\n          \"Shipment Date\": \"$DateFormula-<CW+1M>$\"\n```\nIn AL, read the block via AITTestContext.GetTurnSetup and walk the JSON with `Test Input Json`:\n```al\nlocal procedure ApplyTurnSetup()\nvar\n    TurnSetup: Codeunit \"Test Input Json\";\n    Found: Boolean;\nbegin\n    TurnSetup := AITTestContext.GetTurnSetup(Found);\n    if not Found then\n        exit;\n\n    // Dispatch each action_type entry to a handler that creates records.\n    // Resolve $DateFormula-... values via Test Input Data Tools — values\n    // read through Test Input Json's typed accessors are resolved\n    // automatically.\nend;\n```\nFor a complete dispatcher implementation, check out the SalesValidationAgent3P<https://github.com/microsoft/BCTech/tree/master/samples/BCAgents/SalesValidationAgent/test> sample.\n### Step 5 - configure suite-level setup (optional)\nSuite-level setup holds data that needs to be created only once across all tests in a suite - typical examples are master records every test relies on (locations, customers, posting groups). Define it in a separate YAML file under `.resources/suite_setup/` and reference it by name from the test dataset:\n```yaml\n# .resources/suite_setup/MY-AGENT-SETUP.yaml\nname: MY-AGENT-SETUP\ndescription: Shared master data for the My Agent suite.\nsuite_setup:\n  setup_actions:\n    - action_type: create_customers\n      action_data:\n        - \"No.\": CUST001\n          Name: Validation Customer 01\n```\nApply it once per suite in your test codeunit, gated by IsSuiteSetupDone:\n```al\nlocal procedure SetupPerSuiteTestData()\nvar\n    SuiteSetup: Codeunit \"Test Input Json\";\nbegin\n    if AITTestContext.IsSuiteSetupDone() then\n        exit;\n\n    SuiteSetup := AITTestContext.GetEvalSuiteSetupDataInput();\n    ApplySuiteSetup(SuiteSetup);\n\n    AITTestContext.SetEvalSuiteSetupCompleted();\nend;\n```\nCall SetupPerSuiteTestData from Initialize. The IsSuiteSetupDone flag is sticky across runs - use the Reset Suite Setup action on the AI Eval Suite page after editing the `suite_setup:` content.\n## Configure the evaluation suite\nSet `TestType=\"Agent\"` and `TestRunnerId=\"130451\"` (`Test Runner - Isol. Disabled`) on the suite XML to mark it as an agent evaluation. Each `<Line>` points at a dataset and your test codeunit.\n```xml\n<Root>\n    <AITSuite Code=\"MY-AGENT\" Description=\"My Agent Accuracy Tests\" Dataset=\"MY-DATASET.YAML\"\n              Capability=\"My Agent Capability\" Frequency=\"Daily\" TestRunnerId=\"130451\" TestType=\"Agent\">\n        <Line CodeunitID=\"50200\" Description=\"My Agent - Accuracy\" Dataset=\"MY-DATASET.YAML\" />\n    </AITSuite>\n</Root>\n```\n## Advanced - manual task management\nRunTurnAndWait and FinalizeTurn cover the data-driven flow used by most agent tests. For scenarios that need finer control - for example, starting a task from product code, constructing custom messages, or polling intervention state - `Library - Agent` exposes lower-level helpers. The descriptions below summarize what each method does; see the codeunit symbols for full parameter lists.\n### Task lifecycle\n `CreateTaskAndWait(var AgentTaskBuilder)` - Create a task from a manually configured `Agent Task Builder` and wait for completion.\n `CreateMessageAndWait(var AgentTaskMessageBuilder)` - Append a message to an existing task and wait.\n `ContinueTaskAndWait(var AgentTask)` / `(var AgentTask, UserInput)` - Continue a paused task with default or custom free-text input.\n `WaitForTaskToComplete(var AgentTask)` - Block until a task finishes. Use this in end-to-end scenarios that start the task from product code (for example, by invoking an action) rather than from the library.\n `StopTasks(AgentUserSecurityId)` / `StopAllTasks()` - Cleanup helpers. The recommended flow calls StopTasks from Initialize; use these directly only when you need ad-hoc cleanup outside the standard turn loop.\n `SetAgentTaskTimeout(NewTimeout)` - Override the 30-minute default wait for all `…AndWait` methods.\nExample: building and starting a task manually.\n```al\n[Test]\nprocedure TestAgentReleasesOrders_Manual()\nvar\n    AgentTask: Record \"Agent Task\";\n    AgentTaskBuilder: Codeunit \"Agent Task Builder\";\n    AgentTaskMessageBuilder: Codeunit \"Agent Task Message Builder\";\n    TaskSuccessful: Boolean;\nbegin\n    Initialize();\n\n    AgentTaskMessageBuilder.Initialize('Jane Doe', 'Release all open sales orders');\n    AgentTaskBuilder.Initialize(AgentUserSecurityId, 'Release all open sales orders');\n    AgentTaskBuilder.AddTaskMessage(AgentTaskMessageBuilder);\n\n    TaskSuccessful := LibraryAgent.CreateTaskAndWait(AgentTaskBuilder, AgentTask);\n\n    Assert.IsTrue(TaskSuccessful, StrSubstNo('Agent task did not complete. Status: %1', AgentTask.Status));\nend;\n```\n### Manual intervention handling\n `RequiresUserIntervention(AgentTask)` - Returns true when the task is paused waiting for user input.\n `GetLastUserInterventionRequestDetails(...)` - Read the most recent intervention request, its annotations, and any offered suggestions.\n `GetUserInterventionRequestDetails(LogEntry, ...)` - Read the intervention request attached to a specific log entry.\n `CreateUserInterventionAndWait(var AgentTask, UserInput)` - Reply with free-text input and wait.\n `CreateUserInterventionFromSuggestionAndWait(var AgentTask, SuggestionCode)` - Reply with a suggestion code and wait.\n `ParseUserInterventionRequestType(Text)` - Convert a dataset string (Assistance, ReviewMessage, ReviewRecord, ResumeTask, Retry, Approval, Permission) to the corresponding `Agent User Int Request Type` enum value.\nExample: detect an intervention and respond.\n```al\nif LibraryAgent.RequiresUserIntervention(AgentTask) then\n    TaskSuccessful := LibraryAgent.CreateUserInterventionFromSuggestionAndWait(AgentTask, 'CONFIRM');\n```\nThe valid suggestion codes are defined by the agent developer in the agent's IAgentTaskExecution implementation, specifically in GetAgentTaskUserInterventionSuggestions. Check the agent's implementation to find the available codes before writing assertions against them.\n### Manual output\n `WriteTaskToOutput(var AgentTask, var Output)` - Serialize task details and log entries to JSON.\n `WriteTaskToOutput(var AgentTask, var Output, FromDateTime)` - Same, filtered to entries from a specific point in time. Useful in multi-turn scenarios where you want only the current turn's activity.\n `WriteTurnToOutput(var AgentTask, TurnSuccessful, ErrorReason)` - Sets the answer used by evaluators. FinalizeTurn calls this automatically; use it directly only when you bypass FinalizeTurn.\n## Backwards compatibility: `question:` and GetQuestion\nEarlier agent test datasets used a `question:` element instead of `query:`, and the matching API on `AIT Test Context` was `GetQuestion()`. Both forms are still accepted by the framework, so existing tests don't have to be rewritten. New tests should use `query:` and `GetQuery()` because:\n `query:` supports the task-input/intervention split (title / from / message / attachments vs. intervention.suggestion / intervention.instruction).\n RunTurnAndWait dispatches based on the `query:` shape - it doesn't recognize `question:` as input.\n The validation and continuation features (expected_data.intervention_request, multi-turn continuation queries) are layered on the `query:` form.\nWhen migrating, keep this mapping in mind:\nLegacy|Current\n`question:` (scalar text)|`query:` with title (and optional from / message)\n`AITTestContext.GetQuestion()`|`AITTestContext.GetQuery()`\n## Related information\nBusiness Central Copilot Evaluation<https://github.com/microsoft/BCApps/blob/main/src/Tools/AI%20Test%20Toolkit/README.md>\nSalesValidationAgent3P sample on BCTech<https://github.com/microsoft/BCTech/tree/master/samples/BCAgents/SalesValidationAgent/test>\nTest the Copilot capability in AL<ai-test-copilot>\nWrite AI tests<ai-test-copilot-ai-tests>\nai-test-copilot-datasets\nEvaluation<ai-test-copilot-testtool>\nPermissions in Evaluation<../ai/ai-development-toolkit-permissions>\nBest practices for testing the Copilot capability<ai-test-copilot-bestpractices>"}