{"title":"Best practices for testing the Copilot capability","url":"https://jonas-paprotka.de/docs/dev-itpro/developer/ai-test-copilot-bestpractices/","by":"JOP-AL","license":"Copyright (c) Microsoft Corporation. CC BY 4.0, https://creativecommons.org/licenses/by/4.0/; code samples MIT. From https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/ai-test-copilot-bestpractices, MicrosoftDocs/dynamics365smb-devitpro-pb at 828b6ee (2026-11-19). Changed: includes filled in, links to this site, and for agents shortened without dropping a fact.","credit":"Microsoft Corporation, CC BY 4.0","origin":"https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/ai-test-copilot-bestpractices","source":{"name":"Business Central developer and IT pro docs","repo":"https://github.com/MicrosoftDocs/dynamics365smb-devitpro-pb","commit":"828b6ee72fcd75755627cd6011f3d20899217ab2","date":"2026-11-19"},"checked":"2026-11-19","description":"Learn about best practices for testing the Copilot capability in AL.","date":"2026-10-02","meta":{"topic":"concept-article"},"outline":[{"level":1,"title":"Best practices for testing the Copilot capability","slug":"best-practices-for-testing-the-copilot-capability","at":0,"chars":3786},{"level":2,"title":"Key considerations for LLM-based features","slug":"key-considerations-for-llm-based-features","at":318,"chars":2750},{"level":3,"title":"Measure accuracy at scale","slug":"measure-accuracy-at-scale","at":871,"chars":314},{"level":3,"title":"Create realistic test cases","slug":"create-realistic-test-cases","at":1185,"chars":300},{"level":3,"title":"Validate output safety and tone","slug":"validate-output-safety-and-tone","at":1485,"chars":864},{"level":3,"title":"Test for cross-language compatibility","slug":"test-for-cross-language-compatibility","at":2349,"chars":368},{"level":3,"title":"Track changes across model versions","slug":"track-changes-across-model-versions","at":2717,"chars":351},{"level":2,"title":"Best practices checklist","slug":"best-practices-checklist","at":3068,"chars":340},{"level":2,"title":"Related information","slug":"related-information","at":3408,"chars":378}],"text":"# Best practices for testing the Copilot capability\nTesting AI-driven features like Copilot requires a tailored approach to ensure accuracy, safety, and reliability. This article outlines best practices for testing the Copilot capability in AL, focusing on the unique challenges posed by Large Language Models (LLMs).\n## Key considerations for LLM-based features\nUnlike deterministic systems, LLM-based features require new testing approaches. Consider the following:\n Non-determinism: Identical prompts might produce different results.\n Context sensitivity: Small changes in input phrasing can significantly affect output quality.\n Bias and safety: Language models might reflect or amplify societal and cultural biases.\n> NOTE\n> Always include human-in-the-loop evaluation for user-facing or high-impact scenarios, even if you have automated parts of the test pipeline.\n### Measure accuracy at scale\nTo evaluate Copilot performance broadly:\n Use Evaluation to automate testing and verify thousands of prompts automatically.\n Score outputs for correctness, relevance, and completeness.\n Flag low-confidence responses for human review.\nEvaluation allows automating AI testing at scale.\n### Create realistic test cases\nDesign tests that reflect actual usage:\n Base cases on real-world tasks and intents.\n Apply anonymized user logs for representative prompts.\n Include various phrasing styles and complexity levels.\nBuild test suites that cover both common and edge-case user scenarios.\n### Validate output safety and tone\nA Copilot feature must be accurate-but also safe, respectful, and aligned with your organization's voice. Outputs that appear correct can still fail due to inappropriate tone or harmful implications.\nTest to ensure your Copilot:\n Avoids bias and stereotyping:\n   Uses inclusive, nongendered language.\n   Resists reproducing cultural or societal biases.\n Maintains professional tone:\n   Aligns with your brand's voice.\n   Avoids sarcasm or humor unless appropriate.\n Filters harmful content:\n   Blocks hate speech, profanity, and explicit material.\n   Mitigates prompt abuse and adversarial input.\n Handles adversarial prompts safely:\n   Defends against prompt injection and chaining.\n   Gracefully manages nonsense or confusing queries.\n> TIP\n> Integrate both quality and safety tests into your CI/CD pipeline using Evaluation.\n### Test for cross-language compatibility\nIf your Copilot supports multiple languages:\n Validate input/output handling in each supported locale.\n Involve native speakers to assess linguistic and cultural accuracy.\n Avoid assuming that English test results apply globally.\nLocalizing your test approach is essential to ensure consistent user experience across regions.\n### Track changes across model versions\nLLMs evolve quickly-and updates can unintentionally affect feature behavior. Use regression testing to:\n Rerun existing test suites on updated models.\n Compare current vs. previous outputs side-by-side.\n Identify unexpected changes or regressions.\nMaintain historical baselines for consistency across releases.\n## Best practices checklist\nTask|Description\nAutomate testing|Evaluate large-scale output with batch runs\nDefine realistic prompts|Reflect real-world user behavior\nReview for safety and tone|Detect harmful or biased content\nLocalize testing|Validate multilingual output accuracy\nRun version comparisons|Track regressions from model updates\n## Related information\nBusiness Central Copilot Evaluation<https://github.com/microsoft/BCApps/blob/main/src/Tools/AI%20Test%20Toolkit/README.md>\nBuild the Copilot capability in AL<ai-build-capability-in-al>\nTest the Copilot capability in AL<ai-test-copilot>\nCreate datasets<ai-test-copilot-datasets>\nWrite AI tests<ai-test-copilot-ai-tests>\nEvaluation<ai-test-copilot-testtool>"}