Section 5 · Testing, Deployment, and Maintenance · 10% of exam
5.5 Reviewing Agent Behavior After Changes
Exam objective
Explain how to review and retest agent behavior after changes to instructions, actions, data, models, or deployment configuration.
Changes can affect more than one response
An agent is a connected configuration of routing, instructions, actions, variables, data, permissions, and channels. A change to one component can alter other paths. For example, a revised subagent description may change which requests are routed there; a new action output may affect a later filter; a data update may change grounded responses; and a model or prompt change may alter wording. Identify the exact change and the user outcome it could affect before testing. Record the agent and prompt-template versions involved so results can be traced to the correct configuration.
Run targeted regression tests
Retest the scenarios that directly exercise the changed component, then run the broader regression suite for important tasks. If an action changed, test its inputs, outputs, permissions, success path, and error path. If instructions or routing changed, test multiple phrasings, adjacent topics, and out-of-scope requests. If grounding changed, test questions that depend on updated content and verify sources. If a prompt template changed, test its supported input types and representative records, then test it as part of the agent workflow. Keep previously passing cases so new changes do not silently break reliable behavior.
Use traces and scores to find the cause
Review Testing Center results and Builder interaction details. A trace can show which subagent was selected, which actions ran, what information was returned, and where the path diverged from expectations. Recurring score failures can point to unclear instructions, missing knowledge, incorrect action selection, or a configuration issue. Compare behavior before and after the change, not only the newest response. If the failure occurs only in production, compare target settings, user context, licenses, permissions, active versions, connections, and source readiness. Fix the cause at the responsible layer and rerun the failed cases.
Release with a clear decision
Define what must pass before activation, especially for sensitive or high-impact tasks. A test suite can provide useful evidence and a release baseline, but it is not a guarantee that every live conversation will be correct. Keep a record of test outcomes, accepted limitations, and the activated version. After release, watch for new failure patterns and add important real-world cases to the suite. If an agent underperforms after a change, use a controlled rollback or new version process rather than making untracked edits directly in production. This makes maintenance repeatable and helps teams understand what changed and why.
TIP: After a meaningful change, retest both the affected path and nearby paths that share its routing, actions, variables, data, or prompt templates.
A passing test suite reduces uncertainty but cannot prove that a non-deterministic agent will handle every possible conversation correctly.
Extra concept: When a production issue is understood and fixed, add the scenario to the regression suite so the same failure is easier to catch later.
Official references:
- Use Test Results https://help.salesforce.com/s/articleView?id=ai.agent_testing_center_use_test_results.htm&language=en_US&type=5
- Build an Agent Testing and Observability Plan https://help.salesforce.com/s/articleView?id=ai.agent_test_prep.htm&language=en_US&type=5
- Agentforce Testing Center https://help.salesforce.com/s/articleView?id=ai.agent_testing_center.htm&language=en_US
Key takeaways
- Trace each change to the agent behavior or dependency it can affect.
- Retest targeted cases and broader regression paths using the correct agent and template versions.
- Use results to find root causes, document release evidence, and add new failures to future test suites.
