An agent can give a polished answer and still take the wrong route. It might look up the wrong course or invent an answer when a tool fails. A good evaluation checks both what the user receives and the decisions the workflow made along the way.
Step-by-step setup
Set up a simple pass-or-fail test sheet
What you need A spreadsheet and a test version of the sample n8n workflow or another agent you control.
Make a test worksheet
Create columns for sample input, expected tool call, expected answer, actual answer and pass or fail.
Write expected outcomes
Use the three beginner cases below. Decide what a pass looks like before running the model.
Run each case manually
Use the agent’s test panel with fictional data. Review the tool result in the execution details and copy a short summary into the sheet.
Fix and rerun
After changing a prompt or a tool, rerun the same cases. Add one new test whenever you find a failure that could happen again.
Three test cases to start with
Known course → catalogue lookup → return the approved topics
Unknown course → lookup returns no match → say no match
Missing batch date → no date in lookup → ask admissions instead of guessing
How to tell it worked
All three cases meet their expected outcomes. For the missing-date case, the output includes no invented date.
Write expected outcomes before running the agent
Start with representative cases from the task. Add ambiguous requests, an empty input and an example that requires a reviewer. For each one, describe the allowed answers and tool calls. A good result for an unverified price might be an explicit request for the latest fees rather than a number.
Inspect the run as well as the final text
Record which tool was called, which fields were sent, what the tool returned and whether the approval step was reached. Reviewing a trace can show whether a correct-looking final answer came from the right facts. Use test data and record only what you need to diagnose problems.
Check failure and repetition
Use a mock tool that replies slowly, returns an empty result or produces an error. Send one sample event twice and verify you get the expected number of resulting records. Run several iterations on model-dependent cases because the same input can produce different outputs.
Improve the basics before adding more tools
Sort failures by severity. Correcting an unapproved action or invented claim matters more than polishing a sentence. Re-run your evaluation set after changing a prompt, tool description, model or server. Add a test for each real failure you expect to encounter again.
Try it yourself
Make a test table with a normal enquiry, an unknown course, an out-of-date price, a duplicate ID, a tool timeout and a rejected review. Record pass or fail for the final answer, tool choice and any external action separately.
Common questions
What is a workflow trace?
A trace records the sequence of important steps in a run, such as tool calls, handoffs and validation decisions.
What makes a good approval test?
Try approval, rejection and expiry. Verify that the action can run only after a current approval for the correct item.
