Regular testing is one of the most effective ways to maintain response quality. Run batch tests after every significant change to your agent’s configuration or knowledge base.
Accessing batch tests
Navigate to Testing in the left sidebar. You’ll see a list of all existing tests and their current status.Creating a test
1
Navigate to Testing
Click Testing in the left sidebar to open the batch testing page.
2
Click Create Test
Click the Create Test button in the top-right corner.
3
Enter test details
Give your test a descriptive name and select the agent you want to test. The name should reflect the purpose of the test — for example, “Post-KB Update — [Medicine Name]” or “Guardrail Compliance Check”.
4
Add questions
Add the questions you want your agent to answer. You can use any of the three methods below — or combine them.
5
Run the test
Click Run Test to begin. The RoseRx Platform sends each question to your agent sequentially and captures the response — an estimated runtime is shown based on the number of questions.
Each test can contain up to 100 questions. Questions run one at a time against the live model, so larger tests take longer — the estimated runtime is displayed before you start.
Adding questions
The RoseRx Platform provides three ways to add questions to a batch test:- Generate
- Upload CSV
- Manual
Let RoseRx’s AI generate questions automatically based on your agent’s configuration, knowledge base, and guardrails. This is the fastest way to create comprehensive test coverage.Simply click Generate and RoseRx will create a set of relevant questions tailored to your agent. You can review, edit, or remove any generated question before running the test.
Test statuses
Each batch test moves through the following statuses:While a test is running, the progress bar updates every few seconds. You can navigate away and return — the test will continue running in the background.
Reviewing results
Once a test completes, click into it to review the results. You can filter the results by:- Answer status — Success, Failed, Timeout, or Pending.
- Rating — Good, Acceptable, Poor, or Unrated.
Rating system
Each answer can be rated to track quality over time:
Each test also records a pass rate, giving you a quick quality benchmark across runs.
Question detail
Click any question in the results to see the full detail view:- Question — The full text of the test question.
- Answer — The complete agent response, along with the model used and response latency.
- Rating — Set or change the rating.
- Notes — Add internal notes about the response quality or any issues identified.
- Re-run — Run the question again, for example after fixing a knowledge base gap.
Test actions
From the test list or within a test, you can perform the following actions:- Rename — Change the test name.
- Duplicate — Create a copy of the test with all its questions (useful for re-running after changes).
- Export CSV — Download the test results as a CSV file for offline review or reporting.
- Generate report — Produce a printable report of the test results, ready to share with stakeholders.
- Delete — Permanently remove the test and its results.
- Add questions — Add additional questions to an existing test at any time.
Best practices
Test after KB changes
Run batch tests after updating your Knowledge Base to ensure the agent still provides accurate responses with the new content.
Test guardrail changes
After modifying guardrails or compliance settings, run tests that specifically probe the boundaries you’ve configured.
Build a regression suite
Create a standard set of test questions that you run regularly. Duplicate tests to quickly re-run them after making changes.
Combine methods
Use AI-generated questions for broad coverage and manual questions for targeted edge cases. This gives you the best of both worlds.