agentura run

Run your eval suite locally

Usage

bash

agentura run

What it does

Reads agentura.yaml, runs all eval suites against your agent endpoint, and prints a summary. This is the same logic used in CI on every PR.

Example output

Running eval suites...

✓ accuracy (golden_dataset)  12/15 passed  score: 0.80
✓ quality (llm_judge)         —            score: 0.85
✓ speed (performance)        15/15 passed  score: 1.00

All suites passed.

Exit codes

Code	Meaning
0	All suites passed
1	One or more suites failed or configuration validation error

Use the exit code in your own scripts:

bash

agentura run && echo "Ready to push"

Requirements

agentura.yaml in current directory
Agent endpoint must be running and reachable
For llm_judge suites: GROQ_API_KEY must be set