Validate

The validate command checks evaluation files for schema errors without running them.

Usage

agentv validate evals/my-eval.yaml
agentv validate evals/greeting.eval.ts

Validate multiple files:

agentv validate "evals/**/*.yaml" "evals/**/*.ts"

What It Checks

• YAML/JSONL syntax and TypeScript config default exports
• Required fields: id, input data (input or prompt matrix vars), and at least one grader under assert
• Grader references (command paths, prompt files)
• Target references match entries in providers.yaml
• Rubric structure and field types

For TypeScript eval configs, agentv validate imports and materializes the file through the same core loader used by agentv eval. Supported TypeScript config filenames are *.eval.ts and *.eval.mts; arbitrary .ts helper files are ignored during directory and glob expansion.

When to Use

• Before running evaluations to catch config errors early
• In CI/CD pipelines as a pre-check
• After editing eval files to verify correctness

agentv validate replaces the old eval mock dry-run use case for schema and configuration checks. It does not execute targets and does not produce quality scores. When you need no-live-LLM quality validation, run against an oracle/reference target or use frozen transcript/replay fixtures so graders see real candidate output.