# eval-harness-first

> Build the evaluation harness that gates every fine-tuning run - golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use whe

- Source: https://doutorai.pt/en/components/skills__wshobson-agents__eval-harness-first
- Type: Skills
- Category: General
- Downloads: 0
- Site: Doutor AI (https://doutorai.pt/en) - AI tools for small businesses

An open-source community component, curated and adapted by Doutor AI.

## What it does and how to use it

A **skill** that teaches Claude Code to do general tasks well. It is packaged knowledge the model uses whenever it is relevant.

1. Install the skill (command above).
2. In Claude Code, describe what you need - the skill turns on by itself when the context fits.
3. No commands to memorize: you talk in plain language and the skill does the rest.

## Install (Claude Code)

```bash
mkdir -p .claude/skills/wshobson-agents__eval-harness-first && curl -fsSL https://doutorai.pt/data/lib/skills__wshobson-agents__eval-harness-first.md -o .claude/skills/wshobson-agents__eval-harness-first/SKILL.md
```

Paste this command into the terminal, in your project folder, and the component installs automatically.

## Content

```
---
name: eval-harness-first
description: Build the evaluation harness that gates every fine-tuning run - golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels.
---

# Eval Harness First

The Phase 0 gate for the whole plugin:
`finetuning-method-selection` and every downstream
skill assume this harness exists before a training
config gets written. The harness is not a run-end
side artifact - it is the data-curation engine. The
same labe

- 🔒 Full content on the Pro plan (€7.99/month) -
```

## Related components

- [PocketBase API Rules](https://doutorai.pt/en/components/skills__pocketbase-api-rules): API rules and filter expressions for PocketBase access control. Use when setting permissions, writing filter expressions, configuring who ca
- [PocketBase Collections](https://doutorai.pt/en/components/skills__pocketbase-collections): Collection and schema design for PocketBase. Use when creating collections, designing schemas, adding fields, setting up relations, or choos
- [PocketBase Deploy](https://doutorai.pt/en/components/skills__pocketbase-deploy): Production deployment for PocketBase. Use when deploying PocketBase to a server, setting up Docker, configuring systemd, reverse proxy (ngin
- [PocketBase Hooks](https://doutorai.pt/en/components/skills__pocketbase-hooks): Server-side JavaScript hooks for PocketBase (pb_hooks). Use when writing custom routes, event hooks, cron jobs, sending emails, making HTTP 
- [PocketBase Migrations](https://doutorai.pt/en/components/skills__pocketbase-migrations): Schema migrations and versioning for PocketBase. Use when creating migrations, managing schema versions, syncing collections between environ
- [PocketBase SDK](https://doutorai.pt/en/components/skills__pocketbase-sdk): JavaScript SDK usage for PocketBase client applications. Use when calling PocketBase from frontend or Node.js, authenticating users, subscri

## Save hours every week with AI, no hassle

**Doutor AI** gives you 1,859 ready-to-copy AI prompts, by industry and in plain English - plus a complete technical library for those who want to go further. [See plans →](https://doutorai.pt/en/pricing)

---
Content by Doutor AI (https://doutorai.pt/en). When quoting, please cite the source: https://doutorai.pt/en/components/skills__wshobson-agents__eval-harness-first