Evaluation Harnesses Are an Ethics Problem
We talk about AI safety as if it lives in a policy memo or a fine-tuned refusal layer. In practice, most of what we call "safe" is whatever our evaluation harness decides to measure — and what we choose not to measure quietly becomes acceptable behavior.
That makes eval design an ethics problem before it's an engineering one.
What an eval suite actually encodes
Every benchmark is a value statement:
- What scenarios are in the test set?
- Who wrote the scenarios, and whose harms are represented?
- What counts as a pass — exact match, human rubric, another model as judge?
When a team ships with evals that only cover helpfulness and fluency, they have implicitly decided that consent violations, demographic bias, and overconfidence are out of scope until someone files a ticket.
The harness is the spec. If it's not in the harness, it's not in the product — regardless of what the README claims.
Three layers worth separating
1. Capability evals
Does the model do the task? Accuracy, latency, tool-use success. These are necessary and relatively uncontroversial.
2. Behavior evals
Does the model stay inside its lane? Refusals, citation fidelity, escalation when uncertain. This is where most "agent safety" work actually lives.
3. Impact evals
Who gets hurt when it fails? Red-teaming with adversarial personas, studying downstream effects on real workflows, measuring whether users over-trust outputs.
Most teams stop at layer 1. Layer 2 is uneven. Layer 3 is rare — and it's the layer ethics actually cares about.
A minimal bar before ship
Before calling a system production-ready, I want at least:
- Documented failure modes — not vibes, a list with examples.
- Regression tests for known bad behaviors from prior runs.
- Human review protocol for anything the harness flags at low confidence.
- Versioned eval sets so "we fixed it" is reproducible, not anecdotal.
None of this requires solving alignment. It requires treating measurement with the same seriousness we treat training loss curves.
The uncomfortable part
Better evals slow you down. They surface problems you would rather defer. They create paper trails when something goes wrong.
That friction is the point. Ethics that only exists in principle documents, without a harness that can fail in CI, isn't ethics — it's branding.