Skip to main content

HarnessDeps

@forge/monorepo

@forge/monorepo / backend/src / HarnessDeps

Type Alias: HarnessDeps

HarnessDeps = object

Defined in: backend/src/evaluation/index.ts:25

Properties​

clock?​

readonly optional clock?: () => string

Defined in: backend/src/evaluation/index.ts:46

Returns​

string

graders?​

readonly optional graders?: readonly Grader[]

Defined in: backend/src/evaluation/index.ts:30

Deterministic graders. Defaults to all of them; overridable so a test can narrow the set.

judge?​

readonly optional judge?: Grader

Defined in: backend/src/evaluation/index.ts:38

The judge, for a refusal with no structural signal.

Optional. Without it, a prose-only refusal is reported as unscoreable rather than as a failure — a case the harness could not decide is not the same as a case the platform got wrong, and scoring it as a failure would make the gate fail for want of a model rather than for want of quality.

judgeModelId?​

readonly optional judgeModelId?: string

Defined in: backend/src/evaluation/index.ts:45

The judge's model id, recorded on every judged result.

Passed rather than read from the grader: a Grader is deliberately ignorant of what backs it, and a judged result without a model id cannot answer "did the score move because the model changed".

run​

readonly run: EvalRunner

Defined in: backend/src/evaluation/index.ts:28

Produces the output for a case. The platform under test, injected so the harness is not the runtime.

runId?​

readonly optional runId?: () => string

Defined in: backend/src/evaluation/index.ts:47

Returns​

string

store​

readonly store: EvaluationStore

Defined in: backend/src/evaluation/index.ts:26