Prepare a workspace
Give every candidate a controlled starting point.
On this page
Start from readable project files
evals/query-from-schema/
prompt.md
judge.md
eval.ts # optional preparation
workspace/
schema.sql
package.json
bun.lock
eval.ts
import type { Eval } from "@hona/openeval";
export default {
prepare: [
{ cwd: ".", argv: ["bun", "install", "--frozen-lockfile"] },
],
} satisfies Eval;
Without a workspace, the candidate starts in an empty directory. Preparation runs before the initial evidence snapshot and earns no candidate credit.
Pin external source code
eval.ts · replace the repository and commit
import type { Eval } from "@hona/openeval";
export default {
workspace: {
repository: "https://github.com/example/project.git",
commit: "0123456789abcdef0123456789abcdef01234567",
ref: "main",
overlay: "fixtures/dependencies",
},
prepare: [
{ cwd: "checkout", argv: ["bun", "install", "--frozen-lockfile"] },
],
} satisfies Eval;
| Setting | Purpose |
|---|---|
| repository + commit | Credential-free HTTPS URL and a full 40-character commit SHA |
| ref | Initial branch or revision |
| overlay | Readable files applied to the fetched checkout |
| checkout | Checkout directory name; defaults to checkout |
Declare a synthetic Git history
eval.ts
import type { Eval } from "@hona/openeval";
export default {
workspace: {
ref: "main",
revisions: [
{ ref: "main", directory: "fixtures/main", message: "Initial project" },
{ ref: "next", directory: "fixtures/next", message: "Update schema" },
],
overlay: "fixtures/local",
},
} satisfies Eval;
Revision directories are applied in order and committed by the host. Use overlays for intentional local state. The candidate sees ordinary project files and Git metadata.