The lab record

Results

We run controlled experiments on our own agents and tooling — same brief, one variable, blind judging where it matters. This page is the public record: what we tested, how we tested it, and what the evidence showed, including the parts that surprised us.

Head-to-head build

The Chess Face-Off

The question

Two agents get the identical brief — build real-time online chess for two players — and are told only that they are competing. One carries the full DreamKid stack: project memory, design enforcement, and the house UX doctrine. The other builds bare. Does the stack show up in the product?

How it ran

  • Identical written brief, undisclosed judging criteria, separate working directories and ports.
  • Both builds verified with scripted two-player games through checkmate: real-time sync, illegal-move rejection, mid-game refresh recovery, and console-error sweeps.
  • Blind scoring by two judges — the owner on real hardware, the orchestrating agent on sealed notes written before the owner's verdict — across visual appeal, desktop experience, mobile experience, and stability.

Owner scorecard 32–14 · Agent scorecard 35–31, both for the equipped build

What the evidence showed

  • Both builds played legal chess and passed every functional probe — the gap was never correctness.
  • The equipped build held up on a real phone, where products actually get judged. The bare build passed emulated checks but lost decisively in hands-on scoring on looks and mobile feel.
  • Design enforcement drew from the house design library, giving the winner a consistent, production-grade finish — and when a project calls for a brand-new identity instead, that goes in the brief.
  • Judging on real hardware caught details emulation can't, like how iOS handles small inputs — which is why every test here ends on a real device.
The stack didn't make the agent smarter at chess — it made the product hold up where products actually get used.

Both builds are live. Create a game, send the link to a friend, and judge them yourself.

Equipped build: checkmate dialog over a muted editorial board
Equipped build, desktop — checkmate, scripted verification game
Bare build: win overlay above a classic walnut board
Bare build, desktop — same mate, different hands
Equipped build at phone width with a full-bleed board
Equipped build, 390px
Bare build at phone width with an inset board card
Bare build, 390px
Memory A/B

The Project Brain Test

The question

Same agent, same real production codebase, one difference: one run can read the project's memory — decisions, task queue, lessons, guardrails — and one is told the memory system does not exist. Does persistent memory change what an agent chooses to work on?

How it ran

  • Two isolated worktrees of the same repository, identical open-ended prompts to "continue the most important work."
  • Ground truth established first: the real task queue, priorities, and recorded decisions, independent of either agent.
  • Both runs audited against git and the memory record afterward — claims checked, not trusted.

What the evidence showed

  • The memory-equipped run matched the real queue, fixed a live cross-tenant security gap, found a second one, and left a full written trail for the next session.
  • The bare run anchored on a months-stale spec file it happened to find, built 700 lines against it, missed the queue's top security item entirely, and left no persistent record.
  • Both runs produced competent code. Only one produced the right code.
Memory didn't make the agent more capable — it made the agent aimed. Capability was never the bottleneck; direction was.
Guidance A/B

The Tool Notes Test

The question

Written when-and-how notes exist for every tool the agents use. Do agents behave differently with the notes than without them?

How it ran

  • Two identical sandbox projects seeded with the same traps: a binding prior decision, a known environment pitfall with a recorded workaround, and a guarded legacy file.
  • One agent received the notes, one did not; both were scored against the seeded ground truth.

What the evidence showed

  • Both agents passed the traps — the project record itself carries real protective weight.
  • The notes arm added process discipline: task lifecycle tracking, mid-flight activity reporting, and directive checks the bare arm skipped.
  • That discipline carries a modest token cost — the price of a complete paper trail.
Notes buy process, not correctness. The record protects the work; the notes make the work legible.