Let the model think before the JSON

This field note comes out of Tycho, the program that builds The Ghost: the grading prompt in question is the one Tycho's pipeline uses to score candidate decisions.

A grading prompt built for local open-weight models used a strict discipline: answer cold, emit JSON, nothing else. Ported as-is to Claude, Opus scored 6 of 14 exact — below a 27B local model's 8.

Adding one instruction — think through the item first, then produce JSON — changed the results: Sonnet went from 5/14 to 8/14, Opus from 6/14 to 7/14. The frontier models weren't weaker; the harness was built for a different engine.

Keep the bad rows. First-pass results stayed in the files, labelled as a harness-sensitivity datapoint. A three-point movement on one prompt sentence is information about measurement fragility. A score without its harness is a number without units.

One more finding from the same round: a newer local model needed ~6k tokens or it truncated its own reasoning mid-JSON — 8 of 20 items failed at the pipeline's default 900. Same lesson: the harness is where most first-pass variance lives.

Where this goes next

The next run puts every engine on one harness — think-then-JSON, same prompt, same budget. That shows whether the tight cluster of scores reflects the models or the porting. If it survives one harness, the sheet ran out of resolution; if not, the comparison is worth running again on equal ground.