The latency was the reasoning budget, not the model

This field note comes out of the local-model grading round for Tycho, the program that builds The Ghost.

Qwen3.8-27B ran at default "high" reasoning and measured 38–256 seconds per item, clustering near two minutes. Its predecessor did the same items in about 31 seconds. The natural read: the newer model was too slow.

A colleague said it "smells like we haven't gotten the model working properly yet." Fair point. Same slice, same prompt, reasoning off.

With reasoning disabled, Qwen3.8 was the fastest local engine in the round and finished all 300 items in 70 minutes. The latency had been the thinking budget.

Three specifics from the retest:

Check what landed in the response, not just the flag. The reasoning_effort parameter is accepted whether or not the runtime honours it. What proves it took effect is completion_tokens_details.reasoning_tokens and message.reasoning_content on the way back. All 300 reasoning-off rows came back with zero reasoning content; every earlier row had it. Without that audit trail, "we ran it with reasoning off" is an assertion.

"MLX is faster on Apple Silicon" is a rule of thumb. MLX quantizations landed hours after GGUF. Head-to-head on the same slice, the MLX 4-bit build ran ~40 s/item against GGUF's ~29. Worse, MLX accepted both knobs and silently ignored them; only llama.cpp/GGUF honoured them. GGUF also ships speculative decoding, which accounts for most of its win.

A 4× headline is relative to its own default. Dropping to reasoning_effort: low moved one item from 152 s to 34 s. Over a wider run, "low" still measured 38–256 s. The 4× was real relative to the high tier. It did not make the model fast. Off did.

Where this goes next

Accuracy with reasoning off was measured on the same 300-item slice but hasn't been compared here. That's the open question: does disabling reasoning cost little in agreement, or a lot? The answer decides whether the full pass runs locally at these speeds on hardware we own.