Alvaro Videla Tests JIT Assistance for LLM Arithmetic

Standard tool-use routes for arithmetic - such as PAL, ReAct, and Toolformer - parse the prompt text to extract operands and pass them to Python. Developer Alvaro Videla's Rune project asked a narrower question: can those tool arguments come from the model's own internal activations instead, under a strict no-parser boundary? The no-parser rule means the runtime route cannot see prompt text, regex spans, harness operands, or gold answers - only token IDs and activation vectors.

The activation-derived argument route succeeded in its core claim. Using a frozen Llama model, Videla trained readout probes on layer-22 chunk activations to decode operation type and operands without prompt access. On a filtered DeepMind Mathematics Dataset interpolation split, the route produced exact-answer rates of 100% for gcd (across 1,233 targets), 99.2% for division with remainder, and 98.0% for lcm - gains of 50.2, 81.0, and 96.8 percentage points respectively over the frozen model's unassisted baseline. The result was preregistered on June 2, 2026, with thresholds and operand bounds fixed in advance. A separate hard-negative audit of 10,200 adversarial non-trigger examples produced zero false fires.

The original "JIT replacement" goal - writing the computed correct answer back into the model's residual stream so it would continue generation naturally - did not hold up. Residual writes showed no accuracy advantage over simpler logit or token correction and disturbed surrounding generation more. Videla frames this sharply: "a readable variable is not necessarily a writable register." Demonstrating that arithmetic structure is encoded in activations is a separate problem from reliably writing a corrected state back.

The route is Llama-specific. A Qwen operand-localization sweep returned zero recovery across all sampled positions. The supported operand range is integers 0-9999 with at most 12 generated answer tokens. The Hackaday write-up characterized the project as "sort-of worked" and "deemed a failure" - accurate for the JIT replacement half, but understating the activation-derived argument result, which held up under rigorous controls.

The core finding is that arithmetic prompts leave recoverable operation and operand structure in the residual stream, consistent with the Fourier-style helix encoding described by Kantamneni and Tegmark (2025). Practical value is currently limited by model specificity: unlike text parsers, activation routes require per-model localization. Building model-agnostic operand localizers is the meaningful next step for anyone looking to extend this approach.