GridRule began as a test of latent controls and compositional subgoals. The current work asks a harder question: can an agent gather the right evidence, avoid an irreversible but tempting commitment, and design an experiment that identifies which earlier decision actually changed the outcome?
All three are deterministic, procedural, and text-native. GridRule and FrontierGrid are spatial BFS environments; ContractForge is a compact exact-action contract laboratory. Each targets a different failure boundary.
Can a small model discover latent rules, compose subgoals, and avoid an unrecoverable trap?
Can an agent delay commitment until it has combined all decision-relevant evidence?
{alpha_key, proof_b}Can a decomposition propose the cheapest legal experiment that distinguishes a causal rule from a shortcut?
OR choice, AND prerequisites, SEQ order, and optional GUARDToggle evidence regimes, commit to items and portals, inspect the hidden research labels, and fork the same decision boundary.
GridRule's decoy shares a real door color. Under the irreversible rule, opening it spends the color required later; under the reversible control, the same map remains recoverable. The ON/OFF comparison changes that rule while holding the layouts fixed.
Pass@8 means at least one success in eight sampled rollouts; it is not first-attempt accuracy.
| Policy / condition | Irreversible ON | Reversible OFF | Evidence |
|---|---|---|---|
| Base | 0.367 | 1.000 | held-out |
| Safe trap demos only | 0.734 | 0.900 | 2 train seeds |
| Safe trap + reversible demos | 0.800 | 0.984 | 2 train × 2 eval seeds |
What worked: constrain collection to safe trajectories, keep successful trajectories with a hard filter, select short demonstrations, clone the corrected actions, and mix in reversible examples to preserve navigation.
A July audit found deterministic train/evaluation seed overlap in an earlier cure family. The values above use disjoint evaluation seeds and replace the older absolute cure numbers. The matched causal ON/OFF effect survived.
In reversible tasks, coordinate subgoals trained on single-gate primitives transferred to unseen compositions. Across five small pilot arms, every subgoal arm exceeded the plain baseline on unseen stages; mean pass@1 was 0.440–0.465 versus 0.264. That supports transferable decomposition, but not the broader claim that execution is always the bottleneck.
Reversible depth mainly exposes execution. Hazards expose routing. Crafting exposes order. Irreversibility exposes lookahead before commitment. The environment's useful output is not only success—it is which faculty broke.
The original critic caught only 11 of 106 actual decoy-commit boundaries: 0.104 recall. Hard-negative mining raised boundary recall to 1.000 while overall accuracy stayed 0.971.
Environment-verified typed subgoal reward reached mean final win 0.500 versus 0.525 for outcome reward. Dense distance reward accumulated about 13× faster while wins stayed nearly flat.
When controls were shown, stationary and fast-drift conditions both remained high (0.933 and 0.967). When controls were hidden, both collapsed to zero in the tested probe.
Current interpretation: self-reward is competence-gated. It can stabilize a policy that already proposes meaningful, verifiable subgoals; it did not reliably create that competence from a weak policy or exceed the seed model's ceiling.
FrontierGrid fixes navigation in a compact 7×15 world and makes the evidence structure harder. A claim can make a nearby proof look useful; only a separate counterexample reveals that the shortcut cannot return. Picking the proof or a contaminant is permanent, and the final gate requires an exact clean inventory.
This local probe varies mirrored geometry and controls. It establishes the intended causal trap, not broad model capability.
| Condition | Evidence used | Commitment | Outcome |
|---|---|---|---|
| False-proof route | tempting claim only | invalid proof + bad portal | failed in 12 actions |
| Route + checklist | claim + counterexample | valid required items | 0.80 progress at action cap |
Diagnostic, not a benchmark score: these are single-model, single-seed traces. The checklist corrected proof selection but did not solve within 20 actions. That isolates a remaining execution-efficiency failure.
A budget-20 test-time-evolution runner also repaired one unsafe candidate and won in 20 final actions after 44 total actions. The base trace already won in 18, so this validates the runner—not superiority of the method.
ContractForge removes spatial navigation and makes the causal structure native to the episode. An agent selects one of two sigils, takes an independent ledger, performs ETCH and BIND in the required order, optionally audits, and commits. Authenticity is determined by a latent code–witness relation; “pick the NEAR sigil” is a deliberately perfect shortcut during adaptation.
ORWhich sigil?authentic +1 · forged −1ANDSigil + ledger(−1, −1, −1, +1)SEQWhich order?same actions · +1 vs −1GUARDAudit first?benefit depends on shift riskOn all 500 aligned adaptation contracts, both “choose NEAR” and the code–witness rule win with a perfect efficiency score (RHAE 1.0). Passive four-policy search spends 672 actions, breaks the tie toward the simpler surface rule, and then scores 0% when the authentic sigil moves to FAR. TTRL-like self-imitation spends 1,280 actions on successful aligned traces and also remains at 0% held-out success.
held-out proxy-flip success after passive search or successful-trace imitation
certificate yield at 10 actions per pair; causal edit advantage +2
The matched twin preserves the witness, sigil codes, authentic identity, finish, and operation order while swapping only NEAR/FAR. One pair is enough for exact symbolic prompt or graph memory to repair held-out behavior. A two-feature logistic proxy needs six minibatches—48 certificates and 480 actions—to cross from the shortcut to the causal feature and reach 100%.
| GUARD policy | Aligned RHAE | Proxy-flipped RHAE | Interpretation |
|---|---|---|---|
| No audit | 1.000 | 0.000 | fast when the shortcut holds |
| Audit + repair | 0.694 | 0.391 | worthwhile only above 0.4389 flip probability |
Replicated model-free mechanism test: N=500 at seed 0 and seed 777 produced the same summary. The executor and intervention request are scripted; the two-feature policy contains the causal feature by construction. This is not an LLM, PPO, TTRL, GRPO, or LoRA result.
The original FrontierGrid Credit-DAG probe represents a decomposition as typed relations: alternatives (OR), jointly necessary prerequisites (AND), order (SEQ), guards, options, and switches. It forks matched continuations from the same boundary and asks which edit changed downstream return. ContractForge now realizes OR, all four AND coalitions, SEQ, and GUARD through legal episode actions.
| Probe | Naive local signal | Matched causal result |
|---|---|---|
| Wrong proof / contaminant | achieved 100%; positive hindsight | OR advantage −0.667 each |
| Claim / counterexample | both progress +0.15 | OR advantages −1 vs +2; gap 3.0 |
| Exact-gate prerequisites | each alone has utility 0 | joint interaction 1; Shapley 0.5 each |
A second N=500 horizon sweep, independently repeated with another seed configuration, asked how long a continuation must run before its ordering is trustworthy. Hindsight never recovered the correct poisoned-milestone ranking. Strict outcome intervals reached full certification only at horizon 18 for evidence choices and 22 for milestones.
Scope: the FrontierGrid probes use deterministic scripted continuations with privileged item identities. ContractForge adds a two-feature pairwise update, but no LLM generates the graph, intervention, or policy behavior, and no language-model weights are updated.
Two training seeds and two disjoint evaluation seeds for the final pass@8 table. Strong within this environment; not yet a broad statistical claim.
Competitive with outcome reward, slower and more variable. Dense and blended variants did not improve the conclusion.
Useful failure anatomy, not a model ranking. Broader models, seeds, stages, and budgets remain to be run.
Mirrored procedural instances with a privileged scripted executor. The original exact-gate AND result is a controlled synthetic boundary microprobe.
Legal-action OR, AND, SEQ, and GUARD comparisons plus a matched surface swap. The executor, intervention, symbolic memory, and two-feature learner are privileged mechanism tests.
An LLM-proposed graph and field-preserving intervention, learned pair scheduler, stochastic executor, token-span credit, and LoRA update are next tests—not current results.
The branches commit environment code, probes, and synthesized reports. A complete portable bundle of the main branch’s raw run artifacts is not yet published.
The broad lesson is narrower than “more reward fixes reasoning.” In these environments, the useful signal often lives at a specific commitment boundary. Data collection, evaluation, and credit assignment all improve when they preserve that boundary instead of averaging it away.