An imperfect monitor can teach a model to hide its reward hacks. GRAFT changes where the learning goes: route flagged updates into a forget adapter, then remove it. We examine the architecture, optimizer, anchor training, and evidence from smaller-scale RL and a separate Qwen3-32B SFT experiment.
Read more: https://neuralintel.org
Primary paper: https://arxiv.org/abs/2610.17062
00:00 Can we route the learning?
02:47 Neural Signal Check
05:42 The blind-spot trap
08:43 Frozen base, two new branches
10:38 Why MLP adapters?
13:33 Start without disturbing the base
16:28 Unflagged does not mean clean
18:34 The optimizer can undo your scale
20:44 Read the actual update rule
23:01 What about missed hacks?
25:21 Three ways to game the checker
27:37 Training policy is not deployment policy
29:54 Six controlled toy environments
32:39 Mechanism 2 | inoculation
35:09 How weak can monitoring get?
37:22 Now change the evidence lane
39:59 Elicit the unwanted behavior
42:42 Propensity versus instruction following
44:36 What would you test next?
Source notes: the paper uses routed first moments and unrouted second moments; the 16-point gain is relative to the base model. Zero observed hacks on 99 tasks is not a universal guarantee. On-screen notes reconcile these points with the narration.
Where would you test gradient routing? Share your take below.