Stable Prefix
The stable prefix is TokenPilot's primary mechanism for improving cache hit rates in long-running agent sessions.
How Stable Prefix Works
TokenPilot separates context into two parts:
- Stable prefix: Everything that doesn't change between turns (system prompts, tool schemas, static instructions)
- Dynamic suffix: What changes each turn (new messages, tool outputs, variable content)
By keeping the stable prefix identical across turns, the model API can reuse the cached computation. Only the dynamic suffix is recomputed.
Stabilizer Target
The stabilizer target setting controls where dynamic content is attached:
| Target | Behavior | Cache Efficiency |
|---|---|---|
developer | Dynamic content goes in developer messages | Good |
user | Dynamic content goes in user messages | Best |
The user target (used in aggressive mode) puts dynamic content further from the model's expected structure, maximizing cache reuse. The developer target (used in conservative and normal modes) keeps dynamic content closer to where it logically belongs.
Tuning
bash
# Toggle stabilizer
lightrsi stabilizer on
lightrsi stabilizer off
# Change target
lightrsi stabilizer target developer
lightrsi stabilizer target userNext
- Context Reduction — trimming tool output
- Context Eviction — pruning old context
- Reports and Visuals — see the effect