Knowing what not to compress
100 long-text questions · 97 source inputs · 900 main answers · 48 regression follow-up answers
Context tokens saved
Bars use a 0–100% scale. Savings are weighted by original context tokens; group membership comes from preparation decisions, not answer quality.
Source-aware quality
| Benchmark | Questions | Before | After | Resolved pairs |
|---|
Terra-low screening with Sol-low escalation. Scores: 1 correct, 0.5 partly correct, 0 incorrect. Means use matched resolved pairs; unresolved pairs are unscored and can contain errors. Repeats are not independent documents.
Two accepted news inputs exposed factual regressions
A quantifier changed from one woman to both women, and a salary-payment cause was misattributed. The accepted-input comparison contains three corroborated declining call pairs and 20 unresolved pairs out of 129.
Deeper checks: compress less, or preserve the source
| Case | Setting | Decision | Tokens saved | Fully correct |
|---|
Fresh Terra-medium answers, three repeats per row; matched original-input controls were correct in all three repeats. These are two outcome-selected cases. Higher effort and Sol-high did not reliably fix the 0.50 errors. The complete report retains all settings, uncertainty, costs, source audits and original scores.