43.6% less JSON context
Compared with original formatted JSON. Official structured accuracy stayed at 100% on 10 questions.
UsageTap Compress · measured prompts, modeled API spend
What compression saves—and what the answers retain
TL;DR
What we measured: input size, cost and answer quality.
14.2% lower modeled cost
Across the full cohort, after compression fees. These Terra API estimates assume uncached input and no free compression allowance remaining.
Prose quality was mixed
Official text-matching scores moved in both directions. A separate semantic review also found some incomplete answers. Check the actual answers for your task.
Luna's supplementary review judged the compressed HotpotQA answers correct, but flagged partial Qasper answers in both conditions and one compressed-input summary. One model assessor does not establish unchanged quality.
20 tasks / 19 source inputs · GPT-6.1 Terra, medium reasoning · Three repeats. JSON used deterministic compaction; prose used forced model-assisted mode. This is the full-compression benchmark, not a production-default estimate.
What if my JSON is already minified?
The 43.6% headline uses the original formatted JSON as its baseline. Across the 10 JSON questions: original 54,865 context tokens → minified 37,024 → Compress 30,941. Compress used 16.4% fewer context tokens than minified JSON. These are context-token comparisons; the cost ledger below also includes answer output and compression fees.
These highlights summarize the full cohort. The controls below explore other conditions, workloads and cost scenarios.
Benchmark setup and cost assumptions
OriginalSelected compressionCompression fee
Quality alongside savings
| Benchmark | Metric | Before score | After score | Change | Pairs / questions |
|---|
Observed means on the sampled questions, with each metric kept separate. Repeated calls are not independent documents. Both compression and generation variation can affect score changes; this cohort does not establish broad quality equivalence.
Supplementary semantic review
Luna reviewer scores by benchmark
Original and selected compression; 1 = correct, 0.5 = partly correct, 0 = incorrect. Uncertain pairs excluded.| Benchmark | Before | After | Change | Scored / eligible pairs | Uncertain pairs |
|---|
Supplementary model judgments from one assessor at low reasoning, not official scores or proof of semantic equivalence. Exact duplicate answers share one judgment, so repeated mapped judgments are not independent reviews. Reviewer rationales and evidence are available with each answer below.
Savings at other models’ API prices
Cost-only extrapolation from the same Terra prompt counts and observed output counts. No new model calls. These estimates do not measure Sol, Astra or Opus answer quality, latency, or tokenization; Claude’s actual tokenizer and responses can differ.
| Model | Input | Cached input | Output |
|---|
Projected before and after cost components
| Model | Before input | Before output | Before total | After input | After output | Compression | After total | Net saved | Saved % |
|---|
Context tokens
Proxy token counts; instructions and questions excluded.API-equivalent cost
Matched answer calls; after includes the modeled compression fee.Net savings by workload
After compression fees, under the current pricing scenario.Answer scores across repeats
Official task metric; points are individual calls. Compare within each task.Request-time estimate
Median Codex time plus recorded compression preparation time.Input cost and output cost
API prompt scenario, with output usage measured by Codex.Before and after: the cost ledger
| Input | Pairs | Before API | After API | Compression | After total | Net saved |
|---|
Pricing, cache and compression assumptions
Cache reads, ordinary input and writes are disjoint input categories. A 30-minute write costs 1.25× input; a 12-hour write costs 2.5×. The long-context rate adjustment is applied above 272k input tokens. Scenario controls alter estimates only; they never alter measured results. Changing an early prompt prefix can reduce cache reuse.
Exact token and cost components
| Input / stage | Prompt input | Output | Ordinary input $ | Cached reads $ | Cache writes $ | Output $ | Compression $ | Total $ |
|---|
Before = ordinary input + cached reads + cache writes + output
After = ordinary input + cached reads + cache writes + output + compression charge
Net saving = Before − After
See what compression removes
Original input with removed text struck through. The answer checks below show the measured quality for these tasks.
Original, with removals
Compression details
Answers and quality checks
The published benchmark score is retained exactly. Normalized exact match is supplementary. Review the answers alongside the references: equivalent wording, digits versus spelled-out numbers, and extra detail can affect lexical scores.
Lexical scores can penalize digits versus spelled-out numbers, such as “3” versus “three”; inspect the actual answers and references below. Net cost differences also include variation in output length between calls.
| Variant | Repeat | Status | Actual answer | Official score | Normalized exact match | Semantic review |
|---|
The cases are different tasks and use different metrics. Scores are never averaged across QA F1 and structured-data accuracy. Repeated calls share the same source input; they are not independent documents.
Experiment usage, provenance and limits
API dollar values are modeled estimates; answer generation ran through a Codex subscription. The primary cost comparison prices the application prompt and measured output, excluding Codex's agent instructions/tool envelope. The separate experiment diagnostic prices the complete CLI usage envelope. Output-token usage includes reasoning when reported; reasoning tokens are not added a second time. Prompt counts use a proxy tokenizer. Network and compression times vary with startup/cache conditions. Taxes, regional premiums, Batch/Flex discounts, Fast mode and unknown account-specific terms are excluded. The compression free allowance remaining is unknown; its toggle is a scenario.