Production case study · FocusedFit.ai
Early production results · August 2026
FocusedFit.ai added customer metering. A week later, representative calls reached ~71% input caching and a major context payload was ~73% smaller by size.
The accidental optimization dividend: customer-level usage visibility exposed a cache-breaking instruction, a silent data bug, and duplicated context that the team was not originally trying to find.
Disclosure: FocusedFit.ai is operated by UsageTap co-founder Chris Hefley and developed with founding customer Alvarez Search. This is production evidence from a product operated by the UsageTap team, not an independent customer study.
- Input tokens served from cache
- ~71%
- Major context payload
- 150 KB → 40 KB
- Further compression opportunity
- 3–5%
Representative production calls
After the application was corrected
Estimated, so Runtime Compression stayed off
The original project
Understand and control customer usage.
FocusedFit.ai builds AI-assisted recruiting software with Alvarez Search, processing web-scraped and structured data about recruiting work. It adopted UsageTap not to optimize prompts, but to establish customer-level metering, plans, limits, and a trustworthy usage record for billing support.
After more than three months on another gateway, measured OpenAI usage during development and early production indicated a growing monthly run rate of roughly $600 while only two production users were active.
On August 22, 2026, the team began moving production traffic through UsageTap to attribute AI activity to customers and features, enforce plans and limits, and create the record needed for downstream billing.
Prompt optimization was not the project.
It was an accidental dividend of finally being able to inspect what each production AI call was actually doing.
Discovery 1
Most prompts were missing input-token caching.
Production visibility exposed a clear pattern: most sampled calls received little or no meaningful input-token caching, and the expected savings indicator was usually absent.
Sampling the prompts revealed volatile date and time information inside the system instructions, disrupting the stable prefix that caching depends on. The team first moved it to the end, then removed it as unnecessary. The difference appeared immediately.
- 71.2%: 41,074 of 57,701 input tokens cached on one observed call.
- 70.8%: 41,074 of 58,030 input tokens cached on another observed call.
- ~$0.00739: prompt-caching savings shown on each example call.
- $108.68: estimated cache savings in the August 22–28 reporting view, which recorded 362.9 million cached input tokens.
“Approximately 71% of input tokens served from cache.”
Discovery 2
Compression analysis exposed an upstream application bug.
Runtime Compression entered shadow mode so reductions could be analyzed without changing production prompts. Most requests showed little useful opportunity, so the team inspected the underlying input.
A roughly 150 KB JSON-like block contained Truncated... markers. Source material had been cut off during scraping and database storage, before it ever reached the model. Normal application behavior had made the payload look legitimate.
Discovery 3
The best compression result was fixing the application.
Correcting the truncation exposed another issue: raw scraped content duplicated much of the structured JSON. FocusedFit.ai removed the duplication and replaced the roughly 150 KB context with approximately 40 KB of tightly structured JSON.
Before
~150 KB
Truncated source, raw content, and overlapping structured data.
After
~40 KB
Tightly structured JSON containing only what the task needed.
The payload was approximately 73% smaller by size before Runtime Compression was applied.
This is a payload-size comparison, not a claim of 73% fewer tokens, 73% lower AI cost, or a 73% reduction produced by UsageTap Runtime Compression.
3–5%
The remaining opportunity was not worth enabling.
After incomplete, unnecessary, and duplicated content was removed, UsageTap estimated only another 3% to 5% Runtime Compression opportunity. FocusedFit.ai chose not to enable it.
The compression experiment succeeded by showing that compression was not the best solution.
Before and after
What changed in approximately one week.
Scroll to compare before and after →
| Production signal | Before investigation | After |
|---|---|---|
| Customer usage | Needed reliable customer attribution | Metering, plans, and limits foundation established |
| Input caching | Little or no meaningful caching on most initial sampled calls | ~71% on representative calls |
| Major context payload | ~150 KB | ~40 KB |
| Source data | Scraped data silently truncated | Truncation identified and corrected |
| Duplicated context | Raw source plus structured representation | Structured representation only |
| Runtime Compression | Being evaluated in shadow mode | Only another ~3–5% estimated; not enabled |
These are early production results, not a normalized long-term savings study. The team has not yet isolated the financial effect of every individual change or established a new monthly AI-cost run rate.
What comes next
Measure the economics of a successful recruiting outcome.
The next phase will benchmark representative recruiting tasks while varying model and reasoning level independently, measuring cost per completed task, response time, output quality, and longer-term cache behavior.
The primary workload also moved from GPT-5 to GPT-5.6 Luna with high reasoning. Early experience is promising, but caching, context, and model selection changed close together, so the team is not claiming a measured model advantage until it runs a controlled benchmark.
The goal is not simply to minimize tokens. It is to understand the economics of delivering a successful recruiting outcome.
The takeaway
Metering production AI usage helped the team see where the application was wasting money in the first place.
The savings were not the original project. They were the accidental dividend of finally being able to see, measure, and understand each AI call.