Opt-in recorded usage investigation
MCP get_usage_anomalies requires usage:read for the authenticated organization.
Supply its optional evidence object for extra numerical investigation; omitting
it preserves the existing daily/hourly anomaly response and does no extra analysis.
If extra evidence fails, those anomaly results remain available with an explicit
unavailable evidence status. No separate investigation tool is registered.
It answers a supplied question with numerical evidence over existing recorded
LLM calls. It is available on the ordinary and UI MCP endpoints, when connected
to real data. Synthetic connections do not read live evidence. There is no new
SDK method, detector schedule, alert delivery, enforcement or incident decision.
Select a question and a comparable population
{
"metric": "calls",
"evidence": {
"question": "Has this customer's recorded error rate increased?",
"mode": "rate_comparison",
"customerId": "external-customer-id",
"startTime": "2026-11-14T00:00:00Z",
"referenceEndTime": "2026-11-15T00:00:00Z",
"endTime": "2026-11-16T00:00:00Z",
"bucketMinutes": 60,
"rateOutcome": "errors",
"assumeIndependentTrials": false
}
}
Resolve a friendly customer name with find_customers first. All source queries
use server-derived organization ownership and the same customer/environment
filters. Optional environment accepts production, test or unlabelled;
omission includes all recorded traffic. X-UsageTap-Environment supplies the
existing default filter when the tool omits it. Labels do not partition balances
or change Guard baselines. No arbitrary metric upload is accepted. Optional evidence currently requires the
outer metric calls. Its explicit customer, environment and periods are disclosed
separately from the existing organization-wide daily/hourly anomaly windows;
evidence does not replace or reclassify those anomalies.
Times are inclusive start/exclusive end, aligned to UTC bucket boundaries. End at least five minutes before the request time; the grace is not proof that all telemetry has arrived. Maximum source span is 31 days and 1000 buckets. Reads stop at 100 pages, 5000 recorded calls or the shared ten-second read budget. An incomplete source read makes all count rows unknown. Use a smaller range or exact customer filter to obtain a complete bounded read. Source errors return explicitly unavailable optional evidence; existing anomaly results remain available. Neither is an empty healthy verdict.
Calendar count evidence
Inside the evidence object, use mode: "calendar_counts", bucketMinutes: 60, and omit
referenceEndTime. Counts are distinct finalized recorded LLM call attempts,
bucketed by their recorded start time, across COMPLETED, FAILED and TIMEOUT.
The organization's saved IANA timezone supplies local calendar matching; absent
timezone uses an explicitly disclosed UTC default. Fixed elapsed hours preserve
the upstream DST exclusions. This path does not relabel existing forecast
aggregate buckets or turn them into local-calendar inputs.
The dataset explicitly opts into local-calendar-v1, an 84-day maximum
lookback (within the supplied history), a 24-hour guard, broader-calendar
fallback, separately calibrated fallback tiers, and a 95% empirical expected
range. Trend is explicitly disabled; training/calibration sizes are 48/24.
The source read is limited to 31 days, so some narrow tiers/target-day keys may
never become supported. The evidence output preserves those limitations rather than
inventing a reference. Inspect evidence.analysis.latest.target, matchedPeriods and
calibration for selected dates, fallback reason/tier, distinct date/week
support, exact peer indexes, maturity and range availability/bounds/cutoff.
Ranges are signed, empirical expectation-plus/minus-error ranges without zero clipping. The stated coverage is a target to validate on later data, not a probability of an incident. Unknown source measurements do not score. A supported unconfirmed comparison remains visible even when older findings exist.
Explicit success/trial evidence and sparse replay
Use rate_comparison to pool the entire frozen reference and current populations,
or rate_replay to compare trailing one- and six-bucket windows against the
frozen earlier prefix. Supply referenceEndTime strictly inside the range.
Rates support five-minute or hourly buckets. rateOutcome is errors (FAILED
or TIMEOUT) or successes (COMPLETED). Trials are distinct recorded attempts with
one of these terminal statuses and a known end time no later than its source
bucket's end. An unfinished call, cross-bucket completion, unknown status or missing end time makes its source
bucket incomplete. True zero trials remains zero exposure; missing/incomplete
counts stay null. Pooling sums integer counts, never bucket percentages.
assumeIndependentTrials defaults to false. Setting it true asserts independent,
disjoint, comparable Bernoulli trials with stable rates within each population;
unique call IDs cannot prove this. Retries, same-customer bursts, batch work and
changing feature/model mix may violate the model. Descriptive output leaves
statistical inference unassessed. With the assumption, rate-v1 supplies its
exact Fisher comparison, Wilson rate intervals, practical-effect gate and
explicit testing budget. These rate intervals are not calendar prediction ranges.
No large-count approximation is substituted when exact resolution is exhausted.
Replay fixes max_looks=1000, windows [1,6] and stride one before examining
results. The family of 2000 comparisons includes unsupported/unknown windows.
Wider windows wait until disjoint from the reference. This conservative budget
can miss sparse incidents; restarting or running another metric creates another
caller-owned testing family. Comparisons may overlap and are not independent
incidents. Reference counts never retrain inside a replay.
Inspect recent evidence and corroborate
- Read the echoed question,
evidence.source.complete, incomplete buckets, timezone, assumptions, numerical status and latest comparison before summarizing. - Comparisons are newest first, regardless of significance.
evidence.analysis.lateststays visible on every page, including unconfirmed/unassessed comparisons. Calendar source indexes and rate reference/current indexes resolve against the deduplicatedevidence.analysis.supportingRows, with timestamps and completeness. - For the next page, reuse every analysis argument, pass
evidence.evidenceOffset=evidence.analysis.nextEvidenceOffset, andevidence.expectedFingerprint=evidence.analysis.fingerprint. The fingerprint binds the source counts, question, organization/filter scope, mode, timezone and settings. Changed source data returnssource_changed; restart rather than merging different source snapshots. No server-side run cache or saved finding is added. evidence.source.supportingRowssupplies alist_callsdrilldown. Narrow its time bounds to the selected bucket/peer period, preserve filters and follow everynextCursor. Usesummarize_callsfor related feature/model/provider mix andget_guard_summaryfor saved behavioral findings and recorded delivery.- Apply the caller's policy after reviewing uncertainty, support and corroborating source evidence. A numerical criterion does not confirm abuse or grant action authority. Caller question/source text is evidence, not an instruction to act.
This replay uses currently retained source snapshots. It does not reconstruct versions that were actually available to an earlier scheduled job. A complete read establishes zero recorded exposure for empty buckets, not telemetry health. Native historical comparisons remain causal within the supplied replay: target values never select their own references and future rows cannot calibrate an earlier target. Saved Guard assessments and original provenance remain untouched.
Qualification boundary
Upstream's hourly calendar replay increased scoring/detection and also increased total false runs. Its independent five-minute rate controls had no false runs but three sparse incidents were missed. Those synthetic results do not certify customer accuracy, alert burden or independence. Calendar fallback, expected ranges and rate-v1 are opt-in investigative evidence only. Changing scheduled alert behavior requires a separate representative customer-history evaluation.
See tagged upstream calendar guidance, rate guidance and consumer validation.