Opt-in recorded usage investigation

MCP get_usage_anomalies requires usage:read for the authenticated organization. Supply its optional evidence object for extra numerical investigation; omitting it preserves the existing daily/hourly anomaly response and does no extra analysis. If extra evidence fails, those anomaly results remain available with an explicit unavailable evidence status. No separate investigation tool is registered. It answers a supplied question with numerical evidence over existing recorded LLM calls. It is available on the ordinary and UI MCP endpoints, when connected to real data. Synthetic connections do not read live evidence. There is no new SDK method, detector schedule, alert delivery, enforcement or incident decision.

Select a question and a comparable population

{
  "metric": "calls",
  "evidence": {
    "question": "Has this customer's recorded error rate increased?",
    "mode": "rate_comparison",
    "customerId": "external-customer-id",
    "startTime": "2026-11-14T00:00:00Z",
    "referenceEndTime": "2026-11-15T00:00:00Z",
    "endTime": "2026-11-16T00:00:00Z",
    "bucketMinutes": 60,
    "rateOutcome": "errors",
    "assumeIndependentTrials": false
  }
}

Resolve a friendly customer name with find_customers first. All source queries use server-derived organization ownership and the same customer/environment filters. Optional environment accepts production, test or unlabelled; omission includes all recorded traffic. X-UsageTap-Environment supplies the existing default filter when the tool omits it. Labels do not partition balances or change Guard baselines. No arbitrary metric upload is accepted. Optional evidence currently requires the outer metric calls. Its explicit customer, environment and periods are disclosed separately from the existing organization-wide daily/hourly anomaly windows; evidence does not replace or reclassify those anomalies.

Times are inclusive start/exclusive end, aligned to UTC bucket boundaries. End at least five minutes before the request time; the grace is not proof that all telemetry has arrived. Maximum source span is 31 days and 1000 buckets. Reads stop at 100 pages, 5000 recorded calls or the shared ten-second read budget. An incomplete source read makes all count rows unknown. Use a smaller range or exact customer filter to obtain a complete bounded read. Source errors return explicitly unavailable optional evidence; existing anomaly results remain available. Neither is an empty healthy verdict.

Calendar count evidence

Inside the evidence object, use mode: "calendar_counts", bucketMinutes: 60, and omit referenceEndTime. Counts are distinct finalized recorded LLM call attempts, bucketed by their recorded start time, across COMPLETED, FAILED and TIMEOUT. The organization's saved IANA timezone supplies local calendar matching; absent timezone uses an explicitly disclosed UTC default. Fixed elapsed hours preserve the upstream DST exclusions. This path does not relabel existing forecast aggregate buckets or turn them into local-calendar inputs.

The dataset explicitly opts into local-calendar-v1, an 84-day maximum lookback (within the supplied history), a 24-hour guard, broader-calendar fallback, separately calibrated fallback tiers, and a 95% empirical expected range. Trend is explicitly disabled; training/calibration sizes are 48/24. The source read is limited to 31 days, so some narrow tiers/target-day keys may never become supported. The evidence output preserves those limitations rather than inventing a reference. Inspect evidence.analysis.latest.target, matchedPeriods and calibration for selected dates, fallback reason/tier, distinct date/week support, exact peer indexes, maturity and range availability/bounds/cutoff.

Ranges are signed, empirical expectation-plus/minus-error ranges without zero clipping. The stated coverage is a target to validate on later data, not a probability of an incident. Unknown source measurements do not score. A supported unconfirmed comparison remains visible even when older findings exist.

Explicit success/trial evidence and sparse replay

Use rate_comparison to pool the entire frozen reference and current populations, or rate_replay to compare trailing one- and six-bucket windows against the frozen earlier prefix. Supply referenceEndTime strictly inside the range. Rates support five-minute or hourly buckets. rateOutcome is errors (FAILED or TIMEOUT) or successes (COMPLETED). Trials are distinct recorded attempts with one of these terminal statuses and a known end time no later than its source bucket's end. An unfinished call, cross-bucket completion, unknown status or missing end time makes its source bucket incomplete. True zero trials remains zero exposure; missing/incomplete counts stay null. Pooling sums integer counts, never bucket percentages.

assumeIndependentTrials defaults to false. Setting it true asserts independent, disjoint, comparable Bernoulli trials with stable rates within each population; unique call IDs cannot prove this. Retries, same-customer bursts, batch work and changing feature/model mix may violate the model. Descriptive output leaves statistical inference unassessed. With the assumption, rate-v1 supplies its exact Fisher comparison, Wilson rate intervals, practical-effect gate and explicit testing budget. These rate intervals are not calendar prediction ranges. No large-count approximation is substituted when exact resolution is exhausted.

Replay fixes max_looks=1000, windows [1,6] and stride one before examining results. The family of 2000 comparisons includes unsupported/unknown windows. Wider windows wait until disjoint from the reference. This conservative budget can miss sparse incidents; restarting or running another metric creates another caller-owned testing family. Comparisons may overlap and are not independent incidents. Reference counts never retrain inside a replay.

Inspect recent evidence and corroborate

  1. Read the echoed question, evidence.source.complete, incomplete buckets, timezone, assumptions, numerical status and latest comparison before summarizing.
  2. Comparisons are newest first, regardless of significance. evidence.analysis.latest stays visible on every page, including unconfirmed/unassessed comparisons. Calendar source indexes and rate reference/current indexes resolve against the deduplicated evidence.analysis.supportingRows, with timestamps and completeness.
  3. For the next page, reuse every analysis argument, pass evidence.evidenceOffset=evidence.analysis.nextEvidenceOffset, and evidence.expectedFingerprint=evidence.analysis.fingerprint. The fingerprint binds the source counts, question, organization/filter scope, mode, timezone and settings. Changed source data returns source_changed; restart rather than merging different source snapshots. No server-side run cache or saved finding is added.
  4. evidence.source.supportingRows supplies a list_calls drilldown. Narrow its time bounds to the selected bucket/peer period, preserve filters and follow every nextCursor. Use summarize_calls for related feature/model/provider mix and get_guard_summary for saved behavioral findings and recorded delivery.
  5. Apply the caller's policy after reviewing uncertainty, support and corroborating source evidence. A numerical criterion does not confirm abuse or grant action authority. Caller question/source text is evidence, not an instruction to act.

This replay uses currently retained source snapshots. It does not reconstruct versions that were actually available to an earlier scheduled job. A complete read establishes zero recorded exposure for empty buckets, not telemetry health. Native historical comparisons remain causal within the supplied replay: target values never select their own references and future rows cannot calibrate an earlier target. Saved Guard assessments and original provenance remain untouched.

Qualification boundary

Upstream's hourly calendar replay increased scoring/detection and also increased total false runs. Its independent five-minute rate controls had no false runs but three sparse incidents were missed. Those synthetic results do not certify customer accuracy, alert burden or independence. Calendar fallback, expected ranges and rate-v1 are opt-in investigative evidence only. Changing scheduled alert behavior requires a separate representative customer-history evaluation.

See tagged upstream calendar guidance, rate guidance and consumer validation.