How UsageTap anomaly detection works

Summary

UsageTap learns what is normal for each metric from its own recent history, including trend and recurring calendar patterns. A spike or drop is detected only when the latest finalized value is unusually far from that expected baseline in three ways at once: percentage change, statistical score, and absolute size.

We also apply safeguards for incomplete data, missing buckets, low-volume noise, repeated alerts, and recovery. This layered approach is designed to catch meaningful changes without treating every fluctuation as an incident.

An anomaly is strong evidence that usage changed unexpectedly. It is not, by itself, proof of a particular cause. Use the observed value, expected value, direction, severity, and surrounding periods to investigate what changed.

The one-minute explanation

For every completed hour, day, week, or month, UsageTap:

1. Predicts the value using only earlier, finalized observations.
2. Compares the observed value with that prediction.
3. Measures the difference relative to both normal historical variation and the expected value.
4. Requires the difference to clear statistical, percentage, and minimum-impact gates.
5. Classifies a qualifying change as a warning or critical spike or drop.
6. Groups related signals into incidents and controls when notifications open, repeat, escalate, and resolve.

This is why a value can look visually different without triggering: being different is not enough. The difference must be large compared with the baseline, large compared with ordinary noise, and large enough to matter.

What causes a trigger?

A period creates an anomaly signal only when all of these conditions are true:

• The data is ready. The bucket is finalized, present, and backed by enough earlier training data.
• The absolute impact is meaningful. Tiny numerical changes are ignored, even when their percentage looks large.
• The percentage change is large enough. Spikes and drops have separate thresholds because their practical meaning is different.
• The statistical score is large enough. The difference must also be unusual relative to the metric's recent forecast errors and its natural observation noise.
• Low-volume protection is satisfied. When expected usage is small, a larger percentage change is required to avoid alerts caused by a handful of calls or tokens.

At the balanced defaults, a warning generally requires a robust score of at least 2.5 plus a 75% spike or 50% drop. Critical severity generally requires a score of at least 4 plus a 150% spike or 80% drop. Dollar drops use more conservative 75% and 90% percentage gates. Low-volume protection can raise these percentage gates further.

These are joint gates, not alternatives. For example, an 80% increase does not trigger if it is consistent with the series' normal variability or if the absolute difference is negligible.

Signals, confirmed incidents, and alerts

A signal is one completed period that passes every detector gate. The chart's All signals view shows this point-level evidence.

A confirmed incident reduces one-period noise:

• A critical signal is confirmed immediately.
• A warning normally needs a second related, same-direction warning within the recent cadence window.
• Related signals are grouped so a sustained condition reads as one incident rather than a row of unrelated alerts.

Notification state adds further operational controls:

• The same bucket is evaluated idempotently, so reruns do not count twice.
• A warning incident opens after two anomalous windows; a critical incident opens immediately.
• A warning that becomes critical sends an escalation.
• Reminders are suppressed for 24 hours after a warning notification and 6 hours after a critical notification.
• An incident resolves only after two healthy windows.
• Email and Slack delivery are tracked separately so a failed channel can be retried without resending a successful one.

The chart and notification system use the same core detector and evidence fields. Exploratory chart sensitivity can change which signals are displayed. Chart grouping is a presentation of the selected history; the persisted notification incident is the authoritative alert state.

How to interpret what you see

• Expected is the one-step-ahead estimate for that historical period, calculated without using the observed value being judged.
• Observed is the finalized measured value.
• Deviation is the signed percentage difference from expected. Near a zero baseline, UsageTap reports the absolute difference instead of an infinite percentage.
• Score measures the residual in units of robust historical error. It is similar to a z-score, but uses a robust scale and a metric-specific noise floor.
• Warning means the change cleared the warning gates.
• Critical means the same signal also cleared the stricter critical gates.

An incident is a prompt to investigate deployment changes, traffic mix, customer behavior, retries, failures, model routing, or provider spend. A drop can be as important as a spike: it may indicate an outage, instrumentation failure, or lost traffic.

Data-quality safeguards

The detector distinguishes four data states:

• Observed: a finalized, nonzero measurement.
• Finalized zero: a real zero after the bucket has closed. This can correctly trigger a drop after a healthy baseline.
• Partial: the current incomplete period. It is displayed but excluded from normal detection.
• Missing: a bucket that is not safely finalized or is unavailable. It is not treated as zero.

Usage buckets receive a finalization grace period before a zero is considered real. Time-series continuity is validated, including daylight-saving-length daily periods. If the series has unexpected gaps or duplicates, historical points are withheld from anomaly eligibility rather than scored against a questionable baseline.

Provider spend detection also checks source freshness. Daily provider cost incidents use finalized provider reports; hourly cost is explicitly identified as a UsageTap estimate and is reconciled when authoritative daily data arrives.

Sensitivity controls

Low-volume sensitivity changes how large a percentage move must be when expected call or token usage is small. It does not change the forecast, statistical-score thresholds, incident confirmation, cooldowns, recovery rules, or spend anomaly thresholds.

The sensitivity slider in the public cost analyzer and Spend Intelligence chart is exploratory. It changes the chart's percentage thresholds so you can inspect more conservative or more sensitive interpretations. It does not change saved notification settings.

Why this approach is trustworthy

Several design choices make the detector resistant to common anomaly-detection failures:

• No target leakage: the point being evaluated never contributes to its own expected value.
• Robust estimates: medians, median pairwise slopes, and median absolute deviation reduce the influence of earlier outliers.
• Trend and seasonality: normal growth and recurring hourly, weekday, weekly, or annual patterns are modeled instead of mistaken for anomalies.
• Multiple independent gates: percentage, statistical, and absolute-impact tests must agree.
• Metric-aware noise: counts, tokens, and currency use different lower bounds for ordinary variation.
• Explicit data state: partial and missing values are not silently treated as completed zeros.
• Incident controls: confirmation, cooldown, escalation, recovery, idempotency, and per-channel delivery reduce alert noise after detection.
• Tested edge cases: automated tests cover spikes, drops, low-volume suppression, near-zero baselines, finalized zeros, missing and partial periods, target leakage, time-series continuity, incident confirmation, escalation, cooldowns, recovery, and channel retries.
• Versioned evidence: persisted signals record the detector version used to produce them.

No statistical detector can promise that every anomaly is harmful or that every meaningful business change will be detected. UsageTap therefore presents the evidence behind each result and uses conservative operational confirmation. Trust the signal as a reliable indication that the observed metric departed materially from its learned pattern, then use product and business context to determine why.

Statistical details

1. Rolling one-step-ahead baseline

Let y_t be the observed value for period t. UsageTap estimates E_t, the expected value for that period, from a rolling window containing only finalized values before t.

Default maximum training windows are eight weeks for hourly and daily data, 52 weeks for weekly data, and 36 months for monthly data. Baseline eligibility starts after at least 24 hourly, 14 daily, 8 weekly, or 12 monthly prior observations. The detector also requires at least seven prior residuals before scoring a point.

The robust slope is the median of sampled pairwise slopes over recent lags:

slope = median((y_j - y_i) / (j - i))

Each historical value is projected to the target period with that slope, and the median projection becomes the trend estimate:

T_t = median(y_i + slope * (t - i))

The model then looks for observations in the same seasonal slot. Seasonal periods are 168 hours for hourly data, 7 days for daily data, 4 weeks for weekly data, and 12 months for monthly data. When at least three comparable seasonal observations exist, the expected value is:

E_t = max(0, 0.8 * median(seasonal projections) + 0.2 * T_t)

Until enough seasonal matches exist, E_t = max(0, T_t). The evaluated observation y_t is never included in this calculation.

2. Residual and robust scale

The forecast error, or residual, is:

r_t = y_t - E_t

Only residuals from earlier finalized, non-missing periods are used to estimate historical variation. Their robust scale is the median absolute deviation (MAD), multiplied by the normal-consistency constant:

MAD_scale = 1.4826 * median(|r_i - median(r)|)

To avoid unrealistically large scores when residual history is perfectly flat, UsageTap applies a metric-specific observation-noise floor N(E_t):

counts:   N(E) = sqrt(max(1, E))
tokens:   N(E) = max(0.5, 0.05 * |E|)
currency: N(E) = max(0.01, 0.08 * |E|)
generic:  N(E) = max(0.01, 0.05 * |E|)

The scale and robust standardized residual are:

s_t = max(MAD_scale, N(E_t), machine epsilon)
score_t = r_t / s_t

The computed warning band is E_t ± 2.5 * s_t, with the lower bound clamped to zero. The score is intentionally described as a robust score rather than a classical Gaussian z-test: it is an interpretable standardized residual, but it does not claim a calibrated p-value.

3. Percentage and low-volume adjustment

For a nonzero expected value:

deviation_t = 100 * (y_t - E_t) / |E_t|

When E_t = 0, a positive observed value has infinite relative deviation and must still pass the statistical and absolute gates.

For calls and tokens, the percentage thresholds are multiplied by a smooth low-volume factor:

L(E) = 1 + (M - 1) * exp(-max(0, E) / S)

At the balanced defaults, M = 2.5 and S = 5. Therefore L(E) approaches 2.5 near zero and decays smoothly toward 1 as expected volume grows. This avoids a brittle cutoff between “low” and “normal” volume. Dollar detection sets M = 1, disabling this adjustment.

4. Exact decision rule

Define P_warning and P_critical as the direction-specific percentage thresholds after the low-volume multiplier. A signal is a warning when:

eligible data
AND max(y_t, E_t) >= minimum value
AND |y_t - E_t| >= minimum absolute deviation
AND |deviation_t| >= P_warning
AND |score_t| >= 2.5

It is critical when the warning rule passes and both stricter gates also pass:

|deviation_t| >= P_critical
AND |score_t| >= 4.0

Base percentage thresholds are:

Minimum absolute deviations are 3 calls, $0.50 for currency, and 0.5 for token-valued series. The live forecast path uses a general minimum-value gate of 1. The public billing analyzer permits currency values down to $0.01 before applying the $0.50 absolute-deviation gate.

Because all gates must pass, the detector is conservative in two complementary ways. High-variance series require a larger raw residual to clear the score gate, while low-baseline series require a larger relative move to clear the adjusted percentage gate.

5. Incident state machine

Signals are keyed by organization, provider, metric, scope, direction, cadence, and bucket. An ordinary warning begins in a pending state and opens after two anomalous windows among the three most recent evaluated windows. A critical signal opens immediately.

Open incidents preserve the highest severity and maximum absolute impact observed. A warning-to-critical transition creates an escalation notification. Reminder cooldowns are 24 hours for warnings and 6 hours for critical incidents. Two consecutive healthy windows resolve an open incident. Pending state expires after 48 hours; resolved records are retained for 14 days; delivery and recent signal records have bounded retention.

Email and Slack use independently claimed, idempotent delivery records. A delivery is eligible only when its channel state is pending or failed, which prevents a successful channel from being resent merely because another channel failed.

6. What the method does and does not establish

The model answers a narrow, useful question: Is this finalized observation materially inconsistent with the recent trend, recurring pattern, and ordinary forecast error for this metric?

It does not assign causality, forecast business impact, or produce a universal probability that something is “wrong.” Those conclusions require application context. Keeping that distinction explicit is part of making the result trustworthy.