Fraud Metrics and KPIs
Formulas, denominators and dashboards for measuring a transaction fraud program: loss bps, catch rate, precision, false positives, friction, review load, loss maturity and leading indicators.
Core components
The metric stack
A transaction fraud program needs five families of metrics. Each answers a different question, and none of them is enough alone.
| Family | Question | Core metrics | Primary audience |
|---|---|---|---|
| Outcome | How much did we lose, and is it getting better? | Gross and net fraud loss, loss bps, loss by mechanism | Leadership, finance |
| Control effectiveness | Of the fraud that came at us, how much did we stop, and how cleanly? | Catch rate (recall), precision, value detection rate | Strategy, data science |
| Customer cost | How many good customers did we hurt to get there? | False positive rate, insult rate, approval rate, step-up abandonment | Strategy, product |
| Operational | Can the team keep up, and is it accurate? | Review rate, review hit rate, SLA, handle time, QA accuracy | Ops, QA |
| Leading / exposure | What's about to hit the loss line? | Attack rate, exposure at risk, early-warning signals, time to mitigate | Strategy, on-call |
Formula reference
Define every metric per money path (funding, purchase, payout/withdrawal, reward redemption, P2P send, trade) and per rail. A blended company-wide number hides which path is losing money.
| Metric | Formula | Notes |
|---|---|---|
| Gross fraud loss | Sum of value lost to fraud before recoveries | Include chargebacks lost, ACH/check returns after value left, reimbursed scam claims, clawback failures, written-off promo/reward value. Define per Loss Taxonomy and Attribution. |
| Net fraud loss | Gross loss − recoveries (representment wins, clawbacks, recalls, restitution) | Report both. Recoveries arrive late and lumpy. |
| Loss rate (bps) | Net fraud loss ÷ volume on that money path × 10,000 | The denominator must match the path: card purchase volume for card fraud, ACH funding volume for ACH funding fraud, payout volume for payout fraud. |
| Fraud rate (count) | Confirmed fraud transactions ÷ total transactions | Useful for reward/receipt flows where values are small and uniform. |
| Attack rate | (Fraud stopped + fraud missed) ÷ total attempts | Measures pressure, separately from performance. Needs labels on blocked attempts too. |
| Catch rate / recall (count) | Fraud transactions stopped ÷ (stopped + missed) | "Stopped" must be confirmed fraud; a decline alone doesn't count. |
| Value detection rate (dollar recall) | $ fraud stopped ÷ ($ stopped + $ missed) | Usually the number leadership actually cares about. |
| Precision | Confirmed fraud among actioned ÷ all actioned | "Actioned" = declined, held, reviewed and rejected, or stepped up and failed. Define which. |
| False positive rate (FPR) | Good transactions actioned ÷ all good transactions | A tiny FPR can still mean thousands of insulted customers at scale. |
| False positive ratio | Good actioned ÷ fraud actioned (e.g., "8:1") | Common in card issuing. It's 1/precision − 1, so don't report it next to precision as if they were independent. |
| Insult rate | Good customers declined, held or challenged ÷ good customers who transacted | Counted per customer: one customer declined three times is one insult. |
| Approval rate | Approved ÷ attempted | Split into auto-approved and approved-after-review. Merchant-side teams also track it net of issuer declines. |
| Step-up pass / abandon rate | Passed ÷ challenged; abandoned ÷ challenged | Abandonment is a friction cost, and sometimes a fraud deterrent. Use labels to tell which. |
| Review rate | Sent to manual review ÷ total decisions | Capacity planning input; see Alert Review Operations. |
| Review hit rate | Confirmed fraud ÷ reviewed | Low hit rate = burning analyst time; very high = could probably be automated. |
| Exposure at risk | Value that could be extracted before the funding is final | E.g., instant buying power granted against unsettled ACH. See Funds Availability Holds, Velocity and Exposure Limits. |
| Recovery rate | Recovered $ ÷ gross loss $ | By recovery route: representment, clawback, recall, return. |
| Time to detect / time to mitigate | First fraudulent transaction → first alert; first alert → effective control | The best leading indicator of program agility. |
Dollar-weighted vs count-weighted
The two can point in opposite directions.
Rules of thumb:
- Leadership and finance get dollar-weighted metrics, because loss is denominated in dollars.
- Customer experience metrics are customer-weighted (insult rate) or count-weighted (FPR).
- Model training and evaluation should report both. Optimizing count recall on low-value traffic can leave the expensive tail untouched.
- Network and scheme metrics use their own definitions. Visa's VAMP ratio counts fraud reports plus disputes against settled card-not-present transactions, by count. Nacha's unauthorized return rate is also count-based. Mirror their definitions exactly when tracking compliance (see Card Network Dispute Rules and Nacha Rules and ACH Risk).
Loss maturity and vintage curves
Fraud loss shows up late. Chargebacks, ACH returns, scam reports and reward reversals arrive days to months after the transaction. If you report "losses this month ÷ volume this month", the most recent months always look good. Teams have celebrated a "drop" that was only immature data.
Vintage (cohort) reporting fixes this. Group transactions by the period they happened in (the vintage), then track cumulative loss for each vintage by days since transaction.
| Concept | Definition | How to use it |
|---|---|---|
| Vintage loss curve | Cumulative loss bps for a cohort, plotted against days since transaction | Compare cohorts at the same age ("day-30 loss"), never at different ages |
| Maturity window | Age at which a cohort has realized ~95% of its ultimate loss | Set per rail and mechanism. See Fraud Labeling and Feedback Loops. |
| Completion factor | Historical share of ultimate loss realized by age t | Ultimate estimate = observed loss at age t ÷ completion factor at t |
| Ultimate loss estimate | Projected final loss for an immature cohort | Report with a range. Recalibrate completion factors quarterly. |
Typical late-arriving signals by rail:
- Card fraud disputes arrive within the network's dispute window, which for card-absent fraud is commonly cited as 120 days from processing. Confirm current windows in Card Network Dispute Rules.
- Most ACH debit returns come within two banking days. Consumer unauthorized returns (R05, R07, R10, R11) can use the extended return window of 60 calendar days from settlement; corporate unauthorized returns (R29) keep the two-banking-day window. See Nacha Rules and ACH Risk.
- Check and RDC returns can come quickly, or weeks later for altered or duplicate items. See Check and Mobile Deposit Fraud.
- Scams on push rails are often reported days to weeks later, sometimes only when the victim runs out of money. See Authorized Push Payment (APP) Scams.
- In rewards and cashback, merchant or affiliate reversals and returns can arrive 30–90+ days later. See Cashback, Coupons and Rewards.
Counterfactual measurement
Catch rate needs the fraud you stopped to be labeled. But a declined transaction never matures into a chargeback, so you never learn whether it was really fraud. Options, in order of rigor:
- Holdouts. Let a small random share of flagged traffic through, with exposure caps, and observe outcomes. This is the only clean read on precision for blocking rules. See Experimentation and Champion-Challenger.
- Review outcomes. Treat confirmed-by-review as fraud. This is biased by analyst judgment, so check it with Fraud QA Program sampling.
- Proxy outcomes. Did the customer retry with another instrument, complain, or churn? Did the blocked account show up in a ring? See Entity Link Analysis.
- Score-band extrapolation. Estimate fraud rates in blocked bands from adjacent approved bands. It's cheap but weak at the extremes.
How to build it
| Stage | What exists | Next step |
|---|---|---|
| Crawl | A monthly loss number from finance, maybe chargeback counts | Build one loss ledger by money path and mechanism, reconciled to the GL (see Loss Taxonomy and Attribution) |
| Walk | Loss bps per path; approval and review rates; basic catch rate on review outcomes | Add vintage curves and completion factors. Separate first-party from third-party, and scam from unauthorized. |
| Run | Dollar and count metrics per path and segment; holdouts; insult rate; exposure | Leading indicators with alerting; time-to-mitigate tracked per incident; metrics tied to rule and model inventories |
| Fly | Metrics drive budget and tolerance decisions; loss forecasts by vintage | Risk-adjusted unit economics per product (fraud bps + friction cost + ops cost per $ of volume) |
Ownership: fraud analytics owns definitions and pipelines. Finance owns reconciliation to the general ledger. Strategy owners are accountable for their path's numbers. QA owns accuracy metrics for manual decisions. In practice, a versioned metric dictionary with named owners is one of the most underrated artifacts in a fraud program.
Artifacts and templates
Metric dictionary entry
- Name and one-line definition
- Formula, with numerator and denominator filters spelled out
- Weighting: count, dollar or customer
- Money path, rail and segment scope
- Label source and maturity rule (e.g., "transactions at least 90 days old")
- Source tables and owner
- Known biases (e.g., "blocked attempts are labeled only via review")
- Change log
Example dashboard outlines
Executive (monthly)
| Tile | Content |
|---|---|
| Net fraud loss | $ and bps by money path, mature vintages vs estimated ultimate for recent ones |
| Loss by mechanism | Top 5 mechanisms, trend, share first-party vs third-party vs scam |
| Customer cost | Approval rate, insult rate, step-up abandonment |
| Program cost | Ops cost per decision, vendor spend, cost per $ of loss prevented |
| Exposure | Outstanding unsettled exposure and limits utilization |
| Incidents | SEV count, time to mitigate, loss per incident |
Strategy (weekly, per money path)
| Tile | Content |
|---|---|
| Vintage curves | Last 12 cohorts, loss bps by age |
| Rule and model performance | Precision, recall and volume per rule/model, flagging any that haven't fired or are drifting |
| Segment heatmap | Loss and FPR by tenure, channel, geography, device class, rail |
| Holdout read | Fraud rate in holdout vs treated |
| Attack pressure | Attempt volume, decline reasons, new-pattern sentinels |
Operations (daily)
| Tile | Content |
|---|---|
| Queue health | Inflow, backlog, age, SLA breach count by queue |
| Decision quality | Review hit rate, QA accuracy, overturn rate on appeals |
| Regulatory clocks | Reg E claims approaching day 10 / day 45 or 90; SAR referrals approaching day 30 (see Regulation E and Unauthorized Transfers, BSA-AML and SAR Filing) |
| Staffing | Handle time, utilization, coverage vs forecast |
Model health (weekly)
| Tile | Content |
|---|---|
| Score distribution and PSI | Drift vs training baseline |
| Feature freshness and null rates | From the feature store; see Fraud Data Architecture |
| Calibration | Predicted vs observed fraud rate by score band, on matured labels |
| Latency | p50/p95/p99 decision latency; see Real-Time Decisioning Architecture |
Metrics
Metrics about the metrics program itself:
- Reconciliation gap: fraud loss ledger vs GL fraud loss accounts. Target: explained to within a small tolerance each month.
- Attribution coverage: share of loss dollars with a mechanism label (see Loss Taxonomy and Attribution).
- Label coverage: share of actioned transactions with a resolved label (see Fraud Labeling and Feedback Loops).
- Definition drift: number of metric definitions changed per quarter without a change log entry. Target: zero.
- Forecast error: estimated ultimate loss vs realized, per vintage.
Pitfalls
- Immature denominators. Comparing this month's loss bps to last year's mature number. Always compare vintages at the same age.
- "Catch rate" without missed fraud. If you only count what you caught, recall is 100% by construction.
- Treating declines as fraud. Declined ≠ fraud. Without holdouts or review, precision on blocking rules is unknown.
- One blended loss rate. Mixing payout losses with card disputes and promo abuse hides which money path is bleeding.
- Ignoring friction. A rule that cuts loss by 2 bps while dropping approval by 1% may be destroying value. Price it (see Friction and Customer Experience Tradeoffs).
- Counting scam reimbursements as "fraud loss" without a flag. Scam and unauthorized losses have different liability rules, controls and trends. Keep them separable.
- Optimizing to network thresholds only. Staying under a monitoring ratio is the minimum requirement and shouldn't be the program's goal.
- Goodhart's law in ops. If analysts are scored on handle time, they'll approve faster. Pair speed metrics with QA accuracy.
Related
- Loss Taxonomy and Attribution · Fraud Labeling and Feedback Loops · Friction and Customer Experience Tradeoffs · Exposure Window Modeling
- Experimentation and Champion-Challenger · Rule Governance and Change Management · Model Risk Management
- Alert Review Operations · Fraud QA Program · Manual Review Queues
- Real-Time Transaction Risk Scoring · Machine Learning Models · Rules Engine
- Fraud Program Overview · Fraud Maturity Model · Program Index
Sources and further reading
- Nacha, Unauthorized Return Rate guidance (Mar 2021): 0.5% threshold, unauthorized return codes, and the two 60-day calculation methods
- Nacha, ACH Network Risk and Enforcement Topics (2015): unauthorized (0.5%), administrative (3.0%) and overall (15.0%) return rate levels
- Solidgate, Visa monitoring programs: VAMP ratio construction (fraud + disputes ÷ CNP sales) and thresholds
- Federal Reserve, FraudClassifier model: authorized vs unauthorized classification that makes metrics comparable
- Federal Reserve SR 26-2 attachment, Revised Guidance on Model Risk Management (Apr 2026): outcomes analysis and ongoing monitoring as validation components
- COSO/ACFE Fraud Risk Management Guide, 2nd ed., Executive Summary (2023): data analytics as a point of focus across fraud risk principles
- FTC, reported fraud losses of $12.5B in 2024 (Mar 2025): context on why scam losses need their own line