The Outlier Lab

Fraud Maturity Model

A five-stage maturity rubric for transaction fraud programs across people, process, data, technology and governance, with concrete markers and a self-assessment you can run in an afternoon.

Core components

Five stages

StageNameOne-line description
1ReactiveFraud is handled when it hurts; losses are discovered via chargebacks, returns and complaints
2FoundationalBasic rules, a review queue, a loss ledger and named owners exist for the main money paths
3ManagedGoverned rules and models, QA, risk assessments and launch review; decisions are measured
4OptimizedExperimentation, counterfactual measurement, total-cost optimization, graph and ML at scale, fast adaptation
5AnticipatoryIntel-led and red-teamed; controls ship ahead of attacks; the program shapes product design and industry practice

Most organizations sit at different stages for different money paths, such as stage 3 for card payments and stage 1 for the new rewards program. Assess per money path, then roll up.

Dimensions

Seven dimensions, each scored 1–5:

  1. Governance and ownership
  2. People and operations
  3. Process and lifecycle
  4. Data and labels
  5. Detection technology
  6. Measurement
  7. Adaptation and intelligence

How to build it

The rubric

Dimension1: Reactive2: Foundational3: Managed4: Optimized5: Anticipatory
Governance and ownershipNo named owner for fraud loss; decisions ad hocNamed fraud lead; basic policy; owners for top money pathsFraud policy, risk appetite, decision rights, launch review mandatory; second-line oversight where required (see Three Lines of Defense)Appetite expressed as total cost of fraud; regular risk committee; rules and models governed by materialityFraud requirements built into product design standards; board-level fluency
People and operationsSupport or finance handles fraud part-timeDedicated analysts; SOPs for main queuesCapacity planning; QA with calibration; specialist roles (strategy, ops, analytics)Workforce management; skills matrix; career paths; outsourced ops under QA and access controlsOps findings systematically feed strategy; analysts contribute to intel and red-teaming
Process and lifecycleRules added after incidents, never removedRule changes logged; basic incident handlingStrategy lifecycle with backtest and shadow; incident severity and post-mortems; annual risk assessment (see Fraud Strategy Lifecycle)Emergency path with expiry; periodic rule and model reviews; tabletop exercisesContinuous risk assessment triggered by product and threat change
Data and labelsFraud identified from chargebacks only; no consistent labelsLoss ledger; basic fraud tagging on casesTaxonomy-based labels; label maturity windows defined; event logging across money paths (see Fraud Labeling and Feedback Loops)Entity resolution across accounts, devices and instruments; feature store; QA-audited labels (see Fraud Data Architecture)Shared or consortium data; near-real-time label feedback
Detection technologyProcessor or vendor defaults onlyRules engine on main money paths; deny lists; basic velocityReal-time scoring on money-in and money-out; case management; step-up and holds (see Real-Time Transaction Risk Scoring)ML models governed and monitored; graph analytics; orchestration across vendors (see Graph Analytics)Adaptive controls; agent-aware and synthetic-media-aware defenses; red-team tooling
MeasurementLosses known only in aggregate, after the factLoss by money path; basic approval and decline ratesKPIs per money path: loss bps, catch rate, FPR, review rate; friction metrics (see Fraud Metrics and KPIs)Holdouts and champion-challenger; dollar-weighted metrics; cost of good-customer friction quantified (see Experimentation and Champion-Challenger)Forecasting and scenario modeling of exposure for new products
Adaptation and intelligenceLearns about attacks from lossesPeers and vendors informally consultedIntel digest with owners; unexplained-loss reviews; anomaly sentinelsFormal sharing memberships; time-to-mitigate tracked; red-team before major launchesLeads industry sharing; horizon scanning informs strategy and roadmap (see Threat Intelligence and Horizon Scanning)

Transaction-fraud-specific markers

These markers separate programs that look mature on paper from ones that actually are:

  • Money-out coverage. Every way value leaves (withdrawal, payout, redemption, transfer, refund) has real-time scoring or a rule-based gate. (Stage 3+)
  • Reversibility mapped. For every money-in rail, the program knows the reversal window and limits value extraction within it. (Stage 3+)
  • Account-change linkage. Recent contact, device or payout-destination changes affect money-out decisions. (Stage 3+)
  • Promo economics reviewed. Every incentive is reviewed for single-actor profitability before launch. (Stage 3+)
  • Holdouts exist. Some share of traffic, however small, measures counterfactual fraud. (Stage 4+)
  • QA covers automated decisions as well as manual ones. (Stage 4+)
  • Scam vs unauthorized distinguished in labels and reporting. (Stage 3+; see Authorized Push Payment (APP) Scams)
  • First-party vs third-party distinguished in labels and reporting. (Stage 3+; see First-Party Fraud)
  • Time-to-mitigate for new attacks is measured and trending down. (Stage 4+)
  • Kill switches exist for all live promos and money-out paths, and have been tested. (Stage 3+)

Running a self-assessment

  1. Scope. List money paths, e.g., card funding, ACH funding, withdrawals, P2P send, reward redemption, refunds.
  2. Score. For each money path, score each dimension 1–5 with evidence: a document, dashboard or process artifact. No evidence means the lower score.
  3. Calibrate. Have two people score independently, then reconcile. This is the same principle as QA calibration (see Fraud QA Program).
  4. Weight by exposure. A stage-1 score on a money path that moves 2% of value matters less than a stage-2 score on one that moves 60%.
  5. Pick 2–3 moves. Choose the highest-exposure gaps where one stage of improvement is achievable in two quarters.
  6. Reassess every six months, or after a major incident or launch.

Artifacts and templates

Maturity assessment worksheet (copy)

Money pathDimensionScore (1–5)EvidenceGap to next stageOwnerTarget date
Governance
People and ops
Process
Data and labels
Detection tech
Measurement
Adaptation

Leadership summary outline

  1. Current maturity by money path (heat map)
  2. Exposure-weighted overall score
  3. Top 3 gaps and the risk they create
  4. Proposed moves, cost, and expected impact on loss, friction and speed
  5. What we're deliberately not doing yet, and why

Metrics

Maturity should show up in outcomes. Track these alongside the scores:

MetricExpected trend as maturity rises
Time to mitigate new attacksFalls
Share of losses from previously unknown MOsFalls, then plateaus
Good-customer friction (insult rate, decline rate on good customers)Falls at equal or lower loss
Share of launches reviewedRises to ~100%
Rules and models with owners and current reviewsRises to ~100%
QA critical error rateFalls
Label coverage and timelinessRises

Pitfalls

  • Scoring aspirations. Score only what exists and has evidence. Roadmap items don't count.
  • Averaging away the weak spot. Attackers find the weakest money path. Report the minimum alongside the average.
  • Maturity as the goal. Stage 5 everywhere is expensive and usually unnecessary. Match target maturity to exposure and regulatory expectations.
  • Tech over process. Buying an ML platform doesn't move you to stage 4 if labels are bad and nobody runs holdouts.
  • One-time exercise. A maturity assessment that isn't repeated stops being used to manage anything.

Sources and further reading