Fraud Strategy Lifecycle

Core components

How to build it

Stage 1: Hypothesis

Sources of hypotheses, roughly in order of how often they pay off:

1. Loss review. Confirmed fraud that slipped through. Which features did those cases share that approved good traffic didn't? (See Loss Taxonomy and Attribution.)
2. Analyst and QA observations. Ops sees patterns weeks before they appear in loss data. Give them a structured channel, such as a weekly "patterns seen" form (see Fraud QA Program).
3. Threat intelligence. New MOs reported by peers, consortiums, regulators or law enforcement (see Threat Intelligence and Horizon Scanning).
4. Launch review. Pre-emptive strategies for new products (see New Product and Promo Launch Review).
5. Anomaly sentinels. Unexplained shifts in volume, decline rate or mix (see Playbook - Novel Attack Detection Members).
6. Friction complaints. A strategy that blocks good customers is also a hypothesis: "this rule is wrong."

A good hypothesis statement: "Withdrawals to a newly added bank account within N days of a phone-number change, on accounts older than one year with no prior withdrawals, are disproportionately Account Takeover (ATO)-driven Payout Redirection. Expected impact: $X a month in losses, Y good customers affected."

Stage 2: Analysis and sizing

• Size the loss with matured labels. Include losses already caught by other controls; you need them to measure overlap.
• Size the good population: how many legitimate customers share the pattern. Most proposed strategies are rightly dropped at this step.
• Check overlap. If an existing rule or model already catches 80% of the target, the marginal value of a new rule is the remaining 20%.
• Consider the cheapest action. A step-up challenge, short hold or queue routing may capture most of the value of a decline at a fraction of the friction (see Friction and Customer Experience Tradeoffs).

Stage 3: Design and backtest

Backtest outputs to report every time:

Common traps:

• Look-ahead leakage. Using features that weren't available at decision time, such as a device linkage computed from later sessions. Rebuild features as they existed at decision time.
• Selection bias from existing controls. Fraud that existing rules blocked never produced labels. The backtest only sees fraud that got through. This is the core reason to keep holdouts (see Experimentation and Champion-Challenger).
• Overfitting to one attack. A rule tuned exactly to last month's ring will catch that ring, which may already have moved on. Generalize to the behavior.
• Ignoring adaptation. When you block one path, expect volume to shift to adjacent paths, and plan monitoring for them.

Stage 4: Shadow (silent) mode

Run the logic in production, logging what it would have done without acting.

• Run it long enough to cover weekly seasonality and enough volume for stable estimates (illustrative: 1–2 weeks for high-volume flows).
• Have ops or QA review a random sample of shadow hits blind. This gives an early precision estimate without waiting for labels.
• Compare to the backtest. Large deviations usually mean feature or data differences between offline and production; find them before launch.
• Check whether the new logic adds decision latency or times out (see Real-Time Decisioning Architecture).

Stage 5: Launch

• Staged rollout: a percentage of traffic, or a segment, before full rollout. Hard blocks deserve a smaller first step than review routing.
• Pre-agreed rollback triggers, for example: decline rate rises by more than X points, complaints spike, the queue exceeds SLA.
• Ops readiness: SOP updated, reason codes added, customer messaging ready, support briefed.
• Holdout: where volume and risk allow, keep a small holdout untreated (or treated with a lighter action) to measure true incremental impact over time.
• Change record: what, why, expected impact, owner, rollback plan.

The emergency path

During an active attack, waiting for a backtest means losing money. The emergency path lets you launch first and analyze afterward, under these constraints:

• Authority is limited to the incident commander or on-call strategist (see Fraud Incident Response)
• Prefer reversible actions: holds, step-ups, review routing, tighter limits. Avoid permanent account actions.
• Scope narrowly to the attack signature
• Set an expiry (illustrative: 7–14 days) after which the rule turns off unless it has been through stages 2–4
• Log it in the rule inventory immediately with an "emergency" flag
• Review good-customer impact within 24–48 hours

Most rule sprawl comes from emergency rules that were never backfilled or expired.

Stage 6: Monitor and tune

Review cadence depends on materiality. High-impact strategies are reviewed monthly, everything else quarterly, plus an annual review of the whole inventory (see Rule Governance and Change Management).

Stage 7: Retire

Retire when any of these is true:

• Incremental catch is below the threshold for N consecutive review periods
• Precision is below the threshold and can't be fixed by tuning
• A model or broader rule supersedes it (overlap above X%)
• The underlying product or flow no longer exists
• It is an emergency rule past expiry without backfill

Retire with a watch period. Turn it off, or switch it to shadow mode, and monitor the target population for a few weeks. Some rules have deterrent value that only shows up when they're removed.

Who does what

Artifacts and templates

Strategy proposal template

1. Name and ID. Mechanism tag, money path, rail
2. Problem. Loss or risk addressed, current size and trend
3. Hypothesis. The behavior believed to separate bad from good
4. Logic summary. Plain-language, plus a link to the implementation
5. Action. Decline, hold, step-up, review, limit, or flag-only
6. Backtest. Period, label maturity assumption, catch, precision, incremental catch, dollars, review volume, customer impact by segment
7. Shadow results. Duration, volume, sampled precision, deviations from backtest
8. Rollout plan. Stages, holdout, rollback triggers
9. Ops impact. SOP changes, queue capacity, customer messaging
10. Owner and review date
11. Approvals

Acceptance criteria (illustrative, calibrate to your data)

Metrics

Pitfalls

• Backtesting on immature labels is the most common error, and it always flatters the rule.
• Measuring rules in isolation makes ten rules that each catch the same ring look like ten good rules. Measure incremental contribution.
• Without any holdout, there's no counterfactual, so you can't tell whether losses fell because of the rule or because the attackers left.
• Treating shadow mode as optional lets offline/online feature mismatches, which are common, stay invisible until production.
• Emergency rules that never expire. Set expiry dates and enforce them.
• Ignoring displacement. Losses that move to an adjacent rail or product haven't been prevented.
• A one-way ratchet, where rules get added and never retired, so friction rises steadily and nobody can explain why approval rates fell.

Related

• Fraud Program Overview · Rule Governance and Change Management · Model Risk Management · Experimentation and Champion-Challenger
• Rules Engine · Machine Learning Models · Real-Time Transaction Risk Scoring
• Fraud Labeling and Feedback Loops · Fraud Metrics and KPIs · Loss Taxonomy and Attribution

Sources and further reading

• Federal Reserve SR 26-2, Revised Guidance on Model Risk Management (April 2026): risk-based development, testing and monitoring expectations that map onto the backtest, shadow and monitor stages
• Revised Guidance on Model Risk Management, attachment (SR 26-2): out-of-sample and out-of-time testing, outcomes analysis, ongoing monitoring, controls on early use
• Abrigo, What SR 26-2 means for financial institutions: practitioner view on parallel testing and the missed-activity vs false-positive trade-off for fraud and AML analytics
• COSO/ACFE Fraud Risk Management Guide, 2nd ed., Executive Summary (2023): data analytics as a point of focus across preventive and detective controls
• Federal Reserve FedPayments Improvement, FraudClassifier model: consistent fraud classification that makes hypotheses and loss sizing comparable