Machine Learning Models

What it is

A fraud model is a learned function from features to risk. Algorithm choice matters less than practitioners expect. Gradient-boosted trees remain the workhorse for tabular transaction data, with deep sequence and graph models in specific niches. Success depends on everything around the algorithm: what you call fraud, when you know it, what the model sees at decision time, how its world changes, and how humans use its output.

This note covers those problems from a control-design perspective. Governance (validation, monitoring, documentation) is in Model Risk Management. Label pipelines are in Fraud Labeling and Feedback Loops.

What it stops (and what it doesn't)

Stops well: high-volume, repeating patterns where many weak signals combine: device, behavior, velocity, history, network and amount (Transaction Pattern Anomalies, Velocity Signals, Behavioral Biometrics). Models cut friction by letting you hold or step-up only the riskiest few percent, instead of applying blanket rules.

Struggles with:

• Cold start: new products, new rails and new attacks with no labeled history. Use rules, unsupervised anomaly detection and transfer from related products until labels accumulate.
• Rare, high-severity events: a handful of large losses per year doesn't give enough positives to learn from. Pair the model with policy controls and expert rules.
• Scams: the customer, device and behavior all look normal. Scam models need different features (payee risk, session context, Scam Victim Indicators) and different labels.
• Feedback-dependent outcomes: the model changes which cases get reviewed and which get blocked, so it shapes its own future training data.

Design considerations

Label design

The label defines what the model learns, and a model trained on the wrong label will optimize efficiently for the wrong outcome.

Class imbalance

Fraud is rare, often well under 1% of events. Imbalance matters less for the algorithm than for evaluation and thresholds:

• Use precision-recall and dollar-weighted metrics. Accuracy or ROC AUC alone isn't enough.
• Evaluate at the operating point you'll actually use: the top-k alerts your review team can handle, or the decline rate you can tolerate. Dal Pozzolo et al. argue for precision within the top-k alerts investigators can review each day as the metric that matters operationally.
• Undersampling or class weighting can help training. Recalibrate probabilities afterward if scores feed expected-loss calculations.
• Weight by dollars where loss severity varies widely.

Delayed and biased labels

• Verification latency: chargebacks can take weeks to months, ACH unauthorized returns have their own windows, and claims and returns arrive later still. Your most recent data, the most relevant to current attacks, has the least mature labels. Dal Pozzolo et al. treat this explicitly: in their card-fraud setting, most labels arrive only after a delay of days, which is harmful precisely when patterns are drifting.
• Two label streams: fast but biased feedback from reviewed alerts, and slow but more representative delayed labels from chargebacks and claims. The same research found that training separate models on investigator feedback and on delayed labels, then combining them, beat pooling both into one training set.
• Selection bias: review outcomes exist only for cases the current model or rules flagged. Keep a random review stream for unbiased labels (Manual Review Queues).

Drift

Fraud models face two kinds of drift at once:

• Customer behavior drift: seasonality, product changes, marketing campaigns and macro events shift what normal looks like. Dal Pozzolo et al. attribute a December performance drop in their data to pre-holiday changes in cardholder behavior.
• Adversarial drift: fraudsters change tactics in response to your controls. This drift is targeted and fast.

Controls for drift:

• Monitor feature distributions (population stability), score distributions, alert rates and, once labels mature, precision and recall by segment.
• Use leading indicators that don't wait for labels: review hit rate on top alerts, customer complaint rates, early-reported fraud claims.
• Retrain on a cadence tuned to how fast your environment moves. The cited research found models fed the most recent supervised samples outperformed static ones. Weigh frequent retraining against validation burden (Model Risk Management).
• Keep rules ready for fast response while models catch up (Rules Engine).

Adversarial adaptation

• Probing: adversaries test small transactions, compare outcomes and infer boundaries. Don't return graded decline reasons. Randomize some friction. Watch for test-like patterns (Card Testing).
• Feature degradation: any feature an adversary can observe or influence can lose value over time. Prefer features that are costly to manipulate (entity history, network structure, settled-funds behavior) over ones that are cheap (user agent, declared fields). Monitor feature importance shifts as an early warning.
• Label poisoning: first-party actors may submit false fraud claims or disputes that become training labels. Validate label sources.
• Feedback exploitation: if the model learns that a pattern is "safe" because it was never reviewed, adversaries benefit. Exploration traffic and random review counter this.

Explainability for reviewers

Reviewers make better, faster decisions when they see why a case scored high:

• Show the top contributing features in reviewer-friendly language ("new device + new payout destination within 1 hour") instead of raw feature names.
• Show the account timeline, linked entities and similar past cases alongside the score.
• Don't show the raw score alone, because reviewers anchor on it. Some teams hide the score and show reasons to reduce automation bias. Test both.
• Feed reviewer disagreements back to data science as a signal of model error or missing features (Fraud QA Program).

Build, buy or both

Vendor models bring consortium data and cold-start coverage. In-house models bring your specific labels, features and decision points. Many programs use a vendor score as a feature in an in-house model. Evaluate vendor claims on your data, with matured labels, at your operating point (Vendor Selection and Evaluation).

Implementation path

Customer friction and trade-offs

Models are the main way to reduce friction. Instead of making everyone pass a check, you challenge only the top-risk slice. The trade-off moves to threshold setting: each point on the curve trades fraud caught against good customers challenged. Set thresholds by segment and dollar value, and re-set them when the score distribution drifts. Watch for disparate impact on customer groups via proxy features (Friction and Customer Experience Tradeoffs).

Measuring effectiveness

• Precision and recall at the operating threshold, dollar-weighted, on matured labels.
• Top-k alert precision for review-driven models.
• Lift over the previous model or rules-only baseline, measured with champion-challenger or holdouts.
• Drift metrics: feature and score stability, alert rate stability, performance by segment over time.
• Time from new attack to model adaptation, and how much of the gap rules covered.
• Reviewer agreement rate with model reasons.

Common failure modes

• Evaluating on immature labels, so recent fraud looks like good traffic.
• Training only on reviewed cases (selection bias) or only on approved traffic (survivorship bias).
• Feature leakage: features computed with information not available at decision time.
• No holdouts, so the counterfactual is unknown and the model's impact can't be measured.
• Retraining that silently absorbs a successful attack as "normal."
• Scores shown to reviewers without reasons, which drives automation bias.
• One model for every fraud type, blurring first-party, third-party and scam patterns.
• Vendor models deployed without validation on your own data.

Regulatory and policy notes

• Model risk management: SR 26-2, the revised interagency guidance from the Fed, OCC and FDIC (April 17, 2026), is the current framework; it superseded SR 11-7 (2011) and SR 21-8 (BSA/AML models). It keeps the core expectations of validation, ongoing monitoring, outcomes analysis and effective challenge, including for vendor models, is risk-based, and is described as most relevant to banking organizations over $30B in total assets. Material fraud models at banks are generally in scope. See Model Risk Management.
• Explainability and adverse action: if model outputs feed credit decisions, adverse-action reason requirements may apply. Fraud-only declines are typically treated differently, but check with counsel.
• Fairness and privacy: features using location, device, language or behavioral biometrics raise fairness and privacy questions (Privacy and Data Use in Fraud Prevention).
• GenAI governance: FINRA's 2026 oversight report covers testing, logging, human-in-the-loop review and guardrails for GenAI and agent use. These apply if you use LLMs in fraud workflows.

Mechanisms addressed

Account Takeover (ATO) · Card-Not-Present Fraud · ACH Funding Fraud · Check and Mobile Deposit Fraud · Receipt Fraud (Cashback) · Money Mules · Authorized Push Payment (APP) Scams · Synthetic Identity Fraud · Multi-Accounting · Bust-Out Fraud · Refund and Return Fraud · Payout Redirection

Related controls

Rules Engine · Real-Time Transaction Risk Scoring · Graph Analytics · Manual Review Queues · Consortium and Data Sharing · Transaction Monitoring · Controls Index

Sources and further reading

• Credit Card Fraud Detection: a Realistic Modeling and a Novel Learning Strategy (Dal Pozzolo, Boracchi, Caelen, Alippi, Bontempi): verification latency, alert-feedback selection bias, top-k alert precision, drift; 75M+ transactions
• Credit Card Fraud Detection and Concept-Drift Adaptation with Delayed Supervised Information (Dal Pozzolo et al.): separating feedback and delayed-label learners; seasonal drift
• Federal Reserve SR 26-2: Revised Guidance on Model Risk Management (April 17, 2026): current interagency guidance; supersedes SR 11-7 and SR 21-8; most relevant above $30B in total assets
• Federal Reserve SR 11-7: Guidance on Model Risk Management (2011): the predecessor framework (validation, ongoing monitoring, outcomes analysis, effective challenge)
• FINRA 2026 Annual Regulatory Oversight Report: GenAI: testing, logging, human-in-the-loop and agent guardrails
• FINRA 2026 Annual Regulatory Oversight Report: AML, Fraud and Sanctions: GenAI-driven identity fabrication that degrades verification features