The Outlier Lab

Alert Review Operations

How to run fraud alert review: queue design, prioritization, SLAs tied to money movement, capacity and staffing math, workforce management, decision tooling and outsourced review.

Core components

Queue design

Design queues around decision type and the money clock rather than around which rule fired:

Queue design questionGuidance
What decision does the analyst make?One primary decision per queue (release/hold, approve/decline, close/keep). Mixed decisions cause errors.
What is the money doing while the alert waits?Pre-movement (held: customer waiting, fraud can't progress) vs post-movement (value already gone: recovery only). Pre-movement queues get tight SLAs.
What skills are needed?Separate queues for specialized work: brokerage ACATS, receipt image review, scam intervention calls, seller investigations.
What evidence does the analyst need?Defines the review UI and SOP (see Case Management Systems).
What's the default if the SLA expires?Explicit, per queue: auto-release, auto-decline, extend the hold, escalate. Don't leave it implicit.

Typical queue families:

  • Real-time holds (pre-movement): withdrawals, payouts, high-value transfers, instant funding releases
  • Post-transaction review: card funding, purchases, receipts and offers, refunds and claims
  • Account-level review: new account risk, linked-account clusters, account changes
  • Claims and disputes: Reg E claims, INR/SNAD, chargeback representment (see Regulation E and Unauthorized Transfers)
  • Investigations: rings, mules, insiders (see Investigation Standards)
  • Customer-contact queues: outbound verification, scam intervention calls (see Playbook - Scam Intervention Members)

Prioritization

Within a queue, prioritize by expected value at risk × urgency:

  • Dollar exposure
  • Model score or rule severity
  • Time remaining before the money moves or the hold expires
  • Customer impact (a held paycheck vs a held promo reward)
  • Regulatory clock (Reg E deadlines, SAR timelines)

A simple priority score combining these beats first-in-first-out, which lets low-value alerts delay high-value ones.

SLAs

Queue typeIllustrative SLARationale
Real-time money-out holdMinutes to a few hoursCustomer is waiting; fraud is time-sensitive
Instant funding releaseHoursBalance availability expectations
Post-transaction review24–72 hoursRecovery window; label quality
Receipt and offer review24–72 hoursCustomer expectation; abuse velocity
Reg E claimsRegulatory: generally 10 business days to investigate, or provisional credit with an extended windowRegulation
InvestigationsDays to weeks, case-dependentComplexity

Measure SLAs at percentiles (p50, p90, p99). Averages hide the tail, and the tail is where complaints and losses live.

How to build it

Capacity math

Required FTE = (alerts per day × average handle time in minutes)
             / (productive minutes per analyst per day)
             × (1 + shrinkage buffer)
  • Productive minutes: paid time minus breaks, meetings, training, QA feedback and calibration (often 70–80% of paid time)
  • Shrinkage: absence, attrition, ramp-up of new hires (often 15–30%)
  • Intraday variation: peak hours need more coverage than average volume suggests

Workforce management

  • Forecast alert volume by hour and day, with seasonality, promo calendars and launch calendars (see New Product and Promo Launch Review)
  • Surge plans: cross-trained analysts, overtime triggers, temporary threshold changes agreed with strategy, and an incident escalation path (see Fraud Incident Response)
  • Real-time queue monitoring: age of oldest item, SLA breach risk, inflow vs throughput
  • Skills matrix: which analysts can work which queues; maintain coverage for specialized queues

Decision tooling

The review UI determines decision quality as much as training does:

  • All evidence for the decision on one screen; minimize tab-switching
  • Decision-time snapshot preserved for QA (see Fraud QA Program)
  • Linked entities visible: devices, instruments, payees, accounts (see Entity Link Analysis)
  • Model score with top reasons, so analysts evaluate rather than obey (see Model Risk Management)
  • Structured disposition and reason codes, required; free text alone isn't enough (see Fraud Labeling and Feedback Loops)
  • One-click actions for common outcomes, with confirmation on irreversible ones
  • SOP linked contextually
  • Access logging and least-privilege data visibility (see Insider Access Controls)
  • AI assistance (summaries, suggested actions) clearly labeled, with source data one click away

Feedback to strategy

Ops is the richest source of signal about rule quality. Feed it back through:

Outsourced and offshore review

  • Contracts specify SLAs, QA access, data handling, access controls and audit rights
  • Same SOPs, rubric and QA sampling as in-house; report error rates separately
  • Least-privilege access, monitoring for unusual case access patterns, and controls on actions that release value (see Insider Fraud, Insider Access Controls)
  • In-house ownership of SOPs, calibration and difficult queues

Artifacts and templates

Queue definition template

FieldExample
Queue nameWithdrawal hold – new destination
DecisionRelease / extend hold / decline + restrict
Money statePre-movement (held)
Inflow sourcesRule IDs, model threshold
SLAp90 under 2 hours
SLA-expiry defaultEscalate to senior analyst; do not auto-release
Priority logicAmount × score × time to customer expectation
Required skillsATO and payout SOPs
SOPLink
QA samplingPer Fraud QA Program
OwnerOps lead

Daily ops dashboard

  • Inflow, throughput and backlog by queue
  • Oldest item age and SLA breach forecast
  • Handle time p50/p90
  • Disposition mix by queue and by rule
  • Staffing vs plan
  • Escalations and overrides

Weekly ops-strategy sync agenda

  1. Queue health and SLA performance
  2. Noisy rules (low confirmed-bad rate, high volume)
  3. New patterns seen by analysts
  4. Upcoming launches and expected volume
  5. Tooling pain points
  6. QA themes affecting decisions

Metrics

MetricWhy
SLA attainment (p50/p90/p99) by queueCustomer and fraud impact
Backlog and oldest-item ageEarly warning
Average and p90 handle timeCapacity and tooling
Alerts per analyst-hourProductivity (never alone; pair with QA accuracy)
Confirmed-bad rate by queue and ruleAlert quality
QA accuracy and critical error rateDecision quality (see Fraud QA Program)
Overturn rate (decisions later reversed)Quality, and customer impact
Analyst attrition and time to proficiencySustainability

Pitfalls

  • Productivity targets without quality targets. Analysts optimize for clearing alerts and stop making real decisions.
  • FIFO queues. High-value alerts wait behind low-value ones.
  • Implicit SLA-expiry behavior. An alert that auto-releases after its SLA expires is a hole attackers can find.
  • Rule teams that never see ops cost. Alert volume grows until ops collapses.
  • One queue for everything. Context-switching between mechanisms increases errors.
  • Tooling that hides linked entities. Analysts can't spot rings they can't see.
  • Outsourcing without oversight. Cost savings vanish with one insider incident.

Sources and further reading