ML for phishing, malware, fraud

Why Walmart asks this: The JD says "Explore/apply AI/ML for anomaly detection, phishing detection, malware analysis, fraud prevention," and your resume says Siege AI red-team + detection rules — so the loop probes whether you can turn ML output into defensible security decisions. Walmart is the highest-volume retail target in the world: ~$650–680B revenue (FY24–25), 10,000+ stores, a marketplace, gift cards, pickup/delivery, and hundreds of millions of accounts and transactions. A 0.01% false-positive rate on billions of events still mangles thousands of real customers. Walmart Global Tech has published on exactly this: deep learning for retail fraud (customer "insult" events, chargebacks, sub-1% fraud, label delay), published/presented work on sequence analytics (RNN/transformer over transaction sequences for account takeover, stolen-financial, and velocity abuse) and on mining high-precision, area-specific abuse rules (patent-level detail exists but don't cite numbers you can't back — say "published/presented"), and the "Redemption" gift-card program, which flags scam-loaded cards at transaction time (balance check seconds/minutes after load — especially from overseas — then rapid reload→redemption) and froze ~$4M ($3.96M, the Apr-2016–Jul-2017 cohort, forfeiture 22-USS-000172) that DOJ/Secret Service returned to victims via remission from 2022. Honest caveat if pushed: it captures only a fraction of gift-card fraud, and per ProPublica funds frozen after 2017 haven't yet been returned.

1. The One Rule: Anomalous ≠ Malicious

A model outputs novelty, never intent. The security decision needs three more inputs: context (is the account/asset sensitive? change freeze?), policy (does it violate a rule?) and impact (max loss if wrong). So the pipeline ends in risk rank + human review + step-up, not an auto-block. Saying "the model blocks anything over 0.9" fails the interview. Say: "the model produces a calibrated score; score + business context maps to actions by blast radius — step-up OTP for medium, review queue for high, auto-block only for high-certainty, low-blast-radius cases."

WHAT this looks like at Walmart scale: billions of auth, transaction, and telemetry events per day mean even a well-tuned detector produces thousands of outliers daily, and almost all of them are benign — holiday surges, shift changes, resellers, travel, and deploys. The detection problem is therefore not finding oddity but ranking it so a finite analyst team sees the riskiest few dozen first. WHY ML helps and where it loses: rules catch known bad patterns cheaply but miss novel combinations, while ML finds cross-feature oddity without a signature — yet the attacker adapts by living inside normal ranges, timing abuse to busy periods, and reusing legitimate tools so the behavior looks ordinary. That is why the architecture keeps both: cheap rules funnel out the clearly benign and clearly malicious, and ML scores only the gray middle. Concrete example: a cashier account logs in at 3AM from a new ASN and moves 50x normal refund volume — anomalous on three features, but benign if a Black Friday overnight shift and inventory return wave explain it, malicious if the account is dormant, the device is new, and refunds go to a different instrument. The interview answer is always the same shape: score, then context, then tiered action.

2. Architecture — Anomaly Detection Pipeline

Full checklist flow: sources → features → IsolationForest / Autoencoder / LSTM → scored alerts + SHAP → feedback.

Sources            Features                 Models                Output
─────────          ─────────                ──────                ──────
Auth logs          per-entity baselines     1. IsolationForest    risk score 0–100
DNS/proxy          z / robust-z / MAD       2. Autoencoder        + top SHAP reasons
EDR process        rates, ratios, entropies 3. LSTM next-event    + tier (log/review/
Cloud audit        graph features                 prediction        step-up/block)
Netflow            time-of-day, geo, ASN    ensemble (rank-avg)   + analyst feedback
Transactions       rarity, first-seen
Gift cards                                 SHAP per alert → analyst UI

2.1 Sources → Features

Start from a question, not a dataset ("unusual data egress" → proxy bytes, destination rarity, ASN reputation, user's 30-day p95). Feature families:

• Behavioral baselines: per-entity EWMA mean and variance of logins per day, events per hour, bytes out, and distinct destinations or processes. Each entity is compared to its own history first, so a service account that is always noisy is not flagged for being noisy, only for changing.
• Rarity: inverse document frequency of destination domains, first-seen entities, ASN rarity, and parent-process rarity. Rare does not mean bad, but first-contact with a newly registered domain or an unusual parent for PowerShell is the feature that lets the model catch what allowlists have never seen.
• Temporal: hour-of-day deviation, day-of-week, holiday and Black-Friday flags, and inter-arrival times between actions. The same 3AM login scores differently on a weekday versus a holiday deploy window, which is how seasonal baselines suppress the Black Friday false-positive storm.
• Relational/graph: account-to-device-to-payment-to-address-to-IP degrees, shared-device counts, and communities of accounts sharing a BIN or pickup point. These features catch rings that look normal in isolation but share infrastructure, and they force the attacker to control a whole neighborhood rather than one account.
• Sequence: last-N action types such as session steps, process lineage, and transaction amounts embedded for the LSTM. Order carries the signal here — password reset then email change then high-value checkout is far riskier than the same three events in reverse.
• Context: asset criticality, privileged flags, change-freeze windows, and threat-intel matches. Context does not detect anomaly but decides action: the same score on a domain admin during a freeze outranks the same score on a test account.

2.2 Model choice — know the trade-off

Ensemble: rank-average the three scores (0.4·IF_rank + 0.3·AE_rank + 0.3·LSTM_rank, weights tuned on analyst feedback). Different models fail on different attacks. Do not AND them blindly — an AND kills recall.

WHAT the problem looks like: the question is rarely "is this event odd in isolation" but "is this entity drifting from its own pattern in a way that matches attacker behavior." WHY ML beats rules here: no rule can enumerate every odd combination of time, volume, rarity, and sequence, and static thresholds break across entities with different variances. Antagonist view: the attacker adapts by staying inside per-metric thresholds, splitting activity across accounts and time, reusing legitimate parent processes, and timing abuse to noisy periods when baselines are wide. That is why the ensemble matters — IF catches point outliers, the autoencoder catches broken correlations, and the LSTM catches wrong order — and why absolute counts alone are insufficient without rarity, delta, and graph features. Interview sketch: if asked why not just one model, answer that login anomalies are sequential so LSTM leads, the autoencoder covers the static context, and IF is the cheap backstop, with ranks averaged so one blind spot does not veto the others.

2.3 Scoring → Alerts → SHAP → Feedback

# Pseudo-pipeline (illustrative, sklearn-style — not runnable as-is)
from sklearn.ensemble import IsolationForest
import numpy as np, shap

def robust_z(x, hist, eps=1e-6):
    med = np.median(hist)
    sigma = max(1.4826 * np.median(np.abs(hist - med)), eps)   # MAD -> sigma, zero-variance floor
    if len(hist) < 20: return 0.0                              # min-sample gate
    return (x - med) / sigma

iforest = IsolationForest(n_estimators=300, contamination=0.01,
                          max_samples=256).fit(X_baseline)
raw   = -iforest.score_samples(X)                 # higher = more anomalous
score = calibrate(raw)                            # isotonic/quantile -> 0..100
risk  = 0.6*score + 0.4*business_context(entity)  # risk rank, not verdict
tier  = ("log" if risk<50 else "review" if risk<80 else
         "step_up" if risk<95 else "block")

# Reasons for the analyst: TreeSHAP is exact + fast for GBDT heads (LightGBM/XGB).
# For IsolationForest, shap.TreeExplainer explains mean path length, not the calibrated
# score (shap PR #784, version-fragile) — use feature ablation / per-feature robust-z
# reasons, or a surrogate GBDT trained on iforest scores.
top = explain_iforest(iforest, event)   # ablation or surrogate attributions
reasons = [f"{FEATURES[i]}={'up' if top[i]>0 else 'down'}"
           for i in np.argsort(-abs(top))[:5]]

# Feedback: analyst verdict = weak label with confidence, not ground truth
feedback.append({"event_id": eid, "label": verdict, "confidence": c})

Feedback rules: label only verified incidents as 1, because an analyst closing an alert without action is not proof the event was benign — it often means there was not enough evidence to decide. That distinction matters: training on "no action equals benign" quietly teaches the next model that missed attacks are normal. Keep a small random audit of non-alerted events to estimate missed detections, retrain on a weekly cadence with strictly time-based splits so future data never leaks into training, and monitor both score drift and feature drift with PSI and KS tests to catch upstream logging changes before they masquerade as attacks. Every retrain should compare precision at the fixed review capacity, not just AUC, so the team knows whether the new model actually helps the queue.

2.4 Thresholds and the zero-variance trap

z = (x − μ)/σ breaks when σ = 0 — a host that always logs exactly one event/day gets x/0 = ∞ on any blip, so every tiny change looks infinitely anomalous. Fixes:

Never binary-search a magic threshold: set it at analyst capacity (e.g. 60 reviews/day), tune to hit target precision on the top-K, re-tune monthly.

2.5 Alert journey — from score to analyst decision with numbers

A raw score never reaches an analyst directly. HOW it flows end to end: the entity-hour feature vector is scored by the ensemble, the raw score is calibrated to 0–100 on recent history, and business context adjusts it into a risk rank. The rank maps to a tier — log, review, step-up, or block — and only the review tier and above create queue entries with the top five reasons attached. Suppose the team can review 60 alerts per day and tunes the cut so top-60 precision is 30%: that yields about 18 true cases per day, with 42 false alarms the analysts must still clear. Each decision returns as a weak label with confidence, plus the time spent and whether extra context was needed, so the next tuning cycle can trade threshold against queue load rather than chasing a single metric. Interview sketch: if asked to walk through one alert, narrate sources to features to ensemble score to calibrated risk to tier to SHAP reasons to analyst verdict to retrain, and quote the capacity math to show the threshold is an operations choice, not a model constant.

3. Architecture — Phishing / BEC Detection

Three stacked layers; no single layer is sufficient, and weights must be explainable (CAPE-style modular detectors).

WHAT the problem looks like at retail scale: millions of inbound emails, marketplace messages, and SMS-adjacent lures target associates, suppliers, and customers daily, and a single credential or invoice-redirect success can cascade into account takeover or payout fraud. WHY ML beats rules here but also loses alone: blocklists and regexes catch known malicious domains and attachments cheaply, yet attackers rotate domains in hours and rewrite lures with LLMs, so lexical and intent models must generalize to unseen variants. Antagonist view: the phisher adapts by aging domains, warming reputation, hiding behind shorteners and open redirects, compromising a real mailbox so auth passes, and prompting an LLM to remove urgency and typos. HOW the architecture works: Layer 1 scores the URL and infrastructure without opening the message, Layer 2 scores intent and relationship context from headers and body, Layer 3 checks cryptographic auth alignment, and a final classifier stacks the layer scores into one risk with per-layer reasons preserved for the analyst.

3.1 Layer 1 — URL lexical + infrastructure

Concrete example — one phishing URL walked through: consider http://walmartsupport-login.top/secure/verify?session=48291 linked from a first-contact email. Lexical features fire on length and hyphen count, subdomain depth with a brand token, risky .top TLD, and high path entropy; brand-spoofing fires on small Levenshtein distance to walmart.com combined with registered domain mismatch; infrastructure fires on RDAP age under seven days, privacy-proxy registrant, and a cert issued hours ago from a free issuer; redirect features note a shortener hop that lands off-origin on a page with a credential form posting to a different host. No single feature proves phishing, but the conjunction of young domain plus brand-in-subdomain plus credential-post-off-origin is the pattern the stacked classifier is taught to weight heavily, while preserving each reason for review.

3.2 Layer 2 — NLP on the body (context, not just keywords)

• Intent: requests for credentials, payment re-routes, gift-card or PIN codes, MFA codes, payroll changes, or invoices with new bank details. Intent is the strongest body signal because it describes the payoff action, so a payment-change request from a first-contact sender outweighs any keyword count.
• Psychological signals: urgency ("within 24h"), authority (CEO, IRS, or support impersonation), secrecy ("don't discuss with anyone"), scarcity, and call-to-action density. Each alone is noisy since legitimate ops mail also sounds urgent, but the combination of urgency plus authority plus secrecy is rare in normal traffic and common in BEC.
• Relationship context: first contact, thread hijack where an old thread suddenly carries new bank details, reply-to differing from From domain, display-name brand mismatch, off-hours send, and recipient fan-out. Context is expensive for the attacker to fake because it requires prior history with the victim, which is why first-contact plus fan-out plus off-hours is treated as a gating pattern.
• Stylometry: 47-feature lexical-plus-syntactic-plus-punctuation models with XGBoost reach ~96% on AI-vs-human phishing (paper-reported, dataset-dependent — quote as illustration) — a supporting signal, never a sole verdict. Use it to add weight when intent and context already fire, and expect to recalibrate per LLM family since cross-model generalization is weak.

Weight signals, don't veto on one. Urgency alone fires on every ops email; first-contact + payment change + display-name mismatch + off-hours is the BEC pattern. Feed detector scores into a classifier — explainability comes for free (analyst sees which behaviors fired).

3.3 Layer 3 — Email authentication (SPF / DKIM / DMARC)

The two cases auth cannot catch: (1) compromised legitimate mailbox — mailed from inside, SPF/DKIM/DMARC all pass; only behavior/context detects it. (2) lookalike domain (walmart-support[.]com) that published its own SPF/DKIM and warmed up reputation — lexical + domain age catches it.

3.4 AI phishing has no typos

LLMs removed the old tell (bad grammar). Studies show 86%+ of synthesized phishing bypasses Gmail-style filters, and paraphrase attacks drop DeBERTa classifiers to F1 ≈ 0.38 (figures vary widely by study/dataset — the qualitative point is what survives cross-examination). Attackers prompt: "avoid urgency, avoid deadlines, sound professional." Therefore:

1. Drop keyword blacklists as primary signals: urgency and grammar lists are weak and drifting because an LLM can be prompted to sound calm and professional in one rewrite. Keep them only as low-weight features and never as vetoes, since every ops deadline would otherwise alert.
2. Detect expensive-to-fake context: sender-to-recipient relationship, thread history, auth alignment, and request-type risk (payment change outranks file share, which outranks meeting link). Forging a plausible multi-message history or a legitimate signing domain costs the attacker far more than rewording a lure, so these features survive paraphrase attacks that defeat text classifiers.
3. Model LLM style shifts but recalibrate per family: LLM phishing still shows measurable shifts such as urgency, CTA, authority, and time-pressure densities roughly 3x human phishing with lower lexical diversity — modelable as supporting features, but cross-model generalization is weak. Treat thresholds as per-family settings and re-tune when a new generator appears, rather than assuming one stylometry model covers all LLMs.

3.5 Response — out-of-band verification

Never let a classifier block a CEO request. Verify on a second independent channel (call the known number on file, not the one in the email), because the email channel itself is untrusted once display names and threads can be spoofed. Require dual approval for payment or vendor bank-detail changes so one compromised mailbox cannot move money alone, and quarantine high-risk mail with a visible banner plus a one-click "Report Phishing" loop that returns analyst labels to retraining. HOW it works end to end: the stacked risk score plus request-type risk selects the action — log for low, banner plus quarantine for medium, hold pending out-of-band confirmation for high — and the feedback loop tracks which quarantines were true lures to re-weight layers monthly. Interview sketch: if asked what happens on a high-risk invoice change, answer that the mail is held, the requester is called back on a known number, a second approver signs, and the verdict becomes a labeled example.

4. Architecture — Malware Analysis (Static + Dynamic + Adversarial)

4.1 Static features

4.2 Dynamic features — API call sequences

Static analysis dies to packing because the bytes on disk are not the code that runs; API sequences reflect true behavior once unpacked and are hard to falsify without changing what the malware does. WHAT the problem looks like: a sandbox detonates the sample and logs ordered calls such as CreateFileW, WriteFile, RegSetValue, and HttpSendRequest with arguments, and the task is to separate malicious chains from benign installer chains that use the same APIs. HOW the pipeline works: normalize API names with key parameters, build name-plus-parameter n-grams to keep argument context, link calls that touch the same file or registry key into resource graphs so related actions stay connected even when non-adjacent, and keep sequences process-grouped so multi-process injection is not split into meaningless fragments. Model the sequence with an LSTM or GRU or 1D-CNN plus a small graph encoder, then soft-vote with the static LightGBM head so packing defeats only one branch. Antagonist view: malware inserts benign API noise, delays execution, or splits work across processes to break naive n-grams, which is why parameter-aware features and resource graphs survive longer than name-only truncation. Interview sketch: if asked why dynamic helps, answer that static sees the packer and dynamic sees the payload, and the ensemble keeps both.

4.3 Adversarial ML — the FGSM formula

x_adv = x + ε · sign( ∇_x J(θ, x, y) )

• Symbols: x = original input, ε = budget, J = loss, θ = params. The gradient is taken w.r.t. the input (params frozen), linearizing the loss and moving each feature to maximize loss.
• Why malware is different: binaries are discrete and semantic — you can't flip arbitrary bytes without breaking the executable. Practical attacks are append-based: pad with bytes chosen along the embedding gradient then discretize back to nearest byte (FGM Append), or append benign byte sequences to offset malicious features. FGSM's ε-ball is L∞ by definition — meaningless on discrete executables; practical attacks constrain L0 / append length (bytes added) plus functionality preservation.
• Fixes: adversarial training on generated appends, explicit append-length and entropy-flatness features that flag large low-entropy overlays, ensembles so a transfer attack tuned on one model does not transfer cleanly, and black-box query limits that slow Oracle-style probing.

Adversarial-evasion pair to quote: original — a packed dropper scores 0.92 malicious on byte-entropy plus import features; evasion — the same dropper with 50KB of benign Windows help-text appended drops to 0.41 because the histogram and string stats shift toward benign without changing execution. The defense is to train on such appended variants, add overlay-size features the append cannot hide, and keep the dynamic sandbox path that still observes the payload behavior.

4.4 Drift

Malware mutates continuously as packers, loaders, and toolkits rotate; EMBER-style static features tolerate weeks of drift but degrade on novel families the training set never saw. HOW to respond: monitor detection per family and per packer rather than one global accuracy, so a drop on a single emerging family is visible before the aggregate moves. Retrain only on time-based splits with the test window strictly after training, hold out the newest weeks as a drift probe, and route low-confidence or high-novelty samples to the sandbox and analyst path so the static classifier is never the only line. When per-family recall falls or the sandbox disagrees with static scores more often, treat it as a drift incident: freeze the threshold, add the new family to training with verified labels, and re-tune on precision at the triage capacity. Interview sketch: if asked why performance fell after deployment, answer drift first, show per-family curves and time-split evaluation, and describe the sandbox fallback that buys time while retraining.

5. Architecture — Fraud Prevention (Walmart Scale)

WHAT the problem looks like at retail scale: hundreds of millions of accounts and transactions across stores, online, pickup, delivery, marketplace, and gift cards, with fraud below 1% but losses and customer harm concentrated in account takeover, stolen instruments, and scam-loaded gift cards. A model that is merely accurate still fails if it insults good customers at checkout or misses the short window where gift-card value can be frozen. WHY ML beats rules here and where it loses: velocity caps and blocklists stop the obvious abuse cheaply, but fraudsters rotate devices, instruments, and pickup points faster than rules can be rewritten, so sequence and graph models must learn the shape of abuse. Antagonist view: the fraudster probes as an Oracle, primes aggregates with normal behavior, and poisons retraining with evasions that look legitimate. HOW the pipeline works end to end: transaction and identity events feed sequence, graph, velocity, and context features into supervised and sequence models, the score combines with business impact into a risk tier, and the action is usually step-up or review rather than a hard decline, with verified chargebacks and audits closing the loop.

5.1 Retail fraud taxonomy

5.2 Features that matter

• Sequence/temporal: last-N transactions with amount, category, channel, and geo, plus inter-arrival time, spend versus 30-day baseline, ramp trend, and device risk score time-series (Walmart's sequence-analytics patent scores sequences then trends them). The trend matters more than one amount: a slow ramp of growing purchases followed by a high-resale checkout is the account-takeover shape.
• Graph: account-to-device-to-payment-to-address-to-phone-to-IP edges, shared-entity degree, distance to known-fraud nodes, and store risk rating. A single new device is weak evidence, but five accounts sharing that device with one hop to a blocked entity is a ring signal no per-transaction rule sees.
• Velocity/aggregates: transactions per hour, distinct cards per device per day, returns in 30 and 90 days, and refund ratio. Velocity catches the burst phase of card testing and scalping, while refund ratios catch return abuse that only appears over weeks.
• Identity change: password, email, or payment changes in the last 24 hours plus new device at checkout. Recent identity churn just before a high-value order is the classic takeover precursor, so these features gate the risk upward even when amounts look normal.
• Context: channel (store, online, pickup), high-resale categories such as electronics, gift cards, and tires, and promo eligibility. Context sets the loss and the action: the same score on a gift-card bulk purchase and a grocery run should not receive the same response.

5.3 Models

Delayed labels (chargebacks arrive weeks later) + <1% base rate + drift ⇒ time-aware splits, importance weighting by label delay, periodic retraining — and never train on auto-approved traffic as if verified-benign.

5.4 Adversarial fraud — black-box probing + behavior shaping

Fraudsters treat the detector as an Oracle:

1. Black-box probing to map the boundary: compromise many cards, run repeated transactions at increasing amounts until one blocks, and sacrifice cards to learn the surface. Evasion studies on real banking datasets report 60–100% evasion rates (dataset-dependent — order of magnitude, not a constant), which shows how quickly an Oracle can find the gaps.
2. Surrogate model for transferable evasions: train a local model on observed approve and decline responses, then craft evasions against it; transferability means the same examples often fool the real detector too. The attacker never needs the true weights, only enough query feedback to approximate the boundary.
3. Behavior shaping (priming) to poison aggregates: perform small, normal, on-pattern activity for days or weeks so aggregates like average spend, known-device status, and history length look legitimate — then commit the fraud. Models using absolute aggregates are highly vulnerable here, while delta and change features plus first-seen counters stay robust because priming cannot hide the sudden jump.
4. Poisoning via retraining on evasions: successfully evaded frauds get auto-labeled legitimate, shifting the boundary in the attacker's favor across cycles. The fix is procedural: restrict retraining to verified labels, hold out recent auto-approvals from benign training data, and monitor label-shift and boundary movement as poisoning signals.
5. Concept drift as a weapon: shift behavior gradually so an online baseline follows the drift, then strike once the new normal covers the attack. Defenses are delta features with capped history, periodic re-baselining on verified data, and drift alarms that freeze thresholds when the baseline moves too fast.

Defenses that work start with probe detection — flag one device fingerprint across many accounts, sequential small-then-large amounts, and systematic granularity that looks like boundary mapping rather than shopping. Prefer step-up authentication over binary blocks so uncertain cases pay friction instead of a decline, plant honeypot rules that alert when probed but never trigger on normal traffic, and practice explanation security by never exposing exact decline reasons or thresholds that teach the Oracle. Finally, lean on graph features that force the attacker to control a whole neighborhood of devices, addresses, and instruments rather than burning one account at a time, which raises the cost of every evasion attempt.

5.5 Cost of a false positive

A wrongly declined transaction is not just lost revenue: Walmart's own framing is customer "insult" rate — a good customer told their money is no good at checkout — which drives support contacts, churn, and lost lifetime value far beyond the single basket. HOW to reason about it: each threshold choice trades prevented fraud dollars against insulted-customer dollars plus analyst hours, so business value depends as much on false-positive reduction as on detection. At Walmart volume a 5–10% reduction in false positives or fraud moves profit materially because the denominator is billions of checkouts. Threshold example: with 1M events per day, 0.01% prevalence (100 frauds), TPR 99% and FPR 1%, the model yields 99 true catches against 9,999 false alarms for about 1% precision — operationally unusable without an upstream funnel and a review cap. That is why the team sets the cut at analyst capacity (for example 60 reviews per day), tunes for precision in the top-K, and prefers step-up friction over hard declines in the uncertain band. Interview sketch: if asked whether to favor precision or recall, answer precision first at retail scale, buy recall with step-up and review, and quote insult cost plus queue math.

5.6 Concept-drift response playbook for a fraud model

Fraud drift is expected, not exceptional, because instruments, devices, and scam stories rotate seasonally. HOW the team responds: monitor precision in the top-K, approval and step-up rates, feature PSI and KS, and per-segment recall on account takeover, stolen-financial, and gift-card patterns daily. A drift incident is declared when top-K precision falls, the score distribution shifts, or a new device or pickup pattern spikes without a business explanation. Response is to freeze the threshold so the queue does not flood, route the drifting segment to enhanced step-up and manual review, verify labels before any retrain, retrain on a time-based window with importance weighting for label delay, and only then re-tune the threshold against analyst capacity. Keep the old model as a shadow scorer for one cycle so a bad retrain can be rolled back without losing coverage.

6. Evaluation — Where Most Candidates Fail

6.1 Base rate: precision is the headline, not accuracy/ROC

Fraud is ~0.1–1% of transactions; intrusions are rarer. At 1:10,000, a "99% accurate" detector has a Bayesian detection rate near 1% (Axelsson's base-rate fallacy). ROC looks great while precision is awful on imbalanced data.

Precision = TP / (TP + FP)          # dominated by the benign denominator
PPV_natural = (TPR·π) / (TPR·π + FPR·(1−π)),   π = true prevalence

"Shrink the benign denominator" correctly means: don't evaluate on a downsampled benign set and quote inflated precision. Correct to natural prevalence (PPV_natural), or report PR-AUC, precision@K (K = analyst capacity), alerts/day, and cost per prevented dollar — always with raw TP/FP counts. Worked numbers: 1M events/day, π = 0.01% (100 fraud), TPR 0.99, FPR 0.01 → TP = 99, FP = 9,999 → precision ≈ 1%. Capacity math: at K = 60 reviews/day and 30% precision you catch 18/day — below ~10% precision the queue burns more analyst-hours than the fraud it catches. Operationally, shrink the denominator with an upstream funnel: cheap rules/intel allowlists remove the 95% clearly benign events before the expensive model, so the same precision yields far fewer false alarms.

6.2 Stack signals, then risk-rank

• Stack independent signals into one risk: auth verdict plus URL reputation plus NLP intent plus sender relationship feed a final classifier as one risk score, with each layer weighted by its measured precision on recent labeled data. Keep layers independent so explanations stay readable — the analyst should see which layers fired rather than one opaque number — and re-weight monthly as phishing infrastructure rotates.
• Risk-rank by score times impact: sort a tiered queue by score × business impact so a medium score on a payout change outranks a high score on a newsletter link. Allow auto-actions only in the extreme tier with reversible, low-blast-radius steps such as OTP, hold, or manual review, and never binary approve or deny from one model alone because the benign denominator guarantees false alarms.

6.3 Metrics table

7. Real Interview Q&A (20)

Q1. Design an anomaly detection pipeline end-to-end. Sources (auth/EDR/proxy/cloud/transactions) → per-entity features (baselines, rarity, graph, sequence) → ensemble of IF + autoencoder + LSTM (rank-average) → calibrated risk score + SHAP reasons → tiered action (log/review/step-up/block) → analyst feedback with time-based retraining. Stress funneling (rules first), precision targets, and context enrichment.

Q2. Isolation Forest — how does it work, why use it? Random splits on random features; anomalies isolate in fewer splits, so average path length E(h(x)) is short. Score s(x,n)=2^(−E(h(x))/c(n)) normalizes by tree path length. Unsupervised, high-dim, cheap — good first pass with sparse labels. Weakness: dense/clustered anomalies, correlated features.

Q3. Autoencoder vs LSTM vs Isolation Forest for login anomalies? Ensemble, but lean on LSTM — logins are sequential (time-of-day, prior actions, geo trajectory) and prediction error captures "what should happen next." Autoencoder for the static high-dim context (device/IP/ASN fingerprint); IF as a cheap backstop. Single-model answers are a red flag.

Q4. What features for gift-card fraud? Lead with the published Redemption flags: balance check seconds/minutes after load (especially from overseas), redemption shortly after from a different location, reload→redemption velocity. Secondary: amount vs customer baseline, first gift-card purchase, redemption device/IP ≠ purchaser, many cards redeemed from one IP, in-store high-value load while on a call, PIN-attempt failures. Redemption flags at transaction time and freezes the balance (~$3.96M of the 2016–17 cohort returned via DOJ remission) — but it catches a fraction of total fraud. Prevention beats recovery (gift cards are irreversible).

Q5. How do you detect phishing beyond the spam filter? URL lexical/infrastructure (domain age, homoglyph, redirect chain) + NLP body (intent, urgency/authority/secrecy, relationship context) + auth (SPF/DKIM/DMARC alignment). Weight, don't veto. Add first-contact/payment-change patterns and enforce out-of-band verification for high-risk requests.

Q6. Explain SPF, DKIM, DMARC and what each misses. See §3.3. Key line: DMARC aligns SPF or DKIM with visible From and sets policy; p=none still delivers, lookalikes pass their own auth, and a compromised mailbox passes everything — auth proves domain control, not intent.

Q7. AI phishing has perfect grammar — how do you detect it? Drop typos/grammar and urgency keywords as primary signals. Use expensive-to-fake context: sender↔recipient relationship, thread history, auth alignment, display-name vs domain mismatch, off-hours first contact, request-type risk (payment change > file share). Stylometry (~96% AI-vs-human) supports; recalibrate thresholds per LLM family because generalization is weak.

Q8. Why are urgency/authority/secrecy weak signals? They're persuasion tactics, not proof: urgency fires on every deadline, authority on every exec email. Each detector is noisy; CAPE-style modular scoring combines behaviors in a classifier, keeping verdicts explainable ("urgency + first-contact + payment change").

Q9. How do you analyze a malicious PE with ML? Static: byte histogram/entropy, byte n-grams (4–8) TF-IDF, PE header/imports hashed, parse warnings, strings, Authenticode → EMBER v3 2,568-dim → LightGBM. Dynamic: API name+parameter n-grams, resource graphs, process grouping → LSTM/CNN ensemble. Run both: static is fast/offline, dynamic survives packing.

Q10. What is FGSM? x_adv = x + ε·sign(∇_x J(θ,x,y)) — one-step white-box attack moving input in the loss-maximizing direction, bounded by ε; gradient w.r.t. input, params frozen. For malware, adapt to discrete bytes via embedding gradients + nearest-byte mapping (FGM Append), append bytes, bound by append length / L0 — FGSM is L∞ by definition, which is meaningless on binaries.

Q11. How can attackers evade malware ML? Append benign byte sequences or gradient-crafted padding, pack/obfuscate, modify non-functional regions, generate rewrites. Defenses: adversarial training, padding/entropy features, ensembles against transfer attacks, sandbox/behavioral fallback — no single classifier is the only layer.

Q12. Why does malware detection degrade over time? Concept drift: new families, packers, APIs. Classifiers lose recall on novel families (EMBER-style features are robust for weeks, not years). Monitor per-family performance, retrain on time-based splits, route low-confidence/novel samples to sandbox + analysts.

Q13. Fraudsters don't have your model — how do they evade it? Black-box Oracle behavior: probe with many compromised accounts to map the boundary, train a surrogate, craft transferable evasions. Behavior shaping: small normal transactions for days to prime aggregates, then the big fraud. Poisoning: evasions become "legit" labels at retrain time and shift the boundary.

Q14. How do you defend against probing and shaping? Detect the probe pattern itself (one device fingerprint across accounts, sequential escalation, systematic granularity), use delta/change features so priming doesn't help, graph features so the attacker must control a neighborhood, step-up instead of binary block, don't leak reasons/thresholds, verify labels before retraining, adversarial training on generated evasions.

Q15. Accuracy is 99.9% — is the model good? Meaningless without prevalence. At 0.1% fraud, always-predicting "no fraud" is 99.9% accurate and catches nothing; at 1:10,000, a 1% FPR swamps true positives (base-rate fallacy). Report precision at natural prevalence, PR-AUC, precision@K, alerts/day, cost per prevented dollar.

Q16. Precision or recall — which do you optimize? Precision: the benign denominator dominates, and every FP costs a customer (block → insult/lost sale) or an analyst hour. Buy recall with step-up/review rather than auto-blocks. Set the threshold at analyst capacity and tune to target precision on top-K.

Q17. Your z-score returned infinity. What happened? Zero variance — a constant metric. Fix with a variance floor (max(σ, ε)), MAD-based robust scaling, a min-observation gate, or a count-distribution tail probability. Reverse trap: high variance masks real anomalies (z≈0); robust z on log-scaled features helps.

Q18. The model flags Black Friday traffic as anomalous. Now what? Classic anomalous ≠ malicious: add calendar/holiday context, use same-weekday/hour baselines, suppress known seasonal windows, route volume spikes to capacity planning. Also verify a bot wave isn't hiding inside the spike.

Q19. How does SHAP help, and what are its limits? Per-alert feature attributions make alerts explainable/triageable ("new device, unusual ASN, first-time international") and help find FPs. Limits: correlations split credit, attributions aren't causal, it explains the model not reality, and exposing detailed reasons via an API leaks internals to attackers. Use TreeSHAP for low-latency tree ensembles.

Q20. How do you close the feedback loop safely? Analyst verdicts become weak labels with confidence, not truth. Keep a random audit of non-alerted events to estimate misses, exclude auto-approved traffic from "benign" training data, detect poisoning (label shift, boundary movement), retrain on time-based splits, monitor drift.

8. Hands-On — Feature Lists + Pseudo-Pipeline

anomaly (entity-hour):  events_count, distinct_dsts, bytes_out, failed_auth, new_device,
  new_asn, geo_velocity_kmh, hour_sin/cos, dow, rarity_dst_idf, new_process_ratio,
  parent_rarity, privileged_flag, change_window_flag

phishing (email):  url_count, max_url_len, dots, hyphens, digits, at_sign, ip_host,
  subdomain_depth, path_entropy, min_levenshtein_brand, tld_risk, domain_age_days,
  redirect_hops, cert_age_days, spf, dkim, dmarc, reply_to_mismatch,
  display_name_brand_mismatch, first_contact, urgency/authority/secrecy/cta_density,
  cred_request, payment_change_request, attachment_macro

malware (file):  byte_hist_256, byte_entropy_hist_16x16, ngram_hash_4_8,
  pe_section_entropy, import_hash_1024, has_rich, authenticode_selfsigned,
  parse_warnings_88, strings_paths/urls, api_seq_ngrams, resource_graph_degree

fraud (txn):  amount, amount_vs_30d_p95, txn_velocity_1h, distinct_cards_device_24h,
  new_device, new_payment, address_change_24h, account_age_days, category_risk,
  channel, geo_jump_km, shared_device_accounts, graph_dist_to_fraud, returns_90d,
  refund_ratio, promo_used, giftcard_load, redemption_gap_seconds, device_risk_trend

# End-to-end pseudo-pipeline: score -> explain -> tier -> feedback
def score_event(e):
    f     = extract_features(e)                     # 1: features
    score = (w1*iforest_score(f) +                  # 2: unsupervised ensemble
             w2*autoencoder_recon(f) +
             w3*lstm_next_event_error(entity_seq(e)))
    risk  = combine(calibrate(score), context(e))   # 3: context + impact
    tier  = tier_for(risk)                          # log/review/step_up/block
    if tier != "log":
        queue.add(risk, e, shap_top5(f))            # 4: explain
    return tier
# 5: analysts label; verified labels + random audit sample feed retraining

9. Quick Revision Table

10. Common Mistakes

1. Saying "block" for ML output — no tiers, no human, no appeal path means one false positive becomes a customer insult or a locked account. Always narrate log, review, step-up, and block by blast radius with a calibrated score.
2. Confusing anomaly with maliciousness — holidays, new hires, bulk orders, and deploys are all anomalous but benign. Require context, policy, and impact before calling anything malicious.
3. Quoting accuracy or ROC on imbalanced data — at 0.1% fraud even a useless model looks 99.9% accurate and ROC looks strong while precision collapses. Report precision at natural prevalence, PR-AUC, and precision at K.
4. Downsampling benign data, then claiming the resulting precision — testing on a balanced sample inflates precision because the benign denominator was shrunk artificially. Only an upstream funnel legitimately shrinks the denominator; otherwise correct with PPV_natural.
5. Forgetting zero-variance and min-sample gates — a constant metric gives divide-by-zero and infinite z-scores from as few as three observations. Use a variance floor, MAD scaling, and a minimum history gate with peer-group fallback.
6. Random train/test splits on time-series security data — random splits leak future attacks into training and hide drift. Use strictly time-based splits with the test window after training and a recent drift probe.
7. Training on auto-approvals as benign — evasions that slipped through become legitimate labels and shift the boundary toward the attacker across retrains. Train only on verified labels and audit non-alerted traffic separately to measure misses.
8. Relying on urgency or typo heuristics for phishing — LLM phishing removed the typos and can be prompted to sound calm and professional. Weight relationship context, auth alignment, and request-type risk instead of keyword lists.
9. Treating SPF/DKIM/DMARC as a silver bullet — p=none still delivers, lookalike domains pass their own auth, and compromised mailboxes pass everything. Auth proves domain control, not intent, so behavior and context must still score.
10. Exposing exact reasons or thresholds to users or APIs — detailed decline reasons teach Oracle-style attackers how to probe around the boundary. Return generic friction to users and keep precise attributions inside the analyst UI.
11. Ignoring latency and scale budgets — checkout scoring must fit roughly 100ms inline with LightGBM or Isolation Forest speed, while LSTM and LLM analysis runs async. A model that cannot serve at peak traffic is not deployable no matter its offline score.
12. Running with no drift monitoring and no false-positive budget — a stale model decays silently as families and fraud patterns rotate, and optimizing recall without a precision or queue cap produces alert fatigue. Track drift metrics and fix the threshold at review capacity with monthly re-tuning.

Checklist Coverage