Fraud Model Maintenance: Why Your Detection Rate Drops Six Months After Launch

Abstract

Purpose – Practitioners often say that a fraud detection model loses its edge about six months after launch. This paper examines whether that belief holds up by auditing what the 2008 to 2023 literature claimed about maintaining fraud models. This study was designed and written in 2023 as a retrospective; it makes no claim to have been produced in the period it studies.
Design – Twelve maintenance claims were extracted from a verified corpus of 67 sources published between 2008 and 2023. Each was tested on a seeded synthetic transaction benchmark that instantiates the era's own assumptions (class imbalance, daily batches, changing fraud patterns, delayed labels), under decision rules fixed before the experiments, and against 34 later sources.
Findings – On the benchmark, 4 claims held, 6 were qualified, and 2 were overturned. A never-updated model lost 24% to 58% of its detection rate in six months depending on how long fraud patterns lasted, and nothing when they did not change. Retraining mattered more than the choice between non-linear learners. AUC understated decay, and input-stability monitoring did not detect changes in fraud.
Research limitations – All data are synthetic and results depend on the generator's design; the later literature reading is targeted, not systematic.
Practical implications – Teams should measure the turnover of fraud patterns in their own cases, shorten label delays, and set review cycles against turnover instead of relying on a fixed interval.
Originality – The paper offers a claim-by-claim verdict matrix for the era's fraud model maintenance literature, with a shipped, reproducible benchmark.