Which merchants will stop processing in the next three months, what is really driving it, and does calling them early keep them? A churn model, a causal analysis and a randomised test on 10,000 merchants over 2025.
A Random Forest scores each merchant monthly from point-in-time features: volume against its own three-month average, seasonally adjusted trend, active days, failed payments, payout delays and support tickets. SMOTE balances the training months only, and the alert threshold is tuned on a later month before testing on August and September.
| Test months | Precision | Recall | F1 | ROC-AUC | PR-AUC |
|---|---|---|---|---|---|
| Aug–Sep 2025 | 0.666 | 0.368 | 0.475 | 0.800 | 0.504 |
Accuracy is 92.3%, but with 9.5% of merchants churning, accuracy says little. Precision at the alert threshold sets how much account-manager time is wasted.
In July, merchants on the plugin integration moved to a new payout verification flow. Volume also swings with the seasons, so a before/after comparison mixes the two. Difference-in-differences with merchant and month fixed effects compares plugin merchants to everyone else over the same months.
The friction cut plugin merchants' volume by -8.3% (95% CI -9.1% to -7.5%). The groups moved together before July (pre-trend p = 0.16).
Merchants flagged in September were randomly split. Half got an account-manager call within 24 hours; half got nothing extra.
| Group | Merchants | Churned |
|---|---|---|
| Control | 214 | 70.1% |
| Outreach | 202 | 60.4% |
Churn fell by 9.7 points (p = 0.038), about 20 merchants and $53,506 a year at a 1.5% take rate.
A Flask endpoint returns a churn probability, risk tier and recommended action for each merchant, with p95 latency under 50 ms in tests. A Dockerfile packages the API with a trained model.
All data is synthetic, generated with known effects so every method can be checked against the truth; the A/B outcomes are simulated from an assumed outreach effect. Built with AI-assisted coding (Claude); modeling choices, validation and review by Martins Ogiekpolor.