ml-churn-pipeline - leakage-safe model card

Jigon Yoo
Published

0
  • Project creator

A churn classifier evaluated the way a client should ask for it: split before scaling, a baseline to beat, cross-validation, and explicit limits.

The problem. One lucky accuracy score is not evidence. Churn demos often leak test information into training, ignore the majority baseline, and omit uncertainty entirely.

Committed result: held-out ROC-AUC 0.7982 against a 0.5000 majority baseline; 5-fold cross-validated ROC-AUC 0.819 +/- 0.0298.

1,500 fictional customers are split first into 1,125 training and 375 held-out test rows, and the scaler is fit on training rows only - so the test set never informs the transform.

Against the majority baseline: accuracy 0.7067 vs 0.5893, precision 0.6571 vs 0.0, recall 0.5974 vs 0.0, F1 0.6259 vs 0.0. The model card states what the model does not know as clearly as what it does.

Code: github.com/jigonyoo/ml-churn-pipeline