Calibration and Uncertainty Quantification in Bankruptcy Risk Modeling

Abstract

This study investigates the role of classifier calibration and uncertainty quantification in corporate bankruptcy prediction for private firms. Using a panel of 347,283 German companies observed from 2010 to 2023, the study compares penalized Logistic Regression, Random Forest, and CatBoost in predicting bankruptcies. The models are trained in a time-aware rolling window framework and evaluated on both statistical and economic dimensions. From a model performance perspective, all classifiers exhibit strong discriminatory power, with machine learning models outperforming Logistic Regression in ROC AUC. However, we find that probabilistic models are only modestly calibrated, and cost-sensitive learning severely distorts probability of default estimates without materially improving predictive accuracy. Applying Venn-ABERS, a model-agnostic calibration method from the Conformal Prediction framework, improves calibration for all models and successfully restores distorted probabilities of default for cost-sensitive specifications, while providing probability intervals as a measure of model uncertainty. Embedding these classifiers in an Economic Competition Model, we demonstrate that incorporating calibrated probabilities and probability intervals leads to substantially different lending, default, and profitability outcomes. Overall, our results provide a first comprehensive assessment of Venn-ABERS calibration and uncertainty quantification in private firm bankruptcy prediction using the actual insolvency status of a company and contribute to ongoing discussions on algorithmic transparency and reliable machine learning applications for financial decision-making.