Quantifying Uncertainty for the Prediction of Directional Earnings Changes
Abstract
Our study investigates the predictive performance and uncertainty quantification of directional earnings changes using machine learning methods. While prior literature has focused on improving predictive performance, little attention has been paid to understanding prediction model uncertainty. We apply Logistic Regression, Random Forest, and CatBoost to predict one-year-ahead earnings direction for U.S. public firms using annual data from 2010 to 2022. To quantify uncertainty, we implement a method named Venn-ABERS, which is a model-agnostic approach that provides calibrated probability intervals. These intervals are incorporated into hedge portfolio construction to evaluate predictive and economic significance of our approach. We show that machine learning models consistently outperform Logistic Regression in predictive accuracy and yield higher size-adjusted returns for a 12 month buy and hold portfolio. When model uncertainty is considered for hedge portfolio construction by only shorting stocks with low predictive uncertainty, the resulting portfolios exhibit significantly reduced risk profiles, evidenced by lower return volatility, narrower interquartile ranges, and fewer negative-return trades compared to portfolios without integrating model uncertainty. Moreover, returns are higher for uncertainty-aware portfolios, as filtering out short positions with high uncertainty removes predominantly unprofitable trades while retaining the more profitable shorts. These results highlight the relevance of integrating uncertainty quantification in financial prediction models.