In the ever-evolving world of actuarial science, striking the perfect balance between accuracy and explainability has long been a challenge. Traditional models like Generalized Linear Models (GLMs) are appreciated for their interpretability, while advanced algorithms like Gradient Boosting Machines (GBMs) are sought after for their predictive prowess. But do actuaries need to choose one over the other? With the advent of ensemble modelling, the answer is a resounding no.
Ensemble techniques like bagging and boosting offer a promising middle ground, combining the strengths of multiple models to enhance both predictive performance and transparency. This article explores the mechanics, benefits, and practical applications of ensemble models, demonstrating how actuaries can unlock new levels of precision without sacrificing clarity.
Understanding Ensemble Modelling
Ensemble modelling is the practice of combining predictions from multiple models to create a unified, robust outcome. The two most popular techniques are bagging and boosting, each with unique methods to achieve better results.
Bagging: The Wisdom of the Crowd
Bagging, short for “bootstrap aggregating,” relies on training models on different subsets of data and aggregating their predictions. By random sampling with replacement, bagging reduces random errors, leveraging the “wisdom of the crowd” to improve accuracy.
This concept traces its roots to Sir Francis Galton’s 1906 observation that the median guess in a “guess the weight of an ox” contest was astonishingly close to the actual weight, highlighting the power of collective insights.
A common extension of bagging is the random forest, which adds diversity by training each model on a random subset of features. This approach not only reduces correlation among models but also significantly enhances performance compared to bagging alone.
Boosting: Learning from Mistakes
Boosting takes a different route by building models sequentially, where each subsequent model addresses the errors of its predecessor. The method relies on gradient descent optimization, hence the name gradient boosting machines (GBMs).
One of the most widely used implementations of boosting is XGBoost, known for its speed and efficiency. XGBoost outshines traditional GBMs by incorporating parallelization, regularization techniques, and intelligent handling of missing data, making it a preferred choice for complex datasets.
Combining the Best of Both Worlds
Actuaries often rely on GLMs for their transparency and ease of use, but they may find themselves turning to GBMs for more nuanced predictions. The question then arises: can these models be combined? The answer is yes—and doing so offers unique benefits.
Boosted GLMs
A boosted GLM begins with a standard GLM to capture the main effects and maintain explainability. Residual patterns that the GLM misses are then modeled using XGBoost. These residuals, defined as the difference between the actual response and the GLM’s prediction, are added to the GLM’s output for the final prediction.
However, while this additive adjustment improves performance, it may not align well with premium breakdowns’ inherently multiplicative nature. An alternative is to model the ratio of the response variable to the GLM prediction, enabling better handling of non-linear relationships. This multiplicative approach enhances both coherence and accuracy, as reflected in improved results.
XGBoost-Initialized GLMs
Another innovative approach involves using GLM predictions to initialize an XGBoost model. This technique allows XGBoost to start with a “sketch” of the data’s structure, speeding up convergence and potentially boosting accuracy. While the benefits are more pronounced in larger datasets, this method exemplifies how blending models can streamline complex tasks.
Weighted Averages
A simpler strategy is to compute a weighted average of predictions from both GLM and XGBoost models. However, in some cases—such as the dataset analyzed in this study—the performance gain is negligible, as the XGBoost model alone delivers the best results.
Balancing Performance and Interpretability
While ensembles like boosted GLMs can enhance predictive power, actuaries must remain cautious about overfitting and loss of transparency. Including XGBoost predictions as a covariate in a GLM, for instance, can blur the interpretability of traditional GLM coefficients.
Cross-validation is essential for assessing model performance and ensuring generalizability. In this study, fivefold cross-validation was employed, providing a robust measure of each model’s predictive accuracy.
Lessons from Historical Wisdom
The principles behind ensemble modelling echo Aristotle’s observation that collective effort often surpasses individual capability: “It is possible that the many, though not individually good, yet when they come together may be better, not individually but collectively.”
Ensemble techniques like bagging and boosting embody this idea, proving that even relatively weak models can achieve remarkable results when combined. However, the key to success lies in diversity—models must complement each other, bringing unique strengths to the table.
Final Thoughts
Ensemble modelling represents a significant step forward for actuaries, offering a practical way to balance accuracy and explainability. While it requires additional effort, the rewards—improved performance, greater insights, and enhanced stakeholder confidence—make it well worth the investment.