ROBUST PREDICTIVE MODELING UNDER DATA HETEROGENEITY: INTEGRATING STATISTICAL INFERENCE, ENSEMBLE LEARNING, AND UNCERTAINTY QUANTIFICATION
DOI:
https://doi.org/10.69980/vspxf098Keywords:
Data heterogeneity, Ensemble learning, Predictive modeling, Statistical inference, Uncertainty quantificationAbstract
Robust predictive modeling requires simultaneous consideration of data heterogeneity, predictive performance, and uncertainty. This study developed an integrated framework combining statistical inference, ensemble learning, and uncertainty assessment using heterogeneous bank-marketing data. After duplicate removal, 41,176 observations were analyzed, with successful subscriptions accounting for 11.27% of cases. Logistic regression, Random Forest, XGBoost, and a soft-voting ensemble were evaluated using discrimination, classification, and probability-based metrics. Contact duration was excluded from the primary prospective models and subsequently included in sensitivity analysis. Random Forest achieved the highest prospective ROC-AUC (0.813) and PR-AUC (0.486), whereas XGBoost obtained the lowest Brier score (0.076). The soft-voting ensemble achieved the highest F1-score (0.526) and Matthews correlation coefficient (0.462). Bootstrap confidence intervals showed considerable overlap among model discrimination estimates, indicating uncertainty around small performance differences. Inclusion of contact duration increased ROC-AUC to 0.940–0.950, demonstrating that predictor availability affected performance more strongly than algorithm selection. These findings emphasize the importance of evaluating predictive models through multiple performance measures, uncertainty estimates, and realistic predictor specifications. The proposed framework provides a statistically grounded approach for assessing predictive robustness in heterogeneous data environments.
References
1. Ahmad, N., Wali, B., & Khattak, A. J. (2023). Heterogeneous ensemble learning for enhanced crash forecasts–A frequentist and machine learning based stacking framework. Journal of safety research, 84, 418-434.
2. Angelopoulos, A. N., Bates, S., Fannjiang, C., Jordan, M. I., & Zrnic, T. (2023). Prediction-powered inference. Science, 382(6671), 669-674.
3. Chen, H., Jing, X. Y., Zhou, Y., Li, B., & Xu, B. (2022). Aligned metric representation based balanced multiset ensemble learning for heterogeneous defect prediction. Information and Software Technology, 147, 106892.
4. Cui, S., Wang, Y., Wang, D., Sai, Q., Huang, Z., & Cheng, T. C. E. (2021). A two-layer nested heterogeneous ensemble learning predictive method for COVID-19 mortality. Applied Soft Computing, 113, 107946.
5. Evangelista, E., & Sy, B. D. (2022). An approach for improved students’ performance prediction using homogeneous and heterogeneous ensemble methods. International Journal of Electrical and Computer Engineering, 12(5), 5226.
6. Fang, Z., Wang, Y., Peng, L., & Hong, H. (2021). A comparative study of heterogeneous ensemble-learning techniques for landslide susceptibility mapping. International Journal of Geographical Information Science, 35(2), 321-347.
7. Huang, Y., Feng, X., Li, B., Xiang, Y., Wang, H., Liu, T., & Qin, B. (2024). Ensemble learning for heterogeneous large language models with deep parallel collaboration. Advances in Neural Information Processing Systems, 37, 119838-119860.
8. Kashefian-Naeeini, S., Shokrpour, N., & Pakdel, F. (2024). Optimizing EFL learning: Exploring the role of learner background factors and the nuances of their effects on intrinsic and extrinsic motivation among university students in a mixed-methods study. BMC psychology, 12(1), 535.
9. Liu, J., & Cui, P. (2025, August). Data heterogeneity modeling for trustworthy machine learning. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 (pp. 6086-6095).
10. Majlatow, M., Shakil, F. A., Emrich, A., & Mehdiyev, N. (2025). Uncertainty-aware predictive process monitoring in healthcare: explainable insights into probability calibration for conformal prediction. Applied Sciences, 15(14), 7925.
11. Mendil, M., Mossina, L., & Vigouroux, D. (2023, August). Puncc: a python library for predictive uncertainty calibration and conformalization. In Conformal and Probabilistic Prediction with Applications (pp. 582-601). PMLR.
12. Naim, M. (2026). Bank marketing dataset (UCI) [Data set]. Kaggle
13. Nanglia, S., Ahmad, M., Khan, F. A., & Jhanjhi, N. Z. (2022). An enhanced Predictive heterogeneous ensemble model for breast cancer prediction. Biomedical Signal Processing and Control, 72, 103279.
14. Qian, J., Hansen, L. K., Fafoutis, X., Tiwari, P., & Pandey, H. M. (2020). Robustness analytics to data heterogeneity in edge computing. Computer Communications, 164, 229-239.
15. Salama, M., Abdelkader, H., & Abdelwahab, A. (2022). A novel ensemble approach for heterogeneous data with active learning. International Journal of Engineering Business Management, 14, 18479790221082605.
16. Su, Z., Lu, Z., Wu, Y., Shen, R., & Lu, S. (2025, October). FLSeg: Enhancing Privacy and Robustness in Federated Learning under Heterogeneous Data via Model Segmentation. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 3916-3925). IEEE.
17. Tamuly, P., & Nava, V. (2025). Machine learning based conformal predictors for uncertainty-aware compressive strength estimation of concrete. Construction and Building Materials, 487, 141844.
18. Wang, S., Fu, Y., Li, X., Lan, Y., & Gao, M. (2023). Dfrd: Data-free robustness distillation for heterogeneous federated learning. Advances in Neural Information Processing Systems, 36, 17854-17866.
19. Xie, Y., Li, A., Gao, L., & Liu, Z. (2021). A heterogeneous ensemble learning model based on data distribution for credit card fraud detection. Wireless Communications and Mobile Computing, 2021(1), 2531210.
20. Yuniar, R. F. (2021). The impact of students’ perception on their academic background on their learning motivation. Journal of English Language Teaching and Learning (JETLE), 2(2), 78-84.
21. Zawad, S., Ali, A., Chen, P. Y., Anwar, A., Zhou, Y., Baracaldo, N., ... & Yan, F. (2021, May). Curse or redemption? how data heterogeneity affects the robustness of federated learning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 35, No. 12, pp. 10807-10814).
22. Zefrehi, H. G., & Altınçay, H. (2020). Imbalance learning using heterogeneous ensembles. Expert Systems with Applications, 142, 113005.
23. Zhang, T., Zhang, D. G., Yan, H. R., Qiu, J. N., & Gao, J. X. (2021). A new method of data missing estimation with FNN-based tensor heterogeneous ensemble learning for internet of vehicle. Neurocomputing, 420, 98-110.
24. Zhao, Y., & Deng, W. (2022). Prediction in traffic accident duration based on heterogeneous ensemble learning. Applied Artificial Intelligence, 36(1), 2018643.
25. Zrnic, T., & Candès, E. J. (2024). Active statistical inference. arXiv preprint arXiv:2403.03208.
