A ROBUST STATISTICAL FRAMEWORK FOR MODELING NONLINEAR RELATIONSHIPS IN HIGH-DIMENSIONAL DATA UNDER HETEROSCEDASTICITY AND OUTLIERS
DOI:
https://doi.org/10.69980/ht432337Keywords:
Robust regression, High-dimensional data, Nonlinear modeling, Heteroscedasticity, Outlier contaminationAbstract
In many practical applications, high-dimensional data may have complex nonlinearities, unequal error variance, and influential cases that may make conventional statistical models unreliable. This study proposed a comprehensive statistical model that combines the following methods: regularized feature selection, nonlinear estimation, heteroscedasticity adjustment, and robust loss-based estimation. 146 predictors of the superconducting critical temperature were analysed, and the proposed method was compared to ordinary least squares (OLS), Ridge, LASSO, Elastic Net, Huber regression, and generalized additive modeling. After diagnostic analysis there were high leverage and influential observations, and a Breusch–Pagan statistic of 2818.06 and p < 0.001 was observed. Important predictors also showed evidence of meaningful nonlinear relationships, with spline transformations showing improved explanatory performance. OLS was found to be the most predictive technique, yielding an RMSE of 17.078 and an R2 of 0.742 on test data that were not contaminated, while Huber regression yielded the lowest MAE of 12.898. But with an increasing number of artificial outliers, the performance of OLS and Elastic Net declined as they kept getting worse, while the proposed robust nonlinear framework maintained relatively stable RMSE and R2 values. The results illustrate an important accuracy–robustness trade-off and the importance of the framework proposed here for high-dimensional data with heteroscedasticity, non-linearity and influential observations.
References
1. Adiga, S., & Waghmare, U. V. (2026). Accelerating the search for superconductors using machine learning. Computational Materials Science, 263, 114453.
2. Adomaityte, U., Defilippis, L., Loureiro, B., & Sicuro, G. (2024). High-dimensional robust regression under heavy-tailed data: Asymptotics and universality. Journal of Statistical Mechanics: Theory and Experiment, 2024(11), 114002.
3. Alibagheri, E., Sandoghchi, M., Seyfi, A., Khazaei, M., & Allaei, S. M. V. (2025). A physics-informed machine learning framework for unified prediction of superconducting transition temperatures. Materials Today Physics, 101971.
4. Beyhum, J. (2023). High-dimensional inference robust to outliers with ℓ 1-norm penalization. Communications in Statistics-Theory and Methods, 52(16), 5866-5876.
5. Farnè, M., & Vouldis, A. (2024). ROBOUT: a conditional outlier detection methodology for high-dimensional data: M. Farnè, A. Vouldis. Statistical Papers, 65(4), 2489-2525.
6. Feng, Y., & Wu, Q. (2022). A statistical learning assessment of Huber regression. Journal of Approximation Theory, 273, 105660.
7. Filzmoser, P., & Nordhausen, K. (2021). Robust linear regression for high‐dimensional data: An overview. Wiley Interdisciplinary Reviews: Computational Statistics, 13(4), e1524.
8. Gandomani, M. J., Gandomani, T. J., & Dashti, M. (2025). AI-driven superconductor prediction: an attention-based deep learning approach. Computational Materials Science, 258, 114129.
9. García-Nieto, P. J., García-Gonzalo, E., & Paredes-Sánchez, J. P. (2021). Prediction of the critical temperature of a superconductor by using the WOA/MARS, Ridge, Lasso and Elastic-net machine learning techniques. Neural Computing and Applications, 33(24), 17131-17145.
10. Gashmard, H., Shakeripour, H., & Alaei, M. (2024). Predicting superconducting transition temperature through advanced machine learning and innovative feature engineering. Scientific Reports, 14(1), 3965.
11. Giudici, P., Raffinetti, E., & Riani, M. (2025). Robust machine learning models: linear and nonlinear. International Journal of Data Science and Analytics, 20(2), 1043-1050.
12. Jung, S. G., Jung, G., & Cole, J. M. (2024). Machine-learning predictions of critical temperatures from chemical compositions of superconductors. Journal of Chemical Information and Modeling, 64(19), 7349.
13. Li, S., Dan, Y., Li, X., Hu, T., Dong, R., Cao, Z., & Hu, J. (2020). Critical temperature prediction of superconductors based on atomic vectors and deep learning. Symmetry, 12(2), 262.
14. Liu, B., Zhang, Q., Xue, L., Song, P. X. K., & Kang, J. (2024). Robust high-dimensional regression with coefficient thresholding and its application to imaging data analysis. Journal of the American Statistical Association, 119(545), 715-729.
15. Liu, C. H., & Novikov, G. (2024). Robust sparse regression with non-isotropic designs. Advances in Neural Information Processing Systems, 37, 270-305.
16. Liu, L., Shen, Y., Li, T., & Caramanis, C. (2020, June). High dimensional robust sparse regression. In International Conference on Artificial Intelligence and Statistics (pp. 411-421). PMLR.
17. Ma, X., Wei, L., & Liang, W. (2024). D4R: Doubly robust reduced rank regression in high dimension. Journal of Statistical Planning and Inference, 232, 106162.
18. Metni, H., Takeuchi, I., & Stanev, V. (2023). Predicting the superconducting critical temperature in transition metal carbides and nitrides using machine learning. Physica C: Superconductivity and its Applications, 605, 1354209.
19. Mohammed Rashid, A., Midi, H., Dhhan, W., & Arasan, J. (2022). Detection of outliers in high-dimensional data using nu-support vector regression. Journal of Applied Statistics, 49(10), 2550-2569.
20. Pfeiffer, P., & Filzmoser, P. (2023). Robust statistical methods for high-dimensional data, with applications in tribology. Analytica Chimica Acta, 1279, 341762.
21. Revathy, G., Rajendran, V., & Kumar, P. S. (2021). Prediction study on critical temperature (C) of different atomic numbers superconductors (both gaseous/solid elements) using machine learning techniques. Materials Today: Proceedings, 44, 3627-3632.
22. Roter, B., & Dordevic, S. V. (2020). Predicting new superconductors and their critical temperatures using machine learning. Physica C: Superconductivity and its applications, 575, 1353689.
23. Salibian-Barrera, M. (2023). Robust nonparametric regression: Review and practical considerations. Econometrics and Statistics.
24. Sasaki, H., Takenouchi, T., Monti, R., & Hyvarinen, A. (2020, August). Robust contrastive learning and nonlinear ICA in the presence of outliers. In Conference on Uncertainty in Artificial Intelligence (pp. 659-668). PMLR.
25. Zhuang, C. (2024). SUPERCON dataset: A superconductor materials database from NIMS [Data set]. Kaggle. https://www.kaggle.com/datasets/chaozhuang/supercon-dataset
