COMPARATIVE ANALYSIS OF BIAS AND ROOT MEAN SQUARE ERROR IN RESAMPLING AND PARAMETRIC POWER ESTIMATION
DOI:
https://doi.org/10.69980/sdqawe10Keywords:
Bootstrap Resampling, Parametric Power Estimation, Statistical Bias, Root Mean Square Error, Monte Carlo SimulationAbstract
Power estimation is very important in proper research planning, especially for small samples and different distributional settings. In this study, the accuracy of the resampling-based and parametric statistical power estimation methods was compared using bias and rmse. Secondary Monte Carlo simulation data for 36 conditions (3 levels of pilot size and 4 levels of effect size with three probability distributions) were subjected to a quantitative comparative analysis. Estimation accuracy was assessed using descriptive statistics and at the scenario level. The overall mean RMSE of the resampling-based estimation method (0.2796) was lower than that of the parametric estimation method (0.2991) by approximately 6.53%. The mean absolute bias for the resampling was also lower (0.1094) than that of the parametric estimation (0.1269). However, Parametric estimation had lower RMSE in 19 of 36 scenarios. As the size of the pilot sample increased, both methods were found to have decreasing estimation error. Resampling worked well under the skewed distributions and in moderate to large effect sizes while parametric estimation had marginal benefits under normal or bounded distributions. This suggests that the accuracy of estimation is a function of sample size, size of the effect and distributional characteristics. The study highlights that when choosing statistical power estimation methods, it is important to consider both the bias and RMSE, and that methodological decisions can be condition specific when planning small sample studies.
References
1.Acici, K. (2025). Comparative analysis of machine and deep learning algorithms for Bragg peak estimation in polymeric materials for tissue-sparing radiotherapy. Polymers, 17(15), 2068.
2.Allal, Z., Noura, H. N., Salman, O., & Chahine, K. (2025). Machine learning-based prediction model of wind turbine power generation. Cleaner Energy Systems, 100218.
3.Alrumaidhi, M., Farag, M. M., & Rakha, H. A. (2023). Comparative analysis of parametric and non-parametric data-driven models to predict road crash severity among elderly drivers using synthetic resampling techniques. Sustainability, 15(13), 9878.
4.Awopejo, T. E., Adigun, P. O., Oyekanmi, T. T., Azeez, N. A. A., Adekanye, M. A., & Obisesan, A. (2025). Machine learning-based prediction of magnetic properties from hysteresis curves: A comparative study of Random Forest, Gradient Boosting, XGBoost, LightGBM and Support Vector Regressions. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 8(6), 13456-13479.
5.Carrera, B., & Kim, K. (2024). Comparative analysis of machine learning techniques in predicting wind power generation: A case study of 2018–2021 data from Guatemala. Energies, 17(13), 3158.
6.Chaaban, A. K., & Alfadl, N. (2024). A comparative study of machine learning approaches for an accurate predictive modeling of solar energy generation. Energy Reports, 12, 1293-1302.
7.Cui, B., Chen, W., Weng, D., Wei, X., Sun, Z., Zhao, Y., & Liu, Y. (2023). Observability-constrained resampling-free cubature Kalman filter for GNSS/INS with measurement outliers. Remote sensing, 15(18), 4591.
8.da Silva, E. A., Rocha, D. S., Naves, E. L. M., & Rodrigues, T. B. (2026). Resampling-based power simulation in small samples: a Monte Carlo calibration study (reproducibility package) (Version v1) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.20767954.
9.Deville, L., Theristis, M., King, B. H., Chambers, T. L., & Stein, J. S. (2024). Open‐source photovoltaic model pipeline validation against well‐characterized system data. Progress in Photovoltaics: Research and Applications, 32(5), 291-303.
10.Dong, G., Shen, F., Sun, L., Zhang, M., & Wei, J. (2024). A Bayesian inferred health prognosis and state of charge estimation for power batteries. IEEE Transactions on Instrumentation and Measurement, 74, 1-12.
11.Elsebaie, I. H., Kawara, A. Q., Alharbi, R., & Alnahit, A. O. (2025). Bias correction methods applied to satellite rainfall products over the western part of Saudi Arabia. Atmosphere, 16(7), 772.
12.Ferkous, K., Guermoui, M., Bellaour, A., boulmaiz, T., & Bailek, N. (2024). Enhancing photovoltaic energy forecasting: a progressive approach using wavelet packet decomposition. Clean Energy, 8(3), 95-108.
13.Fitrianto, A., & Linganathan, P. (2022). Comparisons between resampling techniques in linear regression: A simulation study. CAUCHY: Jurnal Matematika Murni dan Aplikasi, 7(3), 345-353.
14.Han, X., Li, X., Yang, J., Tao, W., Han, G., Wang, J., ... & Li, W. (2025). Evaluation and deep learning-based calibration of nearshore sea surface wind speeds from FY-3E GNOS-II and Tianmu-1 missions. Geo-Spatial Information Science, 28(6), 2709-2720.
15.James, G., Witten, D., Hastie, T., Tibshirani, R., & Taylor, J. (2023). Resampling methods. In An introduction to statistical learning: With applications in python (pp. 201-228). Cham: Springer International Publishing.
16.Khan, S., Mazhar, T., Khan, M. A., Shahzad, T., Ahmad, W., Bibi, A., ... & Hamam, H. (2024). Comparative analysis of deep neural network architectures for renewable energy forecasting: enhancing accuracy with meteorological and time-based features. Discover Sustainability, 5(1), 533.
17.Khosravi, K., Farooque, A. A., Karbasi, M., Ali, M., Heddam, S., Faghfouri, A., & Abolfathi, S. (2025). Enhanced water quality prediction model using advanced hybridized resampling alternating tree-based and deep learning algorithms. Environmental Science and Pollution Research, 32(11), 6405-6424.
18.Koukaras, P., Mustapha, A., Mystakidis, A., & Tjortjis, C. (2024). Optimizing building short-term load forecasting: A comparative analysis of machine learning models. Energies, 17(6), 1450.
19.Lee, J., Kwon, D. H., Jeong, H., Nam, G., Hwang, E., Kim, J. H., ... & Kim, H. G. (2025). Improving chlorophyll-a estimation using Sentinel-2 data: a comparative analysis of augmented datasets. GIScience & Remote Sensing, 62(1), 2496551.
20.Martinović, M., Dokic, K., & Pudić, D. (2025). Comparative analysis of machine learning models for predicting innovation outcomes: an applied AI approach. Applied Sciences, 15(7), 3636.
21.Matavel, C. E., Meyer-Aurich, A., & Piepho, H. P. (2024). Model-averaging as an accurate approach for ex-post economic optimum nitrogen rate estimation. Precision Agriculture, 25(3), 1324-1339.
22.Matougui, Z., Djerbal, L., & Bahar, R. (2024). A comparative study of heterogeneous and homogeneous ensemble approaches for landslide susceptibility assessment in the Djebahia region, Algeria. Environmental Science and Pollution Research, 31(28), 40554-40580.
23.Muhammad, M., Ul-Saufie, A. Z., Radi, N. F. A., Noor, N. M., & Gusnanto, A. (2025). Modified resampling strategy for extreme values in imbalanced air pollution data using moving block bootstrapping approach with relevance weighting (MBB-RW). Scientific Reports, 16(1), 290.
24.Nguinabé, J., Rockefeller, R., Khan, N. M., & Rodrigues, P. C. (2025). Bootstrap prediction intervals for the long short term memory (LSTM) recurrent neural network. Expert Systems with Applications, 284, 127728.
25.Pai, A., Mishra, K. K., Jeyan, J. M. L., & Sayal, A. (2025). Enhanced household energy consumption forecasting using multivariate long short-term memory (LSTM) networks with weather data integration. Results in Engineering, 106512.
26.Settu, P., & Ramaiah, M. (2025). A data driven comparison of hybrid machine learning techniques for soil moisture modeling using remote sensing imagery. Scientific Reports, 15(1), 43170.
27.Taghavi-Bayat, A., Gerke, M., & Riedel, B. (2024). Soil moisture retrieval at high spatial resolution over alpine ecosystems on Nagqu-Tibetan plateau: A comparative study on semiempirical and machine learning approaches. Science of Remote Sensing, 9, 100135.
28.Wang, L., & Gao, Y. (2025). Estimating and downscaling ESA-CCI soil moisture using Multi-Source remote sensing images and Stacking-Based ensemble learning algorithms in the Shandian River Basin, China. Remote Sensing, 17(4), 716.
29.Wei, Y., Hashim, H. B., Lai, S. H., Chong, K. L., Huang, Y. F., Ahmed, A. N., ... & El-Shafie, A. (2024). Comparative analysis of artificial intelligence methods for streamflow forecasting. IEEe Access, 12, 10865-10885.
30.Zeng, Z., Tao, L., Su, R., Zhu, Y., Meng, L., Tuheti, A., ... & Chen, C. (2023). Unsupervised transfer learning approach with adaptive reweighting and resampling strategy for inter-subject EOG-based gaze angle estimation. IEEE Journal of Biomedical and Health Informatics, 28(1), 157-168.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Dr. Natalie Brooks, Dr. Matthias Keller, Dr. Charlotte Evans
This work is licensed under a Creative Commons Attribution 4.0 International License.
