SMALL-SAMPLE LOGISTIC REGRESSION WITH PENALIZED LIKELIHOOD AND PRINCIPAL COMPONENT ANALYSIS
DOI:
https://doi.org/10.69980/jpc4eb82Keywords:
Principal component analysis, Firth logistic regression, penalized likelihood, small-sample modelling, binary classificationAbstract
This study examined the applicability of the principal component analysis (PCA) and Firth penalized logistic regression (FPLR) approach to modelling binary outcomes with small samples. An open-access dataset of 23 older adults and their demographic, health-related, and eight predictor variables from questionnaires was used to complete a secondary quantitative analysis. The binary outcome consisted of 11 observations in class 0 and 12 observations in class 1. The eight variables from the questionnaire were standardised and then subjected to PCA, the Kaiser-Meyer-Olkin statistic was used to assess sampling adequacy. PCs 1 and 2 were kept for further Firth logistic regression. The PCA analysis revealed that PC1 and PC2 accounted for 47.71% and 19.46% of the variance, respectively, with a total of 67.18% variance explained. In the penalized regression model, PC1 was found to be associated with the binary outcome (OR = 16.68, p < 0.001), while PC2 was not (OR = 0.77, p = 0.792). For the model, in-sample accuracy, sensitivity, specificity, precision and F1-score were all 100% while the AUC was 1.000. The results are preliminary due to the very limited sample size and lack of external validation, however. In conclusion, the PCA–Firth framework was a parsimonious exploratory framework for small-sample binary classification problems in which predictors are correlated.
References
1. Intergovernmental Panel on Climate Change (IPCC). (2023). Climate Change 2023: Synthesis Report. Geneva: IPCC.
2. World Meteorological Organization (WMO). (2024). State of the Global Climate 2023. Geneva: WMO.
3. India Meteorological Department (IMD). (2024). Annual Climate Summary of India. New Delhi: Ministry of Earth Sciences.
4. Kumar, V., Jain, S. K., & Singh, Y. (2022). Analysis of rainfall variability and trends over India. Theoretical and Applied Climatology, 147(1–2), 341–356.
5. Dash, S. K., Jenamani, R. K., Kalsi, S. R., & Panda, S. K. (2021). Some evidence of climate change in twentieth-century India. Climatic Change, 85(3–4), 299–321.
6. Goswami, B. N., Venugopal, V., Sengupta, D., Madhusoodanan, M. S., & Xavier, P. K. (2006). Increasing trend of extreme rain events over India. Science, 314(5804), 1442–1445.
7. Rajeevan, M., Bhate, J., & Jaswal, A. K. (2008). Analysis of variability and trends of extreme rainfall events over India. International Journal of Climatology, 28(14), 1837–1847.
8. Guhathakurta, P., Sreejith, O. P., & Menon, P. A. (2011). Impact of climate change on extreme rainfall events and flood risk in India. Journal of Earth System Science, 120(3), 359–373.
9. Pai, D. S., Sridhar, L., Rajeevan, M., Sreejith, O. P., Satbhai, N. S., & Mukhopadhyay, B. (2014). Development of a new high spatial resolution rainfall dataset for India. Mausam, 65(1), 1–18.
10. Krishnan, R., Shrestha, A. B., Ren, G., et al. (2020). Climate Change and Land: IPCC Special Report. Cambridge University Press.
11. Singh, D., Tsiang, M., Rajaratnam, B., & Diffenbaugh, N. S. (2014). Observed changes in extreme wet and dry spells during the South Asian monsoon season. Nature Climate Change, 4(6), 456–461.
12. Mishra, V., Kumar, D., & Ganguly, A. R. (2020). Climate change and hydrologic extremes in India. Journal of Hydrology, 589, 125–142.
13. Jain, S. K., Kumar, V., & Saharia, M. (2013). Analysis of rainfall and temperature trends in northeast India. International Journal of Climatology, 33(4), 968–978.
14. Chattopadhyay, S., & Edwards, D. R. (2016). Long-term trends in precipitation and temperature in eastern India. Atmospheric Research, 178–179, 146–157.
15. Pattanaik, D. R., Mohanty, U. C., & Kumar, A. (2019). Climate variability and changing monsoon characteristics over eastern India. Meteorological Applications, 26(3), 450–462.
