SIMULATION-BASED COMPARISON OF PARAMETRIC AND NONPARAMETRIC STATISTICAL METHODS UNDER DISTRIBUTIONAL VIOLATIONS

Abstract

Parametric statistical procedures are commonly used due to their interpretability and efficiency, but can be suboptimal when assumptions of normal data, homogeneity of variance, and resistance to extreme observations are not met. In this study, parametric and nonparametric statistical methods were contrasted with respect to specific violations of their underlying distributions in a Monte Carlo simulation paradigm with empirical validation. Student's independent-samples t-test, Welch's t-test, and the Mann–Whitney U test were tested under the following conditions: normal, skewed, heavy-tailed, heteroscedastic and contaminated distributions. The sample size, effect size and outlier contamination were systematically manipulated, and method performance was evaluated largely by Type I error rates and statistical power. The results indicated that both Student's and Welch's tests were very effective under conditions of approximate normality and were stronger as sample size increased. The Mann-Whitney U test, however, performed better when the distributions were highly skewed, had heavy tails and were contaminated, especially at small and moderate sample sizes. Empirical comparisons also suggested that method choice may affect the results of statistical significance when the data is not normally distributed and when variance is not equal. In general, the results presented here support a condition-dependent philosophy for choosing statistical tests: one should consider the shape of the distribution, sample size, the nature of the variance, and unusual behavior of the data points.