CROSS-CLASSIFIED LOGISTIC MODELING OF CORRECT RESPONSE PROBABILITY IN HIGHER-EDUCATION MATHEMATICS ASSESSMENT
DOI:
https://doi.org/10.69980/rpsstb49Keywords:
cross-classified logistic regression, mathematics assessment, higher education, student heterogeneity, response probabilityAbstract
Assessment data in higher education mathematics courses are frequently characterized by complex dependence because individual students answer several questions while the same questions may be answered by multiple students. The analysis included 9,546 responses from 372 students across 833 mathematics questions. Overall, 46.8% of responses were correct. Some topic-specific differences were detected, with lower odds of correct answers for Differentiation, Probability, Real Functions of a Single Variable, and Fundamental Mathematics compared to Linear Algebra, while Set Theory had higher odds. The proportion of latent variance due to student-level heterogeneity was substantially larger than that due to question-level heterogeneity. The cross-classified model showed better model fit, apparent discrimination, and probabilistic accuracy than traditional logistic regression and single-cluster alternatives, as evidenced by lower WAIC, log loss and Brier scores, and a higher AUC. Sensitivity analyses indicated that the main topic effects and variance structure were robust to changes in the specification. The results are congruent with cross-classified modeling as a systematic method to analyze mathematics assessment data and partition sources of variation at the learner- and item-level.
References
1. Alreshidi, N. A. K. (2023). Enhancing topic-specific prior knowledge of students impacts their outcomes in mathematics. Frontiers in Education, 8, Article 1050468.
2. Azevedo, B. F., Pacheco, M. F., Fernandes, F. P., & Pereira, A. I. (2024). Assessing mathematics learning in higher education [Data set]. UCI Machine Learning Repository.
3. Bjälkebring, P. (2019). Math anxiety at the university: What forms of teaching and learning statistics in higher education can help students with math anxiety? Frontiers in Education, 4, Article 30.
4. Caspari-Sadeghi, S. (2023). Learning assessment in the age of big data: Learning analytics in higher education. Cogent Education, 10(1), 2162697.
5. Di Martino, P., Gregorio, F., & Iannone, P. (2023). The transition from school to university in mathematics education research: New trends and ideas from a systematic literature review. Educational Studies in Mathematics, 113(1), 7-34.
6. Engelbrecht, J., & Borba, M. C. (2024). Recent developments in using digital technology in mathematics education. ZDM–Mathematics Education, 56(2), 281-292.
7. Geisler, S., Rolka, K., & Rach, S. (2023). Development of affect at the transition to university mathematics and its relation to dropout—Identifying related learning situations and deriving possible support measures. Educational Studies in Mathematics, 113(1), 35–56.
8. Guzmán-Valenzuela, C., Gómez-González, C., Rojas-Murphy Tagle, A., & Lorca-Vyhmeister, A. (2021). Learning analytics in higher education: a preponderance of analytics but very little learning? International Journal of Educational Technology in Higher Education, 18(1), 23.
9. Gyamfi, A., & Wren, D. G. (2022). Determining the difficulty and discrimination parameters of a mathematics performance-based assessment. Creative Education, 13(11), 3483–3489.
10. Khasawneh, E., Gosling, C., & Williams, B. (2021). What impact does maths anxiety have on university students? BMC Psychology, 9(1), 37.
11. Park, J. Y., Dedja, K., Pliakos, K., Kim, J., Joo, S., Cornillie, F., Vens, C., & Van den Noortgate, W. (2023). Comparing the prediction performance of item response theory and machine learning methods on item responses for educational assessments. Behavior Research Methods, 55(4), 2109–2124.
12. Rach, S., & Ufer, S. (2020). Which Prior Mathematical Knowledge Is Necessary for Study Success in the University Study Entrance Phase? Results on a New Model of Knowledge Levels Based on a Reanalysis of Data from Existing Studies. International Journal of Research in Undergraduate Mathematics Education, 6(3), 375-403.
13. Rezvanifard, F., Radmehr, F., & Rogovchenko, Y. (2023). Advancing engineering students’ conceptual understanding through puzzle-based learning: A case study with exact differential equations. Teaching Mathematics and Its Applications: An International Journal of the IMA, 42(2), 126-149.
14. Trenholm, S., & Peschke, J. (2020). Teaching undergraduate mathematics fully online: a review from the perspective of communities of practice. International Journal of Educational Technology in Higher Education, 17(1), 37.
15. Vera, J. F. (2022). Distance‐based logistic model for cross‐classified categorical data. British Journal of Mathematical and Statistical Psychology, 75(3), 466-492.
16. Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98-110.
