Reliability and Validity of the Multiple Mini Interview in the General Entrance Examination for Medical Schools at a National University
DOI:
https://doi.org/10.52731/lir.v001.017Keywords:
Multiple Mini Interview (MMI), Generalizability theory (G-theory), medical education, Institutional ResearchAbstract
Introduction: Although MMI is considered an interview technique with good reliability and va-lidity in foreign countries, few universities have implemented it in admissions because of its high cost in Japan. In this study, we examined the reliability and validity of the MMI at a medicaleducation institution that introduced the MMI for general entrance examinations. Methods: Twenty-nine students who took the MMI in 2022 were included in the study. For reliability, G study and D study were conducted based on generalizability theory. For validity, the total score of the MMI and the total score of the achievement test were plotted as a scatter plot and compared with the currently known status. Results: G coefficient was 0.336. There was no significant cor-relation between MMI and test scores.Unwilling applicants had lower MMI scores. Discussion and Conclusion: Regarding reliability, we found that our MMI needs to be improved. The validity of the MMI needs further investigation, but the results suggest that the MMI may be related to factors important to university administration.
References
Moradi, N., Mohammadi, R., & Goldasteh, A. (2019). A Comparative Study of Higher Education Entrance Examinations in Iran with Some Selected Countries. Iranian Journal of Comparative Education, 2(4), 518-532.
Nogami, T., Arai, M., Ishigami, T., Nakada, Y., Matsuda, T., Odaguchi, H., & Hanawa, T. (2021). Comparison of the 2011 and 2019 Kampo Medicine Curricula Across All Japanese Medical Schools. The Tokai Journal of Experimental and Clinical Medicine, 46(2), 75-82.
Benbassat, J., & Baumal, R. (2007). Uncertainties in the selection of applicants for medical school. Advances in Health Sciences Education, 12(4), 509-521.
Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F., & Cleland, J. (2016). How effective are selection methods in medical education? A systematic review. Medical education, 50(1), 36-60.
Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004). An admissions OSCE: the multiple mini‐interview. Medical education, 38(3), 314-326.
Kishi T. (2017). The relationship between Multiple Mini Interview and academic performance of medical students. Tne 81st annual convention of the Japanese Psychological Association pp. 2D-080 (written in Japansese).
Yoshimura, H., Kitazono, H., Fujitani, S., Machi, J., Saiki, T., Suzuki, Y., & Ponnamperuma, G. (2015). Past-behavioural versus situational questions in a postgraduate admissions multiple mini-interview: a reliability and acceptability comparison. BMC medical education, 15(1), 1-9.
Yamada, T., Sato, J., Yoshimura, H., Okubo, T., Hiraoka, E., Shiga, T., ... & Ban, N. (2017). Reliability and acceptability of six station multiple mini-interviews: past-behavioural versus situational questions in postgraduate medical admission. BMC Medical Education, 17(1), 1-7.
Yamada, T., Hiraoka, E., & Takemura, Y. (2021). Predictive Validity of Past-Behavioral Versus Situational Questions in Postgraduate Medical Admissions. Academic Medicine, 96(2), 164-165.
Brennan, R. L. (2000). Performance assessments from the perspective of generalizability theory. Applied Psychological Measurement, 24(4), 339-353.
Streiner, D. L., Norman, G. R., & Cairney, J. (2015). Health measurement scales: a practical guide to their development and use. Oxford University Press, USA. 200-225.
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach's alpha. International journal of medical education, 2, 53.
Shavelson, R. （1991）. Webb, N. Generalizability Theory ; A Primer. Sage Publications, Thousand Oaks, CA.
Brennan, R.L. （2005）. Generalizability theory, New York : Springer Germany.
Oliver, T., Hecker, K., Hausdorf, P. A., & Conlon, P. (2014). Validating MMI scores: are we measuring multiple attributes?. Advances in Health Sciences Education, 19(3), 379-392.
Salvatori, P. (2001). Reliability and validity of admissions tools used to select students for the health professions. Advances in Health Sciences Education, 6(2), 159-175.
Rosenfeld, J. M., Reiter, H. I., Trinh, K., & Eva, K. W. (2008). A cost efficiency comparison between the multiple mini-interview and traditional admissions interviews. Advances in Health Sciences Education, 13(1), 43-58.
Rees, E. L., Hawarden, A. W., Dent, G., Hays, R., Bates, J., & Hassell, A. B. (2016). Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37. Medical Teacher, 38(5), 443-455.
Pau, A., Jeevaratnam, K., Chen, Y. S., Fall, A. A., Khoo, C., & Nadarajah, V. D. (2013). The multiple mini-interview (MMI) for student selection in health professions training–a systematic review. Medical teacher, 35(12), 1027-1041.
Kelly, M. E., Dowell, J., Husbands, A., Newell, J., O‘Flynn, S., Kropmans, T., ... & Murphy, A. W. (2014). The fairness, predictive validity and acceptability of multiple mini interview in an internationally diverse student population-a mixed methods study. BMC Medical Education, 14(1), 1-13.
