Analysis of the Quality of Sudents’ Critical Reasoning Instruments Using Rasch Model
DOI:
10.29303/jppipa.v11i3.9584Published:
2025-03-25Issue:
Vol. 11 No. 3 (2025): MarchKeywords:
Critical reasoning, Instrument, Rasch model, Reliability, ValidityResearch Articles
Downloads
How to Cite
Downloads
Metrics
Abstract
This research aims to analyzes the validity and reliability of a students' critical reasoning questionnaire using the Rasch Model. This quantitative descriptive research employed a non-experimental survey design involving 122 fourth-semester undergraduate students from the College of Teacher Training and Education (STKIP) Melawi. Data were collected through an online questionnaire distributed via Google Forms. Validity was assessed through unidimensionality and item fit analysis, while reliability was measured using person reliability, item reliability, and Cronbach’s Alpha. The results indicate that the instrument has not yet achieved good item validity, with only 4 out of 20 items classified as valid. Reliability analysis showed Cronbach’s Alpha of 0.55 (bad category), item reliability of 0.98 (special category), and person reliability of 0.44 (weak category). Item difficulty analysis identified four difficulty levels: 2 items were very easy, 9 easy, 6 difficult, and 3 very difficult. The Rasch model’s item discrimination analysis produced eight item groups based on an item separation index of H = 9.97. These findings emphasize the need to refine questionnaire items and enhance respondent engagement. A validated instrument will support educational assessments by providing a reliable tool for measuring students’ critical reasoning skills in teacher training institutions.
References
Allen, E., & Seaman, C. (2007). Likert scales and data analysis. Quality Progress, 40(7), 64–65. Retrieved from https://www.bayviewanalytics.com/reports/asq/likert-scales-and-data-analyses.pdf
Alsoudi, S., & Shindi, Y. A. (2023). Rasch rating scale modelling of the Arabic version of the critical thinking disposition scale. Journal of Educational and Psychological Studies [JEPS], 17(4), 359–369. https://doi.org/10.53543/jeps.vol17iss4pp359-369
Anggraini, N., Nabillah, C., Lonan, H. D., & Sain, H. (2023). Application of the Rasch model to test tools in the analysis of survey design. Parameter: Journal of Statistics, 3(1), 19–27. https://doi.org/10.22487/27765660.2023.v3.i1.16412
Asmarita, D., & Prastowo, A. (2023). The learning process of Islamic education to foster students’ critical reasoning at Madrasah Ibtidaiyah Teluk Panji, North Sumatra. DAYAH: Journal of Islamic Education, 6(2), 233. https://doi.org/10.22373/jie.v6i2.18508
Azmi, I., & Prasetya, D. S. B. (2025). Profil berpikir kritis siswa SMP pada mata pelajaran IPA. Journal of Classroom Action Research, 7(1), 163–175. https://doi.org/10.21449/ijate.1210479
Balbuena, S. (2023). Application of the Rasch model in streamlining an instrument measuring depression among college students. International Journal of Assessment Tools in Education, 10(2), 257–278. https://doi.org/10.21449/ijate.1210479
Boone, W. J. (2016). Rasch Analysis for Instrument Development: Why, When, and How? CBE—Life Sciences Education, 15(4), rm4. https://doi.org/10.1187/cbe.16-04-0148
Darmana, A., Sutiani, A., Nasution, H. A., Ismanisa, I., & Nurhaswinda, N. (2021). Analysis of Rasch model for the validation of chemistry national exam instruments. Jurnal Pendidikan Sains Indonesia, 9(3), 329–345. https://doi.org/10.24815/jpsi.v9i3.19618
Denis, B. (2022). Сritical thinking technology as a tool for innovative activity in mathematics lessons. Baltic Journal of Legal and Social Sciences, 2(2), 23–30. https://doi.org/10.30525/2592-8813-2022-2-4
Erfan, M., Maulyda, M. A., Hidayati, V. R., Astria, F. P., & Ratu, T. (2020). Analisis kualitas soal kemampuan membedakan rangkaian seri dan paralel melalui teori tes klasik dan model Rasch. Indonesian Journal of Educational Research and Review, 3(1), 11–19. https://doi.org/10.23887/ijerr.v3i1.24080
Gustianingrum, R. A., Murni, A., & Maimunah. (2023). Analisis kemampuan berpikir kritis peserta didik dalam menunjang penguatan profil pelajar Pancasila. PRISMA, Prosiding Seminar Nasional Matematika, 6(2), 465–471. Retrieved from https://journal.unnes.ac.id/sju/prisma/article/view/66908
Jumadi, J., Sukarelawan, M. I., & Kuswanto, H. (2023). An investigation of item bias in the four-tier diagnostic test using Rasch model. International Journal of Evaluation and Research in Education (IJERE), 12(2), 622–629. https://doi.org/10.11591/ijere.v12i2.22845
Kaur, M. K. (2018). Critical thinking for global peace: A key for sustainable development. International Journal of Trend in Scientific Research and Development, 2(3), 1498–1501. https://doi.org/10.31142/ijtsrd11439
Lailiyah, L., Supriyati, Y., & Komarudin, K. (2018). Analysis of measures items in development of instruments self-assessment (Rasch modeling application). JISAE: Journal of Indonesian Student Assessment and Evaluation, 4(1), 1–9. https://doi.org/10.21009/jisae.v4i1.7819
Laliyo, L. A. R., Kilo, A. L., Paputungan, M., Kunusa, W. R., Dama, L., & Panigoro, C. (2022). Rasch modelling to evaluate reasoning difficulties, changes of responses, and item misconception pattern of hydrolysis. Journal of Baltic Science Education, 21(5), 817–835. https://doi.org/10.33225/jbse/22.21.817
Linacre, J. M. (2012). Winsteps Rasch tutorial 2. Retrieved from https://www.winsteps.com/a/winsteps-tutorial-2.pdf
Loh, S. L., Jamil, N. A., & Ng, K. T. (2022). Establishing the validity and reliability of a program evaluation questionnaire using Rasch measurement model. DINAMIKA: Jurnal Ilmiah Pendidikan Dasar, 14(2), 150–157. https://doi.org/10.30595/dinamika.v14i2.14786
Madzlan, N. A., Rus, R. C., Mustafa, M. C., & Yassin, S. M. (2022). Validity and reliability of survey items in employer perspective construct on the quality of ECCE: Rasch measurement model analysis. World Journal of English Language, 12(2), 288–293. https://doi.org/10.5430/wjel.v12n2p288
Mawardi. (2019). Rambu-rambu penyusunan skala sikap model Likert untuk mengukur sikap siswa. Scholaria: Jurnal Pendidikan Dan Kebudayaan, 9(3), 292–304. https://doi.org/10.24246/j.js.2019.v9.i3.p292-304
Mignogna, G., Carey, C. E., Wedow, R., Baya, N., Cordioli, M., Pirastu, N., Bellocco, R., Malerbi, K. F., Nivard, M. G., Neale, B. M., Walters, R. K., & Ganna, A. (2023). Patterns of item nonresponse behaviour to survey questionnaires are systematic and associated with genetic loci. Nature Human Behaviour, 7(8), 1371–1387. https://doi.org/10.1038/s41562-023-01632-7
Natsir, I., Mayasari, D., & Munfarikhatin, A. (2022). Rasch model: Quality of test instruments to measure students’ ability in algebra. SHS Web of Conferences, 149, 1–5. https://doi.org/10.1051/shsconf/202214901033
Packer, J., Croker, H., Goddings, A.-L., Boyland, E. J., Stansfield, C., Russell, S. J., & Viner, R. M. (2022). Advertising and young people’s critical reasoning abilities: Systematic review and meta-analysis. Pediatrics, 150(6). https://doi.org/10.1542/peds.2022-057780
Purwana, U., Rusdiana, D., & Liliawati, W. (2020). Pengujian kemampuan menginterpretasikan grafik kinematika calon guru fisika: The polytomous Rasch analysis. ORBITA: Jurnal Kajian, Inovasi Dan Aplikasi Pendidikan Fisika, 6(2), 259–266. https://doi.org/10.31764/orbita.v6i2.3264
Putra, M. D. K., & Retnawati, H. (2020). Analisis Rasch terhadap instrumen general self-efficacy scale-12 versi bahasa Indonesia: Studi komparasi rating scale model (RSM) dan partial credit model (PCM). Indonesian Journal of Educational Assesment, 3(1), 52–60. https://doi.org/10.26499/ijea.v3i1.55
Ramdani, A., Jufri, A. W., Jamaluddin, & Setiadi, D. (2020). Kemampuan berpikir kritis dan penguasaan konsep dasar IPA peserta didik. Jurnal Penelitian Pendidikan IPA, 6(1), 119–124. https://doi.org/10.29303/jppipa.v6i1.388
Ridzuan, M. F., Lim, L. H., Fozee, F. A. A., & Nasser, S. N. A. M. (2020). Rasch analysis model: Reliability and validity of superitem test instrument. International Journal of Academic Research in Progressive Education and Development, 9(4), 1–11. https://doi.org/10.6007/IJARPED/v9-i4/8166
Salsabila, F. (2023). Pengujian validitas dan reliabilitas instrumen manajemen diri remaja: Rasch model analysis. Jurnal Bimbingan Dan Konseling Terapan, 7(1), 15–25. https://doi.org/10.30598/jbkt.v7i1.1741
Sari, T. N. I., & Rakhmawati, A. (2024). Analysis the quality of critical thinking and creativity questions in high school biology subjects with the Rasch model. Research and Development in Education (RaDEn), 4(1), 602–616. https://doi.org/10.22219/raden.v4i1.32758
Sumintono, B., & Widhiarso, W. (2014). Aplikasi Model Rasch untuk penelitian ilmu-ilmu sosial. Bandung: Tim Komunika Publishing House
Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan RASCH pada assessment pendidikan. Cimahi: Trim komunikata
Sumintono, B. (2016). Aplikasi Pemodelan Rasch pada asesmen pendidikan: Implementasi penilaian formatif (assessment for learning). Retrieved from https://shorturl.at/uKbbw
Tarigan, E. F., Nilmarito, S., Islamiyah, K., Darmana, A., & Suyanti, R. D. (2022). Analisis instrumen tes menggunakan Rasch model dan software SPSS 22.0. Jurnal Inovasi Pendidikan Kimia, 16(2), 92–96. https://doi.org/10.15294/jipk.v16i2.30530
Tennant, A., & Küçükdeveci, A. A. (2023). Application of the Rasch measurement model in rehabilitation research and practice: Early developments, current practice, and future challenges. Frontiers in Rehabilitation Sciences, 4, 1–17. https://doi.org/10.3389/fresc.2023.1208670
Wahyudi, A., Mahyuddin, M. J., Irawan, A. W., Silondae, D. P., Lestari, M., Basco, F. H., & Kurniawan, S. J. (2020). Model Rasch: Analisis skala resiliensi Connor-Davidson versi Bahasa Indonesia. Advice: Jurnal Bimbingan Dan Konseling, 2(1), 28–35. https://doi.org/10.32585/advice.v2i1.701
Wilson, M., & Dulhunty, M. (2019). Interpreting the relationship between item difficulty and DIF: Examples from educational testing. Journal of Physics: Conference Series, 1379(1). https://doi.org/10.1088/1742-6596/1379/1/012026
Yokhebed, Sutarno, Masykuri, M., Prayitno, B. A., & Sudirman. (2024). Validity and reliability of online socio-scientific issue-based critical reasoning instrument. Jurnal Pendidikan IPA Indonesia, 13(2), 229–238. https://doi.org/10.15294/2hnw4674
Yunus, F. A. N., Suki, N. A., Baser, J. A., Masran, S. H., Marian, M. F., & Rahim, M. B. (2017). Kesahan dan kebolehpercayaan instrumen kompetensi pengajar TVET terhadap pengajaran teknikal berdasarkan pendekatan model Rasch. Online Journal for TVET Practitioners, 2(2). Retrieved from https://publisher.uthm.edu.my/ojs/index.php/oj-tp/article/view/4780
Author Biographies
Yokhebed, Universitas Tanjungpura
Rexy Maulana Dwi Karmadi, Universitas Tanjungpura
Jayanti Silaen, Universitas Tanjungpura
Sherli Meidina, Universitas Tanjungpura
Nur Alfiatun Nikmah, Universitas Tanjungpura
License
Copyright (c) 2025 Yokhebed, Rexy Maulana Dwi Karmadi, Jayanti Silaen, Sherli Meidina, Nur Alfiatun Nikmah
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with Jurnal Penelitian Pendidikan IPA, agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License (CC-BY License). This license allows authors to use all articles, data sets, graphics, and appendices in data mining applications, search engines, web sites, blogs, and other platforms by providing an appropriate reference. The journal allows the author(s) to hold the copyright without restrictions and will retain publishing rights without restrictions.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in Jurnal Penelitian Pendidikan IPA.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
